# Stop Letting Your Coding Agent Blindly Grep Decoy Files: Raw Grep vs. a 0-Token AST Dependency Map (3.15x Faster)

> 💡 **AI Labo Benchmark Series — Experiment `E09` (Guides & Context Hygiene)**
>
> This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.
> * 🚀 **[View All 41 Benchmark Results & Master Ranking](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)**
> * 🛡️ **[Read How Our Proxmox & systemd Anti-Cheating Sandbox Works](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)**

---

## Metadata & Experiment Identity
* **Experiment ID:** `E09` (Backlog Candidate `C005`)
* **Run ID:** `e09-20261028-vm213-01`
* **Sandbox Environment:** Proxmox Isolated VM `VM213` (Debian 13 Trixie, 0 NICs, `virtio-serial` QGA relay, POSIX UID separation)
* **Model:** `gemini-2.5-flash` (`temperature=0.0`, `thinkingBudget=0`)
* **Pattern Under Test:** *"Context Engine / AST Dependency Graph Pre-Injection vs. Monolithic Grep Waste"* — **Free-Form Grep Exploration vs. 0-Token Deterministic AST Symbol & Call-Graph Mini-Map**

---

## 1. Executive Summary (TL;DR)

In any realistic production repository, core domain terms—like `volume_discount`, `error_pct`, `rebate`, `base_cost`, or tier names (`STARTER`, `GROWTH`, `SCALE`)—don't just live in the active execution pipeline. They also appear in legacy migration adapters, Prometheus metrics exporters, compliance archivers, and webhook formatters.

When you hand an AI coding agent `grep` and file-read tools and tell it to explore on its own (**Mode A: `RawGrepExploration`**), keyword searches light up every file that shares domain vocabulary. The agent ends up reading unreferenced auxiliary files, polluting its context window, and burning ~60% of its tokens before finishing the actual fix.

To measure how much exploration waste can be eliminated deterministically, we benchmarked two harness modes on an **8-module multi-tenant API quota & usage billing repository** (3 active pipeline modules containing 7 coupled bugs + 5 unreferenced auxiliary/legacy decoy modules) inside isolated VM `VM213`:

* **Mode A (`RawGrepExploration`):** Standard repository-wide `grep` and iterative file inspection across matching modules.
* **Mode B (`ASTDependencyMap`):** Before calling the LLM, a millisecond Python `ast.parse()` pre-scan (0 tokens) extracts function signatures, constants, and the live call chain (`ACTIVE_PIPELINE` vs. `AUXILIARY_UNREFERENCED`) into an **11-line AST mini-map** injected into Turn 1.

### Key Result
Both modes achieved a **perfect `20 / 20` (100%) pass** on the hidden grading suite. However, pre-injecting the 11-line AST dependency map **eliminated 100% of decoy file reads (`5 ➔ 0` files)**, cut **LLM calls by `-66.7%` (`3 ➔ 1` single-shot pass)**, reduced **input tokens by `-59.1%` (`9,508 ➔ 3,893`)**, cut **output tokens by `-64.5%` (`4,992 ➔ 1,774`)**, lowered **API cost by `-63.0%` (`$0.0259 ➔ $0.0096`)**, and ran **`3.15x` faster (`42.30s ➔ 13.45s`, `-68.2%`)**.

| Metric | Mode A: Free-Form Grep (`RawGrepExploration`) | Mode B: 0-Token AST Map (`ASTDependencyMap`) | Delta / Improvement |
| :--- | :---: | :---: | :---: |
| **Hidden Grading Suite Score (20 Cases)** | `20 / 20` (100%) | **`20 / 20` (100%)** | Identical 100% accuracy |
| **Total Files Inspected (`files_inspected_count`)** | `8 / 8 files` | **`3 / 8 files`** | **-62.5% (`-5` files opened)** |
| **Unreferenced Decoy Files Read (`decoy_files_inspected`)** | `5 / 5 decoys` | **`0 / 5 decoys`** | **-100.0% (zero decoy waste)** |
| **LLM Calls (`model_calls`)** | `3 calls` | **`1 call` (1-shot)** | **-66.7% (`-2` exploration turns)** |
| **Cumulative File Edits (`file_edits`)** | `8 edits` | **`3 edits`** | **-62.5% (`-5` redundant edits)** |
| **Cumulative Input Tokens (`promptTokenCount`)** | `9,508 tokens` | **`3,893 tokens`** | **-59.1% (`-5,615` tokens)** |
| **Cumulative Output Tokens (`candidatesTokenCount`)** | `4,992 tokens` | **`1,774 tokens`** | **-64.5% (`-3,218` tokens)** |
| **Elapsed Execution Time (`elapsed_seconds`)** | `42.30 sec` | **`13.45 sec`** | **-68.2% (`3.15x` faster)** |
| **Estimated API Cost (`estimated_micro_usd`)** | `$0.0259` (`25,852 micro-USD`) | **`$0.0096` (`9,573 micro-USD`)** | **-63.0% cost reduction** |

---

## 2. Experiment Architecture & Repository Layout

```text
+-------------------------------------------------------------------------+
| Proxmox Isolated VM213 (0 NICs, virtio-serial QGA, offline Debian 13)   |
|  Target Repo: 8-Module Multi-Tenant API Quota & Billing Codebase        |
|   - 3 Active Pipeline Files: ingest.py ➔ quota_engine.py ➔ receipt.py   |
|   - 5 Unreferenced Decoys:   legacy_v1_adapter, audit_exporter,         |
|                              webhook_notifier, metrics_collector,       |
|                              csv_archiver                               |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: RawGrepExploration (Keyword Grep + Iterative Inspection)  |  |
|  |  - Turn 1: Repo grep hits domain terms ➔ reads 5 files (3 decoys) |  |
|  |  - Turn 2: Reads remaining 3 files (2 decoys + receipt.py)        |  |
|  |  - Turn 3: Final boundary & non-mutation patch pass               |  |
|  |  --> 3 calls | 8/8 files read (5 decoys) | 9,508 in | 42.30s      |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: ASTDependencyMap (0-Token ast.parse() Call-Graph Map)     |  |
|  |  - Pre-Scan (0 tokens, <5ms): build_ast_dependency_map()          |  |
|  |    Marks 3 files ACTIVE_PIPELINE & 5 files AUXILIARY_UNREFERENCED |  |
|  |  - Turn 1: Opens 0 decoys ➔ patches all 7 bugs in 3 active files  |  |
|  |  --> 1 call  | 3/8 files read (0 decoys) | 3,893 in | 13.45s      |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
```

### Why Keyword `grep` Gets Trapped: Active Modules vs. Vocabulary Decoys

| Module | AST Classification | Key Symbols | Bug / Why `grep` Gets Baited |
| :--- | :--- | :--- | :--- |
| `ingest.py` | **`ACTIVE_PIPELINE`** | `parse_events(raw_events)` `[L6-L37]`, `ALLOWED_ENDPOINTS` | **Bugs 1–3:** Mutates input dict & leaks extra keys; accepts `bool` via `isinstance(x, int)`; rejects free-tier `base_cost == 0` |
| `quota_engine.py` | **`ACTIVE_PIPELINE`** | `settle_quota(events, tier, prepay_credit)` `[L6-L67]`, `ALLOWED_TIERS` | **Bugs 4–6:** Penalizes `error_pct >= 5` instead of `> 5`; checks `subtotal > 10000` instead of `>= 10000`; ignores conflicting rates on duplicate keys |
| `receipt.py` | **`ACTIVE_PIPELINE`** | `format_receipt(summary)` `[L4-L20]` | **Bug 7:** Outputs `0` instead of mandatory signed zero `-0` when discount/credit is zero |
| `legacy_v1_adapter.py` | `AUXILIARY_UNREFERENCED` | `convert_legacy_v1_record`, `estimate_legacy_rebate` | Contains `rebate`, `error_pct`, `base_cost`, `STARTER_V1`, `GROWTH_V1` |
| `audit_exporter.py` | `AUXILIARY_UNREFERENCED` | `compute_audit_digest`, `build_compliance_envelope` | Contains `volume_discount`, `credit_applied`, `STARTER` (offline SOX/SOC2 exporter) |
| `webhook_notifier.py` | `AUXILIARY_UNREFERENCED` | `should_trigger_overage_webhook`, `format_webhook_payload` | Contains `STARTER`, `GROWTH`, `SCALE`, `volume_discount`, `error_pct_max` |
| `metrics_collector.py` | `AUXILIARY_UNREFERENCED` | `render_prometheus_metrics`, `aggregate_endpoint_error_rates` | Contains `error_pct`, `STARTER`, `subtotal` |
| `csv_archiver.py` | `AUXILIARY_UNREFERENCED` | `settlement_lines_to_csv`, `parse_archived_csv_line` | Contains `base_cost`, `error_pct`, `line_total` |

### The Actual 11-Line 0-Token AST Mini-Map Injected in Mode B

Using only Python's standard `ast` library (`build_ast_dependency_map()`), the harness generated this compact structural map in milliseconds before making a single API call:

```text
# AST Symbol & Pipeline Dependency Map (0-Token Deterministic Pre-Scan)
Active Pipeline Chain: ingest.parse_events(raw_events) -> quota_engine.settle_quota(events, tier, prepay_credit) -> receipt.format_receipt(summary)

- ingest.py [ACTIVE_PIPELINE]: funcs=[parse_events(raw_events) [L6-L37]] consts=[ALLOWED_ENDPOINTS]
- quota_engine.py [ACTIVE_PIPELINE]: funcs=[settle_quota(events, tier, prepay_credit) [L6-L67]] consts=[ALLOWED_TIERS]
- receipt.py [ACTIVE_PIPELINE]: funcs=[format_receipt(summary) [L4-L20]] consts=[]
- legacy_v1_adapter.py [AUXILIARY_UNREFERENCED]: funcs=[convert_legacy_v1_record(v1_row) [L23-L39]; estimate_legacy_rebate(gross_amount, error_pct, legacy_tier) [L42-L47]; summarize_legacy_migration_batch(rows) [L50-L58]] consts=[LEGACY_ENDPOINT_MAP, LEGACY_TIER_LIMITS]
- audit_exporter.py [AUXILIARY_UNREFERENCED]: funcs=[compute_audit_digest(tenant_id, endpoint, calls, line_total) [L12-L15]; build_compliance_envelope(settlement_dict, region, auditor_id) [L18-L46]; serialize_compliance_ndjson(envelopes) [L49-L51]] consts=[AUDIT_SCHEMA_VERSION, COMPLIANCE_REGIONS]
- webhook_notifier.py [AUXILIARY_UNREFERENCED]: funcs=[should_trigger_overage_webhook(tier, grand_total, error_pct_max) [L10-L17]; format_webhook_payload(tenant_id, tier, subtotal, volume_discount, grand_total) [L20-L35]] consts=[ALERT_THRESHOLDS]
- metrics_collector.py [AUXILIARY_UNREFERENCED]: funcs=[render_prometheus_metrics(settlement_list) [L4-L18]; aggregate_endpoint_error_rates(raw_events) [L21-L34]] consts=[]
- csv_archiver.py [AUXILIARY_UNREFERENCED]: funcs=[settlement_lines_to_csv(summary) [L14-L20]; parse_archived_csv_line(line_text) [L23-L35]] consts=[CSV_COLUMNS]
```

---

## 3. Turn-by-Turn Token & Cost Breakdown (`usage.json`)

Inspecting all 4 recorded model invocations (`seq 0`–`2` for Mode A, `seq 3` for Mode B) shows exactly how blind `grep` multiplies both input and output tokens:

| Seq | Mode | Node ID | Input Tokens | Output Tokens | Cost (`micro-USD`) | Execution Behavior |
| :---: | :--- | :--- | :---: | :---: | :---: | :--- |
| `0` | **Mode A** (`RawGrep`) | `raw_grep_turn_1_explore_and_patch` | `5,161` | `1,477` | `9,410` (`$0.0094`) | Runs repo-wide `grep`, loads 5 matching files (**including 3 decoys**: `legacy_v1_adapter`, `audit_exporter`, `webhook_notifier`), and makes partial edits to `ingest.py` & `quota_engine.py`. |
| `1` | **Mode A** (`RawGrep`) | `raw_grep_turn_2_refine_receipt_and_bounds` | `2,630` | `1,775` | `8,629` (`$0.0086`) | Loads the remaining 3 files (**including 2 decoys**: `metrics_collector`, `csv_archiver`) plus `receipt.py`; patches `-0` formatting and discount bounds. |
| `2` | **Mode A** (`RawGrep`) | `raw_grep_turn_3_final_patch` | `1,717` | `1,740` | `7,813` (`$0.0078`) | Re-edits the 3 active files to finalize `bool` rejection, input non-mutation, and first-seen ordering (8 cumulative edits). |
| *Mode A Total* | — | *3 turns (5 decoys read)* | *`9,508`* | *`4,992`* | *`25,852` (`$0.0259`)* | *`20 / 20` Pass in `42.30s`* |
| `3` | **Mode B** (`ASTDepMap`) | `ast_dep_map_single_shot_fix` | **`3,893`** | **`1,774`** | **`9,573` (`$0.0096`)** | Sees the 11-line AST mini-map upfront, **ignores all 5 unreferenced decoys**, loads only the 3 active modules, and fixes all 7 bugs in a single shot. |
| *Mode B Total* | — | *1 turn (0 decoys read)* | ***`3,893`*** | ***`1,774`*** | ***`9,573` (`$0.0096`)*** | *`20 / 20` Pass in `13.45s` (`3.15x` faster)* |

Notice that Mode B's **entire run (`$0.00957`) cost almost the exact same as Turn 1 alone of Mode A (`$0.00941`)**, because Mode B never wasted turns reading legacy adapters or re-patching partially understood pipelines.

---

## 4. Practical Takeaway: Use Static Analysis for Navigation, LLMs for Reasoning

1. **Never pay frontier LLM token prices to reconstruct an import graph:** Python's `ast`, TypeScript's compiler API, or `tree-sitter` can build an exact symbol and reachability map in `< 10 ms` for `$0.00`.
2. **Label reachable vs. unreferenced modules explicitly:** Tagging modules as `[ACTIVE_PIPELINE]` vs. `[AUXILIARY_UNREFERENCED]` (and pairing this map with **Tree-sitter Interface Skeletons (`E21`)** for large dependencies) prevents legacy and observability code from hijacking the agent's attention.

---

### 🔗 Continue Reading the Series
* 🚀 **Master Benchmark Hub (All 41 Patterns Ranked):** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)
* 🛡️ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)
* 📐 **Previous Post (`E08`):** [Vibe Coding vs. Spec-First SDD: How Locking an Executable Spec Eliminates Rework and Cuts Input Tokens by 32.9%](https://tappers.hashnode.dev/e08-spec-first-sdd)
* ⚔️ **Next Companion Post (`E10`):** [Why Self-Review Fails in AI Coding Agents: Single-Agent Self-Review vs. an Adversarial Critic (0% vs. 100% Mutant Kill Rate)](https://tappers.hashnode.dev/e10-adversarial-critic)
* 🥈 **Rank #2 Deep Dive (`E21`):** [Why Feeding Full Dependency Files to Your Coding Agent Hurts Accuracy: Full Source vs. Tree-sitter Interface Skeletons](https://tappers.hashnode.dev/e21-treesitter-skeleton-context)