Why a 500-Line AGENTS.md Hurts Coding Agent Accuracy: Monolithic Encyclopedia Prompts vs. a 30-Line Directory-Pointer Index (3.13x Faster)

💡 AI Labo Benchmark Series — Experiment E35 (Context Hygiene & AGENTS.md Architecture)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

🚀 View All 41 Benchmark Results & Master Ranking

🛡️ Read How Our Proxmox & systemd Anti-Cheating Sandbox Works

Metadata & Experiment Identity

Experiment ID: E35 (Backlog Candidate C031)
Run ID: e35-20261029-vm239-01
Sandbox Environment: Proxmox Isolated VM VM239 (ai-e35-239, 0 NICs, PrivateNetwork=yes, NoNewPrivileges=yes, InaccessiblePaths, UID split labexp35 / labjudge35)
Model: gemini-3.8-flash (thinkingLevel: LOW, temperature=0.0)
Pattern Under Test: "OpenAI Harness Engineering (Ryan Lopopolo): Treating AGENTS.md as a Short Table-of-Contents Pointer vs. a 500-Line Monolithic Encyclopedia" — Monolithic Encyclopedia AGENTS.md (MonolithicEncyclopediaAgentsRunner) vs. Directory-Pointer AGENTS.md (DirectoryPointerAgentsRunner)

1. Executive Summary (TL;DR)

One of the most common mistakes teams make when adopting AGENTS.md or CLAUDE.md is treating the root file as a 500-line Monolithic Encyclopedia: every team dumps its billing rules, FX policies, webhook guidelines, PCI checklists, and legacy conventions into one giant markdown file that gets injected into every turn.

As highlighted in OpenAI's Harness Engineering principles, when 95% of an encyclopedia prompt describes unrelated subsystems, it triggers Attention Dilution: the model overlooks critical edge-case invariants for the 3 files it is actually editing because those rules are buried inside 35,000 characters of unrelated prose.

Replacing the monolithic encyclopedia with a 30-line Directory-Pointer AGENTS.md—a lightweight Table of Contents that routes the agent only to the scoped specification pointers ([SPEC-VAL], [SPEC-SET], [SPEC-STM]) matching the active directory—eliminates attention dilution completely.

We tested this head-to-head inside isolated VM VM239 on the 3-module enterprise payment settlement pipeline (validator.py ➔ settlement.py ➔ statement.py):

Mode A (MonolithicEncyclopediaAgentsRunner): Loads a monolithic encyclopedia AGENTS.md across 74 unrelated domain sections (73,336 chars of diluted context across Turns 1–2), overlooking the 5-key dictionary stripping invariant (new_item = dict(item) leaks "extra") and finishing at 19 / 20 (95.0%) in 41.41s.
Mode B (DirectoryPointerAgentsRunner): Injects a concise 3-pointer specification index (DIRECTORY_POINTER_AGENTS_MD) scoped strictly to validator.py, settlement.py, and statement.py, achieving 20 / 20 (100%) in 1 turn (13.24s).

Key Result

Mode A failed validator_01_normalize_and_non_mutation (19 / 20) after 3 turns (46,021 cumulative input tokens and 41.41s). Mode B achieved a 20 / 20 (100%) pass in 1 turn, cutting LLM calls by -66.7% (3 ➔ 1), cumulative input tokens by -95.2% (46,021 ➔ 2,219 tokens—a 20.74x reduction), API cost by -83.1% ($0.0453 ➔ $0.0076), and running 3.13x faster (41.41s ➔ 13.24s, -68.0%).

Metric	Mode A: 500-Line Encyclopedia (`MonolithicEncyclopediaAgents`)	Mode B: Directory-Pointer Index (`DirectoryPointerAgents`)	Delta / Improvement
Hidden Grading Suite Score (`labjudge35`)	`19 / 20` (95.0% — 1 failure)	`20 / 20` (100% — Perfect)	+5.0% accuracy (`19/20 ➔ 20/20`)
Attention-Dilution Traps (`gotcha_traps_hit`)	`2 traps` (Turns 1 & 2)	`0 traps`	-100% (zero rule drift)
LLM Calls (`model_calls`)	`3 calls`	`1 call` (1-shot)	-66.7% (`3.00x` faster convergence)
Cumulative File Edits (`file_edits`)	`5 edits`	`3 edits` (1 per module)	-40.0% fewer edits
Cumulative Tool Output (`tool_output_chars`)	`73,336 chars`	`103 chars`	-99.9% (`712.0x` noise reduction)
Turn 1 Tokens (In / Out)	`1,664 in` / `13 out`	`2,219 in` / `1,593 out`	`+555` in for scoped pointer specs
Turn 2 Tokens (In / Out)	`14,960 in` / `1,260 out`	— (completed on Turn 1)	Avoided 36K-char encyclopedia dump
Turn 3 Tokens (In / Out)	`29,397 in` / `1,592 out`	— (completed on Turn 1)	Avoided secondary dilution bloat
Cumulative Input Tokens (`promptTokenCount`)	`46,021 tokens`	`2,219 tokens`	-95.2% (`20.74x` efficiency)
Cumulative Output Tokens (`candidatesTokenCount`)	`2,865 tokens`	`1,593 tokens`	-44.4% output reduction
Elapsed Execution Time (`elapsed_seconds`)	`41.41 sec`	`13.24 sec`	-68.0% (`3.13x` faster)
Estimated API Cost (`estimated_micro_usd`)	`$0.0453` (`45,260 micro-USD`)	`$0.0076` (`7,638 micro-USD`)	-83.1% (`5.93x` cheaper)

2. Experiment Architecture: Monolithic Encyclopedia vs. Directory-Pointer `AGENTS.md`

+-------------------------------------------------------------------------+
| Proxmox Isolated VM239 (0 NICs, InaccessiblePaths, grader labjudge35)   |
|  Target Task: 3-Module Enterprise Payment Settlement Pipeline           |
|               validator.py ➔ settlement.py ➔ statement.py               |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: MonolithicEncyclopediaAgentsRunner (500-Line Encyclopedia)|  |
|  |  - Turn 1 (1,664 in):  Calls `load_monolithic_encyclopedia`       |  |
|  |                        ➔ dumps 74 unrelated chapters (36,680c)    |  |
|  |  - Turn 2 (14,960 in): Calls `stage_diluted_rule_patch` on 2 files|  |
|  |                        ➔ 36,334 chars of secondary dilution logs  |  |
|  |  - Turn 3 (29,397 in): Overlooks 5-key dict rule in `validator.py`|  |
|  |                        (`new_item = dict(item)` leaks `"extra"`)  |  |
|  |  --> 3 calls | 46,021 in | $0.0453 | 41.41s | 19/20 (1 FAILED)    |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: DirectoryPointerAgentsRunner (3-Pointer Scoped Index)     |  |
|  |  - Injects `DIRECTORY_POINTER_AGENTS_MD` (+555 tokens):           |  |
|  |    * `Pointer [SPEC-VAL] -> validator.py::validate_txns`          |  |
|  |    * `Pointer [SPEC-SET] -> settlement.py::settle_batch`          |  |
|  |    * `Pointer [SPEC-STM] -> statement.py::render_statement`       |  |
|  |  - Turn 1 (2,219 in): Calls `submit_pointer_routed_bundle` once   |  |
|  |  --> 1 call  | 2,219 in  | $0.0076 | 13.24s | 20/20 (100% PASS)   |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

3. Drop-In Implementation Recipe: `resolve_scoped_agents_md`

Keep your root AGENTS.md under 40 lines as a routing table, and dynamically inject only the specification pointers that match the target files being edited:

SCOPED_SPEC_DOCS: dict[str, str] = {
    "validator.py": (
        "Pointer [SPEC-VAL]: ALLOWED_CURRENCIES=('USD','EUR','JPY'); reject empty list & bool; "
        "cnt > 0, cents >= 0, 0 <= bps <= 1000; return fresh 5-key dict without extra keys."
    ),
    "settlement.py": (
        "Pointer [SPEC-SET]: Preserve first-seen `for k in order:`; reserve 10% strictly when `bps > 300`; "
        "inclusive `subtotal >= 15000` discount; `rebate_applied = min(fee_rebate, after_discount)`."
    ),
    "statement.py": (
        "Pointer [SPEC-STM]: Format `{merchant_id}#{currency}: ...` using `#`; "
        "always prefix `-` on `TIER_DISCOUNT: -{val}` and `REBATE_APPLIED: -{val}`."
    ),
}


def resolve_scoped_agents_md(target_files: list[str]) -> str:
    """Resolves only the specification pointers matching the active target files instead of a monolithic manual."""
    lines = ["# ACTIVE DIRECTORY-SCOPED SPECIFICATION POINTERS (Resolved from AGENTS.md TOC):"]
    for f_name in target_files:
        if f_name in SCOPED_SPEC_DOCS:
            lines.append(f"- `{f_name}` -> {SCOPED_SPEC_DOCS[f_name]}")
    return "\n".join(lines)

4. Practical Takeaway: Treat `AGENTS.md` as a Map, Not an Encyclopedia

Keep root AGENTS.md / CLAUDE.md under 30–50 lines: Use it strictly as a directory map pointing to scoped docs/specs/<module>.md files.
Pair a Directory-Pointer AGENTS.md (E35) with a 5-Item MEMORY.md (E20): AGENTS.md routes the agent to the right module specification on demand, while MEMORY.md highlights the top 5 non-obvious edge-case traps (bool subclassing int, $0 authorizations, inclusive boundaries).

🔗 Continue Reading the Series

🚀 Master Benchmark Hub (All 41 Patterns Ranked): I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy
🛡️ Methodology & Sandbox Architecture: How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox
📸 Previous Sequential Post (E34): Eliminating Print-Debugging Loops in Coding Agents: Bare Stack Traces vs. Runtime Frame Variable-Snapshot Injection (-95.0% Input Tokens)
🥇 Related Rank #1 Deep Dive (E20): Stop Your Coding Agent From Stepping on the Same Rake Twice: Cold-Start Exploration vs. a 5-Item MEMORY.md
🪆 Related Progressive Disclosure Deep Dive (E05): Why Role-Based Subagents Slow Down Coding Agents: Subagent Folding + Progressive Skill Disclosure (2.66x Faster)

Why a 500-Line AGENTS.md Hurts Coding Agent Accuracy: Monolithic Encyclopedia Prompts vs. a 30-Line Directory-Pointer Index (3.13x Faster)

Metadata & Experiment Identity

1. Executive Summary (TL;DR)

Key Result

2. Experiment Architecture: Monolithic Encyclopedia vs. Directory-Pointer `AGENTS.md`

3. Drop-In Implementation Recipe: `resolve_scoped_agents_md`

4. Practical Takeaway: Treat `AGENTS.md` as a Map, Not an Encyclopedia

🔗 Continue Reading the Series

Comments

More from this blog

Eliminating Print-Debugging Loops in Coding Agents: Bare Stack Traces vs. Runtime Frame Variable-Snapshot Injection (-95.0% Input Tokens)

Why Hand-Written Example Tests Fail AI Coding Agents: Fixed Sample Tests vs. a Property-Based Invariant Fuzzing Sensor (-95.1% Input Tokens)

Why Coding Agents Need a Pre-Submit Diff Audit Gate: Blind Unchecked Submits vs. Blast-Radius & Entropy Auditing (-95.0% Input Tokens)

Stop Letting Coding Agents Do Manual Multi-Spot String Replacements: Manual Text Edits vs. Parameterized AST Codemod Tools (-94.8% Input Tokens)

Metadata & Experiment Identity

1. Executive Summary (TL;DR)

Key Result

2. Experiment Architecture: Monolithic Encyclopedia vs. Directory-Pointer AGENTS.md

3. Drop-In Implementation Recipe: resolve_scoped_agents_md

4. Practical Takeaway: Treat AGENTS.md as a Map, Not an Encyclopedia

🔗 Continue Reading the Series

Comments

More from this blog

Eliminating Print-Debugging Loops in Coding Agents: Bare Stack Traces vs. Runtime Frame Variable-Snapshot Injection (-95.0% Input Tokens)

Why Hand-Written Example Tests Fail AI Coding Agents: Fixed Sample Tests vs. a Property-Based Invariant Fuzzing Sensor (-95.1% Input Tokens)

Why Coding Agents Need a Pre-Submit Diff Audit Gate: Blind Unchecked Submits vs. Blast-Radius & Entropy Auditing (-95.0% Input Tokens)

Stop Letting Coding Agents Do Manual Multi-Spot String Replacements: Manual Text Edits vs. Parameterized AST Codemod Tools (-94.8% Input Tokens)

2. Experiment Architecture: Monolithic Encyclopedia vs. Directory-Pointer `AGENTS.md`

3. Drop-In Implementation Recipe: `resolve_scoped_agents_md`

4. Practical Takeaway: Treat `AGENTS.md` as a Map, Not an Encyclopedia