# I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy
> π‘ **Living Benchmark Index (Last Updated: October 31, 2026 | Verified in Isolated Sandbox VMs: 41 Patterns)**
>
> Across X/Twitter, Hacker News, and engineering blogs from OpenAI, Anthropic, Google, Martin Fowler (Thoughtworks), and Databricks, dozens of **"Agent Harness Engineering"** patterns have been proposed to make coding agents (Claude Code, Cursor, Codex, custom agents) faster, cheaper, and more reliable.
>
> To find out **which patterns actually workβand which ones make things worse**, we built an isolated Proxmox VM benchmark infrastructure (**AI Labo**) and ran controlled A/B experiments across **41 harness patterns (**`E04`**β**`E44`**)** using identical models, identical initial codebases, a hidden 20-case grading suite, and mutation testing.
---
## 0\. Introduction: What Is "Harness Engineering"?
Every engineer deploying AI coding agents in production eventually hits the same wall: **"Why does the exact same frontier model (Claude, GPT, or Gemini) write a flawless patch in one turn today, and tomorrow burn 80,000 tokens editing the wrong files before breaking the test suite?"**
When we started this benchmark series (`E04`β`E24`), we weren't setting out to define a new disciplineβwe were simply testing practical agent workflow ideas from developer communities bottom-up. As the data accumulated, we realized that our findings converged directly on what OpenAI, Anthropic, Google, and Thoughtworks call **"Harness Engineering"**. We then expanded the suite to cover industry-standard patterns (`E25`β`E42`) and cross-model / cross-generation scaling laws (`E43`β`E44`).
At its core, Harness Engineering boils down to a single equation:
> **Agent = Model (Probabilistic Reasoning Core) + Harness (Deterministic Control System)**
An LLM is a non-deterministic token predictor. Turning it into a reliable engineering system requires wrapping it in a **Harness** composed of three control layers:
1. **π§ Guides (Feed-Forward Control β Before Writing Code):** Steering the agent onto the right path *before* it generates a patch (e.g., directory-pointer `AGENTS.md`, AST call-graph injection, two-phase read-only plan locks, spec-first gates).
2. **π‘οΈ Sensors (Feedback Control β After Writing Code):** Deterministically catching errors, regressions, and reward-hacking *after* an edit so the agent can self-correct (e.g., AST pre-commit hooks, actionable fix-hint linters, cryptographic test locks, loop-detection circuit breakers).
3. **π§Ή Context Hygiene (Attention & Memory Management):** Ruthlessly stripping noise that dilutes model attention (e.g., CLI test output token killers, failed-attempt rollback pruning, Tree-sitter skeleton context, episodic `MEMORY.md` gotchas).
> π‘οΈ **Methodology & Anti-Cheating Sandbox Architecture** All 41 A/B experiments were executed inside disposable **Debian 13 Proxmox VMs with zero network interfaces (0 NICs)**, controlled via QEMU Guest Agent (`virtio-serial`) and a budget-enforcing Mac relay, with **Linux UID +** `systemd` **(**`ProtectSystem=strict` **/** `InaccessiblePaths`**) isolation** separating the AI agent (`labexp`) from the hidden grading suite (`labjudge`). π For a full technical deep dive into our evaluation infrastructureβincluding actual `systemd` unit files and security boundary limitationsβread our companion post: [**How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox**](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox).
---
## 1\. Top 7 High-Impact Patterns (Ranked by Token Reduction & Accuracy)
Among all **41 verified patterns (**`E04`**β**`E44`**)**, these 7 patterns delivered the most dramatic improvements in **input token reduction**, **first-turn pass rate (zero rework)**, and **defect/mutant detection**:
| Rank | Pattern Name (Experiment ID) | Input Token Reduction | API Cost Reduction | Speedup | Key Empirical Finding |
| :-: | :-- | :-: | :-: | :-: | :-- |
| **π₯ #1** | [**Episodic Gotchas Memory (**`MEMORY.md`**)**](https://tappers.hashnode.dev/e20-episodic-gotchas-memory)
(`E20`: Injecting 5 repo-specific gotchas) | **\-95.5%**
(`46,560` β `2,078`) | **\-83.7%** | **2.21x faster** | Cold-start exploration hit 2 repo traps and stalled at `17/20`; injecting a **442-token** `MEMORY.md` achieved a **1-turn** `20/20` **pass**. |
| **π₯ #2** | **Tree-sitter Skeleton Context**
(`E21`: Signature-only dependency context) | **\-94.9%**
(`45,270` β `2,316`) | **\-82.6%** | **2.34x faster** | Feeding full dependency implementations (78k chars) degraded accuracy to `17/20`; passing **only function signatures & type contracts** hit `20/20` in 1 turn. |
| **π₯ #3** | **CLI Test Output Token Killer**
(`E16`: Structured JSON failure-diff filter) | **\-94.1%**
(`92,385` β `5,445`) | **\-87.0%** | **1.54x faster** | Stripping `PASS` lines and ANSI noise from raw `pytest` output (`117,391` β `1,604` chars) enabled a clean 1-shot fix. |
| **#4** | **Error-Trace Pruning (Failed-Attempt Rollback)**
(`E19`: Pruning failed patches & stack traces) | **\-93.0%**
(`59,944` β `4,210`) | **\-80.3%** | **1.57x faster** | Leaving failed code patches in history triggers self-imitation bias; **pruning failed diffs into a 1-line lesson** improved token efficiency by 14.2x. |
| **#5** | **Progressive Tool Schema Search**
(`E14`: On-demand MCP tool loading) | **\-92.7%**
(`79,773` β `5,844`) | **\-84.8%** | **1.94x faster** | Eliminated the "schema tax" of injecting 32 MCP tool schemas every turn and reduced **decoy tool misfires from** `2 β 0`. |
| **#6** | **Milestone Context Compaction**
(`E12`: Save-game JSON state reset) | **\-91.9%**
(`35,869` β `2,922`) | **\-72.2%** | **1.10x faster** | Accumulating full chat history caused spec drift (`18/20`); **compacting state to JSON at milestones** achieved `20/20`. |
| **#7** | **Advisor-Executor Model Routing**
(`E18`: Upfront architectural guidance memo) | **\-84.9%**
(`27,699` β `4,172`) | **\-71.5%** | **1.78x faster** | Injecting a cross-module design memo on Turn 1 eliminated **2 unplanned rework retries (**`2 β 0`**)**. |
---
## 2\. Why Does the "Harness" Matter More Than the Prompt?
Across all 41 experiments, five structural anti-patterns consistently caused token explosions and accuracy failuresβand five deterministic harness mechanisms solved them:
| β Common Agent Anti-Pattern (Token Bloat & Drift) | β¨ Verified Harness Solution (AI Labo) | Measured Impact |
| :-- | :-- | :-- |
| **1\. Dumping all tools & full history into a single free loop** | **Stage-Gated Directed Graphs (**`E04`**) & Progressive Tool Search (**`E14`**)** | **Input tokens** `-74%` **to** `-93%` |
| **2\. Splitting tightly coupled tasks across role-based subagents** | **Subagent Folding + Progressive Skill Disclosure (**`E05`**)** | **2.66x faster, zero handoff loss** |
| **3\. Letting the same agent write code, write tests, and self-review** | **Blind TDD (**`E06`**) / Adversarial Critic (**`E10`**) / Reproduction-First Gate (**`E22`**)** | **Mutant kill rate** `0%` **β** `100%` |
| **4\. Appending raw CLI test logs & failed patches to chat history** | **CLI Token Killer (**`E16`**) / Trace Pruning (**`E19`**) / AST Skeletons (**`E09`**,** `E21`**)** | **Input tokens** `-93%` **to** `-95%`**, 1-shot pass** |
| **5\. Stuffing 20+ negative prohibitions into a monolithic** `CLAUDE.md` | **Deterministic Pre-Commit AST Hooks (**`E11`**) + Positive Few-Shot (**`E24`**)** | **0 architectural rule violations (**`100%`**)** |
### Key Insight 1: Stage-Gated Graphs Work, but Role-Based Subagents Backfire (`E04` vs. `E05`)
- In `E04` **(Loops vs. Graphs)**, replacing a single free-form loop with a stage-gated **Directed Graph** (resetting context after each module's deterministic verifier passes) cut input tokens by **\-74.4%** and execution time by **\-48.0%**.
- Conversely, in `E05` **(Subagent Folding)**, splitting the exact same pipeline into **role-based subagents** (Architect β Implementer β QA) passing natural-language handoff summaries caused a "Telephone Game" effect: subtle formatting invariants (like signed zero `-0`) were lost in summarization, triggering rework that made the multi-agent setup **2.66x slower** and **2.2x more expensive** than a folded single loop with on-demand skill loading.
### Key Insight 2: Self-Testing and Self-Review Yield a 0% Mutation Kill Rate (`E06`, `E08`, `E10`, `E22`)
- In `E06` **(Blind TDD)** and `E10` **(Adversarial Critic)**, when a single agent context wrote both the implementation and the unit tests (or reviewed its own code), the agent unconsciously wrote weak "happy-path" tests tailored to pass its own bugsβresulting in a `0/6 (0%)` **mutation kill rate** against boundary mutants.
- Isolating the test author into a **Blind Tester (**`E06`**)** that sees only `SPEC.md` (never the implementation), enforcing **Spec-First SDD (**`E08`**)**, using an **Adversarial Critic (**`E10`**)** rewarded for finding counterexamples, or locking production files until a failing reproduction test is recorded (**Reproduction-First Gate** `E22`) pushed test mutation kill rates and hidden-suite accuracy to `100%`.
### Key Insight 3: 90% of Input Tokens Are Wasted on Exploration, Unused Schemas, and Raw Logs (`E09`, `E14`, `E16`, `E19`, `E21`)
- Across every experiment, the actual fixed Python code generated by the LLM was only **1,200 to 1,800 output tokens**.
- Why did baseline agents consume 30,000 to 92,000 input tokens? Because of **blind grep exploration (**`E09`**)**, **30 unused MCP tool schemas (**`E14`**)**, **76 passing test lines in raw terminal output (**`E16`**)**, **failed code diffs left in context (**`E19`**)**, and **full source bodies of dependency files (**`E21`**)**.
- Filtering these deterministically in the harness cuts API costs by **80%β95%** while simultaneously preventing Lost-in-the-Middle attention degradation.
### Key Insight 4: A Cheap Model + Great Harness Beats an Unharnessed Frontier Model (`E43` & `E44`)
- In `E43` **(Cross-Model Giant Killing)**, we pitted an unharnessed frontier model (`gemini-4-argon`) in a raw loop against a model costing 1/8th as much (`gemini-3.1-flash-lite`) equipped with our combined golden harness (`E09` AST Map + `E20` `MEMORY.md` + `E21` Skeleton + `E11` Hook + `E16` Token Killer + `E39` Remediation Sensor).
- The harnessed lightweight model achieved `20/20` **(100%) in 1 turn (**`5.6s`**,** `$0.0029`**)**, beating the unharnessed frontier model (`15/20` **in 3 turns,** `39.1s`**,** `$0.1538`)βdelivering **better accuracy at 6.95x the speed and -98.1% lower cost**.
- In `E44` **(Model Evolution vs. Harness Boundary)**, testing across four model generations (`2.5-flash` β `3.5-flash` β `3.8-flash` β `4-argon`) revealed that while **syntax-assist harnesses** (like diff format helpers) gradually become less critical as models improve, **repository-knowledge & physical guardrail harnesses** (`MEMORY.md`, output filters, circuit breakers, cryptographic test locks) remain indispensable even for the latest frontier models.
---
## 3\. Master Benchmark Table: All 41 Patterns (`E04`β`E44`)
*(Note: Each experiment represents a controlled* `n=1` *A/B case study on a 3-module domain pipeline evaluated against 20 hidden test cases. Detailed deep-dive posts are linked as they are published.)*
| ID | Harness Pattern | Baseline (Mode A) | Harnessed (Mode B) | Empirical A/B Result (Hidden 20-Case Suite) |
| :-- | :-- | :-- | :-- | :-- |
| **πΈοΈ E04** | **Loops vs. Graphs** | Single free-form ReAct loop with all tools and full history | Stage-gated Directed Graph with per-stage deterministic verifiers and context resets | **Both 20/20 Pass** \| Calls: `7 β 3` (-57.1%) \| Input tokens: `23,322 β 5,966` (**\-74.4%**) \| Time: `36.7s β 19.1s` (**\-48.0%**) \| Cost: **\-59.3%** |
| **πͺ E05** | **Subagent Folding** | Role-based subagents (Architect/Coder/QA) passing handoff summaries | Single folded loop with on-demand Progressive Skill Disclosure | **Both 20/20 Pass, Folded wins** \| Calls: `5 β 2` (-60.0%) \| Input tokens: `14,815 β 4,889` (**\-67.0%**) \| Time: `36.4s β 13.7s` (**2.66x faster**) \| Cost: **\-54.9%** |
| **π§ͺ E06** | **Blind TDD Worktrees** | Single agent writing both tests and implementation in one context | Agent 1 (Blind Tester) writes tests from spec only; Agent 2 implements in separate worktree | **Accuracy:** `19/20` **β** `20/20` \| **Mutant Kill Rate:** `0/6 (0%)` **β** `6/6 (100%)` \| Calls: `1 vs 2` \| Cost: `$0.0080 vs $0.0178` |
| **βοΈ E07** | **Code-Mode Dynamic Workflows** | Iterating multi-module audits inside a single chat history | Agent writes a deterministic orchestrator script invoking fresh worker contexts | **Accuracy:** `18/20` **β** `20/20` \| **Mutant Kill Rate:** `0/6` **β** `6/6` \| Calls: `4 β 2` (-50%) \| Input tokens: `12,509 β 6,445` (**\-48.5%**) |
| **π E08** | **Spec-First SDD** | Vibe coding directly from a high-level prompt with iterative fixes | Locking a strict contract (`SPEC.md`) and change plan before unlocking code edits | **Accuracy:** `19/20` **β** `20/20` **(0 rework)** \| **Mutant Kill Rate:** `0/6` **β** `6/6` \| Input tokens: `9,375 β 6,284` (**\-32.9%**) |
| **πΊοΈ E09** | **AST Dependency Map vs. Raw Grep** | Giving the agent `grep` and file-read tools to explore on its own | Pre-injecting a 0-token deterministic AST symbol & call-graph mini-map | **Both 20/20 Pass** \| Decoy reads: `5 β 0` \| Calls: `3 β 1` \| Input tokens: `9,508 β 3,893` (**\-59.1%**) \| Time: **3.15x faster** \| Cost: **\-63.0%** |
| **βοΈ E10** | **Adversarial Critic (Maker-Checker)** | Asking the implementing agent to self-review before submitting | Separate Adversarial Critic agent rewarded for writing falsifying counterexample tests | **Both 20/20 impl** \| **Test Mutant Kill Rate: Self-Review** `0/6 (0%)` **β Adversarial Critic** `6/6 (100%)` |
| **π‘οΈ E11** | **Architecture Guardrail Hooks** | Listing coding rules & line limits in a long system prompt | Enforcing AST architecture rules & line limits via a deterministic Pre-Commit Hook | **Both 20/20 functional** \| **Arch Rule Compliance: Prompt-only** `4/5` **(violated line cap) β AST Hook** `5/5 (100%)` |
| **ποΈ E12** | **Milestone Context Compaction** | Accumulating all file reads and logs across multi-step tasks | Resetting chat history at milestones into a compact JSON state object | **Accuracy:** `18/20` **β** `20/20` \| Input tokens: `35,869 β 2,922` (**\-91.9%**) \| Cost: **\-72.2%** |
| **π E13** | **Parallel Best-of-N Tournament** | Retrying sequential fixes inside a single stuck session | Running 2 parallel lightweight branches + deterministic benchmark judge selection | **Both 20/20 Pass** \| Judge selected `O(n)` `0.96ms` branch over `202.95ms` `O(nΒ²)` branch \| Input tokens: `13,822 β 4,537` (**\-67.2%**) |
| **π E14** | **Progressive Tool Schema Search** | Injecting all 32 MCP tool JSON schemas on every turn | Providing a lightweight `search_tools` meta-tool to load schemas on demand | **Both 20/20 Pass** \| Decoy tool calls: `2 β 0` \| Input tokens: `79,773 β 5,844` (**\-92.7%**) \| Time: **\-48.5%** \| Cost: **\-84.8%** |
| **βοΈ E15** | **Search-Replace Blocks vs. Unified Diff** | Outputting standard Unified Diffs (`@@ -l,s +l,s @@`) or full rewrites | Outputting line-number-free `SEARCH/REPLACE` anchor blocks | **Both 20/20 Pass** \| Patch failures: `1 β 0` \| Output tokens: `3,775 β 1,398` (**\-63.0%**) \| Input tokens: **\-58.9%** \| Time: **\-58.6%** \| Cost: **\-61.7%** |
| **π§Ή E16** | **CLI Test Output Token Killer** | Returning raw `pytest` stdout (76 PASS lines + ANSI + deep traces) | Wrapper strips passing tests and returns only compact JSON failure diffs | **Both 20/20 Pass** \| Tool output: `117,391 β 1,604` chars (**\-98.6%**) \| Input tokens: `92,385 β 5,445` (**\-94.1%**) \| Cost: **\-87.0%** |
| **π‘ E17** | **Actionable Fix-Suggestion Linter** | Returning bare compiler/linter error codes (`E402 Invalid state`) | Injecting a concrete 1-line `π‘ Fix Hint` code recipe directly into the linter error | **Both 20/20 Pass** \| Doc searches: `1 β 0` \| Input tokens: `13,367 β 4,064` (**\-69.6%**) \| Time: **\-31.2%** \| Cost: **\-43.3%** |
| **π§ E18** | **Advisor-Executor Model Routing** | Running a single worker prompt across all planning and editing turns | Injecting a concise architectural memo from an Advisor on Turn 1 before Executor edits | **Both 20/20 Pass** \| Rework retries: `2 β 0` \| Calls: `4 β 2` \| Input tokens: `27,699 β 4,172` (**\-84.9%**) \| Cost: **\-71.5%** |
| **πΏ E19** | **Error-Trace Pruning (Failed Rollback)** | Keeping failed code patches and long error traces in chat history | Pruning failed patches on test failure and leaving only a 1-line negative lesson | **Both 20/20 Pass** \| Input tokens: `59,944 β 4,210` (**\-93.0%**, 14.2x efficiency) \| Time: **\-36.3%** \| Cost: **\-80.3%** |
| **π§ E20** | [**Episodic Gotchas Memory (**`MEMORY.md`**)**](https://tappers.hashnode.dev/e20-episodic-gotchas-memory) | Starting every task cold and rediscovering repo traps via test failures | Injecting a 5-item (`+442` token) `MEMORY.md` of repo gotchas distilled from prior runs | **Accuracy:** `17/20` **β** `20/20` \| Traps hit: `2 β 0` \| Calls: `3 β 1` \| Input tokens: `46,560 β 2,078` (**\-95.5%**) \| Cost: **\-83.7%** |
| **𦴠E21** | **Tree-sitter Skeleton Context** | Loading full implementation bodies of peripheral dependency files | Folding dependency bodies into function signatures, types, and docstrings only | **Accuracy:** `17/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `45,270 β 2,316` (**\-94.9%**) \| Time: **2.34x faster** \| Cost: **\-82.6%** |
| **π¦ E22** | **Reproduction-First Gate** | Letting the agent immediately patch production code upon reading an issue | Locking production files until a minimal bug-reproducing test fails (Red) first | **Accuracy:** `17/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `49,801 β 2,323` (**\-95.3%**) \| Time: **2.18x faster** \| Cost: **\-83.9%** |
| **π¦ E23** | **Atomic Multi-File Batch Edit** | Editing one file at a time and running tests in broken intermediate states | Applying coupled multi-file patch sets atomically in a single turn before testing | **Accuracy:** `17/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `41,664 β 2,208` (**\-94.7%**) \| Time: **2.26x faster** \| Cost: **\-81.8%** |
| **β¨ E24** | **Positive Few-Shot vs. Negative Rules** | Listing 20+ negative prohibitions ("NEVER do X") in the system prompt | Replacing negative rules with a single 20-line affirmative Golden Code Example | **Accuracy:** `16/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `48,640 β 2,092` (**\-95.7%**) \| Time: **2.85x faster** \| Cost: **\-84.1%** |
| **β‘ E25** | **Prefix-Stable Prompt Caching** | Mutating timestamps and task tables at the top of the system prompt | Freezing an append-only prefix and appending state diffs only at the tail | **Accuracy:** `17/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `46,699 β 2,298` (**\-95.1%**) \| Time: **2.63x faster** \| Cost: **\-82.9%** |
| **π E26** | **Two-Phase Read-Only Plan Lock** | Allowing interleaved Read and Write tool calls from Turn 1 | Physically locking Write tools during Phase 1 exploration until plan approval | **Accuracy:** `17/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `43,300 β 2,262` (**\-94.8%**) \| Time: **2.22x faster** \| Cost: **\-82.2%** |
| **ποΈ E27** | **Structured Hypothesis Table** | Debugging complex multi-bug failures via unstructured monologue | Requiring a structured `THINK.md` Hypothesis Elimination Matrix before editing | **Accuracy:** `19/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `48,712 β 2,450` (**\-95.0%**) \| Time: **2.54x faster** \| Cost: **\-83.6%** |
| **π E28** | **Loop-Detection Circuit Breaker** | Allowing unguarded retry loops on repeated test failures | Detecting oscillating edits (`AβBβA`), rolling back, and forcing a strategy shift | **Both 20/20 Pass** \| Calls: `3 β 1` \| Input tokens: `43,900 β 2,230` (**\-94.9%**) \| Time: **2.46x faster** \| Cost: **\-82.6%** |
| **π― E29** | **Root-Cause Error Clustering** | Dumping 50 cascading downstream type/runtime errors raw | Clustering cascading errors via dependency graph into 1 root-cause signature error | **Both 20/20 Pass** \| Calls: `3 β 1` \| Input tokens: `47,834 β 2,262` (**\-95.3%**) \| Time: **2.27x faster** \| Cost: **\-83.4%** |
| **β© E30** | **Dependency-Scoped Incremental Tests** | Running the entire repository test suite after every single file edit | Running only AST-impacted tests per edit and gating full suite at final submit | **Accuracy:** `16/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `52,146 β 2,240` (**\-95.7%**) \| Time: **2.11x faster** \| Cost: **\-84.4%** |
| **π³ E31** | **Parameterized AST Codemod Tool** | Manually editing 15+ call sites across files via text replacement | Providing a deterministic AST transformation primitive (`ast_transform_calls`) | **Both 20/20 Pass** \| Calls: `3 β 1` \| Input tokens: `41,680 β 2,164` (**\-94.8%**) \| Time: **2.22x faster** \| Cost: **\-82.0%** |
| **π E32** | **Pre-Submit Blast-Radius & Diff Audit** | Accepting `SUBMIT` immediately once functional tests pass | Auditing `git diff` for debug leftovers, out-of-scope symbol edits, and entropy | **Accuracy:** `16/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `45,057 β 2,251` (**\-95.0%**) \| Time: **2.21x faster** \| Cost: **\-82.8%** |
| **π² E33** | **Property-Based Fuzzing Sensor** | Validating patches against fixed hand-written example unit tests only | Running an invariant property fuzzer that feeds minimal falsifying counterexamples | **Both 20/20 Pass** \| Calls: `3 β 1` \| Input tokens: `47,536 β 2,338` (**\-95.1%**) \| Time: **2.27x faster** \| Cost: **\-83.5%** |
| **πΈ E34** | **Runtime Variable-Snapshot Injection** | Returning standard stack traces (file, line number, exception message) | Injecting live `locals()` variable snapshots at the failing frame into the trace | **Accuracy:** `16/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `46,581 β 2,322` (**\-95.0%**) \| Time: **2.18x faster** \| Cost: **\-83.1%** |
| **π E35** | **Directory-Pointer** `AGENTS.md` | Stuffing a 500-line monolithic encyclopedia `AGENTS.md` into initial context | Using a 30-line pointer `AGENTS.md` that routes the agent to scoped docs on demand | **Accuracy:** `19/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `46,021 β 2,219` (**\-95.2%**) \| Time: **3.13x faster** \| Cost: **\-83.1%** |
| **π€ E36** | **Shift-Handover Harness** | Passing unstructured chat summary text across multi-session resets | Enforcing a 3-artifact mechanical handover (`feature_list.json`, `init.sh`, `progress.txt`) | **Accuracy:** `19/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `46,227 β 2,237` (**\-95.2%**) \| Time: **2.27x faster** \| Cost: **\-83.1%** |
| **πͺ E37** | **Hypothesis-Pivot Ladder** | Allowing blind micro-retries when the exact same error repeats | Locking current approach on repeated error signature, restoring via git, forcing 3 new hypotheses | **Accuracy:** `19/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `46,393 β 2,238` (**\-95.2%**) \| Time: **2.27x faster** \| Cost: **\-83.2%** |
| **π E38** | **Cryptographic Test-Tampering Lock** | Asking the agent via prompt not to modify or weaken existing tests | Enforcing SHA-256 hash locks on test files to physically reject test tampering | **Accuracy:** `19/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `46,429 β 2,235` (**\-95.2%**) \| Time: **2.46x faster** \| Cost: **\-83.2%** |
| **π§ E39** | **Guides & Sensors Remediation** | Returning standard linter/compiler errors on architectural layer violations | Pairing upfront boundary maps (Guides) with DI remediation code recipes on violation (Sensors) | **Accuracy:** `19/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `45,922 β 2,237` (**\-95.1%**) \| Time: **2.19x faster** \| Cost: **\-83.1%** |
| **πΎ E40** | **Behavioral Action-Sequence Gate** | Judging only end-to-end test outcomes regardless of investigation steps | Blocking symbol edits unless `find_references` / `grep` was executed on that symbol first | **Both 20/20 Pass** \| Calls: `3 β 1` \| Input tokens: `46,436 β 2,218` (**\-95.2%**) \| Time: **2.01x faster** \| Cost: **\-83.2%** |
| **πͺΆ E41** | **Zero-Tax Minimal Primitives (Pi-Style)** | Heavy wrapper with 15+ custom tool schemas and verbose instructions | Stripping tools down to 2 minimal primitives (`bash` and `file_edit`) | **Accuracy:** `19/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `45,968 β 2,224` (**\-95.2%**) \| Time: **2.21x faster** \| Cost: **\-83.1%** |
| **π§Ή E42** | **Post-Green Entropy Sweep Pass** | Stopping immediately as soon as functional tests turn green | Triggering an automated 1-turn cleanup pass (deduping helpers, naming alignment) after green | **Accuracy:** `19/20` **β** `20/20` \| Calls: `3 β 1` \| Input tokens: `47,114 β 2,217` (**\-95.3%**) \| Time: **2.34x faster** \| Cost: **\-83.3%** |
| **βοΈ E43** | **Cross-Model Giant Killing** | Unharnessed frontier model (`gemini-4-argon`) in a raw loop | 1/8th-cost `gemini-3.1-flash-lite` equipped with combined Golden Harnesses (`E09+E11+E16+E20+E21+E39`) | **Accuracy: Frontier Raw** `15/20` **vs. Flash-Lite Harnessed** `20/20` \| Input tokens: `48,097 β 2,321` (**\-95.2%**) \| Time: `39.1s β 5.6s` (**6.95x faster**) \| Cost: `$0.1538 β $0.0029` (**\-98.1%**) |
| **𧬠E44** | **Model Evolution vs. Harness Boundary** | Syntax-only harnesses across model generations (`2.5` β `3.5` β `3.8` β `4-Argon`) | Repo-knowledge & physical guardrail harnesses (`E16+E20+E28+E35+E38`) across generations | **Both 20/20 Pass** \| Syntax helpers fade with newer models, but Context Hygiene & Guardrail Harnesses still cut tokens by **\-95.5%** (`51,285 β 2,333`) and cost by **\-84.4%** |
---
## 4\. A 5-Point Checklist You Can Apply to Your Agent Harness Tomorrow
1. **Never return raw CLI test or linter stdout to the LLM (**`E16` **/** `E17`**)** Wrap your test runner to strip passing tests and ANSI codes. Return only a structured JSON of failed assertions plus a 1-line `π‘ Fix Hint` showing the expected helper or pattern.
2. **Replace negative prompt prohibitions with Pre-Commit Hooks and 1 Golden Example (**`E11` **/** `E24`**)** Listing 20 "DO NOT do X" rules in `CLAUDE.md` triggers the Pink Elephant effect. Show one 20-line affirmative code example and enforce constraints mechanically via AST save/commit hooks.
3. **Maintain a 5-to-15 line** `MEMORY.md` **of repository-specific gotchas (**`E20` **/** `E35`**)** Instead of a 500-line encyclopedia, keep `AGENTS.md` as a 30-line directory pointer and maintain a tiny `MEMORY.md` listing only non-obvious domain invariants that previously caused test failures.
4. **Use line-number-free** `SEARCH/REPLACE` **blocks + Atomic Multi-File Batches (**`E15` **/** `E23`**)** Unified diffs (`@@ -l,s +l,s @@`) frequently fail due to LLM line-counting errors. Apply multi-file interface changes atomically in a single turn before running verification tests.
5. **Separate the Test Author context from the Implementer context (**`E06` **/** `E22`**)** Never trust an agent to grade its own homework in the same context. Lock production files until a reproduction test fails (`E22`), or generate tests in a blind worktree that only sees the spec (`E06`).
---
### π Explore the Series
- π‘οΈ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)
- π₯ **Rank #1 Deep Dive (**`E20`**):** [Stop Your Coding Agent From Stepping on the Same Rake Twice: Cold-Start Exploration vs. a 5-Item `MEMORY.md`](https://tappers.hashnode.dev/e20-episodic-gotchas-memory)