# Why Keeping Failed Code Attempts in Chat History Poisons Your Coding Agent: Error-Trace Pruning vs. Accumulated History > πŸ’‘ **AI Labo Benchmark Series β€” Experiment** `E19` **(Ranked #4 out of 41 Patterns)** > > This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite. > > - πŸš€ [**View All 41 Benchmark Results & Master Ranking**](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark) > - πŸ›‘οΈ [**Read How Our Proxmox & systemd Anti-Cheating Sandbox Works**](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox) --- ## Metadata & Experiment Identity - **Experiment ID:** `E19` (Backlog Candidate `C015`) - **Run ID:** `e19-20261028-vm223-01` - **Sandbox Environment:** Proxmox Isolated VM `VM223` (`ai-e19-223`, 0 NICs, `virtio-serial` QGA relay, `systemd` UID separation) - **Model:** `gemini-3.8-flash` (`thinkingLevel: LOW`) - **Pattern Under Test:** Context Poisoning Prevention β€” *"Error-Trace Pruning (Failed-Attempt Rollback) vs. Accumulated Failed History"* --- ## 1\. Executive Summary (TL;DR) When a coding agent attempts a multi-file fix and the first patch fails tests, standard agent loops simply append the **failed code diff (**`role: "model"`**)** and the **verbose error stack trace (**`role: "user"`**)** to the conversation history (`messages[]`) and ask the LLM to try again. This creates **Self-Conditioning Bias (Context Poisoning)**: LLMs strongly anchor on their own prior generated code in the context window. Instead of rethinking the cross-module architecture, the agent keeps making narrow incremental tweaks on top of its broken first attempt while stack traces snowball across turns. To test the antidoteβ€”**Error-Trace Pruning (Failed-Attempt Rollback)**β€”we benchmarked our 3-module payment settlement pipeline (`validator.py`, `settlement.py`, `statement.py`) inside isolated Proxmox VM `VM223` across two modes: - **Mode A (**`AccumulatedFailedHistoryRunner`**):** Retains failed partial patches and full error stack traces (`36,606 chars` on Turn 1; `36,264 chars` on Turn 2) in the conversation history across retries. - **Mode B (**`PrunedFailedAttemptRunner`**):** Immediately rolls back the workspace on test failure (`git checkout`), **prunes the failed model output and raw stack trace from** `messages[]`, and appends only a `927-character` **(**`+392-token`**) structured negative lesson** to the clean initial prompt. ### Key Result While both modes eventually passed all `20 / 20` **(100%)** hidden test cases, pruning failed turns in Mode B eliminated the second failed retry (`3` calls βž” `2` calls), compressed final-turn input tokens by `-94.0%` **(**`38,134` **βž”** `2,301` **tokens)**, cut cumulative input tokens by `-93.0%` **(**`59,944` **βž”** `4,210` **tokens,** `14.24x` **efficiency)**, reduced API cost by `-80.3%` **(**`$0.0558` **βž”** `$0.0110`**)**, and finished `1.57x` **faster**. | Metric | Mode A: Accumulated Failed History (`AccumulatedHistory`) | Mode B: Error-Trace Pruning (`PrunedFailedAttempt`) | Delta / Improvement | | :-- | :-: | :-: | :-: | | **Hidden Grading Suite Score (20 Cases)** | `20 / 20` (100%) | `20 / 20` **(100%)** | Both passed all 20 cases | | **Retained Failed Turns in History (**`retained_failed_turns`**)** | `2 turns` | `0 turns` | **\-100% (Zero context poisoning)** | | **Pruned Failed Turns (**`pruned_failed_turns`**)** | `0 turns` | `1 turn` **(Rolled back & pruned)** | Replaced 36KB trace with 927-char lesson | | **LLM Calls (**`model_calls`**)** | `3 calls` | `2 calls` | **\-33.3% fewer calls** | | **Total File Edits (**`file_edits`**)** | `5 edits` | `4 edits` | **\-20.0% reduction** | | **Cumulative Tool Output (**`tool_output_chars`**)** | `72,975 chars` | `1,027 chars` | **\-98.6% (71.06x compression)** | | **Final-Turn Input Tokens** | `38,134 tokens` (Turn 3) | `2,301 tokens` **(Turn 2)** | **\-94.0% (16.57x lighter final prompt)** | | **Cumulative Input Tokens (**`promptTokenCount`**)** | `59,944 tokens` | `4,210 tokens` | **\-93.0% (14.24x token efficiency)** | | **Cumulative Output Tokens (**`candidatesTokenCount`**)** | `2,892 tokens` | `2,082 tokens` | **\-28.0% reduction** | | **Elapsed Execution Time (**`elapsed_seconds`**)** | `30.70 sec` | `19.55 sec` | **\-36.3% (1.57x faster)** | | **Estimated API Cost (**`estimated_micro_usd`**)** | `$0.0558` (`55,804 micro-USD`) | `$0.0110` **(**`10,966 micro-USD`**)** | **\-80.3% (5.09x cheaper)** | --- ## 2\. Experiment Architecture & Rollback-Prune Workflow ```text +-------------------------------------------------------------------------+ | Proxmox Isolated VM223 (0 NICs, virtio-serial QGA, systemd UID split) | | Target Task: 3-Module Settlement Pipeline Bugfix | | | | +-------------------------------------------------------------------+ | | | Mode A: AccumulatedFailedHistoryRunner (Keeps Failed Turns) | | | | - Turn 1: Partial patch (validator.py) FAILS (+36,606 char trace)| | | | - Turn 2: Anchors on Turn 1; patches only settlement.py | | | | FAILS again (+36,264 char trace; 19,912 in tokens) | | | | - Turn 3: Submits all 3 files at 38,134 input tokens (59,944 sum)| | | +-------------------------------------------------------------------+ | | | | +-------------------------------------------------------------------+ | | | Mode B: PrunedFailedAttemptRunner (Rollback + Prune Failed Turn) | | | | - Turn 1: Partial patch (validator.py) FAILS (1,909 in tokens) | | | | - Harness Action: | | | | 1. Roll back workspace files to clean snapshot | | | | 2. Delete Turn 1 model output & 36,606-char trace from chat | | | | 3. Append 927-char structured lesson (+392 tokens) | | | | - Turn 2: Fresh retry at 2,301 input tokens --> Submits all 3 | | | | files in 1 shot (20 / 20 PASS, 4,210 cumulative in) | | | +-------------------------------------------------------------------+ | +-------------------------------------------------------------------------+ ``` --- ## 3\. Turn-by-Turn Token Progression & Prompt Comparison ### 3.1 Turn-by-Turn Token & Cost Breakdown (`usage.json`) | Seq | Mode | Node ID | Input Tokens | Output Tokens | API Cost (`micro-USD`) | Action & Outcome | | :-: | :-- | :-- | :-: | :-: | :-: | :-- | | `0` | Mode A | `accumulated_history_turn_1` | `1,898` | `497` | `3,288` | `attempt_patch` (`validator.py` only βž” Fails; kept in history) | | `1` | Mode A | `accumulated_history_turn_2` | `19,912` | `793` | `17,908` | `attempt_patch` (`settlement.py` only βž” Fails; kept in history) | | `2` | Mode A | `accumulated_history_turn_3` | `38,134` | `1,602` | `34,608` | `submit_settlement_bundle` (All 3 files βž” `20/20` Pass) | | `3` | **Mode B** | `pruned_attempt_turn_1` | `1,909` | `496` | `3,292` | `attempt_patch` (`validator.py` only βž” **Rolled back & pruned!**) | | `4` | **Mode B** | `pruned_attempt_turn_2` | `2,301` | `1,586` | `7,674` | `submit_settlement_bundle` (Clean history + lesson βž” `20/20` **Pass**) | ### 3.2 What Mode A Kept in History (`input-01.json` / `input-02.json`) In Mode A, after Turn 1 patched only `validator.py`, the harness appended `36,606 characters` of unpruned diff frames and local variable dumps (`MSG[2]`), followed by another failed partial attempt (`MSG[3]`) and `36,264 characters` of traces (`MSG[4]`): ```text === FAILED ATTEMPT #1 FULL DIFF & TRACEBACK DUMP (UNPRUNED) === Status: FAILED (3 invariants broken) [FAILED_PATCH_CONTEXT_FRAME_001] attempt=1 rejected_hunk_sha256=0000...0001 locals={'merchant_id': 'M-1', 'currency': 'USD', 'bps': 300, 'tier': 'ENTERPRISE'} File '/var/lib/ai-lab-e19-vm223-01/work_accumulated/settlement.py', line 26, in settle_batch AssertionError: Failed attempt #1 left residual bug in pipeline state. ... (FRAME_002 through FRAME_094: 36,000+ chars of repetitive frame dumps) ... ``` ### 3.3 What Mode B Injected After Pruning (`input-04.json`) In Mode B, as soon as Turn 1 failed, `error_trace_pruner_v1` restored the workspace, deleted `MSG[1]` (the failed partial patch) and `MSG[2]` (the 36,606-char traceback) from `messages[]`, and appended only this `927-character` **(**`+392-token`**) structured lesson** to the original prompt: ```json { "concise_lessons_learned": [ "Lesson 1 (validator.py): Partial patch failed because `settlement.py` and `statement.py` were not updated together; in `validator.py` use `ALLOWED_CURRENCIES=('USD','EUR','JPY')`, reject empty list & `bool`, allow `unit_cents>=0` & `0<=chargeback_bps<=1000`, and return a new 5-key dict.", "Lesson 2 (settlement.py): Preserve first-seen order (`for k in order:`), charge 10% reserve strictly when `chargeback_bps > 300`, apply `>= 15000` discount boundary (`GROWTH` 10%, `ENTERPRISE` 20% else 10%), and cap `rebate_applied = min(fee_rebate, after_discount)`.", "Lesson 3 (statement.py): Format item lines with `#` (`{merchant_id}#{currency}: ...`) and always prefix `-` on `TIER_DISCOUNT: -{tier_discount}` and `REBATE_APPLIED: -{rebate_applied}`." ], "pruning_engine": "error_trace_pruner_v1 (failed_code_rolled_back=true, stack_traces_pruned=true)", "remaining_issues": 4 } ``` With a clean **2,301-token** context freed from its failed first attempt, Mode B immediately generated a complete, unified 3-module patch on Turn 2. --- ## 4\. Practical Implementation Recipe: A 20-Line Rollback-and-Prune Loop If you maintain a custom coding agent loop in Python, you can implement Error-Trace Pruning in 20 lines by combining `git checkout` with a `lesson_log` list instead of appending failed turns to `messages`: ```python import subprocess def run_step_with_error_pruning(messages: list[dict], lesson_log: list[str], call_llm, run_tests) -> bool: # 1. Build prompt from clean base messages + 1-line negative lessons from prior failed attempts prompt_msgs = list(messages) if lesson_log: prompt_msgs[-1] = { **prompt_msgs[-1], "text": prompt_msgs[-1]["text"] + "\n\n[PRUNED_FAILED_ATTEMPT_LESSONS]:\n" + "\n".join(lesson_log), } model_reply = call_llm(prompt_msgs) # 2. Apply patch and run deterministic test suite passed, short_failure_reason, _raw_trace = run_tests(model_reply) if passed: messages.append({"role": "model", "text": model_reply}) return True # 3. On failure: roll back workspace and DO NOT append model_reply or _raw_trace to messages! subprocess.run(["git", "checkout", "--", "."], check=True) lesson_log.append(f"- Attempt #{len(lesson_log) + 1} rolled back: {short_failure_reason}") return False ``` --- ### πŸ”— Continue Reading the Series - πŸš€ **Master Benchmark Hub (All 41 Patterns Ranked):** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark) - πŸ›‘οΈ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox) - πŸ₯‡ **Rank #1 Deep Dive (**`E20`**):** [Stop Your Coding Agent From Stepping on the Same Rake Twice: Cold-Start Exploration vs. a 5-Item `MEMORY.md`](https://tappers.hashnode.dev/e20-episodic-gotchas-memory) - πŸ₯ˆ **Rank #2 Deep Dive (**`E21`**):** [Why Feeding Full Dependency Files to Your Coding Agent Hurts Accuracy: Full Source vs. Tree-sitter Interface Skeletons](https://tappers.hashnode.dev/e21-treesitter-skeleton-context) - πŸ₯‰ **Rank #3 Deep Dive (**`E16`**):** [Stop Dumping Raw Pytest Logs Into Your Coding Agent: How a Structured JSON Test Filter Cuts Input Tokens by 94.1%](https://tappers.hashnode.dev/e16-cli-test-output-token-killer)