# Escaping Sunk-Cost Retry Loops in Coding Agents: Single-Session Retries vs. a Parallel Best-of-N Tournament (-67.2% Input Tokens)

> 💡 **AI Labo Benchmark Series — Experiment `E13` (Parallel Worktrees & Best-of-N Selection)**
>
> This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite and throughput benchmarks.
> * 🚀 **[View All 41 Benchmark Results & Master Ranking](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)**
> * 🛡️ **[Read How Our Proxmox & systemd Anti-Cheating Sandbox Works](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)**

---

## Metadata & Experiment Identity
* **Experiment ID:** `E13` (Backlog Candidate `C009`)
* **Run ID:** `e13-20261028-vm217-01`
* **Sandbox Environment:** Proxmox Isolated VM `VM217` (`ai-lab-e13-tournament`, 0 NICs, `virtio-serial` QGA relay, UID split `labagent13` / `labjudge13`)
* **Model:** `gemini-3.8-flash`
* **Pattern Under Test:** **Sunk-Cost Single-Session Retry Loop vs. Parallel Best-of-N Branch Tournament + Deterministic Judge**

---

## 1. Executive Summary (TL;DR)

When an AI coding agent tackles a legacy codebase containing **both subtle functional bugs and an algorithmic performance bottleneck** (such as a nested `O(n^2)` loop), running sequential retries inside a single chat session triggers two compounding failure modes:

1. **Quadratic Context Bloat:** Every incremental patch and test-failure log stays in the session history, inflating prompt tokens on every retry.
2. **Structural Anchoring (Sunk-Cost Bias):** Patching inside a single workspace nudges the model toward minimal edits on top of the legacy structure rather than stepping back to redesign the data structure cleanly.

Inspired by parallel `git worktree` tournament workflows in Claude Code and Best-of-N test-time compute scaling, we compared two architectures inside isolated VM `VM217` on a 3-module **payment ledger reconciler** (`ingest.py`, `reconciler.py`, `summary.py`) plagued by 4 boundary bugs and an `O(n^2)` nested-loop bottleneck (`~205 ms` on 2,000 events):

* **Mode A (`SunkCostRetry`):** A single persistent session iteratively patching errors across turns until functional and throughput checks pass.
* **Mode B (`ParallelTournament`):** Spawns **2 independent, clean-context 1-shot branches** (`branch_1_conservative_patch` and `branch_2_clean_o_n_hashmap`) and passes both candidates to a **Deterministic Judge** (functional suite + AST nested-loop detector + `N=2,000` latency benchmark) that automatically selects the winning branch.

### Key Result
Both modes passed **`20 / 20` (100%)** on the hidden grading suite. Yet even counting **both** tournament branches combined, Mode B cut **LLM calls by `-50.0%` (`4 ➔ 2` calls)**, slashed **cumulative input tokens by `-67.2%` (`13,822 ➔ 4,537` tokens)**, lowered **API cost by `-29.2%` (`$0.0232 ➔ $0.0165`)**, and cut **serial wall-clock time by `-22.1%` (`37.28s ➔ 29.04s`, or `~14.5s` when run concurrently)**. Better still, the Deterministic Judge caught that Branch 1's conservative patch left the `O(n^2)` loop intact (`202.952 ms`, score `80/100`) and automatically promoted Branch 2's single-pass `O(n)` hash-map redesign (**`0.962 ms`—`211x` faster**, score `100/100`), beating Mode A's final 3,000-event throughput by **`-19.1%` (`1.280 ms` vs. `1.582 ms`)**.

| Metric | Mode A: Single-Session Retry (`SunkCostRetry`) | Mode B: 2-Branch Tournament (`ParallelTournament`) | Delta / Improvement |
| :--- | :---: | :---: | :---: |
| **Hidden Grading Suite Score (`labjudge13`)** | `20 / 20` (100%) | **`20 / 20` (100%)** | Identical 100% accuracy |
| **3,000-Event Reconciler Latency (`labjudge13`)** | `1.582 ms` (`O(n)`) | **`1.280 ms` (`O(n)`)** | **-19.1% faster (`158x` faster than Branch 1)** |
| **LLM Calls (`model_calls`)** | `4 calls` (Turns 1–4) | **`2 calls` (Branch 1 + Branch 2)** | **-50.0% (`4 ➔ 2` calls)** |
| **Cumulative File Edits (`file_edits`)** | `6 file edits` | **`3 file edits` (winning branch)** | **-50.0% fewer edits** |
| **Cumulative Input Tokens (`promptTokenCount`)** | `13,822 tokens` | **`4,537 tokens` (`2,242 + 2,295`)** | **-67.2% (zero retry history bloat)** |
| **Cumulative Output Tokens (`candidatesTokenCount`)** | `3,432 tokens` | `3,480 tokens` (both branches) | `+1.4%` (virtually identical) |
| **Elapsed Execution Time (`elapsed_seconds`)** | `37.28 sec` | **`29.04 sec` (`~14.5s` parallel)** | **-22.1% serial (`~2.5x` parallel)** |
| **Estimated API Cost (`estimated_micro_usd`)** | `$0.0232` (`23,238 micro-USD`) | **`$0.0165` (`16,453 micro-USD`)** | **-29.2% cheaper than 1 retry loop** |

---

## 2. Experiment Architecture: Sunk-Cost Retries vs. Parallel Tournament

```text
+-------------------------------------------------------------------------+
| Proxmox Isolated VM217 (0 NICs, virtio-serial QGA, grader labjudge13)   |
|  Target Task: 3-Module Payment Ledger Reconciler                        |
|               (ingest.py ➔ reconciler.py ➔ summary.py)                  |
|               Initial State: 4 Boundary Bugs + O(n^2) Loop (~205 ms)    |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: SunkCostRetry (Sequential Retries in One Growing Session) |  |
|  |  - Turn 1 (2,377 in): Patches ingest.py     (Score  30 | 205.0ms) |  |
|  |  - Turn 2 (3,052 in): Rewrites reconciler   (Score  85 |   1.5ms) |  |
|  |  - Turn 3 (4,010 in): Fixes summary -0/#    (Score 100 |   2.0ms) |  |
|  |  - Turn 4 (4,383 in): Final integrated pass (Score 100 |   1.0ms) |  |
|  |  --> 4 calls | 13,822 in | $0.0232 | 37.28s | 3k bench: 1.582 ms  |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: ParallelTournament (2 Clean Worktrees + Code Judge)       |  |
|  |  - Branch 1 [Conservative Patch] (2,242 in / 1,746 out):          |  |
|  |     Passes functional rules, keeps nested O(n^2) loop (202.95 ms) |  |
|  |     --> Judge Score: 80 / 100 (REJECTED)                          |  |
|  |  - Branch 2 [Clean O(n) Hash-Map] (2,295 in / 1,734 out):         |  |
|  |     Single-pass grouped dict + order list (0.962 ms — 211x faster)|  |
|  |     --> Judge Score: 100 / 100 (JUDGE_SELECT_WINNER)              |  |
|  |  --> 2 calls | 4,537 in  | $0.0165 | 29.04s | 3k bench: 1.280 ms  |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
```

---

## 3. Turn-by-Turn & Branch-by-Branch Trace (`usage.json` & `execution-traces.json`)

Inspecting all 6 LLM calls recorded in `VM217` highlights why two parallel 1-shot branches cost **less** than a single sequential 4-turn session:

| Seq | Mode | Node / Branch ID | Input Tokens | Output Tokens | Cost (`micro-USD`) | `N=2,000` Latency | AST `O(n)` Clean? | Judge Score | Status / Bottleneck |
| :---: | :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- |
| `0` | **Mode A** | `sunk_cost_retry_turn_1` | `2,377` | `556` | `3,868` | `205.048 ms` | ❌ `False` | `30 / 100` | `reconciler` bounds/conflicts unfixed, `summary` format wrong, `O(n^2)` slow |
| `1` | **Mode A** | `sunk_cost_retry_turn_2` | `3,052` | `877` | `5,578` | `1.538 ms` | ✅ `True` | `85 / 100` | `summary_format_hashtag_or_minus_zero_failed` |
| `2` | **Mode A** | `sunk_cost_retry_turn_3` | `4,010` | `301` | `4,137` | `2.019 ms` | ✅ `True` | `100 / 100` | Errors cleared; stages final submit |
| `3` | **Mode A** | `sunk_cost_retry_turn_4` | `4,383` | `1,698` | `9,655` | `0.984 ms` | ✅ `True` | `100 / 100` | `SUBMIT` (**1.84x larger input prompt than Turn 1**) |
| *Mode A* | — | *4 sequential turns* | *`13,822`* | *`3,432`* | *`23,238`* | *`0.984 ms`* | *`True`* | *`100`* | *`20 / 20` Pass (`3,000` events: `1.582 ms`)* |
| `4` | **Mode B** | `branch_1_conservative_patch` | **`2,242`** | `1,746` | `8,229` | `202.952 ms` | ❌ `False` | `80 / 100` | Fixed all functional bugs, but kept `for k in seen_keys: for ev in events:` (`O(n^2)`) |
| `5` | **Mode B** | `branch_2_clean_o_n_hashmap` | **`2,295`** | `1,734` | `8,224` | **`0.962 ms`** | ✅ **`True`** | **`100 / 100`** | **`JUDGE_SELECT_WINNER` (`211x` faster than Branch 1)** |
| *Mode B* | — | *2 independent 1-shot branches* | ***`4,537`*** | ***`3,480`*** | ***`16,453`*** | ***`0.962 ms`*** | ***`True`*** | ***`100`*** | *`20 / 20` Pass (`3,000` events: **`1.280 ms`**)* |

---

## 4. Why a Deterministic Judge Beats an LLM Judge in Best-of-N

Look closely at `seq 4` (`branch_1_conservative_patch`) vs. `seq 5` (`branch_2_clean_o_n_hashmap`):
* **Branch 1** wrote a surgical, minimal diff that fixed every single functional edge case (`chargeback_bps == 300 vs 301`, `subtotal == 14999 vs 15000`, `bool` rejection, `#` separator, and `-0` formatting). If you asked an LLM judge to pick the "cleanest, lowest-risk diff" by reading code alone, it could easily pick Branch 1—shipping a `202.95 ms` quadratic time bomb into production!
* Because Mode B's Judge was a **0-token deterministic Python harness** combining unit assertions, an AST nested-loop check, and an `N=2,000` micro-benchmark (`< 15 ms` threshold), it docked Branch 1 to `80 / 100` (`quadratic_or_slow_reconciler(nested_for=True,ms=202.952)`) and promoted Branch 2 (`0.962 ms`, `100 / 100`) with zero human intervention.

---

## 5. Practical Takeaway: Two Strikes, Then Fork Clean Worktrees

1. **Cap in-session retries at 2 turns:** Once an agent fails twice in the same chat thread, prompt tokens start compounding while failed code biases subsequent edits (see **`E19` Error-Trace Pruning** and **`E28` Loop-Detection Circuit Breaker**). Reset to clean `git worktree` branches with orthogonal strategy prompts (e.g., *Conservative Surgical Fix* vs. *Clean Data-Structure Redesign*).
2. **Score tournament branches with code, not prose:** Always pair Best-of-N worktrees with a deterministic scoring script (tests + static AST checks + latency/memory micro-benchmarks) so the fastest, cleanest architecture wins automatically.

---

### 🔗 Continue Reading the Series
* 🚀 **Master Benchmark Hub (All 41 Patterns Ranked):** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)
* 🛡️ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)
* 🛡️ **Previous Sequential Post (`E11`):** [Why Long System Prompts Can't Enforce Code Architecture: Prompt Rules vs. Deterministic Pre-Commit AST Hooks](https://tappers.hashnode.dev/e11-architecture-guardrail-hooks)
* 🔍 **Rank #5 Companion Post (`E14`):** [The Hidden "Schema Tax" of MCP Tools: Dumping 32 Tool Schemas vs. On-Demand Tool Search (-92.7% Input Tokens)](https://tappers.hashnode.dev/e14-progressive-tool-search)
* ✂️ **Next Sequential Post (`E15`):** [Why LLMs Fail at Unified Diffs: Standard Diff Hunks vs. Line-Number-Free SEARCH/REPLACE Blocks (2.41x Faster)](https://tappers.hashnode.dev/e15-search-replace-blocks)