Skip to main content
Hashnode
Tappers Tech Blog

Escaping Sunk-Cost Retry Loops in Coding Agents: Single-Session Retries vs. a Parallel Best-of-N Tournament (-67.2% Input Tokens)

Updated 8 min readMarkdown
T
tappers

💡 AI Labo Benchmark Series — Experiment E13 (Parallel Worktrees & Best-of-N Selection)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite and throughput benchmarks.

Metadata & Experiment Identity

  • Experiment ID: E13 (Backlog Candidate C009)
  • Run ID: e13-20261028-vm217-01
  • Sandbox Environment: Proxmox Isolated VM VM217 (ai-lab-e13-tournament, 0 NICs, virtio-serial QGA relay, UID split labagent13 / labjudge13)
  • Model: gemini-3.8-flash
  • Pattern Under Test: Sunk-Cost Single-Session Retry Loop vs. Parallel Best-of-N Branch Tournament + Deterministic Judge

1. Executive Summary (TL;DR)

When an AI coding agent tackles a legacy codebase containing both subtle functional bugs and an algorithmic performance bottleneck (such as a nested O(n^2) loop), running sequential retries inside a single chat session triggers two compounding failure modes:

  1. Quadratic Context Bloat: Every incremental patch and test-failure log stays in the session history, inflating prompt tokens on every retry.
  2. Structural Anchoring (Sunk-Cost Bias): Patching inside a single workspace nudges the model toward minimal edits on top of the legacy structure rather than stepping back to redesign the data structure cleanly.

Inspired by parallel git worktree tournament workflows in Claude Code and Best-of-N test-time compute scaling, we compared two architectures inside isolated VM VM217 on a 3-module payment ledger reconciler (ingest.py, reconciler.py, summary.py) plagued by 4 boundary bugs and an O(n^2) nested-loop bottleneck (~205 ms on 2,000 events):

  • Mode A (SunkCostRetry): A single persistent session iteratively patching errors across turns until functional and throughput checks pass.
  • Mode B (ParallelTournament): Spawns 2 independent, clean-context 1-shot branches (branch_1_conservative_patch and branch_2_clean_o_n_hashmap) and passes both candidates to a Deterministic Judge (functional suite + AST nested-loop detector + N=2,000 latency benchmark) that automatically selects the winning branch.

Key Result

Both modes passed 20 / 20 (100%) on the hidden grading suite. Yet even counting both tournament branches combined, Mode B cut LLM calls by -50.0% (4 ➔ 2 calls), slashed cumulative input tokens by -67.2% (13,822 ➔ 4,537 tokens), lowered API cost by -29.2% ($0.0232 ➔ $0.0165), and cut serial wall-clock time by -22.1% (37.28s ➔ 29.04s, or ~14.5s when run concurrently). Better still, the Deterministic Judge caught that Branch 1's conservative patch left the O(n^2) loop intact (202.952 ms, score 80/100) and automatically promoted Branch 2's single-pass O(n) hash-map redesign (0.962 ms—211x faster, score 100/100), beating Mode A's final 3,000-event throughput by -19.1% (1.280 ms vs. 1.582 ms).

Metric Mode A: Single-Session Retry (SunkCostRetry) Mode B: 2-Branch Tournament (ParallelTournament) Delta / Improvement
Hidden Grading Suite Score (labjudge13) 20 / 20 (100%) 20 / 20 (100%) Identical 100% accuracy
3,000-Event Reconciler Latency (labjudge13) 1.582 ms (O(n)) 1.280 ms (O(n)) -19.1% faster (158x faster than Branch 1)
LLM Calls (model_calls) 4 calls (Turns 1–4) 2 calls (Branch 1 + Branch 2) -50.0% (4 ➔ 2 calls)
Cumulative File Edits (file_edits) 6 file edits 3 file edits (winning branch) -50.0% fewer edits
Cumulative Input Tokens (promptTokenCount) 13,822 tokens 4,537 tokens (2,242 + 2,295) -67.2% (zero retry history bloat)
Cumulative Output Tokens (candidatesTokenCount) 3,432 tokens 3,480 tokens (both branches) +1.4% (virtually identical)
Elapsed Execution Time (elapsed_seconds) 37.28 sec 29.04 sec (~14.5s parallel) -22.1% serial (~2.5x parallel)
Estimated API Cost (estimated_micro_usd) $0.0232 (23,238 micro-USD) $0.0165 (16,453 micro-USD) -29.2% cheaper than 1 retry loop

2. Experiment Architecture: Sunk-Cost Retries vs. Parallel Tournament

+-------------------------------------------------------------------------+
| Proxmox Isolated VM217 (0 NICs, virtio-serial QGA, grader labjudge13)   |
|  Target Task: 3-Module Payment Ledger Reconciler                        |
|               (ingest.py ➔ reconciler.py ➔ summary.py)                  |
|               Initial State: 4 Boundary Bugs + O(n^2) Loop (~205 ms)    |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: SunkCostRetry (Sequential Retries in One Growing Session) |  |
|  |  - Turn 1 (2,377 in): Patches ingest.py     (Score  30 | 205.0ms) |  |
|  |  - Turn 2 (3,052 in): Rewrites reconciler   (Score  85 |   1.5ms) |  |
|  |  - Turn 3 (4,010 in): Fixes summary -0/#    (Score 100 |   2.0ms) |  |
|  |  - Turn 4 (4,383 in): Final integrated pass (Score 100 |   1.0ms) |  |
|  |  --> 4 calls | 13,822 in | $0.0232 | 37.28s | 3k bench: 1.582 ms  |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: ParallelTournament (2 Clean Worktrees + Code Judge)       |  |
|  |  - Branch 1 [Conservative Patch] (2,242 in / 1,746 out):          |  |
|  |     Passes functional rules, keeps nested O(n^2) loop (202.95 ms) |  |
|  |     --> Judge Score: 80 / 100 (REJECTED)                          |  |
|  |  - Branch 2 [Clean O(n) Hash-Map] (2,295 in / 1,734 out):         |  |
|  |     Single-pass grouped dict + order list (0.962 ms — 211x faster)|  |
|  |     --> Judge Score: 100 / 100 (JUDGE_SELECT_WINNER)              |  |
|  |  --> 2 calls | 4,537 in  | $0.0165 | 29.04s | 3k bench: 1.280 ms  |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

3. Turn-by-Turn & Branch-by-Branch Trace (usage.json & execution-traces.json)

Inspecting all 6 LLM calls recorded in VM217 highlights why two parallel 1-shot branches cost less than a single sequential 4-turn session:

Seq Mode Node / Branch ID Input Tokens Output Tokens Cost (micro-USD) N=2,000 Latency AST O(n) Clean? Judge Score Status / Bottleneck
0 Mode A sunk_cost_retry_turn_1 2,377 556 3,868 205.048 ms ❌ False 30 / 100 reconciler bounds/conflicts unfixed, summary format wrong, O(n^2) slow
1 Mode A sunk_cost_retry_turn_2 3,052 877 5,578 1.538 ms ✅ True 85 / 100 summary_format_hashtag_or_minus_zero_failed
2 Mode A sunk_cost_retry_turn_3 4,010 301 4,137 2.019 ms ✅ True 100 / 100 Errors cleared; stages final submit
3 Mode A sunk_cost_retry_turn_4 4,383 1,698 9,655 0.984 ms ✅ True 100 / 100 SUBMIT (1.84x larger input prompt than Turn 1)
Mode A — 4 sequential turns 13,822 3,432 23,238 0.984 ms True 100 20 / 20 Pass (3,000 events: 1.582 ms)
4 Mode B branch_1_conservative_patch 2,242 1,746 8,229 202.952 ms ❌ False 80 / 100 Fixed all functional bugs, but kept for k in seen_keys: for ev in events: (O(n^2))
5 Mode B branch_2_clean_o_n_hashmap 2,295 1,734 8,224 0.962 ms ✅ True 100 / 100 JUDGE_SELECT_WINNER (211x faster than Branch 1)
Mode B — 2 independent 1-shot branches 4,537 3,480 16,453 0.962 ms True 100 *20 / 20 Pass (3,000 events: 1.280 ms)*

4. Why a Deterministic Judge Beats an LLM Judge in Best-of-N

Look closely at seq 4 (branch_1_conservative_patch) vs. seq 5 (branch_2_clean_o_n_hashmap):

  • Branch 1 wrote a surgical, minimal diff that fixed every single functional edge case (chargeback_bps == 300 vs 301, subtotal == 14999 vs 15000, bool rejection, # separator, and -0 formatting). If you asked an LLM judge to pick the "cleanest, lowest-risk diff" by reading code alone, it could easily pick Branch 1—shipping a 202.95 ms quadratic time bomb into production!
  • Because Mode B's Judge was a 0-token deterministic Python harness combining unit assertions, an AST nested-loop check, and an N=2,000 micro-benchmark (< 15 ms threshold), it docked Branch 1 to 80 / 100 (quadratic_or_slow_reconciler(nested_for=True,ms=202.952)) and promoted Branch 2 (0.962 ms, 100 / 100) with zero human intervention.

5. Practical Takeaway: Two Strikes, Then Fork Clean Worktrees

  1. Cap in-session retries at 2 turns: Once an agent fails twice in the same chat thread, prompt tokens start compounding while failed code biases subsequent edits (see E19 Error-Trace Pruning and E28 Loop-Detection Circuit Breaker). Reset to clean git worktree branches with orthogonal strategy prompts (e.g., Conservative Surgical Fix vs. Clean Data-Structure Redesign).
  2. Score tournament branches with code, not prose: Always pair Best-of-N worktrees with a deterministic scoring script (tests + static AST checks + latency/memory micro-benchmarks) so the fastest, cleanest architecture wins automatically.

🔗 Continue Reading the Series

Comments

Join the discussion

No comments yet. Be the first to comment.

More from this blog