Stopping Blind Micro-Retries on Identical Errors: Death-Spiral Thrashing vs. a 3-Hypothesis Pivot Ladder (-95.2% Input Tokens)

💡 AI Labo Benchmark Series — Experiment E37 (Loop Recovery & Stuck Detection)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

• 🚀 View All 41 Benchmark Results & Master Ranking
• 🛡️ Read How Our Proxmox & systemd Anti-Cheating Sandbox Works

---

Metadata & Experiment Identity

• Experiment ID: E37 (Backlog Candidate C033)
• Run ID: e37-20261029-vm241-01
• Sandbox Environment: Proxmox Isolated VM VM241 (ai-e37-241, 0 NICs, PrivateNetwork=yes, NoNewPrivileges=yes, InaccessiblePaths, UID split labexp37 / labjudge37)
• Model: gemini-3.8-flash (thinkingLevel: LOW, temperature=0.0)
• Pattern Under Test: "LangChain: Improving Deep Agents with Harness Engineering — Stuck Detection & 3-Hypothesis Stop-Loss Pivot Ladder" — Blind Retry on Identical Errors (BlindIdenticalErrorRetryRunner) vs. Hypothesis-Pivot Ladder (HypothesisPivotLadderRunner)

---

1. Executive Summary (TL;DR)

When an AI coding agent hits an error and has no stuck-detection guardrail, its default behavior is Blind Micro-Retry Thrashing: it tweaks a single line around the same failed hypothesis, re-runs the test suite, gets the exact same error signature back, and spirals into context bloat.

A Hypothesis-Pivot Ladder breaks this death spiral: when an error signature repeats, the harness prohibits micro-tweaking the same hypothesis, restores a clean baseline, and forces the agent to climb a structured 3-Hypothesis Pivot Ladder across orthogonal architectural layers (L1_INPUT_CONTRACT ➔ L2_AGGREGATION_MATH ➔ L3_RENDER_FORMAT).

We tested this head-to-head inside isolated VM VM241 on the 3-module enterprise payment settlement pipeline (validator.py ➔ settlement.py ➔ statement.py):

• Mode A (BlindIdenticalErrorRetryRunner): Repeats local micro-edits on identical error signatures (73,062 chars of thrashing logs across Turns 1–2) and submits {**item, ...} in validator.py—leaking extra dictionary keys and stalling at 19 / 20 (95.0%).
• Mode B (HypothesisPivotLadderRunner): Enforces the 3-Hypothesis Structural Pivot Gate (HYPOTHESIS_PIVOT_LADDER_MD + execute_pivot_ladder_and_submit), resolving all 3 module contracts in 1 shot (20 / 20, 100%).

Key Result

Mode A failed validator_01_normalize_and_non_mutation (19 / 20) after 3 turns (46,393 cumulative input tokens). Mode B achieved a 20 / 20 (100%) pass in 1 turn, cutting LLM calls by -66.7% (3 ➔ 1), cumulative input tokens by -95.2% (46,393 ➔ 2,238 tokens—a 20.73x reduction), API cost by -83.2% ($0.0455 ➔ $0.0077), and running 2.27x faster (29.23s ➔ 12.87s, -55.9%).

Metric	Mode A: Blind Identical Retry (BlindIdenticalErrorRetry)	Mode B: 3-Hypothesis Pivot Ladder (HypothesisPivotLadder)	Delta / Improvement
Hidden Grading Suite Score (labjudge37)	19 / 20 (95.0% — 1 failure)	20 / 20 (100% — Perfect)	+5.0% accuracy (19/20 ➔ 20/20)
Identical-Error Thrash Traps (gotcha_traps_hit)	2 traps (Turns 1 & 2)	0 traps	-100% (zero thrashing)
LLM Calls (model_calls)	3 calls	1 call (1-shot)	-66.7% (3.00x faster convergence)
Cumulative File Edits (file_edits)	5 edits	3 edits (1 per module)	-40.0% fewer edits
Cumulative Tool Output (tool_output_chars)	73,062 chars	98 chars	-99.9% (745.5x noise reduction)
Turn 1 Tokens (In / Out)	1,661 in / 12 out	2,238 in / 1,595 out	+577 in for pivot ladder spec
Turn 2 Tokens (In / Out)	15,091 in / 1,250 out	— (completed on Turn 1)	Avoided 36K-char thrashing dump
Turn 3 Tokens (In / Out)	29,641 in / 1,592 out	— (completed on Turn 1)	Avoided secondary rework bloat
Cumulative Input Tokens (promptTokenCount)	46,393 tokens	2,238 tokens	-95.2% (20.73x efficiency)
Cumulative Output Tokens (candidatesTokenCount)	2,854 tokens	1,595 tokens	-44.1% output reduction
Elapsed Execution Time (elapsed_seconds)	29.23 sec	12.87 sec	-55.9% (2.27x faster)
Estimated API Cost (estimated_micro_usd)	$0.0455 (45,498 micro-USD)	$0.0077 (7,660 micro-USD)	-83.2% (5.94x cheaper)

---

2. Experiment Architecture: Blind Retry Loop vs. Hypothesis-Pivot Ladder

+-------------------------------------------------------------------------+
| Proxmox Isolated VM241 (0 NICs, InaccessiblePaths, grader labjudge37)   |
|  Target Task: 3-Module Enterprise Payment Settlement Pipeline           |
|               validator.py ➔ settlement.py ➔ statement.py               |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: BlindIdenticalErrorRetryRunner (Same-Error Micro-Tweaks)  |  |
|  |  - Turn 1 (1,661 in):  Calls `retry_identical_error_patch`        |  |
|  |                        ➔ 36,547 chars of repeated error logs      |  |
|  |  - Turn 2 (15,091 in): Calls `stage_thrashing_tweak` on 2 files   |  |
|  |                        ➔ 36,201 chars of thrashing output         |  |
|  |  - Turn 3 (29,641 in): Submits `{**item, ...}` in `validator.py`  |  |
|  |                        (retains `"extra"` key ➔ fails test #1)    |  |
|  |  --> 3 calls | 46,393 in | $0.0455 | 29.23s | 19/20 (1 FAILED)    |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: HypothesisPivotLadderRunner (3-Hypothesis Pivot Gate)     |  |
|  |  - Injects `HYPOTHESIS_PIVOT_LADDER_MD` (+577 tokens):            |  |
|  |    * `L1_INPUT_CONTRACT` (`validator.py`): fresh 5-key dict, etc. |  |
|  |    * `L2_AGGREGATION_MATH` (`settlement.py`): order, bps>300      |  |
|  |    * `L3_RENDER_FORMAT` (`statement.py`): `#` & `-0` formatting   |  |
|  |  - Turn 1 (2,238 in): Calls `execute_pivot_ladder_and_submit`     |  |
|  |  --> 1 call  | 2,238 in  | $0.0077 | 12.87s | 20/20 (100% PASS)   |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

---

3. Drop-In Implementation Recipe: HypothesisPivotLadder

When an error signature repeats across two consecutive attempts, block local micro-tweaks and inject an orthogonal 3-level hypothesis pivot directive:

class HypothesisPivotLadder:
    """Blocks blind micro-retries on repeated error signatures and forces orthogonal hypothesis pivots."""

    PIVOT_LADDER = [
        (
            "L1_INPUT_CONTRACT",
            "validator.py",
            "Verify strict int (reject bool), unit_cents >= 0, and construct a fresh 5-key dict (strip extra keys).",
        ),
        (
            "L2_AGGREGATION_MATH",
            "settlement.py",
            "Verify first-seen order (remove sorted), bps > 300 reserve, and subtotal >= 15000 discount.",
        ),
        (
            "L3_RENDER_FORMAT",
            "statement.py",
            "Verify `#` merchant-currency delimiter and unconditional `-` sign on `-0` discount/rebate.",
        ),
    ]

    def build_pivot_directive(self, failed_error_keys: list[str]) -> str:
        lines = ["# HYPOTHESIS-PIVOT LADDER DIRECTIVE (Blind Micro-Retry Prohibited):"]
        for level_id, mod_name, check_desc in self.PIVOT_LADDER:
            lines.append(f"- [{level_id} -> `{mod_name}`]: {check_desc}")
        lines.append(f"Active unresolved error keys: {failed_error_keys}")
        return "\n".join(lines)

---

4. Practical Takeaway: Never Allow Two Consecutive Retries With the Same Hypothesis

1. Hash the active error signature after every test run: If error_signature(turn_t) == error_signature(turn_{t-1}), freeze the current approach immediately.
2. Pair the Hypothesis-Pivot Ladder (E37) with the Loop-Detection Circuit Breaker (E28): While E28 catches A➔B➔A file-state oscillations, E37 catches identical error signatures and forces the agent to pivot across architectural layers.

---

🔗 Continue Reading the Series

• 🚀 Master Benchmark Hub (All 41 Patterns Ranked): I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy
• 🛡️ Methodology & Sandbox Architecture: How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox
• 🤝 Previous Sequential Post (E36): Preventing Multi-Session Amnesia in Long-Running Coding Agents: Unstructured Prose Summaries vs. a 3-Artifact Shift-Handover Harness (-95.2% Input Tokens)
• 🔐 Next Sequential Post (E38): Defeating Reward Hacking in AI Coding Agents: Prompt-Only Test Protection vs. a Cryptographic SHA-256 Test-Tampering Lock (2.46x Faster)
• 🔌 Related Circuit Breaker Deep Dive (E28): Breaking A➔B➔A Edit Oscillations in Coding Agents: Unguarded Retry Loops vs. a Loop-Detection Circuit Breaker (-94.9% Input Tokens)