# Stop Your Coding Agent From Cycling Through Refuted Guesses: Free-Text Monologue vs. a Structured Hypothesis Matrix (2.54x Faster)

> 💡 **AI Labo Benchmark Series — Experiment `E27` (Structured Reasoning & Diagnostic Scratchpads)**
>
> This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.
> * 🚀 **[View All 41 Benchmark Results & Master Ranking](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)**
> * 🛡️ **[Read How Our Proxmox & systemd Anti-Cheating Sandbox Works](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)**

---

## Metadata & Experiment Identity
* **Experiment ID:** `E27` (Backlog Candidate `C023`)
* **Run ID:** `e27-20261029-vm231-01`
* **Sandbox Environment:** Proxmox Isolated VM `VM231` (`ai-e27-231`, 0 NICs, `PrivateNetwork=yes`, `NoNewPrivileges=yes`, `InaccessiblePaths`, UID split `labexp27` / `labjudge27`)
* **Model:** `gemini-3.8-flash` (`thinkingLevel: LOW`, `temperature=0.0`)
* **Pattern Under Test:** *"Anthropic `<think>` Tool Research & SWE-agent Structured Diagnostic Scratchpad (Hypothesis Elimination Matrix)"* — **Unstructured Free-Text Monologue (`FreeTextMonologueRunner`) vs. Structured `THINK.md` Hypothesis Elimination Table (`StructuredHypothesisTableRunner`)**

---

## 1. Executive Summary (TL;DR)

When debugging a multi-module pipeline with 7+ interacting defects, letting an AI coding agent reason via unstructured **Free-Text Monologue** ("Let me think... maybe the settlement total is off because of integer division? Or wait, did we check that earlier?") leads to **Hypothesis Cycling**: as tool logs grow across turns, the model loses track of which candidate root causes were already falsified and which boundary contracts remain unverified.

Inspired by Anthropic's `<think>` scratchpad research and SWE-agent diagnostic matrices, we tested replacing free-text debugging monologues with a mandatory **Structured Hypothesis Elimination Table (`THINK.md`)** inside isolated VM `VM231` on the 3-module enterprise payment settlement pipeline (`validator.py` ➔ `settlement.py` ➔ `statement.py`):

* **Mode A (`FreeTextMonologueRunner`):** Reasons through unstructured free-text monologue logs (`92,463` tool output chars across Turns 1–2) before submitting on Turn 3.
* **Mode B (`StructuredHypothesisTableRunner`):** Requires a fixed-schema **Hypothesis Elimination Matrix (`STRUCTURED_HYPOTHESIS_TABLE_MD`)** mapping `Hyp ID ➔ Target Module ➔ Suspected Defect ➔ Diagnostic Probe ➔ Status (CONFIRMED/REFUTED) ➔ Canonical Fix` before any code edit.

### Key Result
Mode A missed `validator_01` (`19 / 20`, 95.0%) after 3 turns (`48,712` input tokens) because it over-constrained `set(item.keys()) != required_keys` instead of stripping extra keys into a clean 5-key dictionary. **Mode B achieved a `20 / 20` (100%) pass in 1 turn**, cutting **LLM calls by `-66.7%` (`3 ➔ 1`)**, **cumulative input tokens by `-95.0%` (`48,712 ➔ 2,450` tokens—a `19.88x` reduction)**, **API cost by `-83.6%` (`$0.0478 ➔ $0.0079`)**, and running **`2.54x` faster (`33.66s ➔ 13.28s`, `-60.6%`)**.

| Metric | Mode A: Free-Text Monologue (`FreeTextMonologue`) | Mode B: Hypothesis Matrix (`StructuredHypothesisTable`) | Delta / Improvement |
| :--- | :---: | :---: | :---: |
| **Hidden Grading Suite Score (`labjudge27`)** | `19 / 20` (95.0% — 1 failure) | **`20 / 20` (100% — Perfect)** | **+5.0% accuracy (`19/20 ➔ 20/20`)** |
| **Hypothesis Cycling Traps (`gotcha_traps_hit`)** | `2 traps` (Turns 1 & 2) | **`0 traps`** | **-100% (zero cycling)** |
| **LLM Calls (`model_calls`)** | `3 calls` | **`1 call` (1-shot)** | **-66.7% (`3 ➔ 1` turn)** |
| **Cumulative File Edits (`file_edits`)** | `5 edits` | **`3 edits` (1 per module)** | **-40.0% fewer edits** |
| **Cumulative Tool Output (`tool_output_chars`)** | `92,463 chars` | **`102 chars`** | **-99.9% (`906.5x` noise reduction)** |
| **Turn 1 Tokens (In / Out)** | `1,655 in` / `18 out` | **`2,450 in` / `1,604 out`** | `+795` in for `H-01`..`H-07` Matrix |
| **Turn 2 Tokens (In / Out)** | `15,832 in` / `1,333 out` | *— (completed on Turn 1)* | Avoided monologue rambling bloat |
| **Turn 3 Tokens (In / Out)** | `31,225 in` / `1,641 out` | *— (completed on Turn 1)* | Avoided secondary cycling bloat |
| **Cumulative Input Tokens (`promptTokenCount`)** | `48,712 tokens` | **`2,450 tokens`** | **-95.0% (`19.88x` efficiency)** |
| **Cumulative Output Tokens (`candidatesTokenCount`)** | `2,992 tokens` | **`1,604 tokens`** | **-46.4% output reduction** |
| **Elapsed Execution Time (`elapsed_seconds`)** | `33.66 sec` | **`13.28 sec`** | **-60.6% (`2.54x` faster)** |
| **Estimated API Cost (`estimated_micro_usd`)** | `$0.0478` (`47,755 micro-USD`) | **`$0.0079` (`7,853 micro-USD`)** | **-83.6% (`6.08x` cheaper)** |

---

## 2. The Verified `THINK.md` Hypothesis Elimination Matrix (`input-03.json`)

Instead of rambling in prose, Mode B locked this 7-row structured diagnostic matrix (`+795` tokens) before touching a single line of code:

| Hyp ID | Target Module | Suspected Defect / Boundary | Diagnostic Probe & Elimination Result | Status | Required Canonical Fix |
| :--- | :--- | :--- | :--- | :---: | :--- |
| **`H-01`** | `validator.py` | `ALLOWED_CURRENCIES` missing `"JPY"` & empty list `[]` allowed | `validate_txns([])` did not raise; `"jpy"` rejected | **CONFIRMED** | Set `ALLOWED_CURRENCIES = ("USD", "EUR", "JPY")` and reject `len(raw_txns) == 0` |
| **`H-02`** | `validator.py` | `isinstance(True, int)` passes in Python & `cents <= 0` rejects `$0` auth | `tx_count=True` accepted; `unit_cents=0` raised `ValueError` | **CONFIRMED** | Check `isinstance(x, bool)` before `isinstance(x, int)`; enforce `cnt > 0`, `cents >= 0`, `0 <= bps <= 1000` |
| **`H-03`** | `validator.py` | In-place mutation `item["merchant_id"] = ...` & extra keys retained | Caller dict mutated; `"extra": 1` leaked into output | **CONFIRMED** | Construct a fresh 5-key dict (`merchant_id`, `currency`, `tx_count`, `unit_cents`, `chargeback_bps`) |
| **`H-04`** | `settlement.py` | `sorted(order)` destroys first-seen merchant-currency order | `M-Z` before `M-A` reordered to `M-A`, `M-Z` | **CONFIRMED** | Iterate `for k in order:` (never `sorted(order)`) |
| **`H-05`** | `settlement.py` | Chargeback reserve `>= 300` vs strictly `> 300` bps | `300 bps` charged 10% reserve instead of `0` | **CONFIRMED** | Charge `(gross * 10) // 100` strictly when `g["chargeback_bps"] > 300` |
| **`H-06`** | `settlement.py` | Tier discount `> 15000` vs inclusive `>= 15000` & rebate cap | `subtotal == 15000` missed discount; `rebate > after_discount` went negative | **CONFIRMED** | Use `subtotal >= 15000` (`GROWTH` 10%/0%; `ENTERPRISE` 20%/10%) and `min(fee_rebate, after_discount)` |
| **`H-07`** | `statement.py` | `:` separator and unsigned `"0"` on zero discount/rebate | Output had `M-1:USD` and `TIER_DISCOUNT: 0` | **CONFIRMED** | Format `{merchant_id}#{currency}` and always prefix `-`: `TIER_DISCOUNT: -{batch['tier_discount']}` |

---

## 3. Why Free-Text Monologue Missed `validator_01` (`19/20`)

In Mode A (`freetext-monologue-bundle.json`), without `H-03` explicitly distinguishing *"rejecting missing required keys"* from *"stripping unapproved extra keys into a fresh 5-key dict"*, Mode A wrote:

```python
# ❌ Mode A (FreeTextMonologueRunner — Turn 3 Final validator.py, 19/20)
        required_keys = {"merchant_id", "currency", "tx_count", "unit_cents", "chargeback_bps"}
        if set(item.keys()) != required_keys:  # <-- BUG: Raises ValueError when caller passes {"extra": 1} instead of stripping it!
            raise ValueError("invalid keys")
```

Mode B, guided by row `H-03` (`"extra": 1 leaked into output ➔ Construct a fresh 5-key dict`), extracted the 5 required fields and dropped extra keys cleanly without raising a false-positive `ValueError`.

---

## 4. Drop-In Implementation Recipe: `HypothesisTableGate`

```python
from dataclasses import dataclass

@dataclass
class HypothesisRow:
    hyp_id: str
    target_module: str
    suspected_defect: str
    probe_result: str
    status: str  # "CONFIRMED" or "REFUTED"
    canonical_fix: str

class HypothesisTableGate:
    """Blocks code edits until all target modules have confirmed/refuted rows in THINK.md."""
    def __init__(self, required_modules: set[str]):
        self.required_modules = required_modules
        self.confirmed_rows: list[HypothesisRow] = []

    def register_table(self, rows: list[HypothesisRow]) -> str:
        covered = {r.target_module for r in rows if r.status in {"CONFIRMED", "REFUTED"}}
        missing = self.required_modules - covered
        if missing:
            raise ValueError(f"THINK.md table incomplete; missing modules: {sorted(missing)}")
        self.confirmed_rows = [r for r in rows if r.status == "CONFIRMED"]
        return f"THINK.md locked with {len(self.confirmed_rows)} confirmed root causes."
```

---

## 5. Practical Takeaway: Force Tabular State for Multi-Bug Debugging

1. **Replace unstructured CoT monologues with a fixed-column Markdown table (`THINK.md`) on multi-bug tasks:** A 6-column table acts as an external working-memory register where every confirmed bug maps 1-to-1 to a required code change.
2. **Keep `REFUTED` rows visible:** Recording *why* a hypothesis was ruled out (`status: REFUTED`) prevents the agent from re-trying the same failed guess three turns later.

---

### 🔗 Continue Reading the Series
* 🚀 **Master Benchmark Hub (All 41 Patterns Ranked):** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)
* 🛡️ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)
* 🔒 **Previous Sequential Post (`E26`):** [Why Claude Code's Plan Mode Works: Interleaved Read-Write Loops vs. a Two-Phase Read-Only Plan Lock (-94.8% Input Tokens)](https://tappers.hashnode.dev/e26-two-phase-readonly-plan-lock)
* 🔌 **Next Sequential Post (`E28`):** [Breaking A➔B➔A Edit Oscillations in Coding Agents: Unguarded Retry Loops vs. a Loop-Detection Circuit Breaker (-94.9% Input Tokens)](https://tappers.hashnode.dev/e28-loop-detection-circuit-breaker)
* 🌿 **Related Context Hygiene Deep Dive (`E19` — Rank #4):** [Why Keeping Failed Code Attempts in Chat History Poisons Your Coding Agent: Error-Trace Pruning vs. Accumulated History](https://tappers.hashnode.dev/e19-error-trace-pruning)