# Eliminating Goal Drift in Multi-File Coding Agents: In-Chat Loops vs. Code-Mode Dynamic Workflows

> 💡 **AI Labo Benchmark Series — Experiment `E07` (Dynamic Code-Mode Orchestration)**
>
> This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.
> * 🚀 **[View All 41 Benchmark Results & Master Ranking](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)**
> * 🛡️ **[Read How Our Proxmox & systemd Anti-Cheating Sandbox Works](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)**

---

## Metadata & Experiment Identity
* **Experiment ID:** `E07` (Backlog Candidate `C003b`)
* **Run ID:** `e07-20261028-vm211-01`
* **Sandbox Environment:** Proxmox Isolated VM `VM211` (`ai-e07-211`, 0 NICs, `virtio-serial` QGA relay, `systemd` UID separation)
* **Model:** `gemini-3.8-flash` (`thinkingLevel: LOW`)
* **Pattern Under Test:** Anthropic Claude Code Dynamic Workflows (*"Stop Babysitting Your Agents / Code-Mode Orchestration"*, Sid Bidasaria & Thariq Shihipar) — **In-Chat Iterative Loop vs. Dynamic Python Workflow Generation**

---

## 1. Executive Summary (TL;DR)

When an AI coding agent audits and refactors multiple files turn-by-turn inside a single growing chat thread (**In-Chat Iterative Loop**), two failure modes reliably appear in later turns:
1. **Quadratic Prompt Bloat:** Every file patch emitted on Turn 1, Turn 2, and Turn 3 stays in chat history and gets re-billed on every subsequent turn.
2. **Goal Drift & Late-Turn Laziness:** By Turn 4, accumulated code diffs crowd out fine-grained specification edge cases from Turn 1, causing the agent to skip boundary checks and write a superficial verification check just to finish.

Anthropic's **Code-Mode Dynamic Workflows** proposes flipping the control model: instead of iterating step-by-step in chat, have the agent first synthesize a **deterministic Python audit & orchestration script (`workflow.py`)** encoding the full contract, run it in Python at zero token cost, and pass the structured audit output to a fresh worker context to fix all modules in a single batch.

We tested both architectures head-to-head on a 3-module financial settlement & risk-scoring pipeline (`validator.py`, `settlement.py`, `manifest.py`, plus `workflow.py`) inside isolated Proxmox VM `VM211`:

* **Mode A (`ChatLoop`):** Patches `validator.py` (Turn 1), `settlement.py` (Turn 2), `manifest.py` (Turn 3), and `workflow.py` (Turn 4) sequentially inside one persistent chat thread.
* **Mode B (`CodeModeWorkflow`):**
  1. **`role_1_workflow_architect` (Turn 1):** Writes a comprehensive 39-line programmatic audit script (`workflow.py`) directly from `TASK.md`, which the Python runtime executes immediately to catch initial bugs (`audit_caught_initial_bugs: true`).
  2. **`role_2_code_mode_fixer` (Turn 2, fresh context):** Receives the locked `workflow.py` and fixes all 3 modules in one pass until `workflow.py` passes 100%.

### Key Result
Mode A suffered late-turn **Goal Drift**, dropping two validation boundaries (`18 / 20` on the hidden suite) and writing a lazy 1-case smoke check on Turn 4 (**`0 / 6` mutant kill rate**). Mode B achieved **`20 / 20` (100%) accuracy** and a **`6 / 6` (100%) mutant kill rate**, while cutting **LLM calls by `-50.0%` (`4 ➔ 2`)**, **cumulative input tokens by `-48.5%` (`12,509 ➔ 6,445`)**, **wall-clock time by `-9.5%`**, and **API cost by `-2.1%`**.

| Metric | Mode A: In-Chat Iterative Loop (`ChatLoop`) | Mode B: Code-Mode Dynamic Workflow (`CodeModeWorkflow`) | Delta / Improvement |
| :--- | :---: | :---: | :---: |
| **Hidden Grading Suite Score (20 Cases)** | `18 / 20` (90.0% — **❌ 2 boundary bugs**) | **`20 / 20` (100% Perfect Pass)** | **+10.0 pt (Zero Goal Drift)** |
| **Workflow Mutant Kill Rate (6 Mutants)** | **`0 / 6` (0.0%)** | **`6 / 6` (100.0%)** | **+100.0 pt audit coverage** |
| **Initial Bugs Caught Deterministically** | No (untested until Turn 4) | **Yes (`audit_caught_initial_bugs: true`)** | 0-token Python audit gate |
| **LLM Calls (`model_calls`)** | `4 calls` | **`2 calls`** | **-50.0% fewer calls** |
| **Cumulative Input Tokens (`promptTokenCount`)** | `12,509 tokens` | **`6,445 tokens`** | **-48.5% prompt reduction** |
| **Cumulative Output Tokens (`candidatesTokenCount`)** | `1,880 tokens` | **`3,003 tokens`** | +59.7% (invested in rich audit code) |
| **Elapsed Execution Time (`elapsed_seconds`)** | `26.42 sec` | **`23.92 sec`** | **-9.5% faster** |
| **Estimated API Cost (`estimated_micro_usd`)** | `$0.0164` (`16,433 micro-USD`) | **`$0.0161` (`16,096 micro-USD`)** | **-2.1% cheaper with 100% quality** |

---

## 2. Architecture & Turn-by-Turn Token Progression

```text
+-------------------------------------------------------------------------+
| Proxmox Isolated VM211 (0 NICs, virtio-serial QGA, systemd UID split)   |
|  Target Task: 3-Module Financial Risk & Settlement Pipeline             |
|               (validator.py ➔ settlement.py ➔ manifest.py)              |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: ChatLoop (Sequential File-by-File Chat Iteration)         |  |
|  |  - Turn 1: patch validator.py   (2,248 in / 425 out)              |  |
|  |  - Turn 2: patch settlement.py  (2,720 in / 814 out)              |  |
|  |  - Turn 3: patch manifest.py    (3,591 in / 303 out)              |  |
|  |  - Turn 4: write workflow.py    (3,950 in / 338 out — lazy check!)|  |
|  |  --> 12,509 in | 18/20 Hidden Cases | 0/6 Mutants Killed          |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: CodeModeWorkflow (Dynamic Script Generation + Fresh Fixer)|  |
|  |  - Turn 1 [Workflow Architect]: Generates 39-line `workflow.py`   |  |
|  |            encoding all boundary & anomaly invariants (2,067 in)  |  |
|  |  - Python Runtime [0 tokens]:   Runs `workflow.py` & locks file   |  |
|  |  - Turn 2 [Code-Mode Fixer]:    Fresh context fixes all 3 modules |  |
|  |            against locked `workflow.py` in 1 batch (4,378 in)     |  |
|  |  --> 6,445 in  | 20/20 Hidden Cases | 6/6 Mutants Killed          |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
```

---

## 3. Code-Level Postmortem: Late-Turn Laziness vs. Programmatic Rigor

### 3.1 Mode A (`ChatLoop`): Goal Drift & Turn-4 Laziness
By Turn 4 (`3,950` input tokens of accumulated patches), Mode A had missed two critical edge cases in its implementation:
1. **`validator_02_empty_and_non_list_rejected`:** Checked `if not isinstance(raw_batches, list):` but forgot to reject empty lists `[]` with `ValueError`.
2. **`settlement_08_invalid_tier_or_bool_reserve_rejected`:** Silently defaulted unknown tiers like `"GOLD"` to `rebate = 0` and omitted negative/boolean validation on `reserve_cents`.

And when asked on Turn 4 to write `workflow.py` to verify the pipeline, the fatigued chat context produced a minimal happy-path script that tested zero negative or boundary conditions:

```python
# ❌ Mode A (`ChatLoop`, Turn 4 `workflow.py`): Superficial happy-path check (0/6 mutants killed)
def run_workflow():
    raw = [
        {"tenant_id": "  t-1 ", "region": "us", "tx_count": 10, "amount_cents": 100000, "risk_score": 75},
        {"tenant_id": "T-1", "region": "US", "tx_count": 5, "amount_cents": 50000, "risk_score": 80},
        {"tenant_id": "T-2", "region": "EU", "tx_count": 2, "amount_cents": 20000, "risk_score": 10},
    ]
    batches = validator.validate_batches(raw)
    st = settlement.compute_settlement(batches, tier="ENTERPRISE", reserve_cents=500)
    txt = manifest.render_manifest(st)
    assert "SETTLEMENT MANIFEST (ENTERPRISE)" in txt
    assert "REBATE: -" in txt
    assert "RESERVE: -500" in txt
    return True
```

### 3.2 Mode B (`CodeModeWorkflow`): Full Contract Encoded in Python on Turn 1
In Mode B, `role_1_workflow_architect` started with a clean `2,067-token` prompt dedicated solely to turning `TASK.md` into executable Python assertions:

```python
# ✅ Mode B (`CodeModeWorkflow`, Turn 1 `workflow.py` excerpt — 6/6 mutants killed)
def bad(fn, *a, **k):
    try: fn(*a, **k); return False
    except ValueError: return True

def run_workflow():
    orig = [{"tenant_id": " t1 ", "region": " us ", "tx_count": 5, "amount_cents": 1000, "risk_score": 70, "extra": 99}]
    orig_copy = [dict(orig[0])]
    v = validate_batches(orig)
    assert orig == orig_copy and "extra" not in v[0]
    assert bad(validate_batches, []) and bad(validate_batches, "not_list")
    for k, val in [("tenant_id", 1), ("tenant_id", " "), ("tenant_id", True), ("region", "XX"),
                   ("tx_count", 0), ("tx_count", True), ("amount_cents", -1), ("amount_cents", False),
                   ("risk_score", -1), ("risk_score", 101), ("risk_score", True)]:
        b = dict(orig_copy[0]); b[k] = val
        assert bad(validate_batches, [b])
    assert bad(compute_settlement, v, "INVALID") and bad(compute_settlement, v, "STANDARD", -1) and bad(compute_settlement, v, "STANDARD", True)
    # Exact boundary checks: risk_score 69 vs 70, anomaly |r1-r2| 50 vs 51, ENTERPRISE 99999 vs 100000, REBATE: -0
    ...
```

---

## 4. Mutation Audit (`M1`–`M6`) Summary (`labjudge07`)

| Mutant ID | Boundary Mutation Injected | Mode A (`ChatLoop`) | Mode B (`CodeModeWorkflow`) |
| :--- | :--- | :---: | :---: |
| **`M1`** | Risk surcharge `>= 70` mutated to strict `> 70` | ❌ Survived | ✅ **Killed** |
| **`M2`** | `ENTERPRISE` rebate `>= 100000` mutated to `> 100000` | ❌ Survived | ✅ **Killed** |
| **`M3`** | `isinstance(tx, bool)` check removed from `tx_count` | ❌ Survived | ✅ **Killed** |
| **`M4`** | Zero-amount batch (`amount_cents == 0`) rejected | ❌ Survived | ✅ **Killed** |
| **`M5`** | Risk anomaly check (`abs(r1 - r2) > 50`) omitted | ❌ Survived | ✅ **Killed** |
| **`M6`** | Zero rebate formatted as `REBATE: 0` instead of `REBATE: -0` | ❌ Survived | ✅ **Killed** |
| **Total** | **6 Boundary Mutants** | **`0 / 6` (0.0%)** | **`6 / 6` (100.0%)** |

---

## 5. Practical Takeaway: Offload Multi-Module Audits From Chat to Code

Whenever an agent task touches **3+ files or repeated cross-file checks**:
1. **Have the agent write a Python audit/orchestration script on Turn 1** instead of stepping through files manually inside chat history.
2. **Pass the script's structured output—not the prior chat log—to a fresh fixer context**, cutting input tokens nearly in half while eliminating late-turn Goal Drift.

---

### 🔗 Continue Reading the Series
* 🚀 **Master Benchmark Hub (All 41 Patterns Ranked):** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)
* 🛡️ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)
* 🧪 **Previous (`E06`):** [Why AI Agents Write Weak Tests for Their Own Code: Single-Agent TDD vs. Two-Agent Blind TDD (0% vs. 100% Mutant Kill Rate)](https://tappers.hashnode.dev/e06-blind-tdd-worktrees)
* 📐 **Next (`E08`):** [Vibe Coding vs. Spec-First SDD: How Locking an Executable Spec Eliminates Rework and Cuts Input Tokens by 32.9%](https://tappers.hashnode.dev/e08-spec-first-sdd)