Stopping Edit-Test Thrashing Across Coupled Modules: Sequential Single-File Edits vs. Atomic Multi-File Batches (2.26x Faster)

💡 AI Labo Benchmark Series — Experiment E23 (Edit Primitives & Cross-Module Transactions)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

🚀 View All 41 Benchmark Results & Master Ranking

🛡️ Read How Our Proxmox & systemd Anti-Cheating Sandbox Works

Metadata & Experiment Identity

Experiment ID: E23 (Backlog Candidate C019)
Run ID: e23-20261028-vm227-01
Sandbox Environment: Proxmox Isolated VM VM227 (ai-e23-227, 0 NICs, PrivateNetwork=yes, NoNewPrivileges=yes, InaccessiblePaths, UID split labexp23 / labjudge23)
Model: gemini-3.8-flash (thinkingLevel: LOW, temperature=0.0)
Pattern Under Test: "Cross-Module Refactoring in Coding Agents: Atomic Multi-File Patching vs. Sequential Edit-Test Thrashing" — Sequential Single-File Edit + Test (SequentialSingleFileEditRunner) vs. Atomic Multi-File Batch Transaction (AtomicMultiFileBatchRunner)

1. Executive Summary (TL;DR)

Many coding agent harnesses wire up an automatic post-edit hook that runs the test suite every time a single file is modified—or expose an edit tool that only accepts one file path per turn.

On single-file bugs, that works fine. On coupled multi-module changes (where validator.py produces normalized records consumed by settlement.py, which produces batch dictionaries formatted by statement.py), running tests after editing just one file guarantees a Broken Intermediate State (CrossModuleContractDesyncError). The resulting cascade of intermediate test failures floods the context window and confuses the agent into partial rewrites.

We tested this inside isolated VM VM227 across the 3-module enterprise payment settlement pipeline (validator.py ➔ settlement.py ➔ statement.py):

Mode A (SequentialSingleFileEditRunner): Edits files incrementally across turns and triggers test runs in broken intermediate states (61,504 chars of cross-module desync logs across Turns 1–2) before submitting on Turn 3.
Mode B (AtomicMultiFileBatchRunner): Injects an explicit cross-module transaction protocol (ATOMIC_MULTIFILE_BATCH_MD) and exposes an Atomic Multi-File Batch tool (apply_atomic_multifile_batch) that applies all 3 interdependent module updates together in a single transaction before running verification tests.

Key Result

Mode A degraded to 17 / 20 (85.0%) after 3 turns (41,664 input tokens), leaving dictionary spread **item key leaks and $0 authorization bugs in validator.py. Mode B achieved 20 / 20 (100%) in 1 turn, cutting LLM calls by -66.7% (3 ➔ 1), cumulative input tokens by -94.7% (41,664 ➔ 2,208 tokens—an 18.87x reduction), API cost by -81.8% ($0.0419 ➔ $0.0076), and running 2.26x faster (28.23s ➔ 12.47s, -55.8%).

Metric	Mode A: Sequential Edit + Test (`SequentialSingleFileEdit`)	Mode B: Atomic Multi-File Batch (`AtomicMultiFileBatch`)	Delta / Improvement
Hidden Grading Suite Score (`labjudge23`)	`17 / 20` (85.0% — 3 failures)	`20 / 20` (100% — Perfect)	+15.0% accuracy (`+3` cases)
Intermediate Desync Traps (`gotcha_traps_hit`)	`2 traps` (Turns 1 & 2)	`0 traps`	-100% (zero intermediate thrashing)
LLM Calls (`model_calls`)	`3 calls`	`1 call` (1-shot)	-66.7% (`3 ➔ 1` turn)
Cumulative File Edits (`file_edits`)	`5 edits`	`3 edits` (1 atomic batch)	-40.0% fewer edits
Cumulative Tool Output (`tool_output_chars`)	`61,504 chars`	`98 chars`	-99.8% (`627.6x` noise reduction)
Turn 1 Tokens (In / Out)	`1,658 in` / `19 out`	`2,208 in` / `1,595 out`	`+550` in for cross-module spec
Turn 2 Tokens (In / Out)	`13,515 in` / `1,239 out`	— (completed on Turn 1)	Avoided Turn 1 desync log bloat
Turn 3 Tokens (In / Out)	`26,491 in` / `1,588 out`	— (completed on Turn 1)	Avoided Turn 2 cascading log bloat
Cumulative Input Tokens (`promptTokenCount`)	`41,664 tokens`	`2,208 tokens`	-94.7% (`18.87x` efficiency)
Cumulative Output Tokens (`candidatesTokenCount`)	`2,846 tokens`	`1,595 tokens`	-44.0% output reduction
Elapsed Execution Time (`elapsed_seconds`)	`28.23 sec`	`12.47 sec`	-55.8% (`2.26x` faster)
Estimated API Cost (`estimated_micro_usd`)	`$0.0419` (`41,922 micro-USD`)	`$0.0076` (`7,638 micro-USD`)	-81.8% (`5.49x` cheaper)

2. Experiment Architecture: Sequential Edit-Test Thrashing vs. Atomic Batch

+-------------------------------------------------------------------------+
| Proxmox Isolated VM227 (0 NICs, InaccessiblePaths, grader labjudge23)   |
|  Target Task: 3-Module Coupled Payment Settlement Pipeline              |
|               validator.py (Producer) ➔ settlement.py ➔ statement.py    |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: SequentialSingleFileEditRunner (Partial Edit ➔ Run Tests) |  |
|  |  - Turn 1 (1,658 in):  Calls `edit_single_file_and_test` ➔        |  |
|  |                        triggers 74 CrossModuleContractDesyncError |  |
|  |                        traces (30,766 chars)                      |  |
|  |  - Turn 2 (13,515 in): Calls `patch_second_file_and_test` ➔       |  |
|  |                        updates validator + settlement while       |  |
|  |                        statement still desynced (30,420 chars)    |  |
|  |  - Turn 3 (26,491 in): Submits all 3 files, but attention drift   |  |
|  |                        leaves `**item` key leak & `cents <= 0`    |  |
|  |  --> 3 calls | 41,664 in | $0.0419 | 28.23s | 17/20 (3 FAILED)    |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: AtomicMultiFileBatchRunner (All-or-Nothing Batch Commit)  |  |
|  |  - Protocol: `ATOMIC_MULTIFILE_BATCH_MD` defines Producer/Consumer|  |
|  |              contracts across all 3 modules upfront               |  |
|  |  - Turn 1 (2,208 in): Calls `apply_atomic_multifile_batch` with   |  |
|  |                       `validator.py`, `settlement.py`, and        |  |
|  |                       `statement.py` in a single atomic payload   |  |
|  |                       BEFORE running verification tests           |  |
|  |  --> 1 call  | 2,208 in  | $0.0076 | 12.47s | 20/20 (100% PASS)   |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

3. Why Intermediate Test Runs Poison Context (`input-01.json` vs. `input-03.json`)

When Mode A ran verification tests while only a subset of modules was staged, the test runner emitted 61,504 characters of CrossModuleContractDesyncError traces (30,766 chars on Turn 1 + 30,420 chars on Turn 2). By Turn 3 (26,491 input tokens), the agent's context window was 92% filled with intermediate desync noise.

Look at how that noise affected Mode A's final validator.py (sequential-edit-bundle.json) compared to Mode B (atomic-batch-bundle.json):

# ❌ Mode A (SequentialSingleFileEditRunner — Turn 3 Final validator.py, 17/20)
        if cnt <= 0 or cents <= 0 or not (0 <= bps <= 1000):  # <-- Fails validator_05 & e2e_03 ($0 auth rejected)
            raise ValueError("numeric bounds violated")
        out.append({
            **item,                                           # <-- Fails validator_01 (spreads unapproved "extra" keys!)
            "merchant_id": mid.strip().upper(),
            "currency": cur.strip().upper(),
        })

# ✅ Mode B (AtomicMultiFileBatchRunner — Turn 1 Single-Shot validator.py, 20/20)
        if cnt <= 0 or cents < 0 or not (0 <= bps <= 1000):   # <-- Passes validator_05 & e2e_03 ($0 auth allowed)
            raise ValueError("numeric bounds violated")
        out.append({                                          # <-- Passes validator_01 (exact 5-key schema)
            "merchant_id": mid.strip().upper(),
            "currency": cur.strip().upper(),
            "tx_count": cnt,
            "unit_cents": cents,
            "chargeback_bps": bps,
        })

4. Practical Takeaway: Defer Verification Until the Entire Change Set Is Applied

Never auto-run integration tests after a single-file edit when refactoring across module boundaries: Allow the agent to stage edits across multiple files (files: {"a.py": ..., "b.py": ..., "c.py": ...}) in a single tool call—or hold test execution until the agent signals that the multi-file batch is complete.
Combine Atomic Multi-File Batches (E23) with SEARCH/REPLACE Blocks (E15): Accepting a dictionary of file_path ➔ [search/replace blocks] gives you the token savings of anchor blocks (-63% output tokens) and the zero-desync guarantee of atomic multi-file commits (-94.7% input tokens).

🔗 Continue Reading the Series

🚀 Master Benchmark Hub (All 41 Patterns Ranked): I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy
🛡️ Methodology & Sandbox Architecture: How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox
🚦 Previous Sequential Post (E22): Fail-Before-Fix in AI Coding Agents: Speculative Patching vs. a Reproduction-First Gate (-95.3% Input Tokens, 100% Accuracy)
✨ Next Sequential Post (E24): Defeating the Pink Elephant Effect in Coding Agents: 25+ Negative 'NEVER DO X' Rules vs. 1 Positive Golden Code Template (2.85x Faster)
✂️ Related Edit Tool Deep Dive (E15): Why LLMs Fail at Unified Diffs: Standard Diff Hunks vs. Line-Number-Free SEARCH/REPLACE Blocks (2.41x Faster)

Stopping Edit-Test Thrashing Across Coupled Modules: Sequential Single-File Edits vs. Atomic Multi-File Batches (2.26x Faster)

Metadata & Experiment Identity

1. Executive Summary (TL;DR)

Key Result

2. Experiment Architecture: Sequential Edit-Test Thrashing vs. Atomic Batch

3. Why Intermediate Test Runs Poison Context (`input-01.json` vs. `input-03.json`)

4. Practical Takeaway: Defer Verification Until the Entire Change Set Is Applied

🔗 Continue Reading the Series

Comments

More from this blog

Stop Running Unrelated Monorepo Test Suites on Every Agent Edit: Full-Suite Runs vs. AST Dependency-Scoped Incremental Tests (-95.7% Input Tokens)

Deduplicating Cascading Test Failures for Coding Agents: Raw Error Dumps vs. AST Root-Cause Error Clustering (-95.3% Input Tokens)

Breaking A➔B➔A Edit Oscillations in Coding Agents: Unguarded Retry Loops vs. a Loop-Detection Circuit Breaker (-94.9% Input Tokens)

Stop Your Coding Agent From Cycling Through Refuted Guesses: Free-Text Monologue vs. a Structured Hypothesis Matrix (2.54x Faster)

Why Claude Code's Plan Mode Works: Interleaved Read-Write Loops vs. a Two-Phase Read-Only Plan Lock (-94.8% Input Tokens)

Metadata & Experiment Identity

1. Executive Summary (TL;DR)

Key Result

2. Experiment Architecture: Sequential Edit-Test Thrashing vs. Atomic Batch

3. Why Intermediate Test Runs Poison Context (input-01.json vs. input-03.json)

4. Practical Takeaway: Defer Verification Until the Entire Change Set Is Applied

🔗 Continue Reading the Series

Comments

More from this blog

Stop Running Unrelated Monorepo Test Suites on Every Agent Edit: Full-Suite Runs vs. AST Dependency-Scoped Incremental Tests (-95.7% Input Tokens)

Deduplicating Cascading Test Failures for Coding Agents: Raw Error Dumps vs. AST Root-Cause Error Clustering (-95.3% Input Tokens)

Breaking A➔B➔A Edit Oscillations in Coding Agents: Unguarded Retry Loops vs. a Loop-Detection Circuit Breaker (-94.9% Input Tokens)

Stop Your Coding Agent From Cycling Through Refuted Guesses: Free-Text Monologue vs. a Structured Hypothesis Matrix (2.54x Faster)

Why Claude Code's Plan Mode Works: Interleaved Read-Write Loops vs. a Two-Phase Read-Only Plan Lock (-94.8% Input Tokens)

3. Why Intermediate Test Runs Poison Context (`input-01.json` vs. `input-03.json`)