Why Coding Agents Need a Pre-Submit Diff Audit Gate: Blind Unchecked Submits vs. Blast-Radius & Entropy Auditing (-95.0% Input Tokens)

💡 AI Labo Benchmark Series — Experiment E32 (Verification & Pre-Submit Gates)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

• 🚀 View All 41 Benchmark Results & Master Ranking
• 🛡️ Read How Our Proxmox & systemd Anti-Cheating Sandbox Works

---

Metadata & Experiment Identity

• Experiment ID: E32 (Backlog Candidate C028)
• Run ID: e32-20261029-vm236-01
• Sandbox Environment: Proxmox Isolated VM VM236 (ai-e32-236, 0 NICs, PrivateNetwork=yes, NoNewPrivileges=yes, InaccessiblePaths, UID split labexp32 / labjudge32)
• Model: gemini-3.8-flash (thinkingLevel: LOW, temperature=0.0)
• Pattern Under Test: "OpenAI Entropy Garbage-Collection Sensor & Pre-Submit git diff Blast-Radius Audit Gate" — Blind Unchecked Submit (BlindUncheckedSubmitRunner) vs. Pre-Submit Blast-Radius & Diff Audit Gate (PreSubmitBlastRadiusAuditRunner)

---

1. Executive Summary (TL;DR)

When an AI coding agent finishes editing files and sees its local smoke check pass, it immediately calls SUBMIT—often leaving behind entropy pollution: shallow copies (dict(item)) that leak extra dictionary keys to downstream callers, unverified boundary conditions (cents <= 0 or bps < 1000), or unintended public signature mutations.

Adding a Pre-Submit Blast-Radius & Diff Audit Gate intercepts the submission before final commit, mechanically auditing modified symbols, downstream caller contracts, and forbidden idioms across the git diff.

We tested this head-to-head inside isolated VM VM236 on the 3-module enterprise payment settlement pipeline (validator.py ➔ settlement.py ➔ statement.py):

• Mode A (BlindUncheckedSubmitRunner): Submits diffs without a pre-submit blast-radius or entropy gate, accumulating 74,181 chars of unaudited diff drift logs across Turns 1–2 and submitting a final bundle that leaks extra dict keys (new_item = dict(item)) and rejects $0 authorizations (cents <= 0) and 1000 bps (0 <= bps < 1000).
• Mode B (PreSubmitBlastRadiusAuditRunner): Enforces a zero-entropy Pre-Submit Blast-Radius & Diff Audit Gate (PRE_SUBMIT_BLAST_RADIUS_AUDIT_MD + pass_blast_radius_audit_and_submit) verifying every modified symbol and downstream caller before commit.

Key Result

Mode A stalled at 16 / 20 (80.0%) after 3 turns (45,057 cumulative input tokens), failing 4 hidden grading tests (validator_01, validator_04, validator_05, and e2e_03). Mode B achieved a 20 / 20 (100%) pass in 1 turn, cutting LLM calls by -66.7% (3 ➔ 1), cumulative input tokens by -95.0% (45,057 ➔ 2,251 tokens—a 20.02x reduction), API cost by -82.8% ($0.0445 ➔ $0.0077), and running 2.21x faster (28.17s ➔ 12.75s, -54.7%).

Metric	Mode A: Blind Unchecked Submit (BlindUncheckedSubmit)	Mode B: Pre-Submit Diff Audit (PreSubmitBlastRadiusAudit)	Delta / Improvement
Hidden Grading Suite Score (labjudge32)	16 / 20 (80.0% — 4 failures)	20 / 20 (100% — Perfect)	+20.0% accuracy (+4 cases)
Entropy Drift Traps (gotcha_traps_hit)	2 traps (Turns 1 & 2)	0 traps	-100% (zero entropy leaks)
LLM Calls (model_calls)	3 calls	1 call (1-shot)	-66.7% (3.00x faster convergence)
Cumulative File Edits (file_edits)	5 edits	3 edits (1 per module)	-40.0% fewer edits
Cumulative Tool Output (tool_output_chars)	74,181 chars	104 chars	-99.9% (713.3x noise reduction)
Turn 1 Tokens (In / Out)	1,656 in / 10 out	2,251 in / 1,597 out	+595 in for blast-radius checklist
Turn 2 Tokens (In / Out)	14,636 in / 1,259 out	— (completed on Turn 1)	Avoided unaudited diff drift logs
Turn 3 Tokens (In / Out)	28,765 in / 1,595 out	— (completed on Turn 1)	Avoided secondary rework bloat
Cumulative Input Tokens (promptTokenCount)	45,057 tokens	2,251 tokens	-95.0% (20.02x efficiency)
Cumulative Output Tokens (candidatesTokenCount)	2,864 tokens	1,597 tokens	-44.2% output reduction
Elapsed Execution Time (elapsed_seconds)	28.17 sec	12.75 sec	-54.7% (2.21x faster)
Estimated API Cost (estimated_micro_usd)	$0.0445 (44,534 micro-USD)	$0.0077 (7,677 micro-USD)	-82.8% (5.80x cheaper)

---

2. Experiment Architecture: Blind Submit vs. Pre-Submit Blast-Radius Audit

+-------------------------------------------------------------------------+
| Proxmox Isolated VM236 (0 NICs, InaccessiblePaths, grader labjudge32)   |
|  Target Task: 3-Module Enterprise Payment Settlement Pipeline           |
|               validator.py ➔ settlement.py ➔ statement.py               |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: BlindUncheckedSubmitRunner (No Pre-Submit Diff Audit)     |  |
|  |  - Turn 1 (1,656 in):  Calls `run_unchecked_draft` ➔ 37,106 chars |  |
|  |                        of unaudited diff drift logs               |  |
|  |  - Turn 2 (14,636 in): Calls `stage_polluted_diff` on 2 files     |  |
|  |                        ➔ 36,760 chars of secondary drift logs     |  |
|  |  - Turn 3 (28,765 in): Submits `new_item = dict(item)` (leaks     |  |
|  |                        extra keys), `cents <= 0`, `bps < 1000`    |  |
|  |  --> 3 calls | 45,057 in | $0.0445 | 28.17s | 16/20 (4 FAILED)    |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: PreSubmitBlastRadiusAuditRunner (Zero-Entropy Diff Gate)  |  |
|  |  - Injects `PRE_SUBMIT_BLAST_RADIUS_AUDIT_MD` (+595 tokens):      |  |
|  |    Audits every modified symbol & downstream caller contract      |  |
|  |    (`validate_txns` ➔ `settle_batch` ➔ `render_statement`)        |  |
|  |  - Turn 1 (2,251 in): Passes blast-radius audit on first shot     |  |
|  |  --> 1 call  | 2,251 in  | $0.0077 | 12.75s | 20/20 (100% PASS)   |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

---

3. Drop-In Implementation Recipe: audit_blast_radius_before_submit

Gate your agent's submit tool with a lightweight AST signature and forbidden-idiom diff auditor:

import ast


def extract_public_signatures(source_code: str) -> dict[str, list[str]]:
    """Extracts public function names and parameter lists from a module."""
    tree = ast.parse(source_code)
    sigs: dict[str, list[str]] = {}
    for node in tree.body:
        if isinstance(node, ast.FunctionDef) and not node.name.startswith("_"):
            sigs[node.name] = [arg.arg for arg in node.args.args]
    return sigs


def audit_blast_radius_before_submit(
    baseline_files: dict[str, str], candidate_files: dict[str, str]
) -> list[str]:
    """Audits public signature stability and blocks known entropy leaks before submit."""
    violations: list[str] = []
    for path, base_src in baseline_files.items():
        cand_src = candidate_files.get(path, "")
        if extract_public_signatures(base_src) != extract_public_signatures(cand_src):
            violations.append(f"{path}: public function signature mutated unexpectedly")
        if "sorted(order)" in cand_src:
            violations.append(f"{path}: forbidden `sorted(order)` remains in diff")
        if "dict(item)" in cand_src and path.endswith("validator.py"):
            violations.append(
                f"{path}: shallow copy `dict(item)` leaks extra keys; construct an explicit 5-key dict"
            )
    return violations

---

4. Practical Takeaway: Never Let SUBMIT Succeed Without Auditing the Diff

1. Check the blast radius of every modified symbol before accepting SUBMIT: Verify that public function signatures match downstream callers and that no temporary debug artifacts or shallow-copy leaks remain in the diff.
2. Combine Pre-Submit Diff Auditing (E32) with Pre-Commit AST Hooks (E11): Use AST hooks during editing and a cross-module blast-radius audit at final submission to guarantee zero-entropy patches.

---

🔗 Continue Reading the Series

• 🚀 Master Benchmark Hub (All 41 Patterns Ranked): I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy
• 🛡️ Methodology & Sandbox Architecture: How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox
• 🌳 Previous Sequential Post (E31): Stop Letting Coding Agents Do Manual Multi-Spot String Replacements: Manual Text Edits vs. Parameterized AST Codemod Tools (-94.8% Input Tokens)
• 🎲 Next Sequential Post (E33): Why Hand-Written Example Tests Fail AI Coding Agents: Fixed Sample Tests vs. a Property-Based Invariant Fuzzing Sensor (-95.1% Input Tokens)
• 🛡️ Related Architecture Hooks Deep Dive (E11): Why Long System Prompts Can't Enforce Code Architecture: Prompt Rules vs. Deterministic Pre-Commit AST Hooks