# Why Coding Agents Need a Pre-Submit Diff Audit Gate: Blind Unchecked Submits vs. Blast-Radius & Entropy Auditing (-95.0% Input Tokens)

> 💡 **AI Labo Benchmark Series — Experiment `E32` (Verification & Pre-Submit Gates)**
>
> This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.
> * 🚀 **[View All 41 Benchmark Results & Master Ranking](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)**
> * 🛡️ **[Read How Our Proxmox & systemd Anti-Cheating Sandbox Works](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)**

---

## Metadata & Experiment Identity
* **Experiment ID:** `E32` (Backlog Candidate `C028`)
* **Run ID:** `e32-20261029-vm236-01`
* **Sandbox Environment:** Proxmox Isolated VM `VM236` (`ai-e32-236`, 0 NICs, `PrivateNetwork=yes`, `NoNewPrivileges=yes`, `InaccessiblePaths`, UID split `labexp32` / `labjudge32`)
* **Model:** `gemini-3.8-flash` (`thinkingLevel: LOW`, `temperature=0.0`)
* **Pattern Under Test:** *"OpenAI Entropy Garbage-Collection Sensor & Pre-Submit `git diff` Blast-Radius Audit Gate"* — **Blind Unchecked Submit (`BlindUncheckedSubmitRunner`) vs. Pre-Submit Blast-Radius & Diff Audit Gate (`PreSubmitBlastRadiusAuditRunner`)**

---

## 1. Executive Summary (TL;DR)

When an AI coding agent finishes editing files and sees its local smoke check pass, it immediately calls `SUBMIT`—often leaving behind **entropy pollution**: shallow copies (`dict(item)`) that leak extra dictionary keys to downstream callers, unverified boundary conditions (`cents <= 0` or `bps < 1000`), or unintended public signature mutations.

Adding a **Pre-Submit Blast-Radius & Diff Audit Gate** intercepts the submission before final commit, mechanically auditing modified symbols, downstream caller contracts, and forbidden idioms across the `git diff`.

We tested this head-to-head inside isolated VM `VM236` on the 3-module enterprise payment settlement pipeline (`validator.py` ➔ `settlement.py` ➔ `statement.py`):

* **Mode A (`BlindUncheckedSubmitRunner`):** Submits diffs without a pre-submit blast-radius or entropy gate, accumulating `74,181` chars of unaudited diff drift logs across Turns 1–2 and submitting a final bundle that leaks extra dict keys (`new_item = dict(item)`) and rejects `$0` authorizations (`cents <= 0`) and `1000` bps (`0 <= bps < 1000`).
* **Mode B (`PreSubmitBlastRadiusAuditRunner`):** Enforces a zero-entropy **Pre-Submit Blast-Radius & Diff Audit Gate** (`PRE_SUBMIT_BLAST_RADIUS_AUDIT_MD` + `pass_blast_radius_audit_and_submit`) verifying every modified symbol and downstream caller before commit.

### Key Result
Mode A stalled at **`16 / 20` (80.0%)** after 3 turns (`45,057` cumulative input tokens), failing 4 hidden grading tests (`validator_01`, `validator_04`, `validator_05`, and `e2e_03`). **Mode B achieved a `20 / 20` (100%) pass in 1 turn**, cutting **LLM calls by `-66.7%` (`3 ➔ 1`)**, **cumulative input tokens by `-95.0%` (`45,057 ➔ 2,251` tokens—a `20.02x` reduction)**, **API cost by `-82.8%` (`$0.0445 ➔ $0.0077`)**, and running **`2.21x` faster (`28.17s ➔ 12.75s`, `-54.7%`)**.

| Metric | Mode A: Blind Unchecked Submit (`BlindUncheckedSubmit`) | Mode B: Pre-Submit Diff Audit (`PreSubmitBlastRadiusAudit`) | Delta / Improvement |
| :--- | :---: | :---: | :---: |
| **Hidden Grading Suite Score (`labjudge32`)** | `16 / 20` (80.0% — 4 failures) | **`20 / 20` (100% — Perfect)** | **+20.0% accuracy (`+4` cases)** |
| **Entropy Drift Traps (`gotcha_traps_hit`)** | `2 traps` (Turns 1 & 2) | **`0 traps`** | **-100% (zero entropy leaks)** |
| **LLM Calls (`model_calls`)** | `3 calls` | **`1 call` (1-shot)** | **-66.7% (`3.00x` faster convergence)** |
| **Cumulative File Edits (`file_edits`)** | `5 edits` | **`3 edits` (1 per module)** | **-40.0% fewer edits** |
| **Cumulative Tool Output (`tool_output_chars`)** | `74,181 chars` | **`104 chars`** | **-99.9% (`713.3x` noise reduction)** |
| **Turn 1 Tokens (In / Out)** | `1,656 in` / `10 out` | **`2,251 in` / `1,597 out`** | `+595` in for blast-radius checklist |
| **Turn 2 Tokens (In / Out)** | `14,636 in` / `1,259 out` | *— (completed on Turn 1)* | Avoided unaudited diff drift logs |
| **Turn 3 Tokens (In / Out)** | `28,765 in` / `1,595 out` | *— (completed on Turn 1)* | Avoided secondary rework bloat |
| **Cumulative Input Tokens (`promptTokenCount`)** | `45,057 tokens` | **`2,251 tokens`** | **-95.0% (`20.02x` efficiency)** |
| **Cumulative Output Tokens (`candidatesTokenCount`)** | `2,864 tokens` | **`1,597 tokens`** | **-44.2% output reduction** |
| **Elapsed Execution Time (`elapsed_seconds`)** | `28.17 sec` | **`12.75 sec`** | **-54.7% (`2.21x` faster)** |
| **Estimated API Cost (`estimated_micro_usd`)** | `$0.0445` (`44,534 micro-USD`) | **`$0.0077` (`7,677 micro-USD`)** | **-82.8% (`5.80x` cheaper)** |

---

## 2. Experiment Architecture: Blind Submit vs. Pre-Submit Blast-Radius Audit

```text
+-------------------------------------------------------------------------+
| Proxmox Isolated VM236 (0 NICs, InaccessiblePaths, grader labjudge32)   |
|  Target Task: 3-Module Enterprise Payment Settlement Pipeline           |
|               validator.py ➔ settlement.py ➔ statement.py               |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: BlindUncheckedSubmitRunner (No Pre-Submit Diff Audit)     |  |
|  |  - Turn 1 (1,656 in):  Calls `run_unchecked_draft` ➔ 37,106 chars |  |
|  |                        of unaudited diff drift logs               |  |
|  |  - Turn 2 (14,636 in): Calls `stage_polluted_diff` on 2 files     |  |
|  |                        ➔ 36,760 chars of secondary drift logs     |  |
|  |  - Turn 3 (28,765 in): Submits `new_item = dict(item)` (leaks     |  |
|  |                        extra keys), `cents <= 0`, `bps < 1000`    |  |
|  |  --> 3 calls | 45,057 in | $0.0445 | 28.17s | 16/20 (4 FAILED)    |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: PreSubmitBlastRadiusAuditRunner (Zero-Entropy Diff Gate)  |  |
|  |  - Injects `PRE_SUBMIT_BLAST_RADIUS_AUDIT_MD` (+595 tokens):      |  |
|  |    Audits every modified symbol & downstream caller contract      |  |
|  |    (`validate_txns` ➔ `settle_batch` ➔ `render_statement`)        |  |
|  |  - Turn 1 (2,251 in): Passes blast-radius audit on first shot     |  |
|  |  --> 1 call  | 2,251 in  | $0.0077 | 12.75s | 20/20 (100% PASS)   |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
```

---

## 3. Drop-In Implementation Recipe: `audit_blast_radius_before_submit`

Gate your agent's `submit` tool with a lightweight AST signature and forbidden-idiom diff auditor:

```python
import ast


def extract_public_signatures(source_code: str) -> dict[str, list[str]]:
    """Extracts public function names and parameter lists from a module."""
    tree = ast.parse(source_code)
    sigs: dict[str, list[str]] = {}
    for node in tree.body:
        if isinstance(node, ast.FunctionDef) and not node.name.startswith("_"):
            sigs[node.name] = [arg.arg for arg in node.args.args]
    return sigs


def audit_blast_radius_before_submit(
    baseline_files: dict[str, str], candidate_files: dict[str, str]
) -> list[str]:
    """Audits public signature stability and blocks known entropy leaks before submit."""
    violations: list[str] = []
    for path, base_src in baseline_files.items():
        cand_src = candidate_files.get(path, "")
        if extract_public_signatures(base_src) != extract_public_signatures(cand_src):
            violations.append(f"{path}: public function signature mutated unexpectedly")
        if "sorted(order)" in cand_src:
            violations.append(f"{path}: forbidden `sorted(order)` remains in diff")
        if "dict(item)" in cand_src and path.endswith("validator.py"):
            violations.append(
                f"{path}: shallow copy `dict(item)` leaks extra keys; construct an explicit 5-key dict"
            )
    return violations
```

---

## 4. Practical Takeaway: Never Let `SUBMIT` Succeed Without Auditing the Diff

1. **Check the blast radius of every modified symbol before accepting `SUBMIT`:** Verify that public function signatures match downstream callers and that no temporary debug artifacts or shallow-copy leaks remain in the diff.
2. **Combine Pre-Submit Diff Auditing (`E32`) with Pre-Commit AST Hooks (`E11`):** Use AST hooks during editing and a cross-module blast-radius audit at final submission to guarantee zero-entropy patches.

---

### 🔗 Continue Reading the Series
* 🚀 **Master Benchmark Hub (All 41 Patterns Ranked):** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)
* 🛡️ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)
* 🌳 **Previous Sequential Post (`E31`):** [Stop Letting Coding Agents Do Manual Multi-Spot String Replacements: Manual Text Edits vs. Parameterized AST Codemod Tools (-94.8% Input Tokens)](https://tappers.hashnode.dev/e31-parameterized-ast-codemod-tool)
* 🎲 **Next Sequential Post (`E33`):** [Why Hand-Written Example Tests Fail AI Coding Agents: Fixed Sample Tests vs. a Property-Based Invariant Fuzzing Sensor (-95.1% Input Tokens)](https://tappers.hashnode.dev/e33-property-based-fuzzing-sensor)
* 🛡️ **Related Architecture Hooks Deep Dive (`E11`):** [Why Long System Prompts Can't Enforce Code Architecture: Prompt Rules vs. Deterministic Pre-Commit AST Hooks](https://tappers.hashnode.dev/e11-architecture-guardrail-hooks)