# Why Long System Prompts Can't Enforce Code Architecture: Prompt Rules vs. Deterministic Pre-Commit AST Hooks

> 💡 **AI Labo Benchmark Series — Experiment `E11` (Deterministic Guardrails & Sensors)**
>
> This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite and AST policy audits.
> * 🚀 **[View All 41 Benchmark Results & Master Ranking](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)**
> * 🛡️ **[Read How Our Proxmox & systemd Anti-Cheating Sandbox Works](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)**

---

## Metadata & Experiment Identity
* **Experiment ID:** `E11`
* **Run ID:** `e11-vm215-20261028-01`
* **Sandbox Environment:** Proxmox Isolated VM `VM215` (0 NICs, `virtio-serial` QGA relay, isolated grader `labjudge11`)
* **Model:** `gemini-3.8-flash`
* **Pattern Under Test:** **Verbose System-Prompt Style Manuals vs. Upfront AST Violation Diagnostics + Deterministic Pre-Commit AST Hooks**

---

## 1. Executive Summary (TL;DR)

To keep AI coding agents from eroding codebase architecture—violating layer boundaries, introducing mutable global state, skipping return type annotations, or bloating files past modular size limits—teams routinely stuff dozens of rules into `CLAUDE.md`, `.cursorrules`, or system prompts (**Prompt-Only Guidelines**).

Yet in production, prompt-only architectural rules suffer from a fundamental flaw: **Silent Soft-Constraint Drift**. While an LLM focuses on making functional unit tests pass, it probabilistically violates structural constraints (especially line-count budgets and cross-layer invariants)—and because functional smoke tests still pass, the agent submits the violation without ever realizing it broke a rule.

We tested this head-to-head inside isolated VM `VM215` on a 3-module **cross-border freight & customs tariff pipeline** (`intake.py` ➔ `tariff.py` ➔ `waybill.py`) whose initial buggy fixtures contained **6 functional bugs plus 14 AST architecture violations across 5 strict policy rules**:

* **Mode A (`BloatedPromptRunner`):** Places a long, emphatic architectural manual (`RULE 1`–`RULE 5`) in the system prompt, while `SUBMIT` runs only functional smoke checks.
* **Mode B (`DeterministicHookRunner`):** Runs a 0-token Python AST linter upfront to inject the exact list of 14 existing AST violations into the initial prompt, and enforces a **hard Pre-Commit AST Hook gate** on `COMMIT_AND_SUBMIT` that mechanically rejects any commit violating `RULE 1`–`RULE 5`.

### Key Result
Both modes passed **`20 / 20` (100%)** on functional tests in a single turn. However, **Mode A failed overall compliance (`4 / 5` architecture rules, `False`)**: it expanded `tariff.py` to **`74` lines** (silently breaching `RULE_5`'s 65-line cap) and submitted anyway. **Mode B achieved `20 / 20` functional accuracy AND `5 / 5` (100%) architectural compliance (`0` violations)** on Turn 1 (`hook_rejections = 0`), writing **`-18.1%` more compact code (`113` vs. `138` total lines)** while reducing **output tokens by `-10.8%`**, **latency by `-5.6%`**, and **API cost by `-6.6%`**.

| Metric | Mode A: Prompt-Only Manual (`BloatedPrompt`) | Mode B: Pre-Commit AST Hook (`DeterministicHook`) | Delta / Improvement |
| :--- | :---: | :---: | :---: |
| **Hidden Functional Suite Score (20 Cases)** | `20 / 20` (100%) | **`20 / 20` (100%)** | Both pass functional tests |
| **AST Architecture Policy Compliance (5 Rules)** | **`4 / 5` (80% — FAILED)** | **`5 / 5` (100% — PASSED)** | **Eliminated silent architectural drift** |
| **Remaining AST Violations at Submit** | `1 violation` (`tariff.py`: `74 > 65` lines) | **`0 violations`** | **100% policy enforcement** |
| **Overall Pass (`Functional 20/20 AND Arch 5/5`)** | ❌ **`False`** | ✅ **`True`** | **Only Hook mode met full spec** |
| **Total Generated Code Size (3 Modules)** | `138 lines` (`46 + 74 + 18`) | **`113 lines` (`35 + 60 + 18`)** | **-18.1% cleaner, tighter code** |
| **LLM Calls (`model_calls`)** | `1 call` | **`1 call`** | Single-shot completion |
| **Input Tokens (`promptTokenCount`)** | `2,535 tokens` | `2,753 tokens` | `+218` tokens (14-item AST audit) |
| **Output Tokens (`candidatesTokenCount`)** | `1,849 tokens` | **`1,650 tokens`** | **-10.8% (`-199` output tokens)** |
| **Elapsed Execution Time (`elapsed_seconds`)** | `15.10 sec` | **`14.26 sec`** | **-5.6% faster** |
| **Estimated API Cost (`estimated_micro_usd`)** | `$0.0088` (`8,835 micro-USD`) | **`$0.0083` (`8,253 micro-USD`)** | **-6.6% cost reduction** |

---

## 2. Experiment Architecture: Prompt Manual vs. Pre-Commit AST Hook

```text
+-------------------------------------------------------------------------+
| Proxmox Isolated VM215 (0 NICs, virtio-serial QGA, grader labjudge11)   |
|  Target Task: Cross-Border Freight & Tariff Pipeline                    |
|               (intake.py ➔ tariff.py ➔ waybill.py)                      |
|               Initial State: Functional Bugs + 14 AST Rule Violations   |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: BloatedPromptRunner (Verbose Prompt Rules, Smoke-Only)    |  |
|  |  - Prompt: Long prose manual explaining RULE 1 through RULE 5     |  |
|  |  - Turn 1: Writes intake (46L), tariff (74L), waybill (18L)       |  |
|  |  - Gate:   Runs functional smoke test only (PASS) ➔ SUBMIT        |  |
|  |  --> Functional: 20/20 | Architecture: 4/5 FAIL (tariff.py = 74L) |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: DeterministicHookRunner (AST Diagnostic + Hard Gate)      |  |
|  |  - Pre-Scan (0 tokens): audit_architecture_guardrails() injects   |  |
|  |    structured list of all 14 initial AST violations + 65L cap     |  |
|  |  - Turn 1: Writes intake (35L), tariff (60L), waybill (18L)       |  |
|  |  - Gate:   COMMIT_AND_SUBMIT runs Pre-Commit AST Hook (5/5 PASS)  |  |
|  |  --> Functional: 20/20 | Architecture: 5/5 PASS (0 violations)    |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
```

---

## 3. Audit of the 5 AST Architecture Policy Rules (`labjudge11`)

The isolated `labjudge11` grader ran deterministic Python `ast` static analysis across the initial buggy fixtures, Mode A's submission, and Mode B's submission:

| Policy Rule ID | Architectural Constraint (AST Check) | Initial Fixture | Mode A (`BloatedPrompt`) | Mode B (`DeterministicHook`) |
| :--- | :--- | :---: | :---: | :---: |
| **`RULE_1_ZERO_IMPORTS`** | Self-contained pure domain core: zero `import` or `from ... import` statements allowed | ❌ `3 violations` (`math`, `decimal`, `os`) | ✅ `0 violations` | ✅ **`0 violations`** |
| **`RULE_2_EXPLICIT_RETURN_TYPE`** | Explicit type contracts: every `def` function must declare an AST `returns` annotation (`-> list`, etc.) | ❌ `3 violations` (missing on all 3 functions) | ✅ `0 violations` | ✅ **`0 violations`** |
| **`RULE_3_NO_MUTABLE_GLOBAL_STATE`** | Immutable module globals: top-level `list`, `dict`, and `set` literals prohibited (must use `tuple`) | ❌ `4 violations` (`ALLOWED_HS_CLASSES = [...]`, etc.) | ✅ `0 violations` | ✅ **`0 violations`** |
| **`RULE_4_DOMAIN_BOUNDARY`** | Strict layer separation: `intake.py` cannot reference contract tiers; `waybill.py` (view layer) cannot perform arithmetic (`*`, `//`) | ❌ `1 violation` (recalculation inside `waybill.py`) | ✅ `0 violations` | ✅ **`0 violations`** |
| **`RULE_5_DOCSTRING_AND_SIZE`** | Every module & function must have a docstring, AND **every file must stay `<= 65` lines** | ❌ `3 violations` (missing docstrings) | ❌ **`1 violation` (`tariff.py` hit `74` lines > `65`)** | ✅ **`0 violations` (`tariff.py` = `60` lines)** |

---

## 4. Why LLMs Violate Prompt Rules—and Why Hooks Fix Both Sides

### Why Mode A Silently Broke `RULE_5`
Inspecting `prompt-bundle.json` shows why prose instructions fail on structural budgets:
* Token-predicting models have **poor real-time line-counting awareness** while generating code token-by-token. Mode A followed local syntax rules (`tuple` globals, `-> dict` return types, docstrings), but spread function signatures and dictionary literals across extra lines in `tariff.py`, pushing it to **`74` lines** (`+9` over the `65`-line limit).
* Because `SUBMIT` only executed functional tests, the harness replied *"Smoke check passed"* and accepted the commit. Without a deterministic sensor, any rule that doesn't crash unit tests will eventually decay.

### Why Mode B Succeeded in 1 Shot With *Fewer* Tokens
Mode B combined a **Feed-Forward Guide** with a **Feedback Sensor** (a preview of the pattern we explore further in **`E39`**):
1. **Upfront 0-Token AST Diagnostic:** Running `audit_architecture_guardrails()` before Turn 1 gave the model exact line numbers for all 14 existing violations and highlighted that `COMMIT_AND_SUBMIT` mechanically blocks any file exceeding `MAX_RULE5_LINES = 65`.
2. **Tighter Code Generation:** Knowing a mechanical AST gate stood at commit time, the model formatted `intake.py` in `35` lines (`-11` vs. Mode A) and `tariff.py` in `60` lines (`-14` vs. Mode A)—clearing the hook on the first call (`hook_rejections = 0`) and saving `10.8%` in output tokens.

---

## 5. Practical Takeaway: Move Rules Out of `CLAUDE.md` and Into Pre-Commit Hooks

1. **If a rule can be checked by an AST parser or linter in 5ms, delete the paragraph from your system prompt and make it a Pre-Commit Hook:** Layer imports (`intake` vs. `waybill`), mutable globals, missing type annotations, and file-length caps belong in `ruff`, `eslint`, `semgrep`, or a 40-line `ast` script—not a 500-line prompt manual.
2. **Pair commit hooks with upfront diagnostics and concrete fix hints:** As we also demonstrate in **`E17` (Actionable Fix-Suggestion Linter)** and **`E24` (Positive Few-Shot vs. Negative Rules)**, deterministic hooks work best when violations return an exact `line:col` plus a 1-line remediation recipe.

---

### 🔗 Continue Reading the Series
* 🚀 **Master Benchmark Hub (All 41 Patterns Ranked):** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)
* 🛡️ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)
* ⚔️ **Previous Post (`E10`):** [Why Self-Review Fails in AI Coding Agents: Single-Agent Self-Review vs. an Adversarial Critic (0% vs. 100% Mutant Kill Rate)](https://tappers.hashnode.dev/e10-adversarial-critic)
* 🗜️ **Rank #6 Companion Post (`E12`):** [How Save-Game Context Compaction Fixes Spec Drift in Multi-Step Coding Agents (-91.9% Input Tokens)](https://tappers.hashnode.dev/e12-milestone-context-compaction)
* 🏆 **Next Sequential Post (`E13`):** [Escaping Sunk-Cost Retry Loops in Coding Agents: Single-Session Retries vs. a Parallel Best-of-N Tournament (-67.2% Input Tokens)](https://tappers.hashnode.dev/e13-parallel-tournament)