💡 AI Labo Benchmark Series — Experiment
E24(Prompt Hygiene & Affirmative Guides)This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.
Metadata & Experiment Identity
- Experiment ID:
E24(Backlog CandidateC020) - Run ID:
e24-20261028-vm228-01 - Sandbox Environment: Proxmox Isolated VM
VM228(ai-e24-228, 0 NICs,PrivateNetwork=yes,NoNewPrivileges=yes,InaccessiblePaths, UID splitlabexp24/labjudge24) - Model:
gemini-3.8-flash(thinkingLevel: LOW,temperature=0.0) - Pattern Under Test: "Defeating the Pink Elephant Effect in Coding Agents: Positive Canonical Examples over Negative 'NEVER DO X' Rules" — Negative Prohibition Walls (
NegativeProhibitionsRunner) vs. Affirmative Golden Code Template (PositiveFewShotTemplateRunner)
1. Executive Summary (TL;DR)
Open almost any long-lived team CLAUDE.md or .cursorrules file and you will see a wall of uppercase negative commands:
- "NEVER use
sorted(order)when iterating keys!" - "DO NOT mutate input dictionaries or copy unknown keys via
dict(item)!" - "NEVER use
< 1000instead of<= 1000!"
In transformer attention mechanisms, repeating NEVER DO <ANTI-PATTERN> dozens of times triggers the Pink Elephant Effect (Ironic Process Theory in Token Prediction): by filling the context window with forbidden token sequences and negative audit logs, you increase attention weight on the very anti-patterns you want the model to avoid, while diluting attention on positive boundary invariants.
We tested this inside isolated VM VM228 on the 3-module enterprise payment settlement pipeline (validator.py ➔ settlement.py ➔ statement.py):
- Mode A (
NegativeProhibitionsRunner): Subjects the agent to a wall of 25+ repetitiveDO NOT / NEVERprohibition rules and negative audit dumps (77,004tool output chars across Turns 1–2). - Mode B (
PositiveFewShotTemplateRunner): Deletes all negativeNEVER DO Xprose and injects a compact 22-line Affirmative Golden Code Template (POSITIVE_FEWSHOT_GOLDEN_TEMPLATE_MD) showing the exact positive Python idioms to emulate.
Key Result
Mode A suffered the worst accuracy degradation in the payment pipeline suite—dropping to 16 / 20 (80.0%, 4 failed test cases) after 3 turns (48,640 input tokens) because it forgot the empty-list check (len(raw_txns) == 0), rejected chargeback_bps == 1000 (0 <= bps < 1000), and rejected $0 authorizations (cents <= 0). Mode B achieved a 20 / 20 (100%) pass in 1 turn, cutting LLM calls by -66.7% (3 ➔ 1), cumulative input tokens by -95.7% (48,640 ➔ 2,092 tokens—a 23.25x reduction), API cost by -84.1% ($0.0477 ➔ $0.0076), and running 2.85x faster (34.87s ➔ 12.23s, -64.9%).
| Metric | Mode A: Negative Prohibition Wall (NegativeProhibitions) |
Mode B: Positive Golden Template (PositiveFewShotTemplate) |
Delta / Improvement |
|---|---|---|---|
Hidden Grading Suite Score (labjudge24) |
16 / 20 (80.0% — 4 failures) |
20 / 20 (100% — Perfect) |
+20.0% accuracy (+4 cases) |
Prohibition Drift Traps (gotcha_traps_hit) |
2 traps (Turns 1 & 2) |
0 traps |
-100% (zero rework) |
LLM Calls (model_calls) |
3 calls |
1 call (1-shot) |
-66.7% (3 ➔ 1 turn) |
Cumulative File Edits (file_edits) |
5 edits |
3 edits (1 per module) |
-40.0% fewer edits |
Cumulative Tool Output (tool_output_chars) |
77,004 chars |
101 chars |
-99.9% (762.4x noise reduction) |
| Turn 1 Tokens (In / Out) | 1,649 in / 12 out |
2,092 in / 1,610 out |
+443 in for 22-line Golden Template |
| Turn 2 Tokens (In / Out) | 15,807 in / 1,335 out |
— (completed on Turn 1) | Avoided Turn 1 prohibition wall |
| Turn 3 Tokens (In / Out) | 31,184 in / 1,654 out |
— (completed on Turn 1) | Avoided Turn 2 cascading drift |
Cumulative Input Tokens (promptTokenCount) |
48,640 tokens |
2,092 tokens |
-95.7% (23.25x efficiency) |
Cumulative Output Tokens (candidatesTokenCount) |
3,001 tokens |
1,610 tokens |
-46.4% output reduction |
Elapsed Execution Time (elapsed_seconds) |
34.87 sec |
12.23 sec |
-64.9% (2.85x faster) |
Estimated API Cost (estimated_micro_usd) |
$0.0477 (47,735 micro-USD) |
$0.0076 (7,607 micro-USD) |
-84.1% (6.28x cheaper) |
2. Experiment Architecture: Prohibition Wall vs. Positive Golden Template
+-------------------------------------------------------------------------+
| Proxmox Isolated VM228 (0 NICs, InaccessiblePaths, grader labjudge24) |
| Target Task: 3-Module Enterprise Payment Settlement Pipeline |
| validator.py ➔ settlement.py ➔ statement.py |
| |
| +-------------------------------------------------------------------+ |
| | Mode A: NegativeProhibitionsRunner ("NEVER DO X" Wall) | |
| | - Turn 1 (1,649 in): Calls `audit_negative_prohibitions` ➔ | |
| | floods context with 74 negative rules | |
| | ("NEVER use sorted(order)...", 38,513c) | |
| | - Turn 2 (15,807 in): Calls `stage_prohibition_retry` ➔ | |
| | another 38,167 chars of prohibition logs | |
| | - Turn 3 (31,184 in): Attention dilution causes 4 missed bounds: | |
| | empty list `[]`, `cents == 0`, `bps == 1000`| |
| | --> 3 calls | 48,640 in | $0.0477 | 34.87s | 16/20 (4 FAILED) | |
| +-------------------------------------------------------------------+ |
| |
| +-------------------------------------------------------------------+ |
| | Mode B: PositiveFewShotTemplateRunner (1 Affirmative Example) | |
| | - Prompt: Injects 22-line `POSITIVE_FEWSHOT_GOLDEN_TEMPLATE_MD` | |
| | showing exact affirmative Python idioms (+443 tokens) | |
| | - Turn 1 (2,092 in): Emulates golden idioms across all 3 files | |
| | in a single turn with zero rework | |
| | --> 1 call | 2,092 in | $0.0076 | 12.23s | 20/20 (100% PASS) | |
| +-------------------------------------------------------------------+ |
+-------------------------------------------------------------------------+
3. The 22-Line Affirmative Golden Code Template (input-03.json)
Instead of listing 25 ways not to write the code, Mode B injected this exact 22-line affirmative code snippet (+443 tokens):
# 1. validator.py — Positive Canonical Idiom:
ALLOWED_CURRENCIES = ("USD", "EUR", "JPY")
# Validate non-empty list, reject isinstance(num, bool) before isinstance(num, int),
# enforce cnt > 0, cents >= 0, 0 <= bps <= 1000, and construct a fresh 5-key dict:
out.append({
"merchant_id": mid.strip().upper(),
"currency": cur.strip().upper(),
"tx_count": cnt,
"unit_cents": cents,
"chargeback_bps": bps,
})
# 2. settlement.py — Positive Canonical Idiom:
# Preserve first-seen `order` list (`for k in order:`), apply 10% reserve strictly when `g["chargeback_bps"] > 300`,
# apply inclusive `subtotal >= 15000` tier discounts (GROWTH: 10% if >=15000 else 0; ENTERPRISE: 20% if >=15000 else 10%),
# and cap `rebate_applied = min(fee_rebate, after_discount)`.
# 3. statement.py — Positive Canonical Idiom:
# Format each line with `#` delimiter and always prefix discount/rebate with `-`:
out.append(f"{item['merchant_id']}#{item['currency']}: tx={item['tx_count']} unit={item['unit_cents']} bps={item['chargeback_bps']} = {item['line_total']}")
out.append(f"TIER_DISCOUNT: -{batch['tier_discount']}")
out.append(f"REBATE_APPLIED: -{batch['rebate_applied']}")
4. Why Mode A Dropped to 16 / 20: Inspecting the 4 Broken Cases
In Mode A (negative-rules-bundle.json), the model was so preoccupied with avoiding the forbidden tokens shouted in the PROHIBITION_RULE wall (sorted(order) and dict(item)) that its final validator.py missed three basic boundary checks (validator_02, validator_04, validator_05, and downstream e2e_03):
# ❌ Mode A (NegativeProhibitionsRunner — Turn 3 Final validator.py, 16/20)
def validate_txns(raw_txns: list) -> list:
if not isinstance(raw_txns, list): # <-- BUG 1: Forgot `or len(raw_txns) == 0` (Fails validator_02)!
raise ValueError("raw_txns must be a list")
...
if cnt <= 0 or cents <= 0 or not (0 <= bps < 1000): # <-- BUG 2: `cents <= 0` rejects $0 auth (Fails validator_05 & e2e_03)!
# <-- BUG 3: `bps < 1000` rejects valid 1000 bps (Fails validator_04)!
raise ValueError("numeric bounds violated")
Meanwhile, Mode B copied the affirmative pattern (len(raw_txns) == 0, cents < 0, 0 <= bps <= 1000, and the 5-key dictionary literal) in one shot (20 / 20).
5. Practical Takeaway: Audit Your CLAUDE.md for NEVER and DO NOT
- Replace every 5 lines of
DO NOT do Xprose with 1 affirmative code block (Do Y like this): LLMs are next-token pattern completion engines; showing the exact target syntax activates in-context imitation directly, whereas negative rules require two-hop reasoning ("what is X, and what should I write instead of X?"). - Enforce true prohibitions in Pre-Commit AST Hooks (
E11), not prompts: If a token sequence is genuinely forbidden (likeimport osin a pure domain layer orsorted(order)), block it mechanically in a 5ms AST hook and keep your prompt 100% affirmative.
🔗 Continue Reading the Series
- 🚀 Master Benchmark Hub (All 41 Patterns Ranked): I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy
- 🛡️ Methodology & Sandbox Architecture: How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox
- 📦 Previous Sequential Post (
E23): Stopping Edit-Test Thrashing Across Coupled Modules: Sequential Single-File Edits vs. Atomic Multi-File Batches (2.26x Faster) - ⚡ Next Sequential Post (
E25): Stop Invalidating Your Coding Agent's KV Cache: Dynamic System Prompt Mutation vs. Append-Only Prefix-Stable Prompt Caching (2.63x Faster) - 🛡️ Related Guardrail Deep Dive (
E11): Why Long System Prompts Can't Enforce Code Architecture: Prompt Rules vs. Deterministic Pre-Commit AST Hooks
Comments
Join the discussionNo comments yet. Be the first to comment.