Stop Returning Bare Linter Error Codes to Coding Agents: Opaque Error Codes vs. Actionable Fix-Hint Linters (-69.6% Input Tokens)

💡 AI Labo Benchmark Series — Experiment E17 (Feedback Sensors & Self-Correction)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

• 🚀 View All 41 Benchmark Results & Master Ranking
• 🛡️ Read How Our Proxmox & systemd Anti-Cheating Sandbox Works

---

Metadata & Experiment Identity

• Experiment ID: E17 (Backlog Candidate C013)
• Run ID: e17-vm221-20261028-01-03
• Sandbox Environment: Proxmox Isolated VM VM221 (ai-e17-221, 0 NICs, PrivateNetwork=yes, ProtectSystem=strict, UID split labexp17 / labjudge17)
• Model: gemini-3.8-flash
• Pattern Under Test: "Human-Oriented vs. Agent-Oriented Linter Messages" (Marmelab / agentlint / OpenAI & Anthropic Harness Guides) — Bare Error Codes (OpaqueErrorLinter) vs. Dynamic 💡 Fix Hint Remediation Recipes (ActionableFixHintLinter)

---

1. Executive Summary (TL;DR)

Traditional compilers and linters were designed for human engineers who already know their team's domain conventions. Returning a terse one-line error like E402 [settlement.py:29]: Ordering invariant or reserve threshold invariant violated feels "token-efficient" at first glance because the linter stdout is only 502 characters long.

For an AI coding agent, however, an opaque error code is an invitation to go on an expensive documentation hunt. When a linter tells the LLM what failed (E401–E404) without telling it how to fix it, the agent has to spend an extra turn calling read_rule_docs or grepping the repo—pulling tens of thousands of characters of unrelated rule manuals into its context window.

We tested this head-to-head inside isolated VM VM221 on a 3-module payment transaction validation, chargeback reserve, and settlement statement pipeline (validator.py, settlement.py, statement.py) containing 4 custom domain contract violations (E401–E404):

• Mode A (OpaqueErrorLinterRunner): Returns standard terse error codes and line numbers (E401–E404, 502 chars). To learn the exact domain constants and thresholds, the agent must call read_rule_docs on Turn 2, loading the 85-rule enterprise rulebook (27,651 chars, including 64 unrelated legacy SWIFT rules) before patching on Turn 3.
• Mode B (ActionableFixHintLinterRunner): Embeds a concrete, 1-line 💡 Fix Hint remediation recipe directly beneath each triggered error (2,145 chars total JSON), allowing the agent to self-correct all 3 modules immediately on Turn 2 with zero external doc lookups.

Key Result

Both modes passed 20 / 20 (100%) on the hidden grading suite. Embedding dynamic 💡 Fix Hint recipes directly into the linter output eliminated documentation lookup turns completely (doc_lookups: 1 ➔ 0, -100%), compressed cumulative tool output by -92.0% (28,252 ➔ 2,248 chars, 12.57x smaller), cut LLM calls by -33.3% (3 ➔ 2), reduced cumulative input tokens by -69.6% (13,367 ➔ 4,064), lowered API cost by -43.3% ($0.0160 ➔ $0.0091), and ran 1.45x faster (21.60s ➔ 14.86s, -31.2%).

Metric	Mode A: Bare Error Codes (OpaqueErrorLinter)	Mode B: Fix-Hint Linter (ActionableFixHintLinter)	Delta / Improvement
Hidden Grading Suite Score (labjudge17)	20 / 20 (100%)	20 / 20 (100%)	Identical 100% accuracy
Rulebook Lookup Turns (doc_lookups)	1 lookup (85-rule manual)	0 lookups	-100% (zero doc exploration)
Cumulative Tool Output (tool_output_chars)	28,252 chars	2,248 chars	-92.0% (12.57x compression)
LLM Calls (model_calls)	3 calls	2 calls	-33.3% (immediate self-heal)
Turn 1 Tokens (In / Out)	1,591 in / 11 out	1,614 in / 12 out	Initial linter call identical
Turn 2 Tokens (In / Out)	1,795 in / 10 out (read_rule_docs)	2,450 in / 1,592 out (Done)	Mode B finishes on Turn 2
Turn 3 Tokens (In / Out)	9,981 in / 1,569 out (Done)	— (completed on Turn 2)	Avoided 9,981-token manual bloat
Cumulative Input Tokens (promptTokenCount)	13,367 tokens	4,064 tokens	-69.6% (3.29x input efficiency)
Cumulative Output Tokens (candidatesTokenCount)	1,590 tokens	1,604 tokens	Identical code volume
Elapsed Execution Time (elapsed_seconds)	21.60 sec	14.86 sec	-31.2% (1.45x faster)
Estimated API Cost (estimated_micro_usd)	$0.0160 (15,989 micro-USD)	$0.0091 (9,064 micro-USD)	-43.3% cost reduction

---

2. Experiment Architecture: Opaque Linter vs. Actionable Fix-Hint Linter

+-------------------------------------------------------------------------+
| Proxmox Isolated VM221 (0 NICs, PrivateNetwork=yes, grader labjudge17)  |
|  Target Task: 3-Module Payment & Chargeback Settlement Pipeline         |
|               validator.py ➔ settlement.py ➔ statement.py               |
|               Initial State: 4 Domain Linter Violations (E401..E404)    |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: OpaqueErrorLinterRunner (Human-Style Bare Error Codes)    |  |
|  |  - Turn 1: Calls `run_opaque_linter` ➔ gets 502 chars of bare     |  |
|  |            error codes (`E401`..`E404`) without target values     |  |
|  |  - Turn 2: Forced to call `read_rule_docs` ➔ dumps 27,651 chars   |  |
|  |            (all 85 rules, including 64 legacy SWIFT rules)        |  |
|  |  - Turn 3: Submits fix with a bloated 9,981-token prompt          |  |
|  |  --> 3 calls | 13,367 in | 28,252 tool chars | 21.60s | 20/20     |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: ActionableFixHintLinterRunner (Agent-Style Fix Recipes)   |  |
|  |  - Turn 1: Calls `run_actionable_linter` ➔ gets 2,145 chars with  |  |
|  |            exact `💡 Fix Hint` code recipes for E401..E404 only   |  |
|  |  - Turn 2: Skips `read_rule_docs` (0 lookups) ➔ patches all 3     |  |
|  |            files in one shot (2,450 input tokens)                 |  |
|  |  --> 2 calls | 4,064 in  | 2,248 tool chars  | 14.86s | 20/20     |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

---

3. Side-by-Side Comparison: Bare Error vs. 💡 Fix Hint Output

Here are the exact linter messages returned for E401–E404 inside VM221:

Rule ID & Location	❌ Mode A (OpaqueErrorLinter — 502 chars total)	✅ Mode B (ActionableFixHintLinter — 2,145 chars total)
E401 (validator.py:21)	E401 [validator.py:21]: Contract violation in transaction normalization, currency allowlist, or input mutation guard.	Same header + 💡 Fix Hint (E401): Set ALLOWED_CURRENCIES = ('USD', 'EUR', 'JPY'), reject empty raw_txns (len(raw_txns) == 0), reject bool on int fields (isinstance(num, bool)), allow unit_cents >= 0 and 0 <= chargeback_bps <= 1000, and append a new dict {...} without mutating item.
E402 (settlement.py:29)	E402 [settlement.py:29]: Ordering invariant or reserve threshold invariant violated.	Same header + 💡 Fix Hint (E402): Iterate for k in order: (remove sorted(order)) to preserve first-seen key order, and change reserve condition to strictly g['chargeback_bps'] > 300 (reserve = (gross * 10) // 100 if g['chargeback_bps'] > 300 else 0).
E403 (settlement.py:37)	E403 [settlement.py:37]: Tier discount schedule or fee rebate cap invariant violated.	Same header + 💡 Fix Hint (E403): Use subtotal >= 15000 (inclusive) for GROWTH (10%) and ENTERPRISE (20% if subtotal >= 15000 else 10%: (subtotal * 10) // 100), and cap rebate_applied = min(fee_rebate, after_discount).
E404 (statement.py:9)	E404 [statement.py:9]: Canonical statement delimiter or signed zero footer format violated.	Same header + 💡 Fix Hint (E404): Format item lines with # (f"{item['merchant_id']}#{item['currency']}: tx=...") and unconditionally format footer discounts with a leading minus sign (f"TIER_DISCOUNT: -{batch['tier_discount']}" and f"REBATE_APPLIED: -{batch['rebate_applied']}").

---

4. Why "Shorter" Linter Output Cost 3.29x More Input Tokens

Look at the paradox in the turn-by-turn token trace (usage.json):

• On Turn 1, Mode A's linter output was 1,643 characters shorter than Mode B's (502 vs. 2,145 chars).
• Yet because 502 characters of bare error codes didn't tell the agent that "JPY" was the missing currency or that 15000 was the inclusive tier boundary, Mode A had to call read_rule_docs on Turn 2—dumping 27,651 characters (all 85 enterprise rules, including 64 irrelevant SWIFT rules E101–E164) into the chat history.
• That single doc lookup inflated Mode A's Turn 3 prompt to 9,981 input tokens—more than 4x larger than Mode B's Turn 2 prompt (2,450 input tokens).

By dynamically attaching only the 4 triggered 💡 Fix Hint recipes at the point of failure, Mode B acted as a Just-in-Time (JIT) Context Injector.

---

5. Practical Takeaway: Turn Every Custom Linter Into a JIT Fix-Hint Teacher

1. Never emit an error code without a 1-line remediation recipe: When wrapping ruff, eslint, mypy, or custom AST linters (E11) for an AI agent, map each rule ID to a concrete 💡 Fix Hint showing the exact helper function, import path, or idiom expected by your codebase.
2. Keep the full style manual out of the prompt: Instead of pre-loading a 28k-character rulebook (or forcing the agent to read it), inject only the remediation snippet for the specific rule that just failed (see also E39 Guides & Sensors Remediation).

---

🔗 Continue Reading the Series

• 🚀 Master Benchmark Hub (All 41 Patterns Ranked): I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy
• 🛡️ Methodology & Sandbox Architecture: How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox
• 🥉 Previous Sequential Post (E16 — Rank #3): Stop Dumping Raw Pytest Logs Into Your Coding Agent: How a Structured JSON Test Filter Cuts Input Tokens by 94.1%
• 🧭 Next Sequential Post (E18 — Rank #7): Benchmarking the Claude Code /advisor Pattern: Solo Executor Loops vs. Upfront Advisor Blueprint Routing
• 🚦 Next Newly Published Deep Dive (E22): Fail-Before-Fix in AI Coding Agents: Speculative Patching vs. a Reproduction-First Gate (-95.3% Input Tokens, 100% Accuracy)