Skip to main content
Hashnode
Tappers Tech Blog

Single Loops vs. Directed Graphs for AI Coding Agents: Cutting Input Tokens by 74.4% Without Losing Accuracy

Updated 9 min readMarkdown
T
tappers

💡 AI Labo Benchmark Series — Experiment E04 (Orchestration & Control Flow)

This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.

Metadata & Experiment Identity

  • Experiment ID: E04 (Backlog Candidate C001)
  • Run ID: e04-vm208-20261027-01
  • Sandbox Environment: Proxmox Isolated VM VM208 (ai-e04-208, 0 NICs, virtio-serial QGA relay, systemd UID separation)
  • Model: gemini-3.8-flash (thinkingLevel: LOW)
  • Pattern Under Test: "Loops and Graphs: How to Stop Babysitting Agents and Only Approve the Last Step" (@hanakoxbt) — Single Free-Form ReAct Loop vs. Stage-Gated Directed Graph

1. Executive Summary (TL;DR)

When building an AI coding agent, the default instinct is to give a single free-form ReAct Loop (Single Loop) access to every tool (inspect_all, edit_file, run_tests, SUBMIT) and let the LLM figure out what to inspect, which file to patch next, when to run tests, and when to stop.

On multi-module tasks, that design suffers from two compounding taxes:

  1. Orchestration Overhead: Over half of the LLM calls are spent deciding how to sequence steps rather than writing code.
  2. Snowballing Context History: Every file read, test log, and previously generated code patch stays in the chat history, inflating prompt tokens on every subsequent turn.

Inspired by the "Loops and Graphs" pattern—which separates inner local Generator + Verifier loops from an outer deterministic Directed Graph—we ran a controlled A/B experiment on a 3-module order & invoice pipeline (parser.py, pricing.py, report.py) containing 6 coupled bugs inside isolated VM VM208:

  • Mode A (SingleLoopRunner): A single free-form agent loop with full tool access across all 3 modules and persistent conversation history.
  • Mode B (DirectedGraphRunner): A 4-node deterministic Directed Graph (node_1_parser ➔ node_2_pricing ➔ node_3_report ➔ node_4_terminal_approval_gate). Each node receives a fresh, minimal context for its single target module, runs a deterministic stage verifier immediately upon edit, and resets chat history before advancing to the next node.

Key Result

Both modes achieved a perfect 20 / 20 (100%) pass on the hidden grading suite and generated virtually the exact same volume of Python code (1,188 vs. 1,189 output tokens—a 1-token difference!). Yet moving workflow orchestration into a stage-gated Directed Graph cut LLM calls by -57.1% (7 ➔ 3 turns), cumulative input tokens by -74.4% (23,322 ➔ 5,966 tokens), execution time by -48.0% (1.92x faster), and API cost by -59.3% ($0.0220 ➔ $0.0089).

Metric Mode A: Single Free-Form Loop (SingleLoop) Mode B: Stage-Gated Directed Graph (DirectedGraph) Delta / Improvement
Hidden Grading Suite Score (20 Cases) 20 / 20 (100%) 20 / 20 (100%) Identical 100% accuracy
LLM Calls (model_calls) 7 calls 3 calls -57.1% (4 orchestration turns eliminated)
Actual File Edits (file_edits) 3 edits 3 edits Identical coding work
Cumulative Input Tokens (promptTokenCount) 23,322 tokens 5,966 tokens -74.4% (~4x context compression)
Cumulative Output Tokens (candidatesTokenCount) 1,188 tokens 1,189 tokens Identical (+1 token)
Elapsed Execution Time (elapsed_seconds) 36.69 sec 19.07 sec -48.0% (1.92x faster)
Estimated API Cost (estimated_micro_usd) $0.0220 (21,950 micro-USD) $0.0089 (8,930 micro-USD) -59.3% cost reduction

2. Experiment Architecture: Single Loop vs. Directed Graph

+-------------------------------------------------------------------------+
| Proxmox Isolated VM208 (0 NICs, virtio-serial QGA, systemd UID split)   |
|  Target Task: 3-Module Order & Invoice Pipeline (6 Coupled Bugs)        |
|               (parser.py ➔ pricing.py ➔ report.py)                      |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: SingleLoopRunner (Traditional Free-Form ReAct Loop)       |  |
|  |  - Turn 1: inspect_all        (1,652 in /   8 out)                |  |
|  |  - Turn 2: run_tests [Fail]   (2,848 in /   8 out)                |  |
|  |  - Turn 3: edit parser.py     (2,883 in / 357 out)                |  |
|  |  - Turn 4: edit pricing.py    (3,324 in / 591 out)                |  |
|  |  - Turn 5: edit report.py     (4,000 in / 209 out)                |  |
|  |  - Turn 6: run_tests [Pass]   (4,297 in /   8 out)                |  |
|  |  - Turn 7: SUBMIT             (4,318 in /   7 out)                |  |
|  |  --> 7 calls | 23,322 in | 1,188 out | 36.69s | 20/20 Pass        |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: DirectedGraphRunner (Stage-Gated Graph + Context Reset)   |  |
|  |  - Node 1 [Fresh]: patch parser.py  + verify_parser  (1,947 in)   |  |
|  |  - Node 2 [Reset]: patch pricing.py + verify_pricing (2,174 in)   |  |
|  |  - Node 3 [Reset]: patch report.py  + verify_report  (1,845 in)   |  |
|  |  - Node 4 [Gate]:  Terminal Approval Gate (AUTO_SUBMIT, 0 tokens) |  |
|  |  --> 3 calls | 5,966 in  | 1,189 out | 19.07s | 20/20 Pass        |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+

The 6 Coupled Bugs Across the 3-Stage Pipeline

  1. Stage 1 (parser.py — parse_items):
    • Allowed qty == 0 and Python bool (isinstance(True, int) == True) instead of rejecting non-positive/boolean quantities with ValueError.
    • Failed to normalize SKU strings (.strip().upper()) and mutated caller input dictionaries in-place.
  2. Stage 2 (pricing.py — compute_invoice):
    • Ignored conflicting unit_price values for the same SKU instead of raising ValueError.
    • Applied WELCOME500 coupon before checking the volume discount and used > 10000 instead of inclusive >= 10000—wiping out the 10% volume discount on exact 10,000 orders (and silently ignoring invalid coupon codes).
    • Used Python banker's rounding (round()) instead of integer floor truncation (// 10) for discounts and tax.
  3. Stage 3 (report.py — format_summary):
    • Hardcoded item quantity as x1 and read the wrong key (invoice.get("discount", 0) instead of discount_total), omitting the mandatory signed zero format (DISCOUNT: -0).

3. Turn-by-Turn Token Anatomy: Where Did 74.4% of Input Tokens Go?

Inspecting all 10 recorded LLM calls (seq 0–6 for Mode A, seq 7–9 for Mode B) reveals why free-form loops waste tokens even when the model makes zero coding mistakes:

Mode Call Seq Node / Action Input Tokens Output Tokens Context State
Mode A 0 inspect_all 1,652 8 Spec only
Mode A 1 run_tests (initial fail) 2,848 8 + All 3 initial files appended
Mode A 2 edit_file (parser.py) 2,883 357 + Initial test failure trace appended
Mode A 3 edit_file (pricing.py) 3,324 591 + Full patched parser.py in history
Mode A 4 edit_file (report.py) 4,000 209 + Full patched pricing.py in history
Mode A 5 run_tests (green) 4,297 8 + Full patched report.py in history
Mode A 6 SUBMIT 4,318 7 2.6x larger than Turn 1 just to say SUBMIT
Mode A Total — 7 turns 23,322 1,188 Snowballing history across all 7 turns
Mode B 7 node_1_parser + auto-verify 1,947 366 Fresh context (parser.py only)
Mode B 8 node_2_pricing + auto-verify 2,174 600 Context reset (pricing.py only)
Mode B 9 node_3_report + auto-verify ➔ AUTO_SUBMIT 1,845 223 Context reset (report.py only)
Mode B Total — 3 turns (+ 0-token gate) 5,966 1,189 Flat ~2,000-token context on every node

Notice the smoking gun on Call 6: Mode A paid 4,318 input tokens just to emit a 7-token {"op": "SUBMIT"} JSON object. Meanwhile, Mode B kept every single node under 2,200 input tokens and transitioned to AUTO_SUBMIT deterministically in Python the instant verify_report_stage passed.

4. All 20 Hidden Grading Suite Results (labjudge04)

Both modes achieved 20 / 20 on the hidden evaluation suite executed by the isolated labjudge04 user:

Stage Test Case IDs Invariants Verified Mode A (SingleLoop) Mode B (DirectedGraph)
Stage 1 (parser.py) parser_01 to parser_06 Trim/uppercase SKU, non-mutation, reject []/bool/float/qty <= 0, allow unit_price == 0 6 / 6 PASS 6 / 6 PASS
Stage 2 (pricing.py) pricing_01 to pricing_08 First-seen SKU order, price conflict check, 9999 vs 10000 boundary, volume-before-coupon order, floor // 10, coupon cap, unknown coupon rejection 8 / 8 PASS 8 / 8 PASS
Stage 3 & E2E (report.py) e2e_01 to e2e_06 DISCOUNT: -0 signed zero, multi-SKU summary, free-item invoice, floor tax E2E, line order, cross-stage ValueError propagation 6 / 6 PASS 6 / 6 PASS
Total 20 Hidden Cases — 20 / 20 (100%) 20 / 20 (100%)

5. Practical Takeaway: Separate Reasoning (Loops) From Control Flow (Graphs)

  1. Don't ask an LLM to be a state machine: Let the LLM write patches inside small, bounded loops, and move deterministic transitions ("load file ➔ run stage test ➔ advance on green ➔ submit") into regular Python/TypeScript graph edges.
  2. Use deterministic stage verifiers as context compressors: Once a module's contract verifier passes, drop the raw editing history before moving to the next module. (Note: As we show in E05, only split into graph nodes when each stage has a deterministic code verifier—never split into role-based subagents passing lossy natural-language summaries!)

🔗 Continue Reading the Series

Comments

Join the discussion

No comments yet. Be the first to comment.

More from this blog