# Cross-Model Giant Killing: How 1/8th-Cost Flash-Lite + Golden Harnesses Beats Unharnessed Frontier Pro (-98.1% Cost, 6.95x Faster)

> 💡 **AI Labo Benchmark Series — Experiment `E43` (Cross-Model Scaling & Giant Killing)**
>
> This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite.
> * 🚀 **[View All Benchmark Results & Master Ranking](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)**
> * 🛡️ **[Read How Our Proxmox & systemd Anti-Cheating Sandbox Works](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)**

---

## Metadata & Experiment Identity
* **Experiment ID:** `E43` (Backlog Candidate `C039`)
* **Run ID:** `e43-20261030-vm247-01`
* **Sandbox Environment:** Proxmox Isolated VM `VM247` (`ai-e43-247`, 0 NICs, `PrivateNetwork=yes`, `NoNewPrivileges=yes`, `InaccessiblePaths`, UID split `labexp40` / `labjudge43`)
* **Models Tested:** Mode A: **`gemini-3.1-pro-preview`** (`$2.00 / $12.00` per 1M tokens) vs. Mode B: **`gemini-3.1-flash-lite`** (`$0.25 / $1.50` per 1M tokens — **1/8th unit price**)
* **Pattern Under Test:** *"AI Labo Scaling Law & Harness Engineering Meta-Analysis: Does a Lightweight Model Equipped with Golden Harnesses Outperform an Unharnessed Frontier Model?"* — **Unharnessed Frontier Pro (`UnharnessedFrontierProRunner`) vs. Harnessed Flash-Lite Giant Killing (`HarnessedFlashLiteGiantKillingRunner`)**

---

## 1. Executive Summary (TL;DR)

When an AI coding agent underperforms in production, the most common instinct is to **throw a more expensive frontier model at the problem**—upgrading from a fast lightweight tier (`$0.25 / $1.50` per 1M tokens) to an 8x more expensive Pro/Frontier tier (`$2.00 / $12.00` per 1M tokens) while keeping the exact same unguarded, noisy agent loop.

To test whether **Harness Engineering beats raw model price**, we pitted an **unharnessed frontier model (`gemini-3.1-pro-preview`)** in a raw loop against **1/8th-cost `gemini-3.1-flash-lite`** equipped with our combined **Golden Harnesses** (`E09` AST Map + `E11` Pre-Commit Hook + `E16` Output Token Killer + `E20` `MEMORY.md` + `E21` Tree-sitter Skeletons + `E39` Guides & Sensors) inside isolated VM `VM247` on the 3-module enterprise payment settlement pipeline (`validator.py` ➔ `settlement.py` ➔ `statement.py`):

* **Mode A (`UnharnessedFrontierProRunner` — `gemini-3.1-pro-preview`):** Runs without harness Guides or Sensors, ingesting `73,962` chars of raw terminal logs across 3 turns (`48,097` input tokens, `1,909` thinking tokens, `39.10s`, `$0.1538`), and **stalls at `15 / 20` (75.0%)** after missing `extra` key stripping (`new_item = dict(item)`) and E2E statement formatting/ordering contracts.
* **Mode B (`HarnessedFlashLiteGiantKillingRunner` — `gemini-3.1-flash-lite`):** Runs the 1/8th-price lightweight model inside the **Golden Guides & Sensors Harness** (`CROSS_MODEL_GIANT_KILLING_HARNESS_MD` + `flash_lite_harnessed_submit`), achieving **`20 / 20` (100% pass) in 1 turn (`5.62s`, `$0.0029`)**.

### Key Result: A `52.6x` Cost Reduction + `+25.0%` Accuracy Gain
Equipped with the Golden Harness, **1/8th-cost `gemini-3.1-flash-lite` beat unharnessed `gemini-3.1-pro-preview` by `+25.0` percentage points (`20/20` vs. `15/20`)**, while cutting **cumulative input tokens by `-95.2%` (`48,097 ➔ 2,321`)**, **execution time by `-85.6%` (`39.10s ➔ 5.62s` — `6.95x` faster)**, and **API cost by `-98.1%` (`$0.1538 ➔ $0.0029` — `52.60x` cheaper)**.

| Metric | Mode A: Unharnessed Frontier Pro (`gemini-3.1-pro-preview`) | Mode B: Harnessed Flash-Lite (`gemini-3.1-flash-lite` + Golden Harness) | Delta / Improvement |
| :--- | :---: | :---: | :---: |
| **Model Unit Pricing (In / Out per 1M)** | `$2.00` in / `$12.00` out | **`$0.25` in / `$1.50` out** | **1/8th (`87.5%` lower) unit price** |
| **Hidden Grading Suite Score (`labjudge43`)** | `15 / 20` (75.0% — 5 failures) | **`20 / 20` (100% — Perfect)** | **+25.0% accuracy (`+5` cases)** |
| **Rework Traps Hit (`gotcha_traps_hit`)** | `2 traps` (Turns 1 & 2) | **`0 traps`** | **-100% (zero rework)** |
| **LLM Calls (`model_calls`)** | `3 calls` | **`1 call` (1-shot)** | **-66.7% (`3.00x` faster convergence)** |
| **Cumulative Tool Output (`tool_output_chars`)** | `73,962 chars` | **`106 chars`** | **-99.9% (`697.8x` noise reduction)** |
| **Turn 1 Tokens (In / Out / Thinking)** | `1,675 in` / `20 out` / `43 think` | **`2,321 in` / `1,421 out` / `141 think`** | 1-shot completion on Turn 1 |
| **Turn 2 Tokens (In / Out / Thinking)** | `15,651 in` / `1,278 out` / `811 think` | *— (completed on Turn 1)* | Avoided 37K-char raw log flood |
| **Turn 3 Tokens (In / Out / Thinking)** | `30,771 in` / `1,594 out` / `1,055 think` | *— (completed on Turn 1)* | Avoided secondary rework bloat |
| **Cumulative Input Tokens (`promptTokenCount`)** | `48,097 tokens` | **`2,321 tokens`** | **-95.2% (`20.72x` efficiency)** |
| **Cumulative Output Tokens (`candidatesTokenCount`)** | `2,892 tokens` | **`1,421 tokens`** | **-50.9% output reduction** |
| **Elapsed Execution Time (`elapsed_seconds`)** | `39.10 sec` | **`5.62 sec`** | **-85.6% (`6.95x` faster)** |
| **Estimated API Cost (`estimated_micro_usd`)** | `$0.1538` (`153,806 micro-USD`) | **`$0.0029` (`2,924 micro-USD`)** | **-98.1% (`52.60x` cheaper)** |

---

## 2. Experiment Architecture: Unharnessed Frontier Pro vs. Harnessed Flash-Lite

```text
+-------------------------------------------------------------------------+
| Proxmox Isolated VM247 (0 NICs, InaccessiblePaths, grader labjudge43)   |
|  Target Task: 3-Module Enterprise Payment Settlement Pipeline           |
|               validator.py ➔ settlement.py ➔ statement.py               |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode A: UnharnessedFrontierProRunner (`gemini-3.1-pro-preview`)   |  |
|  |  - Turn 1 (1,675 in):  Calls `pro_unharnessed_raw_probe`          |  |
|  |                        ➔ 37,154 chars of unfiltered audit logs    |  |
|  |  - Turn 2 (15,651 in): Calls `pro_stage_unverified_patch`         |  |
|  |                        ➔ partial 2-file edit (36,808 chars)       |  |
|  |  - Turn 3 (30,771 in): Fails `validator_01` (`dict(item)` leak)   |  |
|  |                        and 4 E2E statement cases (`15/20`)        |  |
|  |  --> 3 calls | 48,097 in | $0.1538 | 39.10s | 15/20 (5 FAILED)    |  |
|  +-------------------------------------------------------------------+  |
|                                                                         |
|  +-------------------------------------------------------------------+  |
|  | Mode B: HarnessedFlashLiteGiantKillingRunner (`3.1-flash-lite`)   |  |
|  |  - Equips 1/8th-cost `gemini-3.1-flash-lite` with Golden Harness: |  |
|  |    * Guides (`E09` + `E20` + `E21`): `[SPEC-VAL/SET/STM]`         |  |
|  |    * Sensors (`E11` + `E16` + `E39`): Deterministic pre-submit    |  |
|  |      contract & format verification                               |  |
|  |  - Turn 1 (2,321 in): Solves all 20/20 cases in 5.62 seconds      |  |
|  |  --> 1 call  | 2,321 in  | $0.0029 | 5.62s  | 20/20 (100% PASS)   |  |
|  +-------------------------------------------------------------------+  |
+-------------------------------------------------------------------------+
```

---

## 3. Drop-In Implementation Recipe: `run_giant_killing_harness`

Instead of routing every repository task to a slow, expensive frontier model in a raw loop, wrap your fast lightweight model in a deterministic Guides & Sensors harness:

```python
def run_giant_killing_harness(
    task_files: dict[str, str], scoped_guides: str, call_flash_lite_fn, sensor_fn
) -> dict:
    """Pairs 1/8th-cost gemini-3.1-flash-lite with scoped Guides and deterministic Sensors."""
    prompt = (
        "# GUIDES & SENSORS HARNESS FOR FLASH-LITE\n"
        f"{scoped_guides}\n\n"
        "# WORKSPACE FILES\n"
        + "\n".join(f"### {k}\n```python\n{v}\n```" for k, v in sorted(task_files.items()))
    )
    candidate_files = call_flash_lite_fn(model="gemini-3.1-flash-lite", prompt=prompt)
    sensor_errors = sensor_fn(candidate_files)
    return {
        "model": "gemini-3.1-flash-lite",
        "sensor_passed": len(sensor_errors) == 0,
        "errors": sensor_errors,
        "files": candidate_files,
    }
```

---

## 4. Practical Takeaway: Invest in Your Harness Before Upgrading Your Model Tier

1. **A frontier model in a bad harness loses to a small model in a great harness:** Raw terminal noise (`73,962` chars) and undocumented repository gotchas degrade even top-tier models (`15 / 20`).
2. **Use Advisor-Executor Routing (`E18`) when frontier reasoning is genuinely needed:** Consult a frontier model once on Turn 1 to produce a concise architectural blueprint (`E18`), then execute all file edits and verification loops on `flash-lite` / `flash` inside the Golden Harness (`E43`).

---

### 🔗 Continue Reading the Series
* 🚀 **Master Benchmark Hub:** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)
* 🛡️ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)
* 🧹 **Previous Sequential Post (`E42`):** [Why 'Stop-at-Green' Accumulates Technical Debt: Immediate Commit on Green vs. a Post-Green Entropy Sweep Pass (2.34x Faster)](https://tappers.hashnode.dev/e42-post-green-entropy-sweep)
* 🧬 **Next Sequential Post (`E44`):** [Which Harnesses Survive Model Scaling? Syntax Scaffolding vs. the 4 Irreplaceable Core Harnesses (-95.5% Input Tokens)](https://tappers.hashnode.dev/e44-model-evolution-harness-boundary)
* 🧭 **Related Advisor-Executor Routing Deep Dive (`E18`):** [Benchmarking the Claude Code `/advisor` Pattern: Solo Executor Loops vs. Upfront Advisor Blueprint Routing](https://tappers.hashnode.dev/e18-advisor-executor-routing)
