# Preventing Multi-Session Amnesia in Coding Agents: Prose Summaries vs. a 3-Artifact Shift-Handover Harness (-95.2% Input Tokens) > πŸ’‘ **AI Labo Benchmark Series β€” Experiment `E36` (Multi-Session Continuity & State Handover)** > > This post is part of our empirical benchmark series testing AI coding agent harness patterns inside network-isolated Proxmox VMs against a hidden 20-case grading suite. > * πŸš€ **[View All 41 Benchmark Results & Master Ranking](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark)** > * πŸ›‘οΈ **[Read How Our Proxmox & systemd Anti-Cheating Sandbox Works](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox)** --- ## Metadata & Experiment Identity * **Experiment ID:** `E36` (Backlog Candidate `C032`) * **Run ID:** `e36-20261029-vm240-01` * **Sandbox Environment:** Proxmox Isolated VM `VM240` (`ai-e36-240`, 0 NICs, `PrivateNetwork=yes`, `NoNewPrivileges=yes`, `InaccessiblePaths`, UID split `labexp36` / `labjudge36`) * **Model:** `gemini-3.8-flash` (`thinkingLevel: LOW`, `temperature=0.0`) * **Pattern Under Test:** *"Anthropic: Effective Harnesses for Long-Running Agents (`feature_list.json` + `init.sh` + `progress.txt` Shift-Handover Protocol vs. Unstructured Prose Summary)"* β€” **Unstructured Summary Handoff (`UnstructuredSummaryHandoffRunner`) vs. 3-Artifact Shift-Handover Harness (`ShiftHandoverProtocolRunner`)** --- ## 1. Executive Summary (TL;DR) When a long-running coding task spans multiple context resets ("shifts"), handing the next agent session an **unstructured prose chat summary** (*"Most of the payment pipeline refactoring is done; just finish up the remaining checks"*) triggers two failure modes documented in Anthropic's *Effective Harnesses for Long-Running Agents*: **Feature Amnesia** (forgetting subtle edge-case contracts) and **Premature Victory** (declaring the task complete while boundary tests are still failing). Replacing prose handoffs with a deterministic **3-Artifact Shift-Handover Harness**β€”combining a machine-readable contract checklist (`feature_list.json`), an executable verification gate (`init.sh`), and a structured progress log (`progress.txt`)β€”gives the incoming session exact ground truth from Turn 1. We tested this head-to-head inside isolated VM `VM240` on the 3-module enterprise payment settlement pipeline (`validator.py` βž” `settlement.py` βž” `statement.py`): * **Mode A (`UnstructuredSummaryHandoffRunner`):** Receives an unstructured prose handoff summary across 74 unverified checklist paragraphs (`73,841` chars of amnesia rework logs across Turns 1–2), overlooking the 5-key dictionary stripping invariant (`clean_item = dict(item)` leaks `"extra"`) and finishing at **`19 / 20` (95.0%)**. * **Mode B (`ShiftHandoverProtocolRunner`):** Enforces the **3-Artifact Shift-Handover Protocol** (`SHIFT_HANDOVER_HARNESS_MD` + `submit_handover_verified_bundle`), solving **`20 / 20` (100%) in 1 turn**. ### Key Result Mode A failed `validator_01_normalize_and_non_mutation` (`19 / 20`) after 3 turns (`46,227` cumulative input tokens). **Mode B achieved a `20 / 20` (100%) pass in 1 turn**, cutting **LLM calls by `-66.7%` (`3 βž” 1`)**, **cumulative input tokens by `-95.2%` (`46,227 βž” 2,237` tokensβ€”a `20.66x` reduction)**, **API cost by `-83.1%` (`$0.0454 βž” $0.0077`)**, and running **`2.27x` faster (`28.95s βž” 12.75s`, `-55.9%`)**. | Metric | Mode A: Prose Summary Handoff (`UnstructuredSummaryHandoff`) | Mode B: 3-Artifact Shift Handover (`ShiftHandoverProtocol`) | Delta / Improvement | | :--- | :---: | :---: | :---: | | **Hidden Grading Suite Score (`labjudge36`)** | `19 / 20` (95.0% β€” 1 failure) | **`20 / 20` (100% β€” Perfect)** | **+5.0% accuracy (`19/20 βž” 20/20`)** | | **Feature-Amnesia Traps (`gotcha_traps_hit`)** | `2 traps` (Turns 1 & 2) | **`0 traps`** | **-100% (zero amnesia)** | | **LLM Calls (`model_calls`)** | `3 calls` | **`1 call` (1-shot)** | **-66.7% (`3.00x` faster convergence)** | | **Cumulative File Edits (`file_edits`)** | `5 edits` | **`3 edits` (1 per module)** | **-40.0% fewer edits** | | **Cumulative Tool Output (`tool_output_chars`)** | `73,841 chars` | **`100 chars`** | **-99.9% (`738.4x` noise reduction)** | | **Turn 1 Tokens (In / Out)** | `1,665 in` / `13 out` | **`2,237 in` / `1,597 out`** | `+572` in for 3-artifact handover | | **Turn 2 Tokens (In / Out)** | `15,029 in` / `1,258 out` | *β€” (completed on Turn 1)* | Avoided prose handoff drift logs | | **Turn 3 Tokens (In / Out)** | `29,533 in` / `1,591 out` | *β€” (completed on Turn 1)* | Avoided secondary rework bloat | | **Cumulative Input Tokens (`promptTokenCount`)** | `46,227 tokens` | **`2,237 tokens`** | **-95.2% (`20.66x` efficiency)** | | **Cumulative Output Tokens (`candidatesTokenCount`)** | `2,862 tokens` | **`1,597 tokens`** | **-44.2% output reduction** | | **Elapsed Execution Time (`elapsed_seconds`)** | `28.95 sec` | **`12.75 sec`** | **-55.9% (`2.27x` faster)** | | **Estimated API Cost (`estimated_micro_usd`)** | `$0.0454` (`45,404 micro-USD`) | **`$0.0077` (`7,667 micro-USD`)** | **-83.1% (`5.92x` cheaper)** | --- ## 2. Experiment Architecture: Prose Summary vs. 3-Artifact Shift-Handover Harness ```text +-------------------------------------------------------------------------+ | Proxmox Isolated VM240 (0 NICs, InaccessiblePaths, grader labjudge36) | | Target Task: 3-Module Enterprise Payment Settlement Pipeline | | validator.py βž” settlement.py βž” statement.py | | | | +-------------------------------------------------------------------+ | | | Mode A: UnstructuredSummaryHandoffRunner (Prose Chat Summary) | | | | - Turn 1 (1,665 in): Calls `read_unstructured_shift_summary` | | | | βž” 36,932 chars of unverified prose notes | | | | - Turn 2 (15,029 in): Calls `stage_amnesia_partial_patch` | | | | βž” partial 2-file patch (36,586 chars) | | | | - Turn 3 (29,533 in): Overlooks 5-key dict stripping (`clean_item| | | | = dict(item)` leaks `"extra"` key) | | | | --> 3 calls | 46,227 in | $0.0454 | 28.95s | 19/20 (1 FAILED) | | | +-------------------------------------------------------------------+ | | | | +-------------------------------------------------------------------+ | | | Mode B: ShiftHandoverProtocolRunner (3-Artifact Mechanical State) | | | | - Injects `SHIFT_HANDOVER_HARNESS_MD` (+572 tokens): | | | | * `feature_list.json`: Boolean verification status per contract| | | | * `init.sh`: Deterministic diagnostic error keys | | | | * `progress.txt`: Scoped module pointers (`[SPEC-VAL/SET/STM]`)| | | | - Turn 1 (2,237 in): Calls `submit_handover_verified_bundle` | | | | --> 1 call | 2,237 in | $0.0077 | 12.75s | 20/20 (100% PASS) | | | +-------------------------------------------------------------------+ | +-------------------------------------------------------------------------+ ``` --- ## 3. Drop-In Implementation Recipe: `build_shift_handover_payload` Never ask an outgoing LLM session to write a free-form essay for the next session. Generate a deterministic handover payload by running `verify_fn()` (`init.sh`) against `feature_list.json`: ```python import json from typing import Callable def build_shift_handover_payload(features: list[dict], verify_fn: Callable[[], list[str]]) -> str: """Builds a mechanical shift-handover payload combining feature_list.json and init.sh diagnostics.""" active_errors = verify_fn() handover_state = { "protocol": "SHIFT_HANDOVER_V1", "init_sh_status": "PASS" if not active_errors else "FAIL", "active_diagnostic_errors": active_errors, "feature_list": [ { "id": f["id"], "module": f["module"], "contract": f["contract"], "verified": f["error_key"] not in active_errors, } for f in features ], } return ( "# SHIFT-HANDOVER HARNESS STATE (feature_list.json + init.sh)\n```json\n" + json.dumps(handover_state, indent=2, ensure_ascii=False) + "\n```" ) ``` --- ## 4. Practical Takeaway: Ground Every New Session in Executable State, Not Prose 1. **Compute `"verified": true/false` mechanically via `init.sh` at session boot:** Never let an agent mark a feature as done in prose without an automated check flipping the flag in `feature_list.json`. 2. **Pair the Shift-Handover Harness (`E36`) with Milestone Context Compaction (`E12`):** Compact completed milestones into a clean git commit and boot the next stage with the 3-artifact handover state. --- ### πŸ”— Continue Reading the Series * πŸš€ **Master Benchmark Hub (All 41 Patterns Ranked):** [I Benchmarked 41 AI Agent Harness Patterns in Isolated VMs: How to Cut Tokens by 90% Without Losing Accuracy](https://tappers.hashnode.dev/ai-agent-harness-patterns-benchmark) * πŸ›‘οΈ **Methodology & Sandbox Architecture:** [How to Benchmark AI Agents Without Eval Cheating or API Key Leaks: Designing a Proxmox & systemd Sandbox](https://tappers.hashnode.dev/air-gapped-proxmox-systemd-agent-sandbox) * πŸ“– **Previous Sequential Post (`E35`):** [Why a 500-Line AGENTS.md Hurts Coding Agent Accuracy: Monolithic Encyclopedia Prompts vs. a 30-Line Directory-Pointer Index (3.13x Faster)](https://tappers.hashnode.dev/e35-directory-pointer-agents-md) * πŸͺœ **Next Sequential Post (`E37`):** [Stopping Blind Micro-Retries on Identical Errors: Death-Spiral Thrashing vs. a 3-Hypothesis Structural Pivot Ladder (-95.2% Input Tokens)](https://tappers.hashnode.dev/e37-hypothesis-pivot-ladder) * πŸ—œοΈ **Related Context Compaction Deep Dive (`E12`):** [How Save-Game Context Compaction Fixes Spec Drift in Multi-Step Coding Agents (-91.9% Input Tokens)](https://tappers.hashnode.dev/e12-milestone-context-compaction)