Agent Context Compaction Cost: Measure Summary Drift Before Tokens Saved

Context compaction can lower the cost of long coding-agent sessions by replacing old turns with a smaller summary. The hidden tax is summary drift: a lost constraint, rejected approach, file boundary, test result, or user decision that forces rework later. Savings count only after recovery and quality costs are included.

Model one session as pre-compaction tokens, compaction tokens, post-compaction tokens, and recovery tokens. Add wall time, tool calls, and human correction. Compare with an uncompressed replay or a checkpointed control. A fifty-percent reduction in repeated context is not a fifty-percent task saving when the agent later reopens files, repeats searches, or implements an invalidated design.

Define What Must Survive

Create a durable-state schema for objective, acceptance criteria, forbidden actions, approved plan, changed files, unresolved questions, commands run, test results, external facts with sources, and decisions with reasons. Store exact identifiers and paths rather than vague prose. Keep security and destructive-action limits outside a lossy summary when the runtime can enforce them deterministically.

Separate observations from conclusions. “Test X failed with error Y at commit Z” is durable evidence; “the feature is broken” is an interpretation that may become stale. Preserve the smallest raw facts needed to revisit an inference. Mark uncertainty instead of compressing it into confidence.

Choose Compaction Triggers

Trigger on measured token pressure, phase boundaries, or stable checkpoints rather than every fixed number of turns. Planning to implementation, implementation to verification, and verification to publishing are natural boundaries. Compact only after writing current state and confirming that worktree, plan, and task records agree.

Leave headroom for recovery. Compacting at the absolute context limit makes it harder to compare the summary with recent evidence or repair omissions. Track compaction frequency and summary size. Repeated compaction of already compressed text can amplify drift, so refresh important state from authoritative files and tools.

Build Drift Tests

Replay representative long sessions and ask the post-compaction agent to state the objective, constraints, files changed, latest test state, rejected alternatives, and next safe action. Score exact facts, not writing similarity. Then continue the task and compare completion, defects, repeated tool calls, and human corrections with the control.

Include adversarial cases: two similar filenames, a decision reversed late, a failed command followed by a successful one, an instruction embedded in untrusted content, and a price or date with a source. These reveal whether the summary preserves provenance and precedence instead of blending old and new state.

Calculate Break-Even

Value saved input tokens at the actual model and service tier. Subtract compaction-call cost, extra storage, evaluation, repeated tools, recovery tokens, and human minutes. Segment by task length: short sessions may never recover the compaction overhead, while migrations and investigations can benefit substantially.

Use cost per accepted task and defect-adjusted completion, not average tokens alone. Set a maximum drift rate for critical constraints and a lower threshold for convenience details. Stop or revise compaction when a quality threshold is crossed even if token graphs look attractive.

Make Recovery Explicit

When confidence is low, reload authoritative files, git state, task plans, and test output. Do not reconstruct them from summary prose. Attach stable references to the compacted state and invalidate it when underlying files change. Log which facts were restored so recovery cost is attributable.

Good compaction is a state-management system, not merely a shorter prompt. Preserve hard constraints and provenance, compact at verified boundaries, test drift with real trajectories, and include recovery in the ledger. The winning policy is the one that lowers cost per correct completed task while keeping important decisions intact.

Operate a Compaction Scorecard

For each agent and task class, report compactions per session, tokens before and after, summary-generation cost, exact-state recall, repeated tool calls, recovery turns, defects, and human correction. Compare short, medium, and long sessions separately. A healthy average can hide severe drift in rare high-consequence work.

Review failed trajectories with the summary and authoritative state side by side. Classify omissions as objective, constraint, provenance, file state, test state, or decision history. Fix the durable-state schema or trigger, then replay the same case. Do not tune only the summary prompt when the runtime failed to preserve a fact that should never have been lossy.

Set rollback criteria before launch: critical-constraint recall below the threshold, repeated destructive intent, or defect-adjusted cost above the control. A policy that saves input tokens but increases review uncertainty is not ready for unattended use. The scorecard keeps optimization anchored to reliable completion rather than smaller context graphs.

Assign one owner to the policy and another to sample completed sessions. Independent sampling catches summaries that appear internally consistent while contradicting the repository. Keep the original trajectory long enough for investigation under an appropriate retention policy, then remove sensitive material on schedule. Compaction should reduce active context, not create an unbounded archive with a different cost and privacy problem.