Trim CI Logs Before Sending Them to AI: Cut Debugging Tokens Without Hiding the Error

CI logs are among the worst inputs in an AI coding workflow: long enough to be expensive, repetitive enough to distract the model, and often dominated by lines that do not explain the failure. A deterministic reduction step can cut input tokens while keeping the command, first error, relevant stack, and nearby source evidence.

The goal is not aggressive summarization. A model-generated summary can accidentally remove the clue another model needs and adds another paid call. Start with transparent rules that preserve raw logs as an artifact and produce a smaller diagnostic view. The agent can request a specific omitted section if the first view is insufficient.

Why Raw Logs Inflate Context

Package installers repeat progress and dependency notices. Test runners print every passing test before one failure. Compilers report the same root error through dependent modules. Distributed systems add timestamps and request metadata to each line. Stack traces include framework internals that are identical across failures. Sending all of it increases input usage and can push the relevant source out of an agent's attention.

Long logs also compound in a loop. If a 60,000-token transcript is resent after each edit, five attempts may bill far more input than the source patch itself. Context caching can help when prefixes stay stable, but logs often change on every run and create new cache writes.

Preserve a Diagnostic Minimum

• The exact command, working directory, exit code, runtime, and environment identifiers.
• The first causal error, not only the final wrapper message.
• A bounded number of lines before and after the error for local context.
• Application-owned stack frames plus the boundary into a library or runtime.
• Counts and hashes for repeated messages, with one representative example.
• A pointer to the immutable full log so the agent or reviewer can retrieve more.

Apply Lossless Reductions First

Strip ANSI color codes, carriage-return progress frames, and duplicate blank lines. Collapse consecutive identical messages into one line plus a repeat count. Normalize timestamps and request IDs only when their precise values are irrelevant, while keeping a mapping in the raw artifact. Remove known success chatter after recording totals.

For stacks, keep all frames from your repository and a configurable number of surrounding external frames. Do not blindly remove node_modules or vendor paths: the root cause may be an API contract or dependency version. Instead, fold long runs of framework internals behind an expandable marker.

Use Progressive Disclosure

Give the agent a compact failure packet first. Expose tools such as get_raw_log_range, search_raw_log, and get_stack_group so it can retrieve evidence deliberately. Tool descriptions should state line limits and costs. This pattern keeps the common case cheap without creating a hard information ceiling for difficult failures.

Preserve stable identifiers across the compact and raw views. If the summary says ERROR_GROUP_3 occurred 417 times, the retrieval tool should accept that identifier. The agent then asks for a coherent group rather than guessing line offsets.

Measure Savings and Diagnostic Accuracy

Create a corpus of past CI failures with known root causes. Run the agent with raw logs and reduced packets. Compare input tokens, success rate, number of retrieval calls, time to a passing fix, and false diagnoses. A reduction that saves 70% of tokens but lowers fix success by 20 points is not an optimization. A packet that saves 50% with no material success change is.

Track reduction ratio by log source. Test output, compiler diagnostics, deployment logs, and browser traces need different rules. Review cases where the agent fetched the full log; they reveal what the compact representation omitted and guide safe improvement.

Version the reducer with every diagnostic packet. When a fix fails, reviewers need to know which transformation produced the view and be able to reproduce it from the raw artifact. Add golden tests for multiline compiler messages, nested exceptions, interleaved parallel output, Unicode, and truncated uploads. A reducer that corrupts boundaries can create confident but false diagnoses.

Set a hard packet budget and rank sections when it is exceeded. Keep causal errors and repository frames first, then representative warnings, then routine environment detail.

Security Improves Too

Logs may contain tokens, URLs, customer data, and environment values. Redact secrets before either compact or raw content reaches the model, and keep access to the original artifact scoped. Do not rely on truncation as redaction: a credential near the top survives a tail limit. Test reducers with seeded secrets and adversarial multiline values.

Log hygiene is a rare cost control that can also improve signal and reduce exposure. Keep the raw evidence, create a deterministic compact view, and let the agent expand only what it needs. Model the before-and-after input volume with our AI Cost Calculator.