Use Diff Size to Predict AI Coding Cost Without Rewarding Tiny Patches

Diff size is an observable early signal for AI coding cost: larger changes often require more repository reading, generation, tests, and review. It is not a quality target. If teams reward fewer changed lines, agents can hide complexity in dense edits or avoid necessary cleanup. Use diff size for forecasting, then judge value by accepted outcomes.

Collect added, deleted, and modified lines; files and directories touched; generated versus handwritten code; language; task class; model; input and output tokens; tool calls; test runtime; review minutes; retries; and acceptance. Exclude lockfiles, vendored code, snapshots, and formatter-only changes or model them separately. Their line counts behave differently.

Find the Useful Relationship

Plot tokens and review time against log-scaled diff size, file count, and directory spread. Segment bug fixes, features, refactors, migrations, tests, and documentation. A 30-line authentication change may cost more reasoning than a 500-line generated fixture. Report prediction intervals, not one price per line.

Measure repository context separately from output. Small diffs in unfamiliar areas can require reading many files. Track bytes or tokens inspected, searches, and dependency hops. This explains why two equal diffs produce different invoices and helps improve indexing or task scoping.

Build a Forecast, Not a Quota

Before work, estimate likely file count, task class, test suite, risk, and expected diff band. Produce low, expected, and high cost ranges. Update the forecast after planning when the agent has inspected the codebase. The change between forecasts is useful: rapid scope growth can trigger a checkpoint before spend accelerates.

Never stop a task solely because lines exceed the estimate. Ask whether scope, architecture, generated artifacts, or accidental rewrites explain the growth. The safe response may be splitting work, not forcing a compressed patch. Keep budget controls tied to tokens, time, and consequence.

Detect Expensive Editing Patterns

Compare targeted edits with full-file rewrites, repeated formatter churn, regenerated files, and oscillating changes. Track cumulative lines touched across iterations, not only the final diff. An agent may rewrite a file three times and finish with ten changed lines. Cumulative churn reveals the hidden token and test cost.

Attach retry reasons: failed tests, misunderstood requirements, merge conflicts, style checks, or reviewer requests. Fix recurring causes upstream. Better instructions, repository maps, or local validation can lower cost without pressuring the agent to produce artificially small changes.

Include Quality and Review

Normalize by accepted, defect-free changes. Track escaped defects, rollback, review rounds, and time to merge. A larger patch that completes a migration cleanly can be cheaper than five tiny dependent patches. Conversely, large mixed-purpose diffs raise review and rollback cost even when model tokens are modest.

Use directory risk weights. Changes to payments, authentication, infrastructure, or migrations need more tests and review than copy updates. The same diff size should produce different expected total cost because the required assurance differs.

Calibrate Monthly

Retrain or recalibrate when models, agents, repositories, or review practices change. Preserve old coefficients so trend shifts are visible. Validate on later tasks, not the same data used to fit the forecast. Monitor systematic underestimation for particular teams or languages.

Diff size is a useful feature when it stays one feature. Combine it with task shape, context read, risk, tests, and cumulative churn. Forecast ranges, investigate deviations, and optimize cost per accepted change. That supports planning without turning minimal line count into a damaging engineering incentive.

Pilot the Forecast With Advisory Alerts

Run the model for a month without blocking tasks. Show the agent and operator an expected cost band after planning and an updated band when file scope changes. Record whether the alert caused a useful rescope, unnecessary interruption, or no action. Advisory deployment exposes bad assumptions before a forecast becomes policy.

Review the largest under- and over-predictions. Underestimates often reveal hidden repository reading, flaky tests, or cross-cutting dependencies; overestimates may reveal generated code or strong local tooling. Add a feature only when it improves validation data and remains explainable enough for engineers to challenge.

After calibration, alerts can request a checkpoint when expected cost or scope crosses a threshold, but they should never force unsafe truncation. Preserve the user's objective and required quality gates. The forecast earns trust by helping teams plan capacity and investigate churn, not by pressuring every change toward a smaller visible diff.

Make predictions visible in aggregate, not as a developer ranking. Repository age, test architecture, ownership boundaries, and task assignment can create large differences outside individual control. Use the model to improve planning and tooling. If teams fear punishment for a large forecast, they will split, relabel, or understate work, degrading the data and engineering outcome the predictor was meant to improve.