Agent Tool Schema Validation: The Cheapest Invalid Call Is the One Never Sent

Invalid tool calls waste more than one API request. They consume model context, tool latency, retry tokens, sandbox time, rate-limit capacity, and reviewer attention. A strict, versioned schema with deterministic local validation is often one of the highest-return cost controls in a coding-agent system.

Begin with the call path: the model selects a tool, generates arguments, the runtime validates them, authorization checks the requested action, the tool executes, and the result re-enters context. Instrument each boundary. If telemetry records only the final error string, teams cannot tell whether the schema, prompt, runtime, permissions, or external API caused the retry.

Design for Constrained Choices

Use enums for finite operations, explicit required fields, bounded numbers, stable identifiers, and separate read from write tools. Avoid a generic command string when structured fields can express repository, branch, resource, and intended mutation. Reject unknown fields so a model cannot accidentally rely on ignored arguments.

Put semantic rules near the schema: end time must follow start time, production requires an approval token, paths must stay inside the workspace, and destructive operations need exact targets. JSON validity alone does not make a call safe or meaningful.

Calculate the Invalid-Call Tax

For each rejected call, record input and output tokens, tool request cost, elapsed time, correction turns, and whether the task eventually succeeded. If 50,000 monthly tasks average 0.3 invalid calls, each costing $0.04 in model and infrastructure work, direct waste is $600. Ten seconds of added developer wait per task adds 139 hours of latency, which may matter more than the invoice.

Segment by tool, schema version, model, agent release, and error code. Averages hide one tool that causes most failures. Rank fixes by total wasted cost and consequence, not simply rejection count. A rare malformed production call deserves more attention than frequent harmless search typos.

Repair Without Paying for Guessing

Return concise structured errors containing the field, rule, expected type, and safe next action. Do not echo large schemas or raw secrets. Where correction is deterministic, normalize it in code: trim known whitespace, canonicalize a repository identifier, or convert an accepted date format. Never auto-correct a target or permission in a way that changes consequence.

Allow one or two bounded repair attempts based on error class. Repeating the same invalid request should stop the task or escalate. Preserve the full chain under one task ID so a dashboard can attribute correction cost to the original call.

Test the Schema as an API

• Generate valid boundary cases for every operation and optional field.
• Test missing, unknown, wrong-type, oversized, and contradictory inputs.
• Fuzz paths, URLs, shell-like strings, Unicode, and nested payloads.
• Verify that invalid write calls cause no external side effect.

Replay representative agent trajectories against schema changes. A stricter version may prevent unsafe calls but raise refusal or repair cost. Compare accepted-task rate, invalid calls, latency, and side effects. Contract tests should cover both the model-facing description and runtime validator.

Version and Migrate

Publish schema versions and deprecation dates. Keep old versions only long enough for controlled migration, and tag usage. If a tool changes silently, cached prompts and long-running agents may continue generating obsolete fields. A compatibility adapter can be cheaper than immediate retraining, but it needs metrics and an expiry.

Use cost per valid side-effecting call and cost per accepted task as primary units. Track invalid-call rate, first-repair success, p95 correction latency, and unsafe-attempt rate. Review after tool, model, or prompt changes.

Account for Human Time

Schema work consumes design and maintenance labor. Price that effort against avoided retries and reduced incident exposure. A two-day schema refactor costing $2,000 breaks even quickly if it removes $600 of monthly machine waste and recurring support tickets. Keep the assumptions visible.

Do not optimize for zero rejections by accepting arbitrary input. Rejection is a valuable control when it is early, precise, and cheap. The goal is fewer ambiguous attempts and faster safe correction.

Create an Error Budget

Set a maximum invalid-call rate and p95 correction latency for each tool class. Read-only discovery can tolerate more harmless correction than production mutation. When the budget is exceeded, stop adding tool features and inspect the dominant schema, model, or prompt errors. This keeps invalid calls from becoming accepted background noise.

Review a weekly sample of successful calls too. A permissive schema may make rejection metrics look excellent while accepting ambiguous targets or ignored fields. The sample should confirm that the recorded arguments matched the user's intent and that the external effect matched the arguments. Pair rejection data with semantic correctness.

Bottom Line

Constrain tool choices, validate semantics locally, return structured errors, bound repairs, and version contracts. This shifts failure from expensive external execution into cheap deterministic checks. The result is not merely cleaner JSON; it is lower cost and lower consequence per completed coding-agent task.