Can You Tell If Your API Provider Swapped Your Model? Behavioral Fingerprinting Explained

You pay for Opus 4.8 at $5/M in and $25/M out. But if you buy that access through an aggregator, a reseller, or an "OpenAI-compatible" gateway, how do you actually know the tokens you are billed for came from the model on the invoice? A July 2026 paper argues you can check, cheaply, with a trick as simple as asking the model to pick a random number. For anyone budgeting AI coding spend through third-party relays, this is a direct trust-and-cost problem.

The Fingerprint Trick

The paper, "One Token Is Enough: Fingerprinting and Verifying Large Language Models" by Tomas Bruckner of the Prague University of Economics (arXiv 2607.10252, July 2026), starts from a quirk every LLM has: they are terrible at being random. Ask a model "name a random number between 1 and 100" enough times and the answers are not uniform. They cluster. That empirical distribution of answers is stable and distinctive enough to act as a behavioral fingerprint, unique per model.

Across 165 models tested with about 30 queries each, the tells were stark. GPT-4o skews toward 42 and 37. Claude Sonnet 5 heavily favors 47. Qwen3-Max answered 42 all 30 times. Because these biases come from training, not configuration, a provider cannot easily hide them without changing the underlying model, which is exactly the thing you want to detect.

Why This Is a Cost Problem, Not Just a Curiosity

The economic incentive to swap is real. A premium model like Opus 4.8 costs a provider far more to serve than a mid-tier or budget model. If a relay quietly routes your "Opus 4.8" request to something cheaper to run, you keep paying the premium rate while the provider pockets the spread. You would rarely notice from output quality alone on routine coding tasks, because a cheaper model often produces plausible-looking code. What you lose is the reasoning depth and reliability you specifically paid extra for, and on hard tasks that gap becomes rework, wasted retries, and blown budgets.

This is not an accusation against every relay. Plenty are honest. But "trust us" is not an audit, and until now verifying model identity felt like a research problem. The fingerprint method makes it a cheap, routine check.

The Cost of Checking Is Negligible

The paper reports that around 120 requests are enough to identify a model's identity, at an error rate of roughly 10.6%. A verification query is tiny, a few tokens of prompt and a few tokens of answer. Put that against the reference rates:

• 120 requests at, say, 20 input tokens and 5 output tokens each is 2,400 input and 600 output tokens total.
• At Opus 4.8 rates (in $5/M, out $25/M) that is about $0.012 in input plus $0.015 in output, roughly three cents for a full identity check.
• At Haiku 4.5 rates (in $1/out $5) the same check is well under a cent.

Three cents to audit a bill that might run thousands of dollars a month is the cheapest insurance in your stack. Even running it daily across several gateways costs less than a dollar a month.

The 10.6% Error Rate Changes How You Sample

That error rate is the reason you cannot treat a single test as proof. Roughly one identification in ten can be wrong, so one "wrong" fingerprint is not evidence of a swap, it may just be noise. The right posture is statistical: build a baseline distribution against the known first-party model, then run repeated spot-checks against your gateway and compare distributions, not single answers. A swap shows up as a persistent shift in the distribution across many samples, not one odd result. Because each check costs pennies, sampling repeatedly is trivially affordable.

First-Party APIs vs. Third-Party Relays

Direct first-party APIs, Anthropic, OpenAI, Google, and the like, have little incentive to serve you something other than the model you named; their business is the model. The swap risk concentrates in the layers between you and them: aggregators, resellers, and OpenAI-compatible gateways that route across many backends to optimize their own margins. That routing is often a feature you want, it can cut costs, but it also means the model behind a given name can change without notice. Fingerprinting is how you keep that convenience honest.

A Practical Verification Routine

1. Baseline against first-party. Hit the model directly on its official API and collect a few hundred random-number answers to build its reference distribution.
2. Spot-check the gateway. Send the same prompt through your relay under the same model name, at least ~120 requests, and record the distribution.
3. Compare distributions, not single answers. Account for the ~10.6% error rate by relying on the shape across many samples.
4. Automate and schedule it. At pennies per run, a weekly or daily cron check across each provider is effectively free.

If the gateway's distribution drifts persistently from your first-party baseline, that is your cue to ask hard questions, or move traffic. Whether you are paying premium Opus 4.8 rates or budget-tier prices, the point is the same: verify you are getting the model you are billed for, and fold that verification into how you track spend in your AI cost calculator.