OpenAI's 'Useful Intelligence per Dollar': A New Way to Measure AI Coding ROI

On July 17, 2026, OpenAI CFO Sarah Friar published a proposal titled "A Scorecard for the AI Age." Its centerpiece is a four-part metric she calls "Useful Intelligence per Dollar" — a way for CFOs to judge AI spending by the value of the work delivered, not the sticker price of a million tokens. Strip away the boardroom framing and it maps cleanly onto a question every engineering manager already asks: for the money we pour into AI coding, how much accepted, shippable work comes back out?

Token Price Is the Wrong Denominator

The instinct when comparing coding models is to sort by input price and pick the cheapest row. But a token is an input, not an outcome. You do not ship tokens — you ship accepted pull requests, passing test suites, and merged fixes. Two models can bill the identical number of tokens and deliver wildly different amounts of usable work. Friar's argument to CFOs is that vendors must connect capability claims to accepted work. The same discipline turns a fuzzy "which model is cheaper" debate into a number you can actually compute.

A Version You Can Compute Today

For coding, define it as: useful intelligence per dollar = accepted outputs divided by total spend. Flip the fraction and you get the more intuitive figure — cost per accepted task = total spend divided by accepted outputs. "Accepted" is whatever your team already gates on: a PR that passes review and CI, a task the agent finishes without a human rewrite, a bug fix that closes the ticket. The denominator is every dollar you spent getting there, including the attempts that failed.

Worked Example: Cheap Tokens vs High Accept Rate

Assume an agentic coding task burns roughly 120K input tokens (repo context, tool calls, file reads) and 20K output tokens per attempt. Compare the cheapest tier against a frontier model:

• DeepSeek V4 Flash (in $0.09 / out $0.18): 0.12 × $0.09 + 0.02 × $0.18 = about $0.014 per attempt.
• Claude Opus 4.8 (in $5 / out $25): 0.12 × $5 + 0.02 × $25 = $0.60 + $0.50 = $1.10 per attempt.

Now add accept rates. Say the cheap model lands an acceptable result on the first try 40% of the time, and the premium model 85% of the time. Expected attempts to reach one acceptance equals 1 divided by the accept rate: 2.5 attempts for the cheap model, about 1.18 for the premium one. That gives a cost per accepted task, tokens only:

• DeepSeek V4 Flash: $0.014 × 2.5 = about $0.035 per accepted task.
• Opus 4.8: $1.10 × 1.18 = about $1.30 per accepted task.

On raw API spend, the cheap model still wins by a wide margin. A roughly 55x per-token gap does not vanish behind a 2x retry penalty. If tokens were your only cost, cheapest-per-token really would be cheapest-per-accepted-task. But tokens are not your only cost.

Rejected Work Is Not Free

Every failed attempt has to be caught by a human. Someone reads the diff, notices it is wrong, and re-prompts. Fold that in and the picture can invert. Put a developer's loaded time at $75/hour and call each review-and-reject cycle eight minutes — about $10.

• DeepSeek V4 Flash: 2.5 attempts means roughly 1.5 rejected diffs before one lands, plus reviewing the accepted one — call it $25 of human time on top of $0.035 in tokens.
• Opus 4.8: about 0.18 rejected diffs plus one accepted review — roughly $12 of human time on top of $1.30 in tokens.

Fully loaded, the accepted-work cost is about $25 for the "cheap" model and about $13 for the premium one. The model with the higher sticker price delivers more useful intelligence per dollar, because the scarcest, most expensive resource in the loop is not tokens — it is the engineer reviewing them. That is exactly Friar's point: measure accepted, useful work per dollar, and the cheapest token stops being the cheapest outcome.

A Simple Worksheet You Can Run This Week

You do not need OpenAI's full four-part framework to start. For two weeks:

1. Log total API spend per model — most provider dashboards export this directly.
2. Count accepted outputs per model: merged PRs, closed tickets, tasks that shipped without a rewrite.
3. Divide spend by accepted count to get cost per accepted task.
4. Optional but decisive: add review time at your loaded hourly rate for every rejected attempt.

Whichever model gives you the lowest fully-loaded cost per accepted task is your best value — regardless of where it sits on the per-token price list.

What It Changes About Model Selection

The practical rule that falls out is: match model tier to acceptance stakes. For low-stakes, easily verified work — boilerplate, tests, formatting — a cheap model's lower accept rate barely matters, because rejections are quick to catch and its token savings dominate. For high-stakes, hard-to-verify work — subtle refactors, concurrency, security-sensitive paths — a premium model's higher first-try accept rate earns its token premium, because each rejection burns expensive review time and risks shipping a bug that surfaces later. The point of a metric like useful intelligence per dollar is to stop arguing about sticker prices and start measuring delivered work. Plug your own token volumes and accept rates into the AI cost calculator to see where each model lands for your workload.