Cloudflare AI Search Adds Six Models: Price RAG Quality Before You Switch

Cloudflare AI Search now offers six additional Workers AI text-generation models, but a larger menu is not automatically a cheaper retrieval system. The economic question is which model produces an acceptable grounded answer with the fewest retrieval, reasoning, and repair cycles for your own corpus.

According to Cloudflare's August 26 changelog, AI Search added DeepSeek V4 Flash, DeepSeek V4 Pro, GPT-OSS 120B, GPT-OSS 20B, Qwen 3.8 27B, and Kimi K2.7 Code. The two DeepSeek options expose 1,048,576-token context windows, Qwen and Kimi expose 262,144, and the GPT-OSS models expose 128,000. They run on Workers AI and do not require another provider key.

Separate Retrieval Cost From Generation Cost

AI Search still has a pipeline: ingest documents, build or refresh an index, retrieve candidates, assemble context, call a generation model, and return an answer. A model change directly affects the last two stages, but it can indirectly change retrieval behavior. A stronger model may tolerate noisier chunks; a long-context model may encourage teams to send too many chunks. Attribute spend and latency to each stage so a model comparison does not credit retrieval improvements to generation.

Use cost per grounded accepted answer, not price per million tokens, as the main unit. Include all input and output tokens, retries, search operations, failed citations, evaluation labor, and latency. A low-rate model that needs three attempts can cost more than a higher-rate model that answers correctly once. Likewise, a one-million-token window creates capacity, not an instruction to fill it.

Build a Representative Evaluation Set

Sample real developer questions across easy lookup, multi-document synthesis, version conflicts, code examples, and questions that should be refused because the corpus lacks evidence. Preserve expected sources and acceptable answer criteria. Include fresh documents and stale documents to see whether the model respects recency metadata. A benchmark made only of obvious factual lookups will favor speed while hiding the costly failures that appear in production.

Freeze chunking, embeddings, retrieval count, prompt, temperature, and evaluation rubric during the first model comparison. Run multiple repetitions when outputs are nondeterministic. Record answer acceptance, citation precision, unsupported claims, input tokens, output tokens, time to first token, total latency, and any second-pass repair. Only then test model-specific retrieval settings.

Treat Context as a Budgeted Resource

Long context can reduce orchestration calls when a question genuinely spans many files, but irrelevant context increases token cost and can dilute the evidence. Set a maximum retrieved-token budget by question class. Start with a small evidence set, expand only when retrieval confidence or answer evaluation shows a gap, and log every expansion. This progressive approach makes the value of a larger window measurable.

For coding support, keep generated code tied to source versions. Ask the evaluator whether imports, method signatures, flags, and deprecations match the retrieved documentation. An answer that reads well but targets an old SDK creates debugging work outside the model invoice. Include that repair time when calculating cost.

Route by Query Shape

The six models do not need one winner. A small model may handle direct documentation lookup, a coding-specialized model may answer repository questions, and a long-context model may be reserved for architecture synthesis. Define routing rules from observable inputs such as retrieved-token count, number of sources, code density, and risk. Avoid a hidden model cascade that silently spends more whenever the first answer looks uncertain.

Put a ceiling on cascades: maximum models tried, maximum total tokens, and a clear stop condition. When the ceiling is reached, return the retrieved evidence with an uncertainty notice instead of purchasing an unbounded series of guesses. Review routes weekly and remove rules that do not improve accepted-answer cost.

Run a Two-Week Shadow Test

Replay sampled production questions against the incumbent and two candidate models without changing user-visible answers. Redact sensitive material, use the same index snapshot, and score results blind to model identity. Promote a candidate only when it meets the quality floor and improves cost or latency at the task level. Keep a rollback route and preserve model IDs in traces.

Cloudflare's release expands useful choices and removes extra provider-key management for these Workers AI models. The savings still depend on disciplined retrieval, bounded context, and routing evidence. Start with measured workloads, not headline context windows, and pay for capability only where it changes a completed task.

Publish the Decision Record

For every route, document the selected model ID, corpus snapshot, retrieval settings, evaluation date, accepted-answer rate, p50 and p95 latency, and cost range. Include the rejected alternatives and the threshold that would trigger another test. This prevents a temporary benchmark winner from becoming an unexplained permanent default.

Re-run the evaluation after material model, index, embedding, or corpus changes. Sample production misses and add them to the test set without deleting older cases. A living decision record turns six new options into controlled capacity: teams can switch when evidence changes, while finance and engineering retain the same definition of a successful search answer.