What Is Workflow-vs-Agent Architecture? A Cost Decision Framework for Production AI Coding

The Question

Every production AI system faces the same architectural fork: which parts should be run by code, and which should be delegated to an LLM? The trend in 2026 has been away from "let the model orchestrate everything" and toward hybrid systems where deterministic code handles routing, sequencing, and error handling, while the LLM handles the parts that genuinely need language understanding.

This shift matters because the wrong choice can multiply your monthly bill by 3–5x with no quality improvement. Framework releases like Google's ADK 2.0 have made the pattern explicit, but the underlying decision applies to every framework — LangGraph, Temporal, Dify, or your own code.

Defining the Terms

• Workflow (deterministic). Fixed sequence of steps encoded in code. Branches, loops, and error handling are all explicit. The system does exactly the same thing on inputs of the same shape.
• Agent (LLM-driven). An LLM decides at runtime what to do next given a goal, a toolset, and a memory. Sequence, branching, and error recovery emerge from the model's reasoning, not from code.
• Hybrid. A code-level workflow with LLM calls at specific steps. The graph is deterministic; the leaf nodes are probabilistic.

In practice, "hybrid" is where most production systems land. The interesting question is not workflow vs agent as extremes — it's which specific steps you delegate to the model.

Three Cost Dimensions

Every step in your system has a cost profile along three axes:

1. Token cost — dollars per execution. LLM steps have a nonzero token cost; code steps do not.
2. Latency — time from request to response. LLM steps are 200ms–5000ms; code steps are typically <10ms.
3. Failure rate — the probability the step doesn't produce the expected output. Code failure is measurable and reproducible; LLM failure is stochastic.

Every LLM step adds cost on all three axes. Every code step removes cost on all three. The question of workflow-vs-agent is really: which steps have enough language ambiguity to justify the LLM cost?

The Decision Matrix

Worked Example: Coding Assistant Pipeline

Consider a coding assistant that receives a natural language request, searches the codebase, generates code, and commits the change. Two implementations:

All-agent version. Single LLM prompt: "You are a coding assistant. Here are your tools. Do what the user asked." Every internal decision is an LLM turn. Typical cost per task: 30K input + 4K output = $0.10 on Sonnet 5. Average latency: 6–8 seconds.

Hybrid version. Code handles: parsing the request into structured fields, calling ripgrep, batching file reads, applying the edit via git. LLM handles: understanding the request (small model), generating the code (large model). Typical cost per task: 8K input + 2K output = $0.036 on Sonnet 5. Average latency: 2–3 seconds.

Difference: 64% token savings, 62% latency reduction, and fewer failed retries because the deterministic parts don't fail unpredictably. For a team running 5,000 tasks per day, that's $320/day saved, or roughly $115K/year — enough to fund the migration work multiple times over.

When to Prefer Pure Agent Architectures

Not everything wants to be a workflow. The pure agent pattern still wins for:

• Exploratory research. No fixed sequence of steps because you don't know what you're looking for.
• Long-tail customer support. Each case is unique; enumeration is impossible.
• Creative writing. The whole point is variety.
• Prototypes. Speed of iteration matters more than cost per run.

Migration Playbook

If you have a pure-agent system today and want to move toward hybrid:

1. Instrument first. Log every LLM call with its purpose. You cannot optimize what you cannot see.
2. Find the fixed-shape steps. Any LLM call that always returns the same schema is a workflow candidate.
3. Replace one at a time. Migrate the highest-frequency deterministic step first. Measure before and after.
4. Guard the boundary. Where code hands off to an LLM, validate the input; where LLM hands back to code, validate the output.
5. Keep an escape hatch. Some edge cases will require re-agentifying a step. Design for reversibility.

Bottom Line

Workflow-vs-agent is not a binary choice at the system level — it's a decision to make at every step. The default should be code; the exception should be LLM. That inverts the 2023 mindset of "just prompt harder," and for production systems it typically cuts costs 40–60% while making the behavior more predictable and debuggable.