Do You Actually Need an AI Agent?
Probably not. Most production tasks that get labeled 'AI agent work' are really a deterministic pipeline with a single LLM call in the middle, and building a full autonomous agent loop for them wastes money, adds fragility, and slows you down. An agent makes sense only when the number of steps is unknowable in advance and the system must decide what to do next based on intermediate results. Everything else is a workflow.
I am Mahmoud Zalt, an independent senior AI systems architect with 16+ years of production software behind me since 2010. I founded Sista AI, and a year of running autonomous agents in production has made me quick to spot when an agent is the wrong tool entirely. I have designed and shipped AI systems across industries, and the single most common mistake I see is teams reaching for agents when a pipeline would do the job in a third of the time and at a quarter of the cost. If you are evaluating what to build, my AI Agent Development service starts exactly here: with the honest question of whether an agent is the right tool at all. You can also read more about my background.
Agent vs. Workflow: The Actual Difference
These two terms get conflated constantly. Here is the precise distinction I use when scoping any project:
| Dimension | Workflow (Deterministic Pipeline) | AI Agent (Autonomous Loop) |
|---|---|---|
| Control flow | Fixed: defined by you in code | Dynamic: decided by the model at runtime |
| Steps | Known and finite before execution | Unknown until the task resolves |
| LLM role | One or a few steps in a fixed sequence | The orchestrator deciding what to do next |
| Failure modes | Predictable, testable, easy to trace | Difficult to predict or reproduce |
| Cost | Low and bounded per run | Variable, can spiral with long loops |
| Latency | Predictable | Unbounded |
| When to use | You know the steps; only one or two steps need LLM judgment | Open-ended tasks requiring multi-step reasoning and tool selection |
A workflow is 'extract structured data from this PDF, then write it to the database.' An agent is 'research this company, decide which data sources matter, retrieve them, reconcile conflicts, and produce a report.' The difference is not the presence of an LLM. It is whether the LLM is deciding the path or just executing one step on a path you already decided.
The Decision Framework I Use on Every Project
Before writing a single line of agent code, I run through five questions:
- Can you enumerate all the steps right now? If yes, write a pipeline. If the steps depend on what the model finds during execution, consider an agent.
- Is there a meaningful branch point that requires model judgment? A single 'if the sentiment is negative, do X' is a pipeline with a classifier. Thirty possible branches that depend on each other are agent territory.
- What is the blast radius of a wrong step? Agents that can call APIs, modify data, or send messages need strict guardrails and human-in-the-loop checkpoints. The more consequential the actions, the stronger the argument for a deterministic pipeline you control.
- What does the latency budget look like? Agents run multiple LLM calls in sequence. A task with a 2-second response SLA almost never works as an agent loop in production.
- Who maintains this in six months? A pipeline is inspectable by any engineer. An agent with dynamic tool-calling and memory is a debugging problem waiting to happen.
If questions 1 and 2 both point to 'pipeline,' stop there. You just saved weeks of engineering and a non-trivial monthly inference bill.
What Teams Get Wrong: The Agent-First Trap
The most common pattern I see: a team reads about autonomous agents, watches a few demos, and immediately starts building a ReAct loop for what is actually a three-step extract-transform-load task. The result is a system that is harder to test, harder to observe, more expensive per run, and no more capable than a pipeline would have been.
Worked Example: Invoice Processing
A finance team wants to extract line items from scanned invoices, validate totals, and push results to their ERP. Someone proposes an 'AI agent' that autonomously decides how to handle each invoice. Here is what that looks like in reality:
- Agent version: LLM decides each step. It calls an OCR tool, then a validation tool, then an ERP write tool. It might retry, might ask for clarification, might loop. Latency: 8 to 20 seconds per invoice. Cost: 4 to 8 LLM calls per invoice. Failure tracing: difficult.
- Pipeline version: Step 1, call OCR API. Step 2, pass OCR output to a single LLM prompt that extracts structured JSON matching your schema. Step 3, validate totals with deterministic code. Step 4, write to ERP. Latency: 2 to 4 seconds. Cost: 1 LLM call. Failure tracing: trivial.
The pipeline handles 95% of invoices correctly. The remaining 5% with ambiguous layouts go to a human review queue. That is not a limitation. That is the right design. The agent version would have handled the same 95% correctly and added complexity to the 5% that needs human judgment anyway.
When the Trap Gets Expensive
I have reviewed systems where a pipeline-appropriate task was built as a ten-step agent loop. The monthly inference cost was $4,000 to $6,000. Rebuilt as a pipeline with one LLM step, the same throughput cost $300 to $500 per month. The functionality was identical. The agent added zero value over the pipeline version on that task.
When an Agent Is Actually the Right Call
Agents earn their complexity in a specific class of tasks. These are the signals I look for:
- Open-ended research or discovery: The task is 'find relevant information about X' where neither you nor the system knows in advance how many sources to check, which ones are relevant, or how to reconcile conflicts. The number of steps is genuinely unknown.
- Multi-tool coordination with branching: The system needs to choose between 5 or more tools based on intermediate results, and the right sequence varies significantly across inputs.
- Self-correcting loops: The task requires the system to evaluate its own output and decide whether to retry with a different approach. Code generation with test execution and self-repair is a canonical example.
- Long-horizon task decomposition: A user request like 'set up this project environment' that legitimately requires 15 to 40 steps that depend on each other in ways that cannot be fully specified upfront.
Notice what is not on this list: 'classify this text,' 'summarize this document,' 'extract these fields,' 'generate a draft,' 'answer this question from these documents.' Those are all pipeline tasks with one or two LLM steps. The fact that they involve an LLM does not make them agents.
Practical Architecture: The Hybrid Reality
Most production AI systems are neither pure pipelines nor pure agents. They are deterministic pipelines with one or two agent-like nodes embedded inside them. This is the architecture pattern I recommend most often:
The Thin Agent Pattern
Build a deterministic outer pipeline that handles routing, error handling, observability, and data movement. Inside one step of that pipeline, you can have a small agent loop that handles the genuinely ambiguous part of the task. The outer pipeline gives you cost control, predictability, and observability. The inner agent gives you the flexibility you actually need.
Example: a customer support automation system. The outer pipeline classifies the incoming ticket, routes it to the right handler, enforces SLAs, and triggers escalation. Inside the 'complex issue' handler, there is a small agent loop that can call a knowledge base tool, a CRM lookup tool, and a draft-reply tool, running up to three iterations before handing off to a human. You get agent flexibility on the hard cases. You get pipeline determinism everywhere else.
Guardrails Are Not Optional
Any component that calls external APIs or writes to systems needs guardrails regardless of whether it is a pipeline step or an agent action. In practice this means: input validation before the LLM call, output validation (schema + semantic) after it, rate limiting on tool calls, a maximum step count in any loop, and a circuit breaker that hands off to a human or fails closed rather than retrying indefinitely.
For agents specifically, I always implement a hard cap on loop iterations (typically 5 to 10 for most tasks), token budget enforcement per run, and structured logging of every tool call with its inputs and outputs. Without this, debugging a production failure is nearly impossible.
Cost, Observability, and the Evals Problem
One aspect teams consistently underestimate: evals. Before deploying either a pipeline or an agent to production, you need a test set that reflects your real input distribution, a scoring function that defines what 'correct' means for your task, and a baseline you can regress against.
For a pipeline, this is straightforward. You run your 200-example eval set, score outputs, and ship when you hit your threshold. For an agent, the eval problem is harder because you are evaluating a trajectory, not just a final answer. Did the agent take the right steps? Did it avoid unnecessary tool calls? Did it produce a correct result without wasting 15 tokens of context on irrelevant retrieval?
Cost tracking looks different too. Pipeline cost is simple: (input tokens + output tokens) times price per token, times volume. Agent cost requires tracking the full loop: how many iterations did each run take? What was the token cost per step? What was the p95 cost? I have seen agents with a median cost of $0.004 per run and a p95 of $0.12 per run because some inputs triggered long loops. On 1 million daily runs, that tail matters.
Observability minimum for production:
- Trace every run end-to-end with a unique run ID
- Log every LLM call: model, prompt tokens, completion tokens, latency, cost
- Log every tool call: tool name, inputs (sanitized), outputs, latency, success/failure
- Track iteration count per agent run
- Alert on runs that hit your max iteration cap (that is usually a sign of a degenerate input)
Frequently Asked Questions
what is the difference between an ai agent and an automated workflow
A workflow has a fixed sequence of steps that you define in code. An agent has a dynamic sequence where the LLM decides the next step based on intermediate results. Both can use LLMs. The difference is in who controls the control flow: you (workflow) or the model (agent). Most tasks that 'sound like AI' are workflows with one or two LLM steps, not autonomous agents.
do i need an ai agent for rag or document qa
No. A RAG system is a pipeline: retrieve relevant chunks, pass them to an LLM with a prompt, return the answer. That is two deterministic steps with an LLM in the second position. You only need an agent if the retrieval itself needs to be dynamic, for example if the model needs to decide which of several knowledge bases to query, run multiple retrieval passes, and reconcile results. Straightforward RAG is a pipeline.
when should i use tool calling vs building a full agent
Tool calling in a single LLM call (where the model outputs a structured function call and you execute it once) is a pipeline pattern, not an agent. A full agent is when tool outputs are fed back into the model to inform the next decision, repeatedly. Use single tool calling for tasks with a predictable one-step action. Use an agent only when you need genuine multi-step reasoning where each step changes what the next step should be.
how much does an ai agent cost to run in production versus a pipeline
A pipeline with one LLM step typically costs 1 to 3 times the raw inference cost of that call. An agent typically costs 3 to 15 times more per task because of multi-step loops and context accumulation across turns. On high-volume tasks (millions of runs per month), this difference is the dominant cost driver. I have seen teams reduce monthly inference spend by 80% simply by converting an agent-based system to a pipeline after realizing the agent loop added no accuracy benefit.
what are the production risks of using ai agents that pipelines avoid
Agents introduce non-determinism in control flow, making failures harder to reproduce and debug. They have unbounded latency and cost per run if loops are not capped. They are more vulnerable to prompt injection when external data is fed back into context across turns. They require more sophisticated evals because you are testing trajectories, not single outputs. Pipelines fail in predictable, traceable ways. Agents can fail in ways that are difficult to reproduce from logs alone.
can i start with a pipeline and add agent capabilities later
Yes, and this is almost always the right approach. Ship the pipeline first. It will handle 80 to 90 percent of your cases correctly with much less engineering effort. Identify the specific cases where deterministic steps are insufficient. Add a bounded agent node for exactly those cases. This pattern gives you the fastest path to production, the lowest initial cost, and the clearest upgrade path. Starting with a full agent is usually premature.
Ready to Build the Right Thing?
If you are deciding between a pipeline and an agent for a real project, the answer is almost always 'start with the pipeline.' Get it to production. Measure where it falls short. Then add the complexity that the gap actually requires, nothing more. I work with teams at exactly this decision point: auditing existing AI systems that are over-engineered, scoping new systems that are being designed too ambitiously, and building production AI infrastructure that is fast, cost-efficient, and maintainable by a real engineering team.
You can review my work at /projects, read more about my background, or browse other articles on this at /blog. If you have a specific system to build or audit, reach out directly at /contact.
