Single Agent or Multi-Agent? Start with One.
For most production workflows, a single well-built agent with a solid tool set will outperform a multi-agent system in reliability, cost, latency, and debuggability. Reach for multi-agent only when you have genuine parallelism requirements or hard trust boundaries that a single agent cannot satisfy.
I am Mahmoud Zalt, an independent AI systems architect with 16 years building production software. I created Porto SAP, an architectural pattern for keeping large codebases modular, and that same instinct for where to draw boundaries now drives how I split work across agents at Sista AI, the company I founded, where autonomous agents have run in production for the past year. I work directly with engineering teams and founders as an AI architecture consultant. This article is the honest version of the conversation I have with almost every client who arrives excited about 'agent swarms.'
Why Teams Default to Multi-Agent
The pattern is predictable. A team reads a framework README or watches a demo where five agents collaborate on a research task, and the result looks impressive. They then design their system the same way, before they have a single working agent, before they have evals, and before they understand the actual failure modes.
The result is an architecture that looks sophisticated in a diagram and breaks constantly in production. Error messages propagate between agents in ways that are hard to trace. Context gets dropped at handoff boundaries. Costs multiply because each agent step calls the LLM, and a five-step orchestration on GPT-4o can cost ten times what a single well-prompted call would cost. Latency stacks because each agent hop adds a round-trip.
None of this is a problem with multi-agent systems in principle. It is a problem with applying them before the simpler solution has been ruled out.
How High the Single-Agent Ceiling Actually Is
A single agent with well-designed tools can handle far more than most teams assume. The architecture looks like this: one LLM call, a reasoned system prompt, a curated tool set, and a retrieval layer. The agent decides which tools to invoke, invokes them, observes results, and produces a final response. That is it.
What you can fit into that pattern is significant:
- Complex retrieval: hybrid semantic and keyword search over a large corpus, re-ranking, citation extraction
- Multi-step reasoning: chain-of-thought over retrieved context, conditional branching based on intermediate results
- Tool composition: calling a database, an API, a code executor, and a structured output parser in sequence
- Long context: current frontier models support 128k to 1M tokens; many workflows that 'need' multiple agents are really just context management problems
- Structured output: JSON schema enforcement, validation, retry on schema failure
I have built customer-facing agents that handle product recommendation, eligibility checking, scheduling, and escalation routing all inside a single agent with eight tools. The system runs in under two seconds and costs under two cents per session. The same design as a four-agent orchestration would have been slower, more expensive, and harder to eval.
The Two Cases Where Multi-Agent Earns Its Keep
There are exactly two situations where the added complexity of multi-agent is justified. Both require genuine architectural reasons, not aesthetic preference.
1. Genuine Parallelism
If your workflow has tasks that are independent and time-sensitive, running them in parallel reduces wall-clock time. A research pipeline that must query three separate knowledge bases, score results from each, and merge them is a legitimate case. The fan-out and fan-in pattern adds real value when each branch does non-trivial work and the latency reduction matters to the user.
The key word is independent. If task B depends on the output of task A, you do not have parallelism. You have a sequential pipeline, and a single agent with sequential tool calls is simpler.
2. Hard Trust Boundaries
When different parts of a workflow operate under different security contexts, different permission scopes, or must be auditable by different stakeholders, separate agents with explicit handoffs make the boundary visible and enforceable. An agent that browses the web should not have the same database write permissions as the agent that updates customer records. That is a real architectural reason to separate them.
Everything else, including 'the prompt is getting long,' 'this step feels like a different job,' and 'the diagram looks cleaner,' is not sufficient justification.
The Real Tax: Cost, Latency, and Failure Surface
Multi-agent systems are not free. Here is what you are actually paying:
| Dimension | Single Agent | Multi-Agent (4 hops) |
|---|---|---|
| LLM calls per task | 1 to 3 | 4 to 12 |
| Latency (typical) | 1 to 3 s | 4 to 15 s |
| Cost per session | low baseline | 4 to 10x baseline |
| Failure modes | prompt, tool, output | all of above, plus handoff, context loss, orchestrator error |
| Debug surface | one trace | N traces, cross-agent correlation |
| Eval complexity | one eval harness | per-agent evals plus end-to-end |
The failure surface point is under-appreciated. In a single agent, a bad output is visible at one point. In a multi-agent pipeline, a subtly wrong intermediate output from agent 2 corrupts agents 3, 4, and 5. You often only see the failure at the final output and have to work backward through multiple traces to find the source. This is not theoretical. Every production multi-agent system I have reviewed has had incidents of exactly this type.
What Teams Get Wrong When They Do Go Multi-Agent
When multi-agent is the right call, most teams still make the same set of mistakes. Avoiding these saves weeks of debugging.
No Evals at Each Boundary
Teams add an end-to-end eval and call it done. A failure in the middle of the pipeline passes the eval by luck when the downstream agents compensate, or fails opaquely when they cannot. Correct approach: eval each agent independently with representative inputs and expected outputs, then add a system-level eval on top.
Implicit Context Passing
Agents pass raw LLM output to the next agent as a string. The receiving agent now depends on the phrasing of the upstream agent, which is not stable. Correct approach: define explicit typed schemas at every handoff. The upstream agent produces a structured object. The downstream agent receives a structured object. This is non-negotiable.
No Circuit Breaker
Agent 1 returns a low-confidence result. Agent 2 proceeds anyway. Agent 3 proceeds. The user gets a confident-sounding wrong answer. Correct approach: confidence scoring or explicit 'I cannot complete this' output at each stage, with a human-in-the-loop escalation path when any agent falls below threshold.
Orchestrator as God Object
A single orchestrator agent that 'manages' all other agents sounds clean and becomes a bottleneck with a bloated context window and unclear responsibility. Correct approach: prefer direct delegation. The user-facing agent calls sub-agents as tools, not as a managed process. Simpler graph, simpler traces.
A Practical Decision Framework
Before choosing your architecture, answer these five questions in order. Stop as soon as you hit a 'no.'
- Does a single agent with the right tools solve the problem? Build that first. Ship it. Measure it. Do not theorize about what you will need.
- Is there genuine independent parallelism that matters for latency or throughput? If the tasks must run sequentially, multi-agent adds nothing.
- Do different workflow stages require different trust levels or permission scopes? If not, one agent with scoped tools is sufficient.
- Can you eval each proposed agent independently before wiring them together? If you cannot describe what 'good output' looks like for each agent in isolation, you are not ready to compose them.
- Have you modeled the cost and latency at the P95 case? Multi-agent is always more expensive than it looks in the happy path. Model the tail.
If you reach question 5 and the answers still support multi-agent, build it. The framework does not oppose multi-agent. It opposes premature multi-agent.
Observability and Guardrails You Cannot Skip
Regardless of whether you go single or multi, these are non-negotiable for any production agent system.
Structured Traces
Every LLM call should emit: input tokens, output tokens, latency, model version, tool calls made, tool outputs received, and a session or trace ID that links all calls in one user interaction. Tools like LangSmith, Langfuse, or a custom OpenTelemetry pipeline all work. The choice matters less than having it. You cannot debug what you cannot observe.
Input and Output Guardrails
Validate inputs before they reach the LLM: length limits, topic classifiers, PII detection if relevant to your compliance scope. Validate outputs before they reach the user: schema enforcement, toxicity classifiers if the domain warrants it, factual consistency checks against retrieved sources. This is not optional in any user-facing system.
Retry and Fallback Strategy
Define explicitly what happens when a tool call fails, when the LLM returns malformed output, and when the session exceeds a cost ceiling. Retry with backoff on transient errors. Fall back to a simpler path or human escalation when retries are exhausted. Hard-code no tool as 'always available.'
Cost Controls
Set a per-session token budget. Track it. Interrupt gracefully when exceeded rather than allowing runaway context accumulation. In multi-agent systems, pass the remaining budget to each sub-agent so downstream agents do not exceed what the orchestrator has already spent.
Frequently Asked Questions
When should I use a multi-agent system instead of a single agent?
Use multi-agent when you have genuine independent parallelism that reduces user-facing latency by more than the added coordination overhead, or when different workflow stages must operate under different security contexts or permission scopes. Both conditions require concrete evidence, not intuition. If you cannot point to a measured latency benefit or a documented trust boundary, default to a single agent.
Do AI agent frameworks like LangGraph or CrewAI require multi-agent?
No. LangGraph is a graph-based execution framework that works equally well for single-agent state machines and multi-agent pipelines. CrewAI is oriented toward multi-agent, but nothing stops you from using it with one agent. The framework does not determine the architecture. The architecture should be determined by your requirements.
What is the real cost difference between single and multi-agent systems?
A rough rule: every additional agent hop that uses the same model tier costs roughly as much as the base call, plus context overhead from passing state between agents. A four-hop pipeline using GPT-4o can easily cost 5 to 8 times the equivalent single-agent call. With frontier models at current pricing, this becomes significant at scale. Always model cost at your expected daily session volume before committing to an architecture.
Can a single agent handle complex multi-step tasks?
Yes, for most definitions of 'complex.' A single agent with tool use can execute conditional branching, multi-step retrieval, external API calls, code execution, and structured output in one context window. The practical limits are: tasks that are genuinely too long for the context window even with compression, tasks requiring simultaneous independent computation, and tasks where isolation between steps is a security requirement.
How do I know if my agent system is ready for production?
You have: an eval harness with representative inputs and passing rates you are willing to defend, structured tracing on all LLM calls and tool invocations, a defined fallback path for every failure mode, input and output guardrails appropriate to your compliance context, and a cost model at P95 session volume. If any of those are missing, the system is not production-ready regardless of how well it works in demos.
Work With Me on Your Agent Architecture
I help engineering teams and founders design AI agent systems that work in production, not just in demos. That usually means building less than you planned, validating each component with real evals before composing it, and designing for the failure modes the happy path hides. If you are deciding between a single agent and a multi-agent design, or you have a system that is already more complex than it should be, I can give you a clear architecture recommendation fast.
Read more about my work on my background, see what I have shipped, or explore my AI architecture advisory services. When you are ready to talk, get in touch directly.
Book an AI Architecture Review
