Multi-Agent Orchestration Patterns: Supervisor, Swarm, Handoff, and When One Agent Is Enough

Most teams reach for multiple agents too early. The demo looks great: a planner, a researcher, a coder and a reviewer, each with a role and a cute name. Then it ships, and latency triples, costs multiply, and nobody can explain why the reviewer approved something the coder broke.

Multi-agent orchestration is a real tool. It solves specific problems: context overload, parallelism, and separation of permissions. It also adds a coordination layer that is itself a source of failures. This guide covers the four patterns worth knowing, how each one breaks, and how to decide whether you need any of them.

Key Takeaways

• Start with one agent and a good tool set. Add agents only when you can name the specific limit (context, parallelism, or permissions) that a single agent hits.
• There are four workable patterns: pipeline, supervisor, handoff, and swarm. They trade control for flexibility in that order.
• Every agent boundary is a lossy compression of context. Most multi-agent bugs are information lost or distorted at a boundary.
• Cost and latency scale with the number of hops, not the number of agents. Count hops.
• Treat inter-agent messages as untrusted input, the same as tool output.

When One Agent Is Enough

A single agent with well-designed tools handles more than people expect. Before adding a second agent, check whether your problem is really one of these:

Two honest reasons justify multiple agents. The first is context isolation: a sub-agent explores a codebase or reads 40 documents in its own window and returns a 200-token summary, so the parent stays clean. The second is permission isolation: the agent that reads untrusted web content should not hold the credentials that can send email.

"Specialization" on its own is a weak reason. A single model with the right instructions in context is usually as good at each role as a dedicated agent, and it doesn't pay the handoff cost. See Context Engineering for AI Agents for how to get more out of one agent first.

Pattern 1: Pipeline (Sequential Chain)

Agent A's output becomes agent B's input, then C's. The control flow is fixed in code, not decided by a model.

def run_pipeline(task: str) -> str:
    plan = planner.run(task)
    draft = writer.run(plan)
    return reviewer.run(draft)

Use it when the steps are known in advance and each has a clear input and output contract: extract, transform, validate, publish.

How it fails:

• Errors compound. A wrong plan produces a confident, well-written, wrong draft.
• Rigid flow. If step 2 discovers that step 1 was wrong, there's no path back unless you build one.

Mitigation: put a schema between stages and validate it. A typed output ({"steps": [...], "assumptions": [...]}) catches malformed handoffs before they cascade. Add an explicit retry-or-escalate edge from each stage.

A pipeline is closer to a workflow than an agent system, and that's a feature. Deterministic control flow is easier to test, trace and cost-estimate.

Pattern 2: Supervisor (Orchestrator-Workers)

One agent plans and delegates. Workers execute narrow tasks and report back. Only the supervisor talks to the user and decides what happens next.

while not done:
    decision = supervisor.decide(state)      # which worker, with what brief
    result = workers[decision.worker].run(decision.brief)
    state = supervisor.integrate(state, result)
    done = supervisor.is_complete(state) or state.steps > MAX_STEPS

Use it when the task decomposes dynamically (you don't know the subtasks until you start) and you want a single point of control and audit. This is the most common production pattern and the best default when a pipeline is too rigid.

How it fails:

• The supervisor is the bottleneck. Every result flows back through its context. On long tasks its window fills with worker output.
• Bad briefs. The worker only knows what the supervisor wrote. A vague brief produces plausible work on the wrong problem.
• Premature completion. Supervisors tend to declare success after the first reasonable-looking result.

Mitigations:

• Make workers return structured summaries, not raw transcripts. Keep the full trace in storage the supervisor can query on demand.
• Require briefs to include the goal, constraints, and the exact output format expected.
• Give the supervisor an explicit completion checklist and a hard step budget. Our loop engineering guide covers termination design in depth.

Parallelism is the supervisor's biggest win. If three research subtasks are independent, dispatch them concurrently and wait for all three. Wall-clock time drops to the slowest worker instead of the sum.

Pattern 3: Handoff (Routing and Transfer)

Control moves from one agent to another, and the new agent takes over the conversation. There's no central supervisor. Think of a support flow: a triage agent hands to billing, which hands to a refunds specialist.

current = triage_agent
while True:
    reply = current.run(conversation)
    if reply.handoff_to:
        conversation.add_note(reply.handoff_summary)
        current = agents[reply.handoff_to]
        continue
    return reply

Use it when the work is conversational, and different phases need different instructions, tools or permissions. The user experiences one continuous interaction while the active agent changes underneath.

How it fails:

• Context loss at the transfer. The receiving agent lacks what the user already said. Users repeat themselves, and trust drops.
• Ping-pong. Two agents that each think the other owns the problem hand it back and forth.
• Ambiguous ownership. Nobody is accountable for the final outcome.

Mitigations:

• Hand off a structured summary (goal, facts established, what's been tried) plus the relevant history, not the whole transcript and not nothing.
• Cap handoffs per session and detect cycles (A to B to A). On a cycle, escalate to a human or a fallback agent.
• Give every handoff a reason code. Those codes become your best debugging data.

Pattern 4: Swarm (Peer-to-Peer)

Agents communicate directly, share a workspace, and self-organize. No fixed controller. Useful for open-ended exploration, simulations, and research tasks where the structure of the solution is unknown.

Use it when you're exploring, the cost of a wasted branch is low, and you can afford variance.

How it fails: nearly every way the other patterns do, with less visibility. Agents duplicate work, contradict each other, or converge on a shared mistake because they read each other's confident output. Cost is hard to predict because the number of messages isn't bounded by design.

Mitigations: a shared blackboard (one structured store of claims and status) rather than free-form chat; per-agent and global budgets; and a designated verifier whose job is to challenge claims, not to produce them. If you can't tell what each agent contributed, you can't debug the swarm.

Be skeptical of swarms for anything that must be reliable. Most "swarm" success stories turn out, on inspection, to have a supervisor hiding in them.

Choosing a Pattern

A practical rule: use the least flexible pattern that solves the problem. Move right on this table only when the pattern on the left demonstrably can't express what you need. Many production systems are hybrids, such as a supervisor whose workers are small pipelines.

Designing the Boundaries

Whatever pattern you choose, the quality of the system is set at the boundaries between agents.

Define a contract for every message. Name the fields, types and meaning. Free-text handoffs work in demos and fail on edge cases. Validate on receipt and reject malformed messages loudly.

Pass summaries, keep traces. Sending full history across a boundary wastes tokens and buries the signal. Sending too little causes the repeated-question problem. Pass a curated brief and store the full trace for retrieval.

Scope tools per agent. Give each agent the minimum tools it needs. This improves tool selection accuracy and shrinks the blast radius of a compromised agent. It's also the cleanest argument for multiple agents in the first place; see Tool-Calling Agent Sandboxing and Security.

Treat agent output as untrusted input. If one agent reads a web page containing an injected instruction and passes its "summary" to a privileged agent, the injection has crossed the boundary. Sanitize and constrain what flows toward agents with more power.

Budget every loop. Each agent and the system as a whole needs a step cap, token cap and timeout. Multi-agent systems can fail by running up a bill rather than by crashing.

The Cost Math

Cost in a multi-agent system comes from three places, and people usually count only the first:

1. Model calls per agent. Obvious.
2. Repeated context. Every hop re-sends instructions, tool definitions and relevant history. A five-hop chain can send the same 3,000-token system prompt five times.
3. Coordination overhead. The supervisor's planning and integration calls produce no user-visible work.

Rough heuristic: a multi-agent run commonly costs several times the tokens of a well-built single-agent run for the same task. That can be worth it for quality or parallel speed, but you should measure, not assume. Prompt caching and routing cheap models to simple worker roles claw back much of it; see Cutting LLM API Costs with Prompt Caching and Model Routing and The True Cost of Running AI Agents at Scale.

Testing and Observability

You can't debug what you can't see. At minimum:

• Trace every hop with a shared run ID, the sending and receiving agent, the message, token counts and latency. OpenTelemetry for LLM Applications shows how to structure this.
• Evaluate the system, not only the agents. An agent can pass its own tests while the composition fails. Build end-to-end test tasks and track success rate, cost per task and hops per task over time. See How to Evaluate Your LLM Agent.
• Replay failures. Store inputs and message traces so you can re-run a failed case against a changed prompt.
• Watch the hop count. A rising average hop count is often the earliest sign of ping-pong or a confused supervisor.

Frequently Asked Questions

Should I use a framework like LangGraph, or build my own?

Frameworks give you state management, checkpointing and tracing out of the box, which are tedious to build well. Pick one when you need durable state or human-in-the-loop pauses. For a simple pipeline or a small supervisor loop, plain code is often clearer and easier to debug. Whichever you choose, keep your agent logic separable from the framework so you can change it later.

Is the supervisor pattern the same as "agents as tools"?

Nearly. In "agents as tools," the parent calls a sub-agent like any other tool and gets a result back; it never relinquishes control. That is the supervisor pattern with the delegation exposed through the tool-calling interface. It's often the simplest way to implement it.

How many agents is too many?

There's no magic number, but treat each additional agent as needing justification: a distinct permission scope, an isolated context, or real parallelism. If you can't state which, merge it into a neighbor. Systems with more than a handful of agents usually have redundant roles.

How do I stop agents from agreeing with each other's mistakes?

Use independent verification. Give the verifier the original task and the claimed result, not the producer's reasoning, and ask it to check against the source. Where possible, verify with deterministic checks (tests, schema validation, lookups) instead of another model's opinion.

Can I mix models across agents?

Yes, and you often should. Use a stronger model for planning and judgment, and cheaper, faster models for narrow worker tasks. Evaluate the combination end to end, since a weaker worker can silently degrade the whole run.

The Bottom Line

Multi-agent orchestration is an engineering tradeoff, not an upgrade. Each added agent buys you isolation or parallelism and costs you context fidelity, latency and debuggability.

Start with one agent. When it hits a specific limit, pick the least flexible pattern that addresses it: a pipeline for fixed steps, a supervisor for dynamic work, handoffs for conversational phases, a swarm only for exploration. Define strict contracts at every boundary, budget every loop, trace every hop, and measure the system end to end.

If your multi-agent version doesn't beat your single-agent version on a real evaluation, ship the single agent.