Multi-Agent Orchestration
Multi-agent orchestration = coordinating multiple agentic loops so specialized agents can work on separate parts of a larger task, then combining their results into one coherent outcome.
Multi-agent orchestration means one agent coordinating with others to complete complex work, with agents able to run in parallel using isolated context.
Anthropic’s research-system write up frames the same idea: a planner agent decomposes a complex research query, then creates parallel agents that search simultaneously; this introduces coordination, evaluation, and reliability challenges.
1. Architecture Choice
You need to know when to choose:
| Need | Likely answer |
|---|---|
| Simple task, few steps, low ambiguity | Single agent |
| Fixed business process with strict ordering | Deterministic workflow |
| Complex open-ended task with independent subtasks | Multi-agent orchestration |
| Need domain specialists with different tools/prompts | Specialized agents |
| Need broad exploration or research | Parallel fan-out + synthesis |
| Need quality control | Generator + evaluator / critic |
| Need strict compliance | Programmatic guardrails override agent autonomy |
| Need long-running stateful execution | Persistent harness with durable state |
High-yield principle:
Do not use multi-agent orchestration just because it sounds cool. Use it when task complexity, context isolation, specialization, or parallelism justifies the cost and coordination overhead.
Anthropic’s “Building effective agents” guidance says successful implementations often use simple, composable patterns, and warns that extra framework layers can obscure prompts/responses and encourage unnecessary complexity.
2. Core Definition
A multi-agent system consists of multiple Claude-powered agents working together. Each agent may have its own: objective, system prompt, tools, context, model, skills, memory, or state, permissions, output contract
A typical architecture:
User request
↓
Coordinator / orchestrator agent
↓
Task decomposition
↓
Specialist agents run in parallel or sequence
↓
Each agent returns structured findings
↓
Coordinator synthesizes, resolves conflicts, verifies completeness
↓
Final answer / action / artifact
A coordinator delegates to a set of subagents. Each subagent has its own model, system prompt, tools, and context, while the coordinator controls delegation.
3. The Coordinator Agent
The coordinator must have
AgentinallowedToolsto spawn subagents.
Each subagent is defined by an AgentDefinition with:
| Field | Exam meaning |
|---|---|
| Description | Helps coordinator decide when to use the subagent |
| System prompt | Defines subagent behaviour |
| Tool restrictions | Scopes subagent permissions to its role |
The coordinator is the manager of the system. It should not blindly do all the work itself.
Its responsibilities:
| Responsibility | Meaning |
|---|---|
| Understand the user goal | Convert vague request into concrete work plan |
| Decompose task | Break work into independent or sequential subtasks |
| Select subagents | Choose the right specialist for each subtask |
| Assign constraints | Give each agent clear scope, output format, and success criteria |
| Monitor progress | Track which subtasks are complete, blocked, or conflicting |
| Synthesize results | Combine findings into a coherent final response |
| Verify completeness | Check whether the original user goal was satisfied |
| Escalate | Ask user/human/system for approval when needed |
The coordinator is not just a dispatcher. It must inspect the combined output then send targeted follow-up work to the appropriate subagents.
All communication flows through the coordinator. Subagents never communicate directly with one another.
Direct subagent-to-subagent communication is a trap, even if it sounds efficient. The reason is that coordinator-mediated communication gives:
| Benefit | Why |
|---|---|
| Observability | All messages can be logged and monitored centrally |
| Consistent error handling | One place applies recovery policies |
| Controlled information flow | Coordinator decides exactly what context each subagent receives |
So the answer is not “let agents collaborate freely”, its centralize routing through the coordinator.
3.1 The Narrow Decomposition Failure
A research system is asked about a broad topic, such as renewable energy technologies, but the coordinator decomposes the task only into solar and wind. The search agents do excellent work. The synthesis agent accurately combines what it receives. But the final report misses geothermal, tidal, biomass, fusion, etc.
The root cause is not weak search, poor synthesis, insufficient number of subagents, lack of sources
The root cause is: The coordinator decomposed the task too narrowly.
Rule:
If output is incomplete in scope, trace the failure back to coordinator decomposition.
4. Specialist Agents
A specialist agent is useful when the task benefits from a different prompt, tool set, model, or context boundary.
Examples:
| Specialist | Purpose |
|---|---|
| Research agent | Search, collect sources, summarize evidence |
| Code agent | Modify code, inspect files, run tests |
| Security agent | Threat model, review permissions, inspect risky changes |
| Documentation agent | Write user-facing docs or release notes |
| Test agent | Generate and execute tests |
| Critic/evaluator | Check quality, correctness, completeness |
| Compliance agent | Check policy or regulatory constraints |
Specialization is a core pattern: route work to agents with domain-focused system prompts and tools instead of loading one agent with every capability.
5. Context Isolation
Context isolation is one of the strongest reasons to use multiple agents.
A single agent has one context window. If it reads every file, source, policy, and intermediate result, the context becomes noisy. With multiple agents, each subagent can work with a narrower context and return only high-signal output.
Each subagent has its own isolated conversation history; the coordinator can follow up with an earlier subagent. Tools and context are not shared between subagents. If subagents do share a workspace (filesystem or credentials), that sharing can still create security and coordination risks even though their conversation context is isolated.
Subagents do not automatically inherit the coordinator’s conversation history.
They also do not automatically receive the coordinator’s system prompt, previous coordinator messages, results from other subagents, shared memory, global state, and prior invocation history.
6. Parallel Fan-out / Fan-In
A common multi-agent pattern is fan-out / fan-in.
Coordinator
├── Agent A: analyze source 1
├── Agent B: analyze source 2
├── Agent C: analyze source 3
└── Agent D: analyze source 4
All return structured findings
↓
Coordinator synthesizes final answer
Use fan-out when subtasks are independent, similar in shape, bounded, parallelizable, and not dependent on each other’s intermediate results
Good examples:
- Analyze 10 documents independently
- Search different repositories
- Compare vendor options
- Review separate modules
- Generate test cases for separate components
Bad examples:
- Create database schema, then implement API depending on that schema
- Perform financial transaction then audit it
- Update shared files with no locking/merge strategy
- Tasks requiring strict sequential approval
Parallelization is a strong pattern: fan out independent subtasks and have the coordinator synthesize results.
7. Specialization Pattern
Use specialization when different subtasks require different capabilities.
Example:
Coordinator: "Prepare migration plan"
Research agent:
- reads product docs
- gathers constraints
Architecture agent:
- designs migration options
Security agent:
- reviews auth/data risks
Cost agent:
- estimates cost and operational burden
Coordinator:
- compares trade-offs
- produces recommendation
High-yield decision rule:
Specialize agents by capability boundary, not by arbitrary task fragments.
Good specialization:
- Security reviewer with no write access
- Documentation writer with docs repo access
- Database analyst with read-only query tool
Bad specialization:
- Agent 1 handles first paragraph, Agent 2 handles second paragraph
- Every API endpoint gets its own agent even though one agent can reason overall of them
- All agents have all the tools, just in case
8. Escalation Pattern
Escalation means delegating a hard or risky subtask to a stronger agent, different model, human, or deterministic system.
Examples:
| Scenario | Escalation target |
|---|---|
| Cheap model cannot resolve ambiguity | More capable model |
| Agent wants to execute risky command | Human approval |
| Compliance policy is involved | Compliance checker / deterministic rule engine |
| Conflicting source evidence | Critic / evaluator agent |
| Security-sensitive code change | Security reviewer |
Escalation is a valid multi-agent pattern: consult a more capable agent or model for a subset of complex subtasks.
When escalating to a human, the human agent does not have the full conversation transcript. Therefore, the handoff must be self-contained and include Conversation history, recommended action and other relevant data.
Trap:
Escalation is not the same as retrying. Escalation changes authority, capability, or review level.
9. Generator–Evaluator Pattern
This is a very important architecture pattern.
Planner agent:
defines task and success criteria
Generator agent:
creates artifact / code / answer
Evaluator agent:
checks against criteria
Generator:
revises
Coordinator:
accepts final result
Anthropic’s harness-design write-up describes a "planner, generator, and evaluator" architecture for long-running application, using structured criteria and handoff artifacts to improve outcomes. (Anthropic)
Use this when quality matters and output can be evaluated:
- Code generation,
- Design generation,
- Data migration planning,
- Policy analysis,
- Test generation,
- Architecture review.
Trap:
An evaluator agent should use explicit criteria. “Ask another agent if it looks good” is weaker than “evaluate against acceptance criteria, tests, security requirements, and source evidence.”
10. Sequential Handoff Pattern
Sequential handoff is not the same as parallel fan-out.
Agent A: investigate
↓
Agent B: design
↓
Agent C: implement
↓
Agent D: test
↓
Coordinator: final synthesis
Use sequential handoff when each step depends on the previous result.
The handoff should be structured:
{
"objective": "Implement OAuth callback validation",
"completed_work": ["Reviewed auth flow", "Identified missing state validation"],
"remaining_work": ["Add state token verification", "Add tests"],
"files_changed": ["auth/callback.ts"],
"risks": ["Backward compatibility with old sessions"],
"evidence": ["test failure output", "source file references"],
"next_agent_instructions": "Implement only the missing validation and tests."
}
Anthropic’s long-running harness work emphasizes structured artifacts, clean state, progress files, and incremental handoffs so later agents can understand the work quickly after context resets. (Anthropic)
11. Multi-Agent Vs Deterministic Workflow
This is a major distinction.
| Architecture | Best for |
|---|---|
| Deterministic workflow | Known steps, strict order, compliance, repeatability |
| Single agent | Flexible reasoning but small enough for one context |
| Multi-agent | Flexible, complex, decomposable work |
| Hybrid | Agent proposes plan; deterministic system enforces policy |
Examples:
| Scenario | Best choice |
|---|---|
| “Always validate identity, then check balance, then issue refund under $50” | Deterministic workflow |
| “Research unknown causes of latency across services” | Multi-agent |
| “Summarize one support ticket” | Single agent |
| “Generate migration plan, then require human sign-off before execution” | Hybrid |
| “Analyze 30 independent files” | Multi-agent fan-out |
| “Perform irreversible deletion” | Deterministic approval gate |
Heuristic:
Multi-agent orchestration gives flexibility. Deterministic workflows give control. Use each where it belongs.
12. Tool and Permission Boundaries
A common pitfall is giving every agent every tool.
Better pattern:
| Agent | Tools |
|---|---|
| Researcher | read-only search, document fetch |
| Coder | file edit, test runner |
| Security reviewer | read-only repo, static analysis |
| Deployer | deployment tool, but only after approval |
| Coordinator | delegation and synthesis, not necessarily all operational tools |
Tools and MCP servers should be agent-scoped: declare only the tools and servers each agent needs in that agent’s definition.
Security principle:
Least privilege applies per agent, not just per application.
Anthropic’s containment guidance says agent systems need hard environmental boundaries such as sandboxes, filesystem boundaries, and egress controls, because model-level defenses alone cannot be the only protection. (Anthropic)
13. Failure Handling
Multi-agent systems add new failure modes.
| Failure mode | Correct handling |
|---|---|
| One subagent fails | Retry, reassign, or continue with caveat depending on criticality |
| Conflicting outputs | Coordinator requests evidence or sends to evaluator |
| Agent exceeds scope | Enforce task contract and discard irrelevant output |
| Missing source/evidence | Ask subagent for evidence or rerun with stricter output schema |
| Tool permission required | Route permission event to user/human approval |
| Subagent loops | Cap iterations / terminate thread |
| Partial completion | Return partial result with known gaps or continue targeted work |
Blocking permission events should be routed back to the correct subagent/thread that raised them, so the right context resumes once the decision is made.
14. Cost and Latency
Multi-agent systems can be faster but are more expensive in total tokens.
Use multi-agent orchestration when the benefit outweighs:
- extra prompts
- extra context
- duplicate reasoning
- synthesis overhead
- coordination overhead
- harder debugging
- larger blast radius
Anthropic’s orchestration-mode example explicitly warns that fan-out multiplies token usage and should be reserved for work that justifies the cost. (Claude Platform)
Heuristic:
| Requirement | Good answer |
|---|---|
| Need fastest answer to simple question | Single agent |
| Need broad exploration across independent sources | Parallel agents |
| Need predictable low cost | Avoid broad fan-out |
| Need high confidence in complex output | Generator + evaluator |
| Need strict budget | Limit subtask count and concurrency |
15. High-Yield Traps
| Trap | Why wrong | Better answer |
|---|---|---|
| “Use multi-agent for every hard task” | Adds cost and coordination complexity | Use only when decomposable/specialized |
| “Give every agent all tools” | Violates least privilege | Scope tools per agent |
| “Parallelize dependent tasks” | Causes invalid assumptions | Use sequential handoff |
| “Concatenate subagent outputs” | No synthesis or conflict resolution | Coordinator synthesizes |
| “Use evaluator with no criteria” | Weak evaluation | Use explicit rubric/tests |
| “Use model judgment for compliance” | Probabilistic enforcement | Programmatic guardrails |
| “Ignore failed subagent” | Hidden quality risk | Retry, reassign, or report gap |
| “Let agents edit same files freely” | Merge conflicts and overwrites | Isolate, lock, or coordinate writes |
| “Fan out without cap” | Cost/runaway risk | Concurrency and subtask caps |
| “Share full context with all agents” | Noise and leakage | Minimal scoped context |