Multi Agent Systems When One Llm Isn T Enough

Your Agent Hit a Wall — Why One Brain Isn’t Enough

You’ve done the work. In Part 1, you built a ReAct agent that could think step-by-step and call tools. In Part 2, you gave it a full toolbox — weather lookups, math calculations, database queries. In Part 3, you added memory so it could remember who you are and what you’ve been working on.

Your agent is powerful now. It can handle a single, well-scoped task beautifully. Ask it “What’s the weather in Tokyo?” and it calls the weather tool and hands you the answer. Ask it “Remember my name is Alice working on Project Phoenix” and it stores that fact and recalls it later.

But then you push it further. You type:

“Research the latest causal inference methods, then write a tutorial about them, then check your own work for errors.”

And your agent… struggles. It gets confused halfway through. It runs out of context window space. It produces a mediocre tutorial that’s too technical in some places and too vague in others. It can’t catch its own mistakes because it’s too close to the material.

You’ve hit the ceiling. A single LLM agent, no matter how well-built, has fundamental limits:

• Context crowding: One agent has to hold the research, the writing plan, and the review criteria all in the same context window. That’s a lot of tokens.
• Self-correction blindness: It’s hard for an LLM to critique its own output. It doesn’t naturally spot its own errors the way a separate reviewer would.
• Task-switching overhead: Every time the agent switches from “researcher” mode to “writer” mode to “checker” mode, it has to reorient itself. That’s wasted tokens and potential confusion.

This isn’t a failure of your code. It’s a fundamental limitation of the “one brain” approach. Real-world tasks — especially in production — need specialized roles. A researcher thinks differently than a writer. A writer thinks differently than a reviewer. No single LLM call is equally good at all of them.

Anthropic’s production post drives this home: “multi-agent system… outperforming a single…agent by 90.2%” on complex tasks. And the 11x case study shows the same pattern: “moving from a single ReAct agent to a hierarchical LangGraph system” produced dramatic improvements in quality and reliability.

So what’s the solution? Split the job across multiple agents that talk to each other. Each agent stays focused on its specialty. The team covers the whole task.

In this article, you’ll build a multi-agent research-assistant team. It will have a coordinator, a math specialist, a chemistry specialist, and a reviewer. They’ll work together to research a topic, write an explanation, and check for errors — something no single agent could do well.

Let’s start with the intuition.

What Is a Multi-Agent System, Really?

Before we write any code, let’s build a clear mental model. What does it mean for agents to “talk to each other”?

Think of a team of humans working on a project. You don’t ask one person to do everything — you build a team. Same thing for LLMs.

There are two fundamental patterns:

Pattern 1: Group Chat (Debate)

Multiple agents independently solve the same problem, then share their answers and converge on the best one. This is like a panel of experts reviewing the same evidence — each expert brings their own perspective, and the group discussion produces a better result than any single expert would.

Analogy: A panel of doctors reviewing the same patient case. Each doctor has the same training but different experience. They discuss and converge on the best diagnosis.

Pattern 2: Delegation (Coordinator + Specialists)

One agent (the coordinator) receives a complex task, breaks it into sub-tasks, and routes each sub-task to a specialized sub-agent. Each specialist only handles its own domain. The coordinator collects the results and presents the final answer.

Analogy: A project manager + a researcher + a writer + a reviewer. The project manager doesn’t do the research or the writing — they just make sure the right person does the right job.

The survey paper “Multi-Agent Collaboration Mechanisms” (arXiv 2501.06322) calls these patterns “cooperation” and “competition.” But you don’t need the taxonomy. You just need to know: when you have a complex job, you don’t ask one person to do everything — you build a team. Same thing for LLMs.

Here’s a quick textual illustration of the two patterns:

Pattern 1: Group Chat (Round-Robin)
  Agent A -> Agent B -> Agent C -> Agent A -> ...
  Each agent sees the full conversation history.
  All agents have the same goal.

Pattern 2: Coordinator + Specialists
  User -> Coordinator -> Math Specialist
                    -> Chemistry Specialist
                    -> Reviewer
  Coordinator decides who to route to.
  Specialists only see their own domain.

Now you have the mental model. Let’s build it.

AutoGen: Your First Multi-Agent Chat (Two Agents, 15 Lines)

AutoGen is a framework from Microsoft that makes multi-agent conversations easy. Its core abstraction is simple: an AssistantAgent (an LLM with a system prompt) and a UserProxyAgent (a human stand-in that can execute code and provide feedback). Together, they form a “conversation” — the most natural multi-agent primitive.

Let’s install it and build the simplest possible multi-agent system: a Writer and a Critic that collaborate on a blog post.

# Block 1: Install AutoGen
# Run this in your terminal before the next block
# pip install pyautogen

Now, let’s create our first two-agent team:

# Block 2: Two-agent group chat — Writer + Critic
# This block is self-contained

import autogen

# Configuration for the LLM
# You'll need to set your API key as an environment variable or pass it here
llm_config = {
    "config_list": [
        {
            "model": "gpt-4",
            "api_key": "your-api-key-here"  # Replace with your actual key
        }
    ]
}

# Create the Writer agent
# The system_message defines its role and personality
writer = autogen.AssistantAgent(
    name="Writer",
    system_message="You are a skilled technical writer. You write clear, engaging blog posts about data science topics. You focus on explaining concepts intuitively.",
    llm_config=llm_config,
)

# Create the Critic agent
# The UserProxyAgent can simulate a human reviewer
critic = autogen.UserProxyAgent(
    name="Critic",
    human_input_mode="NEVER",  # Fully automated for now
    max_consecutive_auto_reply=3,  # How many times the critic can reply before stopping
    system_message="You are a helpful critic. You review the writer's work and suggest improvements. You always ask for more examples and clearer explanations.",
    code_execution_config=False,  # Don't execute code
)

# Start the conversation
# The critic initiates by asking the writer to create a blog outline
critic.initiate_chat(
    writer,
    message="Write a blog post outline about why correlation doesn't imply causation. Include an introduction, three main points, and a conclusion.",
    max_turns=2,  # Limit the conversation to 2 rounds
)

When you run this, you’ll see a conversation unfold. The Writer proposes an outline. The Critic says “Add more examples — like the ice cream and drowning example.” The Writer revises. The final output is a better outline than either agent could produce alone.

What’s actually going on here?

• The AssistantAgent is the LLM with a specific role (Writer). Its system prompt tells it to focus on clear, engaging writing.
• The UserProxyAgent is a stand-in for a human. It can be configured to either run automatically (like we did) or wait for human input.
• The max_consecutive_auto_reply parameter controls how many times the critic can reply before the conversation stops. This prevents infinite loops.
• The conversation is a back-and-forth: each agent sees the full history and responds.

The improvement wasn’t magic. Separating the writing role from the reviewing role forced the model to produce a second draft — something a single agent rarely does spontaneously. The Critic’s system prompt explicitly says “always ask for more examples,” so it does exactly that.

This group chat pattern is one way to organize multi-agent systems. But what happens when you need more than two roles?

Divide and Conquer: The Coordinator + Specialists Pattern

Group chat works well for small teams with similar expertise. But what if you need a math specialist who only answers math questions, and a chemistry specialist who only answers chemistry questions? You don’t want the math specialist wasting tokens reading chemistry prompts.

Enter the coordinator + specialists pattern. This is like a company with departments. The coordinator is the receptionist who knows which department to forward a call to. Each specialist only handles its own domain.

Let’s build it:

# Block 4: Coordinator + Math Specialist + Chemistry Specialist
# This block is self-contained

import autogen

# LLM configuration
llm_config = {
    "config_list": [
        {
            "model": "gpt-4",
            "api_key": "your-api-key-here"  # Replace with your actual key
        }
    ]
}

# Create the Math Specialist
# Its system prompt is narrow — it only answers math questions
math_specialist = autogen.AssistantAgent(
    name="MathSpecialist",
    system_message="""You are a mathematics expert. You only answer questions about mathematics.
If the question is not about mathematics, say "I can't help you with that — I only handle math questions."
Provide step-by-step solutions when possible.""",
    llm_config=llm_config,
)

# Create the Chemistry Specialist
chemistry_specialist = autogen.AssistantAgent(
    name="ChemistrySpecialist",
    system_message="""You are a chemistry expert. You only answer questions about chemistry.
If the question is not about chemistry, say "I can't help you with that — I only handle chemistry questions."
Explain chemical concepts clearly.""",
    llm_config=llm_config,
)

# Create the Coordinator
# The coordinator decides which specialist to route to
coordinator = autogen.UserProxyAgent(
    name="Coordinator",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=1,  # Only route once per question
    system_message="""You are a coordinator. You receive questions and route them to the right specialist.
If the question is about math, route to MathSpecialist.
If the question is about chemistry, route to ChemistrySpecialist.
If the question is about something else, say "I can't handle that question."""",
    code_execution_config=False,
)

# Test the system
print("=== Test 1: Math question ===")
coordinator.initiate_chat(
    math_specialist,
    message="What is the derivative of x^2?",
    max_turns=1,
)

print("\n=== Test 2: Chemistry question ===")
coordinator.initiate_chat(
    chemistry_specialist,
    message="What is the chemical formula for water?",
    max_turns=1,
)

print("\n=== Test 3: Off-topic question ===")
coordinator.initiate_chat(
    math_specialist,
    message="What is the capital of France?",
    max_turns=1,
)

When you run this, you’ll see the routing in action. The math specialist gives a step-by-step derivative solution. The chemistry specialist explains water’s formula. And when asked about France’s capital, the math specialist politely declines.

What’s the big deal?

Think of it this way: the math specialist doesn’t need to understand chemistry prompts. Its context window stays clean and focused. The coordinator’s routing is naive (keyword matching in the system prompt), but it dramatically improves throughput — the specialist agents don’t waste tokens reorienting.

This pattern is production-proven. Anthropic’s orchestrator-worker system and the 11x sales architecture both use this approach. The 11x case study describes “one supervisor node plus four specialized sub-agents (researcher, positioning-report generator, LinkedIn writer, email writer)” — exactly the coordinator + specialists pattern.

But wait — this works well when you know the roles ahead of time. What about tasks where you need more flexible collaboration?

Debate: When You Want Multiple Opinions, Not Just a Team

Sometimes you don’t want to divide the work. You want multiple agents to independently solve the same problem, then share their answers and converge on the best one. This is the debate pattern.

Let’s use a classic puzzle to show why this matters:

A bat and a ball cost $1.10. The bat costs$1.00 more than the ball. How much is the ball?

A single LLM often answers $0.10 — which is wrong. The correct answer is$0.05 (ball = $0.05, bat =$1.05, total = $$1.10). The mistake happens because the LLM rushes to the intuitive but wrong answer.

Let’s see how debate fixes this:

# Block 3: Multi-agent debate for a math puzzle
# This block is self-contained

import openai  # You'll need to install: pip install openai
import os

# Set your API key
# In production, use environment variables: os.environ["OPENAI_API_KEY"]
client = openai.OpenAI(api_key="your-api-key-here")  # Replace with your actual key

def ask_agent(system_prompt, question):
    """Ask an LLM agent a question and return its answer."""
    response = client.chat.completions.create(
        model="gpt-3.5-turbo",  # Using a cheaper model for the demo
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": question}
        ],
        temperature=0.7,  # Higher temperature = more diverse answers
    )
    return response.choices[0].message.content

# The puzzle
puzzle = "A bat and a ball cost $1.10. The bat costs $1.00 more than the ball. How much is the ball?"

# Round 1: Three agents answer independently
print("=== Round 1: Independent answers ===")
answers = []
for i in range(3):
    system_prompt = f"You are Agent {i+1}. You answer questions independently."
    answer = ask_agent(system_prompt, puzzle)
    answers.append(answer)
    print(f"Agent {i+1}: {answer}")

print()

# Round 2: Agents see each other's answers and revise
print("=== Round 2: Debate round ===")
for i in range(3):
    # Build a prompt that shows the other agents' answers
    other_answers = [answers[j] for j in range(3) if j != i]
    debate_prompt = f"""{puzzle}

You are in a debate round. The other agents said:
- Agent {1 if i != 0 else 2}: {other_answers[0]}
- Agent {2 if i != 1 else 3}: {other_answers[1]}

Review your answer. If you think you were wrong, revise your answer. If you think you were right, explain why.
"""
    
    system_prompt = f"You are Agent {i+1}. You are participating in a debate."
    revised_answer = ask_agent(system_prompt, debate_prompt)
    print(f"Agent {i+1} after debate: {revised_answer}")

When you run this, you’ll see something interesting. In Round 1, one agent might say $0.10, another might say$0.05, and a third might say something else entirely. But in Round 2, after seeing each other’s answers, the agents often converge on the correct answer ($$0.05).

What’s actually going on here?

The debate pattern doesn’t require specialized roles — just simple repetition and a chance to be corrected. The Du et al. (2023) paper showed that this approach improves factuality and reasoning on benchmarks. The key insight: multiple independent perspectives, even from the same model, produce better results than a single pass.

Here’s the catch: you pay for multiple LLM calls for every task. The debate pattern is expensive. But for tasks where accuracy matters — fact-checking, reasoning puzzles, creative brainstorming — the cost is worth it.

The Hard Part: Adding a Human in the Loop (and Why Debugging Is a Nightmare)

So far, all our agents have been fully automated. They talk to each other, route tasks, and converge on answers — all without human intervention.

But what if you need to step in and say “no”?

Autonomous multi-agent systems are brittle. They can hallucinate, misroute, or go in circles without a human check. The TAMAS paper warns: “Collaborative multi-agent systems are highly vulnerable to adversarial propagation between agents.” One agent’s mistake can cascade through the whole system.

AutoGen’s designers knew this. That’s why the UserProxyAgent has a human_input_mode parameter with three settings:

• "NEVER": Fully automated. The agent never asks for human input.
• "TERMINATE": The agent asks for human input only when it wants to end the conversation.
• "ALWAYS": The agent asks for human input after every response.

Let’s see how human_input_mode='ALWAYS' works in practice:

# Block 5: Human-in-the-loop — critic asks for human input
# This block is self-contained

import autogen

# LLM configuration
llm_config = {
    "config_list": [
        {
            "model": "gpt-4",
            "api_key": "your-api-key-here"  # Replace with your actual key
        }
    ]
}

# Create the Writer agent
writer = autogen.AssistantAgent(
    name="Writer",
    system_message="You are a skilled technical writer. You write clear, engaging blog posts about data science topics.",
    llm_config=llm_config,
)

# Create the Critic agent with human_input_mode='ALWAYS'
# This means the critic will ask for human input after every response
critic = autogen.UserProxyAgent(
    name="Critic",
    human_input_mode="ALWAYS",  # Human must approve or edit every response
    max_consecutive_auto_reply=3,
    system_message="You are a helpful critic. You review the writer's work and suggest improvements.",
    code_execution_config=False,
)

# Start the conversation
# The critic will ask for human input after the writer responds
critic.initiate_chat(
    writer,
    message="Write a blog post about the California housing dataset.",
    max_turns=2,
)

When you run this, here’s what happens:

1. The Writer proposes a blog post about the California housing dataset.
2. The Critic pauses and asks for your input (in the terminal).
3. You type: “Actually, the California housing dataset doesn’t exist. Use the Boston housing dataset from scikit-learn instead.”
4. The Critic passes your correction to the Writer.
5. The Writer revises its plan to use the Boston housing dataset.

This is the safety valve. You’re not just a passenger — you can steer. But it means you have to be watching, and that doesn’t scale to hundreds of agents.

The AutoGen Studio paper highlights this as a major pain point: debugging multi-agent conversations is hard. The human_input_mode is part of the solution, but it’s not a silver bullet. In production, you need observability, resumability, and a way to intervene without restarting the whole conversation.

Bringing It All Together: Your First Multi-Agent Research Assistant

Now let’s build the complete system. A multi-agent research assistant that:

1. Receives a user query
2. Routes it to the right specialist (math or chemistry)
3. The specialist returns a draft answer
4. A reviewer checks for errors and clarity
5. The coordinator presents the final answer

# Block 6: Complete multi-agent research assistant
# This block is self-contained

import autogen

# LLM configuration
llm_config = {
    "config_list": [
        {
            "model": "gpt-4",
            "api_key": "your-api-key-here"  # Replace with your actual key
        }
    ]
}

# Create the Math Specialist
math_specialist = autogen.AssistantAgent(
    name="MathSpecialist",
    system_message="""You are a mathematics expert. You only answer questions about mathematics.
If the question is not about mathematics, say "I can't help you with that."
Provide step-by-step solutions when possible.""",
    llm_config=llm_config,
)

# Create the Chemistry Specialist
chemistry_specialist = autogen.AssistantAgent(
    name="ChemistrySpecialist",
    system_message="""You are a chemistry expert. You only answer questions about chemistry.
If the question is not about chemistry, say "I can't help you with that."
Explain chemical concepts clearly.""",
    llm_config=llm_config,
)

# Create the Reviewer
reviewer = autogen.AssistantAgent(
    name="Reviewer",
    system_message="""You are a reviewer. Your job is to check the specialist's answer for:
1. Errors: Is the answer factually correct?
2. Clarity: Is the answer easy to understand?
3. Completeness: Does the answer fully address the question?
If you find any issues, explain what needs to be fixed.""",
    llm_config=llm_config,
)

# Create the Coordinator
coordinator = autogen.UserProxyAgent(
    name="Coordinator",
    human_input_mode="NEVER",
    max_consecutive_auto_reply=2,  # Route, then review
    system_message="""You are a coordinator. You receive questions and route them to the right specialist.
If the question is about math, route to MathSpecialist.
If the question is about chemistry, route to ChemistrySpecialist.
After the specialist answers, route to Reviewer for a quality check.""",
    code_execution_config=False,
)

def main():
    """Main loop for the research assistant."""
    print("Multi-Agent Research Assistant")
    print("Type 'quit' to exit.")
    print()
    
    while True:
        user_query = input("Your question: ")
        if user_query.lower() == "quit":
            break
        
        # Route to the right specialist
        if "math" in user_query.lower() or "derivative" in user_query.lower() or "equation" in user_query.lower():
            specialist = math_specialist
            print("\n[Coordinator] Routing to MathSpecialist...")
        elif "chemistry" in user_query.lower() or "chemical" in user_query.lower() or "molecule" in user_query.lower():
            specialist = chemistry_specialist
            print("\n[Coordinator] Routing to ChemistrySpecialist...")
        else:
            print("\n[Coordinator] I can't determine the right specialist for this question.")
            continue
        
        # Get the specialist's answer
        print(f"[Coordinator] Asking {specialist.name}...")
        coordinator.initiate_chat(
            specialist,
            message=user_query,
            max_turns=1,
        )
        
        # Route to reviewer
        print("\n[Coordinator] Routing to Reviewer for quality check...")
        coordinator.initiate_chat(
            reviewer,
            message=f"Review the following answer for errors and clarity: {user_query}",
            max_turns=1,
        )
        
        print("\n" + "="*50 + "\n")

if __name__ == "__main__":
    main()

When you run this and ask “What is a confounder in causal inference?”, here’s what happens:

1. The coordinator checks the query. It contains “causal inference” — not obviously math or chemistry. The coordinator says it can’t determine the right specialist.

But if you ask “What is the derivative of x^2?”:

1. Coordinator routes to MathSpecialist.
2. MathSpecialist gives a step-by-step solution.
3. Reviewer checks: “This is technically correct, but could include a plain-English explanation.”
4. Coordinator asks MathSpecialist to revise.
5. Final answer includes both the math and the intuition.

The team didn’t just answer the question — it iterated on the quality of the answer because there was a separate agent whose job was to check for clarity. A single agent rarely questions its own output.

A Quick Framework Comparison

You’ve been using AutoGen. But CrewAI does the same thing differently. Here’s the key difference:

• AutoGen: Conversation-driven. Agents talk to each other in a group chat. Coordination emerges from the conversation. Great for prototyping and exploration.
• CrewAI: Role-based. You define agents with specific roles, tasks, and a deterministic sequence. Coordination is top-down and predictable. Great for auditable workflows.

The ZenML comparison puts it well: “CrewAI for auditable/predictable workflows; AutoGen for rapid prototyping where you don’t know the solution path.”

Which should you use? It depends on your problem:

• Are you building a process you understand? Use CrewAI.
• Are you exploring a space you don’t? Use AutoGen.

What You Learned and Where You’re Headed Next

Let’s recap what you’ve learned:

1. The “one brain” ceiling: A single agent struggles with multi-step, multi-perspective work. Real-world tasks need specialized roles.
2. Three patterns for multi-agent systems:
   • Group chat (AutoGen): Agents talk in a round-robin. Good for collaboration and iteration.
   • Coordinator + specialists: One agent routes tasks to domain experts. Good for well-defined roles.
   • Debate: Multiple agents independently solve the same problem, then converge. Good for accuracy and fact-checking.
3. Key trade-offs: More agents = more flexibility but also more complexity, token cost, and failure surface (as the TAMAS paper warns).
4. Frameworks: AutoGen (conversation-driven, good for prototyping) vs. CrewAI (role-driven, good for predictable processes).

In Part 5, you’ll take this multi-agent system and deploy it as a web API with state management, rate limiting, and logging — production-ready.

Check Your Understanding

Remember: What are the three patterns for multi-agent systems described in this article?

Understand: Explain why a single agent struggles with tasks that require both research and review.

Apply: Given a task to “research the latest AI papers, write a summary, and check for errors,” design a multi-agent system. What roles would you create? How would they communicate?

Analyze: Compare the group chat pattern (AutoGen) with the coordinator + specialists pattern. When would you choose one over the other?

Evaluate: The TAMAS paper warns that “collaborative multi-agent systems are highly vulnerable to adversarial propagation between agents.” What does this mean in practice? How would you mitigate this risk?

Create: Design a multi-agent system for a customer support chatbot. It should handle billing questions, technical support, and account management. What agents would you create? What pattern would you use? Write a high-level design document.