Multi-Agent Systems with CrewAI: Orchestrating AI Teams

Introduction

Multi-agent systems represent a significant evolution in AI application design. Instead of relying on a single agent to handle every aspect of a complex task, you can orchestrate multiple specialized agents that collaborate, each focusing on what it does best. CrewAI is a framework built specifically for this paradigm, making it straightforward to define agents with distinct roles, assign them tasks, and manage their interactions.

This article explores how CrewAI works and walks through building a complete research-to-report pipeline.

How CrewAI Works

CrewAI is built around four core concepts:

Agents

An agent is an autonomous unit that performs a specific role. Each agent has:

A role that defines its specialty (e.g., “Research Analyst”)
A goal that describes what it aims to achieve
A backstory that shapes its behavior and decision-making
Tools it can use to accomplish its tasks

Tasks

Tasks are specific assignments given to agents. Each task has:

A description of what needs to be done
An expected output that defines success criteria
An assigned agent responsible for completing it

Crews

A crew is the collection of agents and tasks that work together. The crew defines:

The process for executing tasks (sequential or hierarchical)
The agents and tasks involved
Configuration like verbosity and memory settings

Tools

Tools are capabilities agents can use, such as web search, file reading, API calls, or custom functions. CrewAI integrates with the LangChain tool ecosystem, giving you access to hundreds of pre-built tools.

Installing CrewAI

pip install crewai crewai-tools

Set your API keys:

export OPENAI_API_KEY=your-key-here
export SERPER_API_KEY=your-key-here  # For web search

Defining Agents

Let’s build a research-to-report pipeline with three agents: a researcher, a writer, and an editor.

from crewai import Agent, Task, Crew, Process
from crewai_tools import SerperDevTool, WebsiteSearchTool

# Initialize tools
search_tool = SerperDevTool()
web_tool = WebsiteSearchTool()

# Research Agent
researcher = Agent(
    role="Senior Research Analyst",
    goal="Find comprehensive, accurate information on the given topic",
    backstory="""You are an experienced research analyst with a talent for 
    finding relevant information and identifying key insights. You excel at 
    distinguishing credible sources from unreliable ones and always cite 
    your sources.""",
    tools=[search_tool, web_tool],
    verbose=True,
    allow_delegation=False
)

# Writer Agent
writer = Agent(
    role="Technical Writer",
    goal="Transform research findings into clear, engaging content",
    backstory="""You are a skilled technical writer who specializes in making 
    complex topics accessible. You write with clarity and precision, 
    always structuring content with logical flow and compelling narratives.""",
    verbose=True,
    allow_delegation=False
)

# Editor Agent
editor = Agent(
    role="Content Editor",
    goal="Ensure content is accurate, well-structured, and polished",
    backstory="""You are a meticulous editor with years of experience in 
    publishing. You check for factual accuracy, grammatical correctness, 
    and overall quality. You never compromise on standards.""",
    verbose=True,
    allow_delegation=True
)

Defining Tasks

Each task specifies what an agent should produce:

research_task = Task(
    description="""Conduct thorough research on {topic}. 
    Identify:
    1. Key concepts and definitions
    2. Current state and recent developments
    3. Major challenges and limitations
    4. Notable tools, frameworks, or solutions
    5. Future directions and trends
    
    Use web search to find recent, credible sources. 
    Provide a structured summary with citations.""",
    expected_output="A detailed research report with 5-10 key findings, "
                    "each with source citations and a brief explanation.",
    agent=researcher
)

writing_task = Task(
    description="""Using the research findings, write a comprehensive 
    article on {topic}. The article should:
    1. Have an engaging introduction
    2. Cover all key findings from the research
    3. Include practical examples or code snippets where relevant
    4. Be structured with clear headings
    5. Be between 1000-1500 words
    
    Write in a professional but accessible tone.""",
    expected_output="A well-structured article of 1000-1500 words "
                    "with clear headings and practical examples.",
    agent=writer
)

editing_task = Task(
    description="""Review and edit the article for:
    1. Factual accuracy (cross-check with research)
    2. Grammatical correctness
    3. Logical flow and structure
    4. Clarity and readability
    5. Consistent tone and style
    
    Provide the final polished version.""",
    expected_output="A polished, publication-ready article with no "
                    "errors and excellent readability.",
    agent=editor
)

Assembling the Crew

crew = Crew(
    agents=[researcher, writer, editor],
    tasks=[research_task, writing_task, editing_task],
    process=Process.sequential,
    verbose=True,
    memory=True,
    cache=True
)

result = crew.kickoff(inputs={"topic": "the impact of AI on software testing"})
print(result)

Sequential vs Hierarchical Processes

Sequential Process

Tasks execute in order, with each task’s output passed to the next. This is simple and predictable:

crew = Crew(
    agents=[researcher, writer, editor],
    tasks=[research_task, writing_task, editing_task],
    process=Process.sequential
)

Hierarchical Process

A manager agent coordinates the crew, delegating tasks and managing the workflow. This is more flexible but adds overhead:

crew = Crew(
    agents=[researcher, writer, editor],
    tasks=[research_task, writing_task, editing_task],
    process=Process.hierarchical,
    manager_llm=ChatOpenAI(model="gpt-4o"),
    verbose=True
)

Custom Tools

You can create custom tools for agents to use:

from crewai.tools import tool

@tool("Database Query Tool")
def query_database(query: str) -> str:
    """Query the company database with a SQL query.
    Args:
        query: A SQL query string
    Returns:
        Query results as a formatted string
    """
    # Your database connection logic here
    import sqlite3
    conn = sqlite3.connect("company.db")
    cursor = conn.execute(query)
    results = cursor.fetchall()
    conn.close()
    return str(results)

# Add to an agent
analyst = Agent(
    role="Data Analyst",
    goal="Extract insights from the company database",
    backstory="You are an expert at writing SQL and analyzing data.",
    tools=[query_database]
)

Best Practices

Write detailed backstories: The backstory significantly influences agent behavior. Be specific about expertise, preferences, and standards.
Define clear expected outputs: Vague expected outputs lead to inconsistent results. Specify format, length, and content requirements.
Use memory for context: Enable memory so agents can reference previous interactions and maintain context across tasks.
Limit delegation in sequential mode: Delegation works best in hierarchical mode. In sequential mode, keep agents focused on their assigned tasks.
Choose the right LLM per agent: Not every agent needs the most expensive model. Use smaller models for simpler tasks like formatting or basic analysis.

Conclusion

CrewAI makes multi-agent orchestration accessible by providing a clean abstraction over the complexity of agent coordination. By defining agents with clear roles, tasks with specific outputs, and crews with appropriate processes, you can build sophisticated AI workflows that leverage specialization and collaboration.

The research-to-report pipeline demonstrated here is just one example. The same patterns apply to any multi-step process: customer support routing, code review pipelines, data analysis workflows, and more. Start with a simple sequential crew, then add complexity as your needs grow.