Framework guide

Monitor and control CrewAI crews in production

CrewAI makes it short to define agents with roles, tasks and tools. In production you also need cost per crew, alerts, and a way to pause or change a crew without a deploy. This guide adds that with Agent Control Panel around crew.kickoff. The code was run against the real crewai package before we published it.

By the Agent Control Panel teamPublished 4 min read

On this page
  1. What you get
  2. Set up
  3. Wrap each kickoff
  4. Answer users while the agent is paused
  5. How we tested this code
  6. Frequently asked questions

What you get

  • Every run, with cost. Status, duration, tokens of every model call in the run, the model, the question and the answer, per agent.
  • Pause and stop. From the dashboard, or automatically when an alert rule fires on daily spend, error rate, a spend anomaly or a lost heartbeat.
  • Model swaps without a redeploy. Pick another model in the dashboard; the next run uses it.
  • Approved prompt changes. An owner or admin approves a new prompt; the next run uses it.
  • Liveness. A heartbeat every 30 seconds, so a crashed worker shows up as lost, not as quiet.

Model calls still go from your app to the provider with your own keys. If Agent Control Panel is unreachable, your agent keeps working: see fail-open vs in-path.

Set up

  1. Create an app in Agent Control Panel and copy its API key, agent ID and webhook secret. New apps start in monitor scope; allow the control scope when you are ready to pause and swap models.
  2. Install the SDK next to your framework: pip install agent-control-panel crewai.
  3. Create the client once at startup and call start(), as in the quickstart. The examples import it from your own acp.py.

Wrap each kickoff

The agent’s backstory comes from Agent Control Panel (with your own default), the model comes from it (with your own default), and the kickoff goes through the wrapper. A crew returns a CrewOutput; agent-control-panel 0.1.4 or later reads it as it is: the answer from raw and the tokens of every model call from token_usage.

from crewai import LLM, Agent, Crew, Task
from crewai.tools import tool

from acp import acp  # create_client(...) + acp.start(), see the quickstart

DEFAULT_BACKSTORY = "You are a helpful support assistant."  # used when ACP has no prompt or cannot be reached


@tool("Order status")
def order_status(order_id: str) -> str:
    """Look up an order by its id."""
    return get_order_status(order_id)  # your code


def answer(question: str) -> str:
    backstory = acp.fetch_prompt() or DEFAULT_BACKSTORY   # the approved prompt
    model = acp.get_effective_model("gpt-4.1-mini")        # a model swap from ACP, or your default
    agent = Agent(role="Support agent", goal="Answer the customer's question", backstory=backstory,
                  llm=LLM(model=model), tools=[order_status])
    task = Task(description="Answer this question: {question}", expected_output="A short answer", agent=agent)
    crew = Crew(agents=[agent], tasks=[task])

    # The SDK reads the CrewOutput as it is: the answer from raw, the tokens of every model call from token_usage
    # (agent-control-panel 0.1.4 or later).
    output = acp.wrap_agent_run(lambda: crew.kickoff(inputs={"question": question}),
                                user_input=question, model_used=model)
    return output.raw

CrewAI builds its own system prompt from the role, the backstory and the goal, so the approved prompt from Agent Control Panel goes in as the backstory. Pass the user’s question through inputs rather than formatting it into the task: CrewAI fills {placeholders} in task text, so braces in a question would be read as placeholders.

For a crew with several agents, treat the whole crew as one agent in Agent Control Panel: token_usage adds up every model call of the kickoff. Model ids for other providers carry a prefix in CrewAI (for example anthropic/…), so swap only to ids your LLM accepts. A pushed prompt applies at once when your app receives the signed webhook; otherwise within 60 seconds, and model swaps and pauses within one 30-second heartbeat.

Answer users while the agent is paused

While the agent is paused or stopped, the wrapper raises AgentPausedError before your code runs, so no model is called and no tokens are spent. Catch it where you answer the user:

from agent_control_panel import AgentPausedError

try:
    reply = answer(question)
except AgentPausedError:
    reply = "The assistant is paused right now. Please try again later."

After a restart, the first run waits up to three seconds for Agent Control Panel’s answer, so a pause holds across deploys. If no answer comes in time the run goes ahead.

How we tested this code

On November 11, 2026 we ran this example on Python 3.10 with crewai 1.15.39 and agent-control-panel 0.1.4 (the example as shown, with no adapter), against a stand-in OpenAI-compatible server and a stand-in Agent Control Panel server, so the test needed no API keys. CrewAI used native tool calling: the agent called the tool and then answered, and we checked that:

  • the run was reported once, as a success, with the tokens of both model calls, the model, the question and the answer;
  • a model swap and a pushed prompt were used on the next run (the prompt arrived in CrewAI’s system prompt as the backstory);
  • after a pause, the next call raised AgentPausedError and the model was not called.

What it does not do: Agent Control Panel records each kickoff as one record, not a trace of each agent and task. Async kickoffs, flows and streaming were not part of this test; for those, report the run with report_run when it ends.

Frequently asked questions

Does Agent Control Panel work with CrewAI?

Yes. Wrap crew.kickoff with wrap_agent_run. From agent-control-panel 0.1.4 the SDK reads the CrewOutput as it is: output.raw and output.token_usage.

Where does the managed prompt go in a crew?

Into the agent’s backstory. CrewAI builds its own system prompt from the role, backstory and goal, and the backstory carries the approved prompt from Agent Control Panel.

How do I monitor a crew with several agents?

Register the crew as one agent in Agent Control Panel and wrap the kickoff. The crew’s token_usage adds up every model call, so cost and pause apply to the whole crew.

Do my model calls go through Agent Control Panel?

No. CrewAI calls the provider directly with your key. Agent Control Panel only receives the run report and sends signed commands back.