On this page
The four kinds of tool
1. Tracing and evaluation platforms. You add their SDK, or OpenTelemetry instrumentation, to your code and they record each run as a tree of steps: model calls, tool calls, retrieval, latency and cost. Most add evaluations, datasets and prompt management. They are built for building and debugging. Examples: Langfuse, LangSmith, Arize Phoenix, Opik, Weights & Biases Weave, and AgentOps, which focuses on agent session replays.
2. OpenTelemetry instrumentation into your existing stack. If your company already runs an observability platform, you can emit LLM spans in the standard OpenTelemetry format and keep one place for all telemetry. OpenLLMetry is a set of OpenTelemetry instrumentations for model providers and vector databases that can export to Datadog, Honeycomb and other backends.
3. AI gateways. A proxy between your app and the model providers: you change the base URL and every request passes through it. Gateways route between providers, retry and fall back, issue virtual keys, track spend and apply guardrails; logging comes with the position. Examples: LiteLLM, Portkey’s AI Gateway and Helicone.
4. Agent control panels. A library beside each agent reports its runs and heartbeats, and the panel can change the agent while it runs: pause, stop, switch its model, roll out a prompt, or pause it automatically on a spend or error alert. Agent Control Panel is this kind. It is built for operating agents in production rather than for debugging individual traces.
Side by side
| Tracing and evaluation | OpenTelemetry to your stack | AI gateway | Control panel | |
|---|---|---|---|---|
| How it connects | SDK or OpenTelemetry in your code | Instrumentation library, your exporter | Change the base URL; requests go through it | SDK wraps each agent run |
| In the request path | No | No | Yes | No |
| Best at | Step-by-step traces, evaluations, datasets | One home for all telemetry | Routing, fallbacks, keys, per-request policy | Per-agent health, cost and alerts; acting on running agents |
| Can change a running agent | Usually not (prompt management is the common exception) | No | Per request: route, block, limit | Per agent: pause, stop, model, prompt |
| If the tool is down | You lose telemetry; the agent keeps working | You lose telemetry | Model calls fail unless you build a bypass | Agents keep working (fail-open) |
These are typical patterns, not rules: several tracing platforms also offer a gateway, and gateways log requests. Check each product for the part you need. The trade-off between the last two columns is explained in fail-open vs in-path.
Open-source options
Licences as stated in each project’s repository on November 10, 2026. “Open core” means most of the code is under the licence shown and some enterprise features are under a separate licence.
| Project | Kind | Licence | How it connects |
|---|---|---|---|
| Langfuse | Tracing and evaluation | MIT, open core (enterprise folders separate) | Python and TypeScript SDKs, built on OpenTelemetry; self-hostable |
| Arize Phoenix | Tracing and evaluation | Elastic License 2.0 (source-available, not an OSI licence) | OpenTelemetry-based (OpenInference); self-hostable |
| Opik (Comet) | Tracing and evaluation | Apache-2.0 | SDK and framework integrations; self-hostable |
| Weave (Weights & Biases) | Tracing and evaluation | Apache-2.0 (the toolkit) | SDK |
| AgentOps | Agent session monitoring | MIT (the SDK) | Python SDK with framework integrations |
| OpenLLMetry | OpenTelemetry instrumentation | Apache-2.0 | OpenTelemetry, to any compatible backend |
| LiteLLM | AI gateway | MIT, open core (enterprise folder separate) | Proxy server or Python SDK |
| Portkey AI Gateway | AI gateway | MIT | Proxy |
| Helicone | Gateway and observability | Apache-2.0 | Change the base URL |
LangSmith is a commercial platform (its SDKs are open source). Agent Control Panel’s Node.js and Python SDKs are open source under Apache-2.0; the panel itself is a hosted service.
Seven questions to choose with
- Are you building or operating? While you build and debug, deep traces and evaluations matter most. Once agents serve users, you need to know which agent is unhealthy and be able to act on it.
- Who acts on what you see? If only the developer who wrote the agent will look, a tracing tool is enough. If an on-call engineer or an operator must stop an agent at 2 a.m., they need a control, not a trace.
- Can you accept a new hop in front of the model? A gateway adds a dependency to every call. If it must never take production down, prefer tools that sit beside the call.
- Where must data and keys live? Self-hosting, data residency and who holds the provider API keys narrow the list quickly.
- Which languages and frameworks? Check SDK support for your stack, or whether OpenTelemetry covers it.
- What should happen on a bad day? Alerts on spend, error rate and lost heartbeats, and ideally an automatic pause, limit the damage of a loop or a provider outage.
- What does it cost at your volume? Telemetry pricing is usually per event, trace or seat; estimate with real run counts.
Common combinations
- Tracing platform + control panel. Traces for developers, control and alerts for whoever runs production. The SDKs instrument the same code and do not depend on each other.
- Gateway + control panel. The gateway enforces per-request policy and holds keys; the panel tracks each agent and can pause or re-route it.
- OpenTelemetry to your APM + control panel. One telemetry home for the whole company, plus an operating layer for agents.
If you only need one tool today: debugging a prototype points to a tracing platform; agents already serving customers point to a control panel. More detail in Agent Control Panel vs Langfuse and LangSmith and how to monitor AI agents in production.
Frequently asked questions
What is the best AI agent monitoring tool?
It depends on the job. For debugging and evaluating agents, a tracing platform such as Langfuse or LangSmith. For one place for all telemetry, OpenTelemetry into your existing stack. For routing and per-request policy, a gateway. For operating agents in production, a control panel such as Agent Control Panel.
Are there open-source AI agent monitoring tools?
Yes. Langfuse (MIT, open core), Opik (Apache-2.0), OpenLLMetry (Apache-2.0), Helicone (Apache-2.0), LiteLLM (MIT, open core) and Portkey’s gateway (MIT) are examples. Arize Phoenix is source-available under the Elastic License 2.0.
Do I need a gateway to monitor AI agents?
No. Tracing SDKs, OpenTelemetry and control panels all record runs without sitting in the request path. A gateway is useful when you want routing, fallbacks or per-request policy.
Can a monitoring tool stop an AI agent?
Most monitoring tools only observe. A gateway can refuse individual requests. A control panel such as Agent Control Panel can pause or stop an agent, switch its model or roll out a prompt without a redeploy.