OpenTelemetry for AI agents

AI agents look unusual from the outside, but most of their operational problems are familiar. A request starts somewhere, work fans out to dependencies, one step gets slow or fails, and somebody has to work backward from the symptom. OpenTelemetry already gives us a useful model for that work: traces, spans, attributes, events, logs, and metrics.

The useful shift is to treat an agent run as part of the application trace instead of building a separate telemetry universe for AI. The HTTP request that triggered a support workflow, the database lookup it performed, the model call, the MCP tool, and the final write can belong to the same trace. That makes the AI portion easier to reason about because it keeps the surrounding system visible.

Start with the runtime, not the model

Instrument the service or worker that hosts the agent first. Standard OpenTelemetry instrumentation can cover HTTP clients, servers, databases, queues, and other dependencies without much custom code. Give the process a stable service.name, plus environment and release identity that match the rest of the application.

Once that foundation works, add spans for the operations that are specific to the agent. A model invocation, tool execution, workflow handoff, retrieval step, or planning operation usually deserves a name because automatic instrumentation cannot infer the business meaning.

const run = telemio.agent.startAgent("support-agent", {
  "gen_ai.provider.name": "openai",
  "gen_ai.request.model": "gpt-5"
})

const tool = telemio.agent.startTool("lookup_customer", {
  "gen_ai.tool.type": "datastore"
}, run)

await tool.end()
run.setStatus("ok")
await run.end()

The code above uses @telemio/sdk for explicit agent semantics. It works best beside standard OpenTelemetry instrumentation, not as a replacement for it.

Record the shape of the run

You can answer a surprising number of production questions with metadata. Keep the provider, model, operation, agent or workflow name, tool name, token counts, estimated cost, retry count, status, and duration. Add identifiers that let you join the run to the request, account, job, release, or experiment that created it.

This is enough to investigate questions such as: Which tool is failing most often? Did latency increase after a release? Is one model using more tokens for the same workflow? Are retries hiding a dependency problem? Does a slow browser action lead into an expensive agent run?

MCP and tools are dependencies

An MCP call is still a dependency call. Give it a span with a stable tool or server name. Record success or failure, duration, and the operation that was attempted. If the tool reaches a database or third-party API, keep the downstream trace context when the protocol and runtime allow it.

A tool result does not need to be copied into telemetry. If you need to know whether the same context or result was reused, a stable identifier or fingerprint can provide that relationship without exporting the body.

Keep content capture deliberate

Agent systems can touch support tickets, source code, customer records, credentials, and internal documents in a single run. Logging the full prompt or tool payload by default makes the observability system another store for all of that material.

A safer default is metadata-first capture. Telemio's agent helpers follow that model. When correlation matters, use stable IDs or a SHA-256 fingerprint. When a team genuinely needs sampled content, make the sampling and redaction policy explicit and keep it separate from the basic trace path.

What a useful trace should answer

When you open an agent trace, you should be able to tell what started the run, which application release handled it, which model was called, which tools ran, where time was spent, what failed, how often the system retried, and what the run cost in tokens or money. You should also be able to move outward into the surrounding service, database, queue, browser session, or infrastructure event.

That last part is why OpenTelemetry is a good base. Agent observability becomes more valuable when it stays connected to ordinary software observability.

Set it up

Install the Telemio CLI with npm i -g telemio and run telemio setup otel for repository-aware setup. For explicit JavaScript or TypeScript agent spans, install @telemio/sdk. Coding agents can read the full setup instructions at /agent/setup.md.