Observability for agentic runtimes

An agent runtime is still software. It receives work, calls dependencies, waits on networks, retries failures, changes state, and eventually produces an outcome. The model call is important, but it is only one part of the execution path.

The most useful observability setup keeps that execution path intact. A browser action or queue job can lead into an agent run, which can call a model, retrieve data, execute an MCP tool, hand control to another agent, and write back to a database. Those steps should be understandable as one system rather than five unrelated dashboards.

Instrument the orchestration boundary first

Start with the process that owns the run. Give it normal service identity and standard OpenTelemetry coverage for HTTP, queues, databases, runtime metrics, and outbound dependencies. Then add spans for agent-specific work that automatic instrumentation cannot infer.

request / job / user action
  -> agent or workflow run
    -> model invocation
    -> retrieval
      -> datastore
    -> tool execution
      -> MCP / HTTP / RPC dependency
    -> handoff or child agent
      -> model invocation
    -> persistence / side effect

This shape works whether the orchestration is a short synchronous request or a durable workflow that sleeps, resumes, and moves between workers.

Model calls are not the whole trace

A model span should answer practical questions: which provider and model were used, how long the call took, whether it failed, how many tokens were used, and why the response ended when that information is available. It should not hide the tool calls or service work that made the run succeed.

OpenTelemetry's GenAI conventions are the right place to start for model and agent semantics. They are still evolving, so instrumentation should follow the current conventions rather than freezing a private attribute vocabulary forever.

Tools and MCP are dependencies with business meaning

A tool execution has two useful layers. The agent-level span says what the tool meant to the workflow. The ordinary HTTP, RPC, database, or messaging span says what happened on the wire. Keep both when they answer different questions.

If a tool calls an MCP server, record the stable tool/server identity, operation, duration, status, and failure type. Do not copy the full arguments and response into telemetry by default. The downstream network spans can remain ordinary OpenTelemetry spans.

Long-running agents need durable identity

A background agent may wait in a queue, resume after a checkpoint, retry a step hours later, or hand work to another process. A single in-memory parent span is not always enough.

Keep stable workflow/run identity, attempt number, queue or job identity, checkpoint/resume identity, release version, status, token/cost totals, and handoff relationships. When a direct trace parent no longer makes sense, use propagated context when practical or span links and stable correlation IDs.

Multi-agent systems should show the handoff

Do not flatten every participant into a span called agent. Use low-cardinality agent or workflow names and separate run IDs. A trace should make it obvious which agent held control, which agent or tool received it next, and where latency or failure entered the workflow.

Keep content capture narrow

Prompt text, retrieved documents, tool arguments, source code, and tool results can contain far more sensitive information than ordinary infrastructure telemetry. They are also large and expensive to retain.

Use Telemio where it adds application meaning

In trusted JavaScript and TypeScript runtimes, @telemio/sdk can add explicit agent, tool, context, usage, flag, and experiment telemetry beside standard OpenTelemetry:

npm i @telemio/sdk

The SDK is not a replacement for OpenTelemetry auto-instrumentation. It is the application-context layer. For a repository-aware setup prompt and runtime checklist, see the agentic runtime guide.