An AI agent can return the right answer and still be unhealthy. It may have retried a model call four times, queried the wrong data source, waited on a slow tool, or spent ten times the normal token budget. A conventional uptime dashboard will miss most of that.
The tempting response is to record every prompt, model response, retrieved document, and tool result. That makes debugging easy, but it can also copy customer data, credentials, commercial records, or employee conversations into an observability system with a broader audience and longer retention than the production application.
A better default is structural observability: capture the shape, timing, cost, control decisions, and outcome of each run while leaving content out. OpenTelemetry's current generative-AI conventions provide useful common names for this data. They also mark message content, system instructions, retrieval queries, tool arguments, and tool results as potentially sensitive. Treat that warning as an architecture rule, not a documentation footnote.
Start with the questions operations must answer
Do not begin with a vendor dashboard. Write down the incident and business questions the telemetry must answer:
- Which workflow, agent version, model, and tool combination produced the failure?
- Where did the run spend time: queueing, model inference, retrieval, approval, or tool execution?
- Did a retry, loop, timeout, policy denial, or malformed tool call occur?
- How many input and output tokens did the run consume, and what was its estimated cost?
- Did the agent complete the business outcome, produce a result for review, or stop safely?
- Can an operator correlate the trace with the business record without copying that record into telemetry?
Those questions define a useful minimum. They also prevent teams from collecting content simply because an SDK makes it available.
Build one trace around the business run
Create one root span for the agent run, then child spans for operations with a meaningful duration. A practical tree is:
agent run
├── model call
├── retrieval
├── tool call
├── approval wait
├── model call
└── outcome recording
Use stable workflow and version names rather than user text in span names. Record the agent identifier, workflow version, environment, tenant pseudonym, and a non-sensitive correlation ID on the root. Add the OpenTelemetry gen_ai.operation.name, provider, requested model, response model, finish reasons, and input/output token counts to model spans when available.
For tool spans, record a controlled tool name, operation class such as read or write, attempt number, duration, result status, and error.type. Record whether approval was required and whether it was granted, denied, or expired. Do not put arguments, results, URLs with query strings, database statements, or exception messages into attributes by default.
This structure shows whether the delay belongs to the model, orchestration, a human decision, or a downstream system. It also fits the broader operating model for enterprise AI agents: identity, authority, and evidence must stay connected throughout execution.
Measure four layers, not one
1. Reliability and latency
Track run duration and success rate, plus duration and error rate for model, retrieval, and tool operations. Count timeouts, cancellations, retries, policy denials, invalid tool arguments, exhausted turn limits, and fallback-model use. For streaming interfaces, time to first chunk can expose a degraded experience that total duration hides.
Use percentiles by workflow and model, not a single fleet average. A customer-support lookup and a research workflow have different healthy ranges.
2. Usage and cost
OpenTelemetry defines gen_ai.client.operation.duration and gen_ai.client.token.usage; model spans can also carry input and output token usage. Convert usage to estimated cost in a controlled pricing layer rather than attaching a high-cardinality price table to every span. Track cost per run, per successful outcome, and per workflow version.
Alert on sudden changes in tokens per successful outcome, repeated calls, and tool loops. A cheap failed run is not efficient, and a successful but unusually expensive run may reveal a prompt or routing regression. Apply the same budget discipline described in our AI credit and usage budget guide.
3. Safety and control behavior
Count attempted and executed tool calls separately. Measure approval requests, denials, expired approvals, policy blocks, and stop-control activations. Keep tool identities low-cardinality and classify side effects: read, propose, write, send, purchase, or delete.
These signals reveal an agent that repeatedly asks for forbidden actions even when the policy layer blocks it. Connect alerts to a tested AI agent incident-response and kill-switch procedure, so an operator can stop queued and new work without searching for the right runtime owner during an incident.
4. Business outcomes and quality
Infrastructure success is not task success. Emit a small, governed outcome vocabulary such as completed, needs_review, rejected, abandoned, or failed. Add a reason code and, where appropriate, an evaluation name and score. Keep free-text reviewer notes in the system of record, not the span.
Measure completion rate, human escalation rate, correction rate, and cost per accepted outcome. Sample accepted and rejected runs for offline evaluation using references held in an access-controlled evaluation store. This connects technical changes to the result users actually receive.
Keep content opt-in and isolated
Prompts and outputs are not required for useful tracing. OpenTelemetry explicitly treats input messages, output messages, system instructions, retrieval queries, tool arguments, and tool results as sensitive or potentially sensitive. OpenAI's Agents SDK similarly allows spans to remain enabled while excluding LLM and tool inputs and outputs.
Adopt these defaults:
- Content capture is off in production. Enable it only for an approved workflow, environment, and time window.
- Use an allowlist before export. Export known-safe fields rather than trying to redact every possible secret after collection.
- Fail closed. If filtering or serialization fails, drop the telemetry batch instead of sending the original payload.
- Pseudonymize correlation IDs. Never place emails, account numbers, ticket subjects, or document names in span IDs, names, or labels.
- Separate destinations and retention. Content-bearing diagnostic traces need narrower access, shorter retention, audit logs, and deletion procedures.
- Control cardinality. Keep user IDs, raw URLs, document IDs, and arbitrary errors out of metric labels; use traces or the system of record for investigation.
Hashing sensitive values is not automatically anonymization. A small or predictable value set can often be matched. Use opaque application-generated IDs and keep the mapping in the source system.
Roll out with a privacy test and a failure test
Instrument one important workflow first. Run a synthetic set containing canary secrets, personal data, malformed tool arguments, a timeout, a denied action, and a forced retry loop. Verify that operators can reconstruct the control flow and diagnose every failure without seeing the protected content. Search the observability backend for the canary values; the acceptable result is zero matches.
Then define service objectives around outcomes: accepted completion rate, p95 run duration, tool-error rate, and cost per accepted outcome. Assign an owner to each alert and link it to a response procedure. Review new span attributes like production schema changes, because a harmless-looking label can become a durable data leak.
Good agent observability is not a transcript archive. It is a controlled evidence system. If your traces show what ran, which controls applied, where time and money went, how the run ended, and which business outcome followed, you can operate agents responsibly without turning every customer interaction into a second data store.
