AI Observability: What to Monitor in LLM Applications and AI Agents

ASA AI Team · Updated 2026-11-08 · 4 min read

AI observability is the ability to understand what an LLM application or AI agent is doing in production — which prompts it receives, what it retrieves, which tools it calls, what it answers, how much it costs and whether that behaviour is correct and safe. It extends classic logs, metrics and traces with signals specific to AI.

Key takeaways

Why classic monitoring is not enough

An LLM application can return HTTP 200 in 800 ms and still be wrong, leaking data or following an attacker's instructions. Uptime and latency say nothing about whether the content is correct and safe. AI observability adds visibility into prompts, context, reasoning steps, tool usage and output quality.

What to monitor

FamilySignalsWhy it matters
TracesSpans for each step: retrieval, model call, tool call, guardrail check, final answerReconstruct exactly what happened for any request or incident.
PerformanceLatency per step, time to first token, error and timeout rates, retriesFind slow tools, overloaded models and failing integrations.
CostInput and output tokens per request, user, feature and model; tool-loop countsControl spend and detect runaway agents or abuse.
QualityEvaluation scores (relevance, groundedness, correctness), user feedback, refusal rateDetect regressions after a model or prompt change.
SecurityPrompt-injection and jailbreak detections, guardrail blocks, sensitive data in outputs, unusual tool callsDetect attacks and data leaks early and alert the SOC.
DriftChanges in input topics, retrieval hit rates and answer distributions over timeSpot when real usage diverges from what was tested.

Instrumenting with OpenTelemetry

OpenTelemetry has semantic conventions for generative-AI operations, so model calls, token usage and tool invocations can be captured in a standard way and sent to any compatible backend. Instrument at three levels:

  1. Gateway level — every call to a model provider or local model passes through one point that records model, tokens, latency and errors.
  2. Application level — spans for retrieval, prompt assembly, tool calls and post-processing.
  3. Agent level — each reasoning step and tool decision, linked to the user and session.

Open-source building blocks

Privacy and retention

Prompts and responses often contain personal or confidential data. Redact or tokenise sensitive fields before storage, restrict who can read raw conversations, and set retention periods aligned with your data-protection obligations.

From observability to security

The same traces that explain a bad answer also reveal an attack: a sudden spike in tool calls, an agent reaching an unusual destination, a response containing secrets. Feeding AI telemetry into security monitoring — with detections mapped to OWASP and MITRE ATLAS — closes the loop between AI engineering and the SOC.

Frequently asked questions

What is the difference between AI observability and AI monitoring?

Monitoring tells you that something is wrong, through predefined metrics and alerts. Observability lets you understand why, by tracing every step of a request — prompts, retrieval, tool calls and outputs — and exploring it after the fact.

Should prompts and responses be logged?

Yes, for debugging, quality and security, but with redaction of sensitive data, strict access control and a defined retention period.

Which open-source tools support LLM observability?

OpenTelemetry for collection, and tools such as Langfuse, Arize Phoenix and OpenLLMetry for LLM-specific tracing and evaluation.

How does AI observability help security?

It provides the data to detect prompt injection, data leakage, abnormal tool usage and cost abuse, and to investigate incidents. Sending these events to the SIEM lets the SOC handle AI incidents like any other.

Related guides