/docs/concepts/trace

Trace

A trace is a single agent run — a record of every decision, tool call, message, and artifact, captured exactly as it happened.

What a trace contains

  • Identitytrace_id, optional project_id, and ingest timestamp.
  • Statussuccess, error, or running.
  • Run typeprod, dev, eval, backfill.
  • Steps — a tree of nested operations: LLM calls, retrievals, tools, custom spans.
  • Attachments — image, audio, video, file, and sensor artifacts referenced by step.
  • Metadata — arbitrary JSON you set; queryable as filters and surfaced in the UI.
  • Token usage & cost — rolled up across LLM steps.

The trace lifecycle

  1. Your SDK opens a trace context.
  2. Steps and attachments stream into the context.
  3. The context closes (success or failure) and the trace is committed.
  4. obsrv validates, persists to object storage, indexes for query, embeds for cluster discovery, and publishes on the live tail.

Trace identifiers

Trace IDs use ULIDs prefixed with tr_, so they sort lexicographically by ingest time. SDKs generate trace IDs automatically. Use run_id when you need to correlate obsrv traces with your own job ID, request ID, or replay ID.

What it's not

A trace is not a Datadog span tree. It's a structured, replayable snapshot of an agent run, optimized for inspection. Steps carry first-class fields (model, tokens, cost, attachments), not generic attributes.