Observed signals

Observed signals are real outcomes — refunds, escalations, thumbs-down, items purchased. obsrv treats them as first-class metrics so they roll up next to your synthetic evals.

Recording an observed signal

# Boolean signals
client.record_metric(name="user_thumbs_up", trace_id="tr_…", passed=True)
client.record_metric(name="refund_requested", trace_id="tr_…", passed=False)

# Categorical signals
client.record_metric(
    name="resolution_outcome",
    trace_id="tr_…",
    value="resolved",   # or: "escalated" | "abandoned"
    kind="enum",
)

# Numeric signals
client.record_metric(name="csat", trace_id="tr_…", value=4.6, kind="numeric")

Common signals to wire up

Thumbs up / down on the response.
Conversation reopened within N minutes.
Refund or chargeback associated with the user's next session.
Escalation to a human agent.
Purchase / signup completed after the agent run.

Why observed matters

Synthetic evaluators are biased toward what the prompt thinks "good" looks like. Observed signals tell you whether users actually got what they wanted. Pair them and you have a complete picture.

← Synthetic metrics

Release tagging →