Agent observability 2026 — three open-source platforms, three emphases
Langfuse MIT
6M+ SDK installs/month, 19k+ stars
tracing + prompt management
LLM-as-judge, user feedback, custom evals
session replays, annotation queues
playground + prompt experiments
best for: all-in-one with prompt loop
prompt versions tied to traces
Arize Phoenix Elastic 2.0
agent-specific evaluation focus
trace clustering + anomaly detection
RAG relevancy + retrieval eval
OpenInference auto-instrumentation
pairs with managed Arize AX
best for: RAG relevancy + drift
no prompt versioning alongside, not replacing
Comet Opik Apache 2.0
automated optimization + guardrails
automated prompt A/B experiments
guardrails: PII redaction, topical
LLM-judge hallucination detection
vendor benchmark: ~14x Langfuse
best for: optimization loop
vendor numbers directional; measure your own
common substrate OpenTelemetry GenAI spans (Lesson 23)
all three consume OTel GenAI spans; you can switch without re-instrumenting
content capture: external store + span references (PII-safe)
89% of orgs have agent observability (Maxim field data, 2026)
32% of respondents cite quality as the top production barrier
Datadog v1.37+ maps GenAI attributes natively mixed ops+ML teams fit here
tracing without evaluation is expensive logging. pair traces + LLM-judge + prompt versioning.