Observability Leapfrog

Full Observability Platform

A ground-up rebuild of the observability stack, shipping Phases 0 through 3 in a single release.

Phase 0: Parity

• Scores, sessions, performance metrics, and P95 latency tracking
• thread_id support, OpenTelemetry semconv compatibility, and OTLP/HTTP ingest

Phase 1: Detail UX

• Per-request cost breakdown and token diff view
• Trace waterfall timeline and session playback
• Logprobs visualization and playground integration

Phase 2: Scale & Search

• Full-text search across traces and logs
• Metadata filtering with custom projections
• No-truncation span display for deep debugging

Phase 3: Online Eval Loop

• Sampling rules with AI-powered auto-scorers
• Multi-turn conversation simulation
• Annotation queues with keyboard shortcuts and time-on-item tracking
• Sampling-rule polling runner to close the online evaluation loop