Production Observability
Tracing in Production
Debug the trajectory, not the answer
When a user reports 'it did something weird', rerunning isn't debugging — it's gambling. Production LLM work needs the full trace: every model call (prompt, response, tokens, latency, model version), every tool call, every retrieval, structured as a span tree under one run ID.
The trace — not the final answer — is the unit of debugging. 'The answer was wrong' is undebuggable; 'at step 3 it retrieved the wrong chunk and the model faithfully cited it' is a fix. The telemetry vocabulary exists (OpenTelemetry GenAI conventions); the platforms exist (Langfuse, Braintrust, LangSmith). The discipline is what you have to build.
insight
Log the trajectory, not just the outcome. 'It returned a bad answer' gives you nothing; 'the retriever surfaced stale docs at step 2' gives you a fix.