Being able to answer new questions about what the system is doing from the outside, using traces, metrics and logs, without shipping new code.
You have done this if
You could open one trace and see the prompt, the retrieved chunks, the tool calls and the latency of each step.
Say it in a review
Every request is one trace across gateway, retrieval, model and tools, so we can explain any answer.
On the AI Application map Observability
Read Why observability is not optional for AI systems · Logging the answer tells you almost nothing