Deep systems understanding · Study map 76 of 85
Observability: connect the user symptom to the machine
Suggested preparation: RL post-training: alternate generating and learning
Picture it
Imagine trying to diagnose a sick patient by only looking at their average heart rate over a year. A useful dashboard lets you zoom into the exact moment a symptom occurs and trace it to a specific blocked artery or misfiring nerve. A single aggregate number can easily mask a severe issue affecting just one crucial demographic or feature.
What to understand
- Request traces, phase histograms, SLO/error budgets, and burn rates
- Queue length, active tokens, preemption, cache hits, GPU/network health
Follow the references
1 · Documentation
vLLM · Production metrics ↗Choose stage metrics and histogram buckets that support your SLOs.
2 · Documentation
NVIDIA · DCGM feature overview ↗Add hardware counters only when they help distinguish a suspected bottleneck.
Make it concrete
Create a dashboard specification with a user-facing symptom, a diagnostic metric, and an action for each alert. Avoid logging raw prompts by default.
You understand it when…
TTFT rises while GPU activity falls. What correlated signals would separate a tokenizer bottleneck from slow weight loading?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.