GPU performance: account for every wait
Start at a request timeline, descend to kernels only when the evidence points there, and return to end-to-end latency after every change.
Reason before profiling
Count operations and bytes. Predict whether memory, compute, dispatch, or communication will dominate.
Inspect a timeline
Identify gaps, long kernels, copies, and collectives. Choose one measurable bottleneck.
Documentation · NVIDIA · Nsight Systems user guide ↗
Study the mechanisms
Follow data through a tiled matrix multiply, fused softmax, and tiled attention. Learn why fewer memory trips can matter more than fewer arithmetic operations.
- FlashAttention: avoid writing the giant scratchpad →
- Fusion and CUDA graphs: less dispatch, fewer round trips →
Tutorials · Triton · GPU programming tutorials ↗
Documentation · NVIDIA · CUDA programming guide ↗
Show the evidence
Re-run the same workload and quality check. Record when the optimization stops helping.