Deep systems understanding · Study map 40 of 85
Rooflines: count work per byte moved
Suggested preparation: GPU hardware: workers, shelves, and delivery lanes
Picture it
Imagine a chef chopping vegetables—arithmetic intensity is how many slices they make per vegetable carried from the pantry. If they only make one cut and throw the rest away, they spend all their time walking to the pantry. The roofline model is a visual ceiling that shows your limit: you're either capped by the chef's knife speed (compute-bound) or by how fast they can haul vegetables from the shelves (memory-bound).
What to understand
- FLOPs/byte and the compute-to-bandwidth ratio
- Lower bound: max(FLOPs / FLOPs-per-second, bytes / bytes-per-second)
Follow the references
1 · Book chapter
How To Scale Your Model · Rooflines ↗Work through the matrix multiplication example and locate the crossover.
Keep the distinction. This simple bound ignores some synchronization and sequential phases. Sum stages where they cannot overlap; peak bandwidth is an optimistic ceiling.
Make it concrete
Estimate a lower bound for a dense model step from weight bytes and effective HBM bandwidth. Compare with measured latency and investigate the gap.
You understand it when…
Why can batching improve weight-read efficiency while growing the amount of KV data to read?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.