Deep systems understanding · Study map 39 of 85
GPU hardware: workers, shelves, and delivery lanes
Suggested preparation: Establish a baseline you can trust
Picture it
Picture a vast factory floor teeming with thousands of specialized workers, all eager to assemble parts. Right next to their workbenches are tiny, lightning-fast shelves, while the main warehouse is massive but sits a slow truck-drive away. Even the most efficient workers will stand idle if the trucks can't deliver the raw materials fast enough. To unleash the factory's true potential, you must orchestrate the flow of parts so the workbenches never run dry.
What to understand
- SMs, warps, threads, registers, shared memory, L2, and HBM
- SIMT, occupancy, coalescing, tensor cores, precision, and CPU/PCIe input paths
Follow the references
1 · Documentation
NVIDIA · CUDA programming guide ↗Start with the programming model and memory hierarchy. Draw where one matrix tile lives.
2 · Book chapter
How To Scale Your Model · Rooflines ↗Relate peak arithmetic and memory bandwidth to actual work.
Make it concrete
Compare two hardware specifications by capacity, bandwidth, supported data types, and interconnect. Predict which workload favors each before benchmarking.
You understand it when…
Why does a GPU with more advertised FLOPs not necessarily decode a model faster?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.