Deep systems understanding · Study map 45 of 85
PagedAttention: grow conversations in blocks
Suggested preparation: KV cache: keep the work you can reuse
Picture it
Think of a restaurant where reserving a table means blocking out a massive banquet hall, even for a party of two, leaving the rest of the building empty. PagedAttention acts like a dynamic maître d', seating guests at small, scattered tables and seamlessly linking them together as the party grows. By breaking the memory into flexible, non-contiguous blocks, the system fills every available nook and cranny, serving far more customers at once.
What to understand
- Block tables, internal/external fragmentation, copy-on-write
- Allocation, reclamation, sharing, and eviction
Follow the references
1 · Paper
Kwon et al. · PagedAttention / vLLM ↗Use the virtual-memory analogy and block diagrams. Track two sequences sharing a prefix then diverging.
Keep the distinction. Paging does not eliminate the bytes needed for active KV tensors. Block size trades bookkeeping overhead against wasted space.
Make it concrete
Simulate contiguous reservations versus fixed blocks for mixed output lengths. Count used, reserved, and wasted token slots.
You understand it when…
What memory waste does paging reduce, and what cache size does it not remove?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.