Deep systems understanding · Study map 83 of 85
Lab 3 · Decide whether to split prefill and decode
Suggested preparation: Lab 2 · Keep chats smooth during document bursts
Picture it
Imagine splitting a kitchen so that one team chops vegetables and another cooks them in a different building. This specialization might be brilliantly efficient for a massive banquet, but disastrously slow for a single bowl of soup. A sound design proves exactly when the cost of carrying ingredients across the street is finally justified by the speed of the chefs.
What to understand
- KV-transfer accounting, pool ratios, placement, and load sweeps
- Same hardware budget, same quality, same workload
Follow the references
1 · Paper
Zhong et al. · DistServe ↗Read the evaluation assumptions and design a counterexample workload.
2 · Documentation
NVIDIA · Dynamo ↗Use a supported disaggregated deployment recipe after checking hardware and connector requirements.
Make it concrete
Compare aggregated chunked serving with a disaggregated setup using equal GPUs. If hardware is unavailable, produce an explicitly labeled analytical model and a runnable experiment plan; do not invent measured speedups.
You understand it when…
At what workload does your preferred design change, and what measurement explains the crossover?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.