Explain the mechanism. Defend the tradeoff.
Use these as practice prompts, not memorized scripts or claims about what employers ask. Draw the system, state assumptions, and name the measurement that would change your answer.
GPU activity is high; useful throughput is poor
How would you distinguish recomputation, bandwidth limits, a bloated batch, and failed or late requests?
An 8k document makes everyone’s stream stall
Compare chunked prefill, admission limits, and separate pools. What happens to the document’s TTFT?
The model fits, then OOMs under traffic
Build the memory ledger and explain which term changes with users, context, precision, and architecture.
A two-GPU server is slower than one GPU
Explain the communication pattern, topology, batch, and intended objective before choosing another split.
A faster checkpoint fails important tasks
What evaluation slices and deployment gates prevent a misleading average from approving this change?
A worker dies halfway through a response
Define retry semantics, client behavior, capacity headroom, and cleanup.