Deep systems understanding · Study map 85 of 85
Capstone · Design and defend an LLM service
Suggested preparation: Lab 4 · Survive overload and a failed worker
Picture it
You are the architect of a suspension bridge. Every cable, pillar, and rivet must be justified by the expected winds and the weight of the traffic. You must fuse your choice of materials and structural mechanics into a single, elegant design that survives intense scrutiny, unexpected storms, and tight budgets, knowing exactly which assumptions hold the entire span together.
What to understand
- Model/quality selection → capacity → scheduler/cache → topology
- Deployment, observability, permissions, cost, and recovery
Follow the references
1 · Book chapter
How To Scale Your Model · Transformer inference ↗Revisit the inference-engine design section as a checklist of physical constraints.
2 · Documentation
vLLM · Production metrics ↗Specify exactly how each service target will be observed.
3 · Book chapter
Google SRE Workbook · Canarying releases ↗Finish with release and rollback gates.
Make it concrete
Deliver a workload/SLO contract, architecture diagram, memory and cost sheet, versioned serving config, quality set, load-test report, and failure runbook. Explicitly list unmeasured assumptions and the next experiment that could overturn your choice.
You understand it when…
Defend your design when prompt length doubles, a GPU fails, or the budget is cut by 30%.
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.