Admission control: a bounded waiting room

Picture it

Think of a popular restaurant with a finite number of tables. If you let everyone inside at once, the kitchen collapses, waiters bump into each other, and nobody gets their food on time. Admission control is the bouncer at the door, firmly holding the line to ensure that once a request is seated, it receives the full attention and resources it needs to finish promptly. Protecting the system from overload is what keeps the throughput high.

What to understand

• Token budgets, deadlines, quotas, backpressure, and load shedding
• Fairness, head-of-line blocking, priority, starvation, and output caps

Follow the references

1. 1 · Book chapter

   Google SRE · Handling overload ↗

   Study bounded queues and client throttling. Apply the same reasoning to variable-length generation.

Make it concrete

Drive load above capacity, then add queue/deadline limits. Compare successful requests within SLO, rejected work, memory, and recovery time. Include one noisy tenant.