Load testing: recreate the traffic, not just the average

Picture it

Testing a bridge by walking across it alone tells you nothing about rush hour. Real users arrive in sudden, unpredictable waves, not a steady, polite single-file line. Open-loop testing unleashes this realistic flood, revealing the breaking points where queues pile up and the structure begins to groan under the weight.

What to understand

• Open versus closed loop, trace replay, warm/cold caches
• Arrival process, length distributions, steady state, errors, and tail metrics

Follow the references

1. 1 · Documentation

   vLLM · Serving benchmark reference ↗

   Inspect request-rate, burstiness, dataset, and latency options. Read the current version’s metric definitions.

Make it concrete

Sweep offered load from idle through overload. Count all arrivals, rejections, timeouts, and successful outputs; plot goodput and tails alongside throughput.