Back-of-the-envelope capacity planning

Capacity estimation is the skill of turning requirements into hardware. The standard toolbox:

• QPS math: users × actions/day ÷ 86,400 s, then multiply by peak-to-average ratio (~2–3×).
• Storage math: rows/day × bytes/row × retention.
• Read/write ratio decides caching and replication strategy.
• Latency budgets: sum of network + queue + compute + storage on the critical path.

Power-of-two and "latency numbers every programmer should know" are the cheat sheet (L1 cache ~1 ns, RAM ~100 ns, SSD read ~100 µs, same-DC network ~0.5 ms, cross-continent ~50–150 ms). The point isn't memorizing them — it's using them to sanity-check designs before drawing anything.

Worked example 1: Traverse submissions

• 100+ active users. Assume a heavy DSA day: 20 problems/user/day, each submission POST ~1 KB of payload (plus up to 10 MB for attempts snapshots — but those go to the AI queue, not the hot path).
• Write QPS: 100 users × 20 / 86400 ≈ 0.02/s average — call it 0.1/s at 5× peak. This is nothing. A single Node event loop handles thousands.
• Conclusion the numbers force: Traverse is not I/O-bound on QPS; it's latency-bound on the synchronous work behind the write (the 15 s-per-provider LLM ladder). The 50x fix was removing tail latency, not adding throughput. That's why a single Express process + Prisma is the right architecture at 100 users — and why the resume number (0.2 s) is about p95 per request, not QPS.
• Redis TTL accounting: ~18 cache key families, TTLs 120–600 s → at 100 users, cache size is trivially < 50 MB. The 512 MB maxmemory is dominated by BullMQ job payloads (full code attempts), which is why eviction policy noeviction matters (Day 46).

Worked example 2: Zoth at enterprise scale

Do the interview math out loud for a 5,000-employee company:

• Access requests: even a heavy org does ~500 requests/day → 0.006/s. Approval traffic is human-paced.
• The expiry loop is the real throughput story: worst case 100 expirations per 30 s tick (batch LIMIT 100 in ListExpiredAccessRequests) = 3.3 revocations/s, each publishing one event and one zlink API call. GitHub's API rate limit (~5,000–15,000 req/h for authenticated org tokens) is the actual ceiling — capacity planning here is about downstream API quotas, not your CPU.
• Storage: an access_requests row is ~1 KB, audit rows ~1 KB. 500/day × 3 years ≈ 550 MB. Single Postgres. Correct conclusion: no sharding, no partitioning needed — ever, for this workload, and your design says so implicitly (single instance, composite indexes only).
• Vector store: policy_documents.embedding vector(768) ≈ 3 KB/row; even 100k policy docs = 300 MB — HNSW index keeps cosine search at ms latency. This is why pgvector beat "stand up a vector DB service": the capacity math said the workload fits in the OLTP database.

Worked example 3: the $30 cluster

Zoth's Bicep (infra/main.bicep) provisions 1 × Standard_B2s (2 vCPU, 4 GB). api + emitter + alerting requests: 100m/250m CPU, 128–256 Mi. RabbitMQ: 1 Gi PVC. The envelope: 2 vCPU shared by 3 Go services + RabbitMQ + (worker node OS) — comfortable because Go services are idle 99% of the time at this scale. az aks stop drops burn to ~$0.05/month when idle. The interview line: "I let capacity math, not resume-driven development, pick the topology."

Practice drill

Pick: "design Instagram likes." 500 M users, 10% active daily, 5 likes/user/day → 580 likes/s average, ~1,500/s peak. Likes are idempotent per (user, post). Storage 8 bytes + indexes ≈ 100 B/row → 5 TB/year. Now you've justified: partitioning by post_id, batched writes, and count estimation (don't count 10 M rows per profile view — Day 69, HyperLogLog and count-min sketches).