How a server handles many concurrent requests

When a request arrives, the OS accepts a TCP connection (backlog queue → accept), and the runtime decides who processes it. Two dominant models:

Thread-per-request (Go's model, roughly). Go doesn't spawn OS threads per request; it spawns cheap goroutines (2 KB stacks, scheduled onto M OS threads by the runtime's GMP scheduler). Blocking on I/O parks the goroutine and frees the thread — you get thread-style code with event-loop economics. net/http spawns one goroutine per connection.

Event loop (Node's model). One thread, a poll loop over epoll/kqueue, callbacks/promises. Brilliant for I/O-bound work; catastrophic if any handler blocks the loop (sync crypto, JSON.stringify of a 10 MB object, a busy loop). Concurrency = interleaving, not parallelism — the cluster module or multiple processes buys actual CPU parallelism.

Both then hit the database: Postgres speaks a connection-per-client model with processes per connection, hence connection pools (pgx pool in zoth, Prisma's pool in traverse). A pool of 10–25 is almost always right; more connections = more Postgres processes = more memory and lock contention, not more throughput.

Where your code makes this concrete

Zoth (Go):

• ServerConfig{ReadTimeout: 15, WriteTimeout: 15} (internal/config/config.go) — every connection is bounded; a slow client cannot hold a goroutine forever. Know why this matters: without write timeouts, a stalled client leaks a goroutine per connection (slowloris).
• Middleware chain per request: Recovery → otelhttp → Logging → CORS (Chi). Each is a goroutine-local operation — no locks needed.
• The expiry loop is a separate goroutine (time.Ticker, 30 s), the outbox relay another (5 s) — concurrency via channels/context cancellation, shut down gracefully with a 10 s deadline on SIGTERM.
• The consumer uses prefetchCount = 10 (Qos(10,0,false)) — RabbitMQ's credit-based flow control, the broker-side equivalent of a connection pool.

Traverse (Node):

• express.json({ limit: '10mb', verify: rawBody }) — a 10 MB parse on the event loop. That's why /api/recon (unbounded capture bundles) is mounted before the JSON parser and streamed to disk raw instead (src/lib/reconStore.ts — write to .partial, then rename). That's an event-loop-aware design decision; say it exactly that way.
• Redis clients configured with maxRetriesPerRequest: null — required by BullMQ so a blocked Redis op doesn't reject every in-flight promise.
• Worker concurrency: BullMQ workers (concurrency 10 / 5 / 1) run jobs as separate async tasks on the same loop — the loop is still single-threaded; the 15 s LLM call is a network await, which is exactly what the event loop is good at.
• trust proxy 1 — Express/X-forwarded-for handling so the rate limiter sees real IPs behind nginx/compose networking.

The numbers to keep in your head

• A Go service with 3 endpoints at ~1 rps: ~10 MB RSS idle. Node with auto-instrumentation OTel: ~150–250 MB. Both fine; neither is a bottleneck.
• Postgres: each connection ≈ ~10 MB backend process; 100 connections ≈ 1 GB. Hence pools.
• One Node process saturates ~1 CPU core for CPU-bound work (JSON parsing ~100 MB/s-ish). If a request does CPU work (LCS diff in traverse's AI pipeline bails out above 2000 lines — computeDiff), offload or bound it.

Interview framing

"What happens when 10,000 clients connect at once?" — Go: 10k goroutines (~20 MB), fine; event loop: 10k sockets in epoll, fine — both handle C10K today. The differentiator is one blocking call: in Go it parks a goroutine; in Node it stalls everything. Your traverse code shows the defensive patterns (streaming, timeouts, Promise.allSettled in cache invalidation so one slow Redis op can't fail the batch).