Caching layers, eviction, and TTL semantics
Cache-aside, LRU, TTLs, stampedes — with traverse's CacheService as the implementation study.
Concept → your projects
Cache taxonomies: location (client → CDN → API-local → distributed Redis → DB buffer) — each layer closer to the truth and slower; pattern — cache-aside (app reads, misses load, fills; you own invalidation), read-through/write-through (cache fronts the DB), write-back (async flush — dangerous durability); eviction — LRU (default), LFU (skewed-but-stable keys), TTL (time-bounded staleness), random (surprisingly competitive).
The two hard problems (the famous joke: cache invalidation and naming):
- Invalidation: TTL is the blunt instrument (bounded staleness, zero coordination); explicit invalidation is precise but must find every key (traverse's
delPatternvia SCAN with cluster-aware routing — the keys patternsubmissions:{userId}:*groups per-user keys so one user's invalidation doesn't scan the world). - Stampedes: hot key expires → N concurrent requests all miss → all hit the DB. Fixes: request coalescing (single-flight: one loader, others wait), probabilistic early expiration (xfetch: refresh at TTL·(1±jitter)), or locks. Know single-flight conceptually even though your QPS doesn't need it.
Staleness math: TTL 120 s means p95 staleness ≈ 120 s — the design question is always "what business action tolerates 2 minutes of stale?" Traverse's per-data TTLs answer it per concern: revision stats 120 s (UI comfort), submission lists 300 s, submission-by-id 600 s, groq model list 35 days (near-static). Zoth: no cache at all — reads are 1/s; the right cache decision is often "none" (Day 5's when-not-to-add).
Traverse's CacheService — the implementation study (src/lib/cache.ts):
defaultTTL 3600, per-call overrides;get/set/del/delPattern/exists/incr/mget/mset/wrap(wrap = cache-aside in one function:wrap(key, ttl, loader)).- 2000 ms per-op timeout via
withTimeout— a slow Redis can never hang a request; every method degrades to no-op/null when Redis isn't ready (isRedisReady()). The cache is strictly optional to correctness: DB is always the fallback path. That's the availability posture to recite: "cache down = slower, not broken." - Two Redis clients:
redisClient(cache) andbullRedisClient(queues,maxRetriesPerRequest: null— required by BullMQ so a Redis blip doesn't reject the connection mid-operation). Separation matters: queue client semantics (blocking, retry-forever) are incompatible with cache client semantics (fail-fast). - Cluster support:
delPatternroutes SCAN per node (REDIS_CLUSTER_NODES) — because SCAN is per-node in Redis Cluster; the classic "my delPattern worked locally, deleted nothing in prod" bug, pre-solved.
The write-path doctrine (traverse's, stated in the commit message): "We do NOT cache the write cycle" — writes go DB-first, then enqueue; the workers invalidate caches after derived state updates (invalidateSubmissionCaches, 10 keys in Promise.allSettled — allSettled so one slow del can't fail the batch). Reads: cache-aside with TTLs. Cache never sits between the user and the source of truth.
Bulletproof your resume
Say these answers aloud before moving on
These are the follow-ups an interviewer will actually ask about your projects.
Why does your cache degrade to a no-op instead of erroring when Redis is down?
Because the cache is an optimization layer, not a dependency of correctness: every read path falls through to Postgres, bounded by a 2 s per-op timeout so a hanging Redis adds latency once, never indefinitely. Erroring would turn a slow cache into an outage. The counterpart discipline: nothing is written only to cache — the DB write always happens first, so a cache wipe costs a cold period, never data loss.
How do you choose TTLs?
By staleness tolerance per data type: revision stats 120 s (UX freshness), submission lists 300 s, single submissions 600 s, near-static reference data (model lists) 35 days. The test: what user action is harmed by the staleness window? Stats — none (cosmetic). Notifications unread-count — needs event-driven invalidation more than TTL (it gets explicit invalidation on mark-read). Security-sensitive data — never TTL-cached, invalidated explicitly: zoth caches nothing, and its policy decisions wouldn't be TTL-cacheable without event-driven invalidation on policy writes.
What is a cache stampede and how would you handle one at scale?
Hot key expires; N concurrent requests all miss and all query the DB simultaneously. Fixes: single-flight (one request loads, others await its promise — trivial in-process, needs a distributed lock across replicas), probabilistic early refresh (fetch at TTL×(0.75 + 0.25·rand) to spread expirations), or serving stale while one background refresh happens. At my QPS it's theoretical; the place it would first bite traverse is the leaderboard/stats endpoints on a burst — the fix is per-process single-flight in the wrap() helper, which is a 10-line change I've already scoped.