"We do not cache writes" — Redis policy, persistence, and the queue/cache split

Traverse runs one Redis doing two jobs: BullMQ job storage and the read cache. That cohabitation forced explicit decisions — each a talking point.

noeviction on a shared instance

--maxmemory 512mb --maxmemory-policy noeviction. Default policy is allkeys-lru — under memory pressure, evict whatever's least-recently-used. With BullMQ keys on the same instance, eviction silently deletes queued jobs (a job is just Redis data: hashes, lists, zsets). Noeviction flips failure to loud: writes OOM-error at the cap, check-redis-memory.sh (host cron) alerts at 80%, and the operator adds memory or clears cache keys — jobs never vanish silently. The trade: cache keys also can't be evicted — the cache can OOM the instance if a hot dataset grows. Hence the production fix: split instances —

cache-redis:   maxmemory-policy allkeys-lru, no persistence
queue-redis:   maxmemory-policy noeviction, --appendonly yes (BullMQ + AOF)

The code already supports the split (REDIS_HOST env is one instance; cluster mode supported via REDIS_CLUSTER_NODES) — the compose just hasn't. Say it as a scoped next step, not a deficiency discovered by others.

AOF persistence

--appendonly yes — append-only file of write commands; on restart, replay = state back. For the cache half this is optional luxury; for the queue half it's the difference between "jobs survive the restart" and "a Redis crash drops every pending job." RDB snapshots (the alternative) lose the snapshot-window; AOF with everysec fsync loses ≤1 s. The residual honesty: Redis persistence is still not Postgres durability (no fsync-per-commit by default, no PITR) — which is fine, because job semantics are at-least-once and the DB is the source of truth; losing a 1-second window of jobs means some submissions' derived processing needs the reconciliation sweep from Day 45.

The embedded-redis fossil

start.sh runs redis-server --daemonize inside the backend container in addition to the traverse-redis service. Documented as expected, not a bug — but it's a startup-order fossil: the app connects to the service instance; the embedded one is… present. The interview-grade framing: "It's redundant and I'd remove it; the reason it survives is that removing runtime behavior from a working deployment loses to feature work on the priority list. The lesson I took: never paper over infrastructure ambiguity with 'both run' — the next person can't tell which one is real." (Being able to critique your own repo this way is the bulletproofing.)

The queue/cache split, generalized

Redis roles have incompatible policies — cache wants eviction + failure tolerance; queues want retention + persistence + noeviction. Any system that puts them on one instance has chosen (like traverse) to prioritize job safety over cache elasticity. BullMQ-specific facts to know: jobs are hashes + lists (wait/active) + zsets (delayed) keyed by queue name; removeOnComplete: 1000/24h and removeOnFail: 5000/7d bound Redis growth (completed jobs would otherwise accumulate forever — your values: submission queue keeps 1000 completes for 24 h, 5000 failures for 7 days — retention as Redis hygiene); stalled-job detection re-assigns work when a worker dies mid-job (the at-least-once mechanism).

The zoth contrast

No Redis anywhere: queues are RabbitMQ (durable queues, its own persistence model), no cache (Day 43's measured-no). The contrast in one line: "traverse chose Redis-as-queue because a single VPS + jobs-in-Postgres-adjacent-Redis was the cheapest at-least-once pipeline; zoth chose RabbitMQ because provisioning side effects needed broker-native DLQ/retry semantics and the K8s StatefulSet was already running. Both are right-sized, neither is fashion."