Task queue vs event stream — RabbitMQ vs Kafka vs BullMQ

Three different animals, one confused interview topic. Separate them by semantics first:

Work queue (RabbitMQ, BullMQ, SQS): a message is a task — one consumer takes it, processes it, it's gone. Dispatch is push-based with credit flow control (prefetch/ack). Durability per-message is optional. Replay: none. Multiple independent consumer sets: via separate queues bound to the same exchange (zoth does this — queue per service).

Log/stream (Kafka, Pulsar, Kinesis): a message is a fact appended to a partitioned, ordered, replicated log. Consumers read at their own offset; data persists for retention (days-to-forever) → replay, multiple independent consumers, ordering per partition, massive throughput. Cost: running a broker fleet (KRaft/ZooKeeper historically), partition management, consumer-group rebalancing.

The decision rule (recite verbatim): queue when work must be done once and the truth lives elsewhere; log when the event IS the truth and consumers need history, order, or independence.

The definitive "why RabbitMQ and not Kafka" answer (zoth)

"Zoth's events are side-effect triggers, not a source of truth — the access state lives in Postgres, emitted transactionally via an outbox. So I need work-queue semantics: each access.granted should be claimed by one emitter, with retry accounting, a DLQ, and backpressure via prefetch. Kafka would buy me replay and ordered partitions — but I'd be running a distributed log to get work-queue behavior, paying broker-fleet ops on a $30 cluster for ~5 routing keys and ~1 event/minute. RabbitMQ's cost profile fits: single replica, in-cluster StatefulSet, DLX support built in. And the durability Kafka would add is already provided one layer down — the outbox means the broker can lose anything and we're still correct, because drift detection reconciles within 5 minutes."

If pushed on "Kafka gives ordering": ordering matters when event order per entity changes outcomes — zoth's consumers don't rely on cross-event ordering because each event carries full state (request id + action) and processing is idempotent-per-event; grant→revoke ordering is additionally protected by the goal-state semantics (revoke treats already-revoked as success). A log's ordering is a solution to a problem zoth engineered away.

The BullMQ answer (traverse)

"Traverse's queue is BullMQ on Redis: a work queue with delayed/repeatable jobs, per-queue concurrency, stalled-job detection, and Redis persistence via AOF. RabbitMQ would add broker ops without adding semantics — the jobs are triggers for work whose truth is Postgres. Kafka would add replay for analytics — and if I ever want event-history analytics, the submission table is the log (queryable, indexed); I'd re-derive rather than run a broker."

Be ready for the resume discrepancy (Day 4): if asked "your resume says RabbitMQ for traverse" — "the resume summarizes the pattern; the implementation is BullMQ over Redis — same architectural decision (decouple ingestion from processing), different right-sized broker. Zoth is my actual RabbitMQ project, and I can go arbitrarily deep on it." Confidence, no flinch.

When each would be right (the maturity table)

Note the parenthetical in row 4: an append-only Postgres table IS a legitimate event log — zoth's outbox is a one-partition Kafka with perfect transactional coupling; traverse's Submission table is a replayable event history. You already own "log semantics" in both systems, which is the deepest reason neither needs Kafka.