The idempotency inventory — every duplicate-proofing trick in your code
A full catalog of how two codebases make at-least-once delivery safe to run twice.
Know your own system
Yesterday was the theory; today is the complete inventory, because "how do you avoid double-processing" is the #1 queue question, and your answer should be enumerative.
Zoth — event-driven, at-least-once, safe
- Consumer dedup (the main line).
PostgresDeduper.Seen(internal/queue/consumer/dedup.go):INSERT INTO processed_events(event_id) VALUES($1) ON CONFLICT DO NOTHING— ifRowsAffected == 0, the event was seen. The key:<routingKey>:<event_id | id | request_id>— the routing-key prefix matters: the same request ID granted and revoked must not collide into one dedup entry. Body-SHA256 fallback when no ID field parses. Fail-closed: if the dedup check itself errors, the handler errors (retries, eventually DLQ) rather than risk double side effects. - Idempotent revoke.
PlatformBridge.RevokeAccesstreatszlink.ErrNotFoundas success — "already gone, goal state achieved." So expire-then-manual-revoke, or revoke-then-expire, both converge. - Outbox relay claims.
FOR UPDATE SKIP LOCKED— two relay replicas never publish the same batch (Day 62 full treatment). - Conditional state updates.
UPDATE ... WHERE status=$expected— replayed approval events can't re-transition an already-ACTIVE request (loser gets ErrConflict). - Migrations under advisory lock — concurrent migrators serialize on
pg_advisory_lock(72183921). - Group adds:
ON CONFLICT DO NOTHING.
The residual gap you should be able to name: processed_events grows forever — needs TTL cleanup (a cron deleting rows older than, say, 30 days; dedup windows only need to cover redelivery horizons, not eternity).
Traverse — request-driven, multi-layer
- Request-level:
Submission @@unique([userId, idempotencyKey])— extension retries return 200 + existing row, no double XP. - Solve uniqueness:
@@unique([userId, problemId])— the "was this a first solve?" answer is enforced by the DB, not by check-then-insert (which races). - Revision uniqueness:
@@unique([solveId, revisionNumber, type])—createNextRevisioncatchesP2002and returns the existing revision;initializeMLRevisionearly-returns if any ML revision exists; bulk import countsskippedExisting. - Problem upserts:
@@unique([platform, slug])— bulk import catches P2002 and re-reads the raced row (the correct pattern for find-or-create races). - Set-once semantics:
updateMany({ where: { id, topic: null }, data: { topic } })— AI classification can never overwrite a stored topic. - Notification delivery:
@@unique([notificationId, channel])at write time +FOR UPDATE SKIP LOCKEDwith a 120 s lease at send time + retry backoffmin(60s·2^(n-1), 1h). - Broadcast fanout: jobId
broadcast-fanout-<id>(BullMQ dedupes by job ID) +cursorUserIdcursor making the job itself resumable — crash mid-fanout, re-enqueue, continue from the cursor. - Push drain: repeatable job with fixed
jobId: 'notification-push-drain'— the scheduler can't stack multiple drains. - Ring closure: only
updateMany({where:{closedAt:null}})returningcount === 1emits theRINGS_CLOSEDnotification — the conditional update is the claim. - Bulk import in-batch dedup: normalize keyed by slug keeping the earliest
happenedAt.
The taxonomy to recite
- Natural idempotency (state-setting writes): conditional updates, set-once.
- Deduplicated side effects (queues/webhooks): unique constraints, insert-on-conflict, job IDs.
- Goal-state convergence (external side effects): idempotent revoke.
- Claim-based single execution (workers): SKIP LOCKED / lease / conditional-update-as-claim.
The subtle point interviewers reward: the DB constraint is the last line, not the first. Application-level dedup (check-then-insert) races; constraints make duplicates impossible rather than unlikely. Both your codebases put constraints underneath every dedup decision.
Bulletproof your resume
Say these answers aloud before moving on
These are the follow-ups an interviewer will actually ask about your projects.
Your broker delivers access.granted twice. Walk through exactly what happens.
The consumer extracts the dedup key — routing key 'access.granted' + request_id — and inserts into processed_events with ON CONFLICT DO NOTHING. First delivery: 1 row affected, proceed to the handler. Second delivery: 0 rows affected → duplicate → Ack and drop. If the dedup insert itself errors, the handler returns an error instead of proceeding (fail-closed), the consumer retries, and worst case the message dead-letters for inspection. And even if dedup somehow passed twice, the emitter's revoke/grant are per-request goal-state operations and the state machine's conditional updates would reject re-transitioning an ACTIVE request.
Why key dedup on (routingKey, id) instead of just the ID?
Because the same logical ID legitimately appears under multiple routing keys — request X gets access.granted, later access.revoked, later access.expired events. Keying on ID alone would treat the revoke as a duplicate of the grant and silently skip revocation — a security bug. The routing-key prefix namespaces each lifecycle event separately; a body-hash fallback covers payloads without IDs.
How do you handle find-or-create races without transactions everywhere?
Try the insert, catch the unique-violation error, re-read the existing row. Traverse's bulk import does this for Problem (platform,slug) and Solve (userId,problemId): insert → P2002 → select the winner. It's optimistic concurrency — simpler than SELECT FOR UPDATE under low contention, and the DB constraint is the arbiter. Under heavy contention you'd flip to ON CONFLICT upserts or advisory locks, but the correctness argument never changes: the constraint decides, the app adapts.