The career-application upload flow and the recon streamer
Two file paths in traverse, one conventional and one improvised well.
Know your own system
Flow A — career applications (the conventional path, done correctly)
Route: POST /api/careers/applications with multipart (or JSON + presign, depending on client) → controller validates role/fit fields → file handling: stored to the careers bucket (CAREERS_S3_BUCKET, ap-south-1) under applications/{applicationId}/{filename} → DB row created with status SUBMITTED, resume key, parsed role → email notification to careers@ (the human-fanout step) → admin review flow flips status (UNDER_REVIEW → APPROVED/REJECTED).
Design points to claim (each maps to a principle from Day 79): the DB stores keys and truth, never blobs; the bucket is region- and credential-isolated from the database-backup bucket (blast radius); validation happens before acceptance (size/type limits — name the values from your config); the resume is immutable per application (a re-application is a new row + new key, never an overwrite — append-only thinking, and it makes the admin review auditable: you know exactly what was submitted when); and the notification email is explicit rather than implicit — a human workflow step you can point at.
The upgrade to name: presign → client-direct upload → confirm endpoint that validates (size, magic bytes) and records the key. Benefits: API bandwidth/memory drops to zero for uploads, and the confirm step gives you a place to run content checks and to reconcile orphans (objects uploaded but never confirmed → a sweep job, the same reconciliation instinct as drift detection).
Flow B — recon streaming (the improvised one, and the interesting one)
Constraints that shaped it: the body is unbounded (capture bundles: network responses + DOM snapshots per platform), the client is a browser extension (untrusted-ish, retry-prone), the data is diagnostic (valuable for product insight, not user-facing), and the server runs on a small VPS.
The design: route mounted before express.json (Day 18) → handler consumes the raw request stream and writes chunks to recon/{platform}/... locally (not S3 — diagnostics don't need object-storage durability; local disk + quota is the right cost point) → .partial file then atomic rename on completion (readers never see partial dumps; a crash leaves a .partial that gets overwritten) → per-platform quota (KEEP_PER_PLATFORM) prunes old dumps → a digest email after the flush summarizes what arrived (batch notification instead of per-bundle spam).
Why not presigned direct-to-S3 here? Three reasons worth stating: (1) quota accounting requires a trusted intermediary (the extension shouldn't decide what to keep); (2) the dump format is server-normalized (the API may compress/reformat); (3) the API also sends the digest — one coherent flow. That's the justified deviation (Day 79) — and interviewers respect a deviation with reasons more than a pattern applied blindly.
The atomicity lesson, generalized: .partial + rename is the local-filesystem version of S3 multipart-complete and of the DB's transaction — make incomplete work invisible rather than cleaning it up. The same instinct appears in your outbox (unpublished rows are invisible to consumers), your notification lease (unclaimed rows are invisible), and Postgres MVCC (uncommitted versions are invisible). One principle, four implementations — say that and you sound like you've internalized it rather than memorized it.
Bulletproof your resume
Say these answers aloud before moving on
These are the follow-ups an interviewer will actually ask about your projects.
Why isn't the recon dump stored in S3 like the resumes?
Because the requirements differ: resumes are user-facing artifacts needing durability, per-object ownership, and a lifecycle (multi-region isolation from backups); recon dumps are high-volume diagnostic data with a quota — the right cost point is local disk with KEEP_PER_PLATFORM pruning. Storing them in S3 would add per-object cost and a second credential surface for data we deliberately discard. The rule: match storage to the data's value and lifecycle, not to a single house style.
Explain the .partial-then-rename pattern and where else you use its cousins.
Write to a temp name, then atomically rename when complete — readers only ever see complete files, and a crash leaves an ignorable partial that the next write overwrites. Its cousins in my systems: S3 multipart upload (invisible until complete), the outbox (rows invisible to the relay until committed), the notification lease (rows invisible until claimable), and MVCC itself (uncommitted versions invisible to readers). It's the same principle — make incomplete work invisible instead of cleaning it up — applied at four layers, and it's why none of those paths needs a 'cleanup job' for crashed work.
What would you change about the career application flow?
Move to presign + direct upload + confirm: the API stops proxying megabytes (bandwidth and memory), validation moves to a post-upload check (size, magic bytes — not just extension), and I'd add an orphan sweep (objects uploaded but never confirmed) plus a virus-scan quarantine step, since resumes are untrusted user files. The DB-side design stays as-is: key in the row, immutable per application, status machine for review — that part I'd defend unchanged.