Day 91 of 100·System Design Day·665 words

Synthesis — design Zoth from scratch, cold

A full mock interview: 45 minutes, no notes, using the method from Days 1–89.

Concept → your projects

Run this as a real drill: set a timer, talk out loud, draw on paper. Then compare against the model answer below. This is the exam for the odd days.

The prompt

"Design an enterprise platform that lets employees request temporary access to company resources (repos, cloud roles, SaaS tools), with approvals, automatic expiry, and an audit trail. It must work across multiple companies (tenants)."

The model answer, in interview order

1. Requirements (2 min). FRs: request access to a resource; policy-driven routing/approval; time-bound grant; automatic expiry/revocation; provisioning to external platforms; audit everything. NFRs: safety over speed (a wrong grant is worse than a slow one) → strong consistency for decisions; provisioning can be eventual with a bounded window; audit immutability for compliance; cost-conscious (SMB/mid-market, not hyperscale); multi-tenant isolation as a security requirement.

2. Capacity (3 min). Requests are human-paced: 5,000-employee org ≈ 500 requests/day ≈ 0.01 rps. The real limits are downstream API quotas (GitHub/Azure) and expiry throughput (3–4 revocations/s worst case). Storage: ~1 KB/row, low GB over years. Conclusion: one Postgres, no sharding, no partition gymnastics — and say why the numbers allow simplicity.

3. Data model (5 min). tenants; users (manager_id self-FK for the chain); groups + user_groups; integrations (platform credentials, encrypted); resources (owned via integration); policies (JSONB rules + priority); access_requests (status enum, policy_id, expires_at); approvals (decision authority + source); audit_logs (hash-chained); outbox_events. Constraints: unique keys, FKs with documented cascade actions, enums for closed vocabularies.

4. API (3 min). /access-requests CRUD + transition endpoints (/approve, /deny, /revoke); /policies; /integrations; /audit-logs; health trio (healthz/readyz/aggregate). Contract discipline: nested error envelope, 409 on transition conflicts, tenant from the token, admin-gated mutations.

5. The critical path (5 min — the heart of it).

request → policy match (JSONB, priority-ordered) → route approval (manager chain)
       → approve: self-check + authority check + CAS transition to ACTIVE with expires_at
       → audit row + outbox event (same commit)
       → relay → queue → provisioning worker → platform API (retries, DLQ)
       → expiry loop (30 s) → CAS to EXPIRED → revoke event → platform API
       → drift detector (5 min) reconciles actual vs expected

Justify each mechanism: conditional update for transition safety, transactional outbox so provisioning can't lose events, dedup table for at-least-once consumers, idempotent revoke (not-found = success), drift detection as the reconciliation backstop.

6. Authorization (4 min). Deterministic rules decide (RBAC gates admin surfaces; policy JSONB decides resource access); advisory AI (risk scoring from historical decisions) may inform but is structurally unable to grant — reachable only through a matched policy, writing advisory columns only. State that guarantee as an architectural property, not a hope.

7. Multi-tenancy (3 min). Tenant in every query; resources derived through integrations; credentials encrypted per row (per-tenant KMS as the upgrade); cross-tenant regression tests per controller; RLS as the structural upgrade.

8. Tradeoffs to volunteer (3 min). No Kafka (work-queue semantics + outbox durability), no graph DB (bounded upward walk → recursive CTE), no Redis cache (nothing hot), single-node budget tradeoff with documented upgrade paths.

9. Failure drill (5 min). Broker down → outbox backlog, no loss. Worker crash → redelivery + dedup. Platform outage → retries + DLQ + drift reconciliation. Race approve/expire → CAS, one wins, 409 to the loser. Postgres loss → (in this design) restore; in production: replica + Patreon-style failover with the honest note that IAM should be CP.

10. What I'd build next (2 min). Publisher confirms + persistent messages; sequence-ordered audit chain; escalation SLAs for approvals; RLS; per-tenant KMS; SLOs with burn-rate alerts.

Self-grading rubric

Did you state NFRs with numbers? Did you justify removals (no Kafka/graph/cache)? Did you name the atomicity mechanism (CAS) and the durability mechanism (outbox)? Did you distinguish advisory from authoritative? Did you volunteer weaknesses with fixes? Six yeses = interview-ready.

Bulletproof your resume

Say these answers aloud before moving on

These are the follow-ups an interviewer will actually ask about your projects.

01

You designed this in 45 minutes — what would take longer in reality?

The unglamorous parts: connector breadth (each platform's API quirks, rate limits, error taxonomy), policy authoring UX (JSON rules are powerful and unusable for non-engineers — a real product needs a builder), enterprise identity integration (SCIM, group sync, manager hierarchy from IdPs is messy in practice), and compliance evidence generation (recertification reports, retention controls). The architecture is the fast part; the last 20% is where products live — and I'd say that in an interview because it shows I understand shipping, not just designing.

02

What's the single most important design decision in that whole answer?

The transactional outbox. It's what lets me promise 'provisioning is eventual but never lost' while keeping the request path fast and the broker non-critical — it decouples correctness from infrastructure availability. Everything else (CAS transitions, dedup, idempotent revoke, drift detection) is a supporting cast around that one decision: state changes and their consequences are committed together, and delivery is a separate, retryable concern.

03

How would your design change for 100,000 employees and 40 integrations?

Throughput is still trivial; the pressure is operational and organizational: (1) partition work per integration (per-connector queues so one platform's outage doesn't stall others); (2) cache policy evaluation with event-driven invalidation (hot reads at that scale); (3) partition audit by month for retention; (4) per-tenant fair scheduling; (5) RLS for defense-in-depth as more teams write queries; (6) multi-replica everything with the claim-based patterns that already scale. Notably none of those is a rewrite — the design's primitives (CAS, claims, outbox, dedup) are the ones that scale, which is the real test of an architecture.