FREE · TOPIC 233

When Not To Add Infrastructure

619 words·Updated 2026-07-18·
#system-design#senior-judgment#architecture
# When Not To Add Infrastructure Senior system design is often about declining complexity. Infrastructure is justified only when it buys a real property: lower latency, higher availability, stronger isolation, better operability, or lower cost at scale. The default should be to improve the current system until there is evidence that a new component changes the operating envelope. New infrastructure is not neutral. It creates deployment paths, dashboards, credentials, capacity limits, ownership questions, and failure modes. ## Do Not Add It Yet If - The current system has no measured bottleneck. - A simpler database/index/query change solves the problem. - The team cannot operate the new component. - Failure modes get worse than the problem being solved. - The product requirement is still unstable. - The "scale" is hypothetical and cheap vertical scaling remains available. ## The Burden Of Proof Before adding a queue, cache, search cluster, stream processor, service boundary, or orchestration layer, name the property it must provide. | Proposed Component | Property It Should Buy | Evidence To Ask For | |---|---|---| | Cache | Lower read latency or database load. | Hot keys, query latency, cacheability. | | Queue | Isolation from slow or retryable work. | Request timeouts, retry needs, async tolerance. | | Search index | Better retrieval semantics. | Query patterns SQL cannot serve well. | | New service | Independent scaling or ownership. | Clear API, team boundary, release pressure. | | Multi-region | Availability or latency requirement. | User geography, RTO/RPO, data constraints. | If the evidence is weak, keep the design reversible. Make the module boundary cleaner, add instrumentation, and defer the infrastructure decision. ## Common Overbuilds | Temptation | Simpler First Move | |---|---| | Kafka for every async action | Start with a task queue or outbox. | | Microservices for ownership | Start with module boundaries and clear APIs. | | Redis for every read | Add indexes, fix queries, then cache hot paths. | | Kubernetes for a small app | Use a managed platform or simpler deploy. | | Global multi-region writes | Start with single-primary plus read replicas unless product requires active-active writes. | ## Simpler First Moves - Add the missing index and verify the query plan. - Move slow side effects behind a local job table or outbox. - Add idempotency keys before adding distributed retries. - Split code modules before splitting deployable services. - Increase instance size while measuring whether the bottleneck moves. - Add backpressure and rate limits before scaling every worker pool. These moves are not anti-scale. They preserve learning speed while the product and traffic shape are still changing. ## When It Is Time Infrastructure becomes justified when the system has a repeated, measured problem that simpler changes cannot fix. Good signals include saturated database resources after indexing, user-visible latency from slow dependencies, operational need for durable retries, clear team ownership boundaries, regulatory isolation requirements, or a reliability target that the current topology cannot meet. At that point, the design should include the new component's failure behavior. A cache must define fallback. A queue must define retry, dead-letter, and ordering semantics. A new service must define API ownership, observability, and deployment compatibility. ## Failure Modes - The team adds a cache but still cannot tolerate stale reads. - A queue hides failures until work piles up silently. - A microservice split turns local transactions into partial failures. - A stream processor becomes the only source of truth by accident. - A platform migration absorbs engineering time without improving the product constraint. ## Pro Tip Every new infrastructure component adds an on-call surface. If no one can debug it at 3 AM, it is not a free abstraction.
Primary References & Engineering Sources
  • ·[[wiki/cost-aware-architecture]]
  • ·[[wiki/non-functional-requirements]]
  • ·[[wiki/delegation-and-async-work]]
  • ·[[wiki/reliability-resource-guide]]