FREE · TOPIC 175

Parallel Monolith Read Drain

646 words·Updated 2026-07-18·
#system-design#high-throughput#databases#legacy-modernization
# Parallel Monolith Read Drain When a legacy monolith sends too many reads to a master database, you do not always need to fix every call site first. You can run the same code under a different environment and route known-safe read endpoints to it. ## Problem Legacy code uses: ```text DEFAULT_DB -> master ``` Modern code explicitly uses replicas for reads, but old paths still hit the master. The master is overloaded by reads that could tolerate replica lag. ## Pattern Run two copies of the monolith: | Copy | Config | |---|---| | Monolith 1 | `DEFAULT_DB = master` | | Monolith 2 | `DEFAULT_DB = replica` | Then route with an API gateway: ```mermaid flowchart LR Client --> G[API gateway] G -->|GET /safe-read-route| R[read-drain monolith<br/>default DB = replica] G -->|POST/PUT/fallback| W[legacy monolith<br/>default DB = master] R --> Replica[(replica DB)] W --> Master[(master DB)] Master -.replication.-> Replica ``` KrakenD-style endpoint config supports path and method-specific routing, which is enough for this class of migration. ## Why It Works - No legacy code edit required. - Rollout is per route. - Rollback is a gateway rule change. - Misclassified writes fail against the replica instead of silently overloading master. - The team gains time to do proper query cleanup later. ## Required Discipline Only move routes that tolerate stale reads. Dangerous examples: - read-after-write checkout flows, - account balance reads, - confirmation pages immediately after mutation, - idempotency/retry endpoints that read and write together. Safe-ish examples: - profile display with acceptable lag, - static catalog-like reads, - dashboards with freshness labels, - account metadata where stale reads are not correctness-critical. ## Route Classification Before routing a path to the read-drain monolith, classify it: | Question | Safe Answer | |---|---| | Does it mutate state indirectly? | no writes, no audit side effects required | | Does it require read-after-write consistency? | no, stale data is acceptable | | Does it use transactions or locks? | no master-only behavior | | Does it depend on session state updated by the request? | no | | Can replica lag be shown or tolerated? | yes | If the endpoint is a GET that increments counters, touches "last seen", refreshes tokens, or lazily creates records, it is not a pure read. Either keep it on the master copy or remove the side effect first. ## Rollout Plan 1. Start with metrics for master read QPS by endpoint if available. 2. Add the second monolith with replica-default config. 3. Route one low-risk read endpoint. 4. Watch master QPS, replica lag, error rate, p95/p99 latency. 5. Expand route list gradually. 6. Later, clean the actual legacy code with evidence from the routed endpoints. ## Failure Modes | Failure | Response | |---|---| | replica lag creates confusing UI | add freshness label or route that path back | | endpoint secretly writes | replica errors reveal the route is unsafe | | gateway rule too broad | route by method and exact path, not prefix only | | read-drain fleet is overloaded | shed only drained routes or lower route percentage | | master QPS does not fall | query source was another path or background worker | ## Operational Notes Give the read-drain copy separate dashboards and logs. It should be easy to answer: which routes are drained, how much master QPS was removed, what replica lag users see, and whether errors are concentrated on one route. This pattern buys time; it does not remove the need to fix legacy data-access boundaries. Keep an owner for the follow-up cleanup or the alternate monolith becomes permanent infrastructure. ## Pro Tip This is an environment-level migration. It is useful whenever the code has too many unsafe call sites but the runtime boundary is controllable: - alternate config, - alternate DNS target, - sidecar/proxy route, - feature-flagged backend, - canary deployment.
Primary References & Engineering Sources