Deployments: version the entire inference contract

Picture it

Upgrading a model is like replacing the engine of a plane while it's still flying. The weights, the prompt templates, and the internal runtimes are all tightly interlocking gears; changing just one can unexpectedly shift the entire system's center of gravity. That's why we slowly bleed a tiny sliver of traffic to the new setup—a canary in the coal mine—watching closely for subtle tremors before we commit the entire fleet to the new trajectory.

What to understand

• Immutable artifacts, hashes, provenance, model license/access terms
• Startup/readiness/liveness, shadow traffic, canaries, drain, and rollback

Follow the references

1. 1 · Book chapter

   Google SRE Workbook · Canarying releases ↗

   Use the analysis and rollback sections to define measurable release gates.

2. 2 · Documentation

   Kubernetes · Schedule GPUs ↗

   Review GPU requests and node selection before writing the deployment manifest.

Make it concrete

Write a release manifest and rollback plan. Canary a harmless configuration change against latency, errors, and a fixed quality set; drain active streams before shutdown.