6.1 ML in Production
Interview-ready field notes
The 20-second answer
Production ML is a data and reliability system around a model: make predictions reproducible, monitor them, detect change, and safely improve or roll back.
Core ideas to retain
- Training-serving skew means features or preprocessing differ between training and live inference. Share feature definitions and validate schemas.
- Data drift changes input distributions; concept drift changes the relationship between inputs and outcomes. Both can hurt performance.
- Monitor system health (latency, errors), data quality, prediction distributions, and delayed business/model outcomes by important slices.
- Use versioned data, code, features, model artifacts, and a rollback path. Evaluate offline before staged deployment or A/B testing.
Interview / OA rule
A model can have great offline AUC and fail in production. In an interview, mention feature parity, data leakage checks, monitoring, and a safe release strategy.
One good written resource
Google — Monitoring Production ML Pipelines — read this after the video when you want a clearer mental model, not more pages of notes.
Most asked
Interview questions to practise aloud
Each answer is the level of detail expected for a strong fundamentals round.
What is training-serving skew?
Training inputs or transformations differ from the ones used live. The model then sees a distribution it was not evaluated on.
Data drift vs concept drift?
Data drift changes $P(X)$, the input distribution. Concept drift changes $P(Y|X)$, so the same features imply different outcomes.
What would you monitor after deployment?
Latency/errors, schema and missingness, feature/prediction distributions, outcome metrics when labels arrive, and performance across important user slices.