2.2 Regularization
Interview-ready field notes
The 20-second answer
Regularization deliberately limits or perturbs a model so it learns patterns that transfer beyond its training examples.
Core ideas to retain
- L1 adds and can drive weights exactly to zero; L2 adds and smoothly shrinks them.
- Dropout randomly removes activations during training, reducing brittle co-adaptation. Turn it off for inference.
- Early stopping selects the checkpoint with the best validation score before the model memorizes noise.
- Augmentation adds label-preserving variation, such as image crops or text paraphrases, to increase effective data diversity.
Interview / OA rule
Name the symptom before the fix. Regularization is for a generalization gap, not for a model that cannot fit the training set.
One good written resource
Google — Overfitting and Regularization — read this after the video when you want a clearer mental model, not more pages of notes.
Most asked
Interview questions to practise aloud
Each answer is the level of detail expected for a strong fundamentals round.
L1 vs L2 regularization?
L1 tends to produce sparse, exactly-zero weights; L2 shrinks all weights smoothly and is the usual weight-decay choice in neural nets.
Why does dropout help?
A unit cannot rely on a specific companion always being present, so features become less co-dependent. It behaves like averaging many thinned networks.
Is batch normalization regularization?
Its main job is optimization/stable activations; batch noise can add a mild regularizing effect, but it is not a substitute for validation-based regularization.