Broad intuition · Study map 14 of 85
Backpropagation: trace influence backward
Suggested preparation: Neurons, layers, and representationsDerivatives: a sensitivity meter
Picture it
Picture a network of connected knobs. A bad prediction raises the loss. Backpropagation works backward to measure how a tiny change in each weight would change that loss, including through earlier layers. It multiplies sensitivities along a path and adds them across paths: the calculus chain rule.
What to understand
- Loss → output → intermediate activations → every contributing parameter
- Reverse-mode automatic differentiation versus the optimizer’s update
Follow the references
1 · Video
3Blue1Brown · Backpropagation, intuitively ↗Watch the desired changes travel backward through the digit classifier.
2 · Video
3Blue1Brown · Backpropagation calculus ↗Connect that picture to products of local derivatives. Stop once the chain rule feels concrete.
Keep the distinction. There is usually no single guilty weight, and earlier weights do not create later weights. They affect activations. Backprop computes gradients; SGD or Adam uses them to update parameters.
You understand it when…
Explain why an early-layer weight receives a gradient even though it never directly touches the output.
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.