Deep learning: follow the signals
Understand layers as transformations and training as sensitivity plus updates. Then recognize the different architectures built from those ideas.
Start small
Draw a two-layer network. Keep parameters, activations, gradients, and updates distinct.
- Neurons, layers, and representations →
- Backpropagation: trace influence backward →
- Optimizers: walking with noisy directions →
Video series · Andrej Karpathy · Neural Networks: Zero to Hero ↗
Learn useful coordinates
Follow how an objective arranges embeddings, then how attention changes those representations with context.
Recognize the families
Compare local vision detectors, recurrent state, graph messages, and denoising. Ask what each architecture makes easy and what it compresses away.
- Vision: sliding detectors and image tokens →
- Sequences: recurrence, attention, and state →
- Graphs and recommenders: learning from relationships →
- VAEs, GANs, diffusion, and flow matching →
Book · Dive into Deep Learning ↗