Broad intuition · Study map 6 of 85
Vectors and matrices: coordinates that can move
Suggested preparation: Optimization beyond gradients
Picture it
Think of a vector as coordinates for a thing. A matrix turns those coordinates into a new view. A dot product asks how strongly two directions line up; a learned layer repeats this question using many learned directions.
What to understand
- Shapes, matrix multiplication, dot products, cosine similarity
- Basis changes, projections, SVD, and low-rank approximation
Follow the references
1 · Video + article
3Blue1Brown · Linear transformations and matrices ↗Start with the moving grid and matrix columns. Follow the chapter links to vectors, multiplication, and dot products.
Keep the distinction. Embedding length can contain useful information. Choose cosine or dot product to match how the embedding model was trained.
You understand it when…
What shape maps a batch of 16 vectors with 128 features into 64 features each? Why can a low-rank matrix save work?
This is a study map. Follow the reference for the full explanation; use the question to test your mental model. Browse the reference shelf.