1.1 Linear Algebra
Interview-ready field notes
The 20-second answer
Linear algebra is the language of data and model weights: vectors hold features, matrices transform them, and dot products measure alignment.
Core ideas to retain
- A vector is an ordered list of features; a matrix applies one linear transformation to many features.
- Dot product measures alignment. Cosine similarity normalizes it, so it compares direction rather than length.
- Matrix shapes must line up: . This is the fastest way to catch OA bugs.
- Eigenvectors keep their direction under a transform; PCA keeps the directions with the largest variance. SVD generalizes this idea to rectangular matrices.
Interview / OA rule
For an embedding search question, say: normalize embeddings and rank by dot product/cosine similarity. For PCA, center data first, then retain top-variance directions.
One good written resource
3Blue1Brown — Essence of Linear Algebra — read this after the video when you want a clearer mental model, not more pages of notes.
Most asked
Interview questions to practise aloud
Each answer is the level of detail expected for a strong fundamentals round.
Why use cosine similarity for embeddings?
It compares angle, so a long document vector does not automatically look more similar than a short one. If vectors are L2-normalized, cosine similarity equals their dot product.
What does a matrix multiplication layer do?
It mixes input features into new features. Each output is a weighted sum of inputs, so $Wx+b$ is exactly a learned linear transformation.
PCA vs SVD?
PCA finds high-variance directions in centered data; SVD is a general matrix factorization. PCA is commonly computed with SVD in practice.