Series
Linear Algebra for ML
From vectors and dot products through eigendecomposition and SVD — the mathematical foundations that underpin modern ML systems, built up from geometry with interactive visualizations at every step.
- 1
Vectors: Direction, Magnitude, and the Geometry of Space
Vectors are the atoms of linear algebra — everything else is built on them. This post builds intuition for what a vector is, how addition and scaling work geometrically, and why norms give us a way to measure the world.
2026-07-02
- 2
Inner Products and Cosine Similarity
What does it mean for two vectors to be similar? Inner products measure alignment between vectors — and cosine similarity is just the dot product with magnitudes divided out.
2026-07-02
- 3
Matrices as Linear Maps
A matrix is not just a grid of numbers — it's a function that transforms space. This post builds the geometric intuition for matrix-vector multiplication as rotation, scaling, and shearing.
2026-07-02
- 4
Determinants and Invertibility
The determinant measures how much a matrix stretches or squishes space — and whether it flips orientation. When it's zero, information is lost and the matrix can't be inverted.
2026-07-02
- 5
Eigenvalues and Eigenvectors
Most vectors get rotated and scaled when multiplied by a matrix. Eigenvectors are the special directions that only get scaled — and their scaling factors, the eigenvalues, reveal everything about a matrix's long-term behavior.
2026-07-02
- 10
The Spectral Theorem
Symmetric matrices can always be diagonalized by an orthogonal matrix — their eigenvectors form a natural coordinate system for the data. This is the spectral theorem, and it underlies PCA, kernel methods, and graph Laplacians.
2026-07-02
- 11
Positive Definite Matrices and the Geometry of Optimization
Positive definite matrices define 'bowl-shaped' quadratic forms with a unique minimum. They show up everywhere optimization problems have unique solutions — from least squares to neural network loss landscapes.
2026-07-02
- 12
Linear Algebra in Neural Networks
Every layer of a neural network is a matrix multiplication followed by a nonlinearity. Understanding what these matrices do geometrically — how they stretch, rotate, and project — explains why deep learning works.
2026-07-02
- 13
Putting It All Together — From Vectors to Transformers
A tour through the whole series: how vectors, matrices, eigendecomposition, SVD, and least squares combine to explain the mathematical machinery inside modern ML systems — from PCA to attention to gradient descent.
2026-07-02