Blog Post··5 min read
Determinants and Invertibility
The determinant measures how much a matrix stretches or squishes space — and whether it flips orientation. When it's zero, information is lost and the matrix can't be inverted.
Tag: linear-algebra
Blog Post··7 min read
Eigenvalues and Eigenvectors
Most vectors get rotated and scaled when multiplied by a matrix. Eigenvectors are the special directions that only get scaled — and their scaling factors, the eigenvalues, reveal everything about a matrix's long-term behavior.
Blog Post··9 min read
Inner Products and Cosine Similarity
What does it mean for two vectors to be similar? Inner products measure alignment between vectors — and cosine similarity is just the dot product with magnitudes divided out.
Blog Post··8 min read
Matrices as Linear Maps
A matrix is not just a grid of numbers — it's a function that transforms space. This post builds the geometric intuition for matrix-vector multiplication as rotation, scaling, and shearing.
Blog Post··5 min read
Linear Algebra in Neural Networks
Every layer of a neural network is a matrix multiplication followed by a nonlinearity. Understanding what these matrices do geometrically — how they stretch, rotate, and project — explains why deep learning works.
Blog Post··5 min read
Positive Definite Matrices and the Geometry of Optimization
Positive definite matrices define 'bowl-shaped' quadratic forms with a unique minimum. They show up everywhere optimization problems have unique solutions — from least squares to neural network loss landscapes.
Blog Post··6 min read
Putting It All Together — From Vectors to Transformers
A tour through the whole series: how vectors, matrices, eigendecomposition, SVD, and least squares combine to explain the mathematical machinery inside modern ML systems — from PCA to attention to gradient descent.
Blog Post··4 min read
The Spectral Theorem
Symmetric matrices can always be diagonalized by an orthogonal matrix — their eigenvectors form a natural coordinate system for the data. This is the spectral theorem, and it underlies PCA, kernel methods, and graph Laplacians.
Blog Post··8 min read
Vectors: Direction, Magnitude, and the Geometry of Space
Vectors are the atoms of linear algebra — everything else is built on them. This post builds intuition for what a vector is, how addition and scaling work geometrically, and why norms give us a way to measure the world.
Blog Post··9 min read
SVD: The Geometry of Every Matrix
Every matrix, no matter how ugly, factors into a rotation, a scaling, and another rotation. That single fact — the singular value decomposition — is the engine behind PCA, image compression, latent semantic analysis, and the low-rank tricks that let us fine-tune giant models cheaply.
Blog Post··9 min read
SVD: The Geometry of Every Matrix
Every matrix, no matter how ugly, factors into a rotation, a scaling, and another rotation. That single fact — the singular value decomposition — is the engine behind PCA, image compression, latent semantic analysis, and the low-rank tricks that let us fine-tune giant models cheaply.
Blog Post··6 min read
Linear Algebra for LLMs: Vectors, Matrices, and What They Do
Every forward pass is a sequence of matrix multiplications. Understanding what those matrices do — rotate, scale, project — is the foundation for understanding why transformers work.
Blog Post··8 min read
SVD and Low-Rank Approximation: The Math Behind LoRA
LoRA works because weight matrices are empirically low-rank. SVD explains why, and the math of low-rank approximation explains how LoRA exploits it to fine-tune billion-parameter models on a laptop.
Blog Post··6 min read
Linear Algebra for LLMs: Vectors, Matrices, and What They Do
Every forward pass is a sequence of matrix multiplications. Understanding what those matrices do — rotate, scale, project — is the foundation for understanding why transformers work.
Blog Post··8 min read
SVD and Low-Rank Approximation: The Math Behind LoRA
LoRA works because weight matrices are empirically low-rank. SVD explains why, and the math of low-rank approximation explains how LoRA exploits it to fine-tune billion-parameter models on a laptop.