Series
Ranking and Rank
Two kinds of rank shape modern LLMs: the rank that orders models by preference — Bradley–Terry, ELO, and reward models — and the rank of a matrix — SVD and low-rank adaptation. This series develops both from first principles.
- 1
The Bradley–Terry Model: From ELO Scores to Reward Models
Chatbot Arena ranks LLMs with ELO, InstructGPT trains a reward model on pairwise preferences, and chess has rated players for seventy years. All three rest on the same one-line probabilistic model — Bradley–Terry — which turns out to be logistic regression over comparisons.
2026-06-20
- 1
The Bradley–Terry Model: From ELO Scores to Reward Models
Chatbot Arena ranks LLMs with ELO, InstructGPT trains a reward model on pairwise preferences, and chess has rated players for seventy years. All three rest on the same one-line probabilistic model — Bradley–Terry — which turns out to be logistic regression over comparisons.
2026-06-20
- 2
SVD: The Geometry of Every Matrix
Every matrix, no matter how ugly, factors into a rotation, a scaling, and another rotation. That single fact — the singular value decomposition — is the engine behind PCA, image compression, latent semantic analysis, and the low-rank tricks that let us fine-tune giant models cheaply.
2026-06-20
- 2
SVD: The Geometry of Every Matrix
Every matrix, no matter how ugly, factors into a rotation, a scaling, and another rotation. That single fact — the singular value decomposition — is the engine behind PCA, image compression, latent semantic analysis, and the low-rank tricks that let us fine-tune giant models cheaply.
2026-06-20
- 3
Low-Rank Approximation: From SVD to LoRA and Beyond
Eckart–Young says the truncated SVD is the optimal low-rank approximation. This post turns that theorem into engineering — factorized layers, truncated-SVD compression, LoRA and its descendants QLoRA, DoRA, and GaLore, and nuclear-norm minimization for matrix completion.
2026-06-20
- 3
Low-Rank Approximation: From SVD to LoRA and Beyond
Eckart–Young says the truncated SVD is the optimal low-rank approximation. This post turns that theorem into engineering — factorized layers, truncated-SVD compression, LoRA and its descendants QLoRA, DoRA, and GaLore, and nuclear-norm minimization for matrix completion.
2026-06-20