Series

Equation Cheatsheets

Every key equation, annotated. Each term links to a post explaining the why — the intuition, the derivation, and an interactive visual. A reference layer for the blog.

1. 1

   Cheatsheet: RL Loss Functions

   PPO and GRPO loss functions annotated term-by-term — the clipped surrogate, GAE, value loss, entropy bonus, and group-normalised advantages — with links to the posts explaining each design choice.

   2025-01-10

2. 1

   Cheatsheet: RL Loss Functions

   PPO and GRPO loss functions annotated term-by-term — the clipped surrogate, GAE, value loss, entropy bonus, and group-normalised advantages — with links to the posts explaining each design choice.

   2025-01-10

3. 3

   Cheatsheet: Soft Metrics

   Perplexity, negative log-likelihood, entropy, KL divergence, and cross-entropy annotated term-by-term — what each measures, how they relate, and what values to expect during training.

   2025-01-10

4. 3

   Cheatsheet: Soft Metrics

   Perplexity, negative log-likelihood, entropy, KL divergence, and cross-entropy annotated term-by-term — what each measures, how they relate, and what values to expect during training.

   2025-01-10

5. 4

   Cheatsheet: Attention

   Every equation in scaled dot-product attention and multi-head attention annotated term-by-term — the scaling, the softmax, the heads, RoPE, and KV cache — with links to the posts explaining each design choice.

   2025-01-10

6. 4

   Cheatsheet: Attention

   Every equation in scaled dot-product attention and multi-head attention annotated term-by-term — the scaling, the softmax, the heads, RoPE, and KV cache — with links to the posts explaining each design choice.

   2025-01-10

7. 5

   Cheatsheet: LLM Architectures

   Five LLM architectures — GPT-2, Qwen3-8B, DeepSeek-V3, DeepSeek-R1, GPT-OSS-20B/120B — shown as interactive block diagrams. Click any block to expand equations and parameters. Each model is sourced from its official HF config.json.

   2025-01-10

8. 5

   Cheatsheet: LLM Architectures

   Five LLM architectures — GPT-2, Qwen3-8B, DeepSeek-V3, DeepSeek-R1, GPT-OSS-20B/120B — shown as interactive block diagrams. Click any block to expand equations and parameters. Each model is sourced from its official HF config.json.

   2025-01-10

9. 6

   Cheatsheet: LLM Forward Pass Equations

   The full forward pass, written out as equations, for GPT-2, Qwen3-8B, DeepSeek-V3, and GPT-OSS. Every matrix, every norm, every residual — in the order the computation actually happens.

   2025-01-10

10. 6

    Cheatsheet: LLM Forward Pass Equations

    The full forward pass, written out as equations, for GPT-2, Qwen3-8B, DeepSeek-V3, and GPT-OSS. Every matrix, every norm, every residual — in the order the computation actually happens.

    2025-01-10