Papers

R1-Zero shows that pure RL on a base model — no supervised warmup — can grow chain-of-thought reasoning on its own; R1 adds a small cold-start SFT stage to make that reasoning readable.

GRPO eliminates the value network from PPO by estimating advantage from group-relative reward normalization, halving memory cost while matching quality on outcome-reward tasks.

KIVI quantizes both keys and values to 2 bits, with the twist that keys are quantized per-channel and values per-token. That asymmetry — matched to where each tensor's outliers live — is what makes 2-bit work without any fine-tuning.

LongRoPE extends RoPE-based models beyond 2 million tokens by searching for non-uniform per-dimension rescaling factors via evolutionary algorithms, applying a two-stage extension strategy, and recovering short-context performance through a brief re-fine-tuning step.

V-JEPA extends the JEPA recipe to video: mask spatiotemporal tubes of a clip and predict their representations from the visible context, in latent space. With no pixel reconstruction and no contrastive loss, it learns features that transfer strongly to action recognition — especially motion-driven tasks — at lower compute than pixel-reconstruction baselines.

Speculative decoding uses a cheap draft model to propose K tokens, then verifies them all in one target model forward pass — achieving 2-3× speedup with mathematically guaranteed identical output distribution.

DPO eliminates the reward model training step by directly optimizing preferences as a classification objective.

PagedAttention applies virtual memory paging to KV cache management, eliminating fragmentation and enabling 2-4× throughput improvement in LLM serving.

StreamingLLM identifies the 'attention sink' — the first few tokens absorb a huge share of attention regardless of meaning. Keeping those sink tokens plus a sliding window lets a model generate over effectively unbounded streams without re-computation, at a 22× speedup over the recompute baseline.

Grouped-query attention interpolates between multi-head and multi-query attention: groups of query heads share one key/value head. It cuts the KV cache by the grouping factor while keeping nearly all of multi-head's quality, and it can be uptrained cheaply from an existing multi-head checkpoint.

H2O evicts KV cache entries by cumulative attention mass — a small set of 'heavy hitter' tokens accumulate most of the attention and are kept, the rest are dropped. It holds quality with a fraction of the cache and reports large throughput gains, all without fine-tuning.

Language models consistently perform worse when relevant information appears in the middle of a long context, regardless of model size or context length — a U-shaped performance curve with peaks at the beginning and end that reveals a structural attention bias in models trained on short sequences.

DiT replaces the diffusion U-Net with a Vision Transformer that patchifies image latents and conditions through adaLN-Zero. It finds that diffusion scales with model size and compute the way language models do — DiT-XL/2 reaches FID 2.27 on ImageNet 256×256 — and became the backbone for Sora and Stable Diffusion 3.

I-JEPA learns image representations by predicting the embeddings of masked target blocks from a visible context block — in representation space, not pixel space, and with no hand-designed augmentations. It matches masked autoencoders on ImageNet at roughly 10× less compute and transfers better to spatial tasks.

YaRN extends the effective context window of RoPE-based models beyond their training length by combining non-uniform frequency scaling with an attention temperature correction, achieving competitive long-context performance with far less fine-tuning than linear interpolation.

Classifier-free guidance gets the fidelity boost of classifier guidance without the classifier: train one conditional model that randomly drops its conditioning, then at sampling time extrapolate along the line from the unconditional to the conditional prediction. It is the guidance method behind Stable Diffusion, DALL·E 2, and Imagen.

Constitutional AI replaces human harmlessness labels with AI feedback: the model critiques and revises its own outputs against a written set of principles, then trains on AI-generated preferences (RLAIF).

FlashAttention computes exact attention in tiles that fit in on-chip SRAM, never materializing the N×N score matrix in slow GPU memory. The result is 2–4× faster attention with linear memory instead of quadratic — and FlashAttention-2 nearly doubles it again with better GPU work partitioning.

Latent diffusion runs the diffusion process in the compressed latent space of a pretrained autoencoder rather than on raw pixels, cutting compute by 4–8× at matched quality and introducing cross-attention conditioning. It is the architecture behind Stable Diffusion.

Self-Instruct bootstraps an instruction-tuning dataset from a language model itself: 175 seed tasks expand to 52K instructions via generation, filtering, and ROUGE-based deduplication.

Chinchilla showed that for a fixed compute budget, model size and training tokens should scale roughly equally — and that the large models of the day were badly undertrained. A 70B model trained on 1.4T tokens beat the 280B Gopher at the same compute.

InstructGPT introduced the SFT → reward model → PPO pipeline that became standard RLHF, and showed a 1.3B aligned model can be preferred to 175B GPT-3.

The paper that pushed diffusion past GANs on ImageNet. It introduced adaptive group normalization (AdaGN) for timestep and class conditioning, and classifier guidance — using a separately trained classifier's gradients to steer the reverse process toward a target class.

Rotary Position Embedding encodes absolute position by rotating query and key vectors, so that their dot product depends only on relative position. It became the positional scheme of LLaMA, PaLM, and GPT-NeoX, and it extends to longer contexts cleanly through frequency interpolation.

DDIM keeps DDPM's training objective untouched but replaces the Markovian reverse chain with a non-Markovian one, yielding a deterministic sampler that reaches DDPM-quality images in 20–50 steps instead of 1000 — and a stable, invertible map between noise and image.

DDPM turns image generation into a thousand-step denoising problem: a fixed forward process grinds an image into Gaussian noise, and a U-Net learns to reverse it one step at a time — trained by the deceptively simple objective of predicting the noise.

Pre-LN transformers place LayerNorm inside the residual branch rather than after it, giving bounded gradient magnitudes at initialization and eliminating the need for learning rate warmup.

RMSNorm removes mean-centering and the shift parameter from LayerNorm, achieving ~15% speedup with negligible quality loss — now standard in LLaMA, Gemini, and DeepSeek.

When a proxy reward is optimized past the point where it correlates with true intent, performance on the actual objective collapses.

The paper that introduced the Transformer — multi-head self-attention with no recurrence and no convolution. It set a new BLEU record on WMT 2014 English-German, trained faster than the RNNs it replaced, and became the substrate for every large language model since.

PPO stabilizes policy gradient training by clipping the objective to prevent destructively large policy updates, becoming the dominant algorithm for RLHF.

GAE introduces a single parameter λ that continuously interpolates between high-bias/low-variance TD and low-bias/high-variance Monte Carlo advantage estimates.