Series
The Architecture Playground
Interactive research blogs where you can turn the dials yourself — drag the config of a transformer and watch the weight matrices, KV cache, and expert routing recompute in real time.
Interactive
architecturetransformersinteractivekv-cachemoe
- 1
The Architecture Playground: What a Transformer Config Actually Buys You
An interactive research blog. Drag the config of a decoder-only transformer — hidden size, head counts, FFN type — and watch the parameter count, KV cache, and mixture-of-experts routing recompute live.
2024-06-19
- 1
The Architecture Playground: What a Transformer Config Actually Buys You
An interactive research blog. Drag the config of a decoder-only transformer — hidden size, head counts, FFN type — and watch the parameter count, KV cache, and mixture-of-experts routing recompute live.
2024-06-19