Blog Post··13 min read
Why Your Norms Are Screaming at You
Activation norms, gradient norms, weight norms — what they tell you, why they explode or vanish, and every practical cause from hyperparameters to data bugs.
Tag: debugging
Blog Post··13 min read
Why Your Norms Are Screaming at You
Activation norms, gradient norms, weight norms — what they tell you, why they explode or vanish, and every practical cause from hyperparameters to data bugs.
Blog Post··13 min read
Debugging Transformer Training Runs: Reading the Curves
Most training failures leave signatures in the metrics before they fully manifest. Here's how to read loss curves, gradient norms, learning rate schedules, and activation statistics to diagnose what's going wrong.
Blog Post··13 min read
Debugging Transformer Training Runs: Reading the Curves
Most training failures leave signatures in the metrics before they fully manifest. Here's how to read loss curves, gradient norms, learning rate schedules, and activation statistics to diagnose what's going wrong.