Series
Self-Supervised Learning and World Models
From contrastive learning to JEPA and beyond — how machines learn without labels and why it points toward world models.
- 1
Why Self-Supervised Learning? The Label Bottleneck
Supervised learning requires labels. Labels require humans. At scale, that's the bottleneck. Self-supervised learning sidesteps it by constructing supervision from the data itself.
2024-06-19
- 1
Why Self-Supervised Learning? The Label Bottleneck
Supervised learning requires labels. Labels require humans. At scale, that's the bottleneck. Self-supervised learning sidesteps it by constructing supervision from the data itself.
2024-06-19
- 2
Contrastive SSL: SimCLR, MoCo, and DINO
Contrastive learning teaches a model that two views of the same image should be close in representation space, and views of different images should be far apart. The details of how you enforce this determine everything.
2024-06-19
- 2
Contrastive SSL: SimCLR, MoCo, and DINO
Contrastive learning teaches a model that two views of the same image should be close in representation space, and views of different images should be far apart. The details of how you enforce this determine everything.
2024-06-19
- 3
Masked Autoencoders: Learning by Filling in the Blanks
Mask 75% of an image's patches. Train a model to reconstruct them. The result is a rich visual representation — and the recipe works because pixels are redundant and structure is not.
2024-06-19
- 3
Masked Autoencoders: Learning by Filling in the Blanks
Mask 75% of an image's patches. Train a model to reconstruct them. The result is a rich visual representation — and the recipe works because pixels are redundant and structure is not.
2024-06-19
- 4
JEPA: Predicting in Representation Space
MAE predicts pixels. Contrastive methods match views. JEPA predicts representations of target regions from context regions — in an abstract space where irrelevant details have already been discarded.
2024-06-19
- 4
JEPA: Predicting in Representation Space
MAE predicts pixels. Contrastive methods match views. JEPA predicts representations of target regions from context regions — in an abstract space where irrelevant details have already been discarded.
2024-06-19
- 5
I-JEPA: Self-Supervised Vision at Scale
I-JEPA applies the JEPA idea to images: predict the representations of target patches from a context region, without any view-level augmentations. The result transfers better to semantic tasks than pixel-level methods.
2024-06-19
- 5
I-JEPA: Self-Supervised Vision at Scale
I-JEPA applies the JEPA idea to images: predict the representations of target patches from a context region, without any view-level augmentations. The result transfers better to semantic tasks than pixel-level methods.
2024-06-19
- 6
V-JEPA: Predicting the Future in Representation Space
V-JEPA extends JEPA to video: predict the representations of future or masked frames from context frames. No pixel reconstruction, no contrastive loss — just abstract prediction across time.
2024-06-19
- 6
V-JEPA: Predicting the Future in Representation Space
V-JEPA extends JEPA to video: predict the representations of future or masked frames from context frames. No pixel reconstruction, no contrastive loss — just abstract prediction across time.
2024-06-19
- 7
World Models: The Bigger Picture Behind JEPA
JEPA is a learning architecture. World models are the goal it points toward — internal simulators that can predict the consequences of actions and support planning without interacting with the real world.
2024-06-19
- 7
World Models: The Bigger Picture Behind JEPA
JEPA is a learning architecture. World models are the goal it points toward — internal simulators that can predict the consequences of actions and support planning without interacting with the real world.
2024-06-19