epilogue The missing pieces

Sutskever’s List promises to explain 90% of what matters, and it delivers on that promise with unusual clarity. It reconstructs the path from feature engineering to large-scale supervised learning and from there to the self-supervised breakthroughs that made ChatGPT possible. Yet the remaining 10% is not an afterthought. It encompasses other major traditions in AI that were advancing in parallel with language-model pretraining, including deep reinforcement learning, self-supervised vision, multimodal grounding, diffusion models, and the emergence of scientific foundation models such as AlphaFold. Just as importantly, Sutskever’s List stops at 2020, leaving the subsequent phase of AI development outside its scope. This epilogue will therefore address not only the missing 10% but also the full 100% of what has occurred since.

E.1 Deep reinforcement learning

E.1.1 Masked language models

E.1.2 Self-supervised vision

E.2 The missing 100%

E.2.1 Multimodal foundations

E.2.2 Diffusion models

E.2.3 AI for science

E.2.4 Knowledge distillation

E.2.5 Alignment

E.2.6 Tools and knowledge

E.2.7 Mixture-of-Experts

E.2.8 Reasoning models