chapter eight
8 Simplicity, hidden in complexity
This chapter covers
- How compression reveals secret structure
- How complexity peaks between order and noise
- How simplicity beats overfitting
- Grokking (or memorize, then simplify)
- The question of whether language models are just blurry JPEGs of the internet
Sutskever has argued that any good prediction model is implicitly a good compressor, and vice versa [1]. In one talk, he cites 2017 research on the “sentiment neuron,” in which OpenAI researchers—including Sutskever—trained a long short-term memory (LSTM) to predict the next character in Amazon product reviews, but discovered that a single neuron captured the review’s sentiment [2]. The sentiment neuron arises because predicting the next character forces the model to encode sentiment in a latent variable.