Generative modeling produces new text, images, video, or sound instead of merely classifying existing data. A prompt can elicit informative or creative content from patterns learned during training. section 1.8 section 20.1.1
A prompt-driven application
ChatGPT, Gemini, and Claude generate text from simple prompts, while Midjourney generates images. These applications rely on foundation models rather than special-purpose programming for every new task. section 1.8
The basic training-and-generation pattern is:
- Train a large model to reconstruct missing or corrupted input, such as predicting the next word or denoising an image. section 1.8 section 20.1.1
- Use vast quantities of unlabeled data, avoiding the manual annotations that constrain conventional machine learning. section 1.8
- Store learned knowledge in billions of parameters and use the resulting representations to sample new inputs. section 20.1.1
- Prompt the trained model to retrieve learned patterns and return the output most likely associated with the request. section 1.8
Generative modeling is a subset of deep learning, whose differentiable layers learn weights through loss minimization and gradient descent. Its practical impact is substantial, but predictions of immediate mass unemployment, enormous productivity gains, or imminent superintelligence remain hype rather than established outcomes. section 20.1.1 section 1.10
