liveBook · Manning

Generative modeling

As explained in Grokking Machine Learning, Second Edition

A model that learns the structure of data well enough to generate new data resembling its training dataset.

A spam filter answers a yes-or-no question, and a house-price model produces a number. Generative modeling tackles the harder creative task: producing new text, images, music, or other data that resembles examples it has seen. section 1.13

Generative models learn how a dataset looks well enough to create new data points from that pattern. They belong to unsupervised learning because they can learn from existing data without relying on labels. section 2.10 section 2.14

From face dataset to new faces

Give a model many face images. Rather than classifying whether an image contains a face, it learns the visual structure of those examples and produces realistic-looking faces that were not in the dataset. Similar systems can generate paintings, videos, music, stories, and poetry. section 2.14

Important families include:

  • Generative adversarial networks (GANs), variational autoencoders, and restricted Boltzmann machines (RBMs). section 2.14
  • Generative modeling is harder than classification because creating a convincing example requires learning much more about the data. section 1.13 section 2.14
Figure 1.10 Predictive machine learning is akin to answering questions, such as multiple choice. Generative learning is akin to writing an essay or drawing an image.

Compare predictive learning with generative learning through the contrast between answering a multiple-choice question and writing an essay or drawing an image. section 1.13

How Grokking Machine Learning, Second Edition covers it

How deep it goes
Defines generation and names major model families, but defers detailed algorithms and implementation.
Examples
Uses faces, realistic images, videos, paintings, music, stories, and poetry as generated outputs.
What it assumes
Builds on predictive machine learning, especially neural networks, while introducing generation as a distinct task.
What you can do after
Evaluate/understand how generative models differ from predictive models and what kinds of data they can produce.
How it connects
Connects generative modeling with unsupervised learning, predictive machine learning, neural networks, GANs, variational autoencoders, and RBMs.

Also explained in

Compare how 32 books explain generative modeling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning, Second Edition. The section links point to the parts of the book it draws on.