liveBook · Manning

Generative modeling

As explained in Machines that Think

A generative architecture is a neural-network design organized to learn data patterns and synthesize new examples that follow them.

Generative modeling organizes neural networks to learn patterns in data and synthesize new samples that reflect them. Unlike networks designed to classify, predict, or compress, these architectures aim to create; their design encodes assumptions about how information is organized. section 3.4.6 section 3.4.7

Two approaches illustrate different ways to learn patterns for generation:

  • Variational autoencoders (VAEs) encourage a smooth, continuous latent space, where nearby points can produce similar outputs. section 3.4.6
  • Generative adversarial networks (GANs) train a generator to create samples and a discriminator to distinguish generated samples from real data. section 3.4.6
Figure 3.17 A generative adversarial network trains two models in opposition. The generator begins with random noise and tries to produce images that resemble those in the training set. The discriminator receives a random mixture of real images from the dataset and fake images created by the generator and learns to classify them as genuine or synthetic. During training, each model provides feedback to the other: the discriminator improves its ability to detect fakes, and the generator adjusts its parameters to create images that are increasingly difficult to distinguish from real ones. Through this adversarial loop, the system gradually learns to synthesize new samples that follow the same patterns as the data on which it was trained.

The figure traces a GAN’s feedback loop: the generator starts from random noise, while the discriminator compares generated and real images; each model adjusts as the other improves. section 3.4.6

Generative architectures have been used for images, audio, style transfer, and creative applications. Plausible outputs show what a model has learned about data patterns, but do not by themselves establish human-like understanding. section 3.4.6

How Machines that Think covers it

How deep it goes
Explains latent-space generation in VAEs and adversarial training in GANs without detailing implementation.
Examples
Uses image generation, audio, style transfer, creative applications, and a Picasso-forger analogy as examples.
What you can do after
Understand how VAEs and GANs generate samples through different architectural mechanisms.
How it connects
Connects generative models with neural-network architecture, latent spaces, variational autoencoders, and generative adversarial networks.

Also explained in

Compare how 32 books explain generative modeling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Machines that Think. The section links point to the parts of the book it draws on.