liveBook · Manning

Generative modeling

As explained in Graph Neural Networks in Action

A deep generative model is a generative model that uses neural networks to learn and generate data distributions.

Generative modeling learns the distribution of data so that new examples can be sampled from it. A deep generative model uses neural networks to do this. Unlike a discriminative model, which learns boundaries between classes, a generative model tries to capture the dataspace itself. section 5.0 section 5.1.1

Figure 5.1 Comparison of generative and discriminative tasks. On the left, the discriminative model learns to separate different images of boats and airplanes. On the right, the generative model attempts to learn the entire dataspace, which allows for new synthetic examples to be created such as a boat in the sky or an airplane on water.
figure

Compare a boundary separating boats from airplanes with a model of where their images lie in the dataspace. Modeling the latter can yield synthetic combinations, such as an airplane on water. section 5.1.1

In probability terms, classification asks for a target given an input, P(Y|X); generative modeling is described here as modeling the joint distribution P(X,Y). That is a larger job: recognizing a boat may require a few distinguishing features, while generating plausible images must capture more of the relationships in the data. section 5.1.1

Two practical uses are:

  • Create synthetic examples when collecting or labeling data is costly, including candidate drugs or rare image conditions. Check added examples for bias and noise. section 5.1.2
  • Learn low-dimensional representations without labels, then use them for tasks such as clustering or classification. section 5.1.2

For graphs, an autoencoder compresses data with an encoder and reconstructs it with a decoder; graph autoencoders commonly decode edges with a dot product. A variational graph autoencoder adds a distribution-based, regularized latent space. Use a graph autoencoder for focused reconstruction such as link prediction; favor a variational graph autoencoder when diverse synthetic graphs or subgraphs matter more. The latter can also suit noisy data, while the former is faster on clearly structured graphs. section 5.6 section 5.3.2

Choosing a graph model

To predict missing edges, train a graph autoencoder to reconstruct graph connections. To seek varied small chemical graphs for drug discovery, consider a variational graph autoencoder and its more structured latent space. section 5.3.2

How Graph Neural Networks in Action covers it

How deep it goes
Explains the probability distinction and graph autoencoder mechanisms, then compares architecture trade-offs without a rigorous derivation.
Examples
Uses boat and airplane images, synthetic faces, candidate drugs, and graph edge prediction as examples.
What it assumes
Builds from graph basics for practitioners without requiring prior graph-learning experience.
What you can do after
Apply it via PyTorch Geometric to build graph autoencoders for edge prediction and choose between GAE and VGAE for graph tasks.
How it connects
Connects generative modeling to discriminative models, self-supervised learning, dimension reduction, autoencoders, and variational graph autoencoders.
Cover of Graph Neural Networks in Action
liveBookmanning.com
Read Graph Neural Networks in Action on liveBook Get the book at manning.com

Also explained in

Compare how 29 books explain generative modeling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Graph Neural Networks in Action. The section links point to the parts of the book it draws on.