Variational autoencoder

As explained in Machine Learning Algorithms in Depth

A variational autoencoder is a generative neural network that learns an approximate latent-variable posterior and reconstructs observations from sampled latent representations.

A variational autoencoder (VAE) is a generative encoder–decoder model. An ordinary autoencoder compresses an input through a bottleneck and reconstructs it; a VAE instead encodes the input as parameters of a distribution, samples a latent value, and decodes that sample. Training also structures the latent space so sampled points can produce valid outputs. section 11.1

Figure 11.2 Variational autoencoder architecture, showing an encoder, a sampling layer, and a decoder

The architecture diagram locates the sampling layer between the encoder and decoder. section 11.1

Let x be an observation and z a latent variable. The trainable encoder qϕ(z|x) approximates a posterior distribution; the trainable decoder pθ(x|z) describes reconstructions; and p(z) is the latent prior. The evidence lower bound (ELBO) balances reconstruction from sampled z against a KL-divergence penalty that draws the approximate posterior toward the prior. Thus, optimizing reconstruction alone is not the VAE objective. section 11.1 section 11.1.1

Time-series anomaly detection

Train an LSTM-VAE on nonanomalous sensor signals. For each of n sensors, a Gaussian output uses a mean and variance, giving 2n output parameters. The sampled joint latent representation is encouraged to approximate a standard normal distribution. At detection time, a drop in an input’s log likelihood under the learned normal behavior can be thresholded as an anomaly; a square-wave signal illustrates the result. The detector and its custom loss are implemented from scratch with Keras/TensorFlow using the NAB dataset. section 11.1.1

How Machine Learning Algorithms in Depth covers it

How deep it goes

Defines encoder, decoder, prior, and ELBO terms, then implements an LSTM-VAE detector with a custom loss.

Examples

Works through an LSTM-VAE anomaly detector on the NAB dataset and a square-wave signal.

What it assumes

Builds on encoder–decoder autoencoders while defining the latent variable, encoder and decoder distributions, and prior.

What you can do after

Build it from scratch with Keras/TensorFlow to flag time-series anomalies by thresholding drops in log likelihood.

How it connects

Connects autoencoders, approximate posterior distributions, KL divergence, Gaussian likelihoods, and LSTMs.

liveBookmanning.com

Read Machine Learning Algorithms in Depth on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)A variational autoencoder is a generative neural model that encodes data into a lower-dimensional probabilistic latent representation and decodes that representation into reconstructed or generated data.
• 

  liveBookmanning.com
  Causal AIA variational autoencoder is a generative neural-network model that combines an encoder-defined approximating distribution with a decoder-defined generative distribution over observed data and latent variables.
• 

  liveBookmanning.com
  Learn Generative AI with PyTorchA generative model that learns an encoder-decoder latent-variable representation so it can reconstruct inputs and sample new outputs from latent space.
• 

  liveBookmanning.com
  Outlier Detection in PythonA variational autoencoder is a generative neural-network model that learns a probabilistic latent representation and reconstructs or generates observations from it.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningAn autoencoder that models the latent space probabilistically and is trained with regularizing constraints.
• 

  liveBookmanning.com
  Deep Learning with Python, Third EditionA generative latent-variable model that learns an encoder-decoder structure and a probabilistic latent space for reconstructing and sampling data.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksA variational autoencoder is a neural network that encodes inputs as probability distributions, samples from those distributions, and decodes the samples to reconstruct or generate data.
• 

  liveBookmanning.com
  Generative AI in ActionA variational autoencoder is a generative neural model that encodes data into a probabilistic latent representation and decodes samples from that representation into reconstructed or newly generated data.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA variational autoencoder is a probabilistic autoencoder whose encoder represents inputs as parameters of a latent probability distribution and whose decoder reconstructs or generates data from sampled latent points.
• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA variational autoencoder is a neural network architecture that encodes data into a probabilistic latent space and decodes latent representations to reconstruct or generate data.
• 

  liveBookmanning.com
  Data Without LabelsA variational autoencoder is a neural-network generative model that learns a probabilistic latent representation and reconstructs data from sampled latent variables.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionA variational autoencoder is a probabilistic autoencoder that learns a parameterized latent-variable distribution and decodes samples from it to reconstruct or generate data.

Compare how 13 books explain variational autoencoder →

Related concepts

• encoderhas part · An encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
• autoencoderkind of · An autoencoder is a neural network that encodes input data into an internal representation and decodes it to reconstruct the input or an approximation.
• decodehas part · Decoding is the process of transforming an encoded or internal representation into an interpretable or target-form output.
• generationkind of · Generation is the computational production of outputs from supplied inputs or context, often by applying a model to produce predictions or new content.
• reconstruction lossuses · Reconstruction loss is a measure of the discrepancy between an input or observation and the output a model reconstructs from it or its representation.
• image generationuses · Image generation is the computational creation of new images from learned models, input data, or specified conditions.

This explainer was generated by Manning's knowledge graph from the text of Machine Learning Algorithms in Depth. The section links point to the parts of the book it draws on.