The evidence lower bound (ELBO) is a variational objective for training a variational autoencoder. Minimizing negative ELBO loss indirectly reduces reconstruction error and the KL divergence between the guide distribution Q(Z|…) and the target posterior P(Z|…). section 5.2.3
Training uses stochastic variational inference, optimizing the encoder’s variational distribution with randomly selected data batches to reduce memory use and scale training. section 5.2.3
Pyro training setup
Configure Pyro’s SVI with an Adam optimizer, a VAE model and guide, and Trace_ELBO as the loss; evaluate loss over test batches during training. section 5.2.3
Listing 5.13 Set up the training procedure
from pyro.infer import SVI, Trace_ELBO
from pyro.optim import Adam
pyro.clear_param_store() #1
vae = VAE() #2
train_loader, test_loader = setup_dataloaders(batch_size=256) #3
svi_adam = Adam({"lr": 1.0e-3}) #4
model = vae.training_model #5
guide = vae.training_guide #5
svi = SVI(model, guide, svi_adam, loss=Trace_ELBO()) #5
Inspect the compact Pyro configuration that connects SVI, Adam, the model, the guide, and Trace_ELBO. section 5.2.3
