An objective function defines what training should minimize. A loss function quantifies the discrepancy between model predictions and desired outputs, turning a modeling goal into a measurable training signal. section 1.2.2 section 3.4
Training objectives take different forms according to the model and task:
- Image classification selects an appropriate loss function and optimizer, then trains on batches before evaluating unseen test images. section 3.4
- A variational autoencoder balances reconstruction accuracy against a smooth, continuous latent-space distribution. section 9.1.1
- DALL-E mini minimizes cross-entropy between predicted image-token sequences and ground-truth sequences from a VQGAN encoder. section 12.1.1
Text-to-image training
Image–caption pairs provide the training data. A VQGAN encoder converts each image into discrete tokens, while a BART encoder processes its caption. A BART decoder then predicts the image tokens sequentially, and cross-entropy measures disagreement with the actual token sequence. section 12.1.1
Figure 12.2 Training pipeline of the DALL-E mini model
The pipeline diagram connects captions, pretrained encoders, autoregressive decoding, image tokens, and the training objective. section 12.1.1
