liveBook · Manning

Training

As explained in Learn Generative AI with PyTorch

The iterative process of feeding data through a model, computing loss, and updating parameters.

Training feeds data through a model and adjusts its parameters so it can learn patterns in the data. For a clothing-image GAN, training alternates between updating the discriminator on real samples, updating it on generated samples, and updating the generator. section 4.1.3

A practical check is to inspect generated outputs over time. In the clothing example, the untrained generator produces noise-like images; after training, its outputs begin to resemble clothing. section 4.1.3

Listing 4.3 Training GANs for clothing item generation
for i in range(50):    
    gloss=0
    dloss=0
    for n, (real_samples,_) in enumerate(train_loader):
        loss_D=train_D_on_real(real_samples)            #1
        dloss+=loss_D
        loss_D=train_D_on_fake()                        #2
        dloss+=loss_D
        loss_G=train_G()                                #3
        gloss+=loss_G
    gloss=gloss/n
    dloss=dloss/n    
    if i % 10 == 9:
        print(f"at epoch {i+1}, dloss: {dloss}, gloss {gloss}")
        see_output()                                    #4

See the batch loop: it alternates discriminator and generator updates, then visualizes generated images every ten epochs. section 4.1.3

Text-generation training also depends on preparing suitable input data. The GPT example tokenizes three novels, maps tokens to integers, and divides the resulting sequence into fixed-length training sequences. section 12.1.2

How Learn Generative AI with PyTorch covers it

How deep it goes
Details alternating GAN updates and output checks, while describing LSTM and GPT training at a higher level.
Examples
Uses a clothing-image GAN, an LSTM text generator, and three Hemingway novels for GPT training.
What it assumes
Assumes a constructed model and prepared training data; demonstrates tokenization and sequence preparation for GPT.
What you can do after
Build it from scratch by iterating over batches, alternating discriminator and generator updates, and inspecting generated images.
How it connects
Connects training to GANs, LSTMs, Transformers, tokenization, and training data.

Also explained in

Compare how 24 books explain training →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Learn Generative AI with PyTorch. The section links point to the parts of the book it draws on.