Cover of Learn Generative AI with PyTorch

What you will learn in Learn Generative AI with PyTorch

by Mark Liu

Read it on liveBook Get the book at manning.com

Chapter by chapter

Chapter 1: What is generative AI and why PyTorch?

1.1 Introducing generative AI and PyTorch · 1.1.1 What is generative AI? · 1.1.2 The Python programming language · 1.1.3 Using PyTorch as our AI framework · 1.2 GANs · 1.2.1 A high-level overview of GANs · 1.2.2 An illustrating example: Generating anime faces · 1.2.3 Why should you care about GANs? · 1.3 Transformers · 1.3.1 The attention mechanism · 1.3.2 The Transformer architecture · 1.3.3 Multimodal Transformers and pretrained LLMs · 1.4 Why build generative models from scratch? · 1.5 Summary

What you learn in chapter 1 →

Chapter 2: Deep learning with PyTorch

2.1 Data types in PyTorch · 2.1.1 Creating PyTorch tensors · 2.1.2 Index and slice PyTorch tensors · 2.1.3 PyTorch tensor shapes · 2.1.4 Mathematical operations on PyTorch tensors · 2.2 An end-to-end deep learning project with PyTorch · 2.2.1 Deep learning in PyTorch: A high-level overview · 2.2.2 Preprocessing data · 2.3 Binary classification · 2.3.1 Creating batches · 2.3.2 Building and training a binary classification model · 2.3.3 Testing the binary classification model · 2.4 Multicategory classification · 2.4.1 Validation set and early stopping · 2.4.2 Building and training a multicategory classification model · 2.5 Summary

What you learn in chapter 2 →

Chapter 3: Generative adversarial networks: Shape and number generation

3.1 Steps involved in training GANs · 3.2 Preparing training data · 3.2.1 A training dataset that forms an exponential growth curve · 3.2.2 Preparing the training dataset · 3.3 Creating GANs · 3.3.1 The discriminator network · 3.3.2 The generator network · 3.3.3 Loss functions, optimizers, and early stopping · 3.4 Training and using GANs for shape generation · 3.4.1 The training of GANs · 3.4.2 Saving and using the trained generator · 3.5 Generating numbers with patterns · 3.5.1 What are one-hot variables? · 3.5.2 GANs to generate numbers with patterns · 3.5.3 Training the GANs to generate numbers with patterns · 3.5.4 Saving and using the trained model · 3.6 Summary

What you learn in chapter 3 →

Chapter 4: Image generation with generative adversarial networks

4.1 GANs to generate grayscale images of clothing items · 4.1.1 Training samples and the discriminator · 4.1.2 A generator to create grayscale images · 4.1.3 Training GANs to generate images of clothing items · 4.2 Convolutional layers · 4.2.1 How do convolutional operations work? · 4.2.2 How do stride and padding affect convolutional operations? · 4.3 Transposed convolution and batch normalization · 4.3.1 How do transposed convolutional layers work? · 4.3.2 Batch normalization · 4.4 Color images of anime faces · 4.4.1 Downloading anime faces · 4.4.2 Channels-first color images in PyTorch · 4.5 Deep convolutional GAN · 4.5.1 Building a DCGAN · 4.5.2 Training and using DCGAN · 4.6 Summary

What you learn in chapter 4 →

Chapter 5: Selecting characteristics in generated images

5.1 The eyeglasses dataset · 5.1.1 Downloading the eyeglasses dataset · 5.1.2 Visualizing images in the eyeglasses dataset · 5.2 cGAN and Wasserstein distance · 5.2.1 WGAN with gradient penalty · 5.2.2 cGANs · 5.3 Create a cGAN · 5.3.1 A critic in cGAN · 5.3.2 A generator in cGAN · 5.3.3 Weight initialization and the gradient penalty function · 5.4 Training the cGAN · 5.4.1 Adding labels to inputs · 5.4.2 Training the cGAN · 5.5 Selecting characteristics in generated images · 5.5.1 Selecting images with or without eyeglasses · 5.5.2 Vector arithmetic in latent space · 5.5.3 Selecting two characteristics simultaneously · 5.6 Summary

What you learn in chapter 5 →

Chapter 6: CycleGAN: Converting blond hair to black hair

6.1 CycleGAN and cycle consistency loss · 6.1.1 What is CycleGAN? · 6.1.2 Cycle consistency loss · 6.2 The celebrity faces dataset · 6.2.1 Downloading the celebrity faces dataset · 6.2.2 Process the black and blond hair image data · 6.3 Building a CycleGAN model · 6.3.1 Creating two discriminators · 6.3.2 Creating two generators · 6.4 Using CycleGAN to translate between black and blond hair · 6.4.1 Training a CycleGAN to translate between black and blond hair · 6.4.2 Round-trip conversions of black hair images and blond hair images · 6.5 Summary

What you learn in chapter 6 →

Chapter 7: Image generation with variational autoencoders

7.1 An overview of AEs · 7.1.1 What is an AE? · 7.1.2 Steps in building and training an AE · 7.2 Building and training an AE to generate digits · 7.2.1 Gathering handwritten digits · 7.2.2 Building and training an AE · 7.2.3 Saving and using the trained AE · 7.3 What are VAEs? · 7.3.1 Differences between AEs and VAEs · 7.3.2 The blueprint to train a VAE to generate human face images · 7.4 A VAE to generate human face images · 7.4.1 Building a VAE · 7.4.2 Training the VAE · 7.4.3 Generating images with the trained VAE · 7.4.4 Encoding arithmetic with the trained VAE · 7.5 Summary

What you learn in chapter 7 →

Chapter 8: Text generation with recurrent neural networks

8.1 Introduction to RNNs · 8.1.1 Challenges in generating text · 8.1.2 How do RNNs work? · 8.1.3 Steps in training a LSTM model · 8.2 Fundamentals of NLP · 8.2.1 Different tokenization methods · 8.2.2 Word embedding · 8.3 Preparing data to train the LSTM model · 8.3.1 Downloading and cleaning up the text · 8.3.2 Creating batches of training data · 8.4 Building and training the LSTM model · 8.4.1 Building an LSTM model · 8.4.2 Training the LSTM model · 8.5 Generating text with the trained LSTM model · 8.5.1 Generating text by predicting the next token · 8.5.2 Temperature and top-K sampling in text generation · 8.6 Summary

What you learn in chapter 8 →

Chapter 9: A line-by-line implementation of attention and Transformer

9.1 Introduction to attention and Transformer · 9.1.1 The attention mechanism · 9.1.2 The Transformer architecture · 9.1.3 Different types of Transformers · 9.2 Building an encoder · 9.2.1 The attention mechanism · 9.2.2 Creating an encoder · 9.3 Building an encoder-decoder Transformer · 9.3.1 Creating a decoder layer · 9.3.2 Creating an encoder-decoder Transformer · 9.4 Putting all the pieces together · 9.4.1 Defining a generator · 9.4.2 Creating a model to translate between two languages · 9.5 Summary

What you learn in chapter 9 →

Chapter 10: Training a Transformer to translate English to French

10.1 Subword tokenization · 10.1.1 Tokenizing English and French phrases · 10.1.2 Sequence padding and batch creation · 10.2 Word embedding and positional encoding · 10.2.1 Word embedding · 10.2.2 Positional encoding · 10.3 Training the Transformer for English-to-French translation · 10.3.1 Loss function and the optimizer · 10.3.2 The training loop · 10.4 Translating English to French with the trained model · 10.5 Summary

What you learn in chapter 10 →

Chapter 11: Building a generative pretrained Transformer from scratch

11.1 GPT-2 architecture and causal self-attention · 11.1.1 The architecture of GPT-2 · 11.1.2 Word embedding and positional encoding in GPT-2 · 11.1.3 Causal self-attention in GPT-2 · 11.2 Building GPT-2XL from scratch · 11.2.1 BPE tokenization · 11.2.2 The Gaussian error linear unit activation function · 11.2.3 Causal self-attention · 11.2.4 Constructing the GPT-2XL model · 11.3 Loading up pretrained weights and generating text · 11.3.1 Loading up pretrained parameters in GPT-2XL · 11.3.2 Defining a generate() function to produce text · 11.3.3 Text generation with GPT-2XL · 11.4 Summary

What you learn in chapter 11 →

Chapter 12: Training a Transformer to generate text

12.1 Building and training a GPT from scratch · 12.1.1 The architecture of a GPT to generate text · 12.1.2 The training process of the GPT model to generate text · 12.2 Tokenizing text of Hemingway novels · 12.2.1 Tokenizing the text · 12.2.2 Creating batches for training · 12.3 Building a GPT to generate text · 12.3.1 Model hyperparameters · 12.3.2 Modeling the causal self-attention mechanism · 12.3.3 Building the GPT model · 12.4 Training the GPT model to generate text · 12.4.1 Training the GPT model · 12.4.2 A function to generate text · 12.4.3 Text generation with different versions of the trained model · 12.5 Summary

What you learn in chapter 12 →

Chapter 13: Music generation with MuseGAN

13.1 Digital music representation · 13.1.1 Musical notes, octave, and pitch · 13.1.2 An introduction to multitrack music · 13.1.3 Digitally represent music: Piano rolls · 13.2 A blueprint for music generation · 13.2.1 Constructing music with chords, style, melody, and groove · 13.2.2 A blueprint to train a MuseGAN · 13.3 Preparing the training data for MuseGAN · 13.3.1 Downloading the training data · 13.3.2 Converting multidimensional objects to music pieces · 13.4 Building a MuseGAN · 13.4.1 A critic in MuseGAN · 13.4.2 A generator in MuseGAN · 13.4.3 Optimizers and the loss function · 13.5 Training the MuseGAN to generate music · 13.5.1 Training the MuseGAN · 13.5.2 Generating music with the trained MuseGAN · 13.6 Summary

What you learn in chapter 13 →

Chapter 14: Building and training a music Transformer

14.1 Introduction to the music Transformer · 14.1.1 Performance-based music representation · 14.1.2 The music Transformer architecture · 14.1.3 Training the music Transformer · 14.2 Tokenizing music pieces · 14.2.1 Downloading training data · 14.2.2 Tokenizing MIDI files · 14.2.3 Preparing the training data · 14.3 Building a GPT to generate music · 14.3.1 Hyperparameters in the music Transformer · 14.3.2 Building a music Transformer · 14.4 Training and using the music Transformer · 14.4.1 Training the music Transformer · 14.4.2 Music generation with the trained Transformer · 14.5 Summary

What you learn in chapter 14 →

Chapter 15: Diffusion models and text-to-image Transformers

15.1 Introduction to denoising diffusion models · 15.1.1 The forward diffusion process · 15.1.2 Using the U-Net model to denoise images · 15.1.3 A blueprint to train the denoising U-Net model · 15.2 Preparing the training data · 15.2.1 Flower images as the training data · 15.2.2 Visualizing the forward diffusion process · 15.3 Building a denoising U-Net model · 15.3.1 The attention mechanism in the denoising U-Net model · 15.3.2 The denoising U-Net model · 15.4 Training and using the denoising U-Net model · 15.4.1 Training the denoising U-Net model · 15.4.2 Using the trained model to generate flower images · 15.5 Text-to-image Transformers · 15.5.1 CLIP: A multimodal Transformer · 15.5.2 Text-to-image generation with DALL-E 2 · 15.6 Summary

What you learn in chapter 15 →

Chapter 16: Pretrained large language models and the LangChain library

16.1 Content generation with the OpenAI API · 16.1.1 Text generation tasks with OpenAI API · 16.1.2 Code generation with OpenAI API · 16.1.3 Image generation with OpenAI DALL-E 2 · 16.1.4 Speech generation with OpenAI API · 16.2 Introduction to LangChain · 16.2.1 The need for the LangChain library · 16.2.2 Using the OpenAI API in LangChain · 16.2.3 Zero-shot, one-shot, and few-shot prompting · 16.3 A zero-shot know-it-all agent in LangChain · 16.3.1 Applying for a Wolfram Alpha API Key · 16.3.2 Creating an agent in LangChain · 16.3.3 Adding tools by using OpenAI GPTs · 16.3.4 Adding tools to generate code and images · 16.4 Limitations and ethical concerns of LLMs · 16.4.1 Limitations of LLMs · 16.4.2 Ethical concerns for LLMs · 16.5 Summary

What you learn in chapter 16 →

Concepts explained in this book

discriminatorThe GAN component that learns to distinguish real training images or samples from fake generated ones.
latent spaceA latent representation produced by the encoder and passed to the decoder for reconstruction or generation.
generationA language-modeling task in which new text is produced without conditioning on a specific task label, here using a generic or empty prompt.
transformer architectureThe internal design and structural organization of a Transformer model built around decoder blocks or self-attention components.
tokenizationThe process of splitting text into discrete units such as characters, words, or subwords for modeling.
attentionA neural sequence-processing technique that assigns learned weights to relationships between elements, including self-relationships, to model relevance across a sequence.
generatorThe GAN component that maps latent inputs to synthetic samples intended to look like training data.
training datasetThe source text examples used for model training, viewed here as material whose length and variation affect generation quality.
decoder-only transformerA Transformer design that uses only decoder stacks for autoregressive generation.
encoderAn encoder is a component that transforms input data into an encoded representation for subsequent processing, reconstruction, comparison, or transmission.
token idA numeric identifier assigned to a token so it can be represented as an integer for model processing.
embeddingA continuous numeric encoding that captures the meaning of an input sentence for downstream decoding.
language modelA large language model trained beforehand on broad textual data and reused for downstream tasks.
decodeDecoding is the process of transforming an encoded or internal representation into an interpretable or target-form output.
positional embeddingA positional-encoding method that learns position vectors as parameters rather than using fixed formulas.
arrayAn array with one or more dimensions, used here as the general definition of a tensor.
classifierA task of assigning one of several discrete categories to text sequences.
vocabularyThe number of distinct tokens available in a model's vocabulary.
tensorA multidimensional numerical array used in PyTorch to represent model inputs, outputs, and intermediate values.
MIDIA file containing musical performance data encoded according to the MIDI standard.
masked attentionMasked attention is an attention mechanism in which a mask restricts which positions can influence each other, often preventing sequence positions from accessing future tokens.
cross entropy lossA loss measure that compares predicted token distributions with the target next tokens for multiclass sequence prediction.
feedforward networkA feedforward network is a neural network or subnetwork in which information flows from inputs toward outputs through successive computations without recurrent connections or cycles.
residual connectionA neural-network connection that bypasses intermediate layers and passes earlier activations to later layers.
decoder layerA Transformer building block composed of self-attention and feed-forward components, plus normalization and residual connections in this section.
categorical encodingA vector representation with a single active entry used here to encode token or position identities before projection.
MuseGANA music-generation GAN variant in this chapter that produces multidimensional representations of musical pieces for later conversion to MIDI.
scaled-dot product attentionAn attention method that computes similarity between queries and keys with dot products, scales them, and uses the result to weight values.
generateA function that converts a prompt to token indexes, generates more indexes with sample(), and converts the full sequence back to text.
generate functionA function that turns a text prompt into token indexes, generates additional indexes by sampling, and converts the resulting sequence back to text.
temperatureA sampling control parameter that scales the model’s output distribution before choosing the next token in text generation.
variational autoencoderA generative model that learns an encoder-decoder latent-variable representation so it can reconstruct inputs and sample new outputs from latent space.
multi-head attentionA self-attention variant that splits representations into multiple attention heads to capture different relationships in parallel.
Kullback–Leibler divergenceA divergence measure used as a regularization term to encourage a latent-variable distribution to resemble a normal distribution.
generative adversarial networkA generative adversarial network, a model family trained with a generator and discriminator to synthesize realistic data or transform data between domains.
music TransformerA Transformer model in this chapter that is trained on music sequences and used autoregressively to generate new musical indexes.
music TransformerA Transformer model in this chapter that is trained on music sequences and used autoregressively to generate new musical indexes.
LSTMA recurrent neural network variant with long short-term memory units, mentioned here as an example of a deep learning model that cannot process raw text directly.
machine translationThe task of automatically translating text from one natural language to another.
parameterThe learned numerical parameters of a neural network model after or during training.
music generationThe process of generating new musical event sequences from a trained model, here by extending a prompt of indexed music events.
recurrent neural networkA neural network architecture for sequential data that processes inputs with loops so information can persist across time steps.
subword tokenizationA partial word unit produced by subword tokenization, smaller than a whole word but larger than a character.
transposed convolutionThe 2D computation performed by a transposed convolutional layer to expand an image or feature map.
samplingA prediction strategy that samples next tokens according to predicted probabilities instead of always taking the maximum.
U-NetA U-shaped convolutional encoder-decoder neural network with skip connections, used here to predict and remove noise during diffusion-based image generation.
PyTorchA deep learning library used here for building and training neural networks, especially with dynamic computation graphs and GPU acceleration.
GPT-2A decoder-only Transformer language model used here as the example model whose embeddings and positional encoding are being described.
gradient penaltyAn added loss term that penalizes undesirable gradient behavior during GAN training to improve critic stability.
adamAdam is an adaptive stochastic-gradient optimization algorithm that updates model parameters using moving estimates of the first and second moments of gradients.
diffusion modelA generative model that learns to produce samples by progressively reversing a noising process over discrete time steps.
labelThe reference target values used to supervise training, here the actual noise compared against the model’s prediction.
text-to-image modelThe generative task of producing an image conditioned on a text prompt.
softmaxAn activation that converts raw scores into a probability distribution over multiple classes or positions.
tokenA discrete text unit produced by tokenization, such as a word, punctuation mark, or special character, used as an element of a sequence.
CycleGANA GAN-family image-to-image translation model that learns to map samples between two unpaired visual domains while preserving content via cycle consistency.
mean absolute errorA loss metric that averages the absolute difference between predicted and target values, used here for image reconstruction.
autoencoderA neural network that learns to compress inputs into a latent representation and reconstruct them, used here for digit-image generation and reconstruction.
from scratch implementationA development approach in which a model is implemented and trained directly rather than being treated as a prebuilt black box.
evaluation datasetA held-out subset of data used to monitor training and tune model decisions without directly fitting parameters.
layer normalizationA normalization technique that normalizes activations across features within each layer or sample.
xThe input sequence of music-token indexes used to predict the next elements in this training example construction.
xThe input sequence of music-token indexes used to predict the next elements in this training example construction.
yThe output sequence of music-token indexes shifted one position right from x so the model learns next-element prediction.
yThe output sequence of music-token indexes shifted one position right from x so the model learns next-element prediction.
paddingA setting or operation that adds, fills, or ignores boundary or placeholder positions in tensor or sequence data.
label arithmeticA conditioning technique that forms an intermediate label by weighted averaging two labels to smoothly vary a generated attribute.
beginning-of-sequence tokenThe pair of special sequence markers that denote the beginning and end of a tokenized sequence.
top-k samplingA decoding method that selects the next token from the K most likely candidates after truncating the distribution.
conditional GANA GAN that conditions generation on auxiliary information such as class labels so the output can be steered toward a desired attribute.
GPT modelA family of generative pretrained Transformer language models referenced here as the host family that Codex is integrated into.
encoding arithmeticA latent-space manipulation method that adds and subtracts VAE encodings to synthesize new outputs.
musical noteA musical pitch event encoded in the dataset as one of a fixed set of possible pitch values.
promptThe chat message role that sets assistant behavior in the OpenAI chat-completions format.
epochOne complete pass through the entire training dataset during model training.
LangChainA Python library for building applications with large language models through tools, abstractions, and component chaining.
UNKA special unknown-token symbol used to represent out-of-vocabulary words during tokenization and prompting.
out-of-vocabulary wordAn out-of-vocabulary word or text unit is absent from a tokenizer’s or model’s available vocabulary and may be represented by a special unknown-token marker.
exponential growth curveA curve whose values increase at an accelerating rate according to an exponential relationship.
exponential growth curveA curve whose values increase at an accelerating rate according to an exponential relationship.
convolutional layerA PyTorch convolution layer that applies learned 2D filters to an input tensor to produce feature maps.
reconstruction lossThe loss term that measures pixel-level mismatch between an input image and its reconstruction in a VAE.
Wasserstein distanceA distance measure used in Wasserstein GAN training to compare distributions and provide a smoother optimization signal than standard GAN losses.
overfittingA training failure mode where the model memorizes training data too closely and generalizes poorly to new inputs.
vector arithmeticThe manipulation of latent vectors by weighted averaging to interpolate between generated attributes in latent space.
generative modelingA class of AI systems that create new content such as text, images, code, audio, music, or video rather than only classifying existing inputs.
next-token predictionThe language-modeling objective of predicting the following token from the current and previous tokens in a sequence.
musical measureA musical time unit that groups beats in a composition.
JSB Chorales datasetA dataset of Bach chorales arranged as multitrack piano-roll music and used as training data for music generation models such as MuseGAN.
strideThe step-size parameter that controls how far a convolutional operation moves and, in transposed convolution, how much upsampling occurs.
train_epochA book-defined function that trains the model for one epoch.
backpropagationA PyTorch call that computes gradients of the loss with respect to model parameters.
conditional probabilityA probability of one variable given another, used here for discriminative prediction of Y from X.
hyperparameterTraining or model settings chosen before fitting, such as layer counts, head counts, embedding size, vocabulary size, block size, and dropout rate.
music21A Python library for analyzing, manipulating, and displaying symbolic music data such as notes and piano rolls.
objective functionThe objective value minimized during training, here composed of reconstruction loss and KL divergence for the VAE.
English-to-French translationThe cross-language text translation task from English into French.
cycle consistency lossA loss term that penalizes failure to reconstruct the original input after translating it to another domain and back, used to preserve content in CycleGAN.
denoisingThe process of removing noise from an image estimate one step at a time during generation.
multitrack musicMusic represented as multiple separate tracks or voices that can be edited and processed independently.
convolutional neural networkA neural network architecture using convolution operations, here applied to multidimensional music data to extract spatial features.
queryAn attention query is a vector representation of the information sought by an input position, compared with key representations to determine which information to attend to.
sequence predictionA learning setup where a model predicts the next element in an ordered sequence from earlier elements.
end of text tokenA special end-of-sequence token used to mark completion of decoding or the end of a tokenized sequence.
piano roll representationA grid-based representation of musical notes over time and pitch, commonly used to encode MIDI-like music for computation.
transformersA software library for working with pretrained transformer models and tokenizers from Hugging Face.
trainingThe iterative process of feeding data through a model, computing loss, and updating parameters.
tensor shapeThe tuple of dimension sizes describing the structure of a multidimensional object.
DataLoaderA PyTorch utility that wraps a dataset or iterable to provide batched access, optional shuffling, and iteration during training.
interpolationThe technique of generating intermediate representations between two latent points to explore a smooth latent space.
keyThe attention input that represents the searchable features used to match against a query.
valueIn attention, a value is a vector representation of input content that is combined with other values according to attention weights to produce an output.
logitAn unnormalized model output score produced before conversion to probabilities for sampling or selection.
cross-attentionAn attention operation where queries come from one sequence or representation and keys/values come from another, enabling information transfer between them.
batch normalizationA normalization technique that standardizes activations within minibatches to stabilize and speed neural-network training.
kernel sizeThe spatial size of a convolutional kernel or receptive window applied to an input.
pitchThe perceived highness or lowness of a musical sound, determined here by the note’s frequency and encoded by pitch numbers.
DCGAN / deep convolutional ganA deep convolutional generative adversarial network architecture for image generation.
note-on/note-off eventA musical event indicating that a note begins sounding in the performance encoding.
time-shift eventA musical event in the encoding that advances the represented time between note events.
velocityA musical performance attribute indicating how forcefully a note is played, represented as an event/value in the encoding.
Wasserstein GANA GAN variant that uses the Wasserstein distance as its training objective to improve stability and provide a more informative critic score.
timestepA discrete index into the diffusion/denoising schedule that determines how much noise is added or removed at a given stage.
CLIPA multimodal Transformer model that learns aligned image and text representations for comparing and matching visual and textual inputs.
linear layerA PyTorch layer that applies a learned linear transformation to its input.
grayscale imageImages represented only by intensity values rather than color channels, used here as a simpler target for generation.
encoding interpolationA technique for varying latent representations smoothly to generate intermediate decoded outputs between examples.
accuracyThe fraction or percentage of predictions that match the true labels in a classification task.
music pieceA musical work represented here as a sequence of musical notes and events for Transformer training.
few-shot learningA prompting approach that provides exactly one example to condition a large language model on the task.
musical eventA token-like symbolic unit representing a note or control information in a music sequence used for model training and generation.
musical eventA token-like symbolic unit representing a note or control information in a music sequence used for model training and generation.
sigmoidAn activation function that maps outputs into the range 0 to 1, used here to interpret the discriminator output as a probability.
early stoppingA training control method that stops fitting when validation or other monitored performance no longer improves.
byte pair encodingA subword tokenization method adapted from data compression that merges frequent character pairs to represent text as tokens.
zero-shot promptingA prompting method that gives the model a task without any examples, relying only on instructions and prior knowledge.
stepA subdivision of a musical bar used for sequencing rhythms, melodies, or notes.
long-range dependencyA relationship between distant elements in a sequence whose interpretation depends on information far apart in the input.
GPT-2XLA larger GPT-style decoder-only Transformer model used here as the comparison baseline for architecture size and parameter count.
GPT-2XLA larger GPT-style decoder-only Transformer model used here as the comparison baseline for architecture size and parameter count.
vanishing gradient problemA training failure mode in deep networks where gradient signals become too small to effectively update early layers.
control messageA MIDI event that conveys performance control information such as note on/off timing and related parameters.
tokenizerA text-processing component that splits text into tokens and converts text to a discrete token sequence for modeling.
XLMA pretrained multilingual language model used here as a tokenizer because it handles multiple languages well.
input-output pairA paired training example consisting of an input sequence and its shifted target sequence for next-token prediction.
convolution kernelA learned convolutional kernel whose weights slide across local input regions to produce feature responses.
ReLUA piecewise linear neural-network activation function that outputs zero for negative inputs and the input value for nonnegative inputs.
DALL-EA text-to-image generative model that produces images from textual descriptions.
GELUA smooth activation function used in neural networks to transform layer outputs and improve model performance.
Oxford Flowers datasetA flower-image dataset used as training data for diffusion-model examples.
multicategory classification modelA supervised neural network model that assigns each input to one of more than two discrete classes.
argmaxArgmax is an operation that identifies an argument, position, or index at which a function or collection attains its maximum value.
model architectureThe internal design and structural organization of a neural model, here referring to the overall layout of an autoencoder.
multimodalA generative model that accepts more than one data modality, such as text plus images or audio, as input.
query, key, and valueThe three attention inputs that represent what to match against, what features are matched on, and what content to aggregate.
discriminative modelA model that learns to predict labels from inputs by estimating conditional probabilities and decision boundaries between classes.
feature mapAn intermediate activation map in a neural network whose spatial resolution can be increased or preserved by convolutional operations.
chordA combination of musical notes sounded together, here treated as one of the musical factors to control in generation.
grooveThe rhythmic feel or swing of music that gives it motion and makes it feel compelling to move with.
melodyThe sequence of pitched notes that forms the tune of a musical piece, here one of the controlled generation factors.
styleA musical stylistic attribute that influences the character of generated pieces, distinct from the earlier visual-use sense.
parallelismA training approach that computes multiple sequence elements simultaneously rather than strictly sequentially.
encoder-decoder architectureAn encoder-decoder architecture is a model design in which an encoder transforms input into an intermediate representation and a decoder uses that representation to produce output.
probabilistic vectorA latent representation whose values parameterize a probability distribution rather than a single deterministic code.
probabilistic vectorA latent representation whose values parameterize a probability distribution rather than a single deterministic code.
DDIMSchedulerA scheduler class used to control diffusion-step timing and denoising progression in a diffusion model.
DDIMSchedulerA scheduler class used to control diffusion-step timing and denoising progression in a diffusion model.
label smoothingA training technique that replaces one-hot target labels with softened target distributions to reduce overconfidence and improve generalization.
OpenAI APIAn external programming interface used here to request text-to-speech conversion from OpenAI’s service.
resolutionThe pixel dimensions of an image, used here to compare lower- and higher-resolution generated images.
optimizationAn algorithm that updates model parameters to reduce loss during training.
computation graphA directed graph of mathematical operations and data flow used to compute gradients in deep learning.
octaveThe distance between one level of musical pitch and the next, where frequencies double from one octave to the next.
performance-based representation of musicA music encoding that represents a piece as a sequence of performance events while preserving details such as timing and velocity.
zero paddingThe addition of zero-valued borders around an input to control convolution output size.
noiseRandom perturbation added to an image in a diffusion or denoising setup, which the model learns to predict and remove.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.