liveBook · Manning

Adam

Also called adaptive moment estimation

Adam is an adaptive stochastic-gradient optimization algorithm that updates parameters using exponential moving estimates of the first and second moments of gradients.

How 10 Manning books explain it

Inside Deep Learning: Math, Algorithms, Models

An adaptive optimization algorithm that combines moment estimates with variance-based gradient normalization.

How it is used
Optimizes neural-network training with adaptive gradient updates, including separate generator and discriminator updates in a GAN.
How deep it goes
Explains the gradient moving average and squared-gradient approximation, then shows PyTorch optimizer setup rather than a from-scratch implementation.
Examples
Uses AdamW training and accuracy plots alongside SGD variants, plus separate AdamW optimizers in a GAN training loop.
What you can do after
Apply it via PyTorch’s torch.optim.AdamW with default settings or configure separate optimizers and task-specific betas for a GAN.

Read how Inside Deep Learning: Math, Algorithms, Models explains adam →

Learn Generative AI with PyTorch

How it is used
Uses Adam to update neural-network parameters during supervised and generative-model training.
How deep it goes
Explains gradient history, learning rate, beta settings, and training-loop calls without deriving Adam’s update equations.
Examples
Uses clothing classification, GAN image generation, handwritten-digit reconstruction, and music generation as training examples.
What you can do after
Applies Adam via PyTorch to model parameters, sets learning rates and betas, and performs optimizer steps in a training loop.

Read how Learn Generative AI with PyTorch explains adam →

Build AI Drug Discovery Pipelines

Adam is an adaptive stochastic-gradient optimization algorithm that maintains running estimates of gradient moments to update model parameters.

How it is used
Optimizes neural networks that predict drug-target binding affinities or act as surrogates for docking experiments.
How deep it goes
Explains Adam’s training-loop role and configuration, but does not derive its moment estimates or update equations.
Examples
Demonstrates Adam in PyTorch code for neural-network training, graph-based affinity prediction, and active-learning surrogate modeling.
What you can do after
Apply it via PyTorch to train affinity-prediction models with minibatches and a tunable learning rate.

Read how Build AI Drug Discovery Pipelines explains adam →

Fight Fraud with Machine Learning

Adam is an adaptive stochastic optimization algorithm that updates model parameters using estimates of first and second moments of gradients.

How it is used
Trains deep neural networks for classification tasks such as fake-voice detection and illicit Bitcoin transaction detection.
How deep it goes
Explains Adam’s adaptive rates and momentum, then demonstrates configuration and training loops without deriving its update equations.
Examples
Demonstrates Adam with an AudioTransformer and an MLP trained on fake-voice and Elliptic Bitcoin datasets.
What you can do after
Apply it via PyTorch by configuring Adam, running backpropagation, and monitoring validation and test metrics.

Read how Fight Fraud with Machine Learning explains adam →

Bayesian Optimization in Action

Adam is an adaptive stochastic-gradient optimization algorithm that maintains moving estimates of first and second moments of gradients.

How it is used
Optimizes Gaussian-process parameters by minimizing negative likelihood during gradient-descent training.
How deep it goes
Implements the training loop and convergence checks but does not explain Adam's internal moment estimates.
Examples
Uses constant-mean Gaussian processes, full GP parameter training, 500 iterations, and loss-versus-parameter convergence plots.
What you can do after
Apply it via PyTorch to train Gaussian-process parameters and evaluate whether optimization has converged.

Read how Bayesian Optimization in Action explains adam →

Deep Learning with R, Third Edition

Adam is an adaptive stochastic optimization algorithm that uses estimates of first and second moments of gradients to update parameters.

How it is used
Configures neural-network training by specifying how weight updates reduce the chosen loss.
How deep it goes
Identifies Adam’s optimizer role and default usefulness but does not derive its moment estimates or update equations.
Examples
Uses MNIST digit classification and binary probability classification as concrete Adam configurations.
What you can do after
Apply Adam via Keras to compile and train classification models.

Read how Deep Learning with R, Third Edition explains adam →

How Large Language Models Work

Adam is an adaptive stochastic-gradient optimization algorithm that maintains estimates of gradient moments to adjust parameter updates.

How it is used
Optimizes neural-network parameters during large-scale training as an adaptive form of stochastic gradient descent.
How deep it goes
Explains momentum, local minima, and memory cost through analogy, but stops short of equations or update rules.
Examples
Uses a ball rolling down a loss curve and figures showing parameters, global minima, and local minima.
What you can do after
Evaluate Adam’s optimization benefit against its substantially higher training-memory requirement.

Read how How Large Language Models Work explains adam →

Math and Architectures of Deep Learning

An adaptive optimization algorithm that combines momentum-like first-moment estimates with per-parameter learning-rate adaptation.

How it is used
Uses combined momentum and adaptive learning rates to minimize neural-network loss during iterative training.
How deep it goes
Defines two state vectors, weighted averaging, parameter updates, and bias correction, then stops short of deriving every displayed equation.
Examples
Uses state-vector equations and a PyTorch comparison listing with SGD, momentum, AdaGrad, RMSProp, and Adam.
What you can do after
Apply it via PyTorch by configuring Adam with parameters, a learning rate, and beta coefficients.

Read how Math and Architectures of Deep Learning explains adam →

Machine Learning Algorithms in Depth

Adam is an adaptive optimization method that combines momentum-like first-moment estimates with RMSProp-like second-moment scaling and bias correction.

How it is used
Deploys adaptive stochastic optimization for training neural networks and comparing optimizer performance.
How deep it goes
Explains Adam’s component ideas, smoothing, bias correction, and recommended constants without deriving the full update equation.
Examples
Compares Adam on a LeNet trained with CIFAR-100 using Keras/TensorFlow and validation-accuracy plots.
What you can do after
Apply Adam via Keras/TensorFlow and compare its loss and validation accuracy with other optimizers.

Read how Machine Learning Algorithms in Depth explains adam →

Building Specialized AI Systems

Adam is an adaptive gradient-based optimization algorithm that maintains per-parameter estimates of first and second gradient moments.

How it is used
Supports LLM fine-tuning by adapting update sizes independently across model parameters.
How deep it goes
Defines per-parameter learning-rate adaptation and contrasts Adam with SGD, AdamW, and Adafactor without detailing optimizer internals.
Examples
Uses a hiking-assistant analogy contrasting steep and gentle terrain to illustrate parameter-specific step sizes.
What you can do after
Evaluate optimizer choices against update behavior, model scale, memory demands, and overfitting concerns.

Read how Building Specialized AI Systems explains adam →

Also covered in

What to know first

Related concepts

Questions about adam

What is adam?

Adam is an adaptive stochastic-gradient optimization algorithm that updates parameters using exponential moving estimates of the first and second moments of gradients.

Which Manning books explain adam?

Inside Deep Learning: Math, Algorithms, Models, Learn Generative AI with PyTorch, Build AI Drug Discovery Pipelines, Fight Fraud with Machine Learning, Bayesian Optimization in Action, Deep Learning with R, Third Edition, How Large Language Models Work, Math and Architectures of Deep Learning and 2 more.

What should I know before learning adam?

Manning books that teach adam usually introduce language model, mean squared error, overfitting and objective function first.

What is the difference between adam and AdamW?

Adam is an adaptive stochastic-gradient optimization algorithm that updates parameters using exponential moving estimates of the first and second moments of gradients. AdamW is an adaptive gradient-based optimization algorithm that combines Adam’s moment-based updates with weight decay applied independently of those updates.

What is adam a kind of?

Adam is a kind of optimization. Optimization is the systematic selection or modification of values, configurations, or implementations to improve a defined objective, often subject to constraints.

What is adam also called?

Adam is also called adaptive moment estimation.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.