liveBook · Manning

Optimizer

An optimizer is an algorithm or procedure that adjusts model parameters or decisions to improve an objective.

How 3 Manning books explain it

Distributed Machine Learning Patterns

An optimizer is an algorithm that updates model parameters to minimize a training objective such as loss.

How it is used
Configures parameter updates as part of compiling and training a TensorFlow Keras model.
How deep it goes
Defines the optimizer’s training role and demonstrates Adam, but stops short of explaining update mechanics.
Examples
Uses Adam for MNIST classification and searches learning-rate choices with Keras Tuner.
What you can do after
Configure an optimizer through TensorFlow Keras and vary its learning rate during hyperparameter tuning.

Read how Distributed Machine Learning Patterns explains optimizer →

Evolutionary Deep Learning: Genetic algorithms and neural networks

An optimizer is an algorithm that updates model parameters to minimize or otherwise improve an objective during training.

How it is used
Uses optimizer selection to tune deep-learning training alongside activation and loss functions.
How deep it goes
Compares optimizer outcomes experimentally and demonstrates a GAN failure mode, but does not explain optimizer internals.
Examples
Compares SGD, Adagrad, Adam, and RMSprop, then swaps a GAN discriminator optimizer to expose vanishing-gradient symptoms.
What you can do after
Apply optimizer selection by comparing training results and changing optimizer settings in executable notebooks.

Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains optimizer →

Applied Reinforcement Learning

An optimizer is an algorithm that updates model parameters using an objective function and, commonly, its gradients.

How it is used
Updates deep-network parameters using gradients computed by backpropagation.
How deep it goes
Defines the optimizer’s role after explaining gradient flow, but does not specify an update rule.
Examples
Uses a turn-the-knobs analogy and contrasts small linear-model parameter sets with deep networks’ thousands or millions of parameters.
What you can do after
Understand how gradients from backpropagation guide parameter updates across a deep network.

Read how Applied Reinforcement Learning explains optimizer →

Also covered in

What to know first

Related concepts

Questions about optimizer

What is optimizer?

An optimizer is an algorithm or procedure that adjusts model parameters or decisions to improve an objective.

Which Manning books explain optimizer?

Distributed Machine Learning Patterns, Evolutionary Deep Learning: Genetic algorithms and neural networks and Applied Reinforcement Learning.

What should I know before learning optimizer?

Manning books that teach optimizer usually introduce objective function and recurrent neural network first.

What kinds of optimizer are there?

Kinds of optimizer include BootstrapFewShot, GEPA, LabeledFewShot and MIPROv2.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.