Optimizer
An optimizer is an algorithm or procedure that adjusts model parameters or other decisions to improve an objective function.
Distributed Machine Learning Patterns
An optimizer is an algorithm that updates model parameters to minimize a training objective such as loss.
- How it is used
- Configures parameter updates as part of compiling and training a TensorFlow Keras model.
- How deep it goes
- Defines the optimizer’s training role and demonstrates Adam, but stops short of explaining update mechanics.
- Examples
- Uses Adam for MNIST classification and searches learning-rate choices with Keras Tuner.
- What you can do after
- Configure an optimizer through TensorFlow Keras and vary its learning rate during hyperparameter tuning.
Read how Distributed Machine Learning Patterns explains optimizer →
Evolutionary Deep Learning: Genetic algorithms and neural networks
An optimizer is an algorithm that updates model parameters to minimize or otherwise improve an objective during training.
- How it is used
- Uses optimizer selection to tune deep-learning training alongside activation and loss functions.
- How deep it goes
- Compares optimizer outcomes experimentally and demonstrates a GAN failure mode, but does not explain optimizer internals.
- Examples
- Compares SGD, Adagrad, Adam, and RMSprop, then swaps a GAN discriminator optimizer to expose vanishing-gradient symptoms.
- What you can do after
- Apply optimizer selection by comparing training results and changing optimizer settings in executable notebooks.
Read how Evolutionary Deep Learning: Genetic algorithms and neural networks explains optimizer →
Related concepts
- demonstrationproduces · A demonstration is an example pairing an input, request, or prompt with a corresponding output, answer, or label to illustrate a task or expected behavior.
- LabeledFewShotincludes · LabeledFewShot is a DSPy optimizer that constructs prompts by selecting labeled training examples without verifying language-model predictions.
- GEPAincludes · GEPA (Genetic-Pareto) is a reflective evolutionary optimization method that generates and evaluates candidate prompts using task information and feedback.
- BootstrapFewShotincludes · BootstrapFewShot is a DSPy optimizer that selects or bootstraps demonstrations for a prompt from training examples and successful model-generated outputs.
- MIPROv2includes · MIPROv2 is a DSPy optimizer that searches for effective instructions and demonstrations for language-model programs using performance feedback and a Bayesian proposal model.
- validation setused by · A validation set is held-out data used to assess a model during development and guide model selection or tuning, rather than to fit its parameters.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.