liveBook · Manning

Optimizer

As explained in Distributed Machine Learning Patterns

An optimizer is an algorithm that updates model parameters to minimize a training objective such as loss.

An optimizer updates model parameters during training to reduce the loss. In a TensorFlow Keras workflow, it is selected when compiling the model alongside the loss function and evaluation metrics. section 8.1.1

Choosing Adam in Keras

A handwritten-digit classifier uses Adam with sparse categorical cross-entropy and accuracy, then trains for five epochs and evaluates on MNIST. The example reports approximately 97.6% test accuracy. section 8.1.1

Listing 8.1 Creating a Conda environment
> conda create --name dist-ml python=3.9 -y

Use the model-compilation listing to see where the optimizer is configured in the training workflow. section 8.1.1

How Distributed Machine Learning Patterns covers it

How deep it goes
Defines the optimizer’s training role and demonstrates Adam, but stops short of explaining update mechanics.
Examples
Uses Adam for MNIST classification and searches learning-rate choices with Keras Tuner.
What it assumes
Assumes a defined model, training data, loss function, and evaluation metric before optimizer configuration.
What you can do after
Configure an optimizer through TensorFlow Keras and vary its learning rate during hyperparameter tuning.
How it connects
Links optimizers to loss functions, evaluation metrics, model training, hyperparameters, and learning rates.
Cover of Distributed Machine Learning Patterns
liveBookmanning.com
Read Distributed Machine Learning Patterns on liveBook Get the book at manning.com

Also explained in

Compare how 2 books explain optimizer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Distributed Machine Learning Patterns. The section links point to the parts of the book it draws on.