liveBook · Manning

Optimization

As explained in Grokking Machine Learning

An optimization algorithm that updates model parameters to minimize a loss or objective function.

An optimizer is an algorithm that updates a model’s parameters to reduce its loss. In Keras, you select one when compiling a model. section 10.3

The neural-network example selects Adam and names other optimizers to try:

model.compile(loss = 'categorical_crossentropy', optimizer='adam', metrics=['accuracy'])

See the Keras compile call pairing categorical cross-entropy with Adam. section 10.3

How Grokking Machine Learning covers it

How deep it goes
Defines the optimizer’s role and names alternatives, but does not explain update rules or compare their mechanics.
Examples
Uses Adam in a Keras classifier and suggests rerunning training with SGD, RMSProp, and AdaGrad.
What it assumes
Assumes a Keras model and loss function are defined before compilation.
What you can do after
Apply it via Keras by selecting an optimizer at compilation and comparing training results across alternatives.
How it connects
Connects optimization with neural networks, loss functions, model compilation, and training.
Cover of Grokking Machine Learning
liveBookmanning.com
Read Grokking Machine Learning on liveBook Get the book at manning.com

Also explained in

Compare how 30 books explain optimization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Grokking Machine Learning. The section links point to the parts of the book it draws on.