liveBook · Manning

Locally optimal solution

As explained in Deep Learning with JAX

A point where a function has a value no greater than that of nearby points.

A local minimum is a point where a function has the smallest value among nearby points. It may be local without being the smallest value on the entire domain. section 4.1

Identify a candidate local minimum by:

  • Finding points where the derivative is zero. section 4.1
  • Checking that each candidate is a minimum rather than a maximum or saddle point. section 4.1
  • Using repeated steps opposite the gradient to approach a local minimum during gradient descent. section 2.6

Function minimization

For f(x) = x**4 + 12*x + 1/x, search for points where the derivative is zero, then classify those points. The same idea scales to neural-network training: gradient descent adjusts weights opposite the loss gradient until a stopping criterion is reached. section 4.1 section 2.6

Listing 4.1 A simple function we will use as a model
def f(x):
    return x**4 + 12*x + 1/x    #1

Inspect the compact function definition used for the minimization example. section 4.1

How Deep Learning with JAX covers it

How deep it goes
Defines local minima, distinguishes maxima and saddle points, and outlines derivative checks and iterative gradient descent.
Examples
Demonstrates minimization with f(x) = x**4 + 12*x + 1/x and neural-network weight updates on a loss landscape.
What it assumes
Assumes basic calculus terminology while introducing gradient descent and its neural-network application procedurally.
What you can do after
Evaluate candidates using derivatives and apply gradient descent to reduce a differentiable loss function.
How it connects
Connects local minima with derivatives, maxima, saddle points, gradient descent, neural networks, and loss functions.
Cover of Deep Learning with JAX
liveBookmanning.com
Read Deep Learning with JAX on liveBook Get the book at manning.com

Also explained in

Compare how 9 books explain locally optimal solution →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with JAX. The section links point to the parts of the book it draws on.