Locally optimal solution

As explained in Deep Learning with R, Third Edition

A local minimum is a point whose function value is no greater than the values at sufficiently nearby points.

A locally optimal solution, or local minimum, is a point where the function value is no greater than values at sufficiently nearby points. In optimization, gradient descent can settle there even when a better global minimum exists. section 2.4.3

The learning rate affects whether stochastic gradient descent reaches a useful solution: a rate that is too small can become stuck in a local minimum, while one that is too large can jump to unpredictable locations. section 2.4.3

Figure 2.19 A local minimum and a global minimum

Compare a local minimum with a global minimum on an illustrative loss curve. section 2.4.3

How Deep Learning with R, Third Edition covers it

How deep it goes

Defines local minima and connects them to learning-rate behavior, but stops short of formal analysis or high-dimensional optimization internals.

Examples

Uses one-dimensional loss curves, two-dimensional loss surfaces, and a visual comparison of local and global minima.

What you can do after

Diagnose how an excessively small or large learning rate can prevent useful progress during gradient descent.

How it connects

Connects local minima with stochastic gradient descent, learning rate, loss surfaces, and global minima.

liveBookmanning.com

Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Advanced Algorithms and Data StructuresA local minimum is a point whose function value is no greater than those of sufficiently nearby points, without necessarily being the smallest value over the entire domain.
• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsA solution that is optimal relative to a neighborhood but may not be optimal over the entire feasible search space.
• 

  liveBookmanning.com
  Data Without LabelsA local minimum is a point whose objective value is no greater than that of sufficiently nearby points, even if lower values exist elsewhere.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA local minimum is a point whose objective value is no greater than that of nearby points, without necessarily being the global minimum.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA local minimum is a point where an objective function has a value no greater than that of nearby points, without necessarily being the global minimum.
• 

  liveBookmanning.com
  Sutskever's ListA local minimum is a point whose objective value is no greater than that of nearby points, without necessarily being the global minimum.
• 

  liveBookmanning.com
  Deep Learning with JAXA point where a function has a value no greater than that of nearby points.
• 

  liveBookmanning.com
  Geometry for ProgrammersA local minimum is a point whose function value is no greater than the values at all sufficiently nearby inputs.

Compare how 9 books explain locally optimal solution →

Related concepts

• global optimumcontrasts with · A global optimum is a feasible point or solution whose objective value is at least as favorable as that of every other point or solution in the domain.
• simulated annealingaddressed by · Simulated annealing is a stochastic optimization method that gradually reduces the probability of accepting worse solutions to escape local optima.
• K-means clusteringproduced by · K-means clustering partitions data into a specified number of groups by assigning observations to representative centers and iteratively updating those centers to reduce within-group distances.
• greedy algorithmused by · A greedy algorithm builds a solution incrementally by choosing the locally best available option at each step, without generally accounting for all future consequences or revising prior choices.
• momentumcontrasts with · Momentum is an optimization technique that incorporates a running history of gradients or parameter updates to smooth and accelerate movement during training.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.