Locally optimal solution

As explained in Sutskever's List

A local minimum is a point whose objective value is no greater than that of nearby points, without necessarily being the global minimum.

How Sutskever's List covers it

How deep it goes

Defines local minima and explains how high-dimensional loss surfaces and mini-batch gradients complicate the simple-trap picture.

Examples

Contrasts SVM optimization with neural-network training and uses AlexNet’s mini-batch optimization as a counterpoint to the local-minima concern.

What you can do after

Understand why local minima were considered a training risk and why that concern did not make gradient-based neural-network training ineffective.

How it connects

Connects local minima to convex optimization, gradient descent, mini-batch training, and neural networks.

liveBookmanning.com

Read Sutskever's List on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Advanced Algorithms and Data StructuresA local minimum is a point whose function value is no greater than those of sufficiently nearby points, without necessarily being the smallest value over the entire domain.
• 

  liveBookmanning.com
  Optimization Algorithms: AI techniques for design, planning, and control problemsA solution that is optimal relative to a neighborhood but may not be optimal over the entire feasible search space.
• 

  liveBookmanning.com
  Data Without LabelsA local minimum is a point whose objective value is no greater than that of sufficiently nearby points, even if lower values exist elsewhere.
• 

  liveBookmanning.com
  Grokking AI Algorithms, Second EditionA local minimum is a point whose objective value is no greater than that of nearby points, without necessarily being the global minimum.
• 

  liveBookmanning.com
  Natural Language Processing in Action, Second EditionA local minimum is a point where an objective function has a value no greater than that of nearby points, without necessarily being the global minimum.
• 

  liveBookmanning.com
  Deep Learning with R, Third EditionA local minimum is a point whose function value is no greater than the values at sufficiently nearby points.
• 

  liveBookmanning.com
  Deep Learning with JAXA point where a function has a value no greater than that of nearby points.
• 

  liveBookmanning.com
  Geometry for ProgrammersA local minimum is a point whose function value is no greater than the values at all sufficiently nearby inputs.

Compare how 9 books explain locally optimal solution →

Related concepts

• global optimumcontrasts with · A global optimum is a feasible point or solution whose objective value is at least as favorable as that of every other point or solution in the domain.
• simulated annealingaddressed by · Simulated annealing is a stochastic optimization method that gradually reduces the probability of accepting worse solutions to escape local optima.
• K-means clusteringproduced by · K-means clustering partitions data into a specified number of groups by assigning observations to representative centers and iteratively updating those centers to reduce within-group distances.
• greedy algorithmused by · A greedy algorithm builds a solution incrementally by choosing the locally best available option at each step, without generally accounting for all future consequences or revising prior choices.
• momentumcontrasts with · Momentum is an optimization technique that incorporates a running history of gradients or parameter updates to smooth and accelerate movement during training.

This explainer was generated by Manning's knowledge graph from the text of Sutskever's List. The section links point to the parts of the book it draws on.