Locally optimal solution
A locally optimal solution is a feasible point whose objective value is no worse than those of sufficiently nearby points, though a better solution may exist elsewhere.
Advanced Algorithms and Data Structures
A local minimum is a point whose function value is no greater than those of sufficiently nearby points, without necessarily being the smallest value over the entire domain.
- How it is used
- Identifies when an optimization search has stalled near a solution without establishing the lowest possible cost.
- How deep it goes
- Defines the local–global distinction and examines plateaus, cost-function shapes, and search workarounds without proving global convergence.
- Examples
- Illustrates the concept with edge-crossing costs, a marble on plateaus and slopes, and randomly initialized k-means centroids.
- What you can do after
- Diagnose/debug with it when nearby improvements stop, then assess whether restarts or longer-range moves are worth trying.
Read how Advanced Algorithms and Data Structures explains locally optimal solution →
Optimization Algorithms: AI techniques for design, planning, and control problems
A solution that is optimal relative to a neighborhood but may not be optimal over the entire feasible search space.
- How it is used
- Distinguishes neighborhood-level quality from best-overall quality when describing feasible optimization solutions.
- How deep it goes
- Defines strong and weak local minima through neighborhood comparisons and describes how hill climbing can get stuck.
- Examples
- Uses a search-landscape figure and hill-climbing examples in routing and the 8-puzzle.
- What you can do after
- Evaluate whether a candidate is only locally best and recognize why hill climbing may stop short of a global optimum.
Data Without Labels
A local minimum is a point whose objective value is no greater than that of sufficiently nearby points, even if lower values exist elsewhere.
- How it is used
- Flags why convergence in MDS or k-means need not identify the globally best solution.
- How deep it goes
- Explains the distinction through a mountain analogy and clustering iterations, without a formal proof or guarantee of global optimality.
- Examples
- Uses a mountain descent with a deeper valley elsewhere and randomly initialized k-means centroids.
- What you can do after
- Diagnose/debug with it by questioning apparent convergence and comparing multiple k-means runs.
Read how Data Without Labels explains locally optimal solution →
Grokking AI Algorithms, Second Edition
A local minimum is a point whose objective value is no greater than that of nearby points, without necessarily being the global minimum.
- How it is used
- Frames optimization as a search for strong solutions among many valid possibilities.
- How deep it goes
- Defines local and global bests with analogies but stops short of mathematical tests or search procedures.
- Examples
- Uses restaurant selection, luggage packing, and constrained knapsack combinations to illustrate local and global quality.
- What you can do after
- Evaluate/understand why a good nearby solution may not be globally best and why populations can explore alternatives.
Read how Grokking AI Algorithms, Second Edition explains locally optimal solution →
Natural Language Processing in Action, Second Edition
A local minimum is a point where an objective function has a value no greater than that of nearby points, without necessarily being the global minimum.
- How it is used
- Frames a neural network training failure mode where optimization stops at a nearby error minimum.
- How deep it goes
- Explains nonconvex error surfaces, random starts, and practical monitoring, but does not derive optimization mathematics.
- Examples
- Uses ski slopes, nonconvex error curves, random weight initialization, shuffled samples, and ordered RNN training data.
- What you can do after
- Diagnose training stagnation by monitoring error and apply stochastic shuffling to vary the optimization path.
Read how Natural Language Processing in Action, Second Edition explains locally optimal solution →
Deep Learning with R, Third Edition
A local minimum is a point whose function value is no greater than the values at sufficiently nearby points.
- How it is used
- Explains why stochastic gradient descent may stop at a nearby solution rather than the best point on a loss curve.
- How deep it goes
- Defines local minima and connects them to learning-rate behavior, but stops short of formal analysis or high-dimensional optimization internals.
- Examples
- Uses one-dimensional loss curves, two-dimensional loss surfaces, and a visual comparison of local and global minima.
- What you can do after
- Diagnose how an excessively small or large learning rate can prevent useful progress during gradient descent.
Read how Deep Learning with R, Third Edition explains locally optimal solution →
Sutskever's List
A local minimum is a point whose objective value is no greater than that of nearby points, without necessarily being the global minimum.
- How it is used
- Frames local minima as a perceived obstacle to training multilayer neural networks and a misconception reassessed through AlexNet.
- How deep it goes
- Defines local minima and explains how high-dimensional loss surfaces and mini-batch gradients complicate the simple-trap picture.
- Examples
- Contrasts SVM optimization with neural-network training and uses AlexNet’s mini-batch optimization as a counterpoint to the local-minima concern.
- What you can do after
- Understand why local minima were considered a training risk and why that concern did not make gradient-based neural-network training ineffective.
Read how Sutskever's List explains locally optimal solution →
Deep Learning with JAX
A point where a function has a value no greater than that of nearby points.
- How it is used
- Uses local minima to frame calculus-based optimization and neural-network training through gradient descent.
- How deep it goes
- Defines local minima, distinguishes maxima and saddle points, and outlines derivative checks and iterative gradient descent.
- Examples
- Demonstrates minimization with f(x) = x**4 + 12*x + 1/x and neural-network weight updates on a loss landscape.
- What you can do after
- Evaluate candidates using derivatives and apply gradient descent to reduce a differentiable loss function.
Read how Deep Learning with JAX explains locally optimal solution →
Geometry for Programmers
A local minimum is a point whose function value is no greater than the values at all sufficiently nearby inputs.
- How it is used
- Uses local minima to locate turning points of polynomial functions through derivative analysis.
- How deep it goes
- Defines the geometric turn, links it to derivative roots and sign changes, and notes that a derivative root alone may not produce an extremum.
- Examples
- Uses a cubic polynomial, its quadratic derivative, plotted bumps, and a derivative root that does not create an extremum.
- What you can do after
- Evaluate/understand local minima by interpreting derivative roots and checking whether the derivative changes from negative to positive.
Read how Geometry for Programmers explains locally optimal solution →
Related concepts
- global optimumcontrasts with · A global optimum is a feasible point or solution whose objective value is at least as favorable as that of every other point or solution in the domain.
- simulated annealingaddressed by · Simulated annealing is a stochastic optimization method that gradually reduces the probability of accepting worse solutions to escape local optima.
- K-means clusteringproduced by · K-means clustering partitions data into a specified number of groups by assigning observations to representative centers and iteratively updating those centers to reduce within-group distances.
- greedy algorithmused by · A greedy algorithm builds a solution incrementally by choosing the locally best available option at each step, without generally accounting for all future consequences or revising prior choices.
- momentumcontrasts with · Momentum is an optimization technique that incorporates a running history of gradients or parameter updates to smooth and accelerate movement during training.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.