liveBook · Manning

Saddle point

As explained in Ensemble Methods for Machine Learning

A saddle point is a stationary point where a function curves upward in some directions and downward in others, so it is not a local minimizer.

A saddle point is stationary but not minimizing: its gradient is zero, while the function curves upward in some directions and downward in others. It resembles a bowl-shaped local minimizer, so descent methods cannot distinguish the two from the gradient alone. section 6.1.1

Descent on the Branin function

Starting from an initial point, Newton’s descent can move toward a local minimizer, but another initialization may converge to a saddle point. The Branin function illustrates a saddle between two minimizers, each location having a zero gradient. Convex, well-shaped loss functions reduce this concern; custom non-convex losses require care. section 6.1.1

How Ensemble Methods for Machine Learning covers it

How deep it goes
Defines zero-gradient behavior and directional curvature, but stops short of proving saddle-point detection methods.
Examples
Uses the Branin function and a figure showing a saddle between two local minimizers.
What it assumes
Assumes familiarity with gradients, local minimizers, and descent algorithms.
What you can do after
Evaluate optimization behavior by recognizing when zero gradients may indicate a saddle point rather than a local minimizer.
How it connects
Connects saddle points with local minimizers, gradient descent, Newton’s method, convexity, and loss functions.
Cover of Ensemble Methods for Machine Learning
liveBookmanning.com
Read Ensemble Methods for Machine Learning on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Ensemble Methods for Machine Learning. The section links point to the parts of the book it draws on.