liveBook · Manning

Loss surface

Also called loss landscape

A loss surface is a function that maps model parameter values to their corresponding loss values.

How 3 Manning books explain it

Math and Architectures of Deep Learning

The geometric representation of a loss function over parameter space, where height indicates loss value for each parameter setting.

How it is used
Frames loss minimization as navigation across parameter space during supervised training.
How deep it goes
Defines the surface, maps it to geometry, and links it to gradients and minimization, but stops short of algorithmic variants beyond basic descent.
Examples
Uses concentric circles, concentric spheres, coaxial cylinders, heat maps, and a canyon or ravine analogy.
What you can do after
Apply gradient descent and momentum to move toward lower loss on a neural network surface.

Read how Math and Architectures of Deep Learning explains loss surface →

Sutskever's List

A loss landscape is the function relating a model's parameter values to its loss across parameter space.

How it is used
Frames the apparent difficulty of training deep networks and explains how AlexNet-era optimization challenged that view.
How deep it goes
Explains terrain metaphors, valley curvature, high-dimensional behavior, and mini-batch smoothing without formal analysis.
Examples
Uses a long, narrow valley to contrast oscillating large steps with slow progress along the valley’s base.
What you can do after
Understand why high-dimensional loss surfaces and mini-batch gradient averaging complicate the expectation that optimization gets trapped in poor minima.

Read how Sutskever's List explains loss surface →

Timeless Algorithms, The Foundational Ideas

A loss surface is a geometric representation of how a model's loss varies across possible parameter values.

How it is used
Frames neural-network training as searching parameter values for low loss while prioritizing performance on unseen observations.
How deep it goes
Explains non-convex geometry and optimization behavior, but offers no guarantee of reaching a global minimum.
Examples
Compares neural-network loss surfaces with convex SVM objectives and notes that training runs can differ with initialization and batch order.
What you can do after
Evaluate neural-network solutions by performance on unseen observations rather than training loss alone.

Read how Timeless Algorithms, The Foundational Ideas explains loss surface →

What to know first

Related concepts

Questions about loss surface

What is loss surface?

A loss surface is a function that maps model parameter values to their corresponding loss values.

Which Manning books explain loss surface?

Math and Architectures of Deep Learning, Sutskever's List and Timeless Algorithms, The Foundational Ideas.

What should I know before learning loss surface?

Manning books that teach loss surface usually introduce optimization, parameter, activation function and intercept first.

What is loss surface also called?

Loss surface is also called loss landscape.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.