liveBook · Manning

Decision boundary

As explained in Timeless Algorithms, The Foundational Ideas

A decision boundary is a geometric surface separating regions assigned to different predicted classes.

A decision boundary is the threshold or surface at which a classifier’s predicted class changes. In a linear SVM, it is the hyperplane defined by \(w^T x+b=0\); its position determines the predicted classes. section 8.1.1 section 8.2.2

In support vector machines, the boundary is considered together with the separation around it:

  • A hard-margin SVM places parallel margins around the boundary and chooses the widest separation; the nearest training observations, called support vectors, determine its position. It requires every training observation to remain on or beyond its margin. section 8.1 section 8.2.2
  • A soft-margin SVM allows some observations inside the margin or across the boundary. Slack variables measure violations, while penalty parameter C balances those violations against margin width. section 10.2.7

Temperature classification

For one-dimensional temperature readings labeled safe or unsafe, a threshold separates the classes. When several thresholds classify the training readings correctly, the maximum-margin choice lies farthest from the nearest reading in either class. section 8.1.1

Figure 8.1 Temperature readings appear along a single horizontal axis, increasing from left to right. Circles represent safe conditions, and squares represent unsafe conditions. The vertical dashed line is the decision boundary. The nearest observation on each side is a support vector, and the distance between the boundary and these points defines the margin on either side. Positioning the boundary midway between the support vectors produces the maximum-margin separator.

The figure locates the threshold between safe and unsafe readings and shows how the nearest observations define the margin. section 8.1.1

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines the threshold and hyperplane, then explains margins, support vectors, and soft-margin optimization without resolving uncertainty about future data.
Examples
Uses safe and unsafe temperature readings, then considers noisy or mislabeled observations that can distort a hard-margin separator.
What you can do after
Evaluate how margin width, training observations, and soft-margin penalties shape a classifier’s boundary and its limits.
How it connects
Connects decision boundaries to support vector machines, hyperplanes, margins, support vectors, and generalization.

Also explained in

Compare how 7 books explain decision boundary →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.