liveBook · Manning

Feature space

As explained in Timeless Algorithms, The Foundational Ideas

A feature space is the geometric space whose dimensions correspond to the features used to describe observations.

A feature space is the geometric space defined by the features used to represent observations. Each feature supplies an axis, and each observation becomes a point. For temperature safety, low and high readings may both be unsafe while moderate readings are safe; no single threshold on the temperature axis separates the classes. section 8.1.2

Adding squared temperature as a second coordinate maps each reading t to (t, t²). This polynomial transformation changes the points’ geometric arrangement, not the observations or their labels: a straight line can separate classes that a one-dimensional threshold cannot. This boundary is a hyperplane; the example demonstrates separation, not the maximum-margin boundary. section 8.1.2

SVMs can use richer feature spaces to make nonlinear patterns linearly separable. Manually adding squared features is one approach; kernel methods let SVMs operate in transformed spaces implicitly. section 8.1.2 section 8.0

Figure 8.3 Mapping each temperature reading into a two-dimensional feature space defined by temperature and temperature squared makes the classes linearly separable. The points are schematic rather than exact functional values. The dashed line is a hyperplane separating the safe observations, shown as circles, from the unsafe observations, shown as squares. It demonstrates possible separation, not the optimal maximum-margin boundary.

The diagram shows temperature readings mapped to temperature and temperature-squared coordinates, where a line separates the classes. section 8.1.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Defines feature space and works through a squared-feature transformation, distinguishing possible separation from the maximum-margin boundary.
Examples
Uses temperature readings where unsafe extremes surround safe moderate values, then adds squared temperature as a feature.
What you can do after
Understand how adding a squared feature can make a nonlinear class pattern linearly separable.
How it connects
Connects feature spaces to support vector machines, polynomial transformations, hyperplanes, and kernel methods.

Also explained in

Compare how 3 books explain feature space →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.