liveBook · Manning

Nonlinear mapping

As explained in Timeless Algorithms, The Foundational Ideas

Nonlinearity is the property of a transformation that does not preserve linear combinations, allowing neural networks to represent relationships beyond repeated linear combinations.

A linear map preserves combinations: f(ax + by) = af(x) + bf(y). A nonlinear mapping does not generally preserve them. In neural networks, activation functions introduce nonlinearity so successive transformations can represent relationships that repeated linear combinations cannot. section 11.2.1 section 11.8

Changing the decision problem

Six observations contain three central circles and three surrounding squares. A curved boundary is needed to separate them in the original coordinates. A hidden layer nonlinearly transforms the observations into new coordinates, where a straight boundary can separate the classes. section 11.5.2

Figure 11.3 A nonlinear transformation reorganizes the same six observations so that classes requiring a curved boundary in the input space can be separated by a straight boundary in the learned representation.

See how the same observations move from a curved separation in input space to a straight separation in the learned representation. section 11.5.2

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains nonlinearity through activation functions and a classification geometry example, without deriving activation functions mathematically.
Examples
Illustrates circles and squares reorganized from a curved separation into a straight boundary in learned coordinates.
What you can do after
Understand how nonlinear transformations can simplify class separation by changing the representation of observations.
How it connects
Connects activation functions and nonlinear transformations to neural networks, representation learning, and classification.

Also explained in

Compare how 11 books explain nonlinear mapping →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.