A linear map preserves combinations: f(ax + by) = af(x) + bf(y). A nonlinear mapping does not generally preserve them. In neural networks, activation functions introduce nonlinearity so successive transformations can represent relationships that repeated linear combinations cannot. section 11.2.1 section 11.8
Changing the decision problem
Six observations contain three central circles and three surrounding squares. A curved boundary is needed to separate them in the original coordinates. A hidden layer nonlinearly transforms the observations into new coordinates, where a straight boundary can separate the classes. section 11.5.2
Figure 11.3 A nonlinear transformation reorganizes the same six observations so that classes requiring a curved boundary in the input space can be separated by a straight boundary in the learned representation.

See how the same observations move from a curved separation in input space to a straight separation in the learned representation. section 11.5.2










