Nonlinear mapping

As explained in Math and Architectures of Deep Learning

A property of a function or layer whose output is not a linear transformation of its inputs.

A nonlinear mapping produces outputs that cannot be represented as a linear transformation of its inputs. Matrix-vector multiplication is linear; mapping a vector to its length provides a contrasting nonlinear example because it violates the required combination rule. In machine learning, nonlinearity lets models represent curved decision boundaries and approximate complex classification or estimation functions. section 2.10.2 section 1.6

Nonlinearity matters because it changes what a model can express:

• A weighted sum alone creates a linear model and cannot adequately fit a nonlinear data distribution. section 3.5.3
• Applying a sigmoid to a weighted sum supplies nonlinearity and supports more complex classification tasks. section 1.6
• Stacking nonlinear layers increases expressive power: lower layers can learn simpler features, while higher layers represent more abstract features. section 1.7 section 11.2

Fitting a quadratic function with PyTorch

For data generated by y = x² − x + 2, a linear prediction y_pred = b + wx cannot converge to the underlying curve. Adding a quadratic term gives y_pred = w₀ + w₁x + w₂x²; training then learns coefficients close to 2, −1, and 1. section 3.5.3

Figure 1.5 The sigmoid graph

Use the sigmoid graph to connect the mathematical shape of a nonlinear function with its role after a weighted sum. section 1.6

How Math and Architectures of Deep Learning covers it

How deep it goes

Defines linearity mathematically, contrasts vector-length mapping, and connects sigmoid and layered architectures to expressive power.

Examples

Uses curved separators, sigmoid graphs, dog recognition, vector length, and quadratic regression implemented with PyTorch.

What it assumes

Assumes vectors, matrices, weighted sums, model parameters, and training with error minimization.

What you can do after

Apply it via PyTorch by adding nonlinear terms or nonlinear layers when linear models cannot fit the observed distribution.

How it connects

Connects nonlinear mapping with linear transformation, sigmoid function, expressive power, neural networks, and convolutional layers.

liveBookmanning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA nonlinear transformation maps inputs to outputs in a way that cannot be represented as a linear function.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionA parameterized operation that maps inputs to outputs without preserving linearity, allowing a neural network to model nonlinear relationships.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA process whose behavior cannot be represented as a simple linear relation between inputs and outputs.
• 

  liveBookmanning.com
  Statistics Slam DunkNonlinearity is the property that a relationship cannot be represented adequately by a straight-line pattern or linear function.
• 

  liveBookmanning.com
  Statistics Every Programmer NeedsA nonlinear relationship is a relationship in which changes in one quantity are not represented by a linear function of another.
• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA transformation inserted between linear steps to let a neural network represent functions that are not purely linear.

Compare how 7 books explain nonlinear mapping →

Related concepts

• activation functionproduced by · An activation function maps a neural-network unit’s or layer’s input to an output, often introducing nonlinearity into the network’s computation.
• 80-20 ruleequivalent to · The 80-20 rule is an informal heuristic that roughly 80 percent of an outcome may be associated with 20 percent of its contributing causes or effort.
• linear regressioncontrasts with · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.
• mathematical functionkind of · A mathematical function is a mapping that assigns exactly one output to each permitted input in its domain.
• expressive powerproduces · Expressive power is the capacity of a model, language, or system to represent or convey a range and complexity of functions, patterns, distinctions, or ideas.
• perceptroncontrasts with · A perceptron is a single-layer computational model and linear classifier that combines weighted inputs with a bias and applies a threshold or activation rule.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.