Nonlinear mapping

Q: How deeply does Inside Deep Learning: Math, Algorithms, Models cover nonlinear mapping?

Defines sigmoid and tanh, gives their ranges, and demonstrates tanh placement in a PyTorch network.

As explained in Inside Deep Learning: Math, Algorithms, Models

A transformation inserted between linear steps to let a neural network represent functions that are not purely linear.

A nonlinearity follows a linear operation so a neural network can build complex functions instead of remaining purely linear. These functions are called activation functions. section 2.2.3

Two historical activation functions illustrate the idea:

Sigmoid, σ(x), maps inputs into [0, 1]. section 2.2.3
Hyperbolic tangent, tanh(x), maps inputs into [−1, 1]. section 2.2.3

Insert tanh between linear layers

In PyTorch, use nn.Sequential(nn.Linear(1, 10), nn.Tanh(), nn.Linear(10, 1)). With two linear layers, one activation sits between the hidden and output layers. section 2.2.3

How Inside Deep Learning: Math, Algorithms, Models covers it

Cover of Inside Deep Learning: Math, Algorithms, Models

liveBook manning.com

Read Inside Deep Learning: Math, Algorithms, Models on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Math and Architectures of Deep LearningA property of a function or layer whose output is not a linear transformation of its inputs.
liveBook manning.com

Machine Learning for Tabular Data: XGBoost, Deep Learning, and AIA nonlinear transformation maps inputs to outputs in a way that cannot be represented as a linear function.
liveBook manning.com

Graph Neural Networks in ActionA parameterized operation that maps inputs to outputs without preserving linearity, allowing a neural network to model nonlinear relationships.
liveBook manning.com

Deep Learning with PyTorch, Second EditionA process whose behavior cannot be represented as a simple linear relation between inputs and outputs.
liveBook manning.com

Statistics Slam DunkNonlinearity is the property that a relationship cannot be represented adequately by a straight-line pattern or linear function.
liveBook manning.com

Statistics Every Programmer NeedsA nonlinear relationship is a relationship in which changes in one quantity are not represented by a linear function of another.

Compare how 7 books explain nonlinear mapping →

Related concepts

activation functionproduced by · An activation function maps a neural-network unit’s or layer’s input to an output, often introducing nonlinearity into the network’s computation.
80-20 ruleequivalent to · The 80-20 rule is an informal heuristic that roughly 80 percent of an outcome may be associated with 20 percent of its contributing causes or effort.
linear regressioncontrasts with · Linear regression is a statistical modeling method that estimates a continuous response as a linear combination of predictor variables and fitted coefficients.
mathematical functionkind of · A mathematical function is a mapping that assigns exactly one output to each permitted input in its domain.
expressive powerproduces · Expressive power is the capacity of a model, language, or system to represent or convey a range and complexity of functions, patterns, distinctions, or ideas.
perceptroncontrasts with · A perceptron is a single-layer computational model and linear classifier that combines weighted inputs with a bias and applies a threshold or activation rule.

This explainer was generated by Manning's knowledge graph from the text of Inside Deep Learning: Math, Algorithms, Models. The section links point to the parts of the book it draws on.