How deeply does Math and Architectures of Deep Learning cover function approximation?

Connects surrogate functions, weights, nonlinear activations, perceptron geometry, and approximation capacity, while deferring practical parameter-count limits.

Function approximation

As explained in Math and Architectures of Deep Learning

The task of representing a target input-output mapping with a model, used here to justify nonlinear models for complex real-world processes.

Function approximation treats machine learning as the search for a model that maps inputs to desired outputs when the true transformation is unknown. Training adjusts tunable parameters, or weights, so model outputs match known examples; inference applies the resulting function to new inputs. section 1.2

A weighted sum is the simplest model architecture, but many real classification and estimation problems require nonlinear functions. Applying a nonlinear activation such as the sigmoid to a weighted sum creates a more expressive model for complex decision boundaries. section 1.6

Neural networks extend approximation capacity through perceptrons:

A single perceptron can represent linearly separable logical functions such as AND, OR, NOT, and voting. section 7.3.4
XOR cannot be represented by one perceptron but can be modeled by combining multiple perceptrons. section 7.3.4
A single hidden layer can approximate most functions to arbitrary accuracy in principle, although complicated targets may require many perceptrons; practical models often use multiple layers. section 7.3.4

A perceptron as a geometric function

For binary inputs, weights w₀ = w₁ = 1 and bias −1.5 produce y = ϕ(x₀ + x₁ − 1.5). The threshold separates the input plane at x₀ + x₁ = 1.5, assigning 1 only when both inputs equal 1, which implements AND. section 7.3.4

Figure 7.11 Geometrical views for the perceptrons in figure 7.10. Each dot represents an input point(a [x₀ x₁] vector). The shaded dots map to an output value of 1, and the unshaded dots map to an output value of 0. The thick line indicates the decision boundary.

(a) Logical AND decision boundary: x₀ + x₁ = 1.5

(b) Logical OR decision boundary: x₀ + x₁ = 0.5

Use the decision-boundary diagrams to connect perceptron parameters with geometric partitions of the input space for AND, OR, and NOT. section 7.3.4

How Math and Architectures of Deep Learning covers it

Cover of Math and Architectures of Deep Learning

liveBook manning.com

Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Evolutionary Deep Learning: Genetic algorithms and neural networksA function approximator is a computational model that estimates the outputs of a target function without necessarily reproducing its exact form.

Compare how 2 books explain function approximation →

Related concepts

deep learningaddressed by · Deep learning is a machine-learning approach that uses neural networks with multiple computational layers to learn representations from data.
machine learningequivalent to · Machine learning is a field of computational methods in which systems learn patterns or relationships from data to make predictions, decisions, or other outputs.
DQNimplemented by · DQN is a reinforcement-learning algorithm that uses a neural network to approximate a state-action value function for selecting actions.

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.