Universal approximator

As explained in Deep Learning with PyTorch, Second Edition

A function class that can approximate a wide range of target functions to arbitrary accuracy under suitable conditions.

A universal approximator is a neural-network model that can represent highly nonlinear input/output relationships without requiring a handcrafted formula for the data. Stacking linear transformations with differentiable activations lets the model adapt its capacity to the problem. section 6.1.6

From neurons to nonlinear shapes

A dog-image classifier is difficult to describe with a manually designed mathematical function. A deep network instead composes simple units and estimates their parameters with gradient descent. With Tanh neurons A, B, C, and D, changing weights and biases shifts curves; adding and composing them produces bumps and other increasingly complex outputs. section 6.1.6 section 6.1.6

Figure 6.6 Composing multiple linear units and tanh activation functions to produce nonlinear outputs

Follow the progression from individual linear units and Tanh curves to combined and layered nonlinear outputs. section 6.1.6

How Deep Learning with PyTorch, Second Edition covers it

How deep it goes

Explains nonlinear function composition through Tanh-neuron examples and stops short of formal approximation conditions or proof.

Examples

Uses dog-image classification and composed Tanh neurons with varied weights and biases as concrete examples.

What you can do after

Understand how composing simple neural units creates increasingly complex functions without specifying the exact target formula.

How it connects

Connects neural networks with linear transformations, differentiable activation functions, gradient descent, and model capacity.

liveBookmanning.com

Read Deep Learning with PyTorch, Second Edition on liveBook Get the book at manning.com

Related concepts

• language modelincludes · A language model is a computational model that represents patterns in linguistic tokens to estimate sequence probabilities and predict or generate language from context.
• Gaussian mixture modelincludes · A Gaussian mixture model is a probabilistic model representing a distribution as a weighted combination of multiple Gaussian component distributions.
• Universal Approximation Theoremequivalent to · The Universal Approximation Theorem states that certain sufficiently expressive neural networks can approximate continuous functions on compact domains arbitrarily closely under suitable conditions.

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with PyTorch, Second Edition. The section links point to the parts of the book it draws on.