Activation
An activation is a numerical signal or intermediate representation produced by a neural-network unit or layer during computation.
Deep Learning with Python, Third Edition
The numeric value of a model activation.
- How it is used
- Uses activations to inspect how successive ConvNet layers transform images and to interpret learned filters.
- How deep it goes
- Defines activation tensors, visualizes individual channels, and optimizes filter responses, but does not survey every interpretation technique.
- Examples
- Uses a cat image, a sixth-channel edge visualization, complete activation grids, and optimized Xception filter patterns.
- What you can do after
- Diagnose/debug with it: extract, inspect, visualize, and optimize ConvNet activations to understand learned visual responses.
Read how Deep Learning with Python, Third Edition explains activation →
Natural Language Processing in Action, Second Edition
A hidden layer activation is the numerical output produced by a neural-network unit or layer for a particular input state.
- How it is used
- Uses activations to carry an RNN’s accumulated interpretation of sequential text into processing subsequent tokens.
- How deep it goes
- Explains storage, sequential overwriting, and compact representation, but does not detail the underlying neuron equations.
- Examples
- Uses token sequences, evolving text encodings, and a paint-smudge analogy to illustrate activation values.
- What you can do after
- Evaluate how an RNN activation changes as each token contributes to the sequence representation.
Read how Natural Language Processing in Action, Second Edition explains activation →
Deep Learning with R, Third Edition
An activation is the numerical output produced by a neural-network unit or layer for a given input.
- How it is used
- Uses activations to inspect how a convolutional network decomposes an input into learned features.
- How deep it goes
- Defines activations, extracts layer outputs, examines tensor shapes, and visualizes channels as feature maps.
- Examples
- Uses a held-out cat image, a 178 × 178 × 32 first-layer activation, individual channels, and complete activation grids.
- What you can do after
- Diagnose and interpret convolutional representations by extracting, inspecting, and plotting intermediate activations with Keras.
Read how Deep Learning with R, Third Edition explains activation →
Related concepts
- activation functionproduced by · An activation function maps a neural-network unit’s or layer’s input to an output, often introducing nonlinearity into the network’s computation.
- sparse autoencoderused by · A sparse autoencoder is an autoencoder trained to produce representations in which most encoded units are inactive or near zero.
- l2 normmeasured by · The L2 norm is the nonnegative Euclidean magnitude of a vector, equal to the square root of the sum of its components’ squares, and to absolute value for a scalar.
- neuronproduced by · A neuron is a computational unit in a neural network that transforms inputs, typically using learned parameters and an activation function, to produce an output.
- Swishincludes · Swish, also called SiLU, is a smooth neural-network activation function defined as x multiplied by the sigmoid of βx.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.