An activation is the numerical output produced by a neural-network unit or layer for a particular input. In a convolutional network, activations form feature maps whose channels can encode relatively independent features. section 10.1
A practical way to understand activations is to inspect a trained image model. Given a cat image, a Keras model can return the activations from every convolution and pooling layer. Each result records width, height, and channels; plotting individual channels reveals the visual patterns responding at each stage. section 10.1
Activation inspection typically follows these steps:
- Preprocess an image into the shape expected by the network. section 10.1
- Construct a multi-output Keras model whose outputs are selected layer activations. section 10.1
- Run prediction to obtain a list of activation arrays, then plot channels independently as two-dimensional images. section 10.1
Cat-image convolutional network
The first convolutional layer can produce a 178 × 178 feature map with 32 channels. Inspecting its sixth channel may reveal a diagonal-edge response, while a grid of all channels shows how representations change across the network. section 10.1
Listing 10.3 Instantiating a model that returns layer activations
is_conv_layer <- \(x) inherits(x, keras$layers$Conv2D)
is_pooling_layer <- \(x) inherits(x, keras$layers$MaxPooling2D)
layer_outputs <- list()
for (layer in model$layers) #1
if (is_conv_layer(layer) || is_pooling_layer(layer)) #1
layer_outputs[[layer$name]] <- layer$output #1
activation_model <- keras_model( #2
inputs = model$input,
outputs = layer_outputs
)
Browse the R/Keras construction of a model that returns convolutional and pooling-layer activations. section 10.1
