liveBook · Manning

Activation

An activation is a numerical value or tensor produced by a neural-network unit, layer, or operation during a forward computation.

How 6 Manning books explain it

Quantization and Fast Inference

A numerical output produced by a neural-network layer or operation and passed to subsequent computation.

How it is used
Distinguishes activations from weights and KV cache when choosing quantization targets and precision.
How deep it goes
Works through affine calibration, quantization, linear-layer zero-point correction, and output error; introduces granularity choices without resolving them.
Examples
Compares ReLU activation resolution and tests shifted inputs with and without zero-point correction.
What you can do after
Evaluate activation quantization using tensor ranges, integer-grid resolution, and output error against an FP32 baseline.

Read how Quantization and Fast Inference explains activation →

Rearchitecting LLMs

An activation is a value produced by a neural-model unit during computation.

How it is used
Uses activations to connect inference behavior with explainability and behavior-driven model optimization.
How deep it goes
Defines activations and develops capture, visualization, and cross-model validation workflows.
Examples
Uses a four-token Llama tensor, binned heatmaps, and demographic prompt pairs.
What you can do after
Diagnose model behavior with PyTorch hooks, activation heatmaps, and logit lens without training.

Read how Rearchitecting LLMs explains activation →

Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI

A neural network activation is the output value of a unit or layer during a forward computation.

How it is used
Uses activations to find model-based outliers for diversity sampling.
How deep it goes
Explains layer selection, activation extraction, and rank-based comparison, including limitations of absolute values.
Examples
Illustrates ranked neuron values for ten items and a forward function that returns hidden activations and logits.
What you can do after
Apply it via PyTorch to expose layer outputs and rank low-activation candidates for outlier sampling.

Read how Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI explains activation →

Deep Learning with Python, Third Edition

The numeric value of a model activation.

How it is used
Uses activations to inspect how successive ConvNet layers transform images and to interpret learned filters.
How deep it goes
Defines activation tensors, visualizes individual channels, and optimizes filter responses, but does not survey every interpretation technique.
Examples
Uses a cat image, a sixth-channel edge visualization, complete activation grids, and optimized Xception filter patterns.
What you can do after
Diagnose/debug with it: extract, inspect, visualize, and optimize ConvNet activations to understand learned visual responses.

Read how Deep Learning with Python, Third Edition explains activation →

Natural Language Processing in Action, Second Edition

A hidden layer activation is the numerical output produced by a neural-network unit or layer for a particular input state.

How it is used
Uses activations to carry an RNN’s accumulated interpretation of sequential text into processing subsequent tokens.
How deep it goes
Explains storage, sequential overwriting, and compact representation, but does not detail the underlying neuron equations.
Examples
Uses token sequences, evolving text encodings, and a paint-smudge analogy to illustrate activation values.
What you can do after
Evaluate how an RNN activation changes as each token contributes to the sequence representation.

Read how Natural Language Processing in Action, Second Edition explains activation →

Deep Learning with R, Third Edition

An activation is the numerical output produced by a neural-network unit or layer for a given input.

How it is used
Uses activations to inspect how a convolutional network decomposes an input into learned features.
How deep it goes
Defines activations, extracts layer outputs, examines tensor shapes, and visualizes channels as feature maps.
Examples
Uses a held-out cat image, a 178 × 178 × 32 first-layer activation, individual channels, and complete activation grids.
What you can do after
Diagnose and interpret convolutional representations by extracting, inspecting, and plotting intermediate activations with Keras.

Read how Deep Learning with R, Third Edition explains activation →

Also covered in

What to know first

Related concepts

Questions about activation

What is activation?

An activation is a numerical value or tensor produced by a neural-network unit, layer, or operation during a forward computation.

Which Manning books explain activation?

Quantization and Fast Inference, Rearchitecting LLMs, Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI, Deep Learning with Python, Third Edition, Natural Language Processing in Action, Second Edition and Deep Learning with R, Third Edition.

What should I know before learning activation?

Manning books that teach activation usually introduce adam, backpropagation, gradient and language model first.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.