Pooling

As explained in Natural Language Processing in Action, Second Edition

A pooling layer is a neural-network layer that aggregates neighboring activations to reduce their representation while retaining salient information.

Pooling compresses many activations into fewer values, helping a convolutional neural network summarize whether a learned pattern appears in text. Max pooling selects the strongest response; average pooling computes the mean. section 7.5.2 section 7.2.1

In a word-based CNN, convolution produces one value for each possible kernel position. Pooling reduces that sequence to a compact representation, often a single value per filter, so variable-length text can become a fixed-size encoding when padding and clipping control the input length. section 7.4.1 section 7.2.1

Common pooling choices include:

• Max pooling keeps the largest convolution response and emphasizes the strongest detected match. section 7.5.2 section 7.2.1
• Average pooling takes the mean of values in a window and can represent patterns distributed across several positions. section 7.5.2 section 7.2.1
• Other possible reduction statistics include minimum, standard deviation, and sum. section 7.5.2

Whole-sequence max pooling in PyTorch

After a convolution, calculate its output sequence length and use that length as the MaxPool1d kernel size. For example, a length-40 input with kernel lengths 2 through 6 produces convolution lengths 39, 38, 37, 36, and 35; pooling each complete sequence reduces each branch to length 1. Concatenating five branches yields five pooled outputs before classification.

Figure 7.1 A real-life stencil

Use the stencil figure to visualize convolution as scanning for a pattern and pooling as selecting or averaging its responses. section 7.2.1

How Natural Language Processing in Action, Second Edition covers it

How deep it goes

Explains aggregation choices and tensor-shape consequences, stopping short of detailed pooling mathematics.

Examples

Uses word-based CNNs, disaster tweets, MaxPool1d branches, and an SOS distress-signal exercise.

What it assumes

Assumes tokenization, embeddings, convolution, and tensor shapes while teaching pooling intuition and choices directly.

What you can do after

Apply it via PyTorch to reduce convolution outputs before a neural-network classifier.

How it connects

Connects pooling to convolution, embeddings, padding, clipping, dropout, and classification.

liveBookmanning.com

Read Natural Language Processing in Action, Second Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Inside Deep Learning: Math, Algorithms, ModelsA neural-network operation that downsamples local neighborhoods by aggregating their values.
• 

  liveBookmanning.com
  Math and Architectures of Deep LearningA downsampling operation on feature maps that reduces spatial resolution, often by local averaging or max selection.
• 

  liveBookmanning.com
  Build a Text-to-Image Generator (from Scratch)A pooling layer aggregates values within local spatial windows to produce a reduced or transformed feature representation.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksPooling is a neural-network operation that aggregates values within local regions to reduce spatial or feature-map dimensions.
• 

  liveBookmanning.com
  Graph Neural Networks in ActionA graph operation that reduces a collection of representations to a smaller representation, often for graph-level prediction.
• 

  liveBookmanning.com
  Deep Learning with PyTorch, Second EditionA downsampling technique that reduces spatial resolution by aggregating local regions of a feature map.
• 

  liveBookmanning.com
  AI-Powered SearchPooling is the aggregation of multiple representation values into a single lower-level or fixed-size representation.

Compare how 8 books explain pooling →

Related concepts

• convolutional neural networkused by · A convolutional neural network is a neural-network architecture that applies learned convolutional operations to extract patterns from structured data.
• max poolingincludes · Max pooling is an aggregation operation that replaces each local or grouped region with its maximum value, often to downsample a representation.
• downsamplingequivalent to · Downsampling is the reduction of data quantity, spatial resolution, temporal resolution, or sampling rate by retaining, selecting, or aggregating fewer representative values or observations.
• upsamplingalternative to · Upsampling is a process that increases the number, density, or spatial or temporal resolution of samples or values in a dataset or representation by adding, duplicating, or estimating them.
• gradientuses · A gradient is the vector of partial derivatives of a scalar function with respect to its variables, indicating its local rate and direction of change.

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.