Translation invariance
Translation invariance is the property that a function, representation, or prediction remains unchanged or equivalent when its input is shifted in position.
Deep Learning with PyTorch, Second Edition
The property of a model or representation whose output is unchanged, or changes minimally, when an input object shifts position.
- How it is used
- Motivates convolutional image classifiers that detect an object despite changes in its position.
- How deep it goes
- Explains the failure of fully connected layers and the shared local weights of convolution, then extends the intuition to pooling.
- Examples
- Uses an airplane shifted within an image, a shared-kernel diagram, and a max-pooling tensor-shape example.
- What you can do after
- Evaluate/understand why convolution better handles shifted image patterns than a fully connected layer.
Read how Deep Learning with PyTorch, Second Edition explains translation invariance →
Inside Deep Learning: Math, Algorithms, Models
A property of a model whose output is unchanged, or less changed, when the input is shifted in position.
- How it is used
- Uses pooling to make image-classification predictions more robust to object movement.
- How deep it goes
- Explains shifts, pooling, stride, aliasing, and anti-aliased pooling, but does not establish exact invariance.
- Examples
- Demonstrates one-pixel MNIST shifts and evaluates CIFAR-10 predictions across 8-by-8 crop offsets.
- What you can do after
- Diagnose translation sensitivity by comparing predictions across shifted crops and apply anti-aliased pooling for greater consistency.
Read how Inside Deep Learning: Math, Algorithms, Models explains translation invariance →
Deep Learning with Python, Third Edition
The property that a learned pattern can be recognized at different spatial locations without relearning it from scratch.
- How it is used
- Guides the choice of convolution when local patterns should retain their meaning across spatial or sequential locations.
- How deep it goes
- Explains shared local transformations and practical limits, but stops short of a mathematical derivation of convolution.
- Examples
- Uses moving image edges and textures, plus a Conv1D temperature-forecasting model that achieves about 2.9 degrees validation MAE.
- What you can do after
- Evaluate whether convolutional models suit data whose local patterns behave consistently across locations.
Read how Deep Learning with Python, Third Edition explains translation invariance →
Natural Language Processing in Action, Second Edition
Translation invariance is the property of recognizing a pattern despite changes in its position within an input.
- How it is used
- Supports convolutional neural networks in recognizing text patterns wherever they occur.
- How deep it goes
- Defines the property and contrasts it with fully connected networks, without detailing convolution or model training.
- Examples
- Uses word patterns moving from sentence beginnings to endings and CNN-based named entity recognition as examples.
- What you can do after
- Apply it via convolutional neural networks to recognize patterns across different text positions.
Read how Natural Language Processing in Action, Second Edition explains translation invariance →
Sutskever's List
The property of recognizing a pattern similarly despite changes in its spatial location.
- How it is used
- Uses shared convolutional filters to recognize visual patterns at different image locations.
- How deep it goes
- Explains filter sharing and feature maps, then notes pooling’s added robustness to minor positional variations.
- Examples
- Illustrates location-independent pattern detection with a cat’s face and AlexNet’s 96 first-layer filters.
- What you can do after
- Understand how shared filters support recognition of visual patterns across image locations.
Related concepts
- inductive biaskind of · Inductive bias is the set of assumptions a learning method uses to favor some hypotheses or generalizations over others when learning from data.
- scale invariancecontrasts with · Scale invariance is the property that a relationship or recognized pattern retains its form when the system’s scale or dimensions change.
- poolingaddressed by · Pooling is an operation that aggregates values or representations across regions or axes to produce a reduced or summarized representation.
- convolutional layeraddressed by · A convolutional layer is a neural-network layer that applies shared learned filters to local regions of structured input to produce feature representations.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.