Translation invariance
Translation invariance is the property that a function, representation, or prediction remains unchanged or equivalent when its input is shifted in position.
Deep Learning with PyTorch, Second Edition
The property of a model or representation whose output is unchanged, or changes minimally, when an input object shifts position.
- How it is used
- Motivates image classifiers that recognize local patterns across different positions.
- How deep it goes
- Works through local weighted sums and shared kernel weights, while leaving continuous convolution aside.
- Examples
- Uses shifted airplane images and a large cross detected with small kernels and max pooling.
- What you can do after
- Evaluate how fully connected layers and shared convolution kernels handle patterns at different image positions.
Read how Deep Learning with PyTorch, Second Edition explains translation invariance →
Inside Deep Learning: Math, Algorithms, Models
A property of a model whose output is unchanged, or less changed, when the input is shifted in position.
- How it is used
- Uses pooling to make CNN image classifications more robust to shifts in object position.
- How deep it goes
- Defines invariance, explains max pooling, and tests shifted-image predictions; also shows aliasing limits and an anti-aliased implementation.
- Examples
- Compares predictions for one-pixel-shifted MNIST images and tests shifted CIFAR-10 crops with regular and anti-aliased CNNs.
- What you can do after
- Apply it via PyTorch pooling to improve consistency across image shifts, while checking for remaining aliasing.
Read how Inside Deep Learning: Math, Algorithms, Models explains translation invariance →
Deep Learning with Python, Third Edition
The property that a learned pattern can be recognized at different spatial locations without relearning it from scratch.
- How it is used
- Guides the choice of convolution when local patterns should retain their meaning across spatial or sequential locations.
- How deep it goes
- Explains shared local transformations and practical limits, but stops short of a mathematical derivation of convolution.
- Examples
- Uses moving image edges and textures, plus a Conv1D temperature-forecasting model that achieves about 2.9 degrees validation MAE.
- What you can do after
- Evaluate whether convolutional models suit data whose local patterns behave consistently across locations.
Read how Deep Learning with Python, Third Edition explains translation invariance →
Natural Language Processing in Action, Second Edition
Translation invariance is the property of recognizing a pattern despite changes in its position within an input.
- How it is used
- Supports convolutional neural networks in recognizing text patterns wherever they occur.
- How deep it goes
- Defines the property and contrasts it with fully connected networks, without detailing convolution or model training.
- Examples
- Uses word patterns moving from sentence beginnings to endings and CNN-based named entity recognition as examples.
- What you can do after
- Apply it via convolutional neural networks to recognize patterns across different text positions.
Read how Natural Language Processing in Action, Second Edition explains translation invariance →
Sutskever's List
The property of recognizing a pattern similarly despite changes in its spatial location.
- How it is used
- Uses shared convolutional filters to recognize visual patterns at different image locations.
- How deep it goes
- Explains filter sharing and feature maps, then notes pooling’s added robustness to minor positional variations.
- Examples
- Illustrates location-independent pattern detection with a cat’s face and AlexNet’s 96 first-layer filters.
- What you can do after
- Understand how shared filters support recognition of visual patterns across image locations.
Also covered in
What to know first
- linear layerA linear layer is a neural-network layer that applies a learned affine transformation to its input, typically using weights and a bias.
- convolutional layerA convolutional layer is a neural-network layer that applies shared learned filters to local regions of structured input to produce feature representations.
- convolutional neural networkA convolutional neural network is a neural-network architecture that processes structured inputs with convolutional operations, typically using learned local filters to capture patterns.
- channel dimensionA channel dimension is a dimension of image or tensor data whose positions distinguish separate color components or feature maps.
Related concepts
- max poolingproduced by · Max pooling is an aggregation operation that represents each group of values by its maximum, often reducing the size of a feature representation.
- inductive biaskind of · Inductive bias is the set of assumptions built into a learning method that favors some hypotheses, representations, or generalizations over others when learning from data.
- parameter reuseproduced by · Parameter reuse is the use of the same learned model parameters across multiple inputs, locations, or computation steps.
- convolutional layeraddressed by · A convolutional layer is a neural-network layer that applies shared learned filters to local regions of structured input to produce feature representations.
- poolingaddressed by · Pooling is an operation that aggregates values or representations, often summarizing local regions or combining multiple inputs into a smaller or fixed-size representation.
- scale invariancecontrasts with · Scale invariance is the property that a relationship or recognized pattern retains its form when the system’s scale or dimensions change.
Questions about translation invariance
What is translation invariance?
Translation invariance is the property that a function, representation, or prediction remains unchanged or equivalent when its input is shifted in position.
Which Manning books explain translation invariance?
Deep Learning with PyTorch, Second Edition, Inside Deep Learning: Math, Algorithms, Models, Deep Learning with Python, Third Edition, Natural Language Processing in Action, Second Edition and Sutskever's List.
What should I know before learning translation invariance?
Manning books that teach translation invariance usually introduce linear layer, convolutional layer, convolutional neural network and channel dimension first.
What is the difference between translation invariance and scale invariance?
Translation invariance is the property that a function, representation, or prediction remains unchanged or equivalent when its input is shifted in position. Scale invariance is the property that a relationship or recognized pattern retains its form when the system’s scale or dimensions change.
What is translation invariance a kind of?
Translation invariance is a kind of inductive bias. Inductive bias is the set of assumptions built into a learning method that favors some hypotheses, representations, or generalizations over others when learning from data.
What is translation invariance also called?
Translation invariance is also called translation invariant.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.




