liveBook · Manning

Translation invariance

Also called translation invariant

Translation invariance is the property that a function, representation, or prediction remains unchanged or equivalent when its input is shifted in position.

How 5 Manning books explain it

Deep Learning with PyTorch, Second Edition

The property of a model or representation whose output is unchanged, or changes minimally, when an input object shifts position.

How it is used
Motivates image classifiers that recognize local patterns across different positions.
How deep it goes
Works through local weighted sums and shared kernel weights, while leaving continuous convolution aside.
Examples
Uses shifted airplane images and a large cross detected with small kernels and max pooling.
What you can do after
Evaluate how fully connected layers and shared convolution kernels handle patterns at different image positions.

Read how Deep Learning with PyTorch, Second Edition explains translation invariance →

Inside Deep Learning: Math, Algorithms, Models

A property of a model whose output is unchanged, or less changed, when the input is shifted in position.

How it is used
Uses pooling to make CNN image classifications more robust to shifts in object position.
How deep it goes
Defines invariance, explains max pooling, and tests shifted-image predictions; also shows aliasing limits and an anti-aliased implementation.
Examples
Compares predictions for one-pixel-shifted MNIST images and tests shifted CIFAR-10 crops with regular and anti-aliased CNNs.
What you can do after
Apply it via PyTorch pooling to improve consistency across image shifts, while checking for remaining aliasing.

Read how Inside Deep Learning: Math, Algorithms, Models explains translation invariance →

Deep Learning with Python, Third Edition

The property that a learned pattern can be recognized at different spatial locations without relearning it from scratch.

How it is used
Guides the choice of convolution when local patterns should retain their meaning across spatial or sequential locations.
How deep it goes
Explains shared local transformations and practical limits, but stops short of a mathematical derivation of convolution.
Examples
Uses moving image edges and textures, plus a Conv1D temperature-forecasting model that achieves about 2.9 degrees validation MAE.
What you can do after
Evaluate whether convolutional models suit data whose local patterns behave consistently across locations.

Read how Deep Learning with Python, Third Edition explains translation invariance →

Natural Language Processing in Action, Second Edition

Translation invariance is the property of recognizing a pattern despite changes in its position within an input.

How it is used
Supports convolutional neural networks in recognizing text patterns wherever they occur.
How deep it goes
Defines the property and contrasts it with fully connected networks, without detailing convolution or model training.
Examples
Uses word patterns moving from sentence beginnings to endings and CNN-based named entity recognition as examples.
What you can do after
Apply it via convolutional neural networks to recognize patterns across different text positions.

Read how Natural Language Processing in Action, Second Edition explains translation invariance →

Sutskever's List

The property of recognizing a pattern similarly despite changes in its spatial location.

How it is used
Uses shared convolutional filters to recognize visual patterns at different image locations.
How deep it goes
Explains filter sharing and feature maps, then notes pooling’s added robustness to minor positional variations.
Examples
Illustrates location-independent pattern detection with a cat’s face and AlexNet’s 96 first-layer filters.
What you can do after
Understand how shared filters support recognition of visual patterns across image locations.

Read how Sutskever's List explains translation invariance →

Also covered in

What to know first

Related concepts

Questions about translation invariance

What is translation invariance?

Translation invariance is the property that a function, representation, or prediction remains unchanged or equivalent when its input is shifted in position.

Which Manning books explain translation invariance?

Deep Learning with PyTorch, Second Edition, Inside Deep Learning: Math, Algorithms, Models, Deep Learning with Python, Third Edition, Natural Language Processing in Action, Second Edition and Sutskever's List.

What should I know before learning translation invariance?

Manning books that teach translation invariance usually introduce linear layer, convolutional layer, convolutional neural network and channel dimension first.

What is the difference between translation invariance and scale invariance?

Translation invariance is the property that a function, representation, or prediction remains unchanged or equivalent when its input is shifted in position. Scale invariance is the property that a relationship or recognized pattern retains its form when the system’s scale or dimensions change.

What is translation invariance a kind of?

Translation invariance is a kind of inductive bias. Inductive bias is the set of assumptions built into a learning method that favors some hypotheses, representations, or generalizations over others when learning from data.

What is translation invariance also called?

Translation invariance is also called translation invariant.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.