Shared weight
Weight sharing is a neural-network design in which the same learned parameters are reused across multiple input positions, contexts, or computation steps.
Inside Deep Learning: Math, Algorithms, Models
A convolutional-network technique that reuses the same parameters at multiple input positions instead of learning separate parameters for each position.
- How it is used
- Deploys one learned transformation across spatial positions, hidden layers, or sequence time steps.
- How deep it goes
- Explains reuse mathematically and in PyTorch, then stops short of deriving the full convolution or recurrent optimization equations.
- Examples
- Uses sliding one-dimensional windows, repeated hidden layers, MNIST accuracy comparisons, RNN time steps, and shared autoencoder decoder layers.
- What you can do after
- Apply it via PyTorch by reusing one module object across compatible spatial, layer, or temporal positions.
Read how Inside Deep Learning: Math, Algorithms, Models explains shared weight →
Natural Language Processing in Action, Second Edition
Weight sharing is the reuse of the same learned parameters across multiple inputs, positions, or computation steps.
- How it is used
- Uses one parameter set across sequence positions so recurrent networks can process tokens step by step.
- How deep it goes
- Explains repeated matrix use and serial gradient propagation, but does not provide implementation code or parameter-count analysis.
- Examples
- Uses an unrolled RNN processing text tokens and propagating errors backward through repeated weight-matrix multiplications.
- What you can do after
- Understand how recurrent models reuse parameters and why their training must process sequence errors backward one token at a time.
Read how Natural Language Processing in Action, Second Edition explains shared weight →
Related concepts
- recurrent neural networkused by · A recurrent neural network is a neural network that processes sequences by updating an internal state across successive inputs, allowing earlier inputs to influence later computations.
- convolutional layerused by · A convolutional layer is a neural-network layer that applies shared learned filters to local regions of structured input to produce feature representations.
- convolution kernelconfigures · A convolution kernel is a small array or tensor of weights applied across local input regions to compute weighted responses and detect or transform patterns.
- autoencodercontrasts with · An autoencoder is a neural network that encodes input data into an internal representation and decodes it to reconstruct the input or an approximation.
- layer normalizationused by · Layer normalization standardizes neural-network activations across specified feature dimensions for each example independently, often applying learned scale and bias parameters.
- LSTMuses · An LSTM is a recurrent neural network architecture with gated memory mechanisms that control information flow and help retain or update information across sequences.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.