liveBook · Manning

Identity initialization

As explained in Sutskever's List

Identity initialization sets a layer's parameters so that its initial transformation approximates the identity mapping.

Identity initialization sets a layer’s weights so it begins as a near-no-op: information passes forward unchanged, and training learns useful deviations from that mapping. section 3.5

Dilated-convolution context modules

In a segmentation context module, layers initially pass information through unchanged. Training then learns contextual refinements, stabilizing the module without explicit skip connections or architectural changes. section 3.5

How Sutskever's List covers it

How deep it goes
Defines a weight-based no-op and explains residual refinement, but provides no parameter-level initialization recipe.
Examples
Uses stacked dilated convolutions that initially pass inputs unchanged, then learn contextual refinements for segmentation.
What you can do after
Understand how near-identity weights can stabilize context-module training while allowing segmentation gains without architectural changes.
How it connects
Links identity initialization to residual connections, dilated convolutions, context modules, and semantic segmentation.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Sutskever's List. The section links point to the parts of the book it draws on.