liveBook · Manning

Residual connection

As explained in Quantization and Fast Inference

A residual connection adds a block's input to the output of a transformation applied to that input.

A residual connection adds a transformation’s output to its input: x + sublayer(x). In transformer quantization, errors can accumulate through layers with these additions, contributing to sensitivity. section 3.2

How Quantization and Fast Inference covers it

How deep it goes
Describes the x + sublayer(x) form and error accumulation across layers, without detailing residual-block implementation.
Examples
Illustrates the operation as x + sublayer(x) and links it to quantization errors accumulating across layers.
What you can do after
Understand why quantization errors may accumulate through layers with residual additions.
How it connects
Ties residual connections to transformer attention, LayerNorm, and quantization sensitivity.

Also explained in

Compare how 18 books explain residual connection →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Quantization and Fast Inference. The section links point to the parts of the book it draws on.