liveBook · Manning

Transformer encoder layer

As explained in Applied Reinforcement Learning

A Transformer encoder block containing self-attention and position-wise feedforward sub-layers with residual connections and normalization.

A Transformer encoder layer enriches each zone’s representation by letting it attend to every other zone, then processes each zone independently with a feedforward network. section 9.6

Each sub-layer preserves and stabilizes the representation:

  • Add the sub-layer output to its input through a residual connection, then apply layer normalization. section 9.6
  • Stack layers to build deeper relationships: one layer can capture direct neighbors, while a second can capture indirect links such as zone 1 influencing zone 3 through zone 2. section 9.6
Figure 9.8 Transformer architecture.

Use the architecture figure to follow the encoder layer’s structure. section 9.6

How Applied Reinforcement Learning covers it

How deep it goes
Explains sub-layer functions, residual addition, normalization, and stacked-layer relationships without detailing implementation.
Examples
Uses cooling-floor zones to illustrate direct and indirect relationships across two stacked layers.
What you can do after
Understand how attention, feedforward processing, residual connections, and normalization combine within an encoder layer.
How it connects
Connects self-attention and feedforward networks with residual connections, layer normalization, and stacked Transformer layers.

Also explained in

Compare how 4 books explain Transformer encoder layer →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.