A transformer is a neural-network architecture built around multi-head attention. It is useful when a decision depends on relationships among inputs: a cooling controller, for example, must account for heat moving between data center zones rather than treating each zone independently. section 9.4 section 9.6
A transformer encoder layer combines two sub-layers: multi-head self-attention enriches each zone’s representation using information from other zones, and a position-wise feedforward network processes each enriched representation independently. section 9.6
Each sub-layer uses a residual connection and layer normalization: Output = LayerNorm(x + SubLayer(x)). Adding the input back preserves existing information and gives gradients a direct path through the network. Normalization re-centers and rescales activations to help keep their scale stable. section 9.6
Stacking encoder layers produces deeper representations. In cooling, one layer might capture direct relationships between zones; a later layer can use those representations to capture indirect relationships. section 9.6
Figure 9.8 Transformer architecture.
Follow the encoder architecture and its sub-layers in the diagram. section 9.6
Coordinating data center cooling
A REINFORCE agent controls eight interconnected zones arranged in a ring. Each zone supplies eight features, including temperature, workload, neighbor temperature, and previous actions. The agent chooses a fan speed and chilled-water valve opening for each zone, balancing energy use against thermal safety. A transformer encoder lets the agent represent relationships among zones when coordinating those actions. section 9.7 section 9.4
Listing 9.6 Transformer actor-critic architecture
class TransformerPolicyValue(nn.Module):
def __init__(self, config: Config):
super().__init__()
self.cfg = config
obs_dim = 8 #A
action_dim = 2 #B
self.input_embed = nn.Sequential(
nn.Linear(obs_dim, config.d_model),
nn.LayerNorm(config.d_model),
nn.ReLU()) #C
self.pos_encoding = PositionalEncoding(
config.d_model, max_len=config.n_zones + 1) #D
encoder_layer = nn.TransformerEncoderLayer(
d_model=config.d_model,
nhead=config.n_heads,
dim_feedforward=config.d_ff,
dropout=config.dropout,
activation='gelu',
batch_first=True) #E
self.transformer = nn.TransformerEncoder(
encoder_layer, num_layers=config.n_layers) #F
self.policy_head = nn.Sequential(
nn.Linear(config.d_model, config.d_model),
nn.ReLU(),
nn.Linear(config.d_model, action_dim * 2)) #G
self.value_head = nn.Sequential(
nn.Linear(config.d_model, config.d_model),
nn.ReLU(),
nn.Linear(config.d_model, 1)) #H
self.log_std_min = -5.0 #I
self.log_std_max = 2.0 #I
Inspect the PyTorch actor-critic setup: per-zone input embedding, positional encoding, and stacked transformer encoder layers. section 9.7
Transformers also appear in language models. There, input text is split into tokens and mapped to numerical IDs before flowing through the model to help predict the next token. section 10.2







































