Multi-head attention enriches an input representation by running several independent attention computations in parallel. Each head uses its own learned Query, Key, and Value projections, performs scaled dot-product attention, and contributes an output; the outputs are concatenated and passed through a final linear projection. section 9.5
Figure 9.7 Multi head attention architecture
The architecture diagram shows how separate attention heads process inputs and combine their outputs. section 9.5
In a cooling system, different heads can learn different relationships between zones:
- Thermal proximity between zones. section 9.5
- Similar workload patterns. section 9.5
- Pressure dynamics or shared cooling infrastructure. section 9.5
A Transformer encoder uses multi-head self-attention so each zone can attend to every other zone, followed by a feedforward network. Residual connections and layer normalization wrap these sub-layers; stacking layers can capture indirect zone-to-zone effects. The resulting representations can inform a reinforcement-learning policy. section 9.6 section 9.8
























