A semantic association is a learned relationship that makes one concept more likely to be recalled or predicted given another. In a transformer, repeated exposure to a title and its author can strengthen that relationship through prediction-error updates across the model’s parameters. section 12.2.2
The association is distributed rather than stored as a complete fact in one embedding or attention head. Training updates embeddings, attention projections, feed-forward layers, and output parameters. At prediction time, the prompt representation contributes to logits for possible answers; attention helps construct that representation, while the candidate answers are evaluated as possible outputs. section 12.2.2
A title–author association
After encountering The Count of Monte Cristo and Alexandre Dumas across many contexts, a model may assign Dumas a higher output score than Hugo, Tolstoy, or Dickens when prompted for the author. A simplified dot product illustrates how a candidate’s output weights can score against the prompt representation; it does not describe the full transformer computation. section 12.2.2 section 12.2
This simplified dot product gives Dumas an output score of 0.91 against the example final representation. It illustrates candidate scoring, not a complete or literal account of how a trained transformer stores the association. section 12.2.2
