A language modeling head converts each contextual token representation into vocabulary-sized prediction scores. In a protein language model, it maps transformer outputs to logits for every amino-acid position, supporting masked-token prediction. section 12.3.3
Tracing tensor dimensions
For a batch of 4 sequences padded to length 50, with 128-dimensional representations and a 25-token vocabulary, the head transforms (4, 50, 128) into (4, 50, 25). Each position therefore receives one logit for every amino acid or special token. section 12.3.3

