A [CLS] token is a special Transformer input token prepended to a sequence. It represents no amino acid; through attention, its final hidden state can gather information from every position and summarize the sequence for classification. section 12.3.4
Use [CLS] pooling as one of several ways to convert per-position representations into one sequence vector:
- Use the final [CLS] representation when the model has learned to collect sequence-wide information there. section 12.3.4
- Use mean pooling to average real-token representations, masking padding so meaningless values do not dilute the result. section 12.3.4
- Use max pooling to emphasize the strongest activation at each representation dimension. section 12.3.4


