Tokenization divides text into vocabulary units—such as words, word parts, or punctuation—and maps them to integer identifiers. The same tokenizer maps the same text to the same identifiers. An identifier is only an index: it does not encode the token’s meaning. Instead, it selects a row from a learned embedding matrix, providing the token’s initial numerical representation. section 12.2.1
Training adjusts the selected embedding along with other model parameters through backpropagation. The resulting representations capture statistical relationships in the contexts where tokens appear, rather than storing dictionary definitions. section 12.2.1









