liveBook · Manning

Embedding

As explained in Applied Reinforcement Learning

An embedding is a dense vector representation of an item, such as a token, learned by a model.

An embedding is a dense, learned vector representation of a token. It gives the model more than an integer ID: tokens appearing in similar contexts tend to develop related vectors, located near one another in a high-dimensional space. section 10.2

From token to vector

After “aspirin” is mapped to its vocabulary ID, the model looks up its learned vector. That vector joins the representations of the other tokens; position information is then combined with token embeddings before the sequence enters the transformer. section 10.2

How Applied Reinforcement Learning covers it

How deep it goes
Defines learned high-dimensional token vectors and contextual similarity, but does not detail how their training works.
Examples
Uses “aspirin” and related medical terms to illustrate token vectors and contextual similarity.
What you can do after
Understand how token IDs become learned vectors that carry token and positional information into a transformer.
How it connects
Connects tokenization and token IDs to positional representation and transformer processing.

Also explained in

Compare how 72 books explain embedding →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.