An embedding is a dense numerical vector that represents text—such as a token, sentence, document, or chunk—so its semantic and syntactic meaning can be processed computationally. Nearby vectors represent contextually related items, making embeddings useful when meaning matters more than exact word overlap. section 2.2 section 4.2
Figure 2.3 illustrates the vector representation of words. Although the numerical values may not be immediately interpretable, the visualization effectively showcases the proximity of words in their numeric representation, highlighting their close association based on contextual meaning.
Vector proximity visualizes the central intuition: numerical coordinates are not directly readable, but closeness can reflect contextual association. section 2.2
An encoder creates embeddings by transforming input text into vectors. For sentence-level retrieval, a Sentence Transformer produces one fixed-size vector per sentence, allowing vectors to be stored, indexed, and compared independently. SBERT uses siamese networks and triplet loss during training to emphasize semantic relationships among sentences. section 2.3.1 section 4.2
A semantic retrieval workflow uses embeddings as follows:
- Choose the text unit to represent, such as hotel reviews or semantically coherent document chunks. section 4.1 section 6.4
- Encode each unit with a selected encoder model, producing dense vectors that capture semantic meaning. section 4.2.1 section 6.4
- Encode the incoming query with the same model and compare it with stored vectors using cosine similarity or Euclidean distance. section 3.5 section 4.6
- Index vectors for fast retrieval; bi-encoders permit precomputation and storage, while FAISS or Qdrant can search vectors at scale. section 4.6 section 6.4
Hotel review retrieval
Represent each Paris hotel review as an embedding, then represent a query such as “Hotel with a view of the Eiffel Tower” in the same space. A similarity search ranks reviews whose vectors are closest to the query, returning Eiffel Tower-view reviews even when phrasing differs. Retrieved reviews can then supply context to a generation model. section 4.0 section 6.1
Embedding design is a systems trade-off. Open-source models can run locally or in cloud infrastructure with hardware under your control; API models shift computation to a provider but incur usage costs and reduce control. Higher-dimensional models may improve results at a fee. Long inputs also require care: a fixed-size vector must compress increasingly diverse content, while longer context sharply increases attention computation. section 4.2 section 4.2.1 section 6.3.2







































































