Vectorization maps an input, such as a query or document, to a numerical embedding. An embedding model produces vectors that place semantically similar inputs near one another, so vector comparisons can support search, ranking, and clustering. Unlike a generator, the model’s immediate output is a representation, not an answer. section 3.4
The representation matters as much as the comparison. Earlier methods such as Word2Vec and GloVe assign one vector to each word, leaving ambiguous words with the same representation in different contexts. Transformer-based models instead produce contextual token embeddings: “bank” can differ between a deposit and a riverbank. Embeddings can also represent sentences, documents, or images. section 3.4.1
In a retrieval-augmented generation (RAG) workflow:
- Embed documents and store their vectors in a vector database. section 3.4
- Embed an incoming query and compare it with stored vectors to retrieve relevant passages. section 3.4
- Optionally rerank the passages, then pass the selected context and original query to a language model. section 5.4.3
Compare queries with documents
With a pretrained SentenceTransformer, encode queries using `model.encode(queries, prompt_name='query')` and documents using `model.encode(documents)`. Then calculate `util.cos_sim(query_embeddings, document_embeddings)`. The illustrated two-query, two-document result is a 2 × 2 score matrix; each row shows how one query compares with both documents. section 3.4.1
Figure 5.2 Overview of a RAG system. In the preproduction phase, documents from a knowledge base are chunked and embedded into a vector database. During the production phase, a user query is embedded and used to retrieve top-matching documents based on similarity. An optional reranker refines these results to improve relevance. The final ranked context is then passed, together with the original query, to the language model for response generation.
The diagram separates document vectorization before deployment from query vectorization at retrieval time, and shows where optional reranking and answer generation follow. section 5.4.3
Retrieval quality depends on whether the embedding model captures the relationships the task needs. Model choice is only one design decision: document chunking, retrieval strategy, metadata, and reranking can also affect which context reaches the generator. section 5.4.3
