Embedding-based retrieval represents a query and each document as separate embeddings, then compares those vectors—often with cosine similarity—to produce a relevance score. This architecture is called a bi-encoder because it encodes the two inputs independently. section 13.8
A typical dense-retrieval pipeline separates matching into these stages:
- Encode queries and documents independently into vectors. section 13.8
- Compare the query vector with document vectors using a similarity function such as cosine similarity. section 13.8
- Use approximate nearest-neighbor search to filter toward similar vectors before expensive similarity calculations. section 13.9
- Combine retrieval optimizations such as ANN search, quantization, Matryoshka Representation Learning embeddings, over-requesting, and reranking. section 13.9
Why independent encoding can miss nuance
For the query “mountain hike,” a bi-encoder may retrieve “first time snow hiking” because both concern hiking. A cross-encoder reads the query and document together, allowing attention to distinguish beginner snow hiking from the more specific mountain-hike intent. In practice, bi-encoders support high-volume retrieval, while cross-encoders can rerank a smaller result set with more computation. section 13.8 section 13.9
Figure 13.14 Bi-encoders vs. cross-encoders. Bi-encoders process query and document inputs separately, whereas cross-encoders process them together into a similarity score.
Compare the two architectures visually: bi-encoders process query and document inputs separately, while cross-encoders process them together. section 13.8
