A vector space gives semantic search a geometric foundation: queries and documents become vectors, and retrieval depends on how well their relationships are encoded. Poor representations create relevance gaps that later systems may mistake for generation failures. section 2.0
In retrieval systems, vector spaces support several linked design choices:
- Vector Space Models represent queries and documents as weighted vectors for relevance ranking. section 2.0
- Modern embedding models place text in high-dimensional spaces, such as 1,536 dimensions for text-embedding-3-small. section 10.3
- Vector databases and approximate-nearest-neighbor indexes trade speed, recall, and memory when searching these spaces. section 2.0
Diagnosing retrieval quality
Suppose a query retrieves irrelevant context. Inspect the representation and search layer separately: semantic relationships may be poorly encoded in the vector space, or the ANN index may be sacrificing recall for speed or memory efficiency. section 2.0
Table 2.1 Timeline of information retrieval
|
Era/Phase
|
Year
|
Key Concept
|
Seminal Paper/Event
|
|
Boolean / Keyword Search
|
Pre-1970s
|
Earliest methods of structured text querying using exact
match and set logic (AND/OR/NOT)
|
Based on logical methods
|
|
Inverted Index
|
1970s
|
Data structure that maps terms to documents, enabling
fast, large-scale full-text search
|
|
|
Vector Space Model (VSM) and TF–IDF
|
1975
|
Concept of relevance ranking by representing queries and
documents as weighted vectors
|
G. Salton, A. Wong, and C. S. Yang, "A Vector Space
Model for Automatic Indexing"
|
|
Okapi BM25
|
1994
|
Probabilistic model that refined ranking by introducing
document-length normalization and term saturation
|
S. E. Robertson et al., "Okapi at TREC-3"
|
|
Locality-Sensitive Hashing (LSH)
|
1998
|
Foundation for Approximate Nearest Neighbor (ANN) search
to efficiently find similarities in high-dimensional vector spaces
|
P. Indyk and R. Motwani, "Approximate Nearest
Neighbors: Towards Removing the Curse of Dimensionality"
|
|
Word2vec / Dense Word Embeddings
|
2013
|
Dense
vector representations of words learned from context, capturing semantic
relationships geometrically (e.g., King - Man + Woman ≈ Queen)
|
T. Mikolov et al., "Efficient Estimation of Word
Representations in Vector Space"
|
|
FAISS (Facebook AI Similarity Search)
|
2017
|
High-performance billion-scale implementations of ANN
algorithms (e.g., HNSW, PQ), making dense retrieval feasible in real-world
systems
|
H. Jégou, M. Douze, J. Johnson (Initial release, February
2017)
|
|
Transformers / BERT, Sentence-BERT (SBERT) / Bi-encoders
|
2018, 2019
|
Adapted Transformers to produce fixed-size sentence
embeddings suitable for fast nearest-neighbor search via ANN indices, solving
the latency problem of cross-encoders
|
J. Devlin et al., "BERT: Pre-training of Deep
Bidirectional Transformers..."
|
|
Retrieval-Augmented Generation (RAG)
|
2020
|
Integration of retrieval systems with large language
models (LLMs), using retrieved documents as an external memory source to
prevent hallucination and provide up-to-date context.
|
L. Lewis et al., "Retrieval-Augmented Generation for
Knowledge-Intensive NLP Tasks"
|
Use the timeline to connect vector spaces with the progression from TF–IDF ranking to dense representations and scalable approximate-nearest-neighbor search. section 2.0





