liveBook · Manning

Chapter 2: Revolutions in semantics, scale, and similarity

What you will learn in chapter 2 of Retrieval Augmented Generation, The Foundational Ideas

The chapter introduction explains how advances in semantic representation, scalable vector search, and sentence similarity converged to enable RAG, while identifying retrieval trade-offs that still matter.

After this chapter you will be able to

What it builds on

Section by section

Introduction

The chapter introduction explains how advances in semantic representation, scalable vector search, and sentence similarity converged to enable RAG, while identifying retrieval trade-offs that still matter.

Key terms: RAG, architectural pattern, Word2Vec, FAISS, Sentence-BERT, vector space, retrieval quality, ANN indexes, speed, recall

2.1 Word2Vec (2013): Semantic word embeddings

This section introduces the vocabulary mismatch in keyword search and explains why Word2Vec’s context-based word vectors and computational efficiency mattered for semantic retrieval.

Key terms: purely lexical matching, vocabulary mismatch, Word2Vec, dense vectors, context, semantic embeddings, computational efficiency, large-scale corpora, semantic search

2.1.1 From search engines to semantic understanding

The section explains why earlier text-retrieval methods struggled with synonyms and introduces the context-based idea behind Word2Vec.

Key terms: one-hot encoding, TF-IDF, BM25, keyword matching, Latent Semantic Analysis, LSA, term-document co-occurrence, matrix factorization, Word2Vec, distributional hypothesis

2.1.2 Word2Vec: From words to vectors

This section explains how Word2Vec learns word embeddings from prediction tasks, compares its two architectures, and shows how it makes training over large vocabularies practical.

Key terms: Word2Vec, dense embeddings, distributional hypothesis, geometric proximity, projection layer, embedding matrix, self-supervised learning, co-occurrence structure, Continuous Bag of Words, CBOW

2.1.3 Implementing Word2Vec: From theory to practice

A side-by-side search example shows how Word2Vec makes synonym-only documents visible to a ranking, while averaging word vectors still produces unreliable sentence rankings.

Key terms: BM25, lexical search, term overlap, vocabulary mismatch, word vectors, document vector, cosine similarity, word order, negation, Gensim

2.1.4 Business impact

This section shows how Word2Vec improved search across industries by matching meaning beyond exact keywords, and reports early benchmark evidence of its effectiveness.

Key terms: vocabulary mismatch, Word2Vec representations, semantic search, purely lexical approaches, analogy tasks, MSR Analogy, Google Analogy, WordSim-353

2.1.5 The foundation for dense retrieval

This section explains how Word2Vec made dense word embeddings practical at scale and why searching those embeddings remained a separate challenge.

Key terms: sparse vector, dense vector, hidden layers, Word2Vec, non-linear hidden layer, negative sampling, hierarchical softmax, embeddings, enterprise vocabularies, FAISS

2.1.6 Limitations and the path forward

This section explains Word2Vec’s limits and shows how query–document vocabulary overlap can guide the choice between keyword, semantic, and hybrid retrieval.

Key terms: Word2Vec, granularity limitations, context independence, computational scalability, linear models, distributional statistics, semantic similarity, retriever, vocabulary overlap, query tokens

2.2 FAISS (2017): Billion-scale similarity search

This section introduces the latency problem that arises when semantic vector retrieval reaches web scale and presents FAISS as a response.

Key terms: Word2Vec, retrieval latency, Approximate Nearest Neighbor (ANN), FAISS

2.2.1 Approximate nearest neighbors

This section explains why exhaustive vector search fails at large scale and how FAISS makes approximate similarity search practical through clustering, compression, and GPU acceleration.

Key terms: brute-force similarity search, dot product, Approximate Nearest Neighbor (ANN), vector, normalize, cosine similarity, antonyms, embeddings, semantic search, FAISS

2.2.2 The scale advantage

This section explains how FAISS makes vector search practical at large scale and how to weigh speed, recall, memory, and index-building costs.

Key terms: flat, IVF, HNSW, clusters, multi-layered graph, product quantization, PQ, IVF-PQ, index, brute-force search

2.2.3 The bridge to practical semantic search

This section explains how FAISS made semantic retrieval practical at RAG scale and identifies the search tradeoffs production systems must manage.

Key terms: FAISS, semantic search, similarity search, Approximate Nearest Neighbor, RAG, recall, latency, index choice, GPU acceleration, brute-force search

2.3 Sentence-BERT (2019): Practical sentence similarity

This section explains why BERT captures context but is too costly to use as a cross-encoder for large-scale similarity search, and why its outputs are not ready-made sentence embeddings.

Key terms: Word2Vec, BERT, Transformer, self-attention, cross-encoder, forward pass, similarity score, contextual token representations, [CLS], sentence embedding

2.3.1 From cross-encoders to bi-encoders

This section explains how Sentence-BERT turns BERT into a practical sentence-similarity model by encoding sentences independently and comparing their embeddings.

Key terms: Sentence-BERT, cross-encoder, bi-encoder, Siamese network, shared parameters, contrastive learning, embeddings, semantic similarity, pooling layer, token vectors

2.3.2 Implementation and business impact

This section shows how Sentence-BERT supports practical semantic search, where its benefits appear in applications, and why cheap encoding does not eliminate comparison costs.

Key terms: Word2Vec, Sentence-BERT, sentence embedding, pretrained Sentence-BERT, embeddings, dot product, vocabulary mismatch, pairwise similarity matrix, quadratic cost, batched approximation

2.3.3 Training, performance, and limitations

The section explains how training data shapes Sentence-BERT’s similarity performance and what limits its use in retrieval.

Key terms: Sentence-BERT, Natural Language Inference (NLI), Semantic Textual Similarity (STS), domain adaptation, context length, 512-token limit, domain specificity, forward pass, indexing, fine-tuning

2.3.4 The final piece of the RAG puzzle

The section explains how Sentence-BERT’s sentence embeddings and independent encoding made semantic retrieval fast and scalable enough for RAG.

Key terms: semantic coherence, paraphrase pairs, embedding quality, cross-domain transfer, multilingual capabilities, translation pairs, fine-tuning flexibility, labeled pairs, independent encoding, precomputation

2.4 Ready for revolution

The section identifies the missing link between finding relevant documents and turning them into direct answers.

Key terms: semantic search, retrieved documents, human bottleneck

2.4.1 The modern retrieval stack

The section presents modern enterprise retrieval as a scalable architectural pattern that combines complementary search methods and makes meaning-based search practical across large knowledge bases.

Key terms: architectural pattern, semantic encoding, similarity search, ranking and retrieval, scalable infrastructure, keyword, embedding-based retrieval, Python, vector database, embedding models

2.4.2 From retrieval to synthesis

The section explains the organizational value of semantic retrieval and the remaining gap between finding information and turning it into answers.

Key terms: embedding-based semantic search, keyword search, search latency, throughput, retrieval, generation, synthesis, training data, up-to-date or domain-specific information

2.5 Summary

Where this is used later

Concepts explained in this chapter

← Chapter 1: How RAG research prevents disasters Chapter 3: REALM: Birth of end-to-end trainable RAG →

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.