The distributional hypothesis states that words appearing in similar contexts tend to share semantic properties. It replaces purely lexical matching with context-based similarity: “doctor” and “physician” become related because both occur near terms such as “patient,” “hospital,” and “treatment.” section 2.1.1 section 2.1.2
The hypothesis changes how retrieval and representation systems handle meaning:
- Keyword baselines match exact terms and may treat “automobile” and “car” as unrelated unless synonym lists are added. section 2.1.1
- Word2Vec operationalizes the principle by learning dense vectors in which words used in similar contexts appear near one another. section 2.1.2
- CBOW predicts a target word from its context, while Skip-gram predicts context words from a target word; both learn from raw text. section 2.1.2
- The resulting embeddings support semantic matching in neural information retrieval, including dense passage retrieval systems used in modern RAG. section 2.1.1
Semantic neighbors and analogies
A learned embedding can place “doctor” near “physician” because their surrounding contexts overlap. It can also express an analogy such as vector(“king”) − vector(“man”) + vector(“woman”) ≈ vector(“queen”), although the nearest-neighbor result depends on how competing words are handled. section 2.1.1 section 2.1.2
Figure 2.1 CBOW vs skip-gram architectures.

Compare CBOW and Skip-gram as alternative ways to learn distributional representations from neighboring words. section 2.1.2


