An employee searches for “automobile maintenance,” but the relevant documents say “car repair” or “vehicle service.” Keyword matching can miss them because the words differ. Word2Vec addresses this vocabulary mismatch by learning dense vectors for words from the contexts in which they appear: words used in similar contexts tend to occupy nearby positions in vector space. section 2.1 section 2.1.2
Introduced in 2013, Word2Vec uses a single network to learn representations across a vocabulary, with two training architectures: CBOW and Skip-gram. Its efficient design drops the nonlinear hidden layer used by earlier neural approaches and employs approximations such as negative sampling and hierarchical softmax. This made training on very large corpora practical; it does not make searching millions of resulting vectors fast by itself. section 2.1.2 section 2.1.5
Choose Word2Vec for the problem it solves, not as a complete retrieval system:
- Compared with one-hot encoding, TF-IDF, and BM25, dense word vectors can express similarity between terms that never match exactly. BM25 remains useful when exact terms matter. section 2.1.1 section 2.1.6
- Each word has one static vector. “Bank” cannot receive separate finance and river representations, and phrases or documents need an additional representation method. section 2.1.6
- Embedding quality depends on the training corpus and can inherit its gaps or biases. Large, specialized vocabularies also increase storage and similarity-search demands. section 2.1.2 section 2.1.6
Search ranking: a useful gain with a clear ceiling
For “automobile maintenance,” a BM25 example scores “I bought a new car today” at zero, while a vector-based comparison makes it visible to ranking. But averaging word vectors discards word order, negation, and sentence structure; an irrelevant computer-repair document still ranks just above that car document. The demonstration loads pretrained GloVe vectors, not a trained Word2Vec model: it illustrates the value and limits of word-level semantic vectors, rather than measuring Word2Vec itself. section 2.1.3
--- BM25 (Keyword Search) Results --- Doc 1: Score 1.17 | Text: 'The automobile industry is booming' Doc 2: Score 0.36 | Text: 'Vehicle maintenance can prevent breakdowns' Doc 4: Score 0.33 | Text: 'Automobiles and cars require regular maintenance' Doc 0: Score 0.00 | Text: 'I bought a new car today' Doc 3: Score 0.00 | Text: 'She repaired the computer after the crash' --- Semantic Search Results --- Doc 4: Score 0.79 | Text: 'Automobiles and cars require regular maintenance' Doc 2: Score 0.75 | Text: 'Vehicle maintenance can prevent breakdowns' Doc 1: Score 0.69 | Text: 'The automobile industry is booming' Doc 3: Score 0.55 | Text: 'She repaired the computer after the crash' Doc 0: Score 0.54 | Text: 'I bought a new car today'
Compare the two result lists to see where semantic scoring recovers a nonmatching term and where its ranking remains unreliable. section 2.1.3
Before adding dense retrieval, sample representative queries and measure how often their tokens occur in the document vocabulary. High overlap may favor a simpler keyword baseline. When mismatch is substantial, word vectors offer a starting point; sentence embeddings address longer units of meaning, while scalable vector search addresses lookup speed. section 2.1.6 section 2.5



