Negative sampling makes large-vocabulary training practical by replacing a full probability calculation with a small set of binary checks. For each observed context word, the model asks whether it is the true context or a randomly drawn negative example. section 2.1.2
The baseline problem is the output sum: a vocabulary of 692,000 words requires considering every word at each step. Hierarchical softmax reduces that work through a binary-tree walk, while negative sampling avoids requiring a probability distribution altogether. section 2.1.2
Negative sampling is useful when training must scale:
- It asks only a handful of yes-or-no questions instead of evaluating the entire vocabulary. section 2.1.2
- It helped Word2Vec train on datasets 100–1,000 times larger than earlier approaches, including more than 100 billion words in a day. section 2.1.5
- It supports dense-vector training for large vocabularies, which later enabled embeddings at production retrieval scale. section 2.1.5
A single training decision
Given an observed context word, retain it as the positive example and draw a handful of other words as negatives. Train the model to distinguish the true context from those random alternatives rather than scoring all 692,000 vocabulary entries. section 2.1.2



