Continuous Bag of Words (CBOW) is a Word2Vec architecture that predicts a missing center word from multiple surrounding context words. It learns dense word embeddings from raw text through self-supervised prediction. section 2.1.2
CBOW’s architecture separates prediction from the representation it retains:
- Context-word embeddings are combined by summing them in a projection layer. section 2.1.2
- A vocabulary-wide output layer scores candidate center words. section 2.1.2
- After training, the prediction layer is discarded; the projection rows become the word embeddings. section 2.1.2
Fill-in-the-blank prediction
Given several surrounding words with one center position missing, CBOW combines the surrounding representations and predicts which vocabulary word belongs in the blank. section 2.1.2
Figure 2.1 CBOW vs skip-gram architectures.

Compare the CBOW and Skip-gram information flow in the architecture diagram. section 2.1.2

