Skip-gram is a Word2Vec architecture that learns dense word embeddings by predicting surrounding context words from a target word. It operationalizes the distributional hypothesis: words used in similar contexts tend to receive nearby representations. section 2.1.2
Its shared network follows a compact pipeline:
- Receives a vocabulary-wide one-hot vector for the target word. section 2.1.2
- Maps the target through a typically 100–300-dimensional projection with no activation function; one row per word becomes the learned embedding. section 2.1.2
- Scores every vocabulary word at the output layer for its likelihood of appearing near the target, then discards that prediction layer. section 2.1.2
Figure 2.1 CBOW vs skip-gram architectures.

Compare Skip-gram’s target-to-context direction with CBOW’s context-to-target direction. section 2.1.2

