Skip-gram
Skip-gram is a word-embedding training method that learns representations by predicting context items from a target item.
Natural Language Processing in Action, Second Edition
Skip-gram is a word-embedding training approach that predicts surrounding or skipped words from a context word.
- How it is used
- Trains word embeddings from raw domain text by converting local word relationships into supervised prediction pairs.
- How deep it goes
- Explains prediction direction, window construction, vocabulary encoding, model layers, batching, and device selection, but does not detail optimization theory.
- Examples
- Uses WikiText2, lowercased token sequences, a ten-token window, vocabulary IDs, and 100-dimensional Word2Vec vectors.
- What you can do after
- Builds skip-gram embeddings from scratch with Python and PyTorch using tokenization, windowing, batching, and a Word2Vec model.
Read how Natural Language Processing in Action, Second Edition explains skip-gram →
Retrieval Augmented Generation, The Foundational Ideas
Skip-gram is a Word2Vec architecture that predicts surrounding context words from a target word.
- How it is used
- Learns dense word representations from raw text by predicting nearby context words.
- How deep it goes
- Explains the three-layer architecture and discarded output, but does not detail optimization or training accelerations.
- Examples
- Uses CBOW-versus-Skip-gram architecture diagrams and learned vector relationships as concrete illustrations.
- What you can do after
- Understand how Skip-gram converts context-prediction training into reusable word embeddings.
Read how Retrieval Augmented Generation, The Foundational Ideas explains skip-gram →
Also covered in
- Graph Algorithms for Data Science
- Data Without Labels
- Machines that ThinkIn early access (MEAP)
What to know first
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
Related concepts
- Continuous Bag of Wordscontrasts with · Continuous Bag of Words is a Word2Vec training architecture that predicts a target word from its surrounding context words.
- Word2Vecused by · Word2Vec is a family of shallow neural models that learns vector representations of words from their contexts in text.
- Node2Vecused by · Node2Vec is a graph representation-learning algorithm that learns node embeddings from biased random walks and their context relationships.
- skip-bigrampart of · A skip-bigram is an ordered pair of words separated by zero or more intervening words in a sequence.
Questions about skip-gram
What is skip-gram?
Skip-gram is a word-embedding training method that learns representations by predicting context items from a target item.
Which Manning books explain skip-gram?
Natural Language Processing in Action, Second Edition and Retrieval Augmented Generation, The Foundational Ideas.
What should I know before learning skip-gram?
Manning books that teach skip-gram usually introduce accuracy first.
What is the difference between skip-gram and Continuous Bag of Words?
Skip-gram is a word-embedding training method that learns representations by predicting context items from a target item. Continuous Bag of Words is a Word2Vec training architecture that predicts a target word from its surrounding context words.
What is skip-gram part of?
Skip-gram is part of skip-bigram. A skip-bigram is an ordered pair of words separated by zero or more intervening words in a sequence.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

