N-gram
An n-gram is a contiguous sequence of n consecutive items, such as words, tokens, or characters, drawn from a larger sequence.
Natural Language Processing in Action, Second Edition
An n-gram is a contiguous sequence of up to n extracted elements, such as words, characters, syllables, or symbols.
- How it is used
- Uses n-grams to retain local context in vectorized text and represent character patterns for search and modeling.
- How deep it goes
- Defines n-gram elements and lengths, then configures vectorizers and examines count features and vocabulary growth.
- Examples
- Uses “ice cream,” an algorithmic-bias search query, an encoded message, and character features of names.
- What you can do after
- Apply n-grams via scikit-learn vectorizers to construct word or character features and assess their usefulness for a task.
Read how Natural Language Processing in Action, Second Edition explains n-gram →
LLM Evaluation and Alignment, The Foundational Ideas
An n-gram is a contiguous sequence of n tokens, words, or characters extracted from text.
- How it is used
- Uses n-gram overlap to evaluate generated text and investigate training–evaluation data contamination.
- How deep it goes
- Defines n-gram orders and works through recall calculations, while describing contamination checks as heuristics rather than implementations.
- Examples
- Calculates unigram and bigram recall for candidate and reference summaries and cites 13-gram filtering in GPT-3 benchmark evaluation.
- What you can do after
- Evaluate overlap scores in context, distinguishing shared words from shared phrases and avoiding contamination claims based on high scores alone.
Read how LLM Evaluation and Alignment, The Foundational Ideas explains n-gram →
LLMs in Production
An n-gram is a contiguous sequence of n linguistic units, commonly used to model local patterns in text.
- How it is used
- Uses overlapping word sequences to evaluate whether generated summaries resemble reference summaries without demanding exact matches.
- How deep it goes
- Defines overlap-based scoring and distinguishes n-size variants, longest common subsequences, and skip-bigrams.
- Examples
- Uses paired Zelda summaries and Python calculations of ROUGE-1 and ROUGE-L scores.
- What you can do after
- Evaluate summary similarity with the Python rouge-score library using overlap-based metrics.
Ensemble Methods for Machine Learning
An n-gram is a contiguous sequence of n successive tokens, such as characters or words, extracted from a string.
- How it is used
- Uses n-grams to compare longer string categories and encode high-cardinality or noisy categorical features.
- How deep it goes
- Defines n-grams and places them in token-level similarity, without detailing extraction or comparison algorithms.
- Examples
- Uses employee-position titles, substring patterns, MinHashEncoder, and GapEncoder as practical examples.
- What you can do after
- Apply n-gram-based similarity through specialized encoders for noisy or high-cardinality string categories.
Read how Ensemble Methods for Machine Learning explains n-gram →
Elasticsearch in Action, Second Edition
An n-gram is a contiguous sequence of a specified number of characters or words derived from text.
- How it is used
- Supports typeahead suggestions, spelling correction, and matching broken words through overlapping character or word sequences.
- How deep it goes
- Explains character windows, edge anchoring, word shingles, tokenizer configuration, and analysis output without covering broader relevance tuning.
- Examples
- Uses “coffee” and “Bond” for character tokens, plus book titles for word shingles and typeahead search.
- What you can do after
- Apply it via Elasticsearch tokenizers and search_as_you_type mappings to inspect tokens and implement prefix-oriented search.
Read how Elasticsearch in Action, Second Edition explains n-gram →
Machines that Think
A contiguous sequence of n tokens considered as a unit in language analysis.
- How it is used
- Estimates the likelihood of a next word from the frequency of preceding word sequences in a corpus.
- How deep it goes
- Derives conditional estimates from counts and explains sparsity and backoff, without detailing smoothing techniques.
- Examples
- Uses the frequency of “in the east” relative to “in the” to illustrate a 5% next-word estimate.
- What you can do after
- Understand how corpus counts yield conditional next-word estimates and why longer contexts can make those estimates sparse.
Also covered in
- Deep Learning with R, Third Edition
- Building LLM Applications with DSPy
- Python Concurrency with asyncio
- Learn AI-Assisted Python Programming, Second Edition
- Starting Data Analytics with Generative AI and Python
- Data Without Labels
What to know first
- accuracyAccuracy is the degree to which a result agrees with a specified truth, reference, or criterion, commonly measured as the proportion of evaluated predictions that are correct.
- bag-of-words modelA bag-of-words model represents text using word or token occurrences, counts, or weights while disregarding their order and syntactic relationships.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
- linear regressionLinear regression is a statistical method that models a numeric response as a linear function of one or more predictor variables with fitted parameters.
Related concepts
- bigramincludes · A bigram is an ordered sequence of two consecutive items, such as tokens, words, or symbols, treated as a unit.
- sequence modelkind of · A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations.
- BLEU scoreused by · BLEU is an automatic text-generation evaluation metric that compares candidate text with reference text using modified n-gram precision and a brevity penalty.
Questions about n-gram
What is n-gram?
An n-gram is a contiguous sequence of n consecutive items, such as words, tokens, or characters, drawn from a larger sequence.
Which Manning books explain n-gram?
Natural Language Processing in Action, Second Edition, LLM Evaluation and Alignment, The Foundational Ideas, LLMs in Production, Ensemble Methods for Machine Learning, Elasticsearch in Action, Second Edition and Machines that Think.
What should I know before learning n-gram?
Manning books that teach n-gram usually introduce accuracy, bag-of-words model, language model and linear regression first.
What kinds of n-gram are there?
Kinds of n-gram include bigram, unigram, higher-order n-gram and trigram.
What is n-gram a kind of?
N-gram is a kind of sequence model. A sequence model processes ordered data while accounting for dependencies among its positions to produce predictions or representations.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.





