liveBook · Manning

N-gram model

An n-gram model is a probabilistic language model that estimates sequence probabilities using contiguous sequences of a fixed number of tokens, typically to predict a token from preceding context.

How 2 Manning books explain it

Machines that Think

A language model that estimates the probability of a token from a fixed-length sequence of preceding tokens.

How it is used
Introduces statistical language modeling through local frequency-based prediction and its applications in language technology.
How deep it goes
Explains conditional-frequency estimates, sparsity, smoothing, and context-size trade-offs without implementing a model.
Examples
Calculates the probability of “east” after “in the” from corpus counts and uses an unseen phrase to illustrate sparsity.
What you can do after
Evaluates how context length and sparse data affect the reliability and feasibility of n-gram predictions.

Read how Machines that Think explains n-gram model →

LLMs in Production

An n-gram model is a probabilistic language model that estimates word sequences using a fixed-length context of n tokens.

How it is used
Provides an efficient generative baseline for modeling local word sequences and comparing more capable language models.
How deep it goes
Explains training, generation, scoring, and evaluation, but stops short of modeling syntax or broader linguistic knowledge.
Examples
Uses a Hamlet corpus, padded trigrams, generated continuations, token counts, probability scores, entropy, and perplexity.
What you can do after
Build it from scratch with Python and NLTK, generate continuations, inspect probabilities, and evaluate phrases with entropy or perplexity.

Read how LLMs in Production explains n-gram model →

Also covered in

What to know first

Related concepts

Questions about n-gram model

What is n-gram model?

An n-gram model is a probabilistic language model that estimates sequence probabilities using contiguous sequences of a fixed number of tokens, typically to predict a token from preceding context.

Which Manning books explain n-gram model?

Machines that Think and LLMs in Production.

What should I know before learning n-gram model?

Manning books that teach n-gram model usually introduce language model, perplexity, speech recognition and sequence first.

What is the difference between n-gram model and recurrent neural network?

An n-gram model is a probabilistic language model that estimates sequence probabilities using contiguous sequences of a fixed number of tokens, typically to predict a token from preceding context. A recurrent neural network is a neural network that processes ordered inputs while carrying and updating an internal state across successive steps.

What kinds of n-gram model are there?

Kinds of n-gram model include trigram model.

What is n-gram model a kind of?

N-gram model is a kind of model. A model is an abstract representation of a system, phenomenon, application, or domain that captures relevant structure, behavior, data, or beliefs.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.