liveBook · Manning

LAMBADA

LAMBADA is a language-modeling benchmark that tests whether a model can predict a target word from its preceding passage context.

How 2 Manning books explain it

Rearchitecting LLMs

LAMBADA is a language-modeling benchmark that evaluates whether a model can predict a target word from a textual passage using its preceding context.

How it is used
Tracks model degradation and recovery alongside other benchmarks during pruning and distillation.
How deep it goes
Defines the exact last-word task and compares pruning results, without detailing dataset construction.
Examples
Reports a Gemma-3-270M baseline and Llama-3.2-3B attention-pruning scores.
What you can do after
Apply it via lm-eval to compare model benchmark scores before and after optimization.

Read how Rearchitecting LLMs explains LAMBADA →

Domain-Specific Small Language Models

A narrative text dataset used here to evaluate word-prediction and model accuracy after quantization.

How it is used
Evaluates language-model text understanding through word prediction and compares accuracy before and after quantization.
How deep it goes
Defines the passages, genre labels, and prediction task, but does not describe dataset construction.
Examples
Uses 1,000 validation samples to compare FP16, naive W8A8, and SmoothQuant W8A8 accuracy on OPT-2.7b.
What you can do after
Evaluate model accuracy on narrative word-prediction examples and compare results across quantization settings.

Read how Domain-Specific Small Language Models explains LAMBADA →

What to know first

Related concepts

Questions about LAMBADA

What is LAMBADA?

LAMBADA is a language-modeling benchmark that tests whether a model can predict a target word from its preceding passage context.

Which Manning books explain LAMBADA?

Rearchitecting LLMs and Domain-Specific Small Language Models.

What should I know before learning LAMBADA?

Manning books that teach LAMBADA usually introduce float16 and embedding first.

What is LAMBADA a kind of?

LAMBADA is a kind of benchmark. A benchmark is a defined, repeatable task, workload, test, dataset, or reference standard used to measure or compare systems or methods under specified conditions.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.