LAMBADA
LAMBADA is a language-modeling benchmark that tests whether a model can predict a target word from its preceding passage context.
Rearchitecting LLMs
LAMBADA is a language-modeling benchmark that evaluates whether a model can predict a target word from a textual passage using its preceding context.
- How it is used
- Tracks model degradation and recovery alongside other benchmarks during pruning and distillation.
- How deep it goes
- Defines the exact last-word task and compares pruning results, without detailing dataset construction.
- Examples
- Reports a Gemma-3-270M baseline and Llama-3.2-3B attention-pruning scores.
- What you can do after
- Apply it via lm-eval to compare model benchmark scores before and after optimization.
Domain-Specific Small Language Models
A narrative text dataset used here to evaluate word-prediction and model accuracy after quantization.
- How it is used
- Evaluates language-model text understanding through word prediction and compares accuracy before and after quantization.
- How deep it goes
- Defines the passages, genre labels, and prediction task, but does not describe dataset construction.
- Examples
- Uses 1,000 validation samples to compare FP16, naive W8A8, and SmoothQuant W8A8 accuracy on OPT-2.7b.
- What you can do after
- Evaluate model accuracy on narrative word-prediction examples and compare results across quantization settings.
Read how Domain-Specific Small Language Models explains LAMBADA →
What to know first
- float16float16 is a 16-bit floating-point format, commonly IEEE 754 binary16, with less precision and range than 32-bit floating-point formats.
- embeddingAn embedding is a numerical vector representation of an item in a feature space, encoding selected properties or relationships for computational tasks such as comparison or retrieval.
Related concepts
- benchmarkkind of · A benchmark is a defined, repeatable task, workload, test, dataset, or reference standard used to measure or compare systems or methods under specified conditions.
Questions about LAMBADA
What is LAMBADA?
LAMBADA is a language-modeling benchmark that tests whether a model can predict a target word from its preceding passage context.
Which Manning books explain LAMBADA?
Rearchitecting LLMs and Domain-Specific Small Language Models.
What should I know before learning LAMBADA?
Manning books that teach LAMBADA usually introduce float16 and embedding first.
What is LAMBADA a kind of?
LAMBADA is a kind of benchmark. A benchmark is a defined, repeatable task, workload, test, dataset, or reference standard used to measure or compare systems or methods under specified conditions.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

