liveBook · Manning

Generalization

As explained in Natural Language Processing in Action, Second Edition

Generalization is a model's ability to produce useful predictions for previously unseen inputs, including inputs outside its training examples.

Generalization is a model’s ability to reuse patterns learned from training data when making useful predictions for previously unseen inputs. Inputs outside the training and test sets are out of distribution; extrapolating to them is generalization. section 5.2.4 section 10.1

For names, character n-grams let a classifier use subword patterns rather than memorizing complete names. A TfidfVectorizer can fit on training names and transform unseen names into sparse feature vectors. section 5.2.4

Generalization is not guaranteed intelligence

An LLM may reuse language patterns convincingly without modeling the physical world. Prompting GPT-2 with “There are 2 cows and 2 bulls, how many legs are there?” produces varied answers, including “four,” “30,” and “one,” exposing brittle generalization and hallucination. section 10.1

How Natural Language Processing in Action, Second Edition covers it

How deep it goes
Explains out-of-distribution behavior, demonstrates character n-gram features, and tests LLM brittleness, but stops short of formal generalization theory.
Examples
Uses original names such as Carlana and Cason, character n-gram vectors, and GPT-2’s cow-leg question.
What it assumes
Assumes familiarity with training and test sets, while defining out-of-distribution inputs and motivating subword features through names.
What you can do after
Apply it via character n-gram vectorization and evaluate LLM outputs for reliable transfer beyond observed language patterns.
How it connects
Connects generalization with out-of-distribution examples, NLP, NLU, tokenization, TF-IDF, transformers, hallucination, and large language models.
Cover of Natural Language Processing in Action, Second Edition
liveBookmanning.com
Read Natural Language Processing in Action, Second Edition on liveBook Get the book at manning.com

Also explained in

Compare how 13 books explain generalization →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Natural Language Processing in Action, Second Edition. The section links point to the parts of the book it draws on.