Memorization

As explained in The AI Pocket Book

A behavior where a model stores frequent patterns or solutions from training data rather than learning a general rule.

Memorization is when a model stores frequent patterns or ready-made solutions instead of learning a rule that generalizes. It can produce familiar answers impressively, then fail on an unusual version of the same problem. section 2.8

When the shortcut breaks

A model may recognize cows only when grass appears beneath them. It can perform well on ordinary training-like images but fail to identify a cow on a beach. Similarly, a language model may memorize a common solution and fail when a familiar problem is presented in an uncommon form. section 2.8

Memorization is closely tied to overfitting. In language models, it can also mean reproducing training data nearly verbatim, creating copyright concerns. Techniques that reduce overfitting may mitigate memorization, but eliminating it completely is difficult to guarantee. section 6.1

The practical warning signs are straightforward:

• Typical examples can hide the problem because evaluation resembles the data the model has already seen. section 2.8
• Unusual contexts can expose the missing general rule, as with a cow on a beach or traffic cones placed between lanes. section 2.8
• Nearly verbatim output can reveal that the model retained training text rather than only learning patterns from it. section 6.1

Figure 6.2 Example of GPT-4 output (almost) verbatim text as published by The New York Times. Figure reproduced from The New York Times Company v. Microsoft Corporation, 1:23-cv-11195, (S.D.N.Y.).

Compare the generated passage with the published wording to see what near-verbatim memorization looks like in practice. section 6.1

How The AI Pocket Book covers it

How deep it goes

Defines memorization through examples and links it to overfitting, but stops short of explaining model-level mitigation methods.

Examples

Uses near-verbatim GPT-4 text, cow recognition, boat-crossing variants, and traffic-cone placement to expose memorized shortcuts.

What it assumes

Assumes basic familiarity with model training and evaluation while introducing overfitting, world models, and generalization through examples.

What you can do after

Diagnose memorization by testing uncommon variants and checking whether outputs reproduce training examples nearly verbatim.

How it connects

Connects memorization with overfitting, hallucinations, world models, generalization, and copyright infringement.

liveBookmanning.com

Read The AI Pocket Book on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Introduction to Generative AI, Second EditionModel memorization is the retention of specific training examples or sensitive information in a trained model's parameters or behavior.
• 

  liveBookmanning.com
  Evolutionary Deep Learning: Genetic algorithms and neural networksModel memorization is the phenomenon in which a model fits particular training examples without learning patterns that transfer to new data.
• 

  liveBookmanning.com
  Sutskever's ListMemorization is a model's retention of specific training examples or their details rather than learning patterns that generalize.

Compare how 4 books explain memorization →

Related concepts

• generalizationcontrasts with · Generalization is the extension of a pattern, model, solution, or conclusion beyond the specific examples or cases from which it was derived.
• training datasetproduced by · A collection of examples or observations supplied to a learning procedure to fit or adapt a model or learn its patterns or parameters.
• dropoutaddressed by · Dropout is a neural-network regularization technique that randomly suppresses units, connections, or activations during training to reduce overfitting.
• fine-tuningproduced by · Fine-tuning is adapting a pretrained machine-learning model by training it further on targeted data or objectives for a task, domain, or behavior.
• trainingcontrasts with · Training is a process of developing or improving a person’s or model’s capabilities through instruction, experience, or adjustment using data.

This explainer was generated by Manning's knowledge graph from the text of The AI Pocket Book. The section links point to the parts of the book it draws on.