liveBook · Manning

Model collapse

As explained in Reinforcement Learning from Human Feedback

Model collapse is progressive degradation of a model's effective training distribution and generalization caused by repeated training on model-generated data.

Model collapse is progressive degradation of a model’s effective training distribution when repeated self-training narrows diversity. Rare facts and styles disappear, small errors amplify, and generalization worsens. section 12.1

Collapse risk is highest when pipelines rely on:

  • Unfiltered, repetitive outputs from a single model. section 12.1
  • Synthetic data without real or human data, diverse teachers, deduplication, or strong quality filters. section 12.1
  • Repeatedly replacing the effective training distribution with model-generated examples. section 12.1

A safer synthetic-data pipeline

Mix model-generated examples with real or human data, use diverse teachers, deduplicate records, and apply strong quality filters. These practices largely avoid the collapse regime while supporting synthetic-data use at scale. section 12.1

How Reinforcement Learning from Human Feedback covers it

How deep it goes
Explains distribution narrowing, lost diversity, error amplification, risk conditions, and mitigations without formal modeling.
Examples
Uses repetitive single-model outputs, real-data mixing, diverse teachers, deduplication, quality filters, and synthetic post-training datasets.
What you can do after
Diagnose collapse risk by inspecting output diversity and pipeline reliance on unfiltered, repetitive, single-model generations.
How it connects
Connects model collapse to synthetic data, self-training, generalization, deduplication, quality filtering, and post-training.
Cover of Reinforcement Learning from Human Feedback
liveBookmanning.com
Read Reinforcement Learning from Human Feedback on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Reinforcement Learning from Human Feedback. The section links point to the parts of the book it draws on.