Catastrophic forgetting
Catastrophic forgetting is the substantial loss of previously learned information or capabilities when a model learns new data, tasks, or objectives.
Reinforcement Learning from Human Feedback
Catastrophic forgetting is the rapid loss of previously learned capabilities or information when a model is trained on new objectives or data.
- How it is used
- Frames sequential post-training as a retention problem when adapting models to new tasks.
- How deep it goes
- Derives forward- and reverse-KL objectives, then links on-policy sampling and measured policy drift to retention.
- Examples
- Uses old-versus-new policy modes and matched-performance RL-versus-SFT comparisons to visualize retention.
- What you can do after
- Diagnose/debug retention risk by measuring KL drift from a base policy on specialized inputs.
Read how Reinforcement Learning from Human Feedback explains catastrophic forgetting →
The Art of AI Product Development
Catastrophic forgetting is the loss of previously learned knowledge when a model is trained on new tasks or data.
- How it is used
- Uses evaluation on older tasks to decide whether fine-tuning has damaged retained knowledge.
- How deep it goes
- Defines the failure, identifies overlapping representations as a mechanism, and outlines evaluation and prevention decisions.
- Examples
- Uses acronym resolution, industry-term definition, benchmark regression, and medical, legal, and compliance applications.
- What you can do after
- Diagnose forgetting by comparing performance on previously learned tasks before and during new-task training.
Read how The Art of AI Product Development explains catastrophic forgetting →
Generative AI in Action
Catastrophic forgetting is the loss of previously learned knowledge or capabilities when a model is trained on new data or tasks.
- How it is used
- Frames model adaptation as a balance between task specialization and retaining pretrained knowledge and reasoning.
- How deep it goes
- Explains parameter overwriting and PEFT mitigation, while deferring detailed adaptation-technique trade-offs.
- Examples
- Uses sentiment analysis, spam detection, question answering, summarization, and translation as adaptation examples.
- What you can do after
- Apply it via PEFT by updating a small parameter subset while keeping most pretrained parameters fixed.
Read how Generative AI in Action explains catastrophic forgetting →
Related concepts
- parameter-efficient fine-tuningaddressed by · Parameter-efficient fine-tuning adapts a pretrained model by training a limited subset of its parameters or added components rather than all its parameters.
- low-rank adaptationaddressed by · Low-rank adaptation is a parameter-efficient fine-tuning method that adapts a model by learning low-rank updates to its weights, typically while keeping most original weights fixed.
- Kullback–Leibler divergencemeasured by · Kullback–Leibler divergence is a nonnegative, directed measure of the information lost when one probability distribution approximates another, computed as an expected log ratio under the first distribution.
- learning rateaddressed by · A learning rate is a step-size parameter that controls the magnitude of updates made by an iterative learning or optimization process.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.