liveBook · Manning

Concept drift

As explained in Deep Learning with R, Third Edition

Concept drift is a change over time in the data-generating patterns or relationships that a deployed machine-learning model depends on.

Concept drift occurs when production data changes over time, causing a deployed model’s accuracy to gradually decay. It is common in real-world systems, especially those using user-generated data. section 6.1.2

Changing reviews and fraud patterns

A sentiment model trained on IMDb reviews collected in 2011 may perform worse on reviews from 2020 because vocabulary, expressions, and movie genres evolve. Fraud detection can drift even faster because fraud patterns may change daily. section 6.1.2

Respond to fast-moving drift by:

  • Collecting current production data. section 6.1.2
  • Annotating new examples and retraining the model continually when necessary. section 6.1.2

How Deep Learning with R, Third Edition covers it

How deep it goes
Defines changing production properties, gives domain examples, and stops short of detailed detection methods.
Examples
Uses IMDb sentiment across years and rapidly changing credit-card fraud patterns as examples.
What it assumes
Assumes understanding that models learn patterns from training data and that future data may differ from past data.
What you can do after
Diagnose gradual accuracy decay and respond through current-data collection, annotation, and model retraining.
How it connects
Connects concept drift with production data, sampling bias, data collection, annotation, and model retraining.
Cover of Deep Learning with R, Third Edition
liveBookmanning.com
Read Deep Learning with R, Third Edition on liveBook Get the book at manning.com

Also explained in

Compare how 7 books explain concept drift →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Deep Learning with R, Third Edition. The section links point to the parts of the book it draws on.