Data drift

As explained in Microsoft Azure in Action

Data drift is a change over time in the distribution or characteristics of input data relative to the data used to train a model.

Data drift occurs when incoming request features shift away from a model’s training distribution. Monitoring can alert a team to changes such as review length or sentiment distribution. section 7.4.5

Figure 7.11 The Monitoring dashboard in AI Foundry when your model is first created

The AI Foundry Monitoring dashboard shows endpoint metrics and supports ongoing monitoring for data drift. section 7.4.5

How Microsoft Azure in Action covers it

How deep it goes

Tracks feature changes with examples, but does not specify detection thresholds or remediation steps.

Examples

Uses changes in review length and sentiment distribution as examples of shifting incoming features.

What you can do after

Operate/configure it by setting alerts for shifts in incoming request features.

How it connects

Links data drift to model monitoring, training distributions, and model performance.

liveBookmanning.com

Read Microsoft Azure in Action on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Machine Learning System DesignA discrepancy between the statistical characteristics of data used for model development and those of data encountered in the target operating environment.
• 

  liveBookmanning.com
  Machine Learning Platform EngineeringDrift detection is the identification of changes in data distributions or related model-serving conditions over time.
• 

  liveBookmanning.com
  The Art of AI Product DevelopmentDomain shift is a change between the data distribution encountered during model operation and the distribution represented by its training data.
• 

  liveBookmanning.com
  Reinforcement Learning from Human FeedbackDistribution mismatch is a discrepancy between the probability distribution that generated training data and the distribution of the policy being optimized.
• 

  liveBookmanning.com
  Starting Data Analytics with Generative AI and PythonDomain mismatch is the discrepancy between the data domain used to develop a model and the domain of the data to which it is applied.

Compare how 6 books explain data drift →

Related concepts

• Evidently AIused by · Evidently AI is an open-source Python tool for evaluating, testing, monitoring, and reporting on machine-learning data, models, predictions, and behavior.
• degradationproduces · Degradation is a decline in a system’s or model’s performance, quality, or effectiveness relative to a previous or expected level.
• regressioncontrasts with · A regression is an unintended deterioration in a system’s behavior, performance, or capabilities relative to a prior or expected level, often following a change.
• population stability indexmeasured by · The population stability index (PSI) is a measure of the difference between two probability distributions, often used to quantify shifts in a population or model inputs.
• monitoringmeasured by · Monitoring is the ongoing observation and measurement of a system’s behavior, state, or operational signals to assess its condition and detect changes or problems.

This explainer was generated by Manning's knowledge graph from the text of Microsoft Azure in Action. The section links point to the parts of the book it draws on.