liveBook · Manning

Normalization error

As explained in Delivering Machine Learning Projects: From design to deployment

A normalization error is a mistake caused by inconsistent scales, units, representations, or aggregation conventions across data sources.

A normalization error arises when data sources use inconsistent scales, sampling qualities, or aggregation conventions, distorting the distribution of a fused dataset and any models derived from it. section 5.4.1

Before combining sources, check whether:

  • Entities were collected on the same basis, through principled selection or comparable observation. section 5.4.1
  • Hidden variables, such as offender identity or collection time, are not creating unobserved distortion. section 5.4.1
  • Sensors have not changed quantitatively or qualitatively during the collection period. section 5.4.1

Mismatched measurements

Combining agreement scores measured from 1–5 with scores measured from 1–10, or area-average temperatures with point samples, can create a distorted dataset unless the difference is deliberately assessed. section 5.4.1

How Delivering Machine Learning Projects: From design to deployment covers it

How deep it goes
Checks collection basis, hidden variables, sensor changes, sampling frequency, measurement scales, and aggregation differences.
Examples
Uses sensor frequencies, 1–5 versus 1–10 survey scales, and average versus sampled temperature measurements.
What it assumes
Assumes domain understanding sufficient to interpret source semantics, collection processes, and potential distribution distortions.
What you can do after
Diagnose whether fused data reflects reality and deliberately justify using sources that violate strict consistency conditions.
How it connects
Connects data fusion with sampling, measurement scales, aggregation, hidden variables, sensor data, and model validity.

This explainer was generated by Manning's knowledge graph from the text of Delivering Machine Learning Projects: From design to deployment. The section links point to the parts of the book it draws on.