liveBook · Manning

Provenance

As explained in Data for All

Data lineage is the traceable history of a data item, including its origins, transformations, and uses in creating subsequent data.

Where did this data come from, what changed it, and where is it being used? Provenance—also called data lineage—answers those questions by tracing a data item back to the original data and recording the steps that created it. section 7.1

Data rarely stays in its original form. Raw activity can become aggregated, integrated, derived, synthetic, simulated, or optimization data. Each new asset remains connected to the data used to create it, even when analytics produces something substantially new. section 7.0 section 7.7

Tracking provenance makes data use more accountable:

  • Identify which people, organizations, or systems contributed to a blended dataset. section 7.1
  • Connect permissions, monetization, and compliance obligations to the data that supports a new asset. section 7.1
  • Audit where data has been used and track payments when the same source supports multiple applications. section 7.1
  • Manage and protect data as it moves beyond the activity that originally produced it. section 7.0

Purchase data becomes multiple analytical products

A retailer licenses a person’s purchase data to study personal-care buying, then blends it with household-cleaning purchase and use data. Provenance records both uses, the millions of contributing sources, and the resulting payment and audit obligations. section 7.1

Forecasts retain a connection to everyday activity

Store visits, economic conditions, crime, store expansion, hours, and population rates can produce forecasts for sales, staffing, stocking, shortages, and promotion effectiveness. Those derived streams retain lineage to visits and purchases. section 7.2.3

How Data for All covers it

How deep it goes
Defines lineage and explains its governance value, stopping short of technical implementation details beyond keeping creation and usage records.
Examples
Uses blended purchase data and store-visit forecasting to show how sources support multiple analytical applications and derived data streams.
What it assumes
Assumes only a basic understanding of data and introduces lineage through familiar activities, blended datasets, and forecasts.
What you can do after
Operate lineage records to trace sources, monitor data use, audit applications, and support payment decisions.
How it connects
Connects raw data, aggregated data, integrated data, derived data, synthetic data, analytics, permissions, monetization, and compliance.

Also explained in

Compare how 12 books explain provenance →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data for All. The section links point to the parts of the book it draws on.