liveBook · Manning

Pipeline

As explained in Data for All

A data pipeline is an automated or structured sequence that moves and processes data between sources and destinations.

A data pipeline is a structured flow that supplies data to models, analytics operations, and operational engines. Firms depend on sufficient data volume, variety, and velocity to model and simulate the real world. section 2.5.1

Pipelines are only as effective as the data entering them. Seasonal patterns and ordinary behavior changes can be incorporated, but abrupt disruption may require models to be retrained, recalibrated, or rebuilt. section 2.4.7

The pipeline’s dependence on ongoing data creates practical consequences:

  • A sudden change such as the COVID pandemic can make established predictions unreliable. section 2.4.7
  • Collectively withholding data can interrupt pipelines and alter how companies predict, message, and engage with people. section 2.4.7 section 2.5.1

When the data stops

Platforms that use data to predict behavior around social media, news, shopping, or buy-now-pay-later services cannot produce the same analytical results when the required data is no longer available. section 2.4.7 section 2.5.1

How Data for All covers it

How deep it goes
Explains dependence on data and disruption from external change, but stops short of implementation details.
Examples
Uses the COVID pandemic, social media, news, shopping, and buy-now-pay-later services as examples of pipeline dependence.
What you can do after
Evaluate how changing or withholding data can disrupt predictions and alter companies’ targeting and engagement.
How it connects
Connects data pipelines with models, analytics operations, prediction, and operational engines.

Also explained in

Compare how 48 books explain pipeline →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Data for All. The section links point to the parts of the book it draws on.