1 Introducing the Data Platform

 

This chapter covers

  • Drivers of change in the world of analytics data
  • The growth of data volume, variety and velocity, and why the traditional data warehouse can’t keep up
  • Why data lakes alone aren’t the answer
  • The emergence of the cloud data platform
  • The core building blocks of the cloud data platform
  • Sample use cases for cloud data platforms

1.1    The back story

Every business, whether they realize it or not, requires analytics. It’s a fact. There has always been a need to measure important business metrics and make decisions based on these measurements: Questions like “How many items did we sell last month?” and “What’s the fastest way to ship a package from A to B?” have evolved to “How many new website customers purchased a premium subscription?” and “What does my IoT data tell me about customer behavior?”

Before computers became ubiquitous we relied on ledgers, inventory lists, a healthy dose of intuition and other limited, manual means of tracking and analyzing business metrics. The late 1980s ushered in the concept of a data warehouse – a repository of data combined from multiple sources – which was typically used to produce static reports. Armed with this data warehouse, businesses were increasingly able to shift from intuition-based decision making to  an approach based on data.

1.2    Data warehouses struggle with data Variety, Volume and Velocity

1.2.1 Variety

1.2.2 Volume

1.2.3 Velocity

1.2.4 All the V’s at once

1.3    Data Lakes to the rescue?

1.4    Along came Cloud

1.5    Cloud, data lakes and data warehouses belong together - the emergence of cloud data platforms

1.6    Building blocks of a cloud data platform

1.6.1 Ingestion layer

1.6.2 Storage layer

1.6.3 Processing layer

1.6.4 Serving layer

1.7    How the Cloud Data Platform deals with the 3 V’s

1.7.1 Variety

1.7.2 Volume

1.7.3 Velocity

1.7.4 Two More V’s

1.8    Common Use Cases

sitemap