In part 1, we’ll set up the core infrastructure of our data platform. All other topics discussed in the book (running various workloads, aspects of governance) will be built on top of this infrastructure.
- Chapter 2 discusses storage and various patterns of ingesting and storing data. We’ll look at two Azure services: Azure Data Explorer and Azure Data Lake Storage.
- Chapter 3 covers DevOps and introduces Azure DevOps, as this is a book about data engineering and bringing engineering rigor to data. We’ll see how we can store everything in Git and use automated pipelines for deployment.
- Chapter 4 describes orchestration: how data moves through our platform and how we can schedule various processes. For orchestration and data movement, we’ll use Azure Data Factory. For monitoring, we’ll use Azure Monitor.