chapter two

2 Building an analytics system in Azure

 

This chapter covers:

  • Introducing the six Azure services discussed in this book
  • Joining the services into a working analytics system
  • Calculating fixed and variable costs of these services
  • Applying Microsoft big data architecture best practices

Cloud providers offer a wide selection of services to build a data warehouse and analytics system. Some services are familiar incarnations of on-premises applications: virtual machines, firewalls, file storage, and databases. Increasing in abstraction are services like web hosting, search, queues, and application containerization services. At the highest levels of abstraction are products and services that have no analogue in a typical data center. For example, Azure Functions executes user code without setup of underlying servers, runtimes, or program containers. Moving workloads to more abstract services reduces or eliminates setup and maintenance work, and brings higher guaranteed service levels. Conversely, more abstract services remove access to many configuration settings, and constrain usage scenarios. This chapter introduces the Azure services we’ll use to build our analytics system. These services are abstract to very abstract implementations, which allows you to focus on functionality immediately without needing to spend time on the underlying support systems.

2.1  Fundamentals of Azure architecture

2.1.1  Azure subscriptions

2.1.2  Azure regions

2.1.3  Azure naming conventions

2.1.4  Resource groups

2.1.5  Finding resources

2.2  Lambda architecture

2.3  Azure cloud services

2.3.1  Azure analytics system architecture

2.3.2  Event Hubs

2.3.3  Stream Analytics

2.3.4  Data Lake Store

2.3.5  Data Lake Analytics

2.3.6  SQL Database

2.3.7  Data Factory

2.3.8  Azure PowerShell

2.4  Walk-through of processing a series of event data records

2.4.1  Hot Path

2.4.2  Cold Path

2.4.3  Choosing abstract Azure services