In the previous chapter you explored some services provided by Microsoft Azure: Event Hubs, Stream Analytics, Data Lake, Data Lake Analytics, and SQL Database. You saw at a high level how these services can work together to create an analytics system.
This chapter begins showing you how to design and set up these services to lay the foundation of an analytics system. The first step is learning how to implement a storage system that is secure and scalable using Azure Storage accounts.
Durable storage serves both input and output for your analytics system. You can integrate your existing applications and third-party software with file-based transports. The comma-separated values (CSV) format has long been used to export data from applications. More recently, Extensible Markup Language (XML) and JavaScript Object Notation (JSON) provide greater structure for data interchange. All three formats are stored in plain text files. When delivered as exports from existing systems, these formats provide sources for data processing. After processing, they can hold the output of user queries. Figure 3.1 shows examples of durable storage as sources in the Batch layer, and outputs from the Speed and Serving layers.
