Chapter 10. Ingestion through structured streaming
This chapter covers
- Understanding what is streaming.
- Building your first streaming ingestions.
- Apprehending the different sources of data in streaming.
- Building an application that takes two streams.
- Differentiating discretized streaming and structured streaming.
If you look at data from a few thousand meters (or feet if you are stuck with the imperial system), focus on the data-generation part. Do you see systems that generate batches of data or do you see systems that generate data continuously?
Systems delivering a flow of data, also known as streams, were less popular a few years ago. Streams are definitely getting more traction and understanding streams is the focus of this chapter.
Your mobile phone regularly pings cell towers, if it’s a smartphone (highly probable due to the audience of this book), it will also check email and more.
The bus travelling through (smart) cities are sending their GPS coordinates.
The cash register at your supermarket’s check-out counter generates data as the cashier (or you) pass the items in front of the scanner. A transaction is processed as you pay.
As you bring your car to the garage, a flow of information is collected, stored, sent to various other places like the manufacturers, insurance companies, or reporting companies.