1 Welcome to Grokking Streaming Systems

“If it weren’t for the rocks in its bed, the stream would have no song.”

—Carl Perkins

In this chapter, we will try to answer a few basic questions about streaming systems, starting with “what is stream processing?” and “what are these stream processing systems, or streaming systems, used for?” The objective is to cover some basic ideas that will be discussed in later chapters.

What is stream processing?

Stream processing has been one of the most popular technologies in the recent years in the big data domain. Streaming systems are the computer systems that process continuous event streams.

Streaming system examples

Streaming systems and real time

How a streaming system works

Applications

Backend services

Inside a backend service

Batch processing systems

Inside a batch processing system

Stream processing systems

Inside a stream processing system

The advantages of multi-stage architecture

The multi-stage architecture in batch and stream processing systems

Compare the systems

A model stream processing system

Summary

Exercise