This chapter covers
- What Apache Kafka is and its use cases
- How Kafka fits into enterprise ecosystems
- Architectural overview of Kafka
- Running and using Kafka
Modern enterprise applications are often built from independent components and services that communicate by exchanging messages. Managing this flow of messages becomes increasingly complex as systems scale across multiple servers, data centers, cloud platforms, and geographic regions. The challenge is further heightened by the need for reliability, fault tolerance, and near-real-time performance. Apache Kafka was designed specifically to address these demands, providing a high-throughput, distributed messaging platform capable of handling massive streams of data efficiently.
In this first chapter, we’ll learn what Apache Kafka is, dive into its basic components and their purpose, and discuss where it can be used. More importantly, we’ll look at how Kafka fits into broader enterprise ecosystems. Finally, we’ll cover what is required to use Kafka in terms of hardware, tools, and programming languages.
1.1 What is Apache Kafka, and how does it solve our problems?
Our world runs in real time. When we make a purchase, we expect immediate confirmation. When we order parcels, we expect real-time tracking. When our credit card is used suspiciously, we want to know now, not tomorrow. As customers, we have come to expect organizations to process and react to data as fast as life happens.