This chapter covers
- The uses and properties of logs
- Considering Kafka as a log
- Kafka as a distributed system: partitions and replication
- Taking a closer look at Kafka cluster components
- Kafka in corporate use
In this chapter, we’ll explore the concept of logs and how Kafka operates using a log-based approach. We’ll examine Kafka’s architecture as a distributed system and delve into its core components. Finally, we’ll discuss the practical applications of Kafka in a corporate environment, highlighting its benefits for enterprise data management.
4.1 Logs
We like to describe Kafka as a log. Although the term log is new to many in the context of Kafka, we encounter logs all the time in everyday life in the world of IT.
4.1.1 What exactly is a log?
Our operating systems produce system logs. If we’re responsible for these systems, we use the logs regularly to check the status of the systems and, in the event of errors, to understand how these errors could have occurred. We may even use log monitoring systems that sound alarms when certain events occur and, if necessary (but hopefully not), wake us up in the middle of the night so we can respond to the errors.