1 Getting to know Kafka as an Architect

 

This chapter covers

  • Principles of event-driven architecture
  • Overview of the Kafka ecosystem
  • Utilizing Kafka in enterprise environments

An architect responsible for ensuring different systems within an enterprise work together effectively has many options for managing inter-system communication, with messaging being one of the key methods. Each option offers its unique features for handling large volumes, ensuring reliable delivery, and accommodating various formats. While there is no "one-size-fits-all" solution, Apache Kafka suits very well for a wide range of messaging use cases. It is open source, ensures reliable message handling and is capable of processing large volumes of messages at high speed. One of the main reasons Kafka is the preferred solution is its strong and active community, which is crucial for ongoing development and support. That's why for lots of architects, Kafka has become a natural choice for complex system designs.

1.1 How an architect sees Kafka

1.1.1 Event-driven architecture

1.1.2 Handling myriads of data

1.2 Customer 360: Journey of an event-driven project

1.3 Key players in the Kafka ecosystem

1.3.1 Brokers and clients

1.3.2 Managing cluster metadata

1.4 Architectural principles

1.4.1 The Publish-Subscribe pattern

1.4.2 Reliable delivery

1.4.3 The commit log

1.5 Data as a first-class citizen

1.5.1 Handling data contracts

1.5.2 Schema Registry

1.6 Data integration

1.6.1 Integration patterns

1.6.2 Kafka Connect

1.7 Data transformation

1.7.1 Streaming frameworks

1.8 Impacting operations and infrastructure

1.8.1 Kafka tuning and maintenance

1.8.2 On-premise and cloud options

1.8.3 Solutions from other cloud providers

1.9 Applying Kafka in enterprise

1.9.1 Using Kafka for sending messages

1.9.2 Using Kafka for storing data

1.9.3 How Kafka is different

1.10 Customer 360: The die is cast

1.11 Online Resources

1.12 Summary