5 Kafka in Real World Use Cases

 

This chapter covers

  • Key scenarios where Kafka excels
  • Identifying situations where Kafka may not be the ideal choice
  • Comparing Kafka with other technologies that serve similar purposes.

Gathered in their meeting room once again, Max Sellington, Eva Catalyst, and Rob Routine focused on discussing the use cases for Kafka and its alternatives.

Max (leaning forward): Alright team, let's talk about the use cases for Kafka. Are there any alternatives we should consider? Maybe we want to test something else alongside Kafka?

Eva (nodding): Good point, Max. The closest competitor to Kafka is Apache Pulsar. It has some technical advantages, like multi-tenancy and geo-replication. However, its community is smaller compared to Kafka's.

Rob (interjecting): Another popular option is RabbitMQ, especially for microservices communication. It's known for its simplicity and reliability.

Max: So, besides microservices, what are the other use cases for Kafka?

Rob: Well, Kafka excels in log collection and metrics collection because it's designed to handle high message rates efficiently.

Eva: Exactly. Kafka is also great for data integration, serving as a central hub to stream data between different systems.

Rob: And let's not forget real-time processing. Kafka's great for streaming data in real-time, even though the processing and analytics happen outside of Kafka. It works perfectly as a broker for real-time applications.

5.1 Navigating real-world implementation

5.1.1 Event-driven microservices

5.1.2 Data integration

5.1.3 Collecting logs

5.1.4 Real-time data processing

5.2 Differences with other messaging platforms

5.2.1 Publish-subscribe model

5.2.2 Partitioned data

5.2.3 Lack of broker-side logic

5.2.4 Sequential data access

5.2.5 Message persistence

5.2.6 Limitations in handling large messages

5.2.7 Scalability and high throughput

5.2.8 Fault tolerance

5.2.9 Batch processing

5.2.10 Lack of global ordering

5.3 Kafka Alternatives

5.3.1 RabbitMQ

5.3.2 Apache Pulsar

5.3.3 Solutions from cloud providers

5.4 Online Resources

5.5 Summary