4 Producers: Sourcing data

This chapters covers

Sending messages and the producer
Creating our own producer serializers and partitioners
Examining configuration options to solve a company’s requirements

In the previous chapter, we looked at the requirements that an organization might have regarding their data. Some design decisions we made have practical impacts on how we send data to Kafka. Let’s now enter the world of an event-streaming platform through the portal gate of a Kafka producer. After reading this chapter, you will be well on your way to solving fundamental requirements of a Kafka project by producing data in a couple of different ways.

The producer, despite its importance, is only one part of this system. In fact, we can change some producer configuration options or set these at the broker or topic level. We will discover those options as we get further along, but getting data into Kafka is our first concern in this chapter.

4.1 An example

The producer provides the way to push data into the Kafka system for our example project. As a refresher, figure 4.1 illustrates where producers fit into Kafka.

Figure 4.1 Kafka producers

4 Producers: Sourcing data

This chapters covers

4.1 An example

Figure 4.1 Kafka producers

4.1.1 Producer notes

4.2 Producer options

4.2.1 Configuring the broker list

4.2.2 How to go fast (or go safer)

4.2.3 Timestamps

4.3 Generating code for our requirements

4.3.1 Client and broker versions

Summary

References