A customer-profile service may publish changes to Kafka while another service reads profile, billing, and interaction events to assemble a fuller view. A Kafka client library supplies the application-side behavior for that exchange: producers send records, and consumers retrieve and process them. The application still owns its business logic. section 3.1
A producer client does more than transmit each record immediately. It encodes messages as bytes, selects partitions, assembles batches, sends them to brokers, and can wait for acknowledgments or retry after errors. These behaviors affect throughput and delivery, so client configuration is an architectural decision rather than incidental plumbing. section 3.1.1 section 3.1.2
The integration choice determines how much control the application team takes on:
- A low-level client exposes finer control over message conversion, dead-letter routing, consumer concurrency, and offset handling. section 3.1
- Spring-Kafka and Micronaut-Kafka streamline development but can obscure underlying client behavior. section 3.1
- A REST proxy offers an HTTP integration path; native libraries provide fuller feature access and avoid HTTP overhead. section 4.0
Keep client code straightforward and tune behavior through name-value parameters. A typical service combines application code, a Kafka client library, serialization libraries, other dependencies, and external configuration. Separate configuration allows environments to use different settings without changing application code. Consumer instances can also form a group to divide work across partitions; one instance can handle all partitions when parallel processing is unnecessary. section 3.1.2 section 2.2.2
Figure 3.1 Simplified workflow of message transmission by a producer, including serialization, partition selection, batch assembly, and data transmission to the server, with an optional confirmation step
Follow the producer path from serialization and partition selection through batching and transmission to optional broker confirmation. section 3.1.1
Clients use Kafka’s binary protocol over TCP/IP. Libraries exist in multiple languages and follow the same broad architecture and configuration model. When a client connects, protocol-version negotiation selects a mutually supported API version automatically, allowing compatible clients and brokers to be upgraded independently. section 3.1.1
