A client is an application that communicates with Kafka brokers. It sends messages as a producer, receives them as a consumer, or does both. Brokers sit between these applications, so a producer need not connect directly to every service that uses its messages. section 1.3.1 section 1.4.1
One update, several reactions
When a customer changes address, a service stores the change and publishes an event. Billing, analytics, and order services can each consume that event independently. The producer owns the event structure and makes it reusable, but business requirements still determine which notifications it emits. section 1.4.1
The two client roles have different operational responsibilities:
- A producer formats and sends messages. Its client library can batch sends, distribute messages across partitions, and retry after errors; configuration controls much of this behavior. section 3.1 section 3.1.2
- A consumer subscribes to topics and polls brokers for batches. It deserializes and processes messages, then commits an offset recording where to resume. A commit records a position, not proof that processing succeeded. section 4.1 section 4.2
- Consumers in one group share partitions; each partition belongs to only one group member at a time. More instances can spread the work, but the partition count limits that form of scaling. Consumer lag helps reveal a growing backlog. section 4.1 section 4.3.1
A client application typically combines business logic, a Kafka client library, serialization libraries, and external configuration. Frameworks such as Spring-Kafka and Micronaut-Kafka simplify integration; lower-level clients offer finer control over matters such as offset handling and concurrency. Configuration choices affect reliability as well as throughput: automatic offset commits can occur before processing finishes, while uncommitted messages can be read again after an error. section 3.1 section 3.1.2 section 4.2.4
Aggregating customer data
A customer dashboard can run separate consumers for profile and transaction topics. If both update the same stored customer record, a straightforward read-modify-write sequence can overwrite one update with the other. Optimistic locking is one way to protect that shared record. The issue lies in client-side processing, not in broker delivery alone. section 4.2.13
Keep the division of responsibility clear: Kafka brokers do not deserialize messages to validate their contents or route them by content. Those tasks belong in clients or intermediate processing services. Design clients around the events they need, the positions they commit, and the consequences of failure—not just the connection to a broker. section 5.2.3 section 4.2.4
