A Kafka client is an application outside the Kafka cluster that connects to its brokers. In an online store, one client might publish product price changes while separate clients read them for inventory management and sales analysis. Producers write messages; consumers read them. The applications are clients, not brokers or coordination components. section 4.3.3 section 2.4
Clients do more than produce and consume directly. Kafka Streams can read, transform, and write data, acting as both consumer and producer. Kafka Connect links Kafka with external systems and can avoid custom producer or consumer code. A healthy cluster alone is not enough: the clients moving and processing data also need monitoring. section 4.3.3 section 15.3
Watch a price-change topic
With a broker available at localhost:9092, run `kafka-console-consumer.sh --topic products.prices.changelog --bootstrap-server localhost:9092`. In another terminal, run `kafka-console-producer.sh --topic products.prices.changelog --bootstrap-server localhost:9092` and type `coffee pads 11`. The consumer displays the new message. Start a second consumer with `--from-beginning` to read retained earlier messages as well. Independent consumers can read the same topic without coordinating with each other. section 2.4
# Consumer 2 window
$ kafka-console-consumer.sh \
--topic products.prices.changelog \
--from-beginning \
--bootstrap-server localhost:9092
coffee pads 10
coffee pads 11
coffee pads 12
coffee pads 10
The second consumer command shows how `--from-beginning` retrieves earlier price changes, including a value produced before that consumer started. section 2.4
A client’s connection and reading behavior matter in production:
- It first contacts a bootstrap broker for metadata, then uses the returned partition-leader information. List multiple bootstrap brokers so a client can still start if one is down; each broker must remain individually reachable. section 7.4
- A consumer fetches data rather than receiving unsolicited pushes. It tracks reading progress by offset within each partition; consumer groups divide partitions among members for parallel work. section 6.4 section 9.3
- Monitor client metrics as well as brokers. If a network failure cuts off brokers, consumers can resume after connectivity returns, while producers need an explicit choice about blocking, buffering, dropping messages, or failing. section 15.3 section 16.1.1
