A Kafka client library lets an application communicate with Kafka brokers. Kafka supports client libraries across languages, so producer and consumer applications can integrate with Kafka without being written in Scala or Java. section 1.4
Choose a client according to the application runtime and the features required:
- Use the official Kafka Java library for producers running on the Java Virtual Machine. section 8.1
- Use a library based on librdkafka when the producer does not run in the JVM. section 8.1
- Treat library selection as an operational choice: alternative libraries may be easier to use but can lack features and optimizations. section 8.1
The library carries much of the producer workflow, including serialization, partitioning, buffer management, acknowledgments, timeouts, and retries. Kafka brokers then receive produce requests and persist the data efficiently. section 8.5
A non-JVM producer
A producer application written outside the JVM can communicate with Kafka through a librdkafka-based library; Python examples illustrate the same basic communication concepts used in Java and other languages. section 8.1
