appendix ASetting up a Kafka test environment

 

In this appendix, we’ll provide a brief description of how to set up a Kafka test environment on a single machine, such as a personal laptop. This setup should never be used for a production environment, but it’s great to use throughout the book. In contrast to many blog posts on the internet, we focus on setting up a multi-node environment to be able to see how Kafka survives partial failures.

In figure A.1 we display the test environment, which consists of three Kafka brokers. Broker 1 listens on port 9092, Broker 2 on 9093, and Broker 3 on 9094. To simplify the setup, we’ll run the brokers in dual mode, meaning that the three brokers will act both as controllers and normal brokers. The controllers need additional listeners, and so the controller of Broker 1 will listen on port 9192, Broker 2’s controller will listen on 9193, and Broker 3’s will listen on 9194.

A.1 Operating systems

Apache Kafka runs on the Java Virtual Machine (JVM) and thus supports all major operating systems. We have tested this environment on a macOS 15.2 Sequoia on Apple Silicon and on an Ubuntu 24.04 LTS machine with an Intel CPU. We recommend using the Windows Subsystem for Linux (WSL) on Windows.

A.2 Downloading Kafka

You can always find the most up-to-date version of Kafka at https://kafka.apache.org/downloads. At the time of writing, the latest version is 3.9.0. As some parts of

Figure A.1 Our test environment consists of three Kafka brokers in dual mode.
A group of colorful buttons

AI-generated content may be incorrect.

A.3 Configuring Kafka

A.4 Preparing the data directories

A.5 Starting Kafka

A.6 Stopping Kafka