liveBook · Manning

Kafka-topics.sh

As explained in Apache Kafka in Action

kafka-topics.sh is an Apache Kafka command-line utility for creating, altering, describing, and deleting topics.

Before an online store can record product price changes, it needs a Kafka topic to hold them. kafka-topics.sh is Kafka’s command-line tool for managing topics. Create products.prices.changelog first, then send price updates to it with kafka-console-producer.sh and read them with kafka-console-consumer.sh. section 2.2 section 2.4

$ kafka-topics.sh \
    --create \
    --topic products.prices.changelog \
    --partitions 1 \
    --replication-factor 1 \
    --bootstrap-server localhost:9092
Created topic products.prices.changelog.

This creation command names the topic, starts with one partition and a replication factor of one, and connects to the local cluster through localhost:9092. That small configuration is easy to try, but replication factor one provides no failover. If creation fails, check that Kafka is running and that the topic does not already exist. section 2.2 section 3.1.1

Use --list with --bootstrap-server localhost:9092 to see available topics. Use --describe with --topic and --bootstrap-server to inspect one topic’s configuration. The description reports its topic ID, partition count, replication factor, configuration settings, and partition-level information. This helps people sharing a topic check the parameters they depend on. section 3.1.1

$ kafka-topics.sh \
    --describe \
    --topic products.prices.changelog \
    --bootstrap-server localhost:9092
Topic: products.prices.changelog TopicId: GGYA9u_aRPSd0JRaGn2eBA
PartitionCount: 1 ReplicationFactor: 1 Configs:
  Topic: products.prices.changelog Partition: 0 Leader: 3 Replicas: 3
    Isr: 3 Elr: LastKnownElr:

Compare the reported PartitionCount and ReplicationFactor with the creation command. Here both are one, and the output has one partition line. Partitions support parallel processing; replicas provide redundant storage. Detailed partition and replication behavior is a separate topic. section 3.1.1

The script can also alter and delete topics, although the commands for those operations are not shown here. Plan partition counts carefully: a topic’s partition count cannot be decreased. section 3.3

How Apache Kafka in Action covers it

How deep it goes
Works through creation and description options and interprets output; mentions alteration and deletion without showing their commands.
Examples
Uses a product-price changelog topic on localhost:9092, with creation, listing, and description commands.
What it assumes
Assumes Kafka is installed and accessible; introduces topic creation with a one-partition, one-replica starting configuration.
What you can do after
Operate/configure it to create a topic, list topics, and inspect partition and replication settings.
How it connects
Connects Kafka topics to partitions, replication, producers, and consumers.
Cover of Apache Kafka in Action
liveBookmanning.com
Read Apache Kafka in Action on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.