liveBook · Manning

Replication factor

As explained in Apache Kafka in Action

A replication factor is the number of copies maintained for each Kafka partition.

A product-price topic may be useful for monitoring and historical analysis, but a single stored copy leaves its messages without failover. A topic’s replication factor sets the number of copies maintained for each partition. A factor of 1 means one replica; a factor of 3 normally means one leader and two followers on different brokers. section 2.2 section 4.2.3

Create a replicated topic

On a three-broker cluster, run `kafka-topics.sh --create --topic products.prices.replication --partitions 3 --replication-factor 3 --bootstrap-server localhost:9092`. The factor applies to each of the three partitions, not to the topic as a whole. A factor of 4 would fail here: four replicas require at least four registered brokers. section 4.2.3

$ kafka-topics.sh \
    --describe \
    --topic products.prices.replication \
    --bootstrap-server localhost:9092
Topic: products.prices.replication PartitionCount: 3 ReplicationFactor: 3
Configs:    #1
Topic: products.prices.replication Partition: 0 Leader: 2 Replicas: 2,1,3
Isr: 2,1,3 Elr:   #2
Topic: products.prices.replication Partition: 1 Leader: 3 Replicas: 3,2,1
Isr: 3,2,1 Elr:
Topic: products.prices.replication Partition: 2 Leader: 1 Replicas: 1,3,2
Isr: 1,3,2 Elr:

The `--describe` output reports `ReplicationFactor: 3` and lists three broker IDs under `Replicas` for each partition. `Leader` identifies the broker leading that partition; `Isr` shows its in-sync replicas. section 4.2.3

Replication is warm rather than a backup: followers continually copy new messages from the leader, and a follower can take over if the leader fails. It protects against broker failure, not mistakes such as deleting data. More copies also raise cluster cost. section 4.2.3 section 16.1.2

Choose the factor alongside `min.insync.replicas` and producer `acks=all`. With three brokers, a factor of 3 and `min.insync.replicas=2` allow one broker to fail without affecting producers or consumers. After a second failure, consumers can still read from the remaining replica, but `acks=all` producers cannot continue writing. Setting `min.insync.replicas` to 1 can permit data loss despite `acks=all`. section 16.1.2 section 5.1.2

How Apache Kafka in Action covers it

How deep it goes
Defines the setting, demonstrates creation and inspection, and works through broker-failure trade-offs without detailing the replication protocol.
Examples
Uses product-price topics, `kafka-topics.sh` creation and inspection commands, and a three-broker failure scenario.
What it assumes
Starts with a simple local topic and introduces partitions and brokers before examining replica roles.
What you can do after
Operate/configure it with `kafka-topics.sh`, then assess how replica count and minimum in-sync replicas affect availability.
How it connects
Links Kafka partitions, brokers, leaders and followers to in-sync replicas, producer acknowledgements, and minimum in-sync replicas.

Also explained in

Compare how 4 books explain replication factor →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.