liveBook · Manning

Leader epoch

As explained in Apache Kafka in Action

A leader epoch is a monotonically increasing Kafka partition leadership version that distinguishes current leadership from stale leadership.

A leader epoch is a monotonically increasing version for a Kafka partition’s leadership. Each leader change increments it, allowing brokers to distinguish the current leader from a stale one. section 8.3.2

Rejecting a stale leader

Suppose Broker 3 leads Partition 2 and then becomes unreachable. Kafka elects another leader and increments the partition’s epoch from 0 to 1. If Broker 3 restarts believing it is still leader, its replication messages carry epoch 0 and are rejected because the current epoch is 1. section 8.3.2

$ cat ~/kafka/data/kafka1/\
products.prices.changelog.file-structure-0/leader-epoch-checkpoint
0
1
0 0

Inspect the leader-epoch-checkpoint file to see the initial epoch-and-offset entry. section 8.3.2

$ kafka-broker-stop.sh 3
$ cat ~/kafka/data/kafka1/\
products.prices.changelog.file-structure-0/leader-epoch-checkpoint
0
1
1 0

Compare the checkpoint after a broker stop to observe the epoch advancing from 0 to 1. section 8.3.2

How Apache Kafka in Action covers it

How deep it goes
Explains checkpoint entries and traces broker failure, leader election, restart, epoch comparison, and message rejection.
Examples
Uses leader-epoch-checkpoint files and a stopped-and-restarted broker to demonstrate epoch advancement and stale-message rejection.
What you can do after
Diagnose/debug with checkpoint files and broker-failure scenarios to identify stale leadership during replication.
How it connects
Connects Kafka partitions with leader election, replication, brokers, and log checkpoints.
Cover of Apache Kafka in Action
liveBookmanning.com
Read Apache Kafka in Action on liveBook Get the book at manning.com

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.