A price-update topic can have several copies of each partition. One broker leads the partition; follower brokers fetch messages from that leader into their own logs. The in-sync replica set (ISR) identifies the replicas that are up to date, including the leader. It matters when a broker fails and when a producer waits for confirmation that a message has been replicated. section 4.2.3 section 8.4.1
A follower remains in sync if it has retrieved all messages from the leader within the allowed lag time, 30 seconds by default. The leader tracks follower progress against its Log End Offset (LEO), the end of its log. If a follower fails to catch up within that period, or stops responding to heartbeats, the leader removes it from the ISR. The lag limit is configurable with replica.lag.time.max.ms; followers request new messages every 500 ms by default, adjustable with replica.fetch.wait.max.ms. section 8.4.1
Use kafka-topics.sh --describe to compare Replicas, the brokers assigned a copy, with Isr, the brokers currently in sync. A mismatch can signal a failed or lagging broker. Keeping slow followers out of the ISR also prevents acks=all from waiting on every assigned replica indefinitely. section 3.1.1 section 8.4.1
$ kafka-broker-stop.sh 3
$ kafka-broker-stop.sh 2
$ kafka-topics.sh \
--describe \
--topic products.prices.changelog.min-isr-2 \
--bootstrap-server localhost:9092
Topic: products.prices.changelog.min-isr-2 PartitionCount: 3
ReplicationFactor: 3 Configs: min.insync.replicas=2
Topic: products.prices.changelog.min-isr-2 Partition: 0
Leader: 1 Replicas: 1,2,3 Isr: 1
Topic: products.prices.changelog.min-isr-2 Partition: 1
Leader: 1 Replicas: 2,3,1 Isr: 1
Topic: products.prices.changelog.min-isr-2 Partition: 2
Leader: 1 Replicas: 3,1,2 Isr: 1
After brokers 2 and 3 stop, each partition still lists three assigned replicas, but only broker 1 remains in Isr. Broker 1 has also become leader for every partition. section 5.1.2
ISR size shapes the reliability trade-off. With producer acks=all, the leader waits for every ISR to receive the message before acknowledging it. Set the topic’s min.insync.replicas alongside acks=all to require a minimum number of in-sync replicas. If that minimum is unavailable, the producer gets an error and retries. By contrast, acks=1 acknowledges receipt at the leader sooner, but a leader failure before follower replication can lose the acknowledged message. section 5.1.1 section 5.1.2
$ kafka-console-producer.sh \
--topic products.prices.changelog.min-isr-2 \
--bootstrap-server localhost:9092 \
--producer-property acks=all
>cola all
>[…] WARN [Producer clientId=console-producer] Got error produce response
with correlation id 10 on topic-partition
products.prices.changelog.min-isr-2-2, retrying (2 attempts left).
Error: NOT_ENOUGH_REPLICAS
(org.apache.kafka.clients.producer.internals.Sender)
[…] WARN [Producer clientId=console-producer] Got error produce response
with correlation id 11 on topic-partition
products.prices.changelog.min-isr-2-2, retrying (1 attempts left).
Error: NOT_ENOUGH_REPLICAS
(org.apache.kafka.clients.producer.internals.Sender)
[…] WARN [Producer clientId=console-producer] Got error produce response
with correlation id 12 on topic-partition
products.prices.changelog.min-isr-2-2, retrying (0 attempts left).
Error: NOT_ENOUGH_REPLICAS
(org.apache.kafka.clients.producer.internals.Sender)
[…] ERROR Error when sending message to topic
products.prices.changelog.min-isr-2 with key: null, value: 8 bytes with
error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.NotEnoughReplicasException: Messages are
rejected since there are fewer in-sync replicas than required.
With min.insync.replicas=2 but only one broker still in sync, an acks=all producer reports NOT_ENOUGH_REPLICAS rather than receiving a successful acknowledgement. section 5.1.2
ISR membership also affects reading: the High Watermark marks the offset replicated and committed across all ISRs, and replication delays can affect when messages become available to consumers. Check ISR membership before treating an assigned replica as a current copy or tuning acknowledgement requirements. section 8.5 section 4.2.3
