liveBook · Manning

Cross-cluster replication

As explained in Kafka for Architects

Cross-cluster replication is the asynchronous or synchronous copying of Kafka data between separate clusters.

Cross-cluster replication copies Kafka data between separate clusters to support disaster recovery, geographic availability, and edge-to-central data flows. section 11.4.1 section 12.2.3

In an active-passive design, one cluster serves producers and consumers while a passive cluster receives asynchronous copies. Failover can involve downtime and data loss because RTO and RPO remain greater than zero. section 11.4.1

An active-active design keeps both clusters operational and serving clients. It can avoid planned failover downtime, but simultaneous writes introduce conflict and data-loss risks during an outage. section 11.4.1

Common deployment patterns include:

  • Use MirrorMaker to consume topics from one cluster and produce them to another, providing basic cross-cluster replication. section 12.2.3
  • Use Cluster Linking in Confluent Platform for broker-level, real-time replication without requiring producers or consumers to perform the transfer. section 12.2.3
  • Replicate from an edge cluster to a central cluster after connectivity returns, preserving local processing while supporting aggregation, analytics, and long-term storage. section 12.2.3
Figure 11.4 Replicating data in active-active configuration
A diagram of a cluster of data

AI-generated content may be incorrect.

Compare the active-active architecture used to keep both data centers operational. section 11.4.1

How Kafka for Architects covers it

How deep it goes
Contrasts active-passive and active-active designs and names replication tools, but stops short of configuration procedures.
Examples
Uses passive disaster-recovery clusters, active-active data centers, and edge clusters forwarding events to central systems.
What you can do after
Evaluate replication architectures against availability, failover time, data loss, conflicts, and edge-connectivity requirements.
How it connects
Connects cross-cluster replication with disaster recovery, active-active architecture, edge computing, MirrorMaker, and Cluster Linking.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.