Apache MirrorMaker

As explained in Apache Kafka in Action

Kafka MirrorMaker 2 is Kafka's tool for replicating topics, consumer offsets, and related metadata between Kafka clusters.

Apache MirrorMaker 2 continuously mirrors Kafka data between clusters, providing a practical way to protect against failures and data loss across larger distances. Unlike conventional backups, it copies messages continuously rather than creating occasional snapshots. section 16.3 section 4.4

MirrorMaker uses three Kafka Connect connectors to coordinate cross-cluster replication:

• MirrorSourceConnector mirrors topics and their ACLs. section 16.3
• MirrorCheckpointConnector handles consumer offsets. section 16.3
• MirrorHeartbeatConnector periodically checks connectivity between clusters. section 16.3

Choosing a disaster-recovery topology

An active-passive pairing keeps one cluster available for takeover after failure, whereas an active-active pairing lets both equally capable clusters operate continuously. In active-active deployments, consumers may read from both clusters and must aggregate the data appropriately. MirrorMaker 2 remote topics prevent endless mirroring loops while preserving availability during failures. section 16.4 section 16.4

Figure 4.12 Kafka alone is usually not enough. The Kafka ecosystem offers numerous components to integrate Kafka into our enterprise landscape and thus build a streaming platform.

Place MirrorMaker within the broader Kafka enterprise ecosystem, alongside monitoring, automation, compliance, and other platform components. section 4.4

How Apache Kafka in Action covers it

How deep it goes

Explains the connector roles and recovery topologies, but defers detailed architecture and current operational guidance.

Examples

Uses active-passive, active-active, and hub-and-spoke cluster topologies to illustrate failure handling and continued operation.

What it assumes

Builds on Kafka Connect and assumes familiarity with Kafka clusters, topics, consumer offsets, and ACLs.

What you can do after

Operate cross-cluster replication and evaluate recovery topologies for Kafka data-center failures.

How it connects

Connects Kafka Connect, ACLs, consumer offsets, disaster recovery, remote topics, and active-active replication.

liveBookmanning.com

Read Apache Kafka in Action on liveBook Get the book at manning.com

Related concepts

• star topologyused by · A star topology is a network arrangement in which multiple peripheral nodes connect and communicate through a central hub or node.
• Kafka Connectuses · Kafka Connect is a framework for reliably transferring data between Apache Kafka and external systems through configurable source and sink connectors.
• active-active configurationused by · An active-active configuration is a deployment pattern in which multiple nodes, sites, or environments concurrently serve production workloads and provide redundancy.
• cross-cluster replicationimplements · Cross-cluster replication is the asynchronous or synchronous copying of Kafka data between separate clusters.
• Apache Kafkauses · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.