How deeply does Kafka for Architects cover change-data-capture?

Explains CDC versus snapshots, event representation, connectors, and deployment architecture without detailing connector implementation internals.

Change-data-capture

As explained in Kafka for Architects

Change data capture is a pattern that observes a source system's change records and emits insertions, updates, and deletions as events.

Change data capture (CDC) is an event-driven alternative to repeatedly copying a source system’s current state. It reads change logs and emits insertions, updates, and deletions for downstream systems in real time. section 5.0 section 5.1.2

CDC uses a uniform event representation rather than requiring specialized events for every business change. Kafka provides the central streaming hub, while downstream systems consume the resulting topics independently. section 5.0 section 5.1.2

CDC is useful when:

Systems need database changes made available to multiple destinations without building custom notification services. section 5.1.2
A streaming flow is preferable to waiting for a batch process to complete. section 5.0
The source contains data that changes through direct updates rather than application-managed services. section 5.1.2

Reference-data replication

Reference tables such as occupations, education levels, and country codes can be streamed from source systems through Kafka topics. Kafka Connect source connectors publish the data, while independent sink connectors deliver it to an MS SQL database and Amazon S3. Debezium can provide the CDC integration with source change logs. section 5.1.2

How Kafka for Architects covers it

liveBook manning.com

Read Kafka for Architects on liveBook Get the book at manning.com

Also explained in

liveBook manning.com

Apache Kafka in ActionChange data capture is a technique that monitors and records changes made to a data source rather than repeatedly reading its complete current contents.
liveBook manning.com

ScyllaDB in ActionChange data capture is a database feature that records table mutations in a change-log table so they can be consumed over time.
liveBook manning.com

Architecting an Apache Iceberg LakehouseChange data capture processing handles records representing inserts, updates, and deletes made to a source system.
liveBook manning.com

Kafka Streams in Action, Second EditionChange data capture is a technique for detecting and exporting database record changes such as inserts, updates, and deletes as they occur.

Compare how 5 books explain change-data-capture →

Related concepts

transaction loguses · A transaction log is an ordered record of transactions, transaction-related events, or resulting data changes, maintained as a history and often used for tracking or recovery.
CDC tableproduces · A CDC table is a ScyllaDB table that stores records describing changes made to rows in a corresponding source table.
database tableuses · A database table is a relational database structure that organizes records into rows and defined columns.
Apache Kafkaused by · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.
small fileproduces · A small file is a data file whose size is sufficiently low that file-management, metadata, and scheduling overhead become significant relative to its data content.
Debeziumused by · Debezium is an open-source change data capture platform that captures database changes and streams them to downstream systems.

This explainer was generated by Manning's knowledge graph from the text of Kafka for Architects. The section links point to the parts of the book it draws on.