Change-data-capture
Change-data capture is a technique for detecting and recording changes to a data source, such as insertions, updates, and deletions, so they can be propagated or processed.
Apache Kafka in Action
Change data capture is a technique that monitors and records changes made to a data source rather than repeatedly reading its complete current contents.
- How it is used
- Moves database changes from core systems into Kafka while limiting direct interaction with those systems.
- How deep it goes
- Defines change monitoring, contrasts it with JDBC polling, and walks through PostgreSQL preparation; leaves Debezium installation to its documentation.
- Examples
- Uses missed timestamp-based changes and a PostgreSQL customers table with a Debezium connector.
- What you can do after
- Operate/configure it with Debezium and PostgreSQL, while recognizing why raw captured tables need downstream restructuring.
Read how Apache Kafka in Action explains change-data-capture →
ScyllaDB in Action
Change data capture is a database feature that records table mutations in a change-log table so they can be consumed over time.
- How it is used
- Supports exporting table changes and building histories without repeatedly scanning an entire table.
- How deep it goes
- Explains per-table configuration, generated log fields, partial-update behavior, stream queries, and topology-driven stream generations.
- Examples
- Demonstrates enabling CDC on authors, inserting Charlie Change, updating a bio, and inspecting the resulting log rows.
- What you can do after
- Operate/configure it by enabling CDC, locating its log table, and querying changes through stream IDs.
Kafka for Architects
Change data capture is a pattern that observes a source system's change records and emits insertions, updates, and deletions as events.
- How it is used
- Uses database change events to integrate source systems with multiple downstream destinations in real time.
- How deep it goes
- Explains CDC versus snapshots, event representation, connectors, and deployment architecture without detailing connector implementation internals.
- Examples
- Shows reference tables flowing from PostgreSQL and MongoDB through Kafka topics into MS SQL and Amazon S3.
- What you can do after
- Operate/configure Kafka Connect with source and sink connectors, optionally integrating Debezium for change-log streaming without custom code.
Read how Kafka for Architects explains change-data-capture →
Architecting an Apache Iceberg Lakehouse
Change data capture processing handles records representing inserts, updates, and deletes made to a source system.
- How it is used
- Deploys CDC in streaming ingestion and incremental lakehouse modeling.
- How deep it goes
- Explains stateful processing, delete semantics, engine support, compaction, and read-write tradeoffs without detailing Iceberg specification semantics.
- Examples
- Uses Flink-to-Iceberg Kafka pipelines, equality deletes, position deletes, merge transactions, and downstream synchronization scenarios.
- What you can do after
- Apply CDC via Apache Flink and Iceberg using streaming, checkpointing, delete strategies, and conditional merge operations.
Read how Architecting an Apache Iceberg Lakehouse explains change-data-capture →
Kafka Streams in Action, Second Edition
Change data capture is a technique for detecting and exporting database record changes such as inserts, updates, and deletes as they occur.
- How it is used
- Uses CDC to integrate existing databases with event-streaming applications and downstream consumers.
- How deep it goes
- Explains captured operations, Kafka-topic storage, connector configuration, and the distinction between custom clients and production-ready connectors.
- Examples
- Uses database INSERT, UPDATE, and DELETE events as the central example.
- What you can do after
- Operate CDC by configuring a Kafka Connect connector to publish database changes into a Kafka topic.
Read how Kafka Streams in Action, Second Edition explains change-data-capture →
Related concepts
- transaction loguses · A transaction log is an ordered record of transactions, transaction-related events, or resulting data changes, maintained as a history and often used for tracking or recovery.
- CDC tableproduces · A CDC table is a ScyllaDB table that stores records describing changes made to rows in a corresponding source table.
- database tableuses · A database table is a relational database structure that organizes records into rows and defined columns.
- Apache Kafkaused by · Apache Kafka is a distributed event-streaming platform for publishing, storing, processing, and consuming streams of records.
- small fileproduces · A small file is a data file whose size is sufficiently low that file-management, metadata, and scheduling overhead become significant relative to its data content.
- Debeziumused by · Debezium is an open-source change data capture platform that captures database changes and streams them to downstream systems.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.