liveBook · Manning

Timestamp mode

As explained in Apache Kafka in Action

Timestamp-based mode tracks database changes using a column containing modification timestamps.

Timestamp mode lets a JDBC source connector find database changes through a modification column such as lastupdated. After storing its offset, Kafka Connect queries for records newer than the previously observed timestamp, so updates to existing rows can produce Kafka messages. section 11.8.3

Detecting an updated customer

A connector configured with mode=timestamp and timestamp.column.name=lastupdated can publish an updated customer row, unlike incrementing mode, which notices only new IDs. For example, changing customer ID 3's birthdate can create a Kafka message containing the corrected value. section 11.8.3

Use timestamp mode with these operational safeguards:

  • Choose a timestamp column that changes whenever the tracked row changes; tables without such a column cannot reliably expose updates. section 11.8.3
  • Ensure timestamp values increase monotonically, because equal-time changes can cause Kafka Connect to treat a later record as already read. section 11.8.3
  • Combine timestamp and incrementing modes when needed, using timestamp+incrementing and satisfying the monotonicity requirements for the relevant columns. section 11.8.3
UPDATE customers SET birthdate="1969-12-28" where id=3;....#1

Inspect the concrete customer-row update that timestamp mode is intended to detect. section 11.8.3

How Apache Kafka in Action covers it

How deep it goes
Explains configuration, offset-based querying, combined modes, and missed-change limitations without detailing connector internals.
Examples
Uses a customer table, lastupdated configuration, and a birthdate update to demonstrate producing a corrected Kafka message.
What it assumes
Assumes familiarity with database tables, Kafka topics, and JDBC source connector configuration.
What you can do after
Configure a JDBC source connector to publish timestamp-detectable row updates and recognize conditions that can lose changes.
How it connects
Connects timestamp mode with Kafka Connect, JDBC source connectors, incrementing mode, and change data capture.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.