liveBook · Manning

JDBC connector

As explained in Apache Kafka in Action

A JDBC sink connector is a Kafka Connect connector that writes records from Kafka to a relational database through JDBC.

A JDBC sink connector moves records from Kafka into a relational database through JDBC. It fits integrations where Kafka carries events and a database provides queryable or operational storage. section 11.10

Configure the connector around the failure and transformation behavior the integration requires:

  • Set connection.attempts and connection.backoff.ms to control database reconnection attempts. The default behavior is three attempts separated by 10 seconds. section 11.6.2
  • Use max.retries and retry.backoff.ms to retry failed sink writes, such as writes blocked by temporarily locked database entries. The default is 10 attempts with three-second intervals. section 11.6.2
  • Combine connector retries with errors.retry.timeout and errors.retry.delay.max.ms for broader Kafka Connect error handling; route unprocessable sink records to a dead-letter queue. section 11.10 section 11.6.2
  • Apply Single Message Transformations for simple changes such as renaming fields, masking data, or moving values to keys; reserve complex ETL for other mechanisms. section 11.10

Handling a temporary database outage

During database maintenance or a temporary network outage, extend the JDBC connector's connection retry window rather than relying on the default period. A connector that fails may require a manual restart before its tasks resume, while max.retries and retry.backoff.ms handle individual failed writes. section 11.6.2

How Apache Kafka in Action covers it

How deep it goes
Explains connector-level retries, backoff, dead-letter handling, and transformations without detailing database schema mapping.
Examples
Uses database maintenance, temporary network outages, locked entries, and table-filtered integrations as operational examples.
What you can do after
Operate/configure it by setting database connection attempts, backoff intervals, write retries, and dead-letter handling.
How it connects
Connects Kafka Connect with sink connectors, dead-letter queues, Single Message Transformations, and relational databases.

Also explained in

Compare how 2 books explain JDBC connector →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.