JDBC connector

As explained in Kafka Streams in Action, Second Edition

A JDBC connector is a Kafka Connect connector that reads from or writes to databases through the Java Database Connectivity interface.

A JDBC connector connects Kafka Connect with a database, using JDBC to import table rows into Kafka or support database writes through a sink connector. A source connector uses mode to detect changes and topic.prefix to map each table to a Kafka topic. section 5.3

Configure a JDBC source connector with:

• connector.class, connection.url, username, and password for connector identity and database access. section 5.3
• mode and timestamp.column.name to select rows whose tracked timestamp is newer than the previous import. section 5.3
• A converter and serialization settings, allowing formats such as JSON or Protobuf to vary by connector. section 5.3

PostgreSQL source configuration

A REST API request starts a connector against PostgreSQL with timestamp mode, the ts column, a postgres_ topic prefix, JSON values without schemas, and one task. section 5.3

Figure 5.2 Connect converter changes the data format before the data gets into Kafka or after it leaves Kafka.

Trace where a converter changes data format before it enters Kafka or after it leaves Kafka. section 5.3

How Kafka Streams in Action, Second Edition covers it

How deep it goes

Explains connection, timestamp polling, topic naming, converters, and REST configuration, but stops short of sink-specific operation.

Examples

Uses a PostgreSQL table, timestamp column, Docker initialization, REST configuration, JSON conversion, and a topic prefix.

What it assumes

Assumes basic Kafka Connect and database familiarity while defining converters and explaining connector settings.

What you can do after

Operate/configure a JDBC source connector that imports changed database rows into Kafka topics.

How it connects

Ties JDBC connectors to Kafka Connect, source connectors, sink connectors, converters, Kafka topics, and PostgreSQL.

liveBookmanning.com

Read Kafka Streams in Action, Second Edition on liveBook Get the book at manning.com

Also explained in

• 

  liveBookmanning.com
  Apache Kafka in ActionA JDBC sink connector is a Kafka Connect connector that writes records from Kafka to a relational database through JDBC.

Compare how 2 books explain JDBC connector →

Related concepts

• JDBCuses · JDBC is the standard Java API for connecting applications to relational databases, executing SQL statements, and processing results.
• databaseuses · A database is an organized collection of persistent data managed by software for storage, retrieval, querying, and modification.
• source connectorkind of · A source connector is a Kafka Connect connector that reads data from an external source system and publishes records to Kafka.
• JDBC source pluginincludes · A JDBC source connector is a Kafka Connect connector that reads records from relational databases through JDBC and publishes them to Kafka.

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.