Sink connector
A sink connector is a Kafka Connect connector that reads records from Kafka and exports them to an external destination system.
Kafka Streams in Action, Second Edition
SinkConnector is the Kafka Connect connector type that generates tasks for exporting data from Kafka to an external system.
- How it is used
- Exports Kafka topic records to external applications through configured Kafka Connect plugins.
- How deep it goes
- Explains connector-to-task responsibilities, distributed execution, REST configuration, transforms, and error settings without implementing a custom connector.
- Examples
- Configures an Elasticsearch destination for orientation-student records and chains ValueToKey, ExtractField, and MaskField transforms.
- What you can do after
- Operate/configure it through the Connect REST API to export a topic, transform records, and route failed writes to a dead-letter queue.
Read how Kafka Streams in Action, Second Edition explains sink connector →
Kafka for Architects
A JDBC sink connector is a Kafka Connect connector that reads records from Kafka and writes them to tables in a target relational database.
- How it is used
- Moves Kafka topic records into external systems, including relational databases, through a configured integration pipeline.
- How deep it goes
- Explains consumer-group operation, converters and error handling, but stops short of a JDBC sink configuration walkthrough.
- Examples
- Illustrates address updates written to a service database and reference data delivered to MS SQL and Amazon S3.
- What you can do after
- Operate/configure it in Kafka Connect, choosing a destination plugin and an error-handling response for rejected records.
Apache Kafka in Action
A sink connector is a Kafka Connect connector that reads data from Kafka and writes it to an external destination system.
- How it is used
- Moves Kafka records into external databases, storage systems, messaging systems, or warehouses through standardized integration.
- How deep it goes
- Defines the data flow and covers connector classes, topic selection, task parallelization, consumer groups, and operational error handling.
- Examples
- Uses JdbcSinkConnector, topic.regex, tasks.max, REST configuration, retries, transformations, and dead-letter queues as concrete operational examples.
- What you can do after
- Operate/configure Kafka-to-system integrations with selected topics, parallel tasks, transformations, retries, and failure destinations.
Related concepts
- connectorkind of · A connector is a software element or mechanism that links components, applications, or resources to enable or represent their interaction, communication, or data exchange.
- Kafka Connectused by · Kafka Connect is a framework for reliably transferring data between Apache Kafka and external systems through configurable source and sink connectors.
- errors.toleranceconfigured by · errors.tolerance is a Kafka Connect setting that determines whether connector processing continues after record-processing errors.
- source connectorcontrasts with · A source connector is a Kafka Connect connector that reads data from an external source system and publishes records to Kafka.
- single message transformationuses · A single message transformation is a stateless Kafka Connect operation that independently modifies or filters each record passing through a connector.
- dead-letter topic patternuses · The dead-letter topic pattern redirects messages that cannot be processed to a separate topic for later investigation or handling.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.