Kafka Connect worker
A Kafka Connect worker is a process that manages connectors and runs their tasks within a Kafka Connect cluster.
Kafka Streams in Action, Second Edition
A Connect worker is a Kafka Connect process that runs connector instances and their tasks.
- How it is used
- Runs connector instances and tasks while supplying shared Connect server settings for local or distributed deployments.
- How deep it goes
- Explains worker configuration, converters, plugins, and deployment modes, but stops short of worker internals.
- Examples
- Uses Docker setup, converter configuration, standalone and distributed diagrams, and a JDBC source connector REST request.
- What you can do after
- Operate and configure a Kafka Connect worker for local development or distributed production deployment.
Read how Kafka Streams in Action, Second Edition explains Kafka Connect worker →
Apache Kafka in Action
A Kafka Connect worker is a process in a Kafka Connect cluster that manages connectors and runs their tasks.
- How it is used
- Connects external databases, files, and messaging systems to Kafka through managed connectors and distributed tasks.
- How deep it goes
- Defines worker responsibilities, distributed coordination, configuration, scaling, internal topics, and REST-based status checks.
- Examples
- Uses a twelve-table import with four tasks, worker.properties, the connect-distributed.sh command, and connector status JSON.
- What you can do after
- Operate/configure workers in Distributed Mode and diagnose connector or task placement through the REST API.
Read how Apache Kafka in Action explains Kafka Connect worker →
Related concepts
- Kafka Connectpart of · Kafka Connect is a framework for reliably transferring data between Apache Kafka and external systems through configurable source and sink connectors.
- distributed modeused by · Distributed mode is a Kafka Connect operating mode in which multiple workers coordinate across a cluster to run connector tasks and store internal data.
- taskhas part · A Kafka Streams task is an independently executable processing unit that processes assigned input partitions through a topology and its associated state stores.
- heartbeatuses · A heartbeat is a periodic signal or message indicating that a component or system remains active and reachable.
- group coordinatorused by · A group coordinator is the Kafka broker responsible for managing a consumer group’s membership, liveness, partition assignments, and committed offsets.
- workerkind of · A worker is a person or computational execution unit assigned to perform a task as part of a larger system or activity.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.