Source connector
A source connector is a Kafka Connect connector that reads data from an external source system and publishes records to Kafka.
Kafka Streams in Action, Second Edition
SourceConnector is the Kafka Connect connector type used to configure tasks that import records from an external system into Kafka.
- How it is used
- Imports external records into Kafka topics through configurable Kafka Connect plugins.
- How deep it goes
- Explains connector configuration, task generation, task partitioning, and polling, while deferring parts of the complete implementation.
- Examples
- Uses stock ticker API ingestion, JDBC PostgreSQL configuration, and masked retail sales transactions.
- What you can do after
- Build it from scratch by extending SourceConnector, defining configuration, and generating task configurations for external-source ingestion.
Read how Kafka Streams in Action, Second Edition explains source connector →
Kafka for Architects
A source connector is a Kafka Connect connector that reads data from an external source system and writes it to Kafka.
- How it is used
- Moves external-system data into Kafka topics through configured integration rather than a custom producer.
- How deep it goes
- Explains polling, task parallelism, source offsets, converters, and deployment through a worked JDBC configuration.
- Examples
- Uses customer address replication, reference tables, and a JDBC transaction-table configuration with an incrementing ID.
- What you can do after
- Operate/configure it by installing a plugin, setting source and progress parameters, and deploying a connector through the REST API.
Apache Kafka in Action
A source connector is a Kafka Connect connector that reads data from an external source system and publishes it to Kafka.
- How it is used
- Moves records from databases, files, or other external systems into Kafka through Kafka Connect.
- How deep it goes
- Defines the data flow and connector configuration, then addresses partitioning and task limits without detailing connector implementation internals.
- Examples
- Uses FileStreamSourceConnector, JDBC source connectors, database tables, sequential identifiers, and Debezium’s PostgresConnector.
- What you can do after
- Operate/configure ingestion through connector classes, task limits, and Kafka Connect’s management interfaces.
What to know first
- clientA client is a person, organization, device, or software component that receives or uses a product, service, resource, or functionality provided by another party or system.
- topicA topic is a named channel or category that groups messages or records for publication and delivery to interested consumers.
- consumer groupA consumer group is a logical set of message consumers that consume one or more streams under a shared group identity, with message delivery coordinated among its members.
- fault toleranceFault tolerance is the ability of a system to continue providing correct or acceptable service despite failures in its components or dependencies.
Related concepts
- connectorkind of · A connector is a software element or mechanism that links components, applications, or resources to enable or represent their interaction, communication, or data exchange.
- topicproduces · A topic is a named channel or category that groups messages or records for publication and delivery to interested consumers.
- converteruses · A converter is a component or operation that transforms data or values from one type or representation into another.
- pollinguses · Polling is a technique of repeatedly checking or querying a resource, system, or condition to detect information, events, or changes in state.
- schema evolutionuses · Schema evolution is the controlled modification of a data schema over time while managing compatibility with existing data, producers, consumers, or stored records.
Questions about source connector
What is source connector?
A source connector is a Kafka Connect connector that reads data from an external source system and publishes records to Kafka.
Which Manning books explain source connector?
Kafka Streams in Action, Second Edition, Kafka for Architects and Apache Kafka in Action.
What should I know before learning source connector?
Manning books that teach source connector usually introduce client, topic, consumer group and fault tolerance first.
What is source connector a kind of?
Source connector is a kind of connector. A connector is a software element or mechanism that links components, applications, or resources to enable or represent their interaction, communication, or data exchange.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


