A Kafka Connect sink connector may encounter errors while converting, transforming, or delivering records to an external system. errors.tolerance chooses whether processing stops or continues. section 5.5
Configure the failure policy deliberately:
- Set errors.tolerance to all to keep the connector running despite errors; review the connector logs to determine what failed. section 5.5
- Use none when an error should stop the connector, which can expose production problems quickly but interrupts processing. section 5.5
- Pair all with a dead-letter queue (DLQ) to retain records that Connect cannot deliver; enable context headers to preserve the failure reason. section 5.5
Listing 5.4 REST call to install an elastic connector
$ curl -i -X PUT localhost:8083/connectors/
student-info-elasticsearch-connector/config \
-H "Content-Type: application/json" \
-d ‘{
"connector.class":
"io.confluent.connect.elasticsearch.ElasticsearchSinkConnector",
"connection.url": "http://elasticsearch:9200",
"tasks.max": "1",
"topics": "postgres_orientation_students", #1
"type.name": "_doc",
"value.converter": "org.apache.kafka.connect.json.JsonConverter",
"value.converter.schemas.enable": "false",
"schema.ignore": "true",
"key.ignore": "false",
"errors.tolerance":"all", #2
"errors.deadletterqueue.topic.name":"orientation_student_dlq", #3
"errors.deadletterqueue.context.headers.enable": "true", #4
"errors.deadletterqueue.topic.replication.factor": "1" #5
}’
The Elasticsearch connector configuration shows errors.tolerance set to all alongside a DLQ topic and context headers. section 5.5
