A misconfigured producer can flood a Kafka cluster with data; an uncontrolled consumer can overwhelm brokers with requests. Quotas limit client activity to protect other users and distribute resources more fairly. section 13.3
Choose a limit and an identity to apply it to:
- Set produce and consume byte rates, or limit the broker CPU time allocated to a client. section 13.3
- Start with a conservative default, then give clients that need more capacity specific quotas. Where authentication is available, prefer user-level quotas: clients can choose their own client IDs. section 13.3
- Remember that enforcement is per broker. A client connected to more brokers can exceed its nominal quota across the cluster. section 13.3
kafka-configs.sh --bootstrap-server localhost:9092 \
--alter \
--entity-type clients \
--entity-default \
--add-config 'producer_byte_rate=1048576,\
consumer_byte_rate=2097152,request_percentage=100'
This `kafka-configs.sh` command sets default client quotas: 1 MiB/s for producing, 2 MiB/s for consuming, and a request-percentage limit of 100. section 13.3
kafka-configs.sh --bootstrap-server localhost:9092 \
--alter \
--entity-type users \
--entity-name warehouse \
--add-config 'consumer_byte_rate=5242880,request_percentage=200'
A user-specific command gives the `warehouse` principal its own consume-rate and request-percentage limits. section 13.3
Monitor throttle time after setting quotas. Limits that are too aggressive can disrupt service or cause cascading failures. On managed Kafka, also watch provider-specific limits, such as network bandwidth, because hitting them can cause throttling. section 13.4 section 15.5.5
