liveBook · Manning

CPU time

As explained in Apache Kafka in Action

CPU time is the amount of processor execution time consumed by a client's requests on a broker.

CPU time measures how much broker processor capacity a client’s requests consume. Kafka exposes it as a request quota, alongside producer and consumer byte-rate quotas. section 13.3

Kafka enforces CPU-time quotas separately on each broker. Because usage is calculated per thread, available capacity equals the combined network and I/O thread capacity multiplied by the configured percentage. section 13.3

Use CPU-time quotas with these operational constraints in mind:

  • Prefer user-level quotas when identities must be enforceable; clients can arbitrarily choose their client IDs. section 13.3
  • Treat quotas as overload protection rather than billing guarantees, because a client can connect to multiple brokers and exceed its per-broker allocation overall. section 13.3
  • Start with a conservative default and grant higher limits only to clients that need additional resources; monitor the result because poorly managed quotas can cause problems. section 13.3
((num.io.threads + num.network.threads) * 100)%

The capacity expression makes the per-thread basis of CPU-time quotas explicit: network and I/O thread counts determine the available request percentage. section 13.3

Protecting an online-shop cluster

A default policy can allocate every client a 100% request rate, while limiting producers to 1 MB/s and consumers to 2 MB/s. A periodic newsletter producer can receive a stricter 50% request quota, and a warehouse consumer can receive a higher 200% quota when its workload requires it. section 13.3

How Apache Kafka in Action covers it

How deep it goes
Defines broker and thread-based enforcement, explains limitations, and demonstrates default, client-specific, and user-specific quotas.
Examples
Uses online-shop defaults, a newsletter producer restriction, a warehouse consumer allowance, and a network/I/O thread formula.
What it assumes
Assumes familiarity with Kafka clients and brokers while explaining client IDs, user principals, threads, and quota scope.
What you can do after
Operate/configure Kafka quotas that reserve request capacity and reduce overload risk for selected clients or users.
How it connects
Connects CPU time with quotas, brokers, client IDs, users, producers, consumers, and request processing.

Also explained in

Compare how 4 books explain CPU time →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Apache Kafka in Action. The section links point to the parts of the book it draws on.