liveBook · Manning

Network throttling

As explained in Kafka Streams in Action, Second Edition

Throttling is the deliberate limitation of request or operation frequency by delaying execution between attempts.

Network throttling deliberately spaces requests or operations apart. In a Kafka Connect SourceTask, it keeps calls to an external stock-service API within provider limits while allowing the worker to remain responsive. section 5.6.1

A throttled source task follows this pattern:

  • When no records are available, wait briefly and return null so the worker can pause or shut down. section 5.6.1

Stock ticker source

A StockTickerSourceTask uses timeBetweenPoll to limit requests to a stock API, protecting a plan with request-frequency or daily quotas while loading returned records into Kafka. section 5.6.1

How Kafka Streams in Action, Second Edition covers it

How deep it goes
Explains interval calculation, sleeping, empty-result handling, and worker responsiveness without broader rate-limit strategies.
Examples
Uses a StockTickerSourceTask that polls a stock-service API and returns SourceRecord data.
What it assumes
Assumes familiarity with SourceTask polling and external APIs while demonstrating the throttling logic directly.
What you can do after
Operate a throttled SourceTask that spaces API calls and remains interruptible when no records exist.
How it connects
Connects throttling with Kafka Connect, SourceTask, polling, external APIs, and HTTP requests.
Cover of Kafka Streams in Action, Second Edition
liveBookmanning.com
Read Kafka Streams in Action, Second Edition on liveBook Get the book at manning.com

Also explained in

Compare how 2 books explain network throttling →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Kafka Streams in Action, Second Edition. The section links point to the parts of the book it draws on.