Rate limit
A rate limit is a bound on the frequency or volume of operations permitted within a defined interval, scope, or concurrency limit.
Build AI-Enhanced Web Apps
Throttling is the deliberate reduction or control of request processing or resource consumption to prevent overload.
- How it is used
- Protects AI API endpoints from excessive requests and helps control abuse, backend load, and provider usage.
- How deep it goes
- Works through Redis-backed middleware and a separate authenticated message quota, including thresholds, counters, expiry, 429 responses, and load testing.
- Examples
- Uses a five-request sliding window, a load-test script, and a ten-message daily quota on /api/chat.
- What you can do after
- Apply it via @upstash/ratelimit and Upstash Redis to reject excess requests, then test the limit under load.
Secure APIs
Rate limiting is the restriction of how many requests a client or identity can make during a defined interval.
- How it is used
- Applies rate limits to prevent brute-force attacks, control endpoint abuse, and respond to anomalous request spikes.
- How deep it goes
- Explains thresholds, middleware counters, monitoring, and targeted enforcement, while leaving production storage and full policy design to implementation choices.
- Examples
- Uses login-attempt counters, blocked IPs, locked accounts, endpoint request metrics, and a user-search scraping scenario.
- What you can do after
- Operate/configure API defenses by setting thresholds, instrumenting endpoints, investigating spikes, and applying targeted limits.
Generative AI in Action
Throttling is the deliberate slowing or limiting of requests or operations to control resource use and system load.
- How it is used
- Positions rate limiting as a production API control for managing LLM request load and avoiding service disruption.
- How deep it goes
- Defines rate limits against quotas and gives operational practices and retry code, without detailing a server-side limiting algorithm.
- Examples
- Uses a 240,000-TPM deployment allocation and a Python Tenacity wrapper for chat-completion retries.
- What you can do after
- Operate/configure it by monitoring limits, pacing requests, retrying rate-limit errors, and adjusting deployment TPM allocations.
Build an AI Agent (From Scratch)
A rate limit is a provider-imposed bound on how many requests or operations a client may perform within a period or concurrently.
- How it is used
- Controls concurrent LLM requests during batch evaluation without abandoning parallel execution.
- How deep it goes
- Implements semaphore-guarded calls, retries, and provider-specific tuning; does not implement a time-window quota.
- Examples
- Uses 100 concurrent LLM tasks capped at ten calls and separate OpenAI and Anthropic semaphore settings.
- What you can do after
- Apply it via asyncio.Semaphore and LiteLLM retries, then adjust provider-specific concurrency when rate-limit errors appear.
Read how Build an AI Agent (From Scratch) explains rate limit →
Effective Data Analysis
Rate limiting is a restriction on the number or frequency of requests that a service accepts during a specified period.
- How it is used
- Controls automated API extraction during testing, pagination, and scheduled data pipelines.
- How deep it goes
- Explains request pacing and documentation review, but stops short of specifying universal limits or advanced throttling algorithms.
- Examples
- Uses repeated manual scripts, scheduled pipelines, and paginated forecast retrieval to illustrate when limits arise.
- What you can do after
- Operate/configure API collection by pacing requests, checking restrictions, and accounting for limits in pagination loops.
100 Go Mistakes and How to Avoid Them
CPU throttling is the deliberate reduction of processor execution capacity in response to resource limits, thermal conditions, or power policies.
- How it is used
- Diagnoses unexpectedly limited Go CPU execution in Docker and Kubernetes deployments.
- How deep it goes
- Explains GOMAXPROCS, CFS periods, quotas, and exhaustion scenarios, then stops at deployment guidance rather than prescribing a CFS-aware implementation.
- Examples
- Uses an eight-core node, a 4,000-millicpu limit, four busy threads, and 100 ms CFS periods to trace quota exhaustion.
- What you can do after
- Diagnose/debug with CFS periods, quotas, GOMAXPROCS, and deployment limits when Go workloads become CPU-throttled.
Read how 100 Go Mistakes and How to Avoid Them explains rate limit →
LLMs in Production
A rate limiter is a mechanism that restricts how many requests a client can make to a service within a specified time interval.
- How it is used
- Protects expensive LLM APIs from abuse, overload, bursty workloads, and unfair resource consumption.
- How deep it goes
- Distinguishes fixed-window, sliding-window-log, and token-bucket algorithms with practical trade-offs.
- Examples
- Uses a FastAPI endpoint configured with OAuth2 access keys, SlowAPI, and remote-address request limiting.
- What you can do after
- Operate/configure an LLM API with access-key validation and SlowAPI request limits.
AI Agents in Action, Second Edition
A rate limit is a constraint on how frequently or how many times a system may receive requests within a specified interval.
- How it is used
- Protects agent costs and reliability by limiting repeated or runaway calls to tools and external APIs.
- How deep it goes
- Frames rate limiting as a defense-in-depth safeguard but stops short of specifying algorithms, thresholds, or implementation steps.
- Examples
- Uses runaway agent loops, throttled external APIs, cached popular answers, and customer-support workflows as examples.
- What you can do after
- Operate/configure agent systems with ceilings that limit runaway activity and protect external-service costs and reliability.
Read how AI Agents in Action, Second Edition explains rate limit →
Distributed Machine Learning Patterns
Rate limiting is a technique that restricts the number or frequency of requests a service processes within a specified interval or concurrency bound.
- How it is used
- Protects model serving APIs from accidental overload and denial-of-service attacks while controlling user traffic.
- How deep it goes
- Explains queue admission, concurrency limits, authentication-based policies, monitoring, and auditing without detailing a specific implementation.
- Examples
- Uses four requests under a two-concurrent-request limit and contrasts anonymous with authenticated access.
- What you can do after
- Operate model serving APIs with bounded concurrency and differentiated access limits.
Read how Distributed Machine Learning Patterns explains rate limit →
Pro ASP.NET Core 7, Tenth Edition
Rate limiting is the technique of restricting the number or frequency of requests handled during a specified interval or according to available capacity.
- How it is used
- Protects ASP.NET Core applications from excessive request traffic by limiting processing at endpoints or controllers.
- How deep it goes
- Explains four limiter types, shared options, middleware registration, named policies, and endpoint or controller application.
- Examples
- Demonstrates a one-request-per-15-seconds fixed window, zero queue, controller attributes, endpoint exemption, and a resulting 503.
- What you can do after
- Operate/configure named ASP.NET Core rate-limiting policies and verify limited and exempt routes through browser requests.
Read how Pro ASP.NET Core 7, Tenth Edition explains rate limit →
ScyllaDB in Action
A rate limit is a control that restricts the number of requests or database operations allowed within a specified interval.
- How it is used
- Protects ScyllaDB during hot-partition incidents by reducing upstream demand and keeping service available.
- How deep it goes
- Explains application throttling, per-partition limits, capacity testing, and approximate enforcement without detailing internal implementation.
- Examples
- Uses cassandra-stress measurements and an ALTER TABLE configuration for per-partition read and write limits.
- What you can do after
- Operate and configure traffic controls by tuning application throttles or ScyllaDB per-partition limits.
Starting Data Analytics with Generative AI and Python
A technique that restricts the frequency or volume of requests accepted by a service.
- How it is used
- Controls API-call frequency and volume while managing costs and plan limits in generative AI data-analytics integrations.
- How deep it goes
- Connects implementation guidance with monitoring, caching, error handling, and retries, but does not specify a limiting algorithm.
- Examples
- Applies rate limiting when processing large datasets and recovering from rate-limit errors.
- What you can do after
- Operate API or SDK integrations more reliably while controlling request usage and unexpected costs.
Read how Starting Data Analytics with Generative AI and Python explains rate limit →
Grokking Web Application Security
Rate limiting restricts the number or frequency of operations that a client can perform during a defined period.
- How it is used
- Protects login, sign-up, password-reset, and arbitrary external-API requests from rapid abuse.
- How deep it goes
- Explains progressive delays, IP-based application, lockout risks, shared-address problems, and distributed-attacker limitations.
- Examples
- Uses exponential backoff after incorrect login guesses and per-user limits for requests to arbitrary third-party URLs.
- What you can do after
- Operate rate limiting to slow simple brute-force attacks while balancing account lockouts, shared IP addresses, and distributed attackers.
Read how Grokking Web Application Security explains rate limit →
Get Programming with JavaScript Next: New features of ECMAScript 2015, 2016, and beyond
A rate-limiting technique that ensures a function is not invoked more often than allowed by a specified schedule.
- How it is used
- Limits rocket firing when a game loop checks for spacebar input on every rendered frame.
- How deep it goes
- Defines throttling as wrapping a function and demonstrates its use; timing intervals and implementation details are unspecified.
- Examples
- Demonstrates throttling a ship’s fire action while checking spacebar input during each frame.
- What you can do after
- Apply throttling by wrapping a class-property arrow function and invoking it from the update hook.
Related concepts
- resource limitcontrasts with · A resource limit is an enforced or configured maximum on the amount or rate of resource consumption or service activity within a specified scope.
- API gatewayused by · An API gateway is an intermediary that provides an entry point for client requests to backend APIs or services and routes or otherwise mediates those requests.
- denial-of-service attackaddresses · A denial-of-service attack is an attempt to make a system or service unavailable or degrade its ability to serve legitimate users by exhausting or disrupting its resources.
- brute-force attackaddresses · A brute-force attack systematically tries numerous possible credentials, keys, inputs, or combinations until one succeeds or the search space is exhausted.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.