Rate limit
A rate limit is a constraint on the maximum frequency or volume of operations permitted within a defined interval or capacity.
Build AI-Enhanced Web Apps
Throttling is the deliberate reduction or control of request processing or resource consumption to prevent overload.
- How it is used
- Protects AI application endpoints and paid provider capacity from excessive requests and abuse.
- How deep it goes
- Implements sliding-window middleware and a daily user quota, then recommends load testing and external protection services.
- Examples
- Uses a chat API with a five-request sliding window and a ten-message daily quota, plus an upload-quota proposal.
- What you can do after
- Apply it via Upstash Redis and @upstash/ratelimit, enforce a backend message quota, and test rejection under load.
Fast by Default
Rate limiting restricts the amount of work a client or identity may initiate within a defined period or concurrency bound.
- How it is used
- Protects API capacity by bounding client demand and preserving critical operations during traffic spikes or incidents.
- How deep it goes
- Explains token buckets, concurrency limits, shared Redis state, failure behavior, and operational measurement rather than implementation code.
- Examples
- Uses a Stripe-style payment API to contrast token buckets, concurrent-request caps, and shedding historical reads before charge creation.
- What you can do after
- Operate/configure it by setting request and concurrency thresholds, sharing client state, measuring overload signals, and defining shedding procedures.
Secure APIs
Rate limiting is the restriction of how many requests a client or identity can make during a defined interval.
- How it is used
- Applies rate limits to prevent brute-force attacks, control endpoint abuse, and respond to anomalous request spikes.
- How deep it goes
- Explains thresholds, middleware counters, monitoring, and targeted enforcement, while leaving production storage and full policy design to implementation choices.
- Examples
- Uses login-attempt counters, blocked IPs, locked accounts, endpoint request metrics, and a user-search scraping scenario.
- What you can do after
- Operate/configure API defenses by setting thresholds, instrumenting endpoints, investigating spikes, and applying targeted limits.
Generative AI in Action
Throttling is the deliberate slowing or limiting of requests or operations to control resource use and system load.
- How it is used
- Manages request frequency and resource consumption for production LLM applications.
- How deep it goes
- Distinguishes quotas from rate limits and gives quota-management practices and a bounded retry example.
- Examples
- Illustrates regional TPM allocation across Azure OpenAI deployments and a Tenacity-wrapped chat completion call.
- What you can do after
- Operate/configure deployment allocations and request patterns, and apply retries to handle rate-limit errors.
AI Governance
A rate limit is a configured bound on the number or frequency of requests permitted during a specified period.
- How it is used
- Contains runaway AI actions and request abuse by bounding activity over time or per session.
- How deep it goes
- Describes time-based action caps and complementary scope limits, without specifying enforcement algorithms.
- Examples
- Uses a refund cap and an accidental request loop to illustrate containment of errors and costs.
- What you can do after
- Configure rate limits alongside scope and spending controls to reduce the impact of runaway activity.
Build an AI Agent (From Scratch)
A rate limit is a provider-imposed bound on how many requests or operations a client may perform within a period or concurrently.
- How it is used
- Controls request concurrency during parallel LLM evaluation and informs error handling for custom agent tools.
- How deep it goes
- Demonstrates semaphore placement, provider-specific caps, retries, and tuning; leaves time-window quota enforcement unspecified.
- Examples
- Uses 100 concurrent tasks with a 10-call cap and separate OpenAI and Anthropic semaphores for model evaluation.
- What you can do after
- Apply it using Python’s asyncio.Semaphore and LiteLLM retries to manage concurrent API calls and transient failures.
Read how Build an AI Agent (From Scratch) explains rate limit →
Effective Data Analysis
Rate limiting is a restriction on the number or frequency of requests that a service accepts during a specified period.
- How it is used
- Controls automated API extraction during testing, pagination, and scheduled data pipelines.
- How deep it goes
- Explains request pacing and documentation review, but stops short of specifying universal limits or advanced throttling algorithms.
- Examples
- Uses repeated manual scripts, scheduled pipelines, and paginated forecast retrieval to illustrate when limits arise.
- What you can do after
- Operate/configure API collection by pacing requests, checking restrictions, and accounting for limits in pagination loops.
100 Go Mistakes and How to Avoid Them
CPU throttling is the deliberate reduction of processor execution capacity in response to resource limits, thermal conditions, or power policies.
- How it is used
- Diagnoses unexpectedly limited Go CPU execution in Docker and Kubernetes deployments.
- How deep it goes
- Explains GOMAXPROCS, CFS periods, quotas, and exhaustion scenarios, then stops at deployment guidance rather than prescribing a CFS-aware implementation.
- Examples
- Uses an eight-core node, a 4,000-millicpu limit, four busy threads, and 100 ms CFS periods to trace quota exhaustion.
- What you can do after
- Diagnose/debug with CFS periods, quotas, GOMAXPROCS, and deployment limits when Go workloads become CPU-throttled.
Read how 100 Go Mistakes and How to Avoid Them explains rate limit →
LLMs in Production
A rate limiter is a mechanism that restricts how many requests a client can make to a service within a specified time interval.
- How it is used
- Protects expensive LLM APIs from abuse, overload, bursty workloads, and unfair resource consumption.
- How deep it goes
- Distinguishes fixed-window, sliding-window-log, and token-bucket algorithms with practical trade-offs.
- Examples
- Uses a FastAPI endpoint configured with OAuth2 access keys, SlowAPI, and remote-address request limiting.
- What you can do after
- Operate/configure an LLM API with access-key validation and SlowAPI request limits.
AI Agents in Action, Second Edition
A rate limit is a constraint on how frequently or how many times a system may receive requests within a specified interval.
- How it is used
- Protects agent costs and reliability by limiting repeated or runaway calls to tools and external APIs.
- How deep it goes
- Frames rate limiting as a defense-in-depth safeguard but stops short of specifying algorithms, thresholds, or implementation steps.
- Examples
- Uses runaway agent loops, throttled external APIs, cached popular answers, and customer-support workflows as examples.
- What you can do after
- Operate/configure agent systems with ceilings that limit runaway activity and protect external-service costs and reliability.
Read how AI Agents in Action, Second Edition explains rate limit →
Grokking Software Architecture
A rate limiter restricts the number or frequency of operations permitted from a client or source during a defined interval.
- How it is used
- Protects server-side APIs from aggressive retries, abusive traffic, malicious requests, and overload.
- How deep it goes
- Places enforcement at the API Gateway and illustrates request dropping, but does not specify algorithms or configuration.
- Examples
- Uses one IP address making 500 login attempts in one minute and gateway defenses against denial-of-service traffic.
- What you can do after
- Apply rate limiting at an API Gateway to drop excessive requests before they reach backend services.
Read how Grokking Software Architecture explains rate limit →
Distributed Machine Learning Patterns
Rate limiting is a technique that restricts the number or frequency of requests a service processes within a specified interval or concurrency bound.
- How it is used
- Protects model serving APIs from accidental overload and denial-of-service attacks while controlling user traffic.
- How deep it goes
- Explains queue admission, concurrency limits, authentication-based policies, monitoring, and auditing without detailing a specific implementation.
- Examples
- Uses four requests under a two-concurrent-request limit and contrasts anonymous with authenticated access.
- What you can do after
- Operate model serving APIs with bounded concurrency and differentiated access limits.
Read how Distributed Machine Learning Patterns explains rate limit →
Designing AI Systems
Rate limiting is the controlled restriction of how many requests or tokens a system accepts from a client, workflow, or organization over a period of time.
- How it is used
- Protects organizational budgets and shared model capacity before requests enter provider or fallback workflows.
- How deep it goes
- Defines request, token, daily, concurrency, and scope dimensions but does not specify enforcement algorithms or storage.
- Examples
- Uses per-user, per-workflow, and global scopes alongside request-per-minute, token-per-minute, daily, and concurrency settings.
- What you can do after
- Operate/configure organizational consumption guardrails through a platform-level model service.
Microservices Patterns, Second Edition
A rate limiter pattern restricts the frequency or volume of calls made to a resource or dependency.
- How it is used
- Limits call frequency to help prevent overloading a remote service.
- How deep it goes
- Defines the pattern and its failure when the maximum rate is exceeded, but gives no algorithm or configuration.
- Examples
- Places the pattern in synchronous remote calls and API-composition provider proxies, without a worked rate-limiting scenario.
- What you can do after
- Apply it to limit how frequently a client calls a remote service.
Read how Microservices Patterns, Second Edition explains rate limit →
Pro ASP.NET Core 7, Tenth Edition
Rate limiting is the technique of restricting the number or frequency of requests handled during a specified interval or according to available capacity.
- How it is used
- Protects ASP.NET Core applications from excessive request traffic by limiting processing at endpoints or controllers.
- How deep it goes
- Explains four limiter types, shared options, middleware registration, named policies, and endpoint or controller application.
- Examples
- Demonstrates a one-request-per-15-seconds fixed window, zero queue, controller attributes, endpoint exemption, and a resulting 503.
- What you can do after
- Operate/configure named ASP.NET Core rate-limiting policies and verify limited and exempt routes through browser requests.
Read how Pro ASP.NET Core 7, Tenth Edition explains rate limit →
ScyllaDB in Action
A rate limit is a control that restricts the number of requests or database operations allowed within a specified interval.
- How it is used
- Protects ScyllaDB during hot-partition incidents by reducing upstream demand and keeping service available.
- How deep it goes
- Explains application throttling, per-partition limits, capacity testing, and approximate enforcement without detailing internal implementation.
- Examples
- Uses cassandra-stress measurements and an ALTER TABLE configuration for per-partition read and write limits.
- What you can do after
- Operate and configure traffic controls by tuning application throttles or ScyllaDB per-partition limits.
Starting Data Analytics with Generative AI and Python
A technique that restricts the frequency or volume of requests accepted by a service.
- How it is used
- Controls API-call frequency and volume while managing costs and plan limits in generative AI data-analytics integrations.
- How deep it goes
- Connects implementation guidance with monitoring, caching, error handling, and retries, but does not specify a limiting algorithm.
- Examples
- Applies rate limiting when processing large datasets and recovering from rate-limit errors.
- What you can do after
- Operate API or SDK integrations more reliably while controlling request usage and unexpected costs.
Read how Starting Data Analytics with Generative AI and Python explains rate limit →
Grokking Web Application Security
Rate limiting restricts the number or frequency of operations that a client can perform during a defined period.
- How it is used
- Protects login, sign-up, password-reset, and arbitrary external-API requests from rapid abuse.
- How deep it goes
- Explains progressive delays, IP-based application, lockout risks, shared-address problems, and distributed-attacker limitations.
- Examples
- Uses exponential backoff after incorrect login guesses and per-user limits for requests to arbitrary third-party URLs.
- What you can do after
- Operate rate limiting to slow simple brute-force attacks while balancing account lockouts, shared IP addresses, and distributed attackers.
Read how Grokking Web Application Security explains rate limit →
Get Programming with JavaScript Next: New features of ECMAScript 2015, 2016, and beyond
A rate-limiting technique that ensures a function is not invoked more often than allowed by a specified schedule.
- How it is used
- Limits rocket firing when a game loop checks for spacebar input on every rendered frame.
- How deep it goes
- Defines throttling as wrapping a function and demonstrates its use; timing intervals and implementation details are unspecified.
- Examples
- Demonstrates throttling a ship’s fire action while checking spacebar input during each frame.
- What you can do after
- Apply throttling by wrapping a class-property arrow function and invoking it from the update hook.
Also covered in
- Acing the CCNA Exam: Volume 2 Advanced Networking and Security
- Enterprise RAGIn early access (MEAP)
- Automating API Delivery
- Architecting for AutonomyIn early access (MEAP)
- The Art of Code
- Building LLM Applications with DSPy
- Claude at WorkIn early access (MEAP)
- Latency
- .NET in Action, Second Edition
- Build Financial Software with Generative AI (From Scratch)
- AI for Everyday IT
- Writing for Developers: Blogs that get read
What to know first
- contractA contract is an agreed specification of the conditions, expectations, rights, or obligations that govern behavior or interactions between parties or software components.
- authenticationAuthentication is the process of verifying the claimed identity of a user, service, device, or other entity.
- denial-of-service attackA denial-of-service attack is an attempt to make a system or service unavailable or degrade its ability to serve legitimate users by exhausting or disrupting its resources.
- language modelA language model is a computational model that represents patterns in linguistic sequences to assign probabilities, predict, generate, or evaluate language.
Related concepts
- resource limitcontrasts with · A resource limit is an enforced upper bound on the amount or rate of a resource that an entity may consume within a defined scope or period.
- API gatewayused by · An API gateway is a server-side intermediary that provides an entry point for client requests to backend APIs or services and routes or mediates those requests.
- denial-of-service attackaddresses · A denial-of-service attack is an attempt to make a system or service unavailable or degrade its ability to serve legitimate users by exhausting or disrupting its resources.
- brute-force attackaddresses · A brute-force attack systematically tries numerous possible credentials, keys, inputs, or combinations until one succeeds or the search space is exhausted.
Questions about rate limit
What is rate limit?
A rate limit is a constraint on the maximum frequency or volume of operations permitted within a defined interval or capacity.
Which Manning books explain rate limit?
Build AI-Enhanced Web Apps, Fast by Default, Secure APIs, Generative AI in Action, AI Governance, Build an AI Agent (From Scratch), Effective Data Analysis, 100 Go Mistakes and How to Avoid Them and 11 more.
What should I know before learning rate limit?
Manning books that teach rate limit usually introduce contract, authentication, denial-of-service attack and language model first.
What is the difference between rate limit and resource limit?
A rate limit is a constraint on the maximum frequency or volume of operations permitted within a defined interval or capacity. A resource limit is an enforced upper bound on the amount or rate of a resource that an entity may consume within a defined scope or period.
What kinds of rate limit are there?
Kinds of rate limit include fixed window.
What is rate limit also called?
Rate limit is also called rate limiting, throttling or throttle.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.


















