Rate limiting is a platform-side guardrail that restricts request or token consumption before traffic reaches model providers. It protects budgets and shared capacity from runaway jobs, bot attacks, load tests, and fallback chains that would otherwise shift spending between providers. section 3.6 section 1.1.1
Choose limits according to the resource and failure mode being controlled:
- Per-user limits prevent one user from monopolizing capacity. section 3.6
- Per-workflow limits prevent one application from starving other workflows of shared provider access. section 3.6
- Global limits impose an organization-wide ceiling across users and workflows. section 3.6
- Request limits control call volume; token limits account for expensive, large-context requests. section 3.6
- Concurrency limits restrict how many requests can run simultaneously, complementing rate limits during traffic spikes. section 3.6 section 1.1.1
Listing 3.14 RateLimitConfig dataclass for organizational protection
@dataclass
class RateLimitConfig:
requests_per_minute: int = None #A
tokens_per_minute: int = None #B
requests_per_day: int = None #C
max_concurrent_requests: int = None #D
scope: RateLimitScope = RateLimitScope.WORKFLOW #EUse the configuration model to express request, token, daily, concurrency, and scope constraints in one platform boundary. section 3.6


















