liveBook · Manning

Throughput

As explained in Fast by Default

Throughput is the number of requests or other work items a service handles per unit of time.

Throughput measures how many requests a service handles per second. It complements response speed: a service may respond in 50ms yet bottleneck if it handles only 10 requests per second. section 3.2.2

Use throughput to turn service capacity into an operational target:

  • Measure requests per second alongside response-time percentiles and error rates rather than optimizing speed alone. section 3.2.2
  • Set targets from peak traffic patterns and preserve headroom, typically two to three times normal peak traffic, for unexpected surges. section 3.2.2
  • Connect the metric to user-facing outcomes: insufficient capacity can make pages, applications, dashboards, or checkout flows feel unreliable or unresponsive. section 3.2.2 section 3.6

Capacity under peak load

A service handling 500 requests per second at peak has a concrete throughput baseline. Compare that baseline with expected peak traffic and planned headroom, then monitor it with latency and error-rate data. section 3.2.2

Table 3.4 Backend performance metrics
Metric
Measures
Example
Response time percentiles (e.g. p50, p95, p99)
How fast do requests complete across the distribution of requests
“p95 of 200ms” means 95% of requests complete within 200ms
Throughput
Requests handled per second
Service handles 500 requests per second at peak
Error rate
Percentage of requests that fail
0.05% of payment requests return errors

Use the backend performance metrics table to compare throughput with response-time percentiles and error rate in a single capacity view. section 3.2.2

How Fast by Default covers it

How deep it goes
Defines throughput, contrasts it with response speed, and specifies peak-load targets with surge headroom.
Examples
Uses a service handling 500 requests per second at peak and contrasts fast responses with insufficient request capacity.
What it assumes
Assumes basic awareness of backend services and requests while defining throughput alongside related performance metrics.
What you can do after
Evaluate backend capacity by measuring requests per second against peak traffic, response speed, errors, and planned headroom.
How it connects
Connects throughput with response-time percentiles, error rates, capacity planning, and peak traffic.

Also explained in

Compare how 32 books explain throughput →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Fast by Default. The section links point to the parts of the book it draws on.