liveBook · Manning

Latency degradation experiment

As explained in AI Model Evaluation

A controlled evaluation that deliberately increases or varies response latency to measure effects on system, user, product, or business outcomes.

A latency degradation experiment measures how slower response times affect users and product outcomes. In this controlled A/B test, a control group receives no added delay while test groups experience different delays. The results help teams make evidence-based decisions about slower models, features, and performance improvements. section 4.4 section 4.4.1

Design the experiment around the user experience, not latency alone:

  • Establish a baseline for end-to-end response time and break it down by stages such as feature retrieval, inference, network transfer, and rendering. section 4.4.2
  • Add controlled delays to test variants and retain a no-delay control. Keep infrastructure and resources close to production conditions so resource constraints are not mistaken for the effects of the injected delay. section 4.4.1
  • Measure system performance alongside user and product metrics. Page-load percentiles show how the experience changed; clicks, abandonment, and downstream consumption show how users responded. section 4.4.2 section 4.4.3
  • Limit exposure and duration, define guardrail metrics and stop conditions in advance, and obtain stakeholder approval. The test intentionally worsens the experience for some users. section 4.4
  • Use the results to decide whether to optimize a slower feature, accept the trade-off against business goals, or prioritize performance improvements. section 4.4.1

Movie recommendations

A recommendations page has a current p99 of 900 ms. An experiment injects delay in the personalization backend closest to the client, with a control and five delayed variants. It tests whether slower loading reduces clicks, scrolling, and movie consumption. Track p50, p95, and p99 load times, click-through rate, bounce rate, abandonment, and downstream consumption; replicate the test by device type because latency effects may differ. section 4.4.3

Table 4.4 Latency Degradation Test Results on Mobile
Variant Cell
Added Latency
Avg Page Load Time
CTR
Bounce Rate
Control
0ms
1050ms
23.5%
27.8%
Test Variant 1
100ms
1155ms
22.7%
29.1%
Test Variant 2
200ms
1250ms
20.2%
32.7%
Test Variant 3
300ms
1360ms
16.4%
39.5%
Test Variant 4
400ms
1455ms
12.2%
48.3%
Test Variant 5
500ms
1552ms
8.7%
56.9%

Compare the mobile variants: as added delay rises from 0 to 500 ms, average page load time and bounce rate increase while click-through rate falls. section 4.4.3

The measured response helps locate latency levels that may threaten product goals, but the decision still requires judgment: technical speed alone does not establish whether a feature delivers enough value to justify a slower experience. section 4.4 section 4.4.1 section 4.4.3

How AI Model Evaluation covers it

How deep it goes
Provides experiment variants, environment checks, metrics, safeguards, and a worked product example.
Examples
Tests a movie recommendations page with injected backend delays and compares mobile load time, click-through rate, and bounce rate.
What you can do after
Evaluate latency trade-offs and prioritize optimizations using measured system and product outcomes.
How it connects
Connects A/B testing, end-to-end latency, model inference, and user engagement.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.