Network throttling deliberately limits shadow-traffic volume so infrastructure costs and production risk remain manageable while evaluation still gathers useful evidence. section 4.3.3
Apply throttling against explicit evaluation goals:
- Replicate only a subset of real traffic rather than sending every request to the shadow system. section 4.3.3
- Choose enough traffic to achieve statistical significance, cover important user segments, or exercise selected edge cases. section 4.3.3
- Combine throttling with smart sampling and capacity planning, because duplicate requests can strain production resources. section 4.3.3
A cost-aware shadow run
A team evaluates a new inference system using a sampled portion of live requests, throttles that stream, and runs shadow traffic on cheaper CPUs when GPU capacity is unnecessary. It validates that the reduced volume still covers target segments and edge cases. section 4.3.3
Figure 4.2 illustrates how shadow traffic adds additional load to the supporting infrastructure such as feature database layer, metadata services, caches and user context lookups. This is one of the primary risks of shadow traffic that's worth over communicating so that the machine learning or platform infrastructure teams aren’t surprised by an increase in requests as a result of the shadow traffic evaluation.
Inspect how shadow traffic increases load across feature databases, metadata services, caches, and user-context lookups. section 4.3.3



