liveBook · Manning

Sink component

As explained in AI Model Evaluation

A sink component is a system component that captures outbound actions without delivering them to their external destination.

A sink component captures outbound actions without delivering them to their external destination. In shadow traffic, it lets an experimental system process realistic production requests while users receive responses only from the production system. section 4.3.3

Use a sink component to:

  • Compare experimental and production predictions without exposing users to poor recommendations, incorrect predictions, or experimental failures. section 4.3.3
  • Evaluate the complete request pipeline, including data retrieval, feature extraction, inference, and response formatting. section 4.3.3
  • Monitor realistic traffic patterns, edge cases, bottlenecks, and performance issues before switching users to the new system. section 4.3.3
Figure 4.2 illustrates how shadow traffic adds additional load to the supporting infrastructure such as feature database layer, metadata services, caches and user context lookups. This is one of the primary risks of shadow traffic that's worth over communicating so that the machine learning or platform infrastructure teams aren’t surprised by an increase in requests as a result of the shadow traffic evaluation.

Shadow traffic can increase load on feature databases, metadata services, caches, and user-context lookups, so infrastructure capacity requires explicit validation. section 4.3.3

How AI Model Evaluation covers it

How deep it goes
Explains sink behavior through model comparison, pipeline evaluation, and infrastructure risks, but does not specify implementation code.
Examples
Uses copied production requests, competing model responses, and supporting infrastructure such as feature databases, caches, and metadata services.
What you can do after
Evaluate experimental systems against production behavior without exposing users to their outputs.
How it connects
Connects shadow traffic with production systems, model evaluation, performance metrics, inference, feature extraction, and response formatting.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.