liveBook · Manning

Statefulness

As explained in AI Model Evaluation

A stateful operation is an operation whose effects depend on or modify persistent application or external system state.

Statefulness means that an operation depends on or changes persistent application or external-system state. It makes duplicate evaluation requests risky because they can repeat writes, trigger workflows, or create user-visible side effects. section 4.3.3

Protect stateful systems during shadow-traffic evaluation by:

  • Routing writes to isolated databases or preventing shadow requests from modifying shared state. section 4.3.3
  • Using dry-run adapters, sandboxed APIs, read-only credentials, recorded responses, or mock outputs for external tools. section 4.3.3
  • Intercepting emails and notifications, while preserving production security and data-retention controls. section 4.3.3
Figure 4.2 illustrates how shadow traffic adds additional load to the supporting infrastructure such as feature database layer, metadata services, caches and user context lookups. This is one of the primary risks of shadow traffic that's worth over communicating so that the machine learning or platform infrastructure teams aren’t surprised by an increase in requests as a result of the shadow traffic evaluation.

Shadow traffic can add load across feature databases, metadata services, caches, and user-context lookups, even when state changes are blocked. section 4.3.3

How AI Model Evaluation covers it

How deep it goes
Explains duplicate writes, external side effects, isolation, interception, and resource overhead without detailing implementation code.
Examples
Uses database writes, emails, notifications, tickets, payments, calendars, and workflow-triggering APIs as concrete side-effect examples.
What you can do after
Operate shadow evaluations with isolated stores, dry-run adapters, sandboxed APIs, read-only credentials, or intercepted effects.
How it connects
Connects statefulness to shadow traffic, databases, external APIs, security, privacy, and user-visible effects.

Also explained in

Compare how 15 books explain statefulness →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of AI Model Evaluation. The section links point to the parts of the book it draws on.