An AI recommendation feature can look successful at launch and still lose customers when it starts giving nonsensical advice. Incorrect examples in training data are one possible cause. Validation checks assumptions, data, inputs, outputs, and user reactions at different points; no single check establishes that the product is trustworthy. section 11.1.1 section 10.4
Choose the check that fits the risk and stage of work:
- Before committing, validate the opportunity through research when its value is uncertain or mistakes could cause serious harm. A cheap prototype in a fast-moving setting can instead test assumptions through real-world feedback. section 2.3.3
- Before training, check incoming data for anomalies and inconsistencies, use trusted sources, and filter suspicious or outlier records. After deployment, investigate unusual performance drops or unexpected outputs as possible signs of data poisoning. section 11.1.1
- At the application boundary, validate user queries for formatting and harmful commands. For structured language-model outputs, validate generated data against a predefined schema; Pydantic is one post-generation option. section 11.1.3 section 5.2.3
- With users, validate prototypes that simulate uncertain AI behavior and failures. Observe responses and error tolerance, then keep collecting feedback after launch to find situations the team missed. section 10.4
Figure 5.10 LMs can be used for offline processing, which allows running additional validation of their outputs.
Offline language-model processing leaves room to run additional checks on generated outputs before they are used. section 5.2.3
Validation is a continuing practice, not a launch gate. Schema checks address output format, while data checks, user feedback, and production monitoring expose different failures. Match the effort to the potential harm and keep investigating signals that customer trust is slipping. section 5.2.3 section 11.1.1 section 2.3.3
