An integration test checks whether interacting components work together correctly through their interfaces. In an ML system, this includes the model-serving infrastructure, data infrastructure, user experience, and any application elements connected to the model. section 9.6
Use integration testing as an explicit pre-release delivery task:
- Catalog existing unit and system tests, then identify production components that still lack coverage. section 9.6
- Design tests for the implemented system as a whole, including the production database and feature-extraction code. section 9.6
- Test data infrastructure, user experience, model serving, and other application elements integrated with the ML system. section 9.6
- Automate the preproduction test plan, run it successfully, obtain approval, and secure sign-off before release. section 9.6 section 9.11
Confidential model integration pipeline
When sensitive medical images or models cannot be broadly shared, keep integration testing inside the appropriate security enclave. A separate nonconfidential pipeline can use artificial data and a dummy model, while confidential testing exercises the UI and supporting code against the protected model. Prefer promoting components into the confidential environment before running system and integration tests, so more integration work occurs before production release. section 8.2.2
Figure 8.1 Three different flows to production. Option (a) provides a single secure environment, where artifacts are promoted from dev to test and then to prod. With (b), there is a separate confidential flow that is not accessible from some other environments, and components are promoted into prod from both the secured and confidential environments. A more desirable setup is (c), where we do integration testing after the components are promoted into the confidential environment.
Compare single-environment, separate confidential-flow, and promotion-based testing pipelines before choosing how artifacts move from development through testing to production. section 8.2.2
