A machine learning system is more than a trained model. It combines data, models, and the code and infrastructure that put them to work in a business domain. Data structure and quality affect the models; measured model properties must, in turn, inform system design. Business and ethical requirements belong in those decisions. section 1.2
A tested model cannot deliver value while it sits in a repository. To put it into production, provide data infrastructure that preserves the data environment it was trained for, and infrastructure that retrieves and runs it within the relevant business processes. section 9.0
Before release, make the system-level decisions explicit:
- Choose whether the system is assistive, delegative, or autonomous, and identify the functional and nonfunctional requirements that follow. section 9.11
- Build production data flows, model serving, a suitable user interface, and controls for users. section 9.11
- Arrange logging, monitoring, alerting, user and integration testing, ethical review, and production sign-off. Plan for work after release. section 9.11
An uncertain starting point
A bike retailer has sales and inventory systems and wants to bring their data into one cloud database. A dashboard has a plausible business case, but the value of ML is not yet clear. The project must establish what the data can support, how people will use the result, and which models warrant production integration. section 1.5
Table 10.2 Create and maintain a list of your responsibilities, ensure that your manager or supervisor has access to it, and go over it with them to make sure they agree that it’s accurate. (view table figure)
| System name |
Owner |
Who manages |
Notes |
|---|---|---|---|
| Inventory manager |
Karima |
You |
Reviewed and good 03/22 |
| Demand prediction |
Karima |
You |
Reviewed and good 03/33 |
| Building manager |
Alan |
David |
David isn’t sure about this |
| Auto reply |
Alan |
You |
No governance |
| Inspection |
Josep |
David |
David agreed |
| Entity matcher |
Alan |
You |
No governance |
| Auto review |
Alan |
You |
Last reviewed 01/20 |
Use a system inventory to settle responsibility after deployment: record each system’s business owner, who manages it, and notes requiring follow-up; review the list with management. section 10.2.1
