Cost is the monetary expense of deploying or operating a model and its required components. Treat it as a nonfunctional requirement alongside latency, throughput, memory footprint, privacy, and production robustness. section 8.2.7 section 7.3.1
Assess cost before selecting a model:
- Inventory licensed subcomponents and other infrastructure dependencies that add operating expense. section 8.2.7
- Estimate the effect of execution volume; a model run millions of times can make infrastructure costs prohibitive. section 7.3.1
- Compare cost with latency, memory requirements, carbon impact, and functional performance rather than optimizing one measure in isolation. section 8.2.7 section 7.3.1
- Record the requirement during pre-project or presales work and test it before approving deployment. section 8.2.7
High-volume inference
A classifier that performs well functionally may still be rejected when millions of executions make its infrastructure cost prohibitive. A simpler or pretrained component can reduce project risk, but may trade away performance. section 7.3.1 section 8.2.7
