Degree of innovation sets how novel and ambitious the solution should be, balancing business objectives against the cost and risk of pursuing new methods. section 3.0 section 3.4
Use three practical performance buckets to calibrate the expected innovation:
- Minimum viable ML system: use the simplest and fastest known solution as a baseline or prototype; expect it to mostly work and expose failure modes. section 3.4
- Average human-level ML system: pursue meaningful research and innovation to reach a human-level performance bar. section 3.4
- Best-in-class ML system: invest in maximum performance when small accuracy gains materially affect competitive position, profits, or losses. section 3.4
Competitive domains
Trading, adtech, and search may justify best-in-class ambitions because even a small accuracy shift can produce millions in profit or loss; other systems may not need that level of innovation. section 3.4
