Accuracy is a model-evaluation criterion: it asks whether outputs are factually correct or complete a task correctly. For language-model products, define it around user needs rather than treating it as a universal score. section 5.5 section 8.3 section 5.0
Evaluate accuracy in context:
- Use standardized benchmarks to gauge performance early, then create tailored metrics for the product’s specific needs. section 5.5
- For specialized tasks, build task-specific datasets and validate results in real-world applications with user and stakeholder feedback. section 8.3
- Track user satisfaction and adoption alongside model accuracy; technical performance alone does not establish product value. section 12.4
