A metric is a quantitative measure of a conversational system’s performance or outcomes. Choose measures that reflect business goals and user satisfaction, not merely those that are easiest to collect. section 3.6
Metrics help reveal where an experience is falling short:
- In an insurance claim flow, rising opt-outs and support escalations alongside falling self-service completions exposed a burdensome question. Moving the fax offer until after claim details were read addressed the problem. section 8.1.2
- After deploying fixes, PharmaBot’s team compared failure outcomes with those from the previous two deployments to check whether the changes worked. section 3.5.2
Figure 3.20 Tracking conversation outcomes against the deployment of changes
The dashboard tracks conversation outcomes alongside changes in deployment. section 3.5.2
