User experience measures whether a chatbot interaction satisfies users, keeps them engaged, helps them learn, and enables them to accomplish goals. Unlike NLU accuracy, UX cannot be reduced to one mathematical score: message counts and user totals may show activity without showing a positive experience. section 12.7.1 section 12.7.3
Measure chatbot UX by translating interface events and organizing them with the HEART framework:
- Track events such as starting an interaction, asking a question, or saying thank you. section 12.7.3
- Use HEART's five metric families: happiness, engagement, adoption, retention, and task success. section 12.7.3
- Define adoption carefully: for an education bot, count students who exchange several messages rather than merely subscribing. section 12.7.3
A math education chatbot
A signup alone may indicate outreach success but not meaningful use. Counting students who proceed through several messages gives adoption a closer connection to actual interaction. section 12.7.3
Figure 12.7 Sankey diagram of lesson funnels
A Sankey diagram can visualize lesson funnels and show how users move through stages of a chatbot journey. section 12.7.3
