16 Testing and measuring performance in voice systems
This chapter covers
- Measuring the metrics that matter for voice system performance
- Testing and reporting system performance
- Collecting valid user data
- Discussion of additional system testing pre- and post-rollout
So you’ve built your voice system and applied everything you’ve learned. Now you’re ready for the rest of the world to flock to your implementation. How do you bring it to the world without it breaking in some horrid way? Will you know if people are successful interacting with your VUI or like talking to it? If you followed all the ‘rules,’ why do you even need to worry about any of this?
This chapter explores how to test voice systems and measure their performance in terms of both system accuracy and usability. The information and metrics you capture feed directly into your ability to improve system performance (chapter 17).
16.1 Testing voice system performance
All software needs testing; voice systems are no different. But the details of how you perform testing on a voice system, and across voice development platforms, differ from what you might be used to if you’ve tested non-voice interfaces and have their own special challenges.