5 Building a successful AI assistant

 

This chapter covers

  • Avoiding the most common AI assistant failures
  • Selecting appropriate success metrics for your assistant
  • Knowing which metrics are commonly used for which types of assistants
  • Instrumenting your assistant to measure the right metrics

So far in this book, we have covered why you would build an AI assistant (chapter 1) and how to build an AI assistant (chapters 2–4). In this chapter, we take a different approach, examining the practices that make AI assistants succeed. Specifically, we will focus on how to tell whether an AI assistant is making a positive impact on your bottom line.

When I started writing this chapter, I typed why do AI projects fail in a search engine. There was no shortage of relevant results! Avoiding failure is clearly important when creating an AI assistant. But we should aim higher than not failing and consider how to make our AI projects succeed.

In chapters 1–4, we used Fictitious Inc. as a motivating example. Fictitious Inc. is trying to improve its customer service department by using a conversational AI assistant; the company is getting overloaded with routine customer service questions. It heard that many AI ventures fail and doesn’t want to be one of them.

5.1 AI assistant use cases

5.2 Conversational AI success metrics

5.2.1 Containment

5.2.2 Time to resolution

5.2.3 Net promoter score

5.2.4 Coverage

5.2.5 Instrumenting your conversational AI

5.3 Command interpreter success metrics

5.3.1 Usage

5.3.2 Stickiness

5.3.3 Instrumenting your command interpreter

5.4 Event classifier success metrics

5.4.1 Time to resolution

5.4.2 Number of hand-offs

5.4.3 Other customer satisfaction metrics