11 Building your own classifier

 

This chapter covers

  • Constructing a simple text classifier from the ground up
  • Appraising and critiquing a classifier based on what it learns from a training cycle
  • Seeing how each training example affects the classifier’s learning
  • Building your own classifier

In previous chapters, we learned about how to design and construct an AI assistant, and how to make sure it responds appropriately to the user. AI assistants use classifiers to extract meaning (as an intent) from a user’s natural language input, as shown in figure 11.1. (Some AI platforms refer to the classifier as the natural language understanding [NLU] component.) The intent detected by the classifier is used by the assistant to determine the best response. Thus, getting the right intent from user input is a key part of a well-functioning AI assistant.

Figure 11.1 Orientation for this chapter. How does the classifier work? How can you make it better?

We have briefly discussed that AI assistants are trained, but we have not discussed how that training works. In this chapter, you will learn how to train your own classifier. We will build a simple text classifier for Fictitious Inc.’s conversational AI assistant.

11.1 Why build your own classifier?

11.1.1 Classification is a differentiator

11.1.2 Classification is a core competency

11.1.3 Traceability

11.1.4 To learn

11.1.5 Build or buy?

11.2 Build a simple classifier from first principles

11.2.1 The simplest text classifier

11.2.2 The mathematics behind a simple classifier

11.3 Expanding the simple classifier

11.3.1 Predicting more than one class

11.3.2 An all-in-one classifier

11.3.3 Comparing binary classifiers to all-in-one classifiers

11.4 Extending even further