5 Improving weak understanding for traditional AI
This chapter covers
- Identifying the types of errors a classifier can make
- Establishing a baseline of current classifier performance
- Using data science methodologies to identify and prioritize improvements
- Infusing your traditional AI with generated content to enhance understanding
In this chapter, we will demonstrate a methodical, iterative approach to improving the understanding of a classification-based conversational solution. This chapter builds on the concepts introduced in the previous chapter and uses the output produced by the final exercise in section 4.4 (where you created a test set with the golden intent assigned to each utterance in a format that can be used by your testing tool). Later in this chapter, we’ll explore how large language models can supplement intent-driven output responses to deliver a more robust experience. (If you’re looking for generative AI improvement techniques, feel free to skip ahead to the next chapter.)
We will start by building an improvement plan and identifying the types of errors your classifier may be committing. Next, we’ll iterate through seven improvement cycles to solve the various problems you might see in your own text classifier. Although data science techniques are used, you do not need to be a data scientist to extract meaningful insights about your data using the methodologies presented in this chapter.