chapter three

3 AI, automation, and testing

 

This chapter covers

  • Outlining the value of good testing
  • Appreciating how tools help in testing
  • Identifying when AI tools could be of use in testing

Before we begin our journey into the use of LLMs in testing, we need to take a short detour and ask ourselves the following questions:

  • What is the purpose and value of testing?
  • How can tooling help us what that purpose?
  • When is it appropriate to use AI tools?

It may feel like asking these fundamental questions is an unnecessary activity. But if you are someone who sees testing merely as a confirmatory exercise, as in executing test cases to confirm a requirement is correct, then your mileage from the subsequent chapters will be limited. Understanding the value and performance of testing is critical for determining how tools can be used in a useful way. So in this chapter, we’re going to explore why a deeper understanding of testing can help us to utilize tools effectively. That said, if you are someone who already has that deep understanding of testing, feel free to skim this chapter and move on. For the rest of us, let’s go back to square one by asking the question, why do we test?

3.1 The value of testing

To help us appreciate having a strong foundation of why we need testing in software development, let’s return to the common perspective that testing is a confirmatory exercise. By this we mean that testing is viewed as something that is done to confirm these conditions:

3.1.1 A different way of thinking about testing

3.1.2 A holistic approach to testing

3.2 How tools help with testing

3.2.1 Automation bias

3.2.2 Being selective with tooling

3.3 Knowing when to use LLMs in testing

3.3.1 Generative capabilities

3.3.2 Transformation capabilities

3.3.3 Enhancing capabilities

3.3.4 LLMs in use in testing

3.4 Summary