3 Artificial intelligence, automation, and testing

Before we delve deeper into the use of large language models (LLMs) in testing, let’s ask ourselves the following questions:

It may seem that asking these fundamental questions is unnecessary. But if you are someone who sees testing merely as a confirmatory exercise, as in executing test cases to confirm a requirement is correct, then your mileage from the subsequent chapters will be limited. Understanding the value and performance of testing is critical for determining how tools can be used effectively. Thus, in this chapter, we’re going to explore why a deeper understanding of testing can help us utilize tools. That said, if you are someone who already has that deep understanding, feel free to skim through this chapter and move on. For the rest of us, let’s go back to square one by asking why we test.

3.1 The value of testing

To help us appreciate having a clear understanding of why we need testing in software development, let’s return to the common perspective that testing is a confirmatory exercise. By this, we mean that testing is viewed as something done to confirm the following conditions:

3.1.1 A different way of thinking about testing

3.1.2 A holistic approach to testing

3.2 How tools help with testing

3.2.1 Automation bias

3.2.2 Being selective with tooling

3.3 Knowing when to use LLMs in testing

3.3.1 Generative capabilities

3.3.2 Transformation capabilities

3.3.3 Enhancing capabilities

3.3.4 LLMs in use in testing

Summary