chapter two

2 Specification-based testing

 

This chapter covers

  • How to perform specification-based testing and create test cases based on the specification
  • What boundary testing is and how to create test cases for the boundaries of your program
  • Specification-based testing in practice

Software requirements are, undoubtedly, the most valuable artifact when it comes to software testing. After all, requirements tell us precisely what the software needs to do and what it should not do. Requirements describe the intricacies of the business rules the software has to implement and which we need to validate. Therefore, requirements should be the first artifact you go for when it comes to testing!

In this chapter, we are going to explore ideas related to what we call specification-based testing. These techniques use the requirements of the program as input for testing. By requirements, we mean any document, written as text, that describes what a functionality should do. Think of an agile user story or a UML use case. The specific format does not matter. In simple words, we will discuss how we can use all the information that is available in the requirements to systematically derive a set of tests that fully exercise that requirement.

2.1 The requirement says it all

2.1.1 Step 1: Understanding the requirement, the inputs, and the outputs

2.1.2 Step 2: Just explore what the program does for a bunch of inputs

2.1.3 Step 3: Judiciously explore the possible inputs and outputs and identify the partitions

2.1.4 Step 4: Analyze the boundaries

2.1.5 Step five: Devise test cases

2.1.6 Step six: Automating the test cases

2.1.7 Step 7: Augment the test suite with creativity and experience

2.2 Specification-based testing in a nutshell

2.3 Finding bugs with specification testing

2.4 Specification-based testing in the real world

2.4.1 The process should be iterative and not sequential

2.4.2 How far should I go in doing specification testing?

2.4.3 Partition or boundary? It does not matter

2.4.4 On and off points are enough, but feel free to throw in some in and out points too

2.4.5 Use variations of the same input to facilitate understanding

2.4.6 When the number of combinations explodes, be pragmatic

2.4.7 When in doubt, go for the simplest input

2.4.8 Pick reasonable values for inputs you do not care