8 Property-based testing

 

This chapter covers

  • Understanding property-based testing
  • Fabricating test data using generators
  • Minimizing test case outcomes to give meaningful feedback
  • Using properties to affirm laws

In chapter 7, we worked through the design of a functional library for expressing parallel computations. We introduced the idea that an API should form an algebra —that is, a collection of data types, functions over these data types, and, importantly, laws or properties that express relationships between these functions. We also hinted at the idea that it might be possible to somehow validate these laws automatically. Validation is an important step, as we need to know that the code we write conforms with the laws we have imposed on our program. It would be of great benefit if we could somehow automate this validation process.

This chapter takes us toward a simple but powerful library for automated property-based testing. The general idea of such a library is to decouple the specification of program behavior from the creation of test cases. The programmer focuses on specifying the behavior of a program and giving high-level constraints on the test cases. The framework then automatically generates test cases that satisfy these constraints and runs tests to validate that the program behaves as specified.

8.1 A brief tour of property-based testing

8.2 Choosing data types and functions

8.2.1 Gathering initial snippets for a possible API

8.2.2 Exploring the meaning and API of properties

8.2.3 Discovering the meaning and API of generators

8.2.4 Generators that depend on generated values

8.2.5 Refining the property data type

8.3 Test case minimization

8.4 Using the library and improving the user experience

8.4.1 Some simple examples

8.4.2 Writing a test suite for parallel computations

8.5 Generating higher-order functions and other possibilities

8.6 The laws of generators

8.7 Conclusion