8 Property-based testing

 

This chapter covers

  • Verifying properties of APIs
  • Developing a property-based testing library

In chapter 7, we worked through the design of a functional library for expressing parallel computations. There we introduced the idea that an API should form an algebra─that is, a collection of data types, functions over these data types, and, importantly, laws or properties that express relationships between these functions. We also hinted at the idea that it might be possible to somehow check these laws automatically.

This chapter takes us toward a simple but powerful library for property-based testing. The general idea of such a library is decoupling the specification of program behavior from the creation of test cases. The programmer focuses on specifying the behavior of programs and giving high-level constraints on the test cases; the framework then automatically generates test cases that satisfy these constraints and runs tests to ensure programs behave as specified.

Although a library for testing has a very different purpose than a library for parallel computations, we’ll discover that these libraries have a lot of surprisingly similar combinators. This similarity is something we’ll return to in part 3.

8.1 A brief tour of property-based testing

As an example, in ScalaCheck (http://mng.bz/n2j9), a property-based testing library for Scala, a property looks something like the following listing.

8.1.1 Choosing data types and functions

8.1.2 Initial snippets of an API

8.1.3 The meaning and API of properties

8.1.4 The meaning and API of generators

8.1.5 Generators that depend on generated values

8.1.6 Refining the Prop data type

8.2 Test case minimization

8.2.1 Using the library and improving its usability

8.2.2 Some simple examples

8.2.3 Writing a test suite for parallel computations