chapter eleven

11 Data Oriented Testing

 

This chapter covers

  • The testability of data-oriented code
  • Writing data-oriented tests
  • Working with randomized test data

This chapter will teach you how to write good tests. We’ll learn how to exploit the inherent testability of data-oriented code to create tests that are small, simple, and clear. We’ll also learn how to use data to improve legacy tests and code. Representation remains the essence of programming even in tests. Small changes have outsized impacts on understandability.

Once we can write good tests, we’re going to learn how to write great ones. The remainder of the chapter is devoted to the art and craft of generating randomized test data. It is the most powerful technique I know for creating tests that can actually prove something about a program (or at least make us very confident). The technique is unwieldy and strange at first and requires shifting how we think about the role of software testing, but the reward is a suite that can catch and prevent errors before they reach our users.

11.1 A few definitions

Every developer describes testing differently. What one calls an “Integration test,” another might call a an “acceptance” test, or maybe a “behavioral” or “end-to-end” test. Even the “unit” in “unit test” can involve bike-shedding.

So, just for the purposes of this book, so we can communicate, we’re going to squeeze the entire world of testing into exactly two buckets: Unit Tests and Integration Tests.

11.2 Data orientation makes tests better

11.3 Writing data-oriented tests

11.3.1 Use data to show what’s under test

11.3.2 Data makes tests flexible

11.3.3 Model side-effects with data

11.3.4 Don’t be afraid to introduce new data types

11.4 Avoiding common problems in test suites

11.4.1 Randomize unused inputs to prevent coupling

11.5 Using randomized data in tests

11.5.1 Streams + randomization = infinite test data

11.5.2 Exercise all the states in your program

11.5.3 One test, multiple roles

11.5.4 The most useful tests are often the simplest

11.5.5 Verifying algebraic properties.

11.5.6 Randomization is just another tool

11.5.7 Favor using libraries when generating data

11.6 A few closing thoughts on testing

11.6.1 Data coverage is more important than test coverage

11.6.2 Embrace the humble comment

11.6.3 The best tests are the ones you don’t have to write

11.7 Wrapping up

11.8 Summary