10 Developing a testing strategy

 

This chapter covers

  • Test Levels pros and cons
  • Common anti-patterns in test levels
  • The Test Recipe Strategy
  • Delivery-blocking and non-blocking tests
  • Delivery vs Discovery Pipelines
  • Test Parallelization

Unit tests represent just one of the types of tests you could and should write. In this chapter we’ll discuss how unit testing fits into an organizational test strategy. As soon as we start to look at other types of tests types, we start asking some really important questions:

  • At what level do we want to test various features? (UI, backend, API, unit etc..)
  • How do we decide at which level to test some feature? Do we test it multiple times on many levels?
  • Should we have more functional end-to-end tests or more unit tests?
  • How can we optimize the speed of the tests without sacrificing trust in the tests?
  • Who should write each type of test?

And many more questions...

The answers to these questions is what I'd call a Testing Strategy.

The first step in our journey is to frame the scope of the test strategy in terms of test types.

10.1 Common test types and levels

10.1.1 Criteria for judging a test

10.1.2 Unit tests and component tests

10.1.3 Integration tests

10.1.4 API tests

10.1.5 E2E/UI isolated Tests

10.1.6 E2E/UI system tests

10.2 Test level antipatterns

10.2.1 The end-to-end-only antipattern

10.2.2 Low-level-only test antipattern

10.2.3 Disconnected low-level and high-level tests

10.3 Test Recipes as a strategy

10.3.1 How to write a test recipe

10.3.2 When do I write and use a test recipe?

10.3.3 Rules for a test recipe

10.4 Managing delivery pipelines

10.4.1 Delivery vs Discovery Pipelines