chapter ten

10 Working with different test types

 

This chapter covers

  • Test Levels pros and cons
  • Common anti patterns in test levels
  • The Test Recipe Strategy
  • Delivery Blocking and non blocking tests
  • Delivery vs Discovery Pipelines
  • Test Parallelization

Unit tests represent just one of the types of tests you could and should write. In this chapter we’ll discuss how unit testing fits into an organizational test strategy.

As soon as we start to look at other testing types we can do, we start asking some really important questions:

  • What types of tests do we want to write for this application?
  • At what level to we want to test various features? (UI, backend, API, unit etc..)
  • How do we decide at which level to test some feature? Do we test it multiple times on many levels?
  • Should we have more functional end to end tests or more unit tests?
  • How can we optimize the speed of the tests without sacrificing out trust in the tests?
  • Who should write each type of test?

And many more questions...

The answers to these questions is what I'd call a Testing Strategy.

If you want to cut through all the anti-patterns and go directly to my recommendations on what you might want to do, go directly to section 10.3. If you want to undetstand the reasoning, keep reading.

The first step in our journey is to frame the scope of the test strategy in terms of test types.

10.1 Common Test Types & Levels

10.1.1 Categories for judging a test

10.1.2 Unit Tests (in Memory) + Component Tests (in Memory):

10.1.3 Integration Tests (In Memory)

10.1.4 API Tests (Out of process)

10.1.5 E2E/UI Isolated Tests

10.1.6 E2E/UI system tests

10.2 Test Levels Anti Patterns

10.2.1 E2E only strategy

10.2.2 Low-level-only Test Anti Pattern

10.2.3 Disconnected Low- and high-level test organizations

10.3 Test Recipes as a strategy

10.3.1 A test recipe:

10.3.2 A test recipe is NOT: