chapter five

5 What your test suite tells you about your system’s health

 

This chapter covers

  • The testing pyramid
  • Continuous deployment versus continuous delivery
  • Restoring confidence in your test suite
  • How flakey tests contribute to flakey systems
  • Feature flags
  • Why the Operations team should own the testing infrastructure

If you’re doing any sort of automated testing, you probably keep a watchful eye on the number of test cases. In fact, you probably thump your chest occasionally, overly proud of just how much automation you have. And despite having 1500 automated test cases, a full QA team that does regression on every release, you still end up releasing your software with bugs. Not only that, you sometimes release with embarrassing bugs that highlight the fact that some areas are just not tested at all. You’re not alone.

5.1           The testing pyramid

5.2           Testing structure

5.2.1   Unit testing

5.2.2   Integration tests

5.2.3   End-to-end tests

5.3           Confidence in your test suite

5.3.1   Restoring confidence in your test suite

5.3.2   Vanity metrics

5.4           Continuous deployment versus continuous delivery

5.4.1   Feature flags

5.5           Executing pipelines

5.6           Managing the testing infrastructure

5.7           Summary