12 Testing in production

 

This chapter covers

  • What testing in production is
  • The value of testing in production as part of a testing strategy
  • How to identify what to track in production
  • Setting up testing in production
  • Alternative ways to leverage testing in production

When Eric Ries wrote The Lean Startup in 2011, the strategy and mindset he proposed influenced various industries, including software development. The center of Eric’s thesis is that we want to ensure that we’re delivering something of quality with as minimal waste as possible. As he puts it in a blog post on linkedin.com (http://mng.bz/xMO8),

“The problem is that quality is really in the eye of the beholder. For a for-profit company, quality is defined by what the customer wants. So if we are misaligned with what the customer wants, then all the extra time we take to polish all the edges and get everything right is actually wasted time because we end up pushing the product away from what the customer wants.”

12.1 Planning out testing in production

12.1.1 What to track

12.1.2 Service-level objectives

12.1.3 Service-level agreements

12.1.4 Service-level indicators

12.1.5 What to save

12.2 Setting up testing in production

12.2.1 Setting up a Honeycomb account

12.2.2 Adding Honeycomb to APIs

12.2.3 Advanced querying

12.2.4 Building SLO triggers

12.3 Taking testing in production further

12.3.1 Testing with synthetic users

12.3.2 Testing hypotheses

12.4 Expanding your strategy by testing in production