7 Managing business metrics

 

This chapter covers

  • Identifying metrics that directly impact your business
  • Growing a collection of company-specific business metrics
  • Engaging stakeholders when developing metrics
  • Conducting experiments using multiple metrics

Up to this point in the book, we have described experimental methods that you can use to improve your business metrics. Since the decision to accept or reject a change to your system ultimately hinges on the values of those metrics, it is worth spending some time discussing how to define and employ them effectively. This is the subject of the present chapter.

Section 7.1 starts us out by contrasting metrics that are directly relevant to a business with other, more technical metrics. In section 7.2, we’ll see how business metrics are derived and how they evolve, and we’ll see how metrics have an inherent timescale that affects your ability to measure them. Finally, in section 7.3, we’ll talk about how to use multiple business metrics to evaluate and experiment.

7.1 Focus on the business

7.1.1 Don’t evaluate a model

7.1.2 Evaluate the product

7.2 Define business metrics

7.2.1 Be specific to your business

7.2.2 Update business metrics periodically

7.2.3 Business metric timescales

7.3 Trade off multiple business metrics

7.3.1 Reduce negative side effects

7.3.2 Evaluate with multiple metrics

Summary