chapter four

4 Database trouble & testing in production

 

This chapter covers

  • Design and implement chaos experiments on a popular application (WordPress with MySQL)
  • Implement a chaos experiment adding network latency using the Linux traffic control tool
  • When testing in production might make sense and how to approach it

In this chapter we will apply everything we’ve learned about chaos engineering so far on a real-world example of a common application you might be familiar with. Have you heard of WordPress? It’s a popular blogging engine and content management system. According to some estimates, WordPress accounts for more than a third of all pages on the internet, and for most CMS-backed websites (https://w3techs.com/technologies/details/cm-WordPress). It’s typically paired with a MySQL database, another very popular piece of technology.

Let’s take a vanilla instance of WordPress backed by MySQL and using chaos engineering, try to gain confidence in how reliably we can run it. We’ll try to preemptively guess what conditions might disturb it and design experiments to verify how it fares. Ready? Let’s see what our friends from Glanden are up to these days.

4.1     We’re doing WordPress

4.2     Weak links

4.2.1   Experiment 1: slow disks

4.2.2   Experiment 2: slow connection

4.3     Testing in production

4.4     Summary