8 Application-level fault injection

 

This chapter covers

  • Building chaos engineering capabilities directly into your application
  • Ensuring that the extra code doesn’t affect the application’s performance
  • More advanced usage of Apache Bench

So far, you’ve learned a variety of ways to apply chaos engineering to a selection of different systems. The languages, tools, and approaches varied, but they all had one thing in common: working with source code outside your control. If you’re in a role like SRE or platform engineer, that’s going to be your bread and butter. But sometimes you will have the luxury of applying chaos engineering to your own code.

This chapter focuses on baking chaos engineering options directly into your application for a quick, easy, and—dare I say it—fun way of increasing your confidence in the overall stability of the system as a whole. I’ll guide you through designing and running two experiments: one injecting latency into functions responsible for communicating with an external cache, and another injecting intermittent failure through the simple means of raising an exception. The example code is written in Python, but don’t worry if it’s not your forte; I promise to keep it basic.

Note

I chose Python for this chapter because it hovers at the top of the list in terms of popularity, and it allows for short, expressive examples. But what you learn here is universal and can be leveraged in any language. Yes, even Node.js.

8.1 Scenario

 
 
 
 

8.1.1 Implementation details: Before chaos

 
 
 

8.2 Experiment 1: Redis latency

 
 

8.2.1 Experiment 1 plan

 
 

8.2.2 Experiment 1 steady state

 
 

8.2.3 Experiment 1 implementation

 
 
 

8.2.4 Experiment 1 execution

 
 

8.2.5 Experiment 1 discussion

 
 
 

8.3 Experiment 2: Failing requests

 
 

8.3.1 Experiment 2 plan

 

8.3.2 Experiment 2 implementation

 
 
 
 

8.3.3 Experiment 2 execution

 
 
 

8.4 Application vs. infrastructure

 
 

Summary

 
 
sitemap

Unable to load book!

The book could not be loaded.

(try again in a couple of minutes)

manning.com homepage
test yourself with a liveTest