11 Automating Kubernetes experiments

 

This chapter covers

  • Automating chaos experiments for Kubernetes with PowerfulSeal
  • Recognizing the difference between one-off experiments and ongoing SLO verification
  • Designing chaos experiments on the VM level using cloud provider APIs

In this second helping of Kubernetes goodness, you’ll see how to use higher-level tools to implement chaos experiments. In the previous chapter, you set up experiments manually to build an understanding of how to implement the experiment. But now I want to show you how much more quickly you can go when using the right tools. Enter PowerfulSeal.

11.1 Automating chaos with PowerfulSeal

It’s often said that software engineering is one of the very few jobs where being lazy is a good thing. And I tend to agree with that; a lot of automation or reducing toil can be seen as a manifestation of being too lazy to do manual labor. Automation also reduces operator errors and improves speed and accuracy.

11.1.1 What’s PowerfulSeal?

11.1.2 PowerfulSeal installation

11.1.3 Experiment 1b: Killing 50% of pods

11.1.4 Experiment 2b: Introducing network slowness

11.2 Ongoing testing and service-level objectives

11.2.1 Experiment 3: Verifying pods are ready within (n) seconds of being created

11.3 Cloud layer

11.3.1 Cloud provider APIs, availability zones

11.3.2 Experiment 4: Taking VMs down

Summary