This chapter covers:
- The problems that autoscalers set out to solve.
- Descriptions of how Knative Serving’s autoscaling works when there are zero instances, when there are a few instances, and when there are many instances.
- A walkthrough of the core autoscaling algorithm.
- Descriptions of configuration options and how they affect autoscaling.
Autoscaling awakens the engineering imagination in a way that few topics do. Most of the systems we build seem lifeless or mindless. But to build a system that appears to breathe is somehow uniquely fascinating. Depressingly, though, "autoscaling" turns out to be easy to spell and hard to achieve. The system which today breathes peacefully is tomorrow yelling obscenities.
My goal in this chapter is to explain the basic structure and functioning of the components responsible for management of scaling in Knative Serving: the Autoscaler, the Activator and the Queue-Proxy. Most of the time you will not need to think of these, as they embody the accumulated observations and insights of the Knative authors. But they are dynamic systems and exhibit dynamic complexity, which means that you will occasionally be surprised. A grasp of the components will help you to moderate your surprise.