Continuous control means choosing real-valued signals, such as a fan speed or valve opening, rather than selecting from a fixed set of discrete actions. section 9.1
Why does this matter for operational decisions?
- Discretizing many controls creates an enormous action space: with 8 data-center zones and two controls per zone, 10 levels per control yield 10^16 possible actions. section 9.1
- Coarse levels can miss useful settings: a fan speed of 0.73 may be optimal even when the available choices are only 0.0, 0.5, and 1.0. section 9.1
From data centers to robots
A data-center controller can adjust fan speeds and valve openings. In robotics, continuous-control policies can instead map sensor vectors to joint torques for tasks such as walking, reaching, and grasping. section 9.1 section 11.8.1
Figure 9.1 Value-Based vs. Policy-Based Reinforcement Learning.
Compare value-based and policy-based reinforcement learning approaches to choosing actions. section 9.1
