liveBook · Manning

Continuous control

As explained in Applied Reinforcement Learning

Continuous control is decision-making in which an agent selects real-valued control signals rather than discrete actions.

Continuous control means choosing real-valued signals, such as a fan speed or valve opening, rather than selecting from a fixed set of discrete actions. section 9.1

Why does this matter for operational decisions?

  • Discretizing many controls creates an enormous action space: with 8 data-center zones and two controls per zone, 10 levels per control yield 10^16 possible actions. section 9.1
  • Coarse levels can miss useful settings: a fan speed of 0.73 may be optimal even when the available choices are only 0.0, 0.5, and 1.0. section 9.1

From data centers to robots

A data-center controller can adjust fan speeds and valve openings. In robotics, continuous-control policies can instead map sensor vectors to joint torques for tasks such as walking, reaching, and grasping. section 9.1 section 11.8.1

Figure 9.1 Value-Based vs. Policy-Based Reinforcement Learning.

Compare value-based and policy-based reinforcement learning approaches to choosing actions. section 9.1

How Applied Reinforcement Learning covers it

How deep it goes
Compares action discretizations using a data-center example and connects continuous control to robotics without detailing implementation.
Examples
Uses data-center fan speeds and valve openings, plus robotic walking, reaching, grasping, and joint-torque control.
What you can do after
Evaluate when discretizing controls makes value-based action selection impractical.
How it connects
Connects continuous control to value-based methods, DQN, policy-gradient methods, and PPO.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.