Softplus, log(1 + exp(x)), is a smooth alternative to a hard threshold. In a penalty, it provides a gradual gradient that can guide optimization before a constraint is violated. section 9.7
Data-center cooling
A cooling reward penalizes temperatures and pressure outside safe ranges using smooth hinge penalties. Unlike a hard penalty, these give gradient-based methods a learning signal as conditions approach the limits. section 9.7
Listing 9.5 Reward computation with energy and violation penalties
energy_cost = (self.cfg.w_energy_fan *
(fan ** 2).mean(dim=1) +
self.cfg.w_energy_valve *
(valve ** 2).mean(dim=1)) #A
temp_penalty = soft_hinge_penalty(
self.temp, self.cfg.temp_min, self.cfg.temp_max,
self.cfg.softplus_beta).mean(dim=1) *
self.cfg.w_temp_violation #B
pressure_penalty = soft_hinge_penalty(
self.dp, self.cfg.dp_min, self.cfg.dp_max,
self.cfg.softplus_beta).mean(dim=1) *
self.cfg.w_pressure_violation #C
rewards = -(energy_cost + temp_penalty +
pressure_penalty) #D
See how temperature and pressure penalties combine with energy cost in the reward calculation. section 9.7
