liveBook · Manning

REINFORCE

REINFORCE is a Monte Carlo policy-gradient algorithm that updates policy parameters using sampled returns and the gradient of selected actions’ log probabilities.

How 2 Manning books explain it

Applied Reinforcement Learning

A Monte Carlo policy-gradient algorithm that updates policy parameters using sampled returns and the gradient of the log probability of selected actions.

How it is used
Uses direct policy optimization for coordinated continuous cooling controls where discretizing actions loses precision or becomes impractical.
How deep it goes
Works through sampled gradients and return-to-go, then develops value baselines and an implemented Transformer actor-critic.
Examples
Illustrates updates with a Gaussian fan policy and implements an eight-zone data center cooling simulation.
What you can do after
Builds a REINFORCE-based cooling controller with a Transformer policy, value baseline, and entropy regularization.

Read how Applied Reinforcement Learning explains REINFORCE →

Reinforcement Learning from Human Feedback

REINFORCE is a policy-gradient reinforcement-learning algorithm that updates a policy using sampled returns or rewards.

How it is used
Translates reward-model preferences into language-model behavior during the RL stage of RLHF.
How deep it goes
Derives the Monte Carlo advantage update and implementation components, but stops short of learned value estimation and importance-ratio control.
Examples
Uses sampled language-model completions, reward-model scores, shaped rewards, and formal advantage-gradient equations.
What you can do after
Apply sampled returns and baselines to update a policy toward higher-reward outputs without a learned value function.

Read how Reinforcement Learning from Human Feedback explains REINFORCE →

What to know first

Related concepts

Questions about REINFORCE

What is REINFORCE?

REINFORCE is a Monte Carlo policy-gradient algorithm that updates policy parameters using sampled returns and the gradient of selected actions’ log probabilities.

Which Manning books explain REINFORCE?

Applied Reinforcement Learning and Reinforcement Learning from Human Feedback.

What should I know before learning REINFORCE?

Manning books that teach REINFORCE usually introduce policy gradient method and value function first.

What kinds of REINFORCE are there?

Kinds of REINFORCE include RLOO.

What is REINFORCE a kind of?

REINFORCE is a kind of policy gradient method. A policy gradient method is a reinforcement-learning method that optimizes a parameterized policy by estimating the gradient of expected return with respect to its parameters.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.