liveBook · Manning

Discrete action space

As explained in Applied Reinforcement Learning

A discrete action space is a finite set of distinct actions from which an agent selects.

A discrete action space contains a finite set of choices. In REINFORCE, a policy network assigns each action a probability using a softmax over its output scores, then samples an action. Good returns raise the sampled action’s probability; poor returns lower it. section 9.3

Production choices and language tokens

A factory policy might choose Low, Medium, or High production; with logits [1.0, 2.5, 0.5], it assigns Medium probability 0.76 and may sample it. Language generation uses the same kind of choice over a much larger set: the model selects a next token from its vocabulary. section 9.3 section 10.3

How Applied Reinforcement Learning covers it

How deep it goes
Explains logits, softmax probabilities, sampling, and return-based updates through a production example.
Examples
Uses three production levels and next-token selection from a large language-model vocabulary.
What you can do after
Apply REINFORCE by sampling discrete actions from policy probabilities and adjusting them according to returns.
How it connects
Connects discrete action spaces to REINFORCE, policy gradients, softmax, and Markov decision processes.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.