liveBook · Manning

Categorical distribution

As explained in Applied Reinforcement Learning

A categorical distribution assigns probabilities to a finite set of mutually exclusive outcomes.

A categorical distribution assigns probabilities to discrete actions. In a policy-gradient agent, it can represent action choices such as Low, Medium, and High; a Gaussian distribution serves the corresponding role for continuous actions. section 9.3

Choosing an action

For logits [1.0, 2.5, 0.5], softmax gives approximate probabilities 0.17, 0.76, and 0.10 for Low, Medium, and High. Sampling Medium and receiving a good return raises its probability; a poor return for High would suppress its probability. section 9.3

How Applied Reinforcement Learning covers it

How deep it goes
Works through softmax probabilities for three actions and illustrates reinforcement, while focusing the broader treatment on Gaussian policies.
Examples
Calculates softmax probabilities for Low, Medium, and High actions, then shows how returns shift their likelihoods.
What you can do after
Understand how policy-gradient updates increase the likelihood of good discrete actions and suppress the likelihood of poor ones.
How it connects
Connects categorical distributions with softmax policies, REINFORCE, policy gradients, and Gaussian action distributions.

Also explained in

Compare how 5 books explain categorical distribution →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.