liveBook · Manning

Probability distribution

As explained in Applied Reinforcement Learning

A probability distribution specifies the probabilities assigned to the possible values of a random variable.

A probability distribution assigns probabilities to possible outcomes. In language generation, logits are converted into probabilities for possible next tokens; the model selects one and repeats. Selecting the most probable token is one option, while sampling allows more varied responses. section 10.2

In simulation, distributions describe uncertainty in outcomes such as customer stay durations or weather-related foot traffic. Simulated state transitions can also be counted and normalized to estimate the distribution of the next state. section 1.6.2 section 2.2

Customer-state transitions

Starting from “Buy Pro,” simulate a Markov chain, count the next states, and normalize their frequencies. Compare the resulting distribution with estimates that also condition on earlier states. section 2.2

Figure 10.1 Overview of large language models architecture.

The architecture overview situates probability conversion and token selection at the model’s output stage. section 10.2

How Applied Reinforcement Learning covers it

How deep it goes
Explains logit conversion, token selection, and empirical frequency normalization, but gives no general mathematical treatment.
Examples
Uses next-token decoding and simulated Buy Pro customer-state transitions as examples.
What you can do after
Build it from scratch by simulating transitions and normalizing observed next-state counts into probabilities.
How it connects
Connects distributions to stochastic policies, logits, and Markov chains.

Also explained in

Compare how 33 books explain probability distribution →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Applied Reinforcement Learning. The section links point to the parts of the book it draws on.