liveBook · Manning

Multinomial distribution

As explained in Math and Architectures of Deep Learning

A probability distribution over counts of outcomes from repeated independent trials with multiple mutually exclusive categories.

The multinomial distribution models counts across multiple mutually exclusive, exhaustive outcomes over n random trials. If outcome Ci has probability πi, Xi records how many trials select it, with the counts summing to n. section 5.9.4 section 5.0

For counts k1, …, km, the probability combines the number of possible outcome orderings with the category probabilities: n! ∏i(πi^ki / ki!), where ∑i ki = n. With two categories, the model reduces to the binomial distribution; each individual Xi is also binomially distributed. section 5.9.4

Useful summary statistics follow from the binomial behavior of each category count:

Ten photo-selection trials

Suppose four mutually exclusive classes have probabilities (0.1, 0.42, 0.04, 0.44), representing Einstein, Marie Curie, Gauss, and none of these. The probability of counts (1, 2, 1, 6) across ten trials is evaluated by the multinomial formula. section 5.9.4

How Math and Architectures of Deep Learning covers it

How deep it goes
Defines the probability function, relates counts to binomial distributions, derives moments, and verifies them computationally.
Examples
Uses four photo classes, ten trials, count vector (1, 2, 1, 6), and PyTorch sampling and log-probability checks.
What it assumes
Assumes probability outcomes and trials, while defining mutually exclusive classes and count variables explicitly.
What you can do after
Apply via PyTorch to calculate log probabilities, sample count vectors, and compare empirical and theoretical moments.
How it connects
Connects multinomial distributions to binomial distributions, probability distributions, and machine-learning models.
Cover of Math and Architectures of Deep Learning
liveBookmanning.com
Read Math and Architectures of Deep Learning on liveBook Get the book at manning.com

Also explained in

Compare how 2 books explain multinomial distribution →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Math and Architectures of Deep Learning. The section links point to the parts of the book it draws on.