liveBook · Manning

Probabilistic reasoning

As explained in Timeless Algorithms, The Foundational Ideas

Probabilistic reasoning is inference that represents and updates beliefs using probabilities under uncertainty.

Probabilistic reasoning weighs competing possibilities under uncertainty. Observed data provides evidence about unknown causes, states, parameters, or structures; probabilistic models determine how those possibilities are weighed. section 10.0

P(A∪B)=P(A)+P(B)

For mutually exclusive events, add their probabilities. If the outcomes also cover every possibility, their probabilities sum to one. section 2.3.1

Competing outcomes

A fair die has a 1/6 chance of showing either a 2 or a 5, so the chance of either is 1/3. The two outcomes are mutually exclusive, but do not cover every possible roll. Similar probability allocations describe competing diagnoses or classifier labels. section 2.3.1

This probabilistic framework links ideas across fields: decoding can be expressed as Bayesian model comparison, message passing can perform inference, and cross-entropy can guide model training. section 10.3

How Timeless Algorithms, The Foundational Ideas covers it

How deep it goes
Explains the addition rule, distinguishing mutually exclusive events from collectively exhaustive outcomes through formal conditions and examples.
Examples
Uses a die roll, clinical diagnoses, classifier classes, and candidate words to illustrate probability over competing outcomes.
What you can do after
Understand how probabilities are allocated among mutually exclusive explanations and why exhaustive outcomes sum to one.
How it connects
Connects Bayesian inference, model comparison, message passing, cross-entropy, information theory, and learning.

Also explained in

Compare how 3 books explain probabilistic reasoning →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Timeless Algorithms, The Foundational Ideas. The section links point to the parts of the book it draws on.