liveBook · Manning

P-value

A p-value is the probability, assuming a specified null hypothesis, of obtaining a test statistic or result at least as extreme as the observed one.

How 9 Manning books explain it

Statistics Slam Dunk

A p-value is the probability, under a specified null hypothesis, of obtaining a result at least as extreme as the observed result according to a test statistic.

How it is used
Supports null-hypothesis decisions in normality tests, t-tests, correlation tests, ANOVAs, and regression modeling.
How deep it goes
Defines threshold-based interpretation and demonstrates it across R test outputs, without deriving p-value calculations.
Examples
Uses NBA foul calls, free-throw attempts, salary and win-share Gini coefficients, normality checks, and wins models.
What you can do after
Apply it via base R tests and model summaries to reject or fail to reject a stated null hypothesis.

Read how Statistics Slam Dunk explains p-value →

Statistics Every Programmer Needs

A p-value is the probability, under a specified null hypothesis, of obtaining a result at least as extreme as the observed result.

How it is used
Guides null-hypothesis decisions about regression models, individual predictors, and observed-versus-expected counts.
How deep it goes
Defines the conditional probability, works through threshold decisions, and demonstrates a chi-square computation without deriving a general p-value formula.
Examples
Uses runner-time regression, raisin-variety classification, and a SciPy test of world-population leading digits against Benford’s law.
What you can do after
Apply it via SciPy and statsmodels to interpret significance while checking fit, assumptions, and predictive performance separately.

Read how Statistics Every Programmer Needs explains p-value →

Grokking Statistics

A p-value is a probability, calculated under the null hypothesis, of obtaining a result at least as extreme as the observed result.

How it is used
Uses p-values to judge whether sample evidence challenges a null hypothesis in mean and correlation tests.
How deep it goes
Defines tail-area meaning, contrasts one- and two-tailed calculations, and applies decision rules without deriving proofs.
Examples
Uses tea identification, potato chip weights, and sports drink sales versus temperature datasets.
What you can do after
Apply it via SciPy to calculate p-values and decide whether to reject a specified null hypothesis.

Read how Grokking Statistics explains p-value →

Effective Data Analysis

A p-value is the probability of obtaining a test statistic at least as extreme as the observed value assuming the null hypothesis is true.

How it is used
Uses p-values to interpret test statistics and assess statistical significance in analytics.
How deep it goes
Explains significance thresholds and sample-size limitations, then provides exercises varying alpha and sample size.
Examples
Uses student test scores and simulated independent samples to explore significance and sample size.
What you can do after
Practice interpreting p-values by running independent-samples t-tests and comparing results across thresholds and sample sizes.

Read how Effective Data Analysis explains p-value →

Timeless Algorithms, The Foundational Ideas

A p-value is the probability, under a null hypothesis, of observing a result at least as extreme as the one obtained.

How it is used
Turns sample evidence into a hypothesis-testing decision against a predetermined standard.
How deep it goes
Defines the null-based probability and works through decision thresholds and interpretation, including a near-cutoff result.
Examples
Uses a small clinical trial and beverage fill measurements to illustrate one- and two-tailed decisions.
What you can do after
Evaluate evidence against a predetermined significance level without treating the p-value as the probability that the null hypothesis is true.

Read how Timeless Algorithms, The Foundational Ideas explains p-value →

Grokking Bayes

A p-value is the probability, under a null hypothesis, of obtaining data at least as extreme as the observed data according to a specified test statistic.

How it is used
Evaluates how surprising observed data are under a null hypothesis in classical hypothesis testing.
How deep it goes
Defines null-based extremeness, test statistics, tail probabilities, and threshold decisions, but does not estimate hypothesis probabilities.
Examples
Uses a website conversion test with 500 visitors, 63 conversions, a 0.1 baseline, a right tail, and scipy.stats.binomtest.
What you can do after
Apply it via a classical hypothesis test to quantify null-based extremeness and make a threshold-based rejection decision.

Read how Grokking Bayes explains p-value →

Fight Fraud with Machine Learning

A p-value is the probability, under a statistical test's null hypothesis, of obtaining a result at least as extreme as the observed result.

How it is used
Uses statistical significance to qualify ANOVA-based feature rankings for phishing-fraud detection.
How deep it goes
Defines a practical significance threshold and inspects returned values, but does not derive p-values or test assumptions.
Examples
Uses phishing-page features, including NumNumericChars and PctExtResourceUrls, with scikit-learn p-value output.
What you can do after
Evaluate whether ANOVA F-scores are statistically significant enough to support numerical-feature analysis.

Read how Fight Fraud with Machine Learning explains p-value →

Causal AI

A p-value is the probability, under a null hypothesis, of obtaining a test statistic at least as extreme as the observed one.

How it is used
Uses p-values to assess statistical evidence for dependence or conditional independence while building and checking causal models.
How deep it goes
Explains the definition, threshold rule, bootstrap workflow, and sample-size limitation, but stops short of deriving test statistics.
Examples
Uses E ⊥ T | O, R, a 30-point dataset, bootstrap resampling, confidence intervals, and plots of p-values against data size.
What you can do after
Diagnose/debug conditional-independence tests by comparing thresholds, bootstrapping across sample sizes, and interpreting results cautiously.

Read how Causal AI explains p-value →

LLM Evaluation and Alignment, The Foundational Ideas

A p-value is the probability, under a specified null hypothesis, of obtaining a result at least as extreme as the observed result.

How it is used
Evaluates whether an observed correlation is unlikely under a no-relationship null hypothesis.
How deep it goes
Explains significance, computation, and interpretation, but stops short of treating significance as practical correlation strength.
Examples
Uses human and metric scores, bootstrap resampling, and contrasting p-values to illustrate evaluation decisions.
What you can do after
Evaluate metric evidence by interpreting p-values alongside correlation magnitudes and confidence intervals.

Read how LLM Evaluation and Alignment, The Foundational Ideas explains p-value →

Also covered in

What to know first

Related concepts

Questions about p-value

What is p-value?

A p-value is the probability, assuming a specified null hypothesis, of obtaining a test statistic or result at least as extreme as the observed one.

Which Manning books explain p-value?

Statistics Slam Dunk, Statistics Every Programmer Needs, Grokking Statistics, Effective Data Analysis, Timeless Algorithms, The Foundational Ideas, Grokking Bayes, Fight Fraud with Machine Learning, Causal AI and 1 more.

What should I know before learning p-value?

Manning books that teach p-value usually introduce null hypothesis, variance, mean and anomaly first.

What is the difference between p-value and bayes factor?

A p-value is the probability, assuming a specified null hypothesis, of obtaining a test statistic or result at least as extreme as the observed one. A Bayes factor is the ratio of the marginal likelihoods of two competing models or hypotheses, quantifying their relative support from observed data.

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.