Level of significance
The level of significance is the prespecified maximum probability of rejecting a true null hypothesis in a statistical test.
Timeless Algorithms, The Foundational Ideas
A significance level is the maximum probability of rejecting a null hypothesis when it is true, specified for a statistical test.
- How it is used
- Sets an advance decision threshold for hypothesis tests according to the consequences of false positives and false negatives.
- How deep it goes
- Defines Type I error, explains the error trade-off, and translates alpha into critical regions and a p-value decision rule.
- Examples
- Uses beverage fill volumes, aircraft vibration monitoring, and a small clinical trial to illustrate threshold choices.
- What you can do after
- Evaluate and understand significance thresholds and their consequences when making hypothesis-test decisions.
Read how Timeless Algorithms, The Foundational Ideas explains level of significance →
Grokking Statistics
The significance threshold below which a p-value leads to rejection of the null hypothesis.
- How it is used
- Sets the rejection cutoff for testing a population claim with sample data.
- How deep it goes
- Works through tail areas, critical Z-values, and a complete two-tailed decision in Python.
- Examples
- Uses 95% confidence, ±1.959964 critical Z-values, and 142-gram potato chip bags.
- What you can do after
- Apply it via Python to calculate a critical value and decide whether to reject a null hypothesis.
Read how Grokking Statistics explains level of significance →
What to know first
- standard normal distributionA standard normal distribution is a normal probability distribution with mean zero and standard deviation one.
- meanThe mean is a measure of central location equal to the arithmetic average of numerical observations, or the probability-weighted average of a random variable’s possible values.
- p-valueA p-value is the probability, assuming a specified null hypothesis, of obtaining a test statistic or result at least as extreme as the observed one.
- sample sizeSample size is the number of observations, units, or items included in a sample or group being analyzed.
Related concepts
- critical regionconfigures · A critical region is the set of test-statistic values for which a hypothesis test rejects the null hypothesis at a specified significance level.
- false positiveconfigures · A false positive is a positive result that incorrectly indicates a target condition is present when it is absent.
- p-valuecontrasts with · A p-value is the probability, assuming a specified null hypothesis, of obtaining a test statistic or result at least as extreme as the observed one.
- thresholdkind of · A threshold is a boundary value used to determine whether a measured quantity meets a criterion or changes a classification, decision, or action.
- confidenceequivalent to · Confidence is a degree or quantified measure of certainty or assurance associated with a belief, judgment, estimate, prediction, or result.
- decision ruleconfigures · A decision rule is a criterion or procedure that maps evidence, outcomes, or conditions to a choice or action.
Questions about level of significance
What is level of significance?
The level of significance is the prespecified maximum probability of rejecting a true null hypothesis in a statistical test.
Which Manning books explain level of significance?
Timeless Algorithms, The Foundational Ideas and Grokking Statistics.
What should I know before learning level of significance?
Manning books that teach level of significance usually introduce standard normal distribution, mean, p-value and sample size first.
What is the difference between level of significance and p-value?
The level of significance is the prespecified maximum probability of rejecting a true null hypothesis in a statistical test. A p-value is the probability, assuming a specified null hypothesis, of obtaining a test statistic or result at least as extreme as the observed one.
What is level of significance a kind of?
Level of significance is a kind of threshold. A threshold is a boundary value used to determine whether a measured quantity meets a criterion or changes a classification, decision, or action.
What is level of significance also called?
Level of significance is also called significance level.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.

