Chapter 4: From evidence to decisions: Neyman, Pearson, and the principles of hypothesis testing
The introduction explains why learning from data is not enough to make decisions and how the Neyman–Pearson framework adds explicit, advance rules for controlling errors.
After this chapter you will be able to
- Advance rules for decisions. Explain how the Neyman–Pearson framework turns statistical evidence into disciplined decisions by specifying hypotheses, error rates, and decision rules in advance.
- Interrelated hypothesis testing workflow. Explain that the seven-step sequence is a practical way to view Neyman and Pearson’s mathematical framework, with each step building on earlier decisions or results. State that some choices, such as the significance level, are made before analyzing the data.
- Competing hypothesis roles. Explain how a research question becomes a null hypothesis and an alternative hypothesis, and describe what each represents when evaluating evidence.
- Fixed preanalysis design. Justify defining the competing claims, parameter, and test structure before seeing the data, and keeping them fixed during analysis.
- Significance level and testing errors. Explain what α limits and distinguish Type I from Type II errors in terms of the null hypothesis.
- Contextual error risk tradeoff. Describe how choosing a smaller or larger α changes the balance between false positives and false negatives, and why the appropriate value depends on their consequences.
- Advance selection of alpha. Justify selecting α before examining the data and keeping it fixed throughout the test.
- Test fit and assumptions. Explain how the research question, data type, and a test’s assumptions guide test selection, and why a poor match can make results unreliable or reduce power.
- Advance commitment to the test. Explain why the statistical test is selected before analyzing the data and kept fixed during the analysis.
- Power among valid tests. Explain how power guides a choice among tests for the same hypotheses while Type I error remains controlled, and state the limited case established by the Neyman–Pearson Lemma.
- Purpose of the test statistic. Explain that a test statistic uses the selected test’s formula to summarize the observed sample in one value that can be evaluated against the null hypothesis, but does not itself establish a conclusion.
- Significance level and critical region. Explain how the chosen significance level and the test’s sampling distribution define the critical region, and describe the two-tailed 5% example.
- Equivalent decision rule expressions. Explain how critical regions, critical values, and a p-value comparison express the same decision rule.
- Formal hypothesis test decision. Describe how the observed test statistic or its p-value is compared with the previously established criterion to decide whether to reject or fail to reject the null hypothesis. Distinguish that formal decision from proving a hypothesis or interpreting its practical importance.
- Errors behind either decision. Explain the possible error associated with each decision and how the significance level and statistical power relate to those risks.
- Evidence Without Certainty. Explain what rejecting or failing to reject the null hypothesis allows you to conclude, without treating either decision as proof.
- Contextual and Transparent Conclusions. Describe how to report a test decision in terms of the original research question while stating relevant limitations and showing how the decision was reached.
- Statistical Versus Practical Significance. Distinguish statistical significance from practical importance and describe what else to consider when judging a finding.
- Purpose of the seven-step framework. Explain how separating test design from execution, interpretation, and communication makes the evaluation of evidence transparent and reproducible while explicitly managing the risk of error.
- Shared framework across applications. Explain that research questions, data, and statistical tests can differ while the seven-step Neyman–Pearson framework remains the same, supporting consistent and reproducible decisions under uncertainty.
- Fill-volume hypotheses and direction. State the null and alternative hypotheses for the 12-ounce fill target, and explain why the test is two-tailed and the hypotheses are set before examining the sample.
- Significance level and error trade-off. Explain what α = 0.05 limits in this example and how choosing a smaller or larger α changes the trade-off between false alarms and missed process shifts.
- Test choice and standardized difference. Justify the one-sample, two-tailed t-test and describe what its test statistic measures for the container sample.
- Rejection rule and observed decision. Trace how the preset two-tailed rejection rule is applied to the sample’s t-statistic or p-value to decide whether to reject the target-mean null.
- Evidence-based quality-control conclusion. Distinguish the decision to reject H₀ from proof that the process is off target, and describe the action the evidence supports.
- Safety monitoring hypotheses. State the null and alternative hypotheses for the aircraft sensor and explain why the test looks only for unusually high readings.
- Safety driven error trade-off. Justify the choice of α = 0.10 using the relative costs of missed faults and false alarms, including the trade-off the airline accepts.
- Vibration z-test and score. Explain why the engineers select a one-sample, one-tailed z-test and describe how its z-score is calculated and interpreted for this reading.
- Predefined rule and precautionary action. Trace how the engineers set the rejection rule before examining the result, apply it to the reading, and translate the decision into action without claiming certainty.
- Prespecified rejection rule and decision. Trace how the 5% significance level sets the rejection threshold and why 14 recoveries do not meet it.
- Careful interpretation of borderline evidence. Explain what failing to reject means here, why the cutoff must not be changed afterward, and how the trial result should be reported and followed up.
- Pre-specified decision framework. Explain how the same seven-step hypothesis-testing framework turns uncertainty into a disciplined decision across different applications by establishing hypotheses, significance levels, tests, and decision rules before analyzing data.
- Context-dependent testing choices. Describe why applying the common framework requires choices tailored to the research question, data, assumptions, and consequences of error, including the relative importance of Type I and Type II errors.
- Evidence without certainty. Distinguish a hypothesis-test decision from proof or disproof, and explain that the pre-specified process supports decisions under uncertainty while controlling the long-run probability of error.
- Evidence to decision framework. Explain how Neyman and Pearson’s contribution differs from a statistical test or estimation method: it provides a disciplined, transparent, and reproducible way to decide when evidence justifies a conclusion and action, while recognizing that the decision could be wrong.
- Evidence and action decisions. Explain why statistical evidence informs rather than determines an action, and how the Neyman–Pearson framework uses predefined decision rules and the risks of Type I and Type II errors to judge whether evidence justifies a conclusion and action.
- Reproducible statistical decisions. Explain why researchers applying the same Neyman–Pearson framework to the same data should reach the same statistical conclusion.
- Framework Beyond Individual Tests. Explain how different statistical tests can follow the same Neyman–Pearson decision-making framework, and why that framework remains relevant beyond any particular test.
- Enduring decision framework. Explain why the Neyman–Pearson framework endures as a distinct procedure rather than a particular statistical test: it defines hypotheses, evaluates evidence, and reaches a conclusion using a pre-specified decision rule.
What it builds on
- From prior to posterior: how Bayes’ Theorem works (chapter 2) · Bayesian updating of beliefs from evidence
- Maximum likelihood estimation (chapter 3) · Maximum likelihood estimation of unknown parameters
- Fisher’s approach to statistical inference (chapter 3) · Fisher’s approach extracts evidence from data.
Section by section
Introduction
The introduction explains why learning from data is not enough to make decisions and how the Neyman–Pearson framework adds explicit, advance rules for controlling errors.
- Limits of statistical inference: Distinguish learning from evidence from deciding what conclusion or action should follow. Explain why inference alone does not provide a formal way to balance false positives and false negatives.
Why it matters: It clarifies why making a decision requires more than measuring what the data suggest. - Advance rules for decisions: Explain how the Neyman–Pearson framework turns statistical evidence into disciplined decisions by specifying hypotheses, error rates, and decision rules in advance.
Why it matters: Advance commitments make the move from evidence to action more transparent and reproducible.
Key terms: inference, false positives, false negatives, Neyman-Pearson framework, hypotheses, error rates, decision rules
4.1 The stepwise framework of hypothesis testing
This section frames Neyman and Pearson’s mathematical approach as a practical, interrelated workflow for moving from a research question to a statistically supported conclusion.
- Interrelated hypothesis testing workflow: Explain that the seven-step sequence is a practical way to view Neyman and Pearson’s mathematical framework, with each step building on earlier decisions or results. State that some choices, such as the significance level, are made before analyzing the data.
Why it matters: This framing helps you follow how a test moves from a question to a supported conclusion without treating its steps as isolated tasks.
Key terms: hypothesis testing, seven interrelated steps, significance level
4.1.1 Step 1: state the hypotheses
This section explains how to define competing hypotheses and fix the test’s structure before analyzing data, so evidence can support an objective decision.
- Competing hypothesis roles: Explain how a research question becomes a null hypothesis and an alternative hypothesis, and describe what each represents when evaluating evidence.
Why it matters: The claims you choose determine what the test evaluates and what conclusion it can support. - Initial test structure: Distinguish simple from composite hypotheses and one-sided from two-sided tests, and explain why these choices belong at the start of a test.
Why it matters: Choosing the parameter claims and relevant directions makes clear what kind of departure the test is meant to detect. - Fixed preanalysis design: Justify defining the competing claims, parameter, and test structure before seeing the data, and keeping them fixed during analysis.
Why it matters: Fixing the rules in advance helps ensure that the evidence, rather than hindsight, determines the conclusion.
Key terms: null hypothesis, alternative hypothesis, status quo, simple, composite, parameter of interest, one-sided, two-sided, hypotheses, structure of the test
4.1.2 Step 2: choose a significance level
This section explains how to choose a significance level before testing, based on the consequences of false positives and false negatives.
- Significance level and testing errors: Explain what α limits and distinguish Type I from Type II errors in terms of the null hypothesis.
Why it matters: These definitions make clear which mistake α is intended to limit. - Contextual error risk tradeoff: Describe how choosing a smaller or larger α changes the balance between false positives and false negatives, and why the appropriate value depends on their consequences.
Why it matters: A deliberate α choice reflects which testing error would be more costly in the situation at hand. - Advance selection of alpha: Justify selecting α before examining the data and keeping it fixed throughout the test.
Why it matters: Fixing the rule before seeing results prevents the particular sample from determining the test’s claimed error standard.
Key terms: significance level, α, Type I error, Type II error, null hypothesis, false positive, false positives, false negatives, 0.05, conservative
4.1.3 Step 3: select the test
This section explains how to choose a statistical test that fits the question and data, commit to it before analysis, and prefer greater power when valid alternatives exist.
- Test fit and assumptions: Explain how the research question, data type, and a test’s assumptions guide test selection, and why a poor match can make results unreliable or reduce power.
Why it matters: The test must fit the problem for its result to be useful evidence. - Advance commitment to the test: Explain why the statistical test is selected before analyzing the data and kept fixed during the analysis.
Why it matters: A fixed test keeps the decision from being tailored to the result it is meant to evaluate. - Power among valid tests: Explain how power guides a choice among tests for the same hypotheses while Type I error remains controlled, and state the limited case established by the Neyman–Pearson Lemma.
Why it matters: Among suitable tests, the aim is to detect real effects without giving up control of false positives.
Key terms: research question, type of data, assumptions, statistical power, design decision, decision rule, predetermined rules, Neyman–Pearson Lemma, power, Type I error
4.1.4 Step 4: compute the test statistic
This section explains how the selected test turns observed data into a test statistic and why that value is not yet a decision about the hypotheses.
- Purpose of the test statistic: Explain that a test statistic uses the selected test’s formula to summarize the observed sample in one value that can be evaluated against the null hypothesis, but does not itself establish a conclusion.
Why it matters: The number gives the test something to evaluate without being the decision itself. - Predetermined statistic calculation: Describe why computing the statistic executes an already chosen test rather than introducing a new methodological choice: the selected test and observed data determine the calculation, whether it is done manually or by software.
Why it matters: Keeping the calculation tied to the chosen test makes it an execution of the planned procedure, not a new choice prompted by the result.
Key terms: test statistic, observed data, null hypothesis, statistical test
4.1.5 Step 5: establish the decision rule
This section explains how a predetermined significance level becomes a decision rule for evaluating a hypothesis-test statistic.
- Significance level and critical region: Explain how the chosen significance level and the test’s sampling distribution define the critical region, and describe the two-tailed 5% example.
Why it matters: It connects the error tolerance chosen in advance to a concrete standard for judging a test statistic. - Equivalent decision rule expressions: Explain how critical regions, critical values, and a p-value comparison express the same decision rule.
Why it matters: It lets the reader recognize the same rejection standard whether it is stated as a region, a cutoff, or a p-value comparison. - Advance commitment to the rule: Justify defining the decision rule before examining the results, and distinguish establishing the rule from applying it.
Why it matters: A rule fixed before looking at results is less vulnerable to being adjusted to favor the observed outcome.
Key terms: significance level, α, Type I error, critical region, sampling distribution, two-tailed test, critical values, p-value, null hypothesis, decision rule
4.1.6 Step 6: make a decision
This section explains how a hypothesis test reaches a predefined reject-or-fail-to-reject decision while preserving uncertainty about the truth and practical meaning of the result.
- Formal hypothesis test decision: Describe how the observed test statistic or its p-value is compared with the previously established criterion to decide whether to reject or fail to reject the null hypothesis. Distinguish that formal decision from proving a hypothesis or interpreting its practical importance.
Why it matters: A test gives you a reproducible statistical decision, not a final answer to the practical question. - Errors behind either decision: Explain the possible error associated with each decision and how the significance level and statistical power relate to those risks.
Why it matters: Recognizing both error risks keeps a formal decision from sounding more certain than it is. - Continuous evidence at a cutoff: Explain why two p-values close to a significance threshold can lead to different formal decisions despite providing nearly identical evidence.
Why it matters: A cutoff makes decisions consistent without making nearly identical results fundamentally different.
Key terms: test statistic, p-value, null hypothesis, reject, fail to reject, decision rule, Type I error, Type II error, significance level, α
4.1.7 Step 7: draw a conclusion
This section explains how to turn a hypothesis-test decision into a contextual conclusion that communicates evidence, uncertainty, and practical importance.
- Evidence Without Certainty: Explain what rejecting or failing to reject the null hypothesis allows you to conclude, without treating either decision as proof.
Why it matters: A test result supports a claim about evidence, not a claim of certainty. - Contextual and Transparent Conclusions: Describe how to report a test decision in terms of the original research question while stating relevant limitations and showing how the decision was reached.
Why it matters: Readers can evaluate a conclusion only when they can see both its real-world meaning and its limits. - Statistical Versus Practical Significance: Distinguish statistical significance from practical importance and describe what else to consider when judging a finding.
Why it matters: A detectable effect is not automatically an effect worth acting on.
Key terms: null hypothesis, alternative hypothesis, fail to reject, statistical power, research question, limitations, assumptions, significance level, statistical test, statistical significance
4.1.8 From framework to practice
This section explains why Neyman and Pearson’s seven-step framework makes hypothesis testing a disciplined and reproducible way to draw conclusions under uncertainty.
- Purpose of the seven-step framework: Explain how separating test design from execution, interpretation, and communication makes the evaluation of evidence transparent and reproducible while explicitly managing the risk of error.
Why it matters: A structured test makes it easier to see how evidence led to a conclusion and where error risk was considered.
Key terms: seven-step framework, hypothesis testing, risk of error
4.2 Hypothesis testing in action
Introduces how the same seven-step hypothesis-testing framework applies across different problems to support consistent, reproducible decisions.
- Shared framework across applications: Explain that research questions, data, and statistical tests can differ while the seven-step Neyman–Pearson framework remains the same, supporting consistent and reproducible decisions under uncertainty.
Why it matters: It explains why one testing framework can be useful across different kinds of problems.
Key terms: seven-step framework, Neyman–Pearson procedure, reproducible decision-making
4.2.1 Example 1: beverage fill volumes
The section works through a beverage fill-volume test to show how preselected hypotheses and error limits turn sample evidence into a decision without claiming proof.
- Fill-volume hypotheses and direction: State the null and alternative hypotheses for the 12-ounce fill target, and explain why the test is two-tailed and the hypotheses are set before examining the sample.
Why it matters: Setting the question first prevents the observed fill measurements from determining which result counts as a deviation. - Significance level and error trade-off: Explain what α = 0.05 limits in this example and how choosing a smaller or larger α changes the trade-off between false alarms and missed process shifts.
Why it matters: The error threshold reflects the practical costs of unnecessary intervention versus an undetected fill problem. - Test choice and standardized difference: Justify the one-sample, two-tailed t-test and describe what its test statistic measures for the container sample.
Why it matters: Standardizing the observed difference makes it possible to judge whether the sample is unusual under the target-mean assumption. - Rejection rule and observed decision: Trace how the preset two-tailed rejection rule is applied to the sample’s t-statistic or p-value to decide whether to reject the target-mean null.
Why it matters: A rule fixed before seeing the result makes the rejection decision transparent and reproducible. - Evidence-based quality-control conclusion: Distinguish the decision to reject H₀ from proof that the process is off target, and describe the action the evidence supports.
Why it matters: The test guides a practical response while keeping statistical evidence distinct from certainty.
Key terms: null hypothesis, alternative hypothesis, two-tailed hypothesis test, simple hypothesis, composite hypothesis, significance level, α, Type I error, one-sample, two-tailed t-test, population standard deviation
4.2.2 Example 2: safety monitoring
This section applies the seven-step hypothesis-testing framework to an aircraft vibration reading, showing how a safety-conscious test turns unusual evidence into a precautionary inspection decision.
- Safety monitoring hypotheses: State the null and alternative hypotheses for the aircraft sensor and explain why the test looks only for unusually high readings.
Why it matters: The hypotheses and direction of concern determine what counts as evidence of a possible fault. - Safety driven error trade-off: Justify the choice of α = 0.10 using the relative costs of missed faults and false alarms, including the trade-off the airline accepts.
Why it matters: The significance level reflects the consequences of each kind of mistake, not just a statistical convention. - Vibration z-test and score: Explain why the engineers select a one-sample, one-tailed z-test and describe how its z-score is calculated and interpreted for this reading.
Why it matters: Standardization gives the engineers a way to judge the reading against normal operating variation. - Predefined rule and precautionary action: Trace how the engineers set the rejection rule before examining the result, apply it to the reading, and translate the decision into action without claiming certainty.
Why it matters: A rule set in advance makes the inspection decision reproducible while keeping the conclusion appropriately cautious.
Key terms: null hypothesis, alternative hypothesis, one-tailed hypothesis test, simple, composite, significance level, α = 0.10, Type I error, Type II error, false positives
4.2.3 Example 3: a small clinical trial
A small clinical trial shows how a prespecified hypothesis test yields a reproducible decision when the evidence narrowly misses statistical significance, without resolving whether the drug works.
- Clinical trial test setup: Explain how the recovery question determines the hypotheses and why a one-tailed exact binomial test suits this small trial.
Why it matters: The question and data determine what counts as evidence for an improvement. - Prespecified rejection rule and decision: Trace how the 5% significance level sets the rejection threshold and why 14 recoveries do not meet it.
Why it matters: A rule chosen before seeing the results makes even a close decision transparent and reproducible. - Careful interpretation of borderline evidence: Explain what failing to reject means here, why the cutoff must not be changed afterward, and how the trial result should be reported and followed up.
Why it matters: A formal decision should guide interpretation without turning uncertain evidence into an unjustified claim.
Key terms: null hypothesis, alternative hypothesis, one-tailed exact binomial test, binary outcomes, significance level, rejection region, test statistic, p-value, fail to reject, p-hacking
4.2.4 From examples to principles
The section draws three principles from the examples: hypothesis testing uses a consistent, pre-specified decision framework, adapts its choices to context, and supports decisions without proving hypotheses.
- Pre-specified decision framework: Explain how the same seven-step hypothesis-testing framework turns uncertainty into a disciplined decision across different applications by establishing hypotheses, significance levels, tests, and decision rules before analyzing data.
Why it matters: Setting the rules in advance makes a decision easier to explain and reproduce. - Context-dependent testing choices: Describe why applying the common framework requires choices tailored to the research question, data, assumptions, and consequences of error, including the relative importance of Type I and Type II errors.
Why it matters: The same procedure can serve different problems only when its choices reflect their circumstances and stakes. - Evidence without certainty: Distinguish a hypothesis-test decision from proof or disproof, and explain that the pre-specified process supports decisions under uncertainty while controlling the long-run probability of error.
Why it matters: A test result can guide action without being mistaken for certainty.
Key terms: seven-step procedure, hypotheses, significance levels, statistical tests, decision rules, Type I versus Type II errors, significance level, underlying assumptions, subject-matter knowledge, statistical evidence
4.3 Why it matters
This section explains why the Neyman–Pearson framework matters as a way to turn statistical evidence into decisions while acknowledging the possibility of error.
- Evidence to decision framework: Explain how Neyman and Pearson’s contribution differs from a statistical test or estimation method: it provides a disciplined, transparent, and reproducible way to decide when evidence justifies a conclusion and action, while recognizing that the decision could be wrong.
Why it matters: The framework addresses the enduring problem of turning evidence into sound decisions without pretending those decisions are infallible.
Key terms: Neyman–Pearson framework, statistical evidence, decisions under uncertainty, possibility of error
4.3.1 From evidence to action
This section explains how the Neyman–Pearson framework turns statistical evidence into a basis for decisions under uncertainty.
- Evidence and action decisions: Explain why statistical evidence informs rather than determines an action, and how the Neyman–Pearson framework uses predefined decision rules and the risks of Type I and Type II errors to judge whether evidence justifies a conclusion and action.
Why it matters: It helps distinguish what the data suggests from whether the evidence warrants acting on it.
Key terms: Neyman–Pearson framework, predefined decision rules, Type I and Type II errors, statistical evidence
4.3.2 Making decisions reproducible
This section explains how pre-specified hypothesis-testing rules make statistical decisions reproducible and protect against conclusions shaped by flexible analysis.
- Reproducible statistical decisions: Explain why researchers applying the same Neyman–Pearson framework to the same data should reach the same statistical conclusion.
Why it matters: Explicit rules let others check and reproduce a statistical decision rather than relying on the analyst’s judgment. - Protection against analysis-driven conclusions: Describe how flexible analysis in modern data science can undermine trustworthy conclusions and how setting inference rules before analysis helps.
Why it matters: Deciding the rules first helps prevent a reported finding from being an artifact of how the analysis was adjusted.
Key terms: Neyman–Pearson framework, hypotheses, significance level, statistical test, decision rule, reproducible, p-hacking, statistical significance, statistical inference
4.3.3 A framework beyond statistics
The section explains why Neyman and Pearson’s decision-making framework remains useful even as statistical methods and fields change.
- Framework Beyond Individual Tests: Explain how different statistical tests can follow the same Neyman–Pearson decision-making framework, and why that framework remains relevant beyond any particular test.
Why it matters: It lets you recognize the same decision logic when methods or applications change.
Key terms: Neyman–Pearson framework, significance level, pre-specified decision rule
4.4 An enduring framework for decision-making
The section explains why the Neyman–Pearson framework remains useful for turning uncertain evidence into disciplined decisions across changing fields and technologies.
- Enduring decision framework: Explain why the Neyman–Pearson framework endures as a distinct procedure rather than a particular statistical test: it defines hypotheses, evaluates evidence, and reaches a conclusion using a pre-specified decision rule.
Why it matters: It provides a consistent way to make decisions even as the available statistical tools change. - Applications across changing domains: Describe how the same decision framework applies in established and modern settings by distinguishing a meaningful change or problem from ordinary or random variation.
Why it matters: The same evidence-versus-variation question arises in decisions from manufacturing to model monitoring.
Key terms: Neyman–Pearson framework, pre-specified decision rule, statistical conclusion, routine process variation, random variation, model drift, anomaly detection
4.5 Summary
Where this is used later
- (chapter intro) (chapter 5) · Earlier treatment of drawing conclusions while controlling statistical errors.
- Neyman and Pearson: making decisions while controlling errors (chapter 10) · The later synthesis revisits Neyman and Pearson’s approach to decisions and error control.
Concepts explained in this chapter
- decision ruleA decision rule is a specified mapping from observed evidence or a test statistic to an action or hypothesis-testing decision.
- hypothesis testingHypothesis testing, also called statistical testing, formally evaluates evidence against a null hypothesis under specified assumptions.
- level of significanceA significance level is the maximum probability of rejecting a null hypothesis when it is true, specified for a statistical test.
- null hypothesisA null hypothesis is a baseline statistical claim, commonly asserting that an effect or difference is absent.
- critical regionA rejection region is the set of test-statistic outcomes for which a hypothesis-testing procedure rejects the null hypothesis.
- alternative hypothesisA statistical claim of an effect, difference, or relationship that is contrasted with the null hypothesis.
- p-valueA p-value is the probability, under a null hypothesis, of observing a result at least as extreme as the one obtained.
- test statisticA test statistic is a numerical quantity calculated from data and used to assess a statistical hypothesis.
- one-sided testA one-tailed test is a hypothesis test whose rejection region lies in one direction of the test-statistic distribution.
- critical valueA critical value is a boundary value of a test statistic that separates the rejection region from the non-rejection region.
- Neyman–Pearson frameworkA statistical framework for making hypothesis-testing decisions by controlling error rates under specified hypotheses.
- false positiveA positive test result or classification when the tested condition is absent.
- statistical significanceStatistical significance is an assessment of whether an observed result is sufficiently unlikely under a specified null hypothesis.
- t-testA t-test is a statistical test for comparing means using a t-distribution under specified assumptions.
- two-tailed testA two-tailed test is a hypothesis test whose rejection region includes extreme values in both directions of the test-statistic distribution.
- binomial testA binomial test is an exact statistical test for assessing whether the observed number of successes in fixed independent binary trials is consistent with a specified success probability.
- decision theoryThe formal study of how decision-makers choose among actions under uncertainty and in light of preferences or objectives.
- z-scoreA z-score is a standardized value expressing a measurement's distance from a mean in units of standard deviation.
- z-testA z-test is a statistical hypothesis test that uses a standardized statistic with a standard normal reference distribution under specified assumptions.
- evidenceEvidence is information or observations considered in support of a claim or conclusion.
- practical importancePractical significance is the real-world importance or usefulness of an effect, independent of whether it meets a statistical significance threshold.
- Neyman–Pearson lemmaThe Neyman–Pearson lemma states that a likelihood-ratio rejection rule is most powerful for testing a simple null hypothesis against a simple alternative at a specified significance level.
- uncertaintyUncertainty is the lack of complete information about the state of a system, a quantity, or the outcome of an event.
- composite hypothesisA hypothesis that permits multiple parameter values or probability models.
- simple hypothesisA simple hypothesis specifies one exact value for an unknown parameter or one fully specified probability model.
- standard deviationThe square root of variance, measuring dispersion in the same units as the quantity described.
- tradeoffA trade-off is a decision situation in which improving one outcome requires accepting a reduction in another.
- p-hackingP-hacking is the practice of selectively changing analyses or reporting choices to obtain apparently statistically significant results.
- statistical powerStatistical power is the probability that a test rejects the null hypothesis when a specified alternative condition is true.
- false negativeA negative test result or classification when the tested condition is present.
- assumptionA mathematical assumption is a premise about mathematical properties or conditions required for a result to hold.
- riskRisk is the possibility of an undesirable outcome together with uncertainty about its occurrence or impact.
- error rateAn error rate is the proportion of predictions or classifications that are incorrect.
These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.
