Chapter 6: Coincidence? I think not!

Introduction

The chapter introduction explains the basic logic of testing a claim against sample evidence and distinguishes three ways samples can be compared.

• Null and alternative claims: Explain how the advertised 305 mL soda-can amount serves as a null hypothesis and the claim that cans do not contain about 305 mL serves as its alternative.
  Why it matters: Separating the claim from its competing claim makes clear what the evidence is meant to challenge.
• Sample evidence against a claim: Describe how a sizable sample and its mean and standard deviation help assess whether a population claim is worth challenging, and explain the intuition behind an unlikely sample.
  Why it matters: This keeps an ordinary sample difference from being mistaken for compelling evidence against a population claim.
• Three sample comparison types: Distinguish one-sample, independent-samples, and paired-samples tests by what each compares, including what counts as a data point in a paired test.
  Why it matters: The comparison being made determines which kind of hypothesis test the example calls for.

Key terms: hypothesis, null hypothesis, alternative hypothesis, hypothesis testing, sample mean, standard deviation, population, one-sample, independent samples, paired samples

6.1 Hypothesis testing big ideas

The tea-tasting experiment introduces null and alternative hypotheses, the meaning of a p-value, and the limits of rejecting a hypothesis.

• Competing experimental hypotheses: Describe the null and alternative hypotheses in Muriel’s tea-tasting experiment and what each claims.
  Why it matters: Naming the competing claims makes clear which explanation the experiment puts to the test.
• Chance under the guessing assumption: Explain what the roughly 1.4% p-value means in the tea experiment and distinguish it from the probability that Muriel was guessing after seeing the result.
  Why it matters: Keeping the direction of the probability straight prevents a common misreading of a p-value.
• Limits of rejecting the null: Explain why Muriel’s result gives reason to reject the guessing hypothesis without proving that she has the claimed ability.
  Why it matters: Statistical evidence can challenge an explanation without turning a competing explanation into a certainty.

Key terms: null hypothesis, H0, alternative hypothesis, Ha, p-value, guessing at random, posterior, reject the null hypothesis

6.2 Chasing the truth

The section presents hypothesis testing as an ongoing effort to weigh evidence and revise beliefs, rather than a way to prove absolute truths.

• Provisional conclusions from evidence: Explain why hypothesis testing rarely proves or disproves the null or alternative hypothesis conclusively, and how new evidence can change even well-established explanations.
  Why it matters: It keeps a test result from being mistaken for final proof.
• Humility and active refutation: Describe the attitude the section recommends for objective researchers: remain open to being wrong, guard against confirming their own biases, and try to refute compelling evidence before taking a firm stand.
  Why it matters: Trying to challenge a favored conclusion makes a firm position more defensible.

Key terms: null hypothesis, alternative hypothesis, hypothesis testing, evidence, objective scientists, biases, compelling evidence

6.3 One-sample tests for normal/t-distributions

This section distinguishes one-sample Z-tests from t-tests and explains why t-tests are more common in practice.

• One-sample Z and t-tests: Distinguish Z-tests from t-tests by the distributions they use to compare a sample mean with a population mean, and explain why t-tests are more common in practice.
  Why it matters: This distinction helps you recognize what each test compares and why a researcher might favor a t-test.

Key terms: Z-tests, t-tests, standard normal distribution, t-distribution, sample mean, population mean

6.4 Two-tailed tests

Introduces two-tailed tests as a way to challenge an existing estimate of a population mean using a sample mean.

• Testing an alleged population mean: Explain that a two-tailed test checks whether a sample mean differs from an alleged population mean, which may be a best estimate based on previous studies rather than a known population value.
  Why it matters: It clarifies what claim the test examines without treating an earlier estimate as certain truth.

Key terms: two-tailed tests, sample mean, alleged population mean

6.5 Level of significance for two-tailed tests

The section explains how a two-tailed test’s significance level relates to its confidence level and how the remaining area is divided between the tails.

• Significance and two-tailed area: Explain that the significance level α is the area outside the central confidence level, so α = 1 − C, and that a two-tailed test divides α equally between its tails. Apply this to 95% confidence: α is .05, with .025 in each tail.
  Why it matters: This distinction prevents you from assigning the entire significance level to each tail of a two-tailed test.

Key terms: level of significance, alpha, level of confidence, two tails, critical Z-value

6.6 A two-tailed test

This section uses a potato-chip bag example to set up a two-tailed test and decide whether a sample provides enough evidence to reject an advertised mean.

• Two-tailed hypotheses: Explain why testing whether the mean differs from 142 grams in either direction is two-tailed, and state the null and alternative hypotheses.
  Why it matters: A two-tailed test looks for a difference on either side of the claimed mean, not just a shortage.
• Sampling and chance variation: Justify collecting bags from different stores and explain why a sample mean of 140.2 grams does not, by itself, disprove a 142-gram population mean.
  Why it matters: A measured difference is not automatically convincing evidence against an advertised average.
• Standardized test value: Describe why the section converts the sample mean to a test Z-value and identify the value obtained for the chip-bag sample.
  Why it matters: Putting the observed difference on the standard normal scale lets the test compare it with a rejection threshold.
• Critical-range decision: Apply the section’s 95% critical range to the test Z-value and state the conclusion about the advertised mean.
  Why it matters: The critical range turns the observed difference into a stated reject-or-fail-to-reject decision.

Key terms: two-tailed testing, null hypothesis, alternative hypothesis, H0, Ha, sample, sample mean, sampling bias, variability, standard normal distribution

6.7 Two-tailed p-values

This section explains what a two-tailed p-value measures, how it comes from both tails of a test distribution, and how to compare it with a chosen significance threshold.

• Meaning of a p-value: Explain that a p-value is the probability of a sample result at least as extreme as the observed one, assuming the null hypothesis is true, and distinguish it from the probability that the null hypothesis is true.
  Why it matters: Interpreting the probability correctly prevents a common mistake about what a test has shown.
• Two-tailed p-value area: Describe how a two-tailed p-value uses the areas beyond the observed test Z-value and its symmetric counterpart, and distinguish those boundaries from the preset boundaries for α.
  Why it matters: The two tail areas show which results count as at least as extreme in either direction.
• Decision by p-value comparison: Apply the comparison of p-value with α to decide whether to reject the null hypothesis, using the section’s 0.016 example.
  Why it matters: One comparison gives the test decision for a stated significance level.
• Significance thresholds chosen in advance: Justify choosing a significance threshold with the experiment’s objective in mind before looking at results, rather than changing it afterward to obtain rejection.
  Why it matters: A threshold chosen before seeing the data helps keep a convenient comparison from becoming a way to shop for a favorable result.

Key terms: p-value, null hypothesis, two-tailed test, test Z-value, α, tail area, norm.cdf(), reject the null hypothesis, fail to reject, level of significance

6.8 One-tailed tests

This section explains when a one-tailed test makes sense and how its directional hypotheses differ from those of a two-tailed test.

• One-tailed test tradeoff: Explain that a one-tailed test considers only one direction, why a two-tailed test is generally preferable, and what changes about the tails used for α and the p-value.
  Why it matters: The direction you choose determines which outcomes your test is set up to consider.
• Directional chip-bag hypotheses: State the null and alternative hypotheses for testing whether the mean chip-bag weight is below the advertised 142 grams, and explain why they are opposites.
  Why it matters: Writing complementary hypotheses makes clear which direction the test examines and what the null covers.

Key terms: one-tailed test, two-tailed tests, α, p-value, null hypothesis, alternative hypothesis, H0, Ha, μ

6.9 Level of significance for one-tailed tests

The section explains how a one-tailed test places its significance area and critical Z boundary on the side specified by the alternative hypothesis.

• Direction of the significance tail: Explain how the alternative hypothesis determines whether a one-tailed test places its significance area in the left or right tail.
  Why it matters: The direction of the alternative hypothesis tells you which tail contains the significance area.
• Single critical Z boundary: Describe why a one-tailed test uses a critical Z boundary on only one side, and state what happens when a test value falls in the critical region.
  Why it matters: A single-sided boundary lets you identify the relevant critical region without treating the test as two-tailed.
• Opposite direction Z results: Explain what the section says about a Z-value pointing opposite to the chosen tail.
  Why it matters: The sign of the result can show that it points away from the direction the test was set up to detect.

Key terms: one-tailed tests, significance area, left-tailed test, right-tailed test, alternative hypothesis Ha, critical Z-value, critical Z boundary, critical region, null hypothesis, Z-value

6.10 A one-tailed test

This section applies a left-tailed Z-test to the potato chip example and explains why a one-tailed test is less conservative than a two-tailed test at the same significance level.

• Left-tailed chip-bag decision: Trace the potato chip test from its hypotheses through the Z-value comparison to its conclusion about the bag’s weight.
  Why it matters: It connects a directional claim to a test decision and its practical interpretation.
• One-tail versus two-tail conservatism: Explain why, at the same alpha level, this one-tailed test makes rejection easier than a two-tailed test.
  Why it matters: It explains why choosing the test’s direction changes the threshold for rejecting the null.

Key terms: H0, Ha, test Z-value, critical Z-value, alpha region, left-tailed test, one-tailed test, two-tailed test, alpha level, critical values

6.11 One-tailed p-values

The section explains how to interpret one-tailed p-values, use them to reject a null hypothesis, and decide when a one-tailed test is appropriate.

• One-tailed p-value areas: Explain that a one-tailed p-value measures outcomes at least as extreme as the observed sample in one direction, assuming the null hypothesis is true. Identify the area left of the test Z-value for a left-tailed test and right of it for a right-tailed test.
  Why it matters: The chosen direction determines which probability counts as the p-value.
• P-value rejection decision: Apply the rule that a p-value below α leads to rejection of the null hypothesis. Explain why the potato chip example’s left-tailed p-value of 0.0081 leads to rejection at α = .05.
  Why it matters: Comparing the p-value with α turns the observed result into a test decision.
• Directional test tradeoff: Distinguish when to choose a one-tailed rather than a two-tailed test. Explain that a one-tailed test can have a smaller p-value when results align with the predicted direction, but ignores potential evidence in the other direction.
  Why it matters: Choosing the tail to match the question prevents a smaller p-value from hiding a relevant change in the opposite direction.

Key terms: p-value, one-tailed test, test Z-value, left-tailed test, right-tailed test, α, null hypothesis, two-tailed test, LEFT_TAIL, TWO_TAIL

6.12 Hypothesis testing with smaller samples

This section applies a t-distribution to a small-sample hypothesis test and interprets the result for a college exam-score claim.

• Small-sample t-distribution choice: Explain why this section uses a t-distribution rather than a standard normal distribution for small-sample hypothesis tests, and state how to set its degrees of freedom.
  Why it matters: Choosing the distribution determines how the test accounts for uncertainty in a small sample.
• Directional exam-score test setup: Describe why the college's claim that its students score above the population average calls for a right-tailed, one-tailed t-test at α = 0.01.
  Why it matters: The wording of the claim determines which tail contains evidence against the null hypothesis.
• Exam-test evidence and conclusion: Explain the decision from the exam test's p-value and state what can be reported to the administrator.
  Why it matters: The result distinguishes a sample mean above 100 from sufficient evidence that the college's population mean is higher.

Key terms: t-distribution, standard normal distribution, degrees of freedom, population variance, alternative hypothesis, right-tailed test, one-tailed test, t-value, level of significance, p-value

6.13 Hypothesis testing with proportions

This section applies hypothesis testing to a population proportion, checking sample-size suitability and using a p-value to evaluate a one-sided claim.

• Proportion test setup: Describe how the social-media example sets up a right-tailed hypothesis test, distinguishing the claimed population proportion from the observed sample proportion.
  Why it matters: Keeping the claim separate from the sample result makes the direction of the test clear.
• Sample size prerequisite: Explain what must be checked before performing the proportion hypothesis test, without specifying rules that are not given in this section's text.
  Why it matters: The section treats an adequate sample size as a prerequisite to using this test.
• P-value decision for claim: Explain the decision in the social-media example by comparing its p-value with the stated significance level.
  Why it matters: Comparing the p-value with the chosen threshold connects the sample result to a decision about the claim.

Key terms: proportion, p̂, p, right-tailed test, alternative hypothesis, sample size, hypothesis testing on proportions, p-value, alpha, significance level

6.14 Independent samples

The section adapts hypothesis testing to two independent classroom samples, interprets the resulting test, and notes a limitation of the large-sample approach.

• Independent sample hypotheses: Explain how the claim that Classroom B scores lower than Classroom A determines the hypotheses and the direction of the test.
  Why it matters: The claim determines both what is being tested and which tail to examine.
• Two-sample Z-test adaptation: Describe why the Z-test statistic is adapted for two samples and how the hypothesized difference and sample standard deviations enter the approach.
  Why it matters: A two-sample statistic must account for uncertainty from both classrooms, not just one sample.
• Classroom test conclusion: Explain what conclusion the section draws from the reported p-value at the stated significance level.
  Why it matters: The numerical result matters only when connected back to the original claim about the classes.
• Large-sample method caveat: Explain why the section cautions against treating a sample size above 30 as automatically sufficient for this approach.
  Why it matters: The classroom result should be read with the assumptions of its test in mind.

Key terms: independent samples, null hypothesis, alternative hypothesis, right-tailed test, Z-test statistic, μ1 − μ2, s1 and s2, σ1 and σ2, p-value, 5% level of significance

6.15 Summary