Home › Learning Hub › A Level Maths › S2 5: Hypothesis tests
S2 5

Hypothesis tests

Probability & Statistics 2 · Paper 6

🎯What you need to be able to do

  • Formulate null and alternative hypotheses, and decide between one and two tails.
  • Carry out a test on a population mean using the normal distribution.
  • Carry out a test on a binomial proportion using exact binomial probabilities.
  • Carry out a test on a Poisson mean.
  • Find a critical region and the probability of a Type I error.
  • Define and calculate the probability of a Type II error.
  • State a conclusion in context, without overclaiming.

📚The mathematics

The structure of a test

Every test has the same five steps, and writing them out is how the marks are earned:

  1. State \( \mathrm{H}_0 \) and \( \mathrm{H}_1 \) in terms of a parameter (\( \mu \), \(p\) or \( \lambda \)) — never in words alone.
  2. State the distribution of the test statistic assuming \( \mathrm{H}_0 \) is true.
  3. Calculate the test statistic, or the probability of the observed result or something more extreme.
  4. Compare with the significance level, or with the critical value.
  5. State the conclusion in context.

The null hypothesis is always an equality, \( \mathrm{H}_0: \mu = 500 \). The alternative carries the direction being investigated:

  • One-tailed: \( \mathrm{H}_1: \mu > 500 \) or \( \mathrm{H}_1: \mu < 500 \) — the question suspects a specific direction (“has the mean fallen?”).
  • Two-tailed: \( \mathrm{H}_1: \mu \neq 500 \) — the question asks only whether the mean has changed, with no direction implied.
Decide the number of tails from the question, before seeing the data. For a two-tailed test at 5%, each tail carries 2.5% and the critical value is \( \pm 1.96 \), not \( \pm 1.645 \). Choosing one tail because the data happened to fall that way is not a valid test — it doubles the real error rate. The phrase “has it changed?” means two tails; “has it increased?” means one.

Testing a population mean

When the population is normal with known \( \sigma \), or the sample is large enough for the Central Limit Theorem (S2 4):

\[ z = \frac{\bar{x} - \mu_0}{\sigma/\sqrt{n}} \]

Compare \(z\) with the critical value, or compare the corresponding tail probability with the significance level. The two routes are equivalent; the \(p\)-value route is often clearer to mark and harder to get backwards.

Critical values worth knowing: one-tailed 5% is \( 1.645 \), one-tailed 1% is \( 2.326 \), two-tailed 5% is \( 1.96 \), two-tailed 1% is \( 2.576 \).

Testing a binomial proportion

For small \(n\), use the exact binomial distribution rather than an approximation. Under \( \mathrm{H}_0: p = p_0 \), calculate the probability of the observed value or more extreme, and compare it with the significance level.

“More extreme” means in the direction of \( \mathrm{H}_1 \): for \( \mathrm{H}_1: p > p_0 \) with 10 observed, that is \( \mathrm{P}(X \ge 10) \) — including the observed value itself. Excluding it is a frequent and consequential slip.

Testing a Poisson mean

Identical in structure, using \( \mathrm{Po}(\lambda_0) \) under \( \mathrm{H}_0 \), with \( \lambda \) scaled to the interval given (S2 1). For large \( \lambda \) the normal approximation may be used, with a continuity correction.

Critical regions and Type I errors

The critical region is the set of values of the test statistic that would lead to rejecting \( \mathrm{H}_0 \). For a discrete distribution it is found by trying successive values until the tail probability first drops below the significance level.

The probability of a Type I error — rejecting \( \mathrm{H}_0 \) when it is in fact true — is the probability of landing in the critical region given \( \mathrm{H}_0 \). For a continuous distribution that equals the significance level exactly. For a discrete one it is usually less, because the region cannot be tuned to hit 5% precisely.

A Type II error is failing to reject \( \mathrm{H}_0 \) when it is false. Its probability can only be calculated when a specific alternative value of the parameter is supplied, because “false” on its own does not say by how much.

Never write “accept \( \mathrm{H}_0 \)”. The correct phrase is there is insufficient evidence to reject \( \mathrm{H}_0 \). Failing to find evidence of a change is not evidence that nothing changed — a small sample may simply be unable to detect a real effect. And the conclusion must name the context: “there is evidence at the 5% level that the mean weight of the packets has fallen below 500 g”, not “reject \( \mathrm{H}_0 \)”.

✏️Worked example

(a) A machine fills packets to a mean weight of 500 g with standard deviation 8 g. A random sample of 40 packets has mean weight 497.2 g. Test at the 5% significance level whether the mean weight has fallen. (b) A treatment is claimed to succeed 30% of the time. In a trial of 20 patients, 10 recover. Test at the 5% level whether the success rate is higher than claimed. (c) Find the critical region for the test in (b), and the probability of a Type I error.

(a) \( \mathrm{H}_0: \mu = 500 \), \( \mathrm{H}_1: \mu < 500 \) — one-tailed, because the question asks specifically whether the mean has fallen.

With \( n = 40 > 30 \), the Central Limit Theorem gives \( \bar{X} \sim \mathrm{N}\!\left(500, \dfrac{64}{40}\right) \) under \( \mathrm{H}_0 \). The standard error is \( \dfrac{8}{\sqrt{40}} = 1.2649 \), so

\[ z = \frac{497.2 - 500}{1.2649} = -2.214 \]

The critical value for a one-tailed 5% test is \( -1.645 \). Since \( -2.214 < -1.645 \), the result lies in the critical region. Equivalently, the \(p\)-value is \( \Phi(-2.214) = 0.0134 < 0.05 \).

Conclusion: there is evidence at the 5% level that the mean weight of the packets has fallen below 500 g.

(b) \( \mathrm{H}_0: p = 0.3 \), \( \mathrm{H}_1: p > 0.3 \). Under \( \mathrm{H}_0 \), \( X \sim \mathrm{B}(20, 0.3) \). With \(n\) small, use the exact distribution:

\[ \mathrm{P}(X \ge 10) = 1 - \mathrm{P}(X \le 9) = 0.0480 \]

Since \( 0.0480 < 0.05 \), reject \( \mathrm{H}_0 \).

Conclusion: there is evidence at the 5% level that the success rate of the treatment is higher than 30%.

(c) Test successive values against 5%:

\( \mathrm{P}(X \ge 9) = 0.1133 > 0.05 \)
\( \mathrm{P}(X \ge 10) = 0.0480 < 0.05 \)

So the critical region is \( X \ge 10 \), and the probability of a Type I error is \( 0.0480 \).

Check it. In (c) the Type I error probability, 0.0480, is below the nominal 5% — exactly what should happen for a discrete distribution, where the region can only jump between whole numbers. A value above 0.05 would mean the critical region had been taken one step too far. In (b), note that the observed 10 out of 20 is 50% against a claimed 30%, a large gap, so rejection is unsurprising — though it is only just significant, which is a fair reflection of a trial with only 20 patients.
Include the observed value in the tail. In (b), computing \( \mathrm{P}(X > 10) = 0.0171 \) instead of \( \mathrm{P}(X \ge 10) = 0.0480 \) changes the number by a factor of nearly three. Here both happen to be below 0.05, so the conclusion survives; with 9 recoveries it would not have, since \( \mathrm{P}(X \ge 9) = 0.1133 \) but \( \mathrm{P}(X > 9) = 0.0480 \) — the same slip would flip the verdict.

📝Practise

Work through these, then reveal the answer.

1. A population has known standard deviation 10 and claimed mean 50. A sample of 100 gives a mean of 52. Test at the 5% level whether the mean has changed.
\( \mathrm{H}_0: \mu = 50 \), \( \mathrm{H}_1: \mu \neq 50 \) — "changed" means two-tailed. The standard error is \( \frac{10}{\sqrt{100}} = 1 \), so \( z = \frac{52-50}{1} = 2 \). The two-tailed 5% critical value is 1.96, and \( 2 > 1.96 \), so reject \( \mathrm{H}_0 \). (The \(p\)-value is \( 2(1 - \Phi(2)) = 0.0455 < 0.05 \).) There is evidence at the 5% level that the population mean has changed. Note that against a one-tailed critical value of 1.645 this would have been rejected far more comfortably — the number of tails matters.
2. Explain the difference between a Type I and a Type II error, and give the probability of a Type I error for a continuous test at the 1% level.
A Type I error is rejecting \( \mathrm{H}_0 \) when it is true (a false alarm); a Type II error is failing to reject \( \mathrm{H}_0 \) when it is false (a missed effect). For a continuous test the probability of a Type I error equals the significance level exactly, so 0.01. The probability of a Type II error cannot be stated without a specific alternative value of the parameter.
3. A coin is claimed to be biased towards heads with \( p = 0.4 \) for tails. In 10 tosses only 1 tail appears. Test at the 5% level whether \( p < 0.4 \).
\( \mathrm{H}_0: p = 0.4 \), \( \mathrm{H}_1: p < 0.4 \). Under \( \mathrm{H}_0 \), \( X \sim \mathrm{B}(10, 0.4) \), and the lower tail is \( \mathrm{P}(X \le 1) = 0.6^{10} + 10(0.4)(0.6)^{9} = 0.0060 + 0.0403 = 0.0464 \). Since \( 0.0464 < 0.05 \), reject \( \mathrm{H}_0 \): there is evidence at the 5% level that the proportion of tails is less than 0.4.
4. For the test in question 3, find the critical region and the probability of a Type I error.
\( \mathrm{P}(X \le 1) = 0.0464 < 0.05 \) but \( \mathrm{P}(X \le 2) = 0.1673 > 0.05 \), so the critical region is \( X \le 1 \). The probability of a Type I error is 0.0464 — again below the nominal 5%, as it must be for a discrete test.
5. Faults occur at a mean rate of 5 per week. In one week 10 faults occur. Test at the 5% level whether the rate has increased.
\( \mathrm{H}_0: \lambda = 5 \), \( \mathrm{H}_1: \lambda > 5 \). Under \( \mathrm{H}_0 \), \( X \sim \mathrm{Po}(5) \), and \( \mathrm{P}(X \ge 10) = 1 - \mathrm{P}(X \le 9) = 0.0318 \). Since \( 0.0318 < 0.05 \), reject \( \mathrm{H}_0 \): there is evidence at the 5% level that the fault rate has increased. (The critical region is \( X \ge 10 \), since \( \mathrm{P}(X \ge 9) = 0.0681 > 0.05 \).)
6. A test does not reject \( \mathrm{H}_0 \). A student writes: "This proves the mean is 500." Explain what is wrong, and state the correct conclusion.
A hypothesis test never proves \( \mathrm{H}_0 \); it only asks whether the evidence against it is strong enough. Failing to reject may simply mean the sample was too small to detect a real difference, so a Type II error cannot be ruled out. The correct wording is: "there is insufficient evidence at the 5% level to conclude that the mean differs from 500 g."

🔗Go deeper — other people’s work

These are external resources, not mine. If one stops working, tell me and everything above it on this page still stands.

  • Seeing Theory (Brown University) — frequentist inference and the meaning of a \(p\)-value
  • Khan Academy — significance tests, Type I and Type II errors
  • Cambridge examiner reports — conclusions stated without context lose marks every session