Probability and Statistics 201-SN1-RE • CEGEP, Montreal

Revision sheet: hypothesis tests (201-SN1-RE)

A hypothesis test is rarely lost on the calculation: it is lost on a hypothesis written the wrong way round, a critical value taken from the wrong table or a conclusion that claims more than the data.

What the sheet covers

This sheet gathers the moves that decide the mark, for the three tests of the 201-SN1-RE course: a mean with n≥30n\geq 30, a mean with n<30n<30 using Student's t distribution, a proportion with n≥30n\geq 30.

The thread is always the same: compute UNDER H0H_{0}, with μ0\mu_{0} at the centre and p0p_{0} in the standard error, and conclude only if the data move away from it by more than the level set in advance. Not rejecting H0H_{0} never proves it.

  • 9 mistakes that cost marks
  • 4 figures
  • 15 min reading
Do the chapter's exercises

The thread of the chapter

Everything is computed under H0H_{0}, and nothing proves H0H_{0}: the statistic measures, in standard errors, the distance from the data to the null hypothesis, and only a distance larger than the level set in advance allows a conclusion.

Topics covered

  • Null and alternative hypotheses
  • One-tailed and two-tailed tests
  • Significance level
  • Critical region
  • Test statistic
  • P-value
  • Student's t distribution
  • Test on a mean
  • Test on a proportion
  • Type I error
  • Type II error
  • Power of a test

The essentials

The logic of a test: reasoning under the null hypothesis

  • • H0H_{0} carries the equality on a PARAMETER: μ=μ0\mu=\mu_{0} or p=p0p=p_{0}. H1H_{1} carries what the study wants to establish: ≠\neq (two-tailed), << or >> (one-tailed).
  • • We ASSUME H0H_{0} is true and compute the distribution of the statistic under this hypothesis: centred on μ0\mu_{0}, with the standard error of H0H_{0}.
  • • BEFORE the data, we set the level α\alpha, hence the critical region: the values of the statistic that would be too rare if H0H_{0} were true.
  • • Statistic in the critical region: we reject H0H_{0}. Otherwise, we do not reject it, which does not mean it is true.
01.645−2−1z = 1.20: do not rejectz = 2.30: rejectα = 5%Z under H₀
Under H0H_{0}, ZZ exceeds 1.6451.645 in 5%5\% of cases: z=2.30z=2.30 falls in this orange zone and leads to rejecting H0H_{0}, z=1.20z=1.20 does not.

A test reasons like a proof by contradiction, but a statistical one: it shows that the data are unlikely under H₀, never that H₀ is true.

The three statistics of the course, and their conditions

  • • Mean, n≥30n\geq 30: z=x‾−μ0s/nz=\dfrac{\overline{x}-\mu_{0}}{s/\sqrt{n}}, normal distribution by the central limit theorem, with ss replacing σ\sigma.
  • • Mean, n<30n<30, population assumed NORMAL: t=x‾−μ0s/nt=\dfrac{\overline{x}-\mu_{0}}{s/\sqrt{n}}, Student's t distribution with n−1n-1 degrees of freedom.
  • • Proportion, np0≥5np_{0}\geq 5 and n(1−p0)≥5n(1-p_{0})\geq 5: z=p^−p0p0(1−p0)/nz=\dfrac{\hat{p}-p_{0}}{\sqrt{p_{0}(1-p_{0})/n}}, with p0p_{0} and not p^\hat{p}.
  • • In all three cases, a randomly drawn sample and independent observations (sampling with replacement, or a population large compared with nn).

The conditions are written on the paper before the calculation: a forgotten condition costs the mark even when the statistic is right.

The two errors, and what controls them

  • • Type I error: rejecting H0H_{0} when it is true. Its probability is α\alpha, set by the experimenter.
  • • Type II error: not rejecting H0H_{0} when it is false. Its probability β\beta depends on the true value of the parameter.
  • • Power =1−β=1-\beta: the probability of detecting a gap that exists. For a fixed nn, lowering α\alpha increases β\beta; increasing nn reduces β\beta with α\alpha fixed.

The rules in table form

Each row reads from left to right: the assumptions, then the result. A red cell is not an answer, it is the finding that the form settles nothing and the instruction to rewrite it. Every case is followed by a worked example.

When to reject H0H_{0} at the 5%5\% level

Read the distribution (normal or Student) and the form of H1H_{1}; the last column gives the rejection rule. The two red rows are rules that do not exist.

DistributionH1H_{1}Reject H0H_{0} if
normal μ≠μ0\mu\neq\mu_{0} ∣z∣>1.96|z|>1.96

Example: z=−2.05z=-2.05: ∣z∣>1.96|z|>1.96, we reject.

normal μ>μ0\mu>\mu_{0} z>1.645z>1.645

Example: z=1.80>1.645z=1.80>1.645: we reject, even though 1.80<1.961.80<1.96.

normal μ<μ0\mu<\mu_{0} z<−1.645z<-1.645

Example: z=−1.50>−1.645z=-1.50>-1.645: we do not reject.

Student, 99 df μ≠μ0\mu\neq\mu_{0} ∣t∣>2.262|t|>2.262

Example: n=10n=10, t=2.10<2.262t=2.10<2.262: we do not reject, even though 2.10>1.962.10>1.96.

Student, 99 df μ>μ0\mu>\mu_{0} t>1.833t>1.833

Example: n=10n=10, t=1.90>1.833t=1.90>1.833: we reject.

normal μ>μ0\mu>\mu_{0} ∣z∣>1.96|z|>1.96 no such rule

Example: z=1.80z=1.80: this rule does not reject, the correct rule does.

What to do: One-tailed: the whole area α\alpha is in ONE tail, and the critical value is 1.6451.645 at 5%5\%.

Student, 99 df μ≠μ0\mu\neq\mu_{0} ∣t∣>1.96|t|>1.96 no such rule

Example: t=2.10t=2.10: this rule wrongly rejects, 2.10<2.2622.10<2.262.

What to do: With n<30n<30, the critical value is read from the Student table, with n−1n-1 degrees of freedom.

At 1%1\%: 2.5762.576 two-tailed, 2.3262.326 one-tailed. The smaller the number of degrees of freedom, the further Student's critical value moves from that of the normal distribution.

The mistakes that cost marks

These are the errors I correct most often in session. Each one costs marks on a paper, even when the reasoning behind it is right.

1. Writing the hypotheses about the sample mean

1 mark, and the conclusion loses its object

What not to write

“H0:x‾=120H_{0}:\overline{x}=120 s and H1:x‾<120H_{1}:\overline{x}<120 s.”

What to write

“H0:μ=120H_{0}:\mu=120 s and H1:μ<120H_{1}:\mu<120 s, where μ\mu is the mean reaction time with the catalyst.”

Why: x‾\overline{x} is known as soon as the trials are done: we do not make a hypothesis about a calculated number. The test is about the unknown parameter of the population.

2. Concluding that the null hypothesis is true because it is not rejected

1 to 2 marks, the whole conclusion

What not to write

“We do not reject H₀: the catalyst has no effect on the reaction time.”

What to write

“At the 5% level, the data do not allow us to conclude that the catalyst reduces the mean reaction time.”

120116.1H₀: μ = 120μ = 117β ≈ 0.65x̄ (s)
If the true mean is 117117 s, x‾\overline{x} stays above the bound 116.1116.1 in about 65%65\% of cases (blue area): not rejecting is then the most likely outcome.

Why: A small real gap often goes unnoticed. With n=40n=40 trials and s=15s=15 s, a catalyst that brings the mean down to 117117 s escapes the test in about 65%65\% of cases.

3. Comparing with 1.96 in a one-tailed test

2 marks: the decision and the conclusion are wrong

What not to write

“Right-tailed test at the 5%5\% level: z=1.80<1.96z=1.80<1.96, so we do not reject H0H_{0}.”

What to write

“Right-tailed: we reject if z>1.645z>1.645. Here z=1.80>1.645z=1.80>1.645, so we reject H0H_{0} at the 5%5\% level.”

01.645z = 1.80right-tailed, 5%0−1.961.96z = 1.801.96 rule (two-tailed)
The same statistic z=1.80z=1.80 falls in the rejection zone of the one-tailed test (top) but not in that of the 1.961.96 rule, which belongs to the two-tailed test (bottom).

Why: 1.961.96 leaves 2.5%2.5\% in EACH tail: it is the two-tailed value. In a one-tailed test, the 5%5\% are all in a single tail, beyond 1.6451.645.

4. Reading the p-value as the probability that the null hypothesis is true

1 interpretation mark

What not to write

“p=0.03p=0.03: there is a 3%3\% chance that the fertilizer has no effect.”

What to write

“If the fertilizer had no effect, a mean yield at least this high would appear in only 3%3\% of samples.”

Why: The pp-value is computed assuming H0H_{0} is true: it is P(data at least this extreme∣H0)P(\text{data at least this extreme}\mid H_{0}), not P(H0∣data)P(H_{0}\mid\text{data}). The two differ as in Bayes' theorem.

5. Putting the observed proportion in the standard error of a test

2 marks: here the decision flips

What not to write

“p^=84240=0.35\hat{p}=\frac{84}{240}=0.35 and z=0.35−0.300.35×0.65/240≈1.62<1.645z=\dfrac{0.35-0.30}{\sqrt{0.35\times 0.65/240}}\approx 1.62<1.645: we do not reject.”

What to write

“Under H0H_{0}, σP^=0.30×0.70240≈0.0296\sigma_{\hat{P}}=\sqrt{\frac{0.30\times 0.70}{240}}\approx 0.0296 and z≈1.69>1.645z\approx 1.69>1.645: we reject H0H_{0} at the 5%5\% level.”

Why: The test reasons under H0H_{0}, hence with p=p0p=p_{0}. It is the confidence interval, which has no assumed value, that uses p^\hat{p}.

6. Using the normal distribution with a small sample

2 marks: wrong distribution, wrong decision

What not to write

“n=5n=5, t=2.40>1.96t=2.40>1.96: we reject H0H_{0} at the 5%5\% level.”

What to write

“n=5n=5: Student with 44 degrees of freedom, critical value 2.7762.776. ∣t∣=2.40<2.776|t|=2.40<2.776, so we do not reject H0H_{0}.”

Why: With few data, ss is a poor estimate of σ\sigma: Student's t distribution has heavier tails, and its critical value is larger. Comparing with 1.961.96 takes a risk well above 5%5\%.

7. Choosing the alternative hypothesis after seeing the data

1 mark, and a real risk that is doubled

What not to write

“The question asks whether the measurement of gg is biased. Since x‾<9.81\overline{x}<9.81, I test H1:μ<9.81H_{1}:\mu<9.81.”

What to write

“The question is about a bias in either direction: H1:μ≠9.81H_{1}:\mu\neq 9.81, a two-tailed test, chosen before computing x‾\overline{x}.”

Why: Choosing the side afterwards amounts to keeping 5%5\% in the tail where the data fall AND the possibility of the other one: the true Type I error risk rises to 10%10\%.

8. Forgetting to double the p-value of a two-tailed test

1 mark, and a wrong decision at the 2% level

What not to write

“Two-tailed test, z=2.10z=2.10: p=P(Z>2.10)=0.0179p=P(Z>2.10)=0.0179.”

What to write

“Two-tailed test: p=2×P(Z>2.10)=2×0.0179=0.0358p=2\times P(Z>2.10)=2\times 0.0179=0.0358.”

Why: In a two-tailed test, a gap on the other side would have been just as surprising: both tails count. At the 2%2\% level, 0.01790.0179 would lead to rejecting, 0.03580.0358 would not.

9. Concluding without the context

1 mark, often the last one of the question

What not to write

“z = −2.20 < −1.645, so we reject H₀.”

What to write

“z = −2.20 < −1.645: we reject H₀ at the 5% level. The data allow us to conclude that the mean lead concentration of the network is below the standard.”

Why: The decision is a statistical step; the conclusion is a scientific sentence, with the parameter named and the level stated. The marker grades the two separately.

Which method to choose

Which test statistic?

Look at what is tested (a mean or a proportion), then at the sample size and what is known about the population.

0−1.961.96−2.7762.776normalStudent, 4 df
With 44 degrees of freedom, Student's t distribution is more spread out than the normal distribution: the two-tailed critical value at 5%5\% goes from 1.961.96 to 2.7762.776.
  • If a mean, n≥30n\geq 30 → z=x‾−μ0s/nz=\dfrac{\overline{x}-\mu_{0}}{s/\sqrt{n}}, normal distribution

    Example: n=40n=40, s=0.52s=0.52: standard error 0.0820.082

    the central limit theorem means the population need not be assumed normal

  • If a mean, n<30n<30, population assumed normal → tt, Student's t distribution with n−1n-1 degrees of freedom

    Example: n=12n=12: t0.05, 11=1.796t_{0.05,\,11}=1.796 instead of 1.6451.645

  • If a mean, n<30n<30, strongly skewed data or an outlier → no test of the course applies: more data are needed

    say so on the paper instead of forcing a calculation

  • If a proportion, np0≥5np_{0}\geq 5 and n(1−p0)≥5n(1-p_{0})\geq 5 → z=p^−p0p0(1−p0)/nz=\dfrac{\hat{p}-p_{0}}{\sqrt{p_{0}(1-p_{0})/n}}, normal distribution

    Example: 8484 out of 240240 against p0=0.30p_{0}=0.30: z≈1.69z\approx 1.69

The choice is made from the statement, before any calculation, and the condition that justifies it is written on the paper.

One-tailed or two-tailed: read the verb of the question

Look in the statement for what the study wants to establish, without looking at the data.

  • If “has changed”, “differs”, “is biased”, “matches the stated value” → two-tailed: H1:μ≠μ0H_{1}:\mu\neq\mu_{0}

    Example: measurement of gg compared with 9.819.81 m/s²

  • If “increases”, “exceeds”, “is higher” → right-tailed: H1:μ>μ0H_{1}:\mu>\mu_{0}

    Example: a fertilizer that increases the yield

  • If “decreases”, “reduces”, “is lower than” → left-tailed: H1:μ<μ0H_{1}:\mu<\mu_{0}

    Example: a catalyst that reduces the reaction time

  • If compliance with a maximum standard must be PROVED → H1H_{1}: below the standard, μ<μ0\mu<\mu_{0}

    Example: water declared compliant only if the proof is made

    the most serious error, declaring unsafe water safe, becomes the one the level controls

If the statement says nothing about the direction, the test is two-tailed.

How the answer is expected to be written

A marker ticks steps. Here they are in order, with the concluding sentence expected word for word.

Writing up a complete test

When to use it: The statement asks you to test, to check a claim, or to say whether the data allow a conclusion.

  1. 1 Name the parameter in words, then write H0H_{0} (the equality) and H1H_{1} (what we want to establish), and the level α\alpha.
  2. 2 Write the conditions and the distribution chosen: n≥30n\geq 30 or normal population, np0≥5np_{0}\geq 5, random sample.
  3. 3 Compute the statistic, showing the standard error used (with ss, or with p0p_{0}).
  4. 4 Give the critical value (or the pp-value) and compare.
  5. 5 Decide (reject or do not reject H0H_{0}), then conclude in context, with the level.

Concluding sentence

“t≈2.60>1.796t\approx 2.60>1.796: we reject H0H_{0} at the 5%5\% level. The data allow us to conclude that the fertilizer increases the mean yield of the variety.”

The trap: Skipping the conditions: the calculation is right, but nothing shows that it was allowed.

Marking: Typical breakdown: hypotheses, conditions and distribution, statistic, decision, conclusion in context, in roughly equal parts.

Concluding when you do not reject

When to use it: The statistic does not enter the critical region, or the p-value exceeds the level.

  1. 1 Write the comparison: “z=−1.45>−1.645z=-1.45>-1.645” or “p=0.07>0.05p=0.07>0.05”.
  2. 2 Decide: “we do not reject H0H_{0} at the 5%5\% level”.
  3. 3 Conclude on what the data do NOT allow you to establish, with the vocabulary of the statement.
  4. 4 If the question asks for it, name the limitation: small sample, low power.

Concluding sentence

“At the 5% level, the data do not allow us to conclude that the mean concentration exceeds 20 µg/L.”

The trap: Writing “we accept H₀” or “the concentration is 20 µg/L”: the test never proves the null hypothesis.

Check before you hand in

Five minutes of checking recover more marks than one more problem started in a hurry.

  • The sign of the statistic and the direction of H₁

    In a one-tailed test, you can reject only if the statistic is on the side of H1H_{1}: positive for μ>μ0\mu>\mu_{0}, negative for μ<μ0\mu<\mu_{0}.

    H1:μ>3.20H_{1}:\mu>3.20 and z=−0.8z=-0.8: rejection is impossible, whatever the level; if your calculation concludes to reject, the subtraction is the wrong way round.

  • The p-value and the critical region say the same thing

    p≤αp\leq\alpha exactly when the statistic is in the critical region. Check that the two readings agree.

    z=1.80z=1.80 in a one-tailed test and p=0.0359p=0.0359: p<0.05p<0.05 and z>1.645z>1.645, consistent. A pp-value of 0.070.07 with a rejection reveals a doubling that does not belong there, or a halving that was forgotten.

  • The order of magnitude in standard errors

    Compare ∣x‾−μ0∣|\overline{x}-\mu_{0}| with the standard error: below one standard error, no test rejects at the usual levels; beyond three, all reject at 5%5\%, except with Student and very few data.

    A gap of 0.180.18 for a standard error of 0.0820.082: a little over two standard errors, z≈2.2z\approx 2.2. A zz of 0.350.35 signals a division by ss instead of s/ns/\sqrt{n}.

  • The degrees of freedom

    Student is read with n−1n-1 degrees of freedom, and its critical value is always larger than that of the normal distribution at the same level.

    n=12n=12: row 1111 of the table, 1.796>1.6451.796>1.645. A Student critical value below 1.6451.645 in a one-tailed test at 5%5\% is a reading error.

  • The confidence interval confirms

    In a two-tailed test at level α\alpha, we reject exactly when μ0\mu_{0} falls outside the 1−α1-\alpha confidence interval (same standard error).

    Interval [9.748, 9.800][9.748,\,9.800] and μ0=9.81\mu_{0}=9.81: outside the interval, the test must reject. If it does not, one of the two calculations is wrong.

The typical problem, taken apart

A fertilizer trial on twelve plots

Without fertilizer, the mean yield of a variety is 3.203.20 t/ha. A trial on n=12n=12 plots chosen at random gives x‾=3.47\overline{x}=3.47 t/ha and s=0.36s=0.36 t/ha. The yields are assumed to be normally distributed.

Does the fertilizer increase the mean yield, at the 5%5\% level? We give t0.05, 11=1.796t_{0.05,\,11}=1.796, t0.025, 11=2.201t_{0.025,\,11}=2.201 and t0.01, 11=2.718t_{0.01,\,11}=2.718.

Step 1

μ\mu: mean yield of the variety with fertilizer. H0:μ=3.20H_{0}:\mu=3.20 and H1:μ>3.20H_{1}:\mu>3.20, at level α=0.05\alpha=0.05.

Why

The word “increase” fixes a right-tailed test, before looking at the observed mean.

Step 2

n=12<30n=12<30, σ\sigma unknown, population assumed normal: Student's t distribution with 1111 degrees of freedom.

Why

This line earns the conditions mark; without it, nothing justifies the table used.

Step 3

sn=0.3612≈0.1039\dfrac{s}{\sqrt{n}}=\dfrac{0.36}{\sqrt{12}}\approx 0.1039 and t=3.47−3.200.1039≈2.60t=\dfrac{3.47-3.20}{0.1039}\approx 2.60

Why

We measure the gap from H₀ in standard errors: 0.27 t/ha is a little more than two and a half standard errors.

Step 4

Right-tailed: we reject if t>1.796t>1.796. Here 2.60>1.7962.60>1.796: we reject H0H_{0}.

Why

The critical value comes from row 11 of the table, not 1.645: with twelve plots, the normal distribution would underestimate the risk.

Step 5

2.201<2.60<2.7182.201<2.60<2.718, so 0.01<p<0.0250.01<p<0.025.

Why

The bracket on the p-value tells the strength of the result: rejection at 5% and at 2.5%, not at 1%.

Step 6

Check: 95%95\% lower confidence bound, 3.47−1.796×0.1039≈3.28>3.203.47-1.796\times 0.1039\approx 3.28>3.20.

Why

A one-sided bound above 3.20 gives the same decision as the one-tailed test: two calculations, a single conclusion.

The conclusion, written out

“t≈2.60>1.796t\approx 2.60>1.796: we reject H0H_{0} at the 5%5\% level. The trial data allow us to conclude that the fertilizer increases the mean yield of this variety (0.01<p<0.0250.01<p<0.025).”

The classic mistake on this problem: Comparing 2.602.60 with 1.6451.645, the normal value, or reading the table at 1212 degrees of freedom instead of 1111. Here the decision does not change, but the method loses its marks; with t=1.70t=1.70, the normal distribution would have led to a wrong rejection, since 1.645<1.70<1.7961.645<1.70<1.796.

Learn by heart

  • • The hypotheses are about μ\mu or pp, never about x‾\overline{x} or p^\hat{p}; H0H_{0} carries the EQUALITY.
  • • Critical values of the normal distribution: 1.961.96 (two-tailed at 5%5\%), 1.6451.645 (one-tailed at 5%5\%), 2.5762.576 and 2.3262.326 at 1%1\%.
  • • Mean with n<30n<30: Student with n−1n-1 degrees of freedom, population assumed normal.
  • • Test on a proportion: p0p_{0} in the standard error AND in the conditions np0≥5np_{0}\geq 5, n(1−p0)≥5n(1-p_{0})\geq 5.
  • • pp-value: the probability, IF H0H_{0} is true, of a result at least as extreme; reject if p≤αp\leq\alpha.
  • • α=P(rejecting H0∣H0 true)\alpha=P(\text{rejecting }H_{0}\mid H_{0}\text{ true}); β=P(not rejecting∣H0 false)\beta=P(\text{not rejecting}\mid H_{0}\text{ false}); power =1−β=1-\beta.
  • • Not rejecting H₀ NEVER proves it: the conclusion is that the data do not allow a conclusion.

Frequently asked questions

How do you know whether a hypothesis test is one-tailed or two-tailed?

You decide from the question asked, before looking at the data. If the study wants to establish a specific increase or decrease, with words such as increases, exceeds, reduces or is lower, the test is one-tailed on the side of that word. If it only asks whether the value has changed or differs, in either direction, it is two-tailed. With no indication of direction, take the two-tailed test.

When should you use Student's t distribution rather than the normal distribution in a test on a mean?

When the sample has fewer than 30 observations and the population standard deviation is unknown, you use Student's t distribution with n minus 1 degrees of freedom, provided the population is assumed normal. With 30 observations or more, the central limit theorem allows the normal distribution, even if the data are not normal. Student's critical value is always larger than that of the normal distribution.

What does a p-value of 0.03 mean?

It means that, if the null hypothesis were true, a result at least as far from that hypothesis as the one observed would occur in only 3 percent of samples. So the null hypothesis is rejected at the 5 percent level, but not at the 1 percent level. It is not the probability that the null hypothesis is true.

Can you accept the null hypothesis when the test does not reject it?

No. Not rejecting the null hypothesis only means that the data are not far enough from it to rule out chance. A real but small effect, or a sample that is too small, often gives this result. So you write that the data do not allow you to conclude, at the chosen level, what the study wanted to establish.

Mark this sheet as read or add it to your favourites: a free account, no password, keeps your read sheets and favourites from one visit to the next, tells you which chapter to tackle next and lets you ask for the method or exercise you are missing. Create your space, an email is enough.

Practise it

Corrected exercises: Hypothesis tests, 201-SN1-RE

A method is proved on a paper, not on a sheet. The set for the same chapter takes each of these traps into a problem, with the solution written out step by step.

  • 11 corrected exercises
  • 110 points
  • 180 minutes
Do the exercises
Something missing from this sheet? A method, a trap, an exercise to practise it, a mistake you spotted: tell me, I read every request.

Sending goes through your free space: that is how I can answer you and tell you when the exercise is added. An email is enough, no password. Create your space. Rather not create an account? Write to me directly.

What is it about?
I read every request myself, nobody else sees it. No need to write your name or your school.
When your request is dealt with, you hear about it by email and in your space.

By sending, you agree that I read your request to answer it. It is kept with your account, erased with it, and never published: privacy policy.

This chapter comes back in

The chapters that build on it, later in the year or in the years that follow.

© Ahmed Squalli Houssaini. Revision sheet published at www.letuteurscientifique.ca/en/fiches/cegep-sn1-hypothesis-tests. Free for personal and classroom use; republishing it elsewhere requires written permission (legal notice).

Looking for a 201-SN1-RE tutor in Montreal?

Get in touch for a first session. In a hypothesis test, the method is worth more than the calculation: the hypotheses, the right distribution, the decision and a conclusion written in context earn most of the marks.

Site by Studio Squalli