Advanced Statistics 201-337 • CEGEP, Montreal

Revision sheet: probability and distributions (201-337)

Q: What is the difference between mutually exclusive and independent events?

Mutually exclusive means the two cannot happen at the same time, so their intersection is empty. Independent means that knowing one has happened changes nothing about the probability of the other. Two mutually exclusive events with nonzero probabilities are, on the contrary, strongly dependent, since one happening rules out the other.

Q: Why can a 99% reliable test be wrong two times out of three?

Because the disease is rare. Out of ten thousand people of whom one percent are ill, the test picks up ninety-nine truly ill people, but it also declares positive two percent of the nine thousand nine hundred healthy people, that is, one hundred and ninety-eight false positives. Of all the positives, only one third are actually ill: a positive result is wrong two times out of three.

Q: When does the binomial distribution no longer apply?

As soon as the trials are no longer independent or the probability changes along the way, which is the case when drawing without replacement from a small population. You then need a conditional tree or the hypergeometric distribution. The binomial is tolerated when the sample is no more than about five percent of the population.

Q: What is the continuity correction for?

For moving from a variable that jumps from one whole number to the next to a continuous curve. Each whole value actually occupies a bar of width one, centred on it. To include the value ten in the calculation, you therefore integrate up to ten and a half, and to exclude it, up to nine and a half. Without that half, the probability you get is systematically off.

Q: How do I choose between the binomial, Poisson and normal distributions?

First describe the experiment. A fixed number of trials with two outcomes gives the binomial. A count of rare events over a time span or an area gives Poisson, with a parameter proportional to the observed window. The normal distribution is for continuous variables, and approximates the binomial when the number of trials times each probability exceeds five.

Probability is the chapter where the right formula applied to the wrong experiment gives a perfectly clean and perfectly wrong result. A draw without replacement treated as a binomial sets off no alarm, neither on the exam paper nor on the calculator.

This sheet gathers the decisions that come before the calculation: read whether order matters, read whether there is replacement, read which probability is conditioned on what, and check the conditions before any approximation.

Mark this sheet as read or add it to your favourites: a free account, no password, keeps your read sheets and favourites from one visit to the next, tells you which chapter to tackle next and lets you ask for the method or exercise you are missing. Create your space, an email is enough.

The thread of the chapter

The formula is never chosen from the wording of the question but from the EXPERIMENT it describes: with or without replacement, whether order matters or not, whether the probability is conditional or not.

This chapter is part of 201-337, Advanced Statistics

What is on this sheet

1. The essentials
2. The mistakes that cost marks
3. Which method to choose
4. How the answer is expected to be written
5. Check before you hand in
6. The typical problem, taken apart
7. Learn by heart
8. Frequently asked questions

Topics covered

Conditional probability
Bayes' theorem
Independent events
Mutually exclusive events
Permutations and combinations
Binomial distribution
Poisson distribution
Normal distribution
Continuity correction
Expected value
Variance of a random variable
Hypergeometric distribution

The essentials

Mutually exclusive and independent are unrelated ideas

• MUTUALLY EXCLUSIVE: $P(A\cap B)=0$ , the two cannot happen together.
• INDEPENDENT: $P(A\cap B)=P(A)\times P(B)$ , knowing that one has happened changes nothing about the other.
• Two mutually exclusive events with nonzero probabilities are NEVER independent: if $A$ happens, $B$ becomes impossible.
• Union: $P(A\cup B)=P(A)+P(B)-P(A\cap B)$ , the last term vanishing only when the events are mutually exclusive.

On the left, no overlap: the events are mutually exclusive, hence dependent. On the right, a nonempty intersection: independence becomes possible and is tested with the product.

The two ideas are easy to mix up and describe opposite situations. A question that gives $P(A)$ , $P(B)$ and $P(A\cup B)$ almost always expects you to test independence with the product.

Conditioning, and never flipping the bar

• $P(A\mid B)=\dfrac{P(A\cap B)}{P(B)}$ : what follows the bar becomes the new sample space.
• Bayes: $P(A\mid B)=\dfrac{P(B\mid A)\,P(A)}{P(B)}$ , where $P(B)$ is computed with the law of total probability.
• $P(A\mid B)$ and $P(B\mid A)$ are two different numbers, sometimes very different: a test that is $99\%$ reliable can have only one chance in three of being right when it is positive.

The first branch carries the cause, the second the test. The two “positive” paths weigh

0.0099

and

0.0198

: only one positive in three is ill.

A probability tree makes Bayes almost automatic: the first branch carries the cause, the second the observation, and the conditional probability asked for climbs the tree backwards.

Which distribution, and under what conditions

• BINOMIAL: $n$ INDEPENDENT trials, two outcomes, constant probability $p$ . $P(X=k)=\binom{n}{k}p^{k}(1-p)^{n-k}$ , $\mu=np$ , $\sigma=\sqrt{np(1-p)}$ .
• POISSON: a count of rare events over a time span or an area. $P(X=k)=\dfrac{e^{-\lambda}\lambda^{k}}{k!}$ , with $E(X)=V(X)=\lambda$ , a property that serves as a diagnostic test.
• NORMAL: convert with $z=\dfrac{x-\mu}{\sigma}$ , then read the table. Benchmarks: $68\%$ within $\pm 1\sigma$ , $95\%$ within $\pm 2\sigma$ , $99.7\%$ within $\pm 3\sigma$ .
• Normal approximation to the binomial, allowed when $np\geq 5$ AND $n(1-p)\geq 5$ , with a continuity correction of $0.5$ .

The benchmarks of the normal distribution are read in standard deviations:

68\%

of the population between

-1

and

+1

95\%

between

-2

and

+2

The Poisson parameter is proportional to the length of the observation window: $\lambda=4$ per hour becomes $\lambda=2$ over half an hour. It is the adjustment students forget most often.

Expected value and variance of a discrete variable

• $E(X)=\sum x_{i}P(x_{i})$ : a LONG-RUN average, often a value that $X$ never takes.
• $V(X)=E(X^{2})-[E(X)]^{2}$ , and the standard deviation is its square root.
• A game is fair when $E(X)=0$ for the player; every casino lives off a negative expected value for its customers.

Gains of

-2

0

and

5

with probabilities

0.5

0.3

and

0.2

: the distribution balances at

E(X)=0

, the game is fair.

Writing the unit of $E(X)$ prevents misinterpretation: “ $-0.25$ dollar per game” makes sense, “ $-0.25$ ” does not.

The mistakes that cost marks

These are the errors I correct most often in session. Each one costs marks on a paper, even when the reasoning behind it is right.

1. Treating two mutually exclusive events as independent

2 marks, and the independence question is often worth twice that

What not to write

“ $A$ and $B$ cannot happen together, so they have no influence on each other: $P(A\cap B)=P(A)P(B)$ .”

What to write

“They are mutually exclusive, so $P(A\cap B)=0$ , which differs from $P(A)P(B)$ as soon as both probabilities are nonzero: they are dependent.”

Why: Knowing that $A$ has happened makes $B$ impossible: that is the strongest possible influence, the exact opposite of independence.

2. Turning a conditional probability around

3 marks, it is the reasoning question of the exam

What not to write

“The test detects $99\%$ of ill people, so if I test positive I have a $99\%$ chance of being ill.”

What to write

“ $P(\text{positive}\mid\text{ill})=0.99$ , but $P(\text{ill}\mid\text{positive})=\dfrac{99}{99+198}=\dfrac{1}{3}$ .”

Out of

10{,}000

people of whom

1\%

are ill, a test with

99\%

sensitivity and

98\%

specificity produces

99

true positives against

198

false positives.

Why: When the disease is rare, the false positives coming from the huge healthy population outnumber the true positives. Only the total probability in the denominator brings this out.

3. Applying the binomial to a draw without replacement

2 marks, and the result still looks plausible, so it cannot be caught

What not to write

“We draw $5$ cards from a deck of $52$ , so $X$ follows a binomial distribution with parameters $n=5$ and $p=\dfrac{4}{52}$ .”

What to write

“Without replacement, $p$ changes at every draw: you need a conditional tree or the hypergeometric distribution $\dfrac{\binom{a}{k}\binom{b}{n-k}}{\binom{a+b}{n}}$ .”

Without replacement, the probability of an ace on the second draw is

\dfrac{3}{51}

\dfrac{4}{51}

depending on the first:

p

is not constant, the binomial does not apply.

Why: The binomial requires independent trials with a constant probability. It remains acceptable when the sample is no more than about $5\%$ of the population, which is not the case here.

4. Forgetting the continuity correction

1.5 marks, and the gap grows when

n

is small

What not to write

“ $P(X\leq 10)$ with $\mu=10$ : I take $z=\dfrac{10-10}{\sigma}=0$ , so $0.5$ .”

What to write

“The discrete value $10$ occupies the interval $[9.5, 10.5]$ : I take $z=\dfrac{10.5-10}{\sigma}$ .”

The bar for

10

runs from

9.5

10.5

: to include the whole value

10

under the curve, the bound of integration is

10.5

Why: A histogram is being replaced by a curve: each bar has a width of $1$ , and the area to integrate goes up to the edge of the bar, not to its centre.

5. Taking the expected value for a prediction

1.5 marks on the interpretation question

What not to write

“ $E(X)=2.4$ children per family, so a family has two or three children.”

What to write

“ $E(X)=2.4$ is a long-run average over a large number of families; no family has $2.4$ children.”

The distribution of the number of children balances at

E(X)=2.4

: it is a centre of gravity, and no family has

2.4

children.

Why: The expected value is a centre of gravity, not an observable value. That is exactly why a fair game is defined by $E(X)=0$ and not by “you win as much as you lose”.

6. Confusing permutations and combinations

2 marks, and the error factor is

k!

What not to write

“We choose $3$ delegates out of $10$ : $\dfrac{10!}{7!}=720$ possibilities.”

What to write

“Order does not matter, so $\binom{10}{3}=120$ ; the previous result would be divided by $3!=6$ .”

The

3!=6

orders of

A

B

C

make six podiums but only one committee:

\binom{10}{3}=\dfrac{720}{6}=120

Why: The test fits in one sentence: if permuting the chosen objects gives a different situation, order matters. Three delegates remain the same three delegates whatever order they were drawn in.

7. Keeping the Poisson parameter of another time span

2 marks, and the whole calculation that follows is wrong

What not to write

“ $6$ customers arrive per hour, so over $20$ minutes $\lambda=6$ .”

What to write

“ $\lambda$ is proportional to the length of the window: over $20$ minutes, $\lambda=6\times\dfrac{20}{60}=2$ .”

Six customers per hour, observed over

20

minutes:

\lambda=2

(green). Keeping

\lambda=6

(red) describes a whole hour and shifts the entire distribution.

Why: The Poisson parameter IS the average number of events over the observed window. Changing the window changes the parameter, just as changing units changes a flow rate.

8. Adding probabilities that overlap

1.5 marks, and sometimes a probability greater than

1

What not to write

“ $P(\text{even})+P(\text{greater than }4)=\dfrac{3}{6}+\dfrac{2}{6}=\dfrac{5}{6}$ .”

What to write

“The two events share the $6$ : $P(A\cup B)=\dfrac{3}{6}+\dfrac{2}{6}-\dfrac{1}{6}=\dfrac{4}{6}$ .”

“Even” and “greater than

4

” share the

6

(red): adding it twice gives

\dfrac{5}{6}

instead of

\dfrac{4}{6}

Why: The simple addition rule is reserved for mutually exclusive events. A total greater than $1$ is the most visible sign of this error.

9. Reading the normal table on the wrong side

2 marks, and a reversed conclusion on the quality control

What not to write

“The table gives $0.9332$ for $z=1.5$ , so $P(Z>1.5)=0.9332$ .”

What to write

“The table gives the area to the LEFT: $P(Z>1.5)=1-0.9332=0.0668$ .”

The table gives the area to the LEFT of

1.5

0.9332

(blue); the right tail is its complement,

0.0668

(green).

Why: A common-sense check is enough: a value above the mean cannot concern $93\%$ of the population. Every tail probability is small.

Which method to choose

The four counting cases

Two questions, in this order: does order matter? can an object be repeated?

If order matters and repetition is allowed → $n^{k}$

Example: a $4$ -digit code: $10^{4}=10{,}000$
If order matters, no repetition → permutation $\dfrac{n!}{(n-k)!}$

Example: a podium of $3$ out of $10$ : $720$
If order does not matter, no repetition → combination $\binom{n}{k}=\dfrac{n!}{k!(n-k)!}$

Example: $3$ delegates out of $10$ : $120$
If objects some of which are identical are permuted → $\dfrac{n!}{n_{1}!\,n_{2}!\cdots}$

Example: anagrams of MISSISSIPPI

A special case to know: circular permutations number $(n-1)!$ , because rotating the whole table does not change where the guests sit relative to one another.

Which distribution for which experiment

Describe the experiment in one sentence before naming a distribution.

The distributions the tree decides between: bars bounded by

n

(binomial), unbounded bars of a rare count (Poisson), the curve of a continuous quantity (normal).

If $n$ fixed trials, two outcomes, with replacement or a very large population → binomial distribution

Example: $200$ parts inspected, $3\%$ defective
If drawing without replacement from a small population → hypergeometric distribution, or a conditional tree

Example: $5$ cards drawn from a deck of $52$
If a count of rare events over a time span, a length or an area → Poisson distribution, with $\lambda$ adjusted to the observed window

Example: calls received in $20$ minutes
If a continuous variable, or a binomial with $np\geq 5$ and $n(1-p)\geq 5$ → normal distribution, with the continuity correction in the second case

Example: $P(X\geq 10)$ becomes $P(Y\geq 9.5)$

The Poisson distribution also approximates the binomial when $n$ is large and $p$ small, with $\lambda=np$ . The two approximations then give similar results, which provides a free check.

How the answer is expected to be written

A marker ticks steps. Here they are in order, with the concluding sentence expected word for word.

Writing up a Bayes calculation

When to use it: The question gives the reliability of a test, a detection rate or a proportion of defects, and asks for the reverse probability.

1 Name the events with a letter and a complete sentence, in words.
2 Write the data as conditional probabilities, spotting what is given: $P(\text{positive}\mid\text{ill})$ , not the reverse.
3 Compute the total probability in the denominator along the two paths of the tree.
4 Apply Bayes, then comment on the result in light of how rare the cause is.

Concluding sentence

“ $P(M\mid +)=\dfrac{0.99\times 0.01}{0.99\times 0.01+0.02\times 0.99}=\dfrac{1}{3}$ : despite a reliable test, only one positive in three corresponds to an ill person.”

The trap: Writing the denominator with only the branch of ill people, which mechanically gives back $0.99$ .

Marking: 1 mark for the notation, 1 mark for the total probability, 1 mark for the quotient, 1 mark for the comment.

Justifying a normal approximation

When to use it: The question involves a binomial with a large $n$ and asks for a cumulative probability.

1 Write $n$ and $p$ , then check $np\geq 5$ and $n(1-p)\geq 5$ explicitly.
2 Compute $\mu=np$ and $\sigma=\sqrt{np(1-p)}$ .
3 Apply the continuity correction by moving the bound by $0.5$ in the direction that INCLUDES the whole value asked for.
4 Convert to a z-score $z$ , read the table, and state on which side the area is taken.

Concluding sentence

“ $np=6\geq 5$ and $n(1-p)=194\geq 5$ , so the normal approximation is allowed with $\mu=6$ and $\sigma\approx 2.41$ .”

The trap: Moving the bound the wrong way: for $P(X\geq 10)$ take $9.5$ , for $P(X\leq 10)$ take $10.5$ .

Marking: 1 mark for checking both conditions, 1 mark for the parameters, 1 mark for the correction, 1 mark for reading the table.

Check before you hand in

Five minutes of checking recover more marks than one more problem started in a hurry.

Every probability stays between 0 and 1

Reread every intermediate result. A negative number or one greater than $1$ signals a forgotten intersection or a complement taken wrongly.

$\dfrac{3}{6}+\dfrac{2}{6}=\dfrac{5}{6}$ for two events that share the $6$ : the total swells without exceeding $1$ , but the method is already wrong.
The expected value falls between the minimum and the maximum

Check that $E(X)$ lies between the smallest and the largest possible value, and that the probabilities add up to $1$ .

An expected value of $5.2$ for a variable that goes from $0$ to $4$ betrays a probability copied wrongly.
Poisson: mean equals variance

If the question gives a mean and a variance that are very different, the Poisson distribution does not fit, and you must say so.

Mean $6$ and variance $18$ : the Poisson model must be rejected, the spread is too large.
A tail probability is small

After reading the table, ask whether the event described is rare. A tail probability greater than $0.5$ signals a reading on the wrong side.

$P(Z>1.5)=0.9332$ is impossible: the right value is $0.0668$ .

The typical problem, taken apart

Quality control: at least ten defective parts?

A production line makes parts of which $3\%$ are defective. A random sample of $200$ is taken from continuous production.

What is the probability of finding at least $10$ defective parts in it?

The distribution of the number of defective parts out of

200

, centred on

np=6

: the question is about the right tail, from

10

on.

Step 1

The sample is small compared with the production, so $X$ follows a binomial distribution with parameters $n=200$ and $p=0.03$ .

Why

Justifying the model is worth a mark on its own. It is also what allows us to ignore the lack of replacement, since production is continuous.

Step 2

$np=6\geq 5$ and $n(1-p)=194\geq 5$ : the normal approximation is allowed.

Why

Both conditions are checked BEFORE switching distributions. An approximation used without checking is marked wrong even if the result is right.

Step 3

$\mu=6$ and $\sigma=\sqrt{200\times 0.03\times 0.97}\approx 2.41$ .

Why

The standard deviation is computed from the original binomial, never as the square root of the mean: that mix-up with the Poisson distribution is the most frequent neighbouring error.

Step 4

Continuity correction: $P(X\geq 10)\approx P(Y\geq 9.5)$ , so $z=\dfrac{9.5-6}{2.41}\approx 1.45$ .

Why

We want to INCLUDE the value $10$ , so the bound goes down to $9.5$ . Going up to $10.5$ would exclude precisely the value the question asks us to count.

Step 5

Table: $P(Z<1.45)=0.9265$ , so the probability we want is $1-0.9265\approx 0.074$ .

Why

The table gives the area to the left: subtract it from $1$ for a right tail. The result, about $7\%$ , is indeed a small probability.

Step 6

Check with Poisson, $\lambda=np=6$ : we get about $0.084$ .

Why

Two independent approximations that agree validate the calculation better than a reread. The gap comes from the skewness of the binomial, which the normal curve smooths out.

The conclusion, written out

“The probability of finding at least $10$ defective parts out of $200$ is about $7\%$ , which remains plausible without the line being out of adjustment.”

The classic mistake on this problem: Taking $z=\dfrac{10-6}{2.41}$ without the correction, or worse, $\sigma=\sqrt{6}$ by confusing the binomial with a Poisson distribution.

Learn by heart

• Mutually exclusive: $P(A\cap B)=0$ . Independent: $P(A\cap B)=P(A)P(B)$ . Never both at once if the probabilities are nonzero.
• $P(A\cup B)=P(A)+P(B)-P(A\cap B)$ : the last term may be dropped only if the events are mutually exclusive.
• $P(A\mid B)$ is not $P(B\mid A)$ : Bayes goes from one to the other, and the denominator is a TOTAL probability.
• Order and repetition $n^{k}$ ; order only $\dfrac{n!}{(n-k)!}$ ; neither $\binom{n}{k}$ ; identical objects $\dfrac{n!}{n_{1}!n_{2}!}$ .
• Binomial: $\mu=np$ , $\sigma=\sqrt{np(1-p)}$ . Poisson: $E(X)=V(X)=\lambda$ , adjusted to the observed time span.
• Normal approximation if $np\geq 5$ AND $n(1-p)\geq 5$ , always with the correction of $0.5$ .
• The normal table gives the area to the LEFT; a right tail is taken through the complement.

Frequently asked questions

What is the difference between mutually exclusive and independent events?

Mutually exclusive means the two cannot happen at the same time, so their intersection is empty. Independent means that knowing one has happened changes nothing about the probability of the other. Two mutually exclusive events with nonzero probabilities are, on the contrary, strongly dependent, since one happening rules out the other.

Why can a 99% reliable test be wrong two times out of three?

Because the disease is rare. Out of ten thousand people of whom one percent are ill, the test picks up ninety-nine truly ill people, but it also declares positive two percent of the nine thousand nine hundred healthy people, that is, one hundred and ninety-eight false positives. Of all the positives, only one third are actually ill: a positive result is wrong two times out of three.

When does the binomial distribution no longer apply?

As soon as the trials are no longer independent or the probability changes along the way, which is the case when drawing without replacement from a small population. You then need a conditional tree or the hypergeometric distribution. The binomial is tolerated when the sample is no more than about five percent of the population.

What is the continuity correction for?

For moving from a variable that jumps from one whole number to the next to a continuous curve. Each whole value actually occupies a bar of width one, centred on it. To include the value ten in the calculation, you therefore integrate up to ten and a half, and to exclude it, up to nine and a half. Without that half, the probability you get is systematically off.

How do I choose between the binomial, Poisson and normal distributions?

First describe the experiment. A fixed number of trials with two outcomes gives the binomial. A count of rare events over a time span or an area gives Poisson, with a parameter proportional to the observed window. The normal distribution is for continuous variables, and approximates the binomial when the number of trials times each probability exceeds five.

Practise it

Corrected exercises: Probability and distributions, 201-337

A method is proved on a paper, not on a sheet. The set for the same chapter takes each of these traps into a problem, with the solution written out step by step.

11 corrected exercises
110 points
150 minutes

Do the exercises

Something missing from this sheet? A method, a trap, an exercise to practise it, a mistake you spotted: tell me, I read every request.

Sending goes through your free space: that is how I can answer you and tell you when the exercise is added. An email is enough, no password. Create your space. Rather not create an account? Write to me directly.

Previous sheet Descriptive statistics Next sheet Continuous random variables and density

© Ahmed Squalli Houssaini. Revision sheet published at www.letuteurscientifique.ca/en/fiches/cegep-337-probability-distributions. Free for personal and classroom use; republishing it elsewhere requires written permission (legal notice).

Looking for a 201-337 tutor in Montreal?

Get in touch for a first session. Conditional probability is where almost everyone gets lost: a well-built tree settles the question in two minutes.

Book a session WhatsApp

Site by Studio Squalli

Revision sheet: probability and distributions (201-337)

The essentials

Mutually exclusive and independent are unrelated ideas

Conditioning, and never flipping the bar

Which distribution, and under what conditions

Expected value and variance of a discrete variable

The mistakes that cost marks

1. Treating two mutually exclusive events as independent

2. Turning a conditional probability around

3. Applying the binomial to a draw without replacement

4. Forgetting the continuity correction

5. Taking the expected value for a prediction

6. Confusing permutations and combinations

7. Keeping the Poisson parameter of another time span

8. Adding probabilities that overlap

9. Reading the normal table on the wrong side

Which method to choose

The four counting cases

Which distribution for which experiment

How the answer is expected to be written

Writing up a Bayes calculation

Justifying a normal approximation

Check before you hand in

The typical problem, taken apart

Quality control: at least ten defective parts?

Learn by heart

Frequently asked questions

What is the difference between mutually exclusive and independent events?

Why can a 99% reliable test be wrong two times out of three?

When does the binomial distribution no longer apply?

What is the continuity correction for?

How do I choose between the binomial, Poisson and normal distributions?

Corrected exercises: Probability and distributions, 201-337

See also

Looking for a 201-337 tutor in Montreal?