Conditional probability 201-SN1-RE: revision sheet

The thread of the chapter

After the bar, the sample space changes: $P(A\mid B)$ counts $A$ among only the cases where $B$ occurs. Mutually exclusive, the condition wipes out $A$ ; independent, it changes nothing; turning the bar around changes the denominator, and therefore the number.

Topics covered

Random experiment and sample space
Axioms of probability
Mutually exclusive events
Independent events
Conditional probability
Multiplication rule
Probability tree
Contingency table
Law of total probability
Bayes' theorem
Monty Hall problem
Reliability of systems

The essentials

After the bar, the sample space changes

• Definition, for $P(B)>0$ : $P(A\mid B)=\dfrac{P(A\cap B)}{P(B)}$ . We count $A$ in the new sample space $B$ .
• The same intersection serves both conditional probabilities: $P(A\mid B)$ divides by $P(B)$ , $P(B\mid A)$ divides by $P(A)$ . They are equal only if $P(A)=P(B)$ .
• Within a fixed sample space, the complement rule holds: $P(A\mid B)+P(\overline{A}\mid B)=1$ . Between two sample spaces, nothing holds: $P(A\mid B)+P(A\mid\overline{B})$ can be anything between $0$ and $2$ .
• Information REMOVES cases: knowing that a child of two carriers is healthy removes $aa$ , and the child's probability of being a carrier goes from $\frac{1}{2}$ to $\frac{2}{3}$ .

The same blue area

A\cap B=0.18

is read in column

B

(

0.18

out of

0.2

, that is

0.9

) or in all of the blue (

0.18

out of

0.26

, that is

0.69

): two sample spaces, two numbers.

Before any calculation, write in words “among the ...”: the group you name is the denominator. This one line alone prevents half the errors of the chapter.

Mutually exclusive and independent: two unrelated definitions

• Mutually exclusive: $A\cap B=\varnothing$ . It is a property of the sets, and it shows on a diagram.
• Independent: $P(A\cap B)=P(A)P(B)$ , that is $P(A\mid B)=P(A)$ . It is a property of the NUMBERS, and it shows only through calculation.
• If $P(A)>0$ and $P(B)>0$ , mutually exclusive implies dependent: $P(A\cap B)=0<P(A)P(B)$ .
• If $A$ and $B$ are independent, so are $A$ and $\overline{B}$ , and $\overline{A}$ and $\overline{B}$ : this is what makes the “at least one” calculation through the complement possible.

Never deduce independence from the absence of a causal link: check it with the equality of the numbers, or state it as an assumption of the model, and say so.

The three formulas that do everything

• Multiplication: $P(A\cap B)=P(B)\,P(A\mid B)$ ; on a tree, multiply along a path.
• Law of total probability, for a partition $B_{1}$ , ..., $B_{n}$ : $P(A)=\sum_{j}P(B_{j})\,P(A\mid B_{j})$ ; on a tree, add the paths that lead to $A$ .
• Bayes: $P(B_{i}\mid A)=\dfrac{P(B_{i})\,P(A\mid B_{i})}{\sum_{j}P(B_{j})\,P(A\mid B_{j})}$ : one path divided by all the paths that lead to $A$ .

Bayes' theorem is not a formula of its own: it is the definition of conditional probability, with the multiplication rule in the numerator and the law of total probability in the denominator.

The mistakes that cost marks

These are the errors I correct most often in session. Each one costs marks on a paper, even when the reasoning behind it is right.

1. Turning the bar around without changing the denominator

the whole question, and often the next one

What not to write

“The test detects $95\%$ of contaminated samples, so a positive sample is $95\%$ likely to be contaminated.”

What to write

“ $P(T\mid C)=0.95$ is the sensitivity; the probability of being contaminated given a positive test is $P(C\mid T)=\dfrac{P(C)P(T\mid C)}{P(T)}$ , which depends on the proportion of contaminated samples.”

Same test, sensitivity and specificity

0.95

: a positive is worth

0.16

when

1\%

of the samples are contaminated and

0.83

when

20\%

are.

Why: With a specificity of $0.95$ and $1\%$ of contaminated samples, $P(C\mid T)\approx 0.16$ only: the false positives, drawn from the $99\%$ of uncontaminated samples, outnumber the true ones.

2. Confusing mutually exclusive with independent

1 to 2 marks, and every later calculation done with a product

What not to write

“A muon and an electron have nothing to do with each other: the events $U$ and $E$ are independent.”

What to write

“ $U\cap E=\varnothing$ : $U$ and $E$ are mutually exclusive. Since $P(U\cap E)=0\neq P(U)P(E)=0.15$ , they are dependent.”

On the left,

A

and

B

do not touch:

P(A\cap B)=0

. On the right, they overlap and the intersection is exactly

0.4\times 0.5

: that is what independence looks like.

Why: Mutually exclusive, knowing one rules out the other: that is the most information possible, the opposite of independence.

3. Adding two probabilities that overlap

1 mark per union computed wrongly

What not to write

“ $P(M)+P(N)=0.72+0.32=1.04>1$ : the model is wrong.”

What to write

“ $P(M\cup N)=P(M)+P(N)-P(M\cap N)=0.72+0.32-0.12=0.92$ .”

Why: The plain sum is the axiom for DISJOINT events; in general, the intersection is counted twice.

4. Completing to 1 across two different sample spaces

1 mark, plus the wrong tree that follows

What not to write

“ $P(R\mid A)=0.75$ , so $P(R\mid\overline{A})=0.25$ .”

What to write

“ $P(\overline{R}\mid A)=0.25$ ; $P(R\mid\overline{A})$ has to be computed: $\dfrac{P(R)-P(R\cap A)}{1-P(A)}=\dfrac{0.06}{0.68}\approx 0.088$ .”

Why: On a tree, it is the branches coming out of the SAME node that add up to 1, never two branches from different nodes.

5. Forgetting that information removes cases

the whole genetics question

What not to write

“The healthy sister of an affected child, whose two parents are carriers, is a carrier with probability $\frac{1}{2}$ .”

What to write

“Given that she is healthy, $P(Aa\mid\text{healthy})=\dfrac{1/2}{3/4}=\dfrac{2}{3}$ .”

Why: $\frac{1}{2}$ is the probability BEFORE the information; the case $aa$ is ruled out, and the sample space now weighs only $\frac{3}{4}$ .

6. Taking independence given a state for plain independence

2 marks

What not to write

“Each child of the couple is affected with probability $\frac{1}{180}$ ; for two children, $1-\left(1-\frac{1}{180}\right)^{2}$ .”

What to write

“Given that both parents are carriers, the children are independent: $P=\frac{2}{3}\times\frac{1}{30}\times\left(1-\left(\frac{3}{4}\right)^{2}\right)=\frac{7}{720}$ .”

Why: Two children of the same parents share the same hidden cause, the parental genotypes: their status is linked as long as that cause is unknown.

7. Adding reliabilities in parallel

1 to 2 marks

What not to write

“Two power supplies with reliabilities $0.90$ and $0.85$ in parallel: $0.90+0.85$ , or else $0.90\times 0.85$ .”

What to write

“The block fails if both fail: $1-0.10\times 0.15=0.985$ , assuming the failures are independent.”

Why: The sum exceeds $1$ ; the product is the formula for a SERIES connection. In parallel, it is the probabilities of failure that are multiplied.

8. Counting the remaining outcomes instead of conditioning

the whole question

What not to write

“After an empty door is opened, two doors are left: a one-in-two chance for each.”

What to write

“If the person opening knows and never reveals the prize, $P(\text{prize behind the other door})=\frac{2}{3}$ ; if the door is opened at random, $\frac{1}{2}$ .”

Why: What matters is the probability of the observation in each case, not the number of cases left: the same move does not teach the same thing under a different protocol.

Which method to choose

Which tool for which form of question

Look at the form of the data and the verb of the question, before any calculation.

If COUNTS cross-classified by two criteria → contingency table: divide the cell by the total of the row or column of the condition

Example: $36$ detections out of $80$ summer samples: $P(D\mid Su)=0.45$
If percentages “among the ...” given step by step → probability tree: multiply along a path, add the relevant paths

Example: $0.6\times 0.05+0.15\times 0.16+0.25\times 0.08=0.074$
If an EFFECT is observed and the probability of a CAUSE is asked for → Bayes' theorem: one path over the sum of the paths that lead to the effect

Example: contaminated well, probability that it is tubular: $\frac{0.030}{0.074}\approx 0.405$
If “at least one” over independent trials → go through the complement: $1-$ the product of the probabilities of failure

Example: $1-0.45^{3}\approx 0.909$

never the sum of the probabilities of success
If “are they independent?” → compare $P(A\cap B)$ with $P(A)P(B)$ , or $P(A\mid B)$ with $P(A)$

Example: $0.6\times 0.5=0.3=P(A\cap B)$ : independent
If components in series or in parallel → series: product of the reliabilities; parallel: $1-$ product of the probabilities of failure

Example: $1-0.05\times 0.20=0.99$

If no branch applies, name the events, write the data in symbols ( $P(\cdot)$ or $P(\cdot\mid\cdot)$ ), then spot what is missing: the form of the missing data points to the formula.

How the answer is expected to be written

A marker ticks steps. Here they are in order, with the concluding sentence expected word for word.

Proving Bayes' theorem

When to use it: The question asks you to prove, establish or justify Bayes' theorem for a partition.

1 State the hypotheses: $B_{1}$ , ..., $B_{n}$ form a partition of $\Omega$ , $P(B_{j})>0$ for every $j$ , and $P(A)>0$ .
2 Write the definition: $P(B_{i}\mid A)=\dfrac{P(B_{i}\cap A)}{P(A)}$ .
3 Transform the numerator with the definition of $P(A\mid B_{i})$ : $P(B_{i}\cap A)=P(A\mid B_{i})P(B_{i})$ .
4 Transform the denominator: the $A\cap B_{j}$ are disjoint with union $A$ , so by additivity $P(A)=\sum_{j}P(A\mid B_{j})P(B_{j})$ .
5 Substitute and conclude.

Concluding sentence

“For every $i$ , $P(B_{i}\mid A)=\dfrac{P(A\mid B_{i})\,P(B_{i})}{\sum_{j=1}^{n}P(A\mid B_{j})\,P(B_{j})}$ .”

The trap: Writing the denominator without saying why: it is the disjointness of the $A\cap B_{j}$ and the additivity axiom that justify it.

Marking: 1 mark for the hypotheses, 1 for the numerator, 2 for the justified denominator

Justifying independence or dependence

When to use it: “Are $A$ and $B$ independent? Justify.”

1 Compute $P(A\cap B)$ , then the product $P(A)P(B)$ , each on its own line.
2 Compare the two numbers, with the sign = or ≠.
3 Conclude in one sentence in context, without invoking causes.

Concluding sentence

“ $P(A\cap B)=0.18$ and $P(A)P(B)=0.12$ : since $0.18\neq 0.12$ , the events $A$ and $B$ are not independent.”

The trap: Answering “no, because one influences the other”: the causal argument earns no marks, only the comparison of the numbers does.

Marking: 1 mark for the two calculations, 1 for the comparison and the conclusion

Check before you hand in

Five minutes of checking recover more marks than one more problem started in a hurry.

The branches of one node add up to 1

On the tree, add the probabilities carried by the branches coming out of each node.

$0.05+0.95=1$ under $T$ ; a node that gives $0.05+0.84$ signals a copying error.
The posterior probabilities add up to 1

After Bayes, add the probabilities of the causes given the effect: the sum is 1.

$0.405+0.324+0.270\approx 1$ ; a sum of $1.2$ says that a denominator is wrong.
A total probability lies between the conditional probabilities

$P(A)$ is a weighted mean of the $P(A\mid B_{j})$ : it falls between the smallest and the largest.

$P(C)=0.074$ does lie between $0.05$ and $0.16$ ; a result of $0.29$ betrays a sum without weights.
The intersection is smaller than each event

$P(A\cap B)\le\min\left(P(A),P(B)\right)$ and $P(A\cup B)\ge\max\left(P(A),P(B)\right)$ .

With $P(A)=0.32$ , an intersection of $0.4$ is impossible: a conditional probability was probably taken for the intersection.

The typical problem, taken apart

A song identification app reports a rare species

In a forest at dawn, three bird species sing similar songs: $X$ , rare, $4\%$ of the songs; $Y$ , $36\%$ ; $Z$ , $60\%$ . An app classifies each song. It reports “ $X$ ” for $90\%$ of the songs of $X$ , $5\%$ of the songs of $Y$ and $2\%$ of the songs of $Z$ .

a)A song chosen at random is classified as “ $X$ ”. What is the probability that it really comes from $X$ ?b)A second song from the same bird is also classified as “ $X$ ”; assume that, given the species, the two classifications are independent. What does this probability become?

The setup of the question: three species, then the app's classification; it is the three paths ending in “

X

” that count.

Step 1

Let $X$ , $Y$ , $Z$ denote the species and $K$ “the app reports $X$ ”. Data: $P(X)=0.04$ , $P(K\mid X)=0.90$ , $P(K\mid Y)=0.05$ , $P(K\mid Z)=0.02$ .

Why

Writing each percentage with its bar forces you to ask “among whom?”: $90\%$ is taken among the songs of $X$ , not among the reports.

Step 2

Law of total probability: $P(K)=0.04\times 0.90+0.36\times 0.05+0.60\times 0.02=0.036+0.018+0.012=0.066$ .

Why

The denominator of Bayes is computed separately, on its own line: it is the one people forget, or reduce to the single path of $X$ .

Step 3

Bayes: $P(X\mid K)=\dfrac{0.036}{0.066}\approx 0.545$ .

Why

The bar is turned around: the question asks about the cause (the species) given the effect (the report).

Step 4

Two reports: $P(X\mid K_{1}\cap K_{2})=\dfrac{0.04\times 0.9^{2}}{0.04\times 0.9^{2}+0.36\times 0.05^{2}+0.60\times 0.02^{2}}=\dfrac{0.0324}{0.03354}\approx 0.966$ .

Why

Independence is assumed only GIVEN the species: square each likelihood, never the probability $0.545$ .

Step 5

Check: $P(Y\mid K)\approx 0.273$ and $P(Z\mid K)\approx 0.182$ , and $0.545+0.273+0.182=1$ .

Why

The probabilities of the causes given the effect form a distribution: their sum checks the whole calculation in five seconds.

The conclusion, written out

A single “ $X$ ” report gives only about an even chance that the bird really belongs to the rare species, because the two common species, by far the majority, produce almost as many false reports. Two matching reports raise this probability to about $0.97$ , in the model where the classifications are independent given the species.

The classic mistake on this problem: Answering $0.90$ to question a): that is $P(K\mid X)$ , the performance of the app on species $X$ , not $P(X\mid K)$ .

Learn by heart

• $P(A\mid B)=\dfrac{P(A\cap B)}{P(B)}$ : the condition goes in the DENOMINATOR.
• $P(A\mid B)\neq P(B\mid A)$ in general: same numerator, different sample space.
• Independent: $P(A\cap B)=P(A)P(B)$ . Mutually exclusive: $A\cap B=\varnothing$ . Both at once: only if one of them has probability zero.
• Union: $P(A\cup B)=P(A)+P(B)-P(A\cap B)$ , ALWAYS; the plain sum only if mutually exclusive.
• At least one, independent trials: $1-\prod(1-p_{i})$ .
• Bayes = one path of the tree divided by all the paths that lead to the observation.
• Series: $\prod p_{i}$ ; parallel: $1-\prod(1-p_{i})$ , independent failures.

Frequently asked questions

What is the difference between mutually exclusive and independent events?

Two mutually exclusive events cannot happen together: their intersection is empty. Two independent events are such that knowing one does not change the probability of the other: the probability of the intersection is the product of the two probabilities. When both probabilities are nonzero, mutually exclusive events are never independent, because knowing one rules out the other.

How do you know when to use Bayes' theorem?

Use Bayes' theorem when the question gives the probability of an effect given each cause, for example the probability of a positive test given that a sample is contaminated, and asks for the reverse: the probability of a cause given the observed effect. Compute one path of the tree and divide it by the sum of all the paths that lead to the observed effect.

Why is P(A given B) not equal to P(B given A)?

The two probabilities have the same numerator, the probability that A and B happen together, but not the same denominator. Given B, you count among the cases where B occurs; given A, among the cases where A occurs. The two numbers are equal only if A and B have the same probability. Mixing them up is the most frequent error of the chapter.

How do you prove Bayes' theorem in CEGEP?

Start from the definition of conditional probability. The numerator, the probability of the cause and the effect together, is written as the probability of the cause times the probability of the effect given the cause. The denominator, the probability of the effect, is split over the partition of causes: the pieces are disjoint, and the additivity axiom gives the law of total probability. Substitute and conclude.

Why should you switch doors in the Monty Hall problem?

Because the host knows where the prize is and always opens a losing door. The host's choice depends on where the prize is: if it is behind the door you did not choose, the host is forced to open the other one. The calculation with Bayes' theorem gives two chances in three by switching. If the door were opened at random and turned out empty, the answer would be one chance in two.

Mark this sheet as read or add it to your favourites: a free account, no password, keeps your read sheets and favourites from one visit to the next, tells you which chapter to tackle next and lets you ask for the method or exercise you are missing. Create your space, an email is enough.

Practise it

Corrected exercises: Probability and conditional probability, 201-SN1-RE

A method is proved on a paper, not on a sheet. The set for the same chapter takes each of these traps into a problem, with the solution written out step by step.

11 corrected exercises
110 points
150 minutes

Do the exercises

Something missing from this sheet? A method, a trap, an exercise to practise it, a mistake you spotted: tell me, I read every request.

Sending goes through your free space: that is how I can answer you and tell you when the exercise is added. An email is enough, no password. Create your space. Rather not create an account? Write to me directly.

Previous sheet Counting Next sheet Discrete and continuous random variables

This chapter comes back in

The chapters that build on it, later in the year or in the years that follow.

Discrete and continuous random variables

© Ahmed Squalli Houssaini. Revision sheet published at www.letuteurscientifique.ca/en/fiches/cegep-sn1-conditional-probability. Free for personal and classroom use; republishing it elsewhere requires written permission (legal notice).

Revision sheet: probability and conditional probability (201-SN1-RE)

The essentials

After the bar, the sample space changes

Mutually exclusive and independent: two unrelated definitions

The three formulas that do everything

The mistakes that cost marks

1. Turning the bar around without changing the denominator

2. Confusing mutually exclusive with independent

3. Adding two probabilities that overlap

4. Completing to 1 across two different sample spaces

5. Forgetting that information removes cases

6. Taking independence given a state for plain independence

7. Adding reliabilities in parallel

8. Counting the remaining outcomes instead of conditioning

Which method to choose

Which tool for which form of question

How the answer is expected to be written

Proving Bayes' theorem

Justifying independence or dependence

Check before you hand in

The typical problem, taken apart

A song identification app reports a rare species

Learn by heart

Frequently asked questions

What is the difference between mutually exclusive and independent events?

How do you know when to use Bayes' theorem?

Why is P(A given B) not equal to P(B given A)?

How do you prove Bayes' theorem in CEGEP?

Why should you switch doors in the Monty Hall problem?

Corrected exercises: Probability and conditional probability, 201-SN1-RE

This chapter comes back in

See also

Looking for a 201-SN1-RE tutor in Montreal?