Probability Theory and Statistics

Lecture 3: Conditional probability and independence, part 1

16 September 2019

Lecturer: Mikhail Zhitlukhin (Михаил Валентинович Житлухин)

Email: [email protected]
 Conditional probability
The conditional probability of event A given event B is the probability
that A happens if we know that B has already happened.
Example
A = “more than 3 points on a die”, B = “even number of points”
P (A) = 1/2, P (B) = 1/2

A
1 3 5 Probability of A given B =
P ({4, 6}) 2
=
2 4 6 P ({2, 4, 6}) 3
B

1/11
Definition
The conditional probability of event A given event B is

P (A ∩ B)
P (A | B) = assume P (B) 6= 0
P (B)

The conditional probability is the “fraction which A ∩ B takes up in B”.

A A∩B B

2/11
Example
A football team is going to play 2 games. It will win the 1st game with
probability 0.7, win the 2nd game with probability 0.5, and win the both
games with probability 0.4.
If the team lose the 1st game, what is the probability to win the 2nd
game?
P (win 2nd game | lose 1st game) = ?

Answer: 1/3

3/11
Properties of conditional probability

1. Probability multiplication rule: P (A ∩ B) = P (A | B)P (B)

Also true: P (A ∩ B ∩ C) = P (A | B ∩ C)P (B | C)P (C), etc.

2. P (A ∪ C | B) = P (A | B) + P (C | B) − P (A ∩ C | B)

3. P (A | B) = 1 − P (A | B)

4. If A ⊂ B, then P (A | B) = PP (B)
(A)

If B ⊂ A, then P (A | B) = 1.

But the following is not true (even when B ∩ C = ∅):

5. P (A | B ∪ C) = P (A | B) + P (A | C)

4/11
 The total probability formula
For any events A, B such that P (B) 6= 0 the total probability formula
holds:
P (A) = P (A | B) · P (B) + P (A | B) · P (B)

Proof

P (A | B) · P (B) + P (A | B) · P (B)
B

A∩B
= P (A ∩ B) + P (A ∩ B) = P (A)
∩
A

B B The last line is because A∩B and A∩B are disjoint

Ω random events and (A ∩ B) ∪ (A ∩ B) = A.

5/11
Example
If the team is in good form, it will win a game with probability 70%.
If the team is in bad form, it will win a game with probability 30%.
Suppose 60% of the time the team is in good form. What is the
probability to win a game?

Answer: 54%

6/11
Probability tree
The previous problem can be also solved using a “probability tree”

0.7 win P = 0.6 · 0.7 = 0.42

p=
good form

p=
0 .6 0 .3 lose
p=
p=
0.4 0.3 win P = 0.4 · 0.3 = 0.12
p=

bad form
p=
0 .7 lose

Then P (win) = 0.42 + 0.12 = 0.54.

7/11
 Bayes’ formula
Example – the inverse problem for the team
Good form (60% of time): 70% win, 30% lose
Bad form (40% of time): 30% win, 70% lose
If the team wins a game, what is the probability it is in good form?
A = “win”, A = “lose”, B = “good form”, B = “bad form”

P (B | A) = ?

Thomas Bayes, 1701 – 1761

8/11
Bayes’ formula
Let P (A) > 0 and 0 < P (B) < 1. Then

P (A | B) · P (B)
P (B | A) =
P (A | B) · P (B) + P (A | B) · P (B)

Proof
probability multiplication rule
z }| {
P (A ∩ B) P (A | B) · P (B)
P (B | A) = =
P (A) P (A | B) · P (B) + P (A | B) · P (B)
| {z }
total probability formula

Answer in the problem: 7/9.

9/11
 Bonus
“Don’t worry. . . ”
Suppose a medical test for some dangerous disease gives a correct result
in 99% of cases, and only 1% of the population have this disease.
If a patient receives a positive result of the test, what is the probability
of having the disease?

Answer: 50% (find it using Bayes’ formula)

10/11
 Reading
Basic reading
• Wonnacotts: § 3.4 (but Bayes’ theorem is given as an exercise)
• Newbold, Carlson, Thorne: § 3.3 (starting from “Conditional
probability”; skip “Statistical Independence”), § 3.5

Detailed reading
• Durret: § 3.1, 3.2, 3.3
• Bertsekas, Tsitsiklis: § 1.3, 1.4

Video we watched
• youtube.com/watch?v=HGO_q0d3hsA (part 1)
• youtube.com/watch?v=eVhLltpH97Y (part 2)
(we watched part 1, starting from 29:00)
11/11