§4 Conditional probability

§4.1 Introduction
4.1.1 Definition. Let B be an event such that P(B) > 0. Then the conditional probability P(A|B)
of A given B is
P(A ∩ B)
P(A|B) = .
P(B)
4.1.2 Physical interpretation:
If the event B has occurred, what do we know about the probability of event A?
Example.
Dial a mobile phone no. at random. It is natural to assume that

P(respondent is male) = P(respondent is female) ≈ 1/2.

Ask the person whether he/she weighs over 70 kg and the answer is “yes” say. Then, naturally,

P(respondent is male | over 70 kg) > P(respondent is female | over 70 kg).

Example.
Toss 2 dice. Given first dice scores 1,

P(2nd dice scores 6 | 1st dice scores 1) = P(2nd dice scores 6),

since the two events {1st dice scores 1} and {2nd dice scores 6} are independent (discussed
later).

4.1.3 Note that

(i) P(Ω|B) = P(Ω ∩ B)/P(B) = 1;

(ii) if A ∩ C = ∅, then P(A ∪ C|B) = P(A|B) + P(C|B);
(iii) if A1 ⊂ A2 ⊂ · · · , then P(A1 ∪ A2 ∪ · · · | B ) = limn→∞ P(An |B).

(i), (ii) and (iii) together confirm that for any fixed event B, P(·|B) defines a probability for
the events of Ω.

14
4.1.4 Example.
A mother has two kids. You ask, “are they both boys? ” The mother says “No”. You know
now that one of the two kids must be a girl. What is the probability that the mother has a
son? Assume that a kid has equal probability (= 1/2) to be of either sex.
Solution:
Take Ω = {bb, bg, gb, gg} with equally likely outcomes (each with probability 1/4), where “ bg” means
that the elder kid is a boy and the younger one is a girl, etc.
Define A = {mother has a son} = {bb, bg, gb}, and B = {both kids are boys} = {bb}.
The required probability is
P(A ∩ B c ) P({bg, gb}) 2/4 2
P(A|B c ) = c
= = = .
P(B ) P({bg, gb, gg}) 3/4 3

4.1.5 (Birthday problem)

N guests (none twins) born in 2001 were gathered. What is the probability that at least two
have same birthday?
Solution:
Let Aj = {guests 1, 2, . . . , j have different birthdays}, j = 1, 2, . . . , N .
Clearly, if N > 365, then AN = ∅, hence

P(at least 2 have same birthday) = P(AcN ) = 1.

Consider the case N ≤ 365. Noting that AN ⊂ AN −1 ⊂ · · · ⊂ A2 ,

P(AN ) = P(AN ∩ AN −1 ) = P(AN |AN −1 )P(AN −1 ) = P(AN |AN −1 )P(AN −1 |AN −2 )P(AN −2 )
N
Y −1
= · · · = P(AN |AN −1 )P(AN −1 |AN −2 ) · · · P(A3 |A2 )P(A2 ) = P(A2 ) P(Aj+1 |Aj ).
j=2

In general,
365 − k 364
P(Ak+1 |Ak ) = (k = 2, . . . , 365), and P(A2 ) = .
365 365
Thus
N
Y −1
P(at least 2 have same birthday) = P(AcN ) = 1 − P(A2 ) P(Aj+1 |Aj )
j=2
   
364 366 − N 364!
=1− ··· =1− .
365 365 (365 − N )! 365N −1

15
 Clearly, as N increases, the above probability increases.
Numerical example:

0.4757, N = 22,
P(at least 2 have same birthday) ≈
0.5073, N = 23.

Thus, at least 23 persons are required for P(at least 2 have same birthday) to exceed 50%.

4.1.6 Theorem. Suppose that A ⊂ B1 ∪ B2 ∪ · · · ∪ Bn , where the events B1 , . . . , Bn are mutually

exclusive and have nonzero probabilities. Then

(i) (Law of total probability)

P(A) = P(A|B1 )P(B1 ) + P(A|B2 )P(B2 ) + · · · + P(A|Bn )P(Bn );

(ii) (Bayes’ Theorem)

P(A|Bj )P(Bj )
P(Bj |A) = .
P(A|B1 )P(B1 ) + · · · + P(A|Bn )P(Bn )
Proof:
Note that P(A ∩ Bj ) = P(A|Bj )P(Bj ). For (i),
n n


X X
P(A) = P A ∩ (B1 ∪ · · · ∪ Bn ) = P (A ∩ B1 ) ∪ · · · ∪ (A ∩ Bn ) = P(A ∩ Bj ) = P(A|Bj )P(Bj ).
j=1 j=1

For (ii),
P(Bj ∩ A) P(A|Bj )P(Bj )
P(Bj |A) = = .
P(A) P(A|B1 )P(B1 ) + · · · + P(A|Bn )P(Bn )

§4.2 Simpson’s paradox

4.2.1 Example.
There are 2 treatments for a disease, A and B. Apply A to some patients, B to others and
record the outcomes: either R (recovered) or D (dead). Suppose we observe the following:

Men (M) Women (W)

R D R D
A 20 80 40 20
B 50 160 15 5

16
 Suppose the probability of recovery can be reliably derived from the above observed record.

(a) For men receiving treatment A, the probability of recovery is

20 1
P(R|A ∩ M ) = = .
100 5
(b) For men receiving treatment B, the probability of recovery is
50 5
P(R|B ∩ M ) = = .
210 21
(c) For women receiving treatment A, the probability of recovery is
40 2
P(R|A ∩ W ) = = .
60 3
(d) For women receiving treatment B, the probability of recovery is
15 3
P(R|B ∩ W ) = = .
20 4
(a) and (b) imply that for men, B is better.
(c) and (d) imply that for women, B is better.
But, for all patients receiving A, we have
 
60 3 65 13 3
P(R|A) = = , P(R|B) = = < ,
160 8 230 46 8
which implies that A is better!!

4.2.2 Simpson’s paradox may arise if we combine results for 2 groups when different numbers and
proportions of each group receive each treatment, and when the recovery rates for each treat-
ment are markedly different between the 2 groups. In such circumstances, direct combination
may hide, or even reverse, the apparent effects.

§4.3 Diagnosis paradox

4.3.1 Example.
Let C = {person has cancer} and D = {diagnose cancer}. Assume

P(D|C) = 0.95, P(Dc |C c ) = 0.95, P(C) = 0.005.

17
 Thus, the diagnosis has high probability (0.95) of being correct and the cancer is quite rare
(0.5% incidence rate).
Now, a person has been diagnosed cancer, what is the probability of his really having cancer?
Answer:
0.95 × 0.005
P(C|D) = = 0.087.
0.95 × 0.005 + 0.05 × 0.995
For reference, if the person has been diagnosed as having no cancer, the probability of his really not
having cancer is P(C c |Dc ) = 0.99975.

4.3.2 Other applications:

A witness (to an accident, crime, flying saucer etc.) thinks he observed (D) an event (C)
which is known to be rare. Even if his powers of observation are good (i.e. both P(D|C) and
P(Dc |C c ) are high), we should NOT trust his evidence too much.

§4.4 *** More challenges ***

4.4.1 (Gambler’s ruin) A man gambles with a wealthy woman as follows. He tosses a fair coin repeatedly;
if it comes up heads then the woman pays him one dollar, but if it comes up tails then he pays her
one dollar. He plays this game repeatedly until either he runs out of money or he accumulates a
sum of N dollars. Suppose that he starts with a capital of k (0 < k < N ) dollars. What is the
probability that he is ultimately bankrupted?
Hint: If Ak denotes the event that he is eventually bankrupted with an initial capital of k
dollars, then we have P(Ak | {first toss shows heads}) = P(Ak+1 ).
What happens if the woman is incredibly rich and the man does not want to stop at any fixed
amount of N dollars?

4.4.2 (a) A and B take turn to roll a fair dice, with A rolling first. The winner is the first to roll a six.
What is the probability that A wins?
(b) A (possibly unfair) coin is tossed repeatedly and the outcome recorded as a sequence of H’s
(heads) and T’s (tails). Define each set of adjacent H’s and each set of adjacent T’s a run.
Find the probability that a run of r heads is observed before a run of s tails.

4.4.3 One of 3 prisoners, A, B and C, is chosen randomly to be executed the next morning. Only the
warden knows who is going to die but he is not allowed to tell the prisoners until just before the
execution.

18
In the evening, prisoner A asks the warden, “Please, tell me which of B and C is not going to die.
If both are not to die, just tell me one name at random. Since I know anyway that one of them is
not going to die, you will not be giving me any information.”
The warden thought about it for a while, and replied, “I cannot tell you this. For now you think
you have only 1/3 chance of dying. Suppose I told you that B is not to die. You would then think
that you have a 1/2 chance of dying since either you or C will be executed. Thus, in effect, I would
have given you some information.”
Was the warden right?

19