Conditional Probability

Example 6. There are five white balls numbered 1, 2, 3, 4, 5 and three black balls numbered 1, 2, 3 in
the box. We chose one ball and it turned out to be white. What is the probability that it has an
even number? An obvious answer is 25 . On the other hand we have classical probability scheme with
= {(1, w), (2, w), . . . , (5, w), (1, b), (2, b), (3, b)}
A – the event that a chosen ball has even number A = {(2, w), (4, w), (2, b)}
B – the event that a chosen ball is white B = {(1, w), (2, w), . . . , (5, w)}

|AflB|
2 |A fl B| P(A fl B)
= = =
| |
5 |B| |B| P(B)
| |

Definition 4. Assume ( , F, P) is a probability space and A, B œ F are events such that P(B) > 0. The
conditional probability of an event A under the condition that B occurred is given by:

P(A fl B)
P (A | B) =
P(B)

Fact 2. For fixed B œ F that satisfies P(B) > 0, the triple ( , F, P(· | B)) is a probability space

Proof. ’E œ F

P(E fl B)
P(E | B) = >0
P(B)
P (B fl ) P(B)
P( | B) = = =1
P (B) P(B)

(An )n=1 - countable sequence of mutually exclusive events (pairwise disjoint)

Œ

A B tŒ tŒ qŒ
P(( n=1 An ) fl B) P ( n=1 (An fl B)) P (An fl B) ÿ
Œ
€ Œ
P An | B = = = n=1 = P (An | B)
n=1
P(B) P(B) P(B) n=1

Fact 3. 1. A, B œ F and A fl B = ÿ ∆ P(A | B) = 0

2. A, B œ F and B ™ A ∆ P(A | B) = 1
Theorem 4. Assume ( , F, P) is a probability space end A1 , . . . , An œ F are events satisfying
P (A1 fl A2 fl . . . fl An ) > 0. Then

P (A1 fl A2 fl . . . fl An ) = P (An | A1 fl A2 fl . . . fl An≠1 ) P (An≠1 | A1 fl . . . fl An≠2 ) . . . P (A2 | A1 ) P (A1 )

Proof.
P (An | A1 fl . . . fl An≠1 ) · P (An≠1 | A1 fl . . . fl An≠2 ) · . . . · P (A2 | A1 ) · P (A1 )
(A((( (( (
P (A1 fl . . . fl An ) P (A
(( 1fl(. (.. fl n≠1 ) ((
P (A
( 1 fl A2 )
(( ( · (
(( ( · · · · · ⇠) ·⇠ ⇠⇠
P (A 1)
P (A ( ( ( ) P (A( ( ( ( ) P
⇠ ⇠
(A
( ( (n fl . . . fl A n≠1 ( ( 1 fl . . . fl A n≠2 1

7
Example 7. There are n ≠ 1 white balls and one black ball in the box. In each step, we withdraw one ball
until we get the black ball. What is the probability that we will finish after k steps if a) we withdraw without
returning, b) we withdraw with returning?
Let us denote white balls with w1 , w2 , . . . , wn≠1 and the black balls with b. We have

= {(b), (w1 , b) , . . . , (wn≠1 , b) , (w1 , w2 , b) , . . .}

Let Ai be an event that the i-th ball is white

! " ! "
P Āk fl Ak≠1 fl Ak≠2 fl . . . fl A1 = P Āk | Ak≠1 fl . . . fl A1 P (Ak≠1 | Ak≠2 fl . . . fl A1 ) . . . P (A2 | A1 ) P (A1 )
(2)
Case a)
! " 1
P (Ai | Ai≠1 fl . . . fl A1 ) = n≠i
n≠i+1 , P Āk | Ak≠1 fl . . . fl A1 = n≠k+1 , thus (2) is

1 n≠k+1 n≠k+2 n≠2 n≠1 1

· · · ..., · =
n≠k+1 n≠k+2 n≠k+3 n≠1 n n
Case b)
3 4
n≠1 ! " n≠1
P (Ai | Ai≠1 fl . . . fl A1 ) = , P Āk | Ak≠1 fl . . . fl A1 = 1≠ thus (2) is
n n
3 4 3 4k≠1
n≠1 n≠1 n≠1 1 n≠1
1≠ · ... · =
n n n n n
tn
Definition 5. We say that family of events (Bk )k=1 is a partition of iff Bk = and Bk ’s are pairwise
n
k=1
disjoint. We define a countable partition analogously.
Theorem 5. (Law of the total probability) Assume ( , F, P) is a probability space, A œ F and (Bk )k is a
n

partition (finite or countable) of such that P (Bk ) > 0 for every k. Then:

ÿ
P(A) = P (A | Bk ) P (Bk )
k

t t
Proof. Events (A fl Bk )k are pairwise disjoint and A = A fl =Afl k Bk = k (A fl Bk )
A B
€ ÿ ÿ
P(A) = P A fl Bk = P (A fl Bk ) = P (A | Bk ) P (Bk )
k k k

Theorem 6. With the same conditions as in the above theorem, if P(A) > 0 then for every k

P (A | Bk ) P (Bk )
P (Bk | A) = q
n P (A | Bn ) P (Bn )

Proof.
P (A fl Bk ) P (A | Bk ) · P (Bk )
P (Bk | A) = =q
P(A) n P (A | Bn ) P (Bn )

8
Example 8. Assume there are two boxes: Box I and Box II. Box I contains w1 white balls and b1 black
balls. Box II contains w2 white balls and b2 black balls. In the experiment, we first choose a box and then a
ball from the chosen box.
1. What is the probability that a chosen ball is white
2. Assume that a chosen ball is white. What is the probability that it was taken from Box I.
A - an event that a chosen ball is white.
B1 - an event that Box I was chosen.
B2 - an event that Box II was chosen.
B1 fl B2 = ÿ, B1 fi B2 = hence {B1 B2 } is partition of
P (B1 ) = P (B2 ) = 12 , P (A | B1 ) = w1w+b
1
1
, P (A | B2 ) = w2w+b
2
2
1 2
1. P(A) = P (A | B1 ) · P (B1 ) + P (A | B2 ) P (B2 ) = 12 w1w+b1
1
+ Ê2
w2 +b2
w1
P(A|B1 )P(B1 )
2. P (B1 | A) = = w1
w1 +b1
w2
w1 +b1 + w2 +b2
P(A)

Example 9. A test detecting a virus gives a correct result in 95% of cases. For a patient the test shows
positive result. Given that 1% of the population is infected, what is the probability that the patient with a
positive test is actually infected? What is the probability that a patient with negative result is infected?
Note that if we give a test to all the people, then 0.01 · 0.95 + 0.99 · 0.05 receive a positive result (infected
people with a positive test and healthy people with a positive test). Note that there are many healthy people
with positive tests: 0.95 · 0.05 of the population. This amount is approximately five times more than the
amount of infected people with positive test 0.01 · 0.95, so the probability that a random patient with a
positive test has a virus is approximately 1/6.
With formulae: let A be the event of getting a positive test and B be the event of being healthy. Then
P(B) = 0.99, P(B) = 0.01, P(A|B) = 0.95, P(A|B) = 0.95 hence P(A|B) = 1 ≠ P(A|B) = 0.05.
P(A|B)P(B) 0.01·0.95
Then P(B|A) = P(A|B)P(B)+P(A|B)P(B)
= 0.01·0.95+0.99·0.05 = 0.161017

P(A|B)P(B) 0.01·0.05
P(B|A) = P(A|B)P(B)+P(A|B)P(B)
= 0.01·0.05+0.99·0.95 = 0.00053

Independence of events
Assume A and B are events and P(B) > 0. Events A and B should be independent if the information that
B occurred does not influence the probability that A occurs. Thus we should have P(A | B) = P(A) which
means P(A fl B) = P(A)P(B)
Definition 6. Assume ( , F, P) is a probability space. Events A, B œ F are independent if P(A fl B) =
P(A)P(B)
Remark 4. If P(A) = 0 then for any B œ F the events A and B are independent. Similarly if P (A) = 1.
Definition 7. Events A1 , A2 , . . . , An are independent if for all 2 6 k 6 n and for any sequence 1 6 i1 <
i2 < . . . < ik 6 n we have:

P (Ai1 fl Ai2 fl . . . fl Aik ) = P (Ai1 ) · P (Ai2 ) · . . . P (Aik )

Definition 8. We say events A1 , A2 , . . . , An are pairwise independent of for any i, j œ {1, . . . , n}, i ”= j, the
event Ai and Aj are independent.
Remark 5. Of course, if A1 , . . . , An are independent, then they are pairwise independent.

9
Example 10. We throw a die twice. Let A - in the first throw, we got an even number of spots, B - in the
second throw, we get an even number of spots, C- the sum of spots is even.

1
P(A) = P(B) = P(C) =
2
1
P(A fl B) = P(A fl C) = P(B fl C) = ∆ A, B, C are pairwise independent
4
1
But A fl B µ C and P(A fl B fl C) = P(A fl B) = ”= P(A)P(B)P(C)
4

For any (possibly infinite) number of events, we define independence by:

Definition 9. Assume {Ai }iœI is a family of events. We say that these events are independent iff for any n
and pairwise different i1 , i2 , . . . , in œ I the events Ai1 ,..., Ain are independent.
Definition 10. Assume ( , F, P) is a probability space and F1 , F2 , . . . , Fn , are ‡-algebras contained in F.
We say that ‡ -algebras F1 , . . . , Fn are independent if for any A1 œ F1 , A2 œ F2 , . . . , An œ Fn we have:

P (A1 fl . . . fl An ) = P (A1 ) . . . P (An )

Theorem 7. Under the assumptions of the above definition ‡-algebras F1 , . . . , Fn are independent if and
only if any events A1 œ F1 , . . . , An œ Fn are independent.

Proof. ≈ trivial
∆ We need to show that for any 2 6 k 6 n and for any sequence 1 6 i1 < i2 < . . . < ik 6 n we have

P (Ai1 fl Ai2 fl . . . fl Aik ) = P (Ai1 ) P (Ai2 ) . . . P (Aik ) (3)

Consider the events:

I
Ai if i = il for some l
Bi =
otherwise

Of course B1 œ F1 , . . . , Bn œ Fn
P (B1 fl... flBn ) = P (B1 ) . . . P (Bn ) which is equivalent to (3)

Definition 11. Let A œ F be an event. Then ‡(A) is the smallest ‡-algebra that contains A, i.e. ‡(A) =
{ÿ, , A, Ā}
Fact 4. Assume events A1 , A2 , . . . , An are independent. Then ‡-algebras ‡ (A1 ) , . . . , ‡ (An ) are independent.

Proof. We need to show that for any Bi œ ‡ (Ai ) we have

P (B1 fl . . . fl Bn ) = P (B1 ) . . . P (Bn ) (4)

If at least one Bk = ÿ then (4) is satisfied. If for some j we have Bj = , then we can omit this event in
both sides of (4). Hence it is enough to show that for any sequence 1 6 i1 < i2 < . . . < ik Æ n

P (Bi1 , fl . . . fl Bik ) = P (Bi1 ) . . . P (Bik )

10
 ) *
where Bij œ Aij , Āij

! "
P Āi1 fl Ai2 fl . . . fl Aik = P (Ai2 fl . . . fl Aik \ Ai1 fl Ai2 fl . . . fl Aik ) = P (Ai2 fl . . . fl Aik ) ≠ P (Ai1 fl Ai2 fl . . . fl Aik ) =
! "
= P (Ai2 ) . . . P (Aik ) ≠ P (Ai1 ) P (Ai2 ) . . . P (Aik ) = (1 ≠ P (Ai1 )) P (Ai2 ) . . . .P (Aik ) = P Āi1 P (Ai2 ) . . . P (Aik )

By induction, the result follows.

Infinitely and finitely often

This topic will be used a lot later when speaking about the convergence of random variables!
Definition 12. Assume (An )n=1 is a sequence of events, i.e. Ak œ F for every k. We say that:
Œ

1. Events in (An )n=1 occur infinitely often iff An occurs for infinite number of indices n œ N.
Œ

{An i.o.} = {Ê | Ê œ An . for Œ number of indices n œ N}

2. Events in (An )n=1 occur finitely often iff An occurs for at most finite number of indices n œ N
Œ

{An f.o.} = {Ê | Ê œ An for finitely many indices n œ N}

Theorem 8.
Œ €
‹ Œ ‹
€
{An i.o.} = An and {An f.o.} = {An i.o.} = Ān .
m=1 nØm m=1 nØm

In particular {An i.o.} œ F and {An f.o.} œ F

Proof. The second part follows from the definition of ‡-algebra.

uΠFor
t the first one, note that it is enough to
show the result for {An i.o.}. We first show that {An i.o.} ™ m=1 nØm An . Suppose Ê œ An for an infinite
t uΠt
number of indices. Then for any m Ø 1 we have that Ê œ nØm An . Hence Ê œ m=1 nØm An . Suppose
next that t
Êœ/ {An i.o.}. Then Ê œ An for only finely
uΠmanyt indices n. Let k be the largest of these indices.
Then Ê œ/ nØm An for all m Ø k + 1. Thus Ê œ / m=1 nØm An

Theorem 9 (Borel-Cantellli). Let ( , F, P) be a probability space. Let (An )n=1 be a sequence of events
Œ

qŒ
1. If n=1 P (An ) < Πthen P({An i.o.}) = 0
qŒ
2. If (An )n=1 are independent and i=1 P (An ) = Πthen P({An i.o.}) = 1
Œ

Proof. We will show separately 1) and 2)

uŒ t uŒ
1. {An i.o.} = m=1 n>m An = m=1 Bm , where B1 ´ B2 ´ B3 . . .

A B Q R
Œ
‹ € Œ
ÿ
P ({An i.o.}) = P Bm = lim P (Bm ) = lim P a An b 6 lim P (An ) = 0
mæŒ mæŒ mæŒ
m=1 n>m n=m

qŒ
where the last equality follows from the convergence of n=1 P (An ) < Œ.

11
2. We will show P ({An f.o.}) = 0

Œ ‹
€ Œ
€ ‹
{An f.o.} = Ān = Bm , Bm = Ān
m=1 n>m m=1 n>m

and B1 ™ B2 ™ B3 ™ . . .

A B Q R
Œ
€ ‹
P ({An f.o.}) = P Bm = lim P (Bm ) = lim P a Ān b
mæŒ mæŒ
m=1 n>m
um+k
For any fixed m let Ck,m = n=m Ān . One can easily see that:

C1,m ´ C2,m ´ C3,m ´ . . . (5)

Œ
‹ ‹
Ck,m = Ān (6)
k=1 nØm

Thus
A Œ
B
‹
P ({An f.o.}) = lim P Ck,m = lim lim P (Ck,m ) = lim lim P(Am )P(Am+1 ) . . . P(Am+k ) =
mæŒ mæŒ kæŒ mæŒ kæŒ
k=1
k
Ÿ k
Ÿ qm+k
P(An )
= lim lim (1 ≠ P(Am+i )) Æ lim lim e≠P(Am+i ) = lim lim e≠ n=m = lim 0 = 0
mæŒ kæŒ mæŒ kæŒ mæŒ kæŒ mæŒ
i=0 i=0

12