Chapter 2 - Axioms of Probability 1
Chapter 2 - Axioms of Probability 1
2 - Axioms of Probability
Content
2.1 Introduction
2.2 Sample Spaces & Events
2.3 Definitions of probability
2.4 Axioms of probability
2.5 Some Propositions involving probability
1. Introduction
We introduce the concept of the probability of an event occurring. Then we’ll show how
probabilities can be computed/calculated.
Sample Space
Sample space (S): “The set of all possible outcomes of the random experiment.”
Examples
The following examples all provide sample spaces for the possible outcomes:
• The sex of a new-born child is:
o S = {g, b}
▪ g = girl, b = boy
• The order of finish in a race among 7 horses:
o S = {all 7! Permutations of (1,2,3,4,5,6,7)}
▪ Each number represents a specific horse
• Flipping two coins:
o S = {(h, h), (h, t), (t, h), (t, t)}
▪ h = heads, t = tails
• Tossing two dice:
o S = {(i, j): i, j = 1,2,3,4,5,6}
▪ Consists of 36 points (6×6)
• Lifetime of a transistor (non-negative real numbers)
o S = {x: 0 ≤ x < ∞}
Events
Event: “Any subset of the sample space.”
• An event has occurred, when any of the outcomes in the event is obtained when the
experiment is performed.
Examples
The following examples all provide events for the possible outcomes:
• The sex of a new-born child is:
o E = {g} o F = {b}
▪ Event that child = girl. ▪ Event that child = boy.
• The order of finish in a race among 7 horses:
o E = {all the orders beginning with 3}
▪ The event that horse 3 wins the race.
• Flipping two coins:
o E = {(h,h), (h,t)}
▪ The event that heads appears on the first coin
• Tossing two dice:
o E = {(1,6), (2,5), (3,4), (4,3), (5,2), (6,1)}
▪ The event that the sum of the two dice is equal to 7.
• Lifetime of a transistor (non-negative real numbers)
o E = {x: 0 ≤ x < 5}
▪ The event that the transistor doesn’t last longer than 5 hours.
Example
• 𝐸 ∪ 𝐹 = {(ℎ, ℎ), (ℎ, 𝑡), (𝑡, ℎ)} is the event where heads appears at least on one coin.
Intersection
• 𝐸 ∩ 𝐹 = 𝐸𝐹 = 𝐸 & 𝐹 → both events occur.
• When more than two events are in intersection: ⋂∞
𝑖=1 𝐸𝑖
Example
• 𝐸 ∩ 𝐹 = {(ℎ, 𝑡), (𝑡, ℎ)} is the event where heads & tails appears on one coin each.
Example
• Event E is where the sum of the dice equals 7 & F is where the sum equals 6.
• 𝐸 ∩ 𝐹 = ∅ → there is no such outcome where both occur simultaneously.
Complementary Events
• The complement EC or E’ of an event E consists of all the elements in S that are not
elements of E.
• Note: SC = ∅, since the experiment must result in some outcome.
Example
• Event E is where the sum of the dice equals 7.
• 𝐸′ → all the outcomes where the sum of the two dice is not equal to 7.
Subsets
• 𝐸 ⊂ 𝐹 → E is a subset of F, E is contained in F & every element in E is an element in F.
Supersets
• 𝐸 ⊃ 𝐹 → E is a superset of F, F is contained in E & every element in F is an element in E.
Equal Events
• 𝐸 ⊂ 𝐹 & 𝐹 ⊂ 𝐸 → events E and F are equal in terms of having the same elements.
Venn Diagrams
• A graphical representation for illustrating logical relations among events.
• Sample space = large rectangle
• Events = circles contained in the rectangle
• Overlapping circles indicate common elements in from different events.
Notation
• 𝑛(𝑆) → number of outcomes in the sample space (S).
• 𝑛(𝐸) → number of outcomes for the event (E).
Examples
(d) E ⊂ F
Laws
Commutative Laws
𝐸∪𝐹 =𝐹∪𝐸 𝐸∩𝐹 =𝐹∩𝐸
Associative Laws
(𝐸 ∪ 𝐹) ∪ 𝐺 = 𝐸 ∪ (𝐹 ∪ 𝐺) (𝐸 ∩ 𝐹) ∩ 𝐺 = 𝐸 ∩ (𝐹 ∩ 𝐺)
Distributive Laws
(𝐸 ∪ 𝐹) ∩ 𝐺 = 𝐸𝐺 ∪ 𝐹𝐺 (𝐸 ∩ 𝐹) ∪ 𝐺 = (𝐸 ∪ 𝐺) ∩ (𝐹 ∪ 𝐺)
Examples
De Morgan’s laws:
𝑛 𝑐 𝑛 𝑛 𝑐 𝑛
𝑐
(⋃ 𝐸𝑖 ) = ⋂ 𝐸𝑖 (⋂ 𝐸𝑖 ) = ⋃ 𝐸𝑖 𝑐
𝑖=1 𝑖=1 𝑖=1 𝑖=1
Examples
(𝐸 ∪ 𝐹)𝑐 = 𝐸 𝑐 𝐹 𝑐 (𝐸 ∩ 𝐹)𝑐 = 𝐸 𝑐 ∪ 𝐹 𝑐
Definitions of Probability in terms of
Relative frequency
• The probability that event E will occur during an experiment that is repeated using the
same space (S) is defined as: limiting relative frequency of E
𝑛(𝐸)
𝑃(𝐸) = lim
𝑛→∞ 𝑛
Measure of Belief
• A personal/subjective view of probability is the measure of an individual’s degree of
belief in the statements that they are making.
o E.g. I am 90% percent that Barca will win against Bayern Munich tonight.
Examples
Example 1
Suppose that in a 7-horse race, you believe that each of the first 2 horses has a 20% chance
of winning, horses 3 and 4 each have a 15% chance, and the remaining 3 horses have a 10%
chance each. Would it be better for you to wager at even money that the winner will be one
of the first three horses or to wager, again at even money, that the winner will be one of the
horses 1, 5, 6, and 7?
Solution:
On the basis of your personal probabilities concerning the outcome of the race, your
probability of winning the first bet is 0.55 (0.2 + 0.2 + 0.15) whereas it is 0.5 (0.2+0.1+0.1+0.1)
for the second bet. Hence, the first wager is more attractive.
Note:
• A person’s subjective probabilities aren’t always accurate/consistent & should be verified.
3. Axioms of Probability
𝑛(𝐸)
The convergence of to a constant limiting value is an assumption/axiom. Since this
𝑛
single assumption is extraordinarily complicated, we shall make smaller axioms.
Assume that for every event E in a sample space S there exists a number P(E) = probability of
the event E occurring. This number P(E) satisfies the following 3 axioms:
1. 𝟎 ≤ 𝑷(𝑬) ≤ 𝟏 for every event E.
2. 𝑷(𝑺) = 𝟏
3. 𝑷(⋃∞ ∞
𝒊=𝟏 𝑬𝒊 ) = ∑𝒊=𝟏 𝑷(𝑬𝒊 ) provided E1, E2, … are mutually exclusive (not intersections)
This assumption of the existence of a set function P, defined on the events of a sample space
S satisfies the 3 axioms above, constitutes the modern mathematical approach to probability.
Strong law of large numbers states that:
𝑛(𝐸)
𝑃(𝐸) = lim ( )
𝑛→∞ 𝑛
This means if the experiment is repeated over & over again, then with probability 1, the
proportion of time during which any specific event E occurs will equal P(E).
Examples
Example 3.1
If our experiment consists of tossing a coin and if we assume that a head is as likely to appear
as a tail, then we would have:
𝑷({𝑯}) = 𝑷({𝑻}) = 𝟎. 𝟓
If, however, the coin was biased and we believed that a head were twice as likely to appear
as a tail, then we would have:
𝟐
𝑷({𝑯}) =
𝟑
𝟏
𝑷({𝑻}) =
𝟑
Example 3.2
If a die is rolled & supposing that all 6 sides are equally likely to appear, then we would have:
𝟏
𝑷({𝟏}) = 𝑷({𝟐}) = 𝑷({𝟑}) = 𝑷({𝟒}) = 𝑷({𝟓}) = 𝑷({𝟔}) =
𝟔
From Axiom 3, it would thus follow that the probability of rolling an even number equals:
𝑷({𝟐, 𝟒, 𝟔}) = 𝑷({𝟐}) + 𝑷({𝟒}) + 𝑷({𝟔})
𝟏 𝟏 𝟏
= + +
𝟔 𝟔 𝟔
𝟑
=
𝟔
= 𝟎. 𝟓
4. Some Simple Propositions
Proposition 0
• P(∅) = 0 → the null event has probability 0 of occurring.
o 𝑃(𝑆) = ∑∞ ∞
𝑖=1 𝑃(𝐸𝑖 ) = 𝑃(𝑆) + ∑𝑖=2 𝑃(∅)
Proposition 1
Since EC & E are always mutually exclusive & since 𝑬𝒄 ∪ 𝑬 = 𝑺, we have that:
𝑃(𝑆) = 𝑃(𝐸 𝑐 ∪ 𝐸) = 𝑃(𝐸) + 𝑃(𝐸 𝑐 ) = 1
𝑃(𝐸 𝑐 ) = 1 − 𝑃(𝐸)
The probability that an event does not occur is 1 minus the probability that it does occur.
Example
3 5
If the 𝑃(𝐻) = when tossing a coin then the 𝑃(𝑇) =
8 8
Proposition 2
If the event E is contained in the event F, then the probability of E is no greater than the
probability of F.
𝐸 ⊂ 𝐹 ⇒ 𝑃(𝐸) ≤ 𝑃(𝐹)
Proof
Since 𝐸 ⊂ 𝐹, it follows that:
𝐹 = 𝐸 ∪ 𝐸𝑐 𝐹
From axiom 3 we get:
𝑃(𝐹) = 𝑃(𝐸) + 𝑃(𝐸 𝑐 𝐹)
Since 𝑃(𝐸 𝑐 𝐹) ≥ 0, proposition 2 is proved.
Example
The 𝑃(1) < 𝑃(𝑜𝑑𝑑) when rolling a die.
𝐼 𝐼𝐼 𝐼𝐼𝐼
Since:
𝐸 ∪ 𝐹 = 𝐼 ∪ 𝐼𝐼 ∪ 𝐼𝐼𝐼
𝐸 = 𝐼 ∪ 𝐼𝐼
𝐹 = 𝐼𝐼 ∪ 𝐼𝐼𝐼
Therefore:
𝑃(𝐸 ∪ 𝐹) = 𝑃(𝐼) + 𝑃(𝐼𝐼) + 𝑃(𝐼𝐼𝐼)
𝑃(𝐸) = 𝑃(𝐼) + 𝑃(𝐼𝐼)
𝑃(𝐹) = 𝑃(𝐼𝐼) + 𝑃(𝐼𝐼𝐼)
This shows that:
𝑃(𝐸 ∪ 𝐹) = 𝑃(𝐸) + 𝑃(𝐹) − 𝑃(𝐸 ∩ 𝐹)
Examples
Example 4.1
Travis is taking two movies along on his holiday vacation. With probability 0.5, he will like the
first movie; with probability 0.4, he will like the second movie; and with probability 0.3, he
will like both movies. What is the probability that he likes neither movie?
Solution:
Let E = the first movie, and F = the second movie
𝑃(𝐸) = 0.5
𝑃(𝐹) = 0.4
𝑃(𝐸 ∩ 𝐹) = 0.3
Therefore:
𝑃(𝐸 ∪ 𝐹) = 𝑃(𝐸) + 𝑃(𝐹) − 𝑃(𝐸 ∩ 𝐹)
= 0.5 + 0.4 − 0.3
= 0.6
The probability that he likes neither is thus:
𝑃(𝐸 𝐶 ∩ 𝐹 𝑐 ) = 𝑃([𝐸 ∪ 𝐹]𝑐 )
= 1 − 𝑃(𝐸 ∪ 𝐹)
= 1 − 0.6
= 𝟎. 𝟒
0.4
Boole’s Inequality
𝑛 𝑛
𝑃 (⋃ 𝐸𝑖 ) = ∑ 𝑃(𝐸𝑖 )
𝑖=1 𝑖=1
Additional Example
Suppose you are given that 𝑷(𝑨) = 𝟎. 𝟑, 𝑷(𝑨 ∩ 𝑩𝑪 ) = 𝟎. 𝟐 and 𝑷(𝑨 ∪ 𝑩) = 𝟎. 𝟔.
Hint draw a Venn Diagram.
A: 0.3 B
𝑃(𝐴𝐶 ∩ 𝐵) = 𝑃(𝐴 ∪ 𝐵) − 𝑃(𝐴) = 0.3
0.4
Calculate:
a) P(at least one of A,B)
Solution:
The probability of A or B.
𝑃(𝐴 ∪ 𝐵) = 𝟎. 𝟔
b) P(exactly one of A,B)
Solution:
The probability of (A and not B) or (not A and B)
Examples
Example 5.1
If two dice are rolled, what is the probability that the sum of the upturned faces will equal 7?
Solution:
Table of the sum of two dice: vertical: outcome on die 1, horizontal: outcome on die 2
1 2 3 4 5 6
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
There are 36 possible outcomes and each of them are equally likely.
Therefore:
𝒏(𝑺) = 𝟑𝟔
Event E is the event where the sum of the upturned faces equal 7. From the table above we
see that that happens 6 times out of 36 outcomes.
Therefore:
𝒏(𝑬) = 𝟔
Therefore:
𝒏(𝑬) 𝟔 𝟏
𝑷(𝑬) = = =
𝒏(𝑺) 𝟑𝟔 𝟔
Example 5.2
If 3 balls are “randomly drawn” from a bowl containing 6 white and 5 black balls, what is the
probability that one of the balls is white and the other two black?
Solution:
When order is important: Permutations
(𝑾, 𝑩, 𝑩) + (𝑩, 𝑾, 𝑩) + (𝑩, 𝑩, 𝑾)
𝑷(𝑬) =
𝟑 𝒃𝒂𝒍𝒍𝒔
(𝟔 × 𝟓 × 𝟒) + (𝟓 × 𝟔 × 𝟒) + (𝟓 × 𝟒 × 𝟔)
=
𝟏𝟏 × 𝟏𝟎 × 𝟗
𝟒
=
𝟏𝟏
When order is not important: Combinations
𝟔 𝟓
( )( ) 𝟒
𝑷(𝑬) = 𝟏 𝟐 =
𝟏𝟏 𝟏𝟏
( )
𝟑
Probability tree solution:
General remark
When k items are taken randomly from a set of n items, they can either be ordered selection
or unordered set of items selected.
Ordered
Assume each new selection is equally likely to be any of the unselected items of the set.
Unordered
𝒏
Assume that all ( ) possible subsets of k items are equally likely to be selected.
𝒌
General example
Suppose 5 people are to be randomly selected from a group of 20 individuals consisting of
10 married couples, and we want to determine the probability that the 5 chosen are all
unrelated.
Unordered
𝟐𝟎
We would assume that all ( ) possible outcomes are equally likely.
𝟓
𝟏𝟎
Firstly 5 of the 10 couples have a member selected: ( ). Then 1 of the 2 members of each
𝟓
𝟓
𝟐
of these couples is selected: ( ) .
𝟏
𝟏𝟎
( ) × 25 168
𝑃(𝑁) = 𝟓 = = 𝟎. 𝟓𝟐𝟎𝟏
𝟐𝟎 323
( )
𝟓
Ordered
Therefore 𝟐𝟎 × 𝟏𝟗 × 𝟏𝟖 × 𝟏𝟕 × 𝟏𝟔 equally likely outcomes, of which 𝟐𝟎 × 𝟏𝟖 × 𝟏𝟔 ×
𝟏𝟒 × 𝟏𝟐 are unrelated.
𝟐𝟎 × 𝟏𝟖 × 𝟏𝟔 × 𝟏𝟒 × 𝟏𝟐 168
𝑃(𝑁) = = = 𝟎. 𝟓𝟐𝟎𝟏
𝟐 × 𝟏𝟕𝟎 × 𝟏𝟗 × 𝟏𝟖 × 𝟏𝟔 323
Example 5.3
A committee of 5 is to be selected from a group of 6 men and 9 women. If the selection is
made randomly, what is the probability that the committee consists of 3 men and 2 women?
Solution:
𝟏𝟓 𝟔 𝟗
𝑛(𝑆) = ( ) = 𝟑 𝟎𝟎𝟑 & 𝑛(𝐸) = ( ) ( ) = 𝟕𝟐𝟎
𝟓 𝟑 𝟐
𝟔 𝟗
( )( ) 𝟐𝟒𝟎
𝑷(𝑬) = 𝟑 𝟐 =
𝟏𝟓 𝟏𝟎𝟎𝟏
( )
𝟓
Example 5.4
An urn contains n balls, one of which is special. If of these balls k are withdrawn one at a
time, with each selection being equally likely to be any of the balls that remain at the time,
what is the probability that the special ball is chosen?
Solution:
𝒏
𝑛(𝑆) = ( ) where event E is the event where the one ball is special.
𝒌
Therefore:
1 𝑛−1
( )( )
𝑷(𝑬) = 1 𝑘 − 1
𝑛
( )
𝑘
(𝑛 − 1)!
1×
(𝑘 − 1)! (𝑛 − 𝑘)!
=
(𝑛)!
(𝑘)! (𝑛 − 𝑘)!
(𝑛 − 1)! (𝑘)! (𝑛 − 𝑘)!
= ×
(𝑘 − 1)! (𝑛 − 𝑘)! (𝑛)!
(𝑛 − 1)! 𝑘(𝑘 − 1)!
= ×
(𝑘 − 1)! 𝑛(𝑛 − 1)!
𝒌
=
𝒏
Alternate calculations are on page 66.
Example 5.5
A 5-card poker hand is said to be a full house if it consists of 3 cards of the same denomination
and 2 other cards of the same denomination (of course, different from the first
denomination). Thus, a full house is three of a kind plus a pair. What is the probability that
one is dealt a full house?
Solution:
𝟓𝟐
𝑛(𝑆) = ( ) → 5 cards chosen from 52 available.
𝟓
𝟒 𝟒
Full house implies: choosing 3 of a kind ( ) and a pair ( ) of a different kind.
𝟑 𝟐
There are 13 different cards: therefore initially 13 types for the triplet & then 12 for the pair.
4 4 13 2 4 4
13 × 12 × ( ) ( ) ( ) ( ) ( ) ( )
𝑷(𝑭𝒖𝒍𝒍 𝑯𝒐𝒖𝒔𝒆) = 3 2 = 2 1 3 2 = 𝟎. 𝟎𝟎𝟏𝟒
52 52
( ) ( )
5 5
Example 5.6: Birthday Problem/Paradox
If n people are present in a room, what is the probability that no two of them celebrate their
birthday on the same day of the year? How large need n be so that this probability is less
than 0.5?
Solution:
We assuming that they can celebrate their birthday on any one of the 365 days. Therefore:
𝑛(𝑆) = (𝟑𝟔𝟓)𝒏 . The probability that no two celebrate the same birthday is thus:
(365)(364)(363) … (365 − 𝑛 + 1)
𝑷(𝒏𝒐 𝟐 𝒃𝒊𝒓𝒕𝒉𝒅𝒂𝒚𝒔) =
365𝑛
When 𝑛 ≥ 23 the 𝑷(𝒏𝒐 𝟐 𝒃𝒊𝒓𝒕𝒉𝒅𝒂𝒚𝒔) < 𝟎. 𝟓
365 1
That is because every pair of individuals has the probability of 2 = . If there are 23
365 365
23
people, there are ( ) = 253 different pairs of individuals.
2
Example 5.7
A total of 36 members of a club play tennis, 28 play squash, and 18 play badminton.
Furthermore, 22 of the members play both tennis and squash, 12 play both tennis and
badminton, 9 play both squash and badminton, and 4 play all three sports. How many
members of this club play at least one of three sports?
Solution:
Let E = tennis, F = squash & G = badminton.
36 28
6 18 1
4
8 5
1
18
𝑃(𝑋 ≥ 1) = 𝑃(𝐸 ∪ 𝐹 ∪ 𝐺)
= 𝑃(𝐸) + 𝑃(𝐹) + 𝑃(𝐺) − 𝑃(𝐸𝐹) − 𝑃(𝐸𝐺) − 𝑃(𝐹𝐺) + 𝑃(𝐸𝐹𝐺)
36 + 28 + 18 − 22 − 12 − 9 + 4 43
= =
𝑁 𝑁
Hence, we can conclude that 43 members play at least one of the three sports.
Note:
• The probability of the union of three events is equal to:
𝐏(𝐄 ∪ 𝐅 ∪ 𝐆) = 𝐏(𝐄) + 𝐏(𝐅) + 𝐏(𝐆) − 𝐏(𝐄𝐂 𝐅𝐆) − 𝐏(𝐄𝐅 𝐂 𝐆) − 𝐏(𝐄𝐅𝐆𝐂 ) − 𝟐𝐏(𝐄𝐅𝐆).
• The probability that exactly one of the events E or F occurs.
𝐏(𝐄) + 𝐏(𝐅) − 𝟐𝐏(𝐄𝐅)
• The probability of event E and not F occurring is equal to:
𝐏(𝐄𝐅 𝐜 ) = 𝑷(𝑬) − 𝑷(𝑬𝑭)
•
• An urn contains M red and N blue balls. If a random sample of size r is chosen, what is the
probability that it contains exactly k red balls?
𝑴 𝑵
( )( )
𝑷(𝑬) = 𝒌 𝒓 − 𝒌
𝑴+𝑵
( )
𝒓