PROBABILITY & STATISTICS

Sample Spaces
Chapter 2: Probability
Probability
LEARNING OBJECTIVES
Addition Rules

Conditional Pro.
1. Sample Spaces and Events (KG Mẫu và Biến cố)
2. Interpretations of Probability (Cách tiếp cận xác suất)
Multi. Rules
3. Addition Rules (Quy tắc cọng)
Total Pro. Rule
4. Conditional Probability (Xác suất có đk)
Independence
5. Multiplication and Total Probability Rules (Quy tắc
Bayes’Theorem
nhân và CT xác suất toàn phần)
Summary 6. Independence (Biến độc lập)

13/02/25
7. Bayes’ Theorem
Department of Mathematics 1/38
 Sample spaces and events
Sample Spaces
Sample Spaces

Probability
Definition
Definition Random
Randomexperiment
experiment
Addition Rules
•• AnAnexperiment
experimentthat thatcan
canresult
resultin
indifferent
differentoutcomes,
outcomes,
Conditional Pro. even
eventhough
thoughititisisrepeated
repeatedininthe
thesame
samemanner
mannerevery
every
time,
time,isiscalled
calledaarandom
randomexperiment.
experiment.
Multi. Rules
•• The
Theset
setof
ofall
allpossible
possibleoutcomes
outcomesof ofaarandom
random
Total Pro. Rule experiment
experimentisiscalled
calledthe
thesample
samplespace
spaceofofthe
the
experiment.
experiment.The
Thesample
samplespace
spaceisisdenoted
denotedasasS.S.
Independence
•• AnAnevent
eventisisaasubset
subsetof
ofthe
thesample
samplespace
spaceof
ofaarandom
random
Bayes’Theorem
experiment.
experiment.
Summary

13/02/25 Department of Mathematics 2/38

 Sample spaces and events
Sample spaces
Sample Spaces

Probability

Addition Rules
Example
Conditional Pro.
Random experiment: Roll a die
Multi. Rules
Sample space: S ={1, 2, 3, 4, 5, 6}
Total Pro. Rule Event: E1 = {Die is even}={2, 4, 6}
Independence E2 = {Die is odd}={1, 3, 5}
Bayes’Theorem

Summary

13/02/25 Department of Mathematics 3/38

 Sample spaces and events
Sample spaces
Sample Spaces
Tree Diagrams (Sơ đồ cây)
Probability
Sample spaces can also be described graphically with
Addition Rules tree diagrams.
Conditional Pro.
– When a sample space can be constructed in
several steps or stages, we can represent each of
Multi. Rules the n1 ways of completing the first step as a
Total Pro. Rule
branch of a tree.
– Each of the ways of completing the second step
Independence can be represented as n2 branches starting from
Bayes’Theorem the ends of the original branches, and so forth.

Summary

13/02/25 Department of Mathematics 4/38

 Sample spaces and events
Sample spaces
Sample Spaces

Example
Probability
A probability experiment consists of tossing a
Addition Rules
coin and then rolling a six-sided die. Describe the
Conditional Pro. sample space.
Multi. Rules Tree diagram:
Total Pro. Rule

Independence

Bayes’Theorem H1 H2 H3 H4 H5 H6 T1 T2 T3 T4 T5 T6

Summary The sample space has 12 outcomes:

S = {H1, H2, H3, H4, H5, H6, T1, T2, T3, T4, T5, T6}
13/02/25 Department of Mathematics 5/38
 Sample spaces and events
Sample spaces
Sample Spaces
Example
Probability Each message in a digital communication system is classified
as to whether it is received within the time specified by the
Addition Rules
system design. If three messages are classified, use a tree
Conditional Pro.
diagram to represent the sample space of possible outcomes.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem

Summary

13/02/25 Department of Mathematics 6/38

 Sample spaces and events
Sample Spaces
Sample Spaces
Basic Set Operations
Probability

The union of two events is the event that consists of all

Addition Rules
outcomes that are contained in either of the two events.
Conditional Pro. We denote the union as E1E2.
Multi. Rules The intersection of two events is the event that consists
of all outcomes that are contained in both of the two
Total Pro. Rule
events. We denote the intersection as E1E2.
Independence
The complement (Biến cố đối) of an event in a sample
Bayes’Theorem space is the set of outcomes in the sample space that are
not in the event. We denote the component of the event E
Summary as E’.

13/02/25 Department of Mathematics 7/38

 Sample spaces and events
Sample Spaces
Sample Spaces
Venn Diagrams
Probability mutually exclusive

Addition Rules

Conditional Pro.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem

Summary

13/02/25 Department of Mathematics 8/38

 Sample spaces and events
Sample Spaces
Sample Spaces
Important properties:
Probability

Addition Rules A  (B  C) = (A  B)  C

Conditional Pro.
A  (B  C) = (A  B)  C

Multi. Rules
A  (B  C) = (A  B)  (A  C)
(A  B)’ = A’  B’
Total Pro. Rule
(A  B)’ = A’  B’
Independence
A = (A  B)  (A  B’)
Bayes’Theorem

Summary

13/02/25 Department of Mathematics 9/38

 Interpretations of Probability
Sample Spaces
Introduction
Probability
Probability
There are three approaches to assessing the probability of an
Addition Rules
uncertain event:
1. a priori classical probability (XS cổ điển tiên nghiệm):
Conditional Pro. the probability of an event is based on prior knowledge of
the process involved.
Multi. Rules

Total Pro. Rule 2. empirical classical probability (XS cổ điển thực

nghiệm): the probability of an event is based on observed
Independence data.
Bayes’Theorem 3. subjective probability (XS chủ quan): the probability of
an event is determined by an individual, based on that
Summary person’s past experience, personal opinion, and/or analysis
of a particular situation.
13/02/25 Department of Mathematics 10/38
 Interpretations of Probability
Sample Spaces
Equally Likely Outcomes
Probability
Probability
Whenever
Wheneveraasample
samplespace
spaceconsists
consistsof
ofNNpossible
possibleoutcomes
outcomesthat
thatare
are
Addition Rules equally
equallylikely,
likely,the
theprobability
probabilityof
ofeach
eachoutcome
outcomeisis1/N.
1/N.
Conditional Pro.
1. a priori classical probability
Multi. Rules
X number of ways the event can occur
Total Pro. Rule Probability of Occurrence  
T total number of possible outcomes

Independence 2. empirical classical probability

Bayes’Theorem number of favorable outcomes observed

Probability of Occurrence 
total number of outcomes observed
Summary

13/02/25 Department of Mathematics 11/38

 Interpretations of Probability
Sample Spaces

Example priori classical probability

Probability
Probability
Find the probability of selecting a face card (Jack, Queen,
Addition Rules or King) from a standard deck of 52 cards.
Conditional Pro.

Multi. Rules X number of face cards

Probability of Face Card  
Total Pro. Rule
T total number of cards

Independence X 12 face cards 3

 
T 52 total cards 13
Bayes’Theorem

Summary

13/02/25 Department of Mathematics 12/38

 Interpretations of Probability
Sample Spaces
Example empirical classical probability
Probability
Probability Find the probability of selecting a male taking statistics
Addition Rules
from the population described in the following table:

Conditional Pro.
Taking Stats Not Taking Stats Total
Multi. Rules Male 84 145 229
Female 76 134 210
Total Pro. Rule
Total 160 279 439
Independence

Bayes’Theorem number of males taking stats

Probability of Male Taking Stats 
total number of people
Summary
84
 0.191
439
13/02/25 Department of Mathematics 13/38
 Interpretations of Probability
Sample Spaces
Axioms of Probability (Các tiên đề xác suất)
Probability
Probability
Probability is a number that is assigned to each member of a
Addition Rules collection of events from a random experiment that satisfies the
following properties:
Conditional Pro.
If S is the sample space and E is any event in a random
Multi. Rules experiment,

Total Pro. Rule

(1) P(S) = 1
(2) 0 ≤ P(E) ≤ 1
Independence
(3) For two events E1 and E2 with E1∩E2 = Ø
Bayes’Theorem
P(E1  E2) = P(E1) + P(E2)
Summary

13/02/25 Department of Mathematics 14/38

 Addition Rules
Sample Spaces
The special addition rule
Probability

Addition Rules
Rules
1. If A and B are mutually exclusive events,
Addition
P(A  B) = P(A) + P(B)
Conditional Pro.
2. A collection of events, E1, E2, …, Ek is said to be
Multi. Rules
mutually exclusive (xung khắc) if for all pairs,
Total Pro. Rule
Ei ∩ Ej = Ø
Independence
For a collection of mutually exclusive events,
Bayes’Theorem
P(E1  E2 …  Ek) = P(E1) + P(E2) +…+ P(Ek)
Summary

13/02/25 Department of Mathematics 15/38

 Addition Rules
Sample Spaces
The general addition rule
Probability

Addition Rules
Rules
1. Two events: A and B are any events,
Addition
P(A  B) = P(A) + P(B) – P(A ∩ B)
Conditional Pro.

Multi. Rules
2. Three events:
Total Pro. Rule

Independence P(A  B  C) = P(A) + P(B) + P(C) - P(A ∩ B)

Bayes’Theorem
- P(B ∩ C) - P(C ∩ A) + P(A ∩ B ∩ C)

Summary

13/02/25 Department of Mathematics 16/38

 Addition Rules
Sample Spaces
Example
Probability Find the probability of selecting a male or a statistics
Addition Rules
Addition Rules
student from the population described in the following
table:
Conditional Pro.
Taking Not Taking Total
Multi. Rules Stats Stats
Male 84 145 229
Total Pro. Rule
Female 76 134 210
Independence Total 160 279 439

Bayes’Theorem
P(Male or Stat) = P(M) + P(S) – P(M and S)
Summary
= 229/439 + 160/439 – 84/439 = 305/439
13/02/25 Department of Mathematics 17/38
 Conditional Probability
Sample Spaces
Conditional Probability
Probability
To introduce conditional probability, consider an
Addition Rules
example involving manufactured parts.
Conditional Pro.
Conditional Pro.
Let D denote the event that a part is defective and let F
Multi. Rules
denote the event that a part has a surface flaw.
Total Pro. Rule

Then, we denote the probability of D given, or

Independence
assuming, that a part has a surface flaw as P(D|F). This
Bayes’Theorem notation is read as the conditional probability of D
Summary
given F, and it is interpreted as the probability that a
part is defective, given that the part has a surface flaw.
13/02/25 Department of Mathematics 18/38
 Conditional Probability
Sample Spaces
Example 2-17(page 32)
Probability

Addition Rules

Conditional Pro.
Conditional Pro.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem

Summary

13/02/25 Department of Mathematics 19/38

 Conditional Probability
Sample Spaces
Example 2-17(page 32)
Probability

Addition Rules

Conditional Pro.
Conditional Pro.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem

Summary

13/02/25 Department of Mathematics 20/38

 Conditional Probability
Sample Spaces
Definition
Definition Conditional
ConditionalProbability
Probability
Probability
The
Theconditional
conditionalprobability
probabilityof
ofan
anevent
eventBBgiven
givenan
anevent
event
Addition Rules A,
A,denoted
denotedas
asP(B|A),
P(B|A),isis
Conditional Pro.
Conditional Pro. P(B|A)
P(B|A)==P(B
P(B∩∩A)/P(A)
A)/P(A)
Multi. Rules for
forP(A)
P(A)>>0.0.
Total Pro. Rule Special case: all outcomes are equally likely
Independence

Bayes’Theorem

Summary

13/02/25 Department of Mathematics 21/38

 Conditional Probability
Sample Spaces
Example 2-23 (page 43)
Probability

Addition Rules

Conditional Pro.
Conditional Pro. Note that in this example all four of the following
probabilities are different:
Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem
Here, P(D) and P(D|F) are probabilities of the same
event, but they are computed under two different states
Summary of knowledge. Similarly, P(F) and P(F|D) are computed
under two different states of knowledge.
13/02/25 Department of Mathematics 22/38
 Conditional Probability
Sample Spaces
Example
Probability Of the cars on a used car lot, 70% have air conditioning
Addition Rules
(AC) and 40% have a CD player (CD). 20% of the cars
have both.
Conditional Pro.
Conditional Pro.
What is the probability that a car has a CD player, given
Multi. Rules that it has AC ?
Total Pro. Rule
CD No CD Total
P(CD and AC)
Independence P(CD | AC) 
AC 0.2 0.5 0.7 P(AC)
Bayes’Theorem No AC 0.2 0.1 0.3 0.2
 .2857
0.7
Summary Total 0.4 0.6 1.0

13/02/25 Department of Mathematics 23/38

 Multiplication rules
Sample Spaces
Multiplication Rule
Probability
P(A ∩ B) = P(A|B)P(B) = P(B|A)P(A)
Addition Rules

Conditional Pro.
Example
The probability that an automobile battery subject to high
Multi. Rules
Multi. Rules engine compartment temperature suffers low charging current
is 0.7. The probability that a battery is subject to high engine
Total Pro. Rule compartment temperature is 0.05.
Independence The probability that a battery is subject to low charging current
and high engine compartment temperature is
Bayes’Theorem

Summary
C={a battery suffers T ={a battery is subject to high
low charging current} engine compartment temperature}
13/02/25 Department of Mathematics 24/38
 Total Probability Rule
Sample Spaces
Partition of an event
Probability

Addition Rules Figure 2-15

Partitioning an event
Conditional Pro. into two mutually
exclusive subsets.
Multi. Rules

Total Pro.
Total Pro. Rule
Rule

Independence
Figure 2-16
Partitioning an
Bayes’Theorem event into several
mutually
Summary exclusive subsets.

13/02/25 Department of Mathematics 25/38

 Total Probability Rule
Sample Spaces
Total Probability Rule: two events
Probability

Addition Rules P(B) = P(BA) + P(BA’) = P(B|A)P(A) + P(B|A’)P(A’)

Conditional Pro.
Total Probability Rule: multiple events
Multi. Rules

Total Pro.
Pro. Rule
Rule
E1  E2  … Ek =S
Total

Independence

Bayes’Theorem

Summary

13/02/25 Department of Mathematics 26/38

 Total Probability Rule
Sample Spaces
Example 2-27 (page 48)
Probability

Addition Rules

Conditional Pro.

Multi. Rules

Total Pro.
Total Pro. Rule
Rule

Independence

Bayes’Theorem

Summary

13/02/25 Department of Mathematics 27/38

 Independence
Sample Spaces
Definition
Definition
Probability
Two
Twoevents
eventsisiscalled
calledindependent
independentififany
anyone
oneof
ofthe
the
Addition Rules following
followingequivalent
equivalentstatements
statementsisistrue:
true:
Conditional Pro. (1)
(1)P(A|B)
P(A|B)==P(A)
P(A)
Multi. Rules (2)
(2)P(A
P(A∩∩B)
B)==P(A)P(B)
P(A)P(B)
Total Pro. Rule (3)
(3)P(B|A)
P(B|A)==P(B)
P(B)
Independence
Independence
Proposition: If A and B are independent events, then
Bayes’Theorem so are events A and B’, events A’ and B, and events A’
Summary
and B’.

13/02/25 Department of Mathematics 28/38

 Independence
Sample Spaces
Example 2-31 (page 52)
Probability
A day’s production of 850 manufactured parts contains 50
Addition Rules
parts that do not meet customer requirements. Two parts are
selected at random, without replacement, from the batch. Let
Conditional Pro. A ={the first part is defective}, and let B ={the second part is
defective}.
Multi. Rules
We suspect that these two events are not independent because
Total Pro. Rule knowledge that the first part is defective suggests that it is
less likely that the second part selected is defective.
Independence
Independence
P(B|A) = 49/849 the two events
Bayes’Theorem are not
independent, as
Summary we suspected.

13/02/25 Department of Mathematics 29/38

 Independence
Sample Spaces
Definition
Definition
Probability
The
Theevents
eventsEE11,,EE22,,…,
…,EEnnare
areindependent
independentififandandonly
onlyififfor
for
any
anysubset
subsetof ofthese events Ei1 , Ei2 , ... , Eik
theseevents
Addition Rules

Conditional Pro.
P(Ei1  Ei2 ... Eik ) P(Ei1 )P(Ei2 ) ... P(Eik )
Multi. Rules

Total Pro. Rule

Exercise
Independence
Independence
Two coins are tossed. Let A denote the event “at most
Bayes’Theorem one head on the two tosses,” and let B denote the event
“one head and one tail in both tosses.” Are A and B
Summary
independent events?

13/02/25 Department of Mathematics 30/38

 Independence
Sample Spaces
Example 2-34 (page 53)
Probability
The following circuit operates only if there is a path of
Addition Rules functional devices from left to right. The probability that each
device functions is shown on the graph. Assume that devices
Conditional Pro. fail independently. What is the probability that the circuit
operates?
Multi. Rules
P(T B) = 1- P[(T B)’]
Total Pro. Rule
= 1-P(T’ ∩B’)
Independence
Independence
P(T’ ∩B’) = P(T’)P(B’)
Bayes’Theorem
= (1-0.95)2 =0.052
Summary
P(T B) = 1 - 0.052 =0.9975
13/02/25 Department of Mathematics 31/38
 Bayes’ Theorem
Sample Spaces
Bayes’ Theorem
Probability

Addition Rules If E1, E2, …, Ek are k mutually exclusive and exhaustive

events and B is any event,
Conditional Pro.
P(B | E1 )P( E1 )
P( E1 | B) 
Multi. Rules P(B | E1 )P( E1 )  P(B | E2 )P(E2 ) ... P(B | Ek )P(Ek )
Total Pro. Rule
for P(B)  0

Independence
In special case:
Bayes’Theorem
Bayes’Theorem P ( B | A) P ( A)
P( A | B)  for P ( B )  0
Summary P( B)

13/02/25 Department of Mathematics 32/38

 Bayes’ Theorem
Sample Spaces
Example
Probability In a state where cars have to be tested for the emission of
pollutants, 25% of all cars emit excessive amount of pollutants.
Addition Rules
When tested, 99% of all cars that emit excessive amount of
Conditional Pro. pollutants will fail, but 17% of all cars that do not emit
excessive amount of pollutants will also fail. What is the
Multi. Rules probability that a car that fails the test actually emits excessive
amounts of pollutants?
Total Pro. Rule

Independence Let A denote the event that a car fails the test and B the
event that it emits excessive amounts of pollutants:
Bayes’Theorem
Bayes’Theorem
P(B) = 0.25, P(A|B) = 0.99 and P(A|B’) = 0.17.
Summary
We have to find P(B|A)?

13/02/25 Department of Mathematics 33/38

 Bayes’ Theorem
Sample Spaces

P( A | B) P( B)
Probability P( B | A) 
P( A)
Addition Rules
where
Conditional Pro.
P(A) = P(AB) + P(AB’) = P(A|B)P(B) + P(A|B’)P(B’)
Multi. Rules
= 0.375
Total Pro. Rule
Substitution into the formula for P(B|A) yields
Independence
P(B|A) = 0.66
Bayes’Theorem
Bayes’Theorem

Summary

13/02/25 Department of Mathematics 34/38

 Bayes’ Theorem
Sample Spaces
Example
Probability
Two events A and B are such that P[A∩B] = 0.15,
Addition Rules
P[A∪B] = 0.65, and P[A|B] = 0.5. Find P[B|A].

Conditional Pro.

Multi. Rules

Total Pro. Rule

Independence

Bayes’Theorem
Bayes’Theorem

Summary

13/02/25 Department of Mathematics 35/38

 Random Variables
Sample Spaces

Probability
Definition
Definition
Addition Rules AArandom
randomvariable
variableisisaafunction
functionthat
thatassigns
assignsaareal
real
number
numbertotoeach
eachoutcome
outcomein inthe
thesample
samplespace
spaceof
ofaa
Conditional Pro.
random
randomexperiment.
experiment.
Multi. Rules
AArandom
randomvariable
variableisisdenoted
denotedbybyananuppercase
uppercaseletter
lettersuch
such
Total Pro. Rule as
asX.
X.After
Afterananexperiment
experimentisisconducted,
conducted,thethemeasured
measured
value
valueofofthe
therandom
randomvariable
variableisisdenoted
denotedby
byaalowercase
lowercase
Independence letter
lettersuch
suchas
asxx 70
70milliamperes.
milliamperes.
Bayes’Theorem
Bayes’Theorem

Summary

13/02/25 Department of Mathematics 36/38

 Random Variables
Sample Spaces

Probability

Definition
Definition
Addition Rules
AAdiscrete
discreterandom
randomvariable
variableisisaarandom
randomvariable
variablewith
withaa
Conditional Pro. finite
finite(or
(orcountably
countablyinfinite)
infinite)range.
range.
Multi. Rules AAcontinuous
continuousrandom
randomvariable
variableisisaarandom
randomvariable
variablewith
with
an
aninterval
interval(either
(eitherfinite
finiteor
orinfinite)
infinite)of
ofreal
realnumbers
numbersfor
for
Total Pro. Rule
its
itsrange.
range.
Independence

Bayes’Theorem
Bayes’Theorem

Summary

13/02/25 Department of Mathematics 37/38

 Summary
Sample Spaces
We have studied:
Probability
1. Sample Spaces and Events
Addition Rules 2. Interpretations of Probability
Conditional Pro. 3. Addition Rules

Multi. Rules 4. Conditional Probability

Total Pro. Rule

5. Multiplication and Total Probability Rules
6. Independence
Independence
7. Bayes’ Theorem
Bayes’Theorem
Homework: Read slides of the next lecture.
Summary
Summary

13/02/25 Department of Mathematics 38/38