Chapter 2

Probability
 2

Introduction
Statisticians are often dealing with either numerical (Quantitative)
data, representing counts or measurements, or categorical
(Qualitative) data. Any recording of information, whether it be
numerical or categorical, is an observation.

Statisticians use the word experiment to describe any process that

generates a set of data.

An experiment that can result in different outcomes, even though it is

repeated in the same manner every time, is called a random
experiment.
 3

Introduction

The branch of mathematics that studies a likelihood or a chance of a

phenomenon happening is known as probability.
 4

Introduction
All statistical experiments have three things in common:

➢ The experiment can have more than one possible outcome.

➢ Each possible outcome can be specified in advance.

➢ The outcome of the experiment depends on chance.

 5

Sample Space
The set of all possible outcomes of a statistical experiment is called
the sample space and is represented by the symbol S.

Each outcome in a sample space is called an element or a member of

the sample space, or simply a sample point. The number of all sample
points is n.

n=6
 6

Sample Space
A sample space is discrete if it consists of a finite or countable infinite set of
outcomes (S={10,20,30,40} , S={1,2,3,…})

A sample space is continuous if it contains an interval (either finite or infinite)

of real numbers (S={x I 10 <x<20 }, S=R- ={x I x<0 }, S={x I 100 ≤ x ≤ 250 )

More than one sample space can be used to describe the outcomes of an
experiment.
 7

Sample Space
➢ Randomly select a camera and record the recycle time of a flash.
S = R+ = {x | x > 0}, the positive real numbers, is continuous infinite.

➢ Suppose it is known that all recycle times are between 1.5 and 5
seconds.
S = {x | 1.5 ≤ x ≤ 5} is continuous finite.

➢ It is known that the recycle time has only three values(low, medium
or high).
S = {low, medium, high} attributes.

➢ Does the camera conform to minimum recycle time specification

S = {yes, no} attributes.
 8

Sample Space
➢ Randomly select two cameras and record the recycle time of a flash. If the objective
of the analysis is to consider only whether the cameras conform to the
manufacturing specifications or not.
S = {yy, yn, ny, nn}, attributes.

➢ If we are interested only in the number of conforming cameras in the sample.

S = {0, 1, 2} is discrete finite.

➢ Consider an experiment in which cameras are tested until the flash recycle time fails
to meet the specifications.
S = {n, yn, yyn, yyyn,…} attributes.

➢ Consider the number of cameras tested until the flash recycle time fails to meet the
specifications.
S = {1, 2, 3, 4,…} discrete infinite.
 9

Sample Space
Sample spaces with a large or infinite number of sample points are best
described by a statement or rule.

What is the sample space of an experiment that represents the cities in the
world with a population over 1 million?

We read the previous rule as: S is the set of all x such that x is a city with a
population over 1 million.

Suppose a random experiment is to select an item from a production batch

until one defective item is observed. What is the sample space?
 10

Sample Space
 11

Sample Space
An experiment consists of flipping a coin and then flipping it a second
time if a head occurs. If a tail occurs on the first flip, then a die is
tossed once. Construct a tree diagram and determine the sample
space.
 12

S = {OOO, OOL, OLO, OLL, LOO, LOL, LLO, LLL}

The number of sample points in the sample space is
2*2*2=8
 13

The number of sample points in the sample space is

2*2*3*4= 48
 14

Events
An event (E) is a subset of a sample space of a random experiment.

The complement of an event A with respect to S is the subset of all

elements of S that are not in A. We denote the complement of A by the
symbol A` or 𝐴ഥ 𝑜𝑟 𝐴𝑐 .
 15

Events
➢ When tossing a die S={1,2,3,4,5,6}. The event A is getting an outcome that
is divisible by 3:
The event B is getting an outcome that is divisible by 21:
The event C is getting and outcome that is not divisible by 21:

➢ If three items are selected from a manufacturing batch to observe if the

item is defective or not.
S={DDD, DDN, DND, DNN, NDD, NDN, NND, NNN}
The event A is getting more than one defective item in the sample:
The event B is getting at most one defective item:

➢ If the sample space is the life in years of an electric component.

S={t I t>=0}. The event A is getting a component whose life is at most 4
years:
The event B is getting an item with life between 2 and 6 years:
 16

Events
The intersection of two events A and B, denoted by the symbol A ∩ B, is the
event containing all elements that are common to A and B (the elements in
events A and B)
Example: Let A = {a, b, c} and B = {b, c, d, e}; then A ∩ B = {b, c}
Let C = {t I 4< t <8} D = {t I 6 ≤ t ≤12}; then C ∩ D = {t I 6 ≤ t< 8} continuous finite.

Two events A and B are mutually exclusive, or disjoint, if A ∩ B = φ, that is, if

A and B have no elements in common.

The union of the two events A and B, denoted by the symbol A∪B, is the event
containing all the elements that belong to A or B or both.
 17

Discrete Events
In the game of snakes and ladders, a fair die is thrown. If event
E1 represents all the events of getting a natural number less than 4,
event E2 consists of all the events of getting an even number and
E3 denotes all the events of getting an odd number. List the sets
representing the following:

i)E1 or E2 or E3
ii)E1 and E2 and E3
iii)E1 but not E3
The sample space is given as S = {1 , 2 , 3 , 4 , 5 , 6}
E1 = {1,2,3} E2 = {2,4,6} E3 = {1,3,5}
i)E1 or E2 or E3= E1∪ E2 ∪ E3= {1, 2, 3, 4, 5, 6}
ii)E1 and E2 and E3 = E1∩ E2 ∩ E3 = ∅
iii)E1 but not E3 = E1 ∩ E3`= {2}
 18

Continuous Events
If M = {x I 3<x<9 } and N = {y I 5<y<12} calculate:
i) M ∪ N
ii) M ∩ N
iii) M ∩ N`
iv) M`∪ N

i) M ∪ N = {z I 3<z<12}
ii) M ∩ N = {z I 5<z<9}
iii) N` = {y≤5 , y≥12} M ∩ N`= {z I 3<z ≤ 5}
iv) M` = {x≤3 , x≥9} M`∪ N= {z I z≤3 , z>5 }
 19

Venn Diagrams
Venn diagrams are the diagrams that are used to represent the sets, relation
between the sets and operation performed on them, uses circles (overlapping,
intersecting and non-intersecting), to denote the relationship between sets.

Mutually exclusive events

A∩B=φ
 20

Events Laws
➢ Commutative law (event order is unimportant):
A ∩ B = B ∩ A and A ∪ B = B ∪ A

➢ Distributive law (like in algebra):

(A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C)
(A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)

➢ Associative law (like in algebra):

(A ∪ B) ∪ C = A ∪ (B ∪ C)
(A ∩ B) ∩ C = A ∩ (B ∩ C)
 21

Events Laws
➢ DeMorgan’s law:
(A ∪ B) = A ∩ B The complement of the union is the intersection of
the complements.
(A ∩ B) = A ∪ B The complement of the intersection is the union of
the complements.

➢ Complement law:
(A) = A
 22

Exercises
 23

Counting Sample Points

We use the counting techniques to count the number of points in the
sample space without listing each element. The counting methods we
will study are:

➢ Multiplication rule.
➢ Permutations (linear and circular)
➢ Partitioning
➢ Combinations

We should know where to use each one of them based on the case.
 24

Multiplication Rule

Suppose that two coins are tossed in parallel with the two dices, how
many sample points are there?
 25

Multiplication Rule

AND = Multiply OR = Sum

 26

Permutations (select & arrange)

When we are interested in a sample space that contains elements
as all possible orders or arrangements of a group of objects.

For example:
we may want to know how many different arrangements are
possible for sitting 6 people around a table (circular).

we may want to know how many different orders are possible for a
lock with 4 numbers (linear).

A permutation is an arrangement of all or part of a set of objects.

IN PERMUTATION ORDER MATTERS

 27

Permutations-Distinct objects
Permutations
(select & arrange)
The order matters
Distinct Objects

When we have n distinct When we have n distinct

When we have n distinct objects,
objects, in how many ways can objects, taking r objects at a
in how many ways can we arrange
we arrange them: time, in how many ways can
them in a circle:
(ABC) we arrange r items:
We can arrange them in n! way We can select & arrange r
We can arrange them in a circle in
(n factorial) items from n distinct objects by
(n-1)! Way
n! = n(n-1)(n-2)…(2)(1) nPr ways:
Special case: 0!=1
Example: In how many ways can
Example: In how many ways Example: In how many ways
we arrange 6 students around a
can we arrange 6 students on 6 can we arrange 6 students out
table?
chairs? of ten on 6 chairs?
no. of ways=
no. of ways= 10!
(6-1)!=5!=5*4*3*2*1=120 way 𝑃
10 6 = = 151,200 𝑤𝑎𝑦
6!=6*5*4*3*2*1=720 way 4!
 28

Permutations-Distinct objects
 29

Permutations-Distinct objects
A witness to hit and run accident told the police that the license number contained the
letters RLS followed by 4 digits, the first of which is 4. If the witness cannot recall the
last 3 digits, but he is certain that all 4 digits are different.

RLS4 _ _ _
0,1,2,3,4,5,6,7,8,9
Because of the 3 digits are different 4 will be out.
Therefore, the answere will be 9P3= 9!/(9-3)!=(9x8x7x6!)/(6!) = 504

What will be the solution if the numbers can be repeated?

 30

Permutations-Distinct objects
objects
 31

Counting techniques-General Questions

In a certain state, automobile license plates display three letters followed by three digits.
How many such plates are possible if repetition of the letters:
➢ Is not allowed?
➢ Is allowed?
Note: repetition of numbers is allowed.

No Repetition With Repetition

If there is repetition, we cannot use the permutation formula. We can solve questions with the
multiplication rule only.
 32

Permutations-Similar objects (Partitioning)

Often, we are concerned with the number of ways of partitioning a set of n objects into
r subsets called cells. A partition has been achieved if the intersection of every possible
pair of the r subsets is the empty set φ and if the union of all subsets gives the original
set (S). The order of the elements within a cell is of no importance.
 33

Permutations-Similar objects (Partitioning)

 34

Permutations-Similar objects (Partitioning)

 35

Combinations (select)

𝒏𝑪𝒓 =

IN COMBINATION ORDER DOES NOT MATTER

 36

Combinations (select)

AND = Multiply OR = Sum

 37

Combinations (select)

If there is repetition (with replacement), we cannot use the combination formula.

 38
The relation between Permutations & Combinations
If S = {a, b, c, d},
In how many ways can we select three letters from S?
Select Combination (order does not matter)
n=4 r=3 𝟒 𝟒!
𝑪
𝟒 𝟑 = = = 𝟒 𝒘𝒂𝒚𝒔
𝟑 𝟏! 𝟑!

Select then arrange, we calculate in how many ways can we select three
items from four then we arrange them by r! ways:
𝟒!
𝟒𝑷𝟑 = 𝟒𝑪𝟑 ∗ 𝟑! = 𝟑! = 𝟒 ∗ 𝟑 ∗ 𝟐 ∗ 𝟏 = 𝟐𝟒 𝒘𝒂𝒚𝒔
𝟏! 𝟑!
 39
The relation between Permutations & Combinations

# of permutations ≥ # of combinations (they are equal only when r equals zero or 1, proof it)

Q1: A club has nine members. In how many ways can a committee of three be
chosen from the members of this club?

Q2: A committee of seven—consisting of a chairman, a vice chairman, a

secretary, and four other members—is to be chosen from a class of 20
students. In how many ways can this committee be chosen?

Q3:There are 30 people in a group. If all shake hands with one another , how
many handshakes are possible?
 40

Exercises
 41

Probability of an Event

“The chance of rain today is 30%”

Is a statement that quantifies our feeling about the possibility of rain. The
likelihood of an outcome is quantified by assigning a number from the
interval [0, 1] to the outcome (or a percentage from 0 to 100%). Higher
numbers indicate that the outcome is more likely than lower numbers. A 0
indicates an outcome will not occur. A probability of 1 indicates that an
outcome will occur with certainty.
 42

Probability of an Event
We introduce probability for discrete sample spaces—those with
only a finite (or countably infinite) set of outcomes.

Probability is used to quantify the likelihood, or chance, that an

outcome of a random experiment will occur. It is the chance of
something happening out of a defined number of opportunities.
𝐹𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦
Probability = Relative Frequency =
𝑇𝑜𝑡𝑎𝑙

For each element, a weight or “probability” between 0 and 1, will be

assigned.

The total weights (i.e., probabilities) = 1

 43

Probability of an Event

No
intersection
 44

Probability of an Event

In the previous example, all sample points are equally likely, which means they have
the same weight, how to find the probability if the sample points are not equally
likely?
 45

Probability of an Event
 46

Probability of an Event
 47

Probability of an Event
 48

Probability of an Event

What is the probability that two defective parts are selected in the sample?
P=535,095/15,890,700=0.034

What is the probability that no defective parts are selected? (P=0.676)

 49

Additive Rules

Proof that using

Venn diagrams
 50

Additive Rules
 51

Additive Rules
 52

Additive Rules
 53

Additive Rules
 54

Additive Rules
 55

Additive Rules
56
 57
Exercises
 58
Conditional Probability, Independence, and the Product Rule
Your two favourite foods are Pizza and Mansaf. Let A represent the event that you
eat Pizza for breakfast, and B represent the event that you eat Mansaf for lunch. On
a randomly selected day, the probability that you will eat Pizza for breakfast, P(A), is
0.6. The probability that you will eat Mansaf for lunch, P(B), is 0.5. The conditional
probability that you eat Mansaf for lunch given that you eat Pizza for breakfast,
P(B/A), is 0.4. What is the probability that you eat Pizza for breakfast given that you
eat Mansaf for lunch?

P(eating Pizza for BF)=P(A)=0.6 P(eating Mansaf for L)=P(B)=0.5

P(eating Pizza for BF given eating Mansaf for L) P(eating Mansaf for L given eating Pizza for BF)
= P(A/B)=? = P(B/A)=0.4
 59
Conditional Probability, Independence, and the Product Rule
The probability of an event B occurring when it is known that some event A has
occurred is called a conditional probability and is denoted by P(B|A). The symbol
P(B|A) is usually read “the probability that B occurs given that A occurs” or simply
“the probability of B, given A.”

Using the product rule or multiplicative rule:

 60
Conditional Probability, Independence, and the Product Rule

This means that occurrence of one

event has no impact on the
probability of occurrence of the
other event.
 61
Conditional Probability, Independence, and the Product Rule
Suppose that our sample space S is the population of adults in a small town who
have completed the requirements for a college degree. They are categorized
according to gender and employment status. The data are given in the following
Table:

One of these individuals is to be selected at random. What is the probability that a man
is chosen, given that he is already employed?

Let the event M represent that the person chosen is a man.

Let the event E represent that the person is employed.
 62
Conditional Probability, Independence, and the Product Rule
𝑃(𝑀 ∩ 𝐸)
𝑃 𝑀𝐸 =
𝑃(𝐸)

𝑛(𝑀 ∩ 𝐸) 460
𝑃 𝑀∩𝐸 = =
𝑛(𝑆) 900

𝑛(𝐸) 600
𝑃 𝐸 = =
𝑛(𝑆) 900

𝑃(𝑀 ∩ 𝐸)
𝑃 𝑀𝐸 =
𝑃(𝐸)
460/900 460 23
= = =
600/900 600 30
 63
Conditional Probability, Independence, and the Product Rule
Example: The probability that a regularly scheduled flight departs on time is P(D)=0.83;
the probability that it arrives on time is P(A)=0.82; and the probability that it departs and
arrives on time is P(D ∩ A)=0.78. Find the probability that a plane:

(a) Arrives on time, given that it departed on time.

(b) Departed on time, given that it arrived on time.
(c) Arrives on time, given that it did not depart on time.

𝑃(𝐴∩𝐷) 0.78
𝑎. 𝑃 𝐴 𝐷 = = = 0.94 𝑃 𝐴` 𝐷 = 1 − 𝑃 𝐴 𝐷 =0.06
𝑃(𝐷) 0.83

𝑃(𝐴∩𝐷) 0.78
𝑏. 𝑃 𝐷 𝐴 = = = 0.95 𝑃 𝐷` 𝐴 = 1 − 𝑃 𝐷 𝐴 =0.05
𝑃(𝐴) 0.82

𝑃(𝐴∩𝐷`) 𝑃 𝐴 −𝑃(𝐴∩𝐷) 0.82−0.78

c. 𝑃 𝐴 𝐷` = = = = 0.24
𝑃(𝐷`) 𝑃(𝐷`) 0.17
 64
Conditional Probability, Independence, and the Product Rule

Are A and B dependent?

A= {3, 5, 7, 9} P(A) = 4/8

 65
Conditional Probability, Independence, and the Product Rule

Are the events dependent?

 66
Conditional Probability, Independence, and the Product Rule
 67
Conditional Probability, Independence, and the Product Rule
The following table provides an example of 400 parts classified by surface flaws and as
(functionally) defective. Find: P(F), P(D), P(D/F), P(D`/F), P(D/F`), P(D`/F`)

40 28
𝑃 𝐹 = 400 𝑃 𝐷 = 400

𝑃(𝐷 ∩ 𝐹) 10/400 10 30 𝑃 𝐷` ∩ 𝐹 30/400 30

𝑃 𝐷𝐹 = = = 𝑃 𝐷` 𝐹 = 1 − 𝑃 𝐷 𝐹 = = = =
𝑃(𝐹) 40/400 40 40 𝑃 𝐹 40/400 40

𝑃(𝐷 ∩ 𝐹`) 18/400 18 342 𝑃 𝐷` ∩ 𝐹` 342/400 342

𝑃 𝐷 𝐹` = = = 𝑃 𝐷` 𝐹` = 1 − 𝑃 𝐷 𝐹` = = = =
𝑃(𝐹`) 360/400 360 360 𝑃 𝐹` 360/400 360
 68
Conditional Probability, Independence, and the Product Rule

+ =1

+ =1 + =1

+ + + =1
342/400 + 18/400 + 30/400 + 10/400 = 1
Question: Is D & F dependent events?

(Hint: If P(D/F)=P(D) or P(F/D)=P(F) or P(D∩F)=P(D)*P(F) then they are independent)

 69
Conditional Probability, Independence, and the Product Rule
Suppose that we have a fuse box containing 20 fuses, of which 5 are defective. If 2 fuses are selected
at random and removed from the box what is the probability that both fuses are defective? What is the
probability that the second one is defective given that the first one is defective?

a. If the selection is without replacement.

b. If the selection is with replacement.

Solution:

A: the event that the first selected part is defective.

B: the event that the second selective part is defective.

Without replacement (dependent events): With replacement (independent events):

5 4 1 5 5 1
𝑃 𝐴∩𝐵 = = 𝑃 𝐴∩𝐵 = =
20 19 19 20 20 16

4 5
𝑃 𝐵𝐴 = 𝑃 𝐵𝐴 =
19 20
 70
Conditional Probability, Independence, and the Product Rule

One bag contains 4 white balls and 3 black balls, and a second bag contains 3
white balls and 5 black balls. One ball is drawn from the first bag and placed
unseen in the second bag.
What is the probability that a ball now drawn from the second bag is black?

Let B1, B2, and W1 represent, respectively, the drawing of a black ball from
bag 1, a black ball from bag 2, and a white ball from bag 1.

We are interested in the union of the mutually exclusive events B1 ∩ B2 and

W1 ∩ B2
 71
Conditional Probability, Independence, and the Product Rule
The various possibilities and their probabilities are:

4W, 3B

B W
𝑜𝑟

3W, 6B 4W, 5B

3W, 5B
𝑃 (𝐵1 ∩ 𝐵2 𝑜𝑟 (𝑊1 ∩ 𝐵2 )] = 𝑃((𝐵1 ∩ 𝐵2 ) + 𝑃(𝑊1 ∩ 𝐵2 )
= P 𝐵1 𝑃 𝐵2 𝐵1 + P 𝑊1 𝑃 𝐵2 𝑊1
3 6 4 5
= +
7 9 7 9
 72
Conditional Probability, Independence, and the Product Rule
 73
Conditional Probability, Independence, and the Product Rule
 74
Conditional Probability, Independence, and the Product Rule

Remember DeMorgan’s law

 75
Conditional Probability, Independence, and the Product Rule
 76
Exercises

When B ⊂ 𝐴 𝐴⋂𝐵 = 𝐵 𝐴⋃𝐵 = 𝐴

Remember DeMorgan’s law

 77
Homework
 78
Total Probability Rule
suppose that in semiconductor manufacturing, the probability is √ √
0.2 that a chip subjected to high levels of contamination during √ √
𝑷 𝑭 𝑯 𝑷 𝑭 𝑯`
manufacturing and 0.8 is the probability that a chip is subjected
to not high level of contamination. The contamination causes the
chip failure. The probability that a chip fails given that it is not
subjected to a high level of contamination equals 0.005 while the ?
probability of a chip failure given that it is subjected to a high Total Prob.
Rule
level of contamination equals 0.01. What is the probability that a
chip fails?
H: a chip is subjected to a high level of contamination. F = 𝐹 ∩ 𝐻 ∪ 𝐹 ∩ 𝐻`
F: a chip fails. P F = P 𝐹 ∩ 𝐻 + 𝑃 𝐹 ∩ 𝐻`
= 𝑃 𝐹 𝐻 𝑃 𝐻 + 𝑃 𝐹 𝐻` 𝑃 𝐻`
Total probability rule (two events)
𝑃 𝐻 = 0.2 𝑃 𝐻` = 0.8 𝑃 𝐹 𝐻` = 0.005 𝑃 𝐹 𝐻 = 0.1

P F = P 𝐹 ∩ 𝐻 + 𝑃 𝐹 ∩ 𝐻` = 𝑃 𝐹 𝐻 𝑃 𝐻 + 𝑃 𝐹 𝐻` 𝑃 𝐻`
=0.1*0.2+0.005*0.8=0.024
 79
Total Probability Theorem

Proof!

A = 𝐴 ∩ 𝐵1 ∪ 𝐴 ∩ 𝐵2 ∪ 𝐴 ∩ 𝐵3 ∪ … 𝐴 ∩ 𝐵𝑘
𝑷 𝑨 𝑩𝟏 𝑷 𝑨 𝑩𝟐 𝑷 𝑨 𝑩𝒌
P(A)=P 𝐴 ∩ 𝐵1 + P 𝐴 ∩ 𝐵2 + P 𝐴 ∩ 𝐵3 + … P 𝐴 ∩ 𝐵𝑘 = σ𝑘𝑖=1 P 𝐴 ∩ 𝐵𝑖

P(A)=P(𝐵1 )P 𝐴 𝐵1 + P(𝐵2 )P 𝐴 𝐵2 + P(𝐵3 )P 𝐴 𝐵3 +… P(𝐵𝑘 )P 𝐴 𝐵𝑘

=σ𝑘𝑖=1 P(𝐵𝑖 )P 𝐴 𝐵𝑖

We call a partition: collectively exhaustive (their union is S) and ?

mutually exclusive (their intersection is Φ) events Total Prob.
Rule
 80
Total Probability Theorem

A: the product is defective. P(A)=?

B1: the product is made by machine B1. P(B1)=0.3
B2: the product is made by machine B2. P(B1)=0.45
B3: the product is made by machine B3. P(B1)=0.25
P(A/B1)= 0.02 P(A/B2)=0.03 P(A/B3)=0.02

P(A) = P(A∩B1) + P(A∩B2) + P(A∩B3)

P(A) = P(B1)P(A/B1) + P(B2)P(A/B2) +
?
P(B3)P(A/B3)
Total Prob.
P(A) = 0.3*0.02 + 0.45*0.03 + 0.25*0.02 =0.0245 Rule
 81

Bayes’ Rule

𝑷 𝑨 𝑩𝟏 𝑷 𝑨 𝑩𝟐 𝑷 𝑨 𝑩𝒌

𝑷 𝑩𝟏 𝑨 𝑷 𝑩𝟐 𝑨 ? Bayes’ Rule 𝑷 𝑩𝒌 𝑨

Total probability rule

P(A)=
 82

Bayes’ Rule

? Bayes’ Rule
 83

Bayes’ Rule
 84

Bayes’ Rule

?
Total Prob.
Rule
? Bayes’ Rule
 85

Bayes’ Rule

a. What is the probability that a product using one of these chips

fails?

b. If an item is selected randomly and found to be failed, what is the

probability that it is the probability that it is not subjected to high
level of contamination?
 86
Exercises
 87
General Questions for Chapter 2
 88
Course Structure