Chapter 4: Elementary probability

Definition:

• Probability is the chance of occurring of an outcome of an experiment

• It is the measure of how likely an outcome of an experiment is to occur

Definition of some probability terms

• Experiment (E) :is any statistical process that can be repeated several times and in any trial
of which the outcome is unpredictable

• It is also called random experiment or probability experiment

Example: If a fair die is rolled once it is possible to list all the possible outcomes

i.e.1, 2, 3, 4, 5, 6 but it is not possible to predict which outcome will occur

1
 …cont
Example of experiment and the possible outcome

⊲ Tossing a coin only once, S = {Head (H), Tail (T)}

⊲ Tossing a coin two times, S = {HH,HT, TH, TT}

⊲ Rolling a die, S = {1, 2, 3, 4, 5, 6}

⊲ Selecting an item from a production lot, S = {Defective, Non-defective}

⊲ Introducing a new product, S = {Success, Failure}

2
 Con…
Sample Space (S): is a set consisting all possible outcomes of a given experiment

- If a probability experiment has 𝑀 possible outcomes and done 𝑛 times, then the sample
space (S) has 𝑀𝑛 total elements.

Example: What is the sample space for the following experiment

A. Rolling of a die one time.

B. Toss a coin two times

Answer: A) S = {1, 2, 3, 4, 5, 6} = 61 B) S = (HH, HT, TH, TT) = 22

• Event: It is a subset of sample space or

- It is a statement about one or more outcomes of a random experiment 3

 Con..
• They are denoted by capital letters

Example: Considering the above experiment, let A be the event of odd numbers, B be the
event of even numbers, and C be the event of number 8. then list the elements of A, B and C

Answer: A = {1, 3, 5} B = {2, 4, 6} C = {8}

• Note: If sample space (S) has 𝑛 members, then there are exactly 2n subsets or events.
Equally Likely Events: Events which have the same chance of occurring.

• Complement of an Event: the complement of an event A means non- occurrence of A.

- It is denoted by 𝐴′ contains those points of the sample space which don’t belong to A

Example: from the above example A is the complement of B and viscera 4

 Con…
• Elementary Event: an event having only a single element or sample point.

Example: Let A is the event of getting a number 2 from rolling a die once. A = {2}

• Mutually Exclusive Events: two or more events are mutually exclusive, if they have no
outcome in common (i.e. they cannot occur together simultaneously)

Example: Let A is the event of odd numbers and B is the event of even numbers in rolling of a
die once. The we have

A = {1, 3, 5} B = {2, 4, 6}

- Here, A and B are mutually exclusive events since A∩B = ø

5
 Con…
• Independent Events: Two events are independent if the occurrence of one does not affect
the probability of the other occurring.

• Dependent Events: Two events are dependent if the first event affects the outcome or
occurrence of the second event in a way the probability is changed.

6
 Con…
Counting Rules
• In order to calculate probabilities, we have to know
- The number of elements of an event
- The number of elements of the sample space
• In order to determine the number of outcomes, one can use
• The addition rule
• The multiplication rule
• Permutation rule
• Combination rule
• To list the outcomes of the sequence of events, we can use tree diagram

7
 Addition Rule
• Addition Rule: states that if a task can be done (accomplished) by any of the k procedures,

• where 𝑖𝑡ℎ procedures has 𝑛𝑖 alternatives, then the total number of ways of doing the task
is
𝑘
𝑛1 + 𝑛2 + 𝑛3 … + 𝑛𝑘 = 𝑖=1 𝑛𝑖

Example: A student goes to the nearest Cafe to have a breakfast. He can take tea, coffee, or
milk with bread, cake and sand witch. How many possibilities does he have?

Solution: Using tree diagram

The student has

3*3 = 9 possibilities
8
 Con…
Multiplication rule
• If a choice consists of k steps of which the first can be made in n1 ways, the
second can be made in n2 ways…, the kth can be made in nk ways, then the
whole choice can be made in (n1* n2*……….* nk) ways

Example: The digits 0, 1, 2, 3, and 4 are to be used in 4 digit identification card.

How many different cards are possible if
A. Repetitions are permitted
B. Repetitions are not permitted

9
 Con…
Answer: A)
1𝑠𝑡 digit 2𝑛𝑑 digit 3𝑟𝑑 digit 4𝑡ℎ digit
5 5 5 5

There are four steps

Step 1: Selecting the first digit, this can be made in 5 ways

Step 2: Selecting the second digit, this can be done in 5 ways

Step 3: Selecting the third digit, this can be made in 5 ways

Step 4: Selecting the fourth digit, it can be made in 5 ways

 5*5*5*5 = 625 different cards can be made

10
 Con…
B)
1𝑠𝑡 digit 2𝑛𝑑 digit 3𝑟𝑑 digit 4𝑡ℎ digit
5 4 3 2

Step 1: Selecting the first digit, this can be made in 5 ways

Step 2: Selecting the second digit, this can be done in 4 ways

Step 3: Selecting the third digit, this can be made in 3 ways

Step 4: Selecting the fourth digit, it can be made in 2 ways

 5*4*3*2 = 120 different cards can be made

11
 Con…
Permutation

• An arrangement of 𝑛 objects in a specified order is called permutation of the objects

Permutation Rules:

1. The number of permutations of n distinct objects taken all together is 𝑛!

Where 𝑛! = 𝑛*(𝑛 − 1)!*(𝑛 − 2)! *…..* 3*2*1

2. The arrangement of 𝑛 objects in a specified order using 𝑟 objects at a time is called the
permutation of 𝑛 objects taken 𝑟 objects at a time.

𝑛!
- It is written as P(n, r) =
𝑛−𝑟 !
12
 Con…
3. The number of permutations of 𝑛 objects in which k1 are alike k2 are alike ---- etc is
𝑛!
P (n, r) =
𝑘1 !∗𝑘2 !……..∗𝑘𝑛 !

Examples:

1. Suppose we have a letters A,B, C, D

A. How many permutations are there taking all the four?

B. How many permutations are there two letters at a time?

2. How many different permutations can be made from the letters in the word “CORRECTION

13
 Con..
Answer: 1)

A) 𝑛 = 4  4! = 4*3*2*1 = 24 different permutations

B) 𝑛 = 4, 𝑟 = 2
4! 4∗3!
 P (𝑛, 𝑟) = P (4, 2) = = = 4 different permutations
4−1 ! 3!

2) Here 𝑛 =10, 2 are C, 2 are O, 2 are R, 1E, 1T, 1I 1N

𝐾1 = 2, 𝐾2 = 2, 𝐾3 = 2, 𝐾4 = 1, 𝐾5 =1, 𝐾6 = 1, 𝐾7 = 1
10! 10∗9∗8∗7∗6∗5∗4∗3∗2!
 = = 10*9*7*6*5*4*3*2
2!∗2!∗2!∗1!∗1!∗1!∗1! 2!∗2!∗2!∗1!∗1!∗1!∗1!

= 453600 permutations
14
 Con…
Combination
• A selection of objects with out regard to order is called combination.
Example: Given the letters A, B, C, and D list the permutation and
combination for selecting two letters.
Answer:
Permutation Combination
AB BA CA DA AB BC AC
AC BC CB DB BD AD DC
AD BD CD DC
Note: In permutation AB is different from BA. But in combination AB is the
same as BA.
15
 Con…
Combination rule
Rule 1:The number of combinations of 𝑟 objects selected from 𝑛 objects is
denoted by
𝑛 𝑛!
C 𝑟
=
𝑛−𝑟 !𝑟!
Example: In how many ways a committee of 5 people be chosen out of 9
people?
Answer: 𝑛 = 9, 𝑟 = 5
𝑛 9 9! 9! 9∗8∗7∗6∗5! 9∗8∗7∗6
C 𝑟
=C 5
= = = = = 9*2*7 = 126 ways
9−5 !5! 4!5! 4!∗5! 4∗3∗2∗1

• Rule 2: If the selection has k steps, by selecting r1 of n1 objects, r2 of n2, ..., rk

of nk objects, then the total number of ways of doing this selection is equal
16
to
 Con…

Example: Among 15 clocks there are two defectives .In how many ways can an inspector chose three
of the clocks for inspection so that:
A. There is no restriction
B. None of the defective clock is included
C. Only one of the defective clocks is included
D. Two of the defective clock is included

Answer

n = 15 clocks, defective clock =2, non-defective clocks = 13

A. If there is no restriction select three clocks from 15 clocks and this can be done in 17
 Con…
n = 15, r =3
𝑛 𝑛! 15 15! 15∗14∗13∗12! 15∗14∗13
𝑟
=  3
= = = = 2730/6 = 455
𝑛−𝑟 !𝑟! 15−3 !3! 12!3! 6

B. None of the defective clocks is included

• This is equivalent to zero defective and three non defective, which can be
done in

2 13 2! 13! 2! 13!
C 0
*C 3
= * = *
2−0! 0! 13−3 !3! 2!0! 10!3!

2∗1 13∗12∗11∗10! 1716

= * = 1* = 286 possible ways
2∗1 10!3∗2 6
18
 Con…
C. Only one of the defective clocks is included.
• This is equivalent to one defective and two non defective, which can be done
in:

2 13 2! 13! 2! 13!
C 1
*C 2
= * = *
2−1! 1! 13−2 !2! 1!1! 11!2!

13∗12∗11!
= 2*
11!2!

156
= 2*
2

= 156 possible ways

19
 Con…
D. Two of the defective clock is included.
• This is equivalent to two defective and one non defective, which can be done in:

2 13 2! 13! 2! 13!
C 2
*C 1
= * = *
2−2! 2! 13−1 !1! 0!2! 12!1!

13∗12!
= 1*
12!1!

= 1*13

= 13 possible ways

20
 Some Probability Rules/Axioms

• Let S be a sample space associated with a random experiment. Then with any event E, in
this sample space, we associate a real number called probability of E satisfying the following
properties (axioms).

21
…cont

22
…cont

23
…cont

24
 Random variables and probability distribution
 What is random variable?
• A random variable is a numerical description of the outcomes of the experiment.

- or it is a numerical valued function defined on sample space, usually denoted by capital

letters.

• A random variable takes a possible outcome and assigns a number to it.

Example: Flip a coin three times, let X be the number of heads in three tosses.

A. Find the sample space(S) B. Find the possible values of X

25
 Con…
Soln: A. S = {HHH, HHT, HTH, THH, HTT,, THT, TTH, TTT}

A. X(HHH) = 3, X(HHT) = X(HTH) = X(THH)=2, X(HTT) = X(THT) = X(TTH) =1, X(TTT) = 0

Then the possible values of X are

X = (0, 1, 2, 3)

What is probability distribution?

• Probability distribution is a distribution which consists of a value a random variable can

assume and the corresponding probabilities of the values

Example: Consider the experiment of tossing a coin three times. Let X be the number of
heads. Then construct the probability distribution of X. 26
 Con…
solution

1. First identify the possible value that X can assume.

2. Calculate the probability of each possible distinct value of X and express X in the form of
frequency distribution
X= x 0 1 2 3
P(X =x) 1/8 3/8 3/8 1/8

• Probability distribution is denoted by P for discrete and by f for continuous random

variable.

27
 Con…

Properties of Probability Distribution

- P(x) ≥ 0 if X is discrete random variable

- f(x) ≥ 0, if X is continuous variable

- 𝑥𝑃 𝑋 = 𝑥 = 1, if X is discrete random variable

- 𝑓 𝑥 𝑑𝑥 = 1 if X is continuous random variable

• If X is continuous random variable, then

𝑏
- P(a < X < b) = 𝑎
𝑓 𝑥 𝑑𝑥

28
 Con…

- P(a < X < b) = P(a ≤ X < b) = P(a < X ≤ b) = P(a ≤ X ≤ b)

• If X is discrete random variable then

29
 Con…

• Probability means area for continuous random variable

𝑏
- that means, P(a < X < b) = 𝑎
𝑓 𝑥 𝑑𝑥

Introduction to expectation

• Let a discrete random variable X assume the values X1, X2, ….,Xn with the probabilities P(X1),
P(X2), ….,P(Xn) respectively.

- Then the expected value of X ,denoted as E(X) is defined

𝑛
E(X) = X1 P(X1) + X2 P(X2) +…..+ Xn P(Xn) = 𝑖= Xi P(Xi)

30
 Con…
• Let X be a continuous random variable assuming the values in the interval (a, b) such that
𝑏
𝑎
𝑓 𝑥 𝑑𝑥 = 1 ,then

𝑏
𝐸 (X) = 𝑎
𝑥𝑓 𝑥 𝑑𝑥

Examples: What is the expected value of a random variable X obtained by tossing a coin
three times where X is the number of heads

Soln: First construct the probability distribution of X

X=x 0 1 2 3
P(X = x) 1/8 3/8 3/8 1/8
31
 Con…
 𝐸(X) = X1 P(X1) + X2 P(X2) + X3 P(X3) + X4 P(X4)
= 0*1/8+1*3/8+2*3/8+3*1/8

= 3/8+6/8+3/8

= 12/8

=1.5 /////
Mean and Variance of a random variable
• Let X be given random variable, then

• The expected value of X is its mean

 Mean of X = 𝐸(X)
32
 Con…
• The variance of X is given by

Variance of X = var(X) = E(𝑋 2 ) - [𝐸 𝑋 ]2

𝑛
Where, E(𝑋 2 ) = 𝑖=1 𝑥𝑖
2 P(X= 𝑥𝑖 ) if X is discrete random variable

E(𝑋 2 ) = 𝑥 2 𝑓 𝑥 𝑑𝑥 if X is continuous variable

Example 2: Suppose that X is a continuous random variable with pdf of

then find the mean value and variance of X.

33
…cont

34
 …cont
Exercise: 1. A coin is tossed two times. Let X be the number of heads. Find the mean value
and the standard deviation of X.

2. Suppose that X is a continuous random variable with pdf of

a. f(x) = 1 + x, −1 ≤ x < 3 b. f(x) = 2x,1 ≤ x < 3

Example: What is the variance of a random variable X obtained by tossing a coin three times
where X is the number of heads.

Soln: E (X) = 1.5 (from the above example)

𝑛
E(𝑋 2 ) = 𝑖=1 𝑥𝑖
2
P(X= 𝑥𝑖 ) = 02 *1/8+12 *3/8+22 *3/8+32 *1/8

35
 Con…
= 3/8+12/8+9/8 = 24/8 = 3////

Var(X) = E(𝑋 2 ) - [𝐸 𝑋 ]2

= 3 – (1.5)2

= 3 – 2.25

= 0.75////

36