Chapter 5: Discrete Probability

• Historic
• Introduction (5.1)
• Discrete Theory (5.2)
• Expected Value &
Variance (5.3)

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Sixth Edition, Mc Graw-Hill, 2011
2
Historic
• The theory of probability was first developed
more than three hundred years ago by Blaise
Pascal for gambling purposes

• Later, the French mathematician Laplace, who

also was interested in gambling defined the
probability of an event

• The probability of an event led to the

development of much of basic probability theory

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
3
Historic (cont.)

• Probability theory plays an important role in

genetics where the goal is to help understand
the inheritance of traits

• In computer science, it is used in complexity

theory (average complexity of algorithms,
expert systems for medical diagnosis, etc)

• Unlike in deterministic algorithms, in

probabilistic algorithms, the output of a program
may change given the same input (random
choices are taken!)
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
4 Introduction to
Discrete Probability (5.1)
• Introduction

– In the eighteenth century, Laplace defined

the probability of an event as the number of
successful outcomes divided by the number
of possible outcomes

– The probability of obtaining an odd number

when rolling a dice is equal to 3/6 = ½
(assume die is fair!)
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
5 Introduction to Discrete Probability (5.1) (cont.)

• Finite probability

– Experiment is a procedure that yields one of

the given set of possible outcomes

– Sample space of the experiment is the set of

possible outcomes

– An event is a subset of the sample space

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
6 Introduction to Discrete Probability (5.1) (cont.)

– Definition 1:

the probability of an event E, which is a

subset of a finite sample space S of equally
likely outcomes, is
E
p( E )  .
S

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
7 Introduction to Discrete Probability (5.1) (cont.)

– Example: An urn contains 4 blue balls and 5

red balls. What is the probability that a ball
chosen from the urn is blue?

Solution: to calculate the probability, note

that there are 9 possible outcomes and 4 of
these possible outcomes produce a blue ball.
Hence, the probability that a blue ball is
chosen is 4/9.

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
8 Introduction to Discrete Probability (5.1) (cont.)

– Example: Find the probability that a hand of 5 cards

in poker contains 4 cards of one kind.

Solution: By the product rule, the number of hands of

5 cards with 4 cards of one kind is the product of the
number of ways to pick one kind, the number of
ways to pick the 4 of this kind out of the 4 in the deck
of this kind, and the number of ways to pick the 5th
card. This is C(13, 1) C(4, 4) C(48, 1).
Since there is a total of C(52, 5) different hands of 5
cards, the probability that a hand contains 4 cards of
one kind is
C ( 13 ,1 )C ( 4 ,4 )C ( 48 ,1 ) 13 * 1 * 48
  0.00024.
C ( 52 ,5 ) 2 ,598 ,960
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
9 Introduction to Discrete Probability (5.1) (cont.)

• Probability of combinations of events

– Theorem 1:

Let E be an event in a sample space S. The

probability of the event E , the
complementary event of E, is given by
p( E )  1  p( E ).

Proof: To find the probability of the event E,

note that E = |S| - |E|. Hence,
|S||E| |E|
p( E )   1  1  p( E )
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
|S| |S|
10 Introduction to Discrete Probability (5.1) (cont.)

– Example: A sequence of 10 bits is randomly

generated. What is the probability that at
least one of these bits is 0?

Solution: Let E be the event that at least one

of the 10 bits is 0. Then E is the event that
all the bits are 1s. Since the sample space S
is the set of all bit strings of length 10. It
follows that
|E| 1 1 1023
p( E )  1  p( E )  1   1  10  1   .
|S| 2 1024 1024
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
11 Introduction to Discrete Probability (5.1) (cont.)

– Theorem 2:

Let E1 and E2 be events in the sample space S.

Then
p( E 1  E 2 )  p( E 1 )  p( E 2 )  p( E 1  E 2 ).

Proof: Using the formula for the number of elements in

the union of two sets, it follows that
E1  E 2  E1  E 2  E1  E 2 .
Hence,
E1  E 2 E1  E 2  E1  E 2
p( E 1  E 2 )  
S S
E1 E 2 E1  E 2
  Discrete
© by Kenneth H. Rosen,  Mathematics & its
 Applications,
p( E 1 ) Seventh 2 ) Mc
p( EEdition, E1  E
p(Graw-Hill, 2 ).
2011
S S S
12 Introduction to Discrete Probability (5.1) (cont.)

– Example: What is the probability that a

positive integer selected at random from the
set of positive integers not exceeding 100 is
divisible by either 2 or 5?
Solution:
E1 = event that integer selected is divisible by 2
E2 = event that integer is divisible by 5
E1  E2 = event that integer divisible by either 2 or 5
E1  E2 = event that integer divisible by both 2 & 5,
or equivalently, that is divisible by 10
Since |E1| = 50, |E2| = 20 and | E1  E2| = 10,
p( E 1  E 2 )  p( E 1 )  p( E 2 )  p( E 1  E 2 )
50 20 10 3
    .
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
100 100 100 5
13
Probability Theory (5.2)
• Assigning probabilities
– Let S be the sample space of an experiment with a
finite or countable number of outcomes.
p(s) is the probability assigned to each outcome s
a) 0  p(s)  1 s  S
and
b)  p( s )  1
sS
There are n possible outcomes, x1, x2, …, xn, the
two conditions to be checked are:
a) 0  p(xi)  1 i = 1, 2, …, n
and
b) i  n The function p from the set of all
 p x i   1 events of the sample S is called
 1Kenneth H. Rosen, Discrete Mathematics
©iby probability distribution
& its Applications, Seventh Edition, Mc Graw-Hill, 2011
14
Probability Theory (5.2) (cont.)

– Example: What probabilities should we

assign to the outcomes H(heads) and T(tails)
when a fair coin is flipped? What probabilities
should be assigned to these events when the
coin is biased so that heads comes up twice
as often as tails?
Solution:
• unbiased coin: p(H) = p(T) = ½
• biased coin since:
 p( H )  2 p( T ) 1 2
  p( T )  and p(H)  .
 p( H )  p( T )  1 3 3
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
15
Probability Theory (5.2) (cont.)

– Definition 2:

The probability of the event E is the sum of

the probabilities of the outcome in E. That is,
p( E )   p( s )
sE

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
16
Probability Theory (5.2) (cont.)
– Example:
Suppose that a die is biased so that 3
appears twice as often as each other number
but that the other 5 outcomes are equally
likely. What is the probability that an odd
number appears when we roll this die?
Solution: We want to find the probability of
the event E = {1, 3, 5}
p(1) = p(2) = p(4) = p(5) = p(6) = 1/ 7
p(3) = 2/ 7
It follows that:
p(E) = p(1) + p(3) + p(5) = 1/ 7 + 2/ 7 + 1/ 7
= 4/ 7
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
17
Probability Theory (5.2) (cont.)

• Conditional probability

– Definition 3:

Let E and F be events with p(F) > 0. The

conditional probability of E given F, denoted
p(E|F), is defined as
p( E  F )
p( E | F )  .
p( F )

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
18
Probability Theory (5.2) (cont.)

– Example: A bit string of length 4 is generated at

random so that each of the 16 bit strings of length 4 is
equally likely. What is the probability that it contains at
least 2 consecutive 0s, given that its first bit is a 0?
(We assume that 0 bits and 1 bits are equally likely)

Solution:
E = event that a bit string of length 4 contains at least 2
consecutive 0s.
F = event that the first bit of a bit string of length 4 is a 0.
The probability that a bit string of length 4 has at least 2
consecutive 0s, given that its first bit is equal 0, equals
p( E  F )
p( E | F )  .
© by Kenneth H. Rosen, Discrete Mathematics & itspApplications,
( F ) Seventh Edition, Mc Graw-Hill, 2011
19
Probability Theory (5.2) (cont.)

Solution (cont.):

Since E  F = {0000, 0001, 0010, 0011, 0100}, we

see that p(E  F) = 5/16. Since there are 8 bit
strings of length 4 that start with a 0, we have
p(F) = 8/16 = ½. Consequently,
5 / 16 5
p( E | F )   .
1/ 2 8

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
20
Probability Theory (5.2) (cont.)
• Random variables
– They are numerical values associated with
the outcome of an experiment
– Definition 5:
A random variable is a function from the
sample space of an experiment to the set of
real numbers. That is, a random variable
assigns a real number to each possible
outcome.
Remark: Note that a random variable is a function. It
is not a variable, and it is not random!
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
21
Probability Theory (5.2) (cont.)

– Example: Suppose that a coin is flipped 3

times. Let X(t) be the random variable that
equals the number of heads that appear
when t is the outcome. Then X(t) takes the
following values:

X(HHH) = 3,
X(HHT) = X(HTH) = X(THH) =2,
X(TTH) = X(THT) = X(HTT) = 1,
X(TTT) = 0.
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
22
Expected Value & Variance (5.3)
• Introduction

– The expected value is very useful in

computing the average-case complexity of
algorithms

– Another useful measure of a random variable

is its variance

– The variance tells us how spread out the

values of this random variable are
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
23
Expected Value & Variance (5.3) (cont.)

• Expected values

– Definition 1:

The expected value (or expectation) of the

random variable X(s) on the sample space S
is equal to E ( X )  p( s ) X ( s ). 
sS

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
24
Expected Value & Variance (5.3) (cont.)

– Example: Expected value of a die

Let X be the number that comes up when a
die is rolled. What is the expected value of
X?

Solution: The random variable X takes the

values 1, 2, 3, 4, 5, or 6, each with probability
1/6. It follows that
1 1 1 1 1 1 21 7
E( X )  * 1  * 2  * 3  * 4  * 5  * 6   .
6 6 6 6 6 6 6 2

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
25
Expected Value & Variance (5.3) (cont.)

– Theorem 1:

If X is a random variable and p(X =r) is the

probability that X = r, so that
p X  r    sS , X ( s ) r p( s )

then E ( X )   p X  r  r .
r X ( S )

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
26
Expected Value & Variance (5.3) (cont.)

– Example: What is the expected value of the

sum of the numbers that appear when a pair
of fair dice is rolled?

Solution:
Let X be the random variable equal to the
sum of the numbers that appear when a pair
of dice is rolled.
We have 36 outcomes for this experiment.
The range of X is {2, 3, 4, 5, 6, 7, 8, 9, 10,
11, 12}.
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
27
Expected Value & Variance (5.3) (cont.)
Solution (cont.):
p(X = 2) = p(X = 12) = 1/36,
p(X = 3) = p(X = 11) = 2/36 = 1/18,
p(X = 4) = p(X = 10) = 3/36 = 1/12,
p(X = 5) = p(X = 9) = 4/36 = 1/9,
p(X = 6) = p(X = 8) = 5/36,
p(X = 7) = 6/36 = 1/6.
1 1 1 1 5 1
 E( X )  2*  3* * 4 *  5 *  6 * 7 *
36 18 12 9 36 6
5 1 1 1 1
 8*  9 *  10 *  11*  12 *
36 9 12 18 36
 7.
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
28
Expected Value & Variance (5.3) (cont.)

• Linearity of expectations

– Theorem 3:

If Xi, i = 1,2 , …, n with n a positive integer,

are random variables on S, and if a and b
are real numbers, then
a) E(X1 + X2 + … + Xn)
= E(X1) + E(X2) + … + E(Xn)
b) E(aX + b) = aE(X) + b.

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
29
Expected Value & Variance (5.3) (cont.)

• Average-case computational complexity

– Computing the average-case computational
complexity of an algorithm can be interpreted
as computing the expected value of a
random variable
– Principle: Let the sample space of an
experiment be the set of possible inputs aj,
(j = 1, 2, .., n), and let X(aj) computes the
number of operations used by the algorithm
when aj is an input.

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
30
Expected Value & Variance (5.3) (cont.)

– Then, the average-case complexity of the algorithm

is:
n
E ( X )   p( X ( a j ))* ( X ( a j ))  exp ected value of X.
j 1

– The average-case computational complexity of an

algorithm is usually more difficult to determine than
its worst-case computational complexity

– Example: Average-case complexity of the linear

search algorithm (p.384)
Exercise to think about.
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
31
Expected Value & Variance (5.3) (cont.)
• Variance
– The expected value of a random variable provides
the average value, but the variance tells us how
widely its values are distributed

– Definition 4:

Let X be a random variable on a sample space S.

The variance of X, denoted by V(X), is
V( X )   ( X ( s )  E (( X ))2 p( s ).
sS

The standard deviation of X, denoted (X), is defined

to be V ( X ) .
© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
32
Expected Value & Variance (5.3) (cont.)
– Theorem 6:

If X is a random variable on a sample S, then

V(X) = E(X2) – E(X)2.

Proof: Note that

V ( X )   ( X ( s )  E ( X ))2 p( s )
sS

  X ( s )2 p( s )  2 E ( X ) X ( s ) p( s )  E ( X )2  p( s )
sS sS sS

 E ( X 2 )  2 E ( X ) E ( X )  E ( X )2
 E ( X 2 )  E ( X )2

© by Kenneth H. Rosen, Discrete Mathematics & its Applications, Seventh Edition, Mc Graw-Hill, 2011
Dr. Djamel CSE 504 Discrete Structures & Foundations of
33
Expected Value & Variance (5.3) (cont.)

– Example: Variance of the value of a die

What is the variance of the random variable
X, where X is the number that comes up
when a die is rolled?

Solution: V(X) = E(X2) – E(X)2. We know that

E(X) = 7/2. To find E(X2) note that X2 takes
the values i2 = 1, 2, …, 6, each with
probability 1/6.
 E( X 2 ) 
1 2
6
 91
1  22  32  4 2  5 2  6 2  .
6

2
91  7  35
Conclusion: V ( X )      .
6  2
© by Kenneth H. Rosen, Discrete Mathematics 12
& its Applications, Seventh Edition, Mc Graw-Hill, 2011