ST2504 – INTRODUCTION TO PROBABILITY

THEORY
Test 1, Friday, 21st March 2024
• Venue: ETF 3
Important Discrete Distributions
• In this section we study the most common discrete random variables
and their probability distributions.
• These probability distributions are very important in the theory of
probability and statistics and are also of practical importance as many
real life phenomena can be modelled using one or more of these
probability distributions.
Common Discrete Probability Distributions
• Bernoulli Distribution
• Binomial Distribution
• The Negative Binomial
• The Geometric Distribution
• The Hyper Geometric
• The Poison Distribution
• Note: Each probability distribution above has a probability mass
function, the mean, the variance and the moment generating
function
Bernoulli Distribution
• Definition: A Bernoulli trial (experiment) is one that possesses the
following properties:
• (a) The trial results in one of the two mutually exclusive outcomes.
• (b) The probability of success on a single trial is 𝑝.
• (c) The r.v. of interest, X, is the success (occurrence) of the event 𝐴, or
failure (non-occurrence) of the event 𝐴, denoted by 𝐴,ҧ of an event.
Bernoulli Random Variable
• Let’s consider an experiment that can result in one of the two
possible outcomes, which we will refer to as success and failure.
• The probability of success is p, 0 < p <1, and the probability of failure
is 1 – p.
• Such an experiment is called a Bernoulli trial.
• The sample space for a Bernoulli trial is:
• S = {success, failure} or S = {s, f}
Example
• Toss a fair coin. Let “head” be a success and “tail” a failure. Then the
experiment is called a Bernoulli trial.

1, if a " head " occurs

X =
0, otherwise

• Consider the sex of a child to be born

1, if the child is a boy
X =
0, if the child is a girl
Cont.
• The probability mass function for X is
P ( X = 0 ) = P ( failure ) = 1 − p
P ( X = 1) = P ( success ) = p

The distribution may be represented with the probability mass function

P( X = x) = p q x 1− x
I0,1

Where 𝑞 = 1 − 𝑝
Cont.

• The indicator function 𝐼 0,1 indicates that X can take either 0 or 1 as

possible values.
• For all values of X outside 0,1 , 𝑃 𝑋 = 𝑥 = 0.
• 𝑃 𝑋 = 2 = 0 because 2  0,1
Cont.
• The Bernoulli distribution is a legitimate probability distribution.
• Proof:
P( X = x)  0,
1 1

 P( X = x) =  P( X = x) = P( X = 0) + P( X = 1)
x =0 x =0

= p + (1 − p)
=1
Theorem
• Suppose 𝑋 is a Bernoulli r.v. with parameter 𝑝. Then
• 1) E[X] = p
• 2) V[X] = pq = p(1-p)
• 3) MX(t) = pet + q

• Proof the above inequalities.

Example
• A fair die is tossed once.
1, 𝑖𝑓 𝑎 3 𝑜𝑐𝑐𝑢𝑟𝑠
• Let 𝑋 = ቊ
0, 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

• Find 𝐸 𝑋 𝑎𝑛𝑑 𝑉𝑎𝑟(𝑋)

Bernoulli Distribution _ Summary
P ( X = x ) = p x q1− x I0,1

E[X] = p
V[X] = pq = p(1-p)
MX(t) = pet + q

Note that we have:

- A probability mass function
- The mean, the variance and the moment generating
function
Uses of Moment Generating Functions
• We can use the moment generating functions to find the first and
second moments. E.g.

E ( X ) = M (t = 0) = M X (t)
'

t
X

 2
E ( X 2 ) = M X'' (t = 0) = M X (t)
t
• These are evaluated at 𝑡 = 0.
Exercise
• Let 𝑋 be a Bernoulli probability mass function with
M X (t ) = pe + q
t

derive the mean and variance of 𝑋.

Solution
M X (t ) t =0 = ( pet + q )
d ' d
dt dt
= pet
t =0

=p

M X (t ) = ( pet )
d '' d
dt t =0 dt
= pet
t =0

=p

Now V ( X ) = E ( X 2 ) − [ E ( X )]2
= p − p2
= p (1 − p )
Binomial Experiment
• An experiment that consists of 𝑛 independent Bernoulli trials each
with probability of success p is called a binomial experiment or trial.
• The binomial trial has the following features:
• 1. There are 𝑛 independent Bernoulli trials,
• 2. 𝑛 is a fixed positive integer,
• 3. The probability of success on a single trial is p and remains the
same from trial to trial. The probability of failure is q = 1 – p
• 4. The trials are independent
Cont.
• Let the random variable X be the total number of successes in a
binomial experiment. Then X is called a binomial r.v. with parameters
𝑛 𝑎𝑛𝑑 𝑝, and we write 𝑋~𝐵𝑖𝑛𝑜𝑚(𝑛, 𝑝) in short.

• The parameter of a probability distribution is a value that

characterizes the probability distribution.
• The Bernoulli distribution has one parameter, 𝑝 and the binomial
distribution has two parameters, 𝑛 𝑎𝑛𝑑 𝑝.
Cont.
• The probability distribution for a Binomial rv is:
n k
P ( X = k ) = PX ( k ) =   p (1 − p ) , k = 0,1,2,..., n
n−k

k 
• Where 𝑃𝑋 𝑘 = 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑘 𝑠𝑢𝑐𝑐𝑒𝑠𝑠
• 𝑛 = 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑖𝑧𝑒/𝑡𝑜𝑡𝑎𝑙 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑟𝑖𝑎𝑙𝑠
• 𝑝 = 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑠𝑢𝑐𝑐𝑒𝑠𝑠
• 1 − 𝑝 = 𝑞 = 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑓𝑎𝑖𝑙𝑢𝑟𝑒
• 𝑘 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑒𝑠
𝑛
• = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑤𝑎𝑦𝑠 𝑡𝑜 𝑜𝑏𝑡𝑎𝑖𝑛 𝑘 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑒𝑠 𝑖𝑛 𝑛 𝑡𝑟𝑖𝑎𝑙𝑠
𝑘
Cont.
• The Binomial distribution is called so because the probabilities are the
terms in the binomial expansion of 𝑝 + 𝑞 𝑛
• Binomial Theorem:
n
 n  x n− x
( a + b) =    p q
n

x =0  x 
Cont.
• The binomial distribution is a legitimate probability distribution. That
is,
• (a) P (k )  0
X
n
• (b)  P ( k ) = 1,
k =0
X

• Try it!
Proof

n n
n x
 P( X = x) =    p (1 − q )
n− x

x =o x =0  x 

=  p + (1 − p )  by binomial theorem
n

=  p + 1 − p = 1
n
Cont.
• Given the probability distribution of a binomial distribution, evaluate
the following probabilities when 𝑛 = 5 𝑎𝑛𝑑 𝑝 = 0.3.
• a)𝑃 𝑋 ≥ 4 ,
• 𝑏) 𝑃 0 ≤ 𝑋 < 3 ,
• 𝑐) 𝑃(𝑋 ≤ 4)
Cont.
• 𝑎) 0.0308
• 𝑏) 0.8369
• 𝑐) 0.9976
Example
• Let the r.v. X be the total number of heads when a fair coin is tossed 4
times.
• Is this a Bernoulli or binomial trial/experiment?
• Are the conditions of a Binomial trial satisfied?
• Find the probability mass function for X.
Solution
• 1. There are 𝑛 independent Bernoulli trials,
• 2. 𝑛 is a fixed positive integer,
• 3. The probability of success on a single trial is p and remains the
same from trial to trial. The probability of failure is q = 1 – p
• 4. The trials are independent
Example
• We can see that the 4 conditions for a binomial experiment are
satisfied. Here p = 0.5, n = 4. Now, the probability mass function of X
is
 4  k ( 4− k )
P( X = k ) =   0.5 0.5 , k = 0,1,2,3,4
k 
• The probability mass function of X can be represented as
PMF

k 0 1 2 3 4 Total
PX(k) 0.0625 0.25 0.375 0.25 0.0625 1
Graph
• The graph for PX(k) is
Cont.
• The pmf is symmetrical about x = 2.
Example
• 1. The probability that the patient recovers from a rare blood disease is 0.4.
If 10 people are known to have contracted the disease, what is the
probability that
• a) exactly 3 patients survive the disease?
• b) from 3 to 5 patients survive the disease?
• 2. Suppose that a student is given a test with 10 true-false questions. Let X
be the number of questions guessed correctly by the student.
• i) Write down the probability distribution (pmf) of X
• ii) What is the probability that the student guesses all questions correctly?
• Iii) What is the probability that the student guesses 7 or more questions
correctly?
Solution 1
• A Binomial experiment with n = 10 trials and p = 0.4
• Now,
10 
p[ X = 3] =   ( 0.4 ) ( 0.6 )
3 7

3 

P[3  X  5] = P[ X = 3] + P[ X = 4] + P[ X = 5]
10  10  10 
=   ( 0.4 ) ( 0.6 ) +   ( 0.4 ) ( 0.6 ) +   ( 0.4 ) ( 0.6 )
3 7 4 6 5 5

3  4  5 
Solution 2
• Suppose that the student guesses completely
1, for a correct guess
xi = 
0, Otherwise
i = 0,1,2,3,4,5,6,7,8,9,10

X ~ Binim(np, npq)
• i)  n  x n− x
P  X = x =   p q I x

 x {0,1,2,..., n }
 Cont.
• Let X be a random variable with a binomial distribution. Then

E ( X ) = np
V ( X ) = npq
M X (t ) = ( pe + q )
t n

• Using the direct method, proof the above theorems.

Binomial Theorem
n
 n  x n− x
(a + b) =   a b
n

x =0  x 

Example
𝐿𝑒𝑡 𝑎 = 2, 𝑏 = 3, 𝑛 = 4
Cont.
4
 4  x 4− x
( 2 + 3) =    2 3
4

x =0  x 

 4 0 4  4 1 3  4 2 2  4 3 1  4 4 0
( 5) =   2 3 +   2 3 +   2 3 +   2 3 +   2 3
4

0 1   2 3  4
625 = 81 + 216 + 216 + 96 + 16
625 = 625
Derive the moment generating function
 n  x n− x
P  X = x =   p q I x

 x {0,1,2,..., n }

E ( X ) = np n
 n  x n− x
(a + b) =   a b
n

V ( X ) = npq x =0  x 

M X (t ) = ( pe + q )
t n
Exercise
Nine percent of students in ST2504 have their bank balances greater
than M500. Suppose that 10 students are selected at random to be
interviewed about their bank balances
i) What is the probability that two students will have their bank
balances greater than M500?
ii) What is the probability that none will have a bank balance greater
than M500?
Solution
• Let n=10, p=0.09, q=0.91, P(X=2)=0.1714, P(X=0)=0.3894
Poison Distribution
• Several useful applications of probability theory deal with time
intervals and the number of events that occur within the given time
intervals.
• - number of phone calls received in an hour
• - number of insurance claims per year
• - occurrence of accidents in a factory in a week
• - arrival of patients to an emergency wards
• - bacterial growth in a certain time interval
• If we let X be the rv representing the number of occurrences in each
of the cases mentioned, then X is a discrete rv.
Cont.
• The time instant at which we begin observing the phenomenon is
labelled 0.
• Let t be a fixed time period of time of length t.
• [0,t] is the fixed time interval
• X is the number of events that occur in the interval [0,t].
Properties of a Poison Experiment
• 1. The probability of an occurrence is the same for any two intervals
of equal length.
• 2. The occurrence or non-occurrence in any interval is independent of
the occurrence or non-occurrence in any other interval.
• 3. Two or more simultaneous events are impossible
Definition
• Let X be a rv with a Poison distribution. The probability distribution
function of X is given by:
e−  x
P[ X = x] = PX ( x) = ( x)
I0,1,...
x!
 0
• 𝑃[𝑋 = 𝑥] = 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦 𝑜𝑓 𝑥 𝑜𝑐𝑐𝑢𝑟𝑟𝑒𝑛𝑐𝑒𝑠 𝑖𝑛 𝑎𝑛 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙
• = 𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑 𝑣𝑎𝑙𝑢𝑒/ 𝑚𝑒𝑎𝑛 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑜𝑐𝑢𝑟𝑟𝑒𝑛𝑐𝑒𝑠 𝑖𝑛 𝑎𝑛 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙
Cont.
• It is clear that if λ>0, then P[X=x)>o for all x. To verify that P[X=x]
satisfies the second condition for a probability function, note that a
Taylor’s series expansion of a function f(x) about a point a is
f '(a )( x − a ) f "(a )( x − a ) 2 f "'(a )( x − 3)3
f ( x ) = f (a ) + + + + ...
1! 2! 3!

• Now suppose that f ( x) = e x and consider the expansion about 0, i.e,

a = 0, we have
Cont.
e0 x 2 e0 x 3 e0 x 4
e =e +e x+
x 0 0
+ + + ...
2! 3! 4!
x 2 x3 x 4
= 1 + x + + + + ...
2! 3! 4!

• Which can be written in short as

 k
x
ex = 
k =0 k!
Cont.
• Proof that

e−  x
P[ X = x] = PX ( x) = ( x)
I0,1,...
x!
 0

• Is a true pdf.
Cont.
• Therefore,
−
 
e  x  x


x =0
PX ( x) = 
x =0 x!
= e  =e e = 1
−

x =0 x!
− 
Exercise
• Given a poison pmf, find the following:
•
E(X )
Var ( X )
M X (t )
Cont.

 E  X ( X − 1)  =  2

E  X − X  = 
2 2

E( X ) − E( X ) = 
2 2

E( X ) =  + 
2 2
Cont.

Var ( X ) = E ( X ) − [ E ( X )]
2 2

=  + −
2 2

=
Theorem
• Let X be a rv with a Poison distribution. Then
−
e  x
1. P[ X = x] = ( x)
I0,1,...
x!
2. E ( X ) = 
3. Var ( X ) = 
 ( et −1)
4. M x (t ) = e
Home Work
• Proof the theorems using the indirect method
Cont.
• Remark
• The parameter of a rv with a poison distribution is λ.
• λ = (rate of occurrence) (duration)
Exercise
• Suppose that the occurrence of accidents in a large industrial plant
satisfies the three assumptions for the Poison process. Suppose also
that accidents occur at the rate of ½ per week. Let X be the number of
accidents that will occur in a period of the next six weeks.
• A) What is the parameter λ?
• B) Write down the pdf of X.
• C) Find E(X), Var(X) and mgf of X
• D) What is the probability that exactly 3 accidents will occur?
• E) What is the probability that no accidents will occur?
Solution
• A) Rate of accidents = ½ per week
• Duration = 6 weeks
• λ = (rate)(duration)= 0.5 * 6 = 3
− x
• B) P[ X = x] = e
I ( x)
0,1,...
x!
e −3 3x
=
x!
Example
• If the number of accidents occurring on a highway each day is a
Poison rv with parameter λ = 3, what is the probability that no
accidents occur today?
Poison Approximation to Binomial
Distribution
• The binomial distribution can be approximated by a Poison
distribution when the parameter n is large and p is small (typically
when n ≥ 20 and p ≤ 0.05). It can be proved that under those
conditions:
n x n− x e − np
( np ) x

 x p (1 − p ) 
  x!
e−  x
= , for  = np
x!
Cont.
• The advantage is that the Poison probabilities are much simpler to
compute.
Cont.
• Suppose that the probability that a corn seed from a certain batch
does not germinate equals 0.02. If we plant 200 of these seeds, what
is the probability that
• A) At most 5 seeds will not germinate?
• B) Exactly 3 will not germinate?
• C) At least 3 will not germinate?
Example 2
• If the probability that an individual suffers an adverse
reaction from a particular drug is known to be 0.001,
determine the probability that out of 2000 individuals, (a)
exactly three and (b) more than two individuals will suffer
an adverse reaction.
Geometric Distribution
• Suppose that a sequence of independent Bernoulli trials are
performed.
• Also assume that the probability of success 𝑝 remains the same
throughout.
• Let the rv X be the number of trials needed to get the first success.
• Then X is called a Geometric Random Variable.
Cont.
• Geometric distribution is the number of trials required to obtain the
first success in a sequence of independent Bernoulli trials in which
the probability of success at each trial is 𝑝.
Cont.
• The sample space S for the experiment contains the countably infinite
set of sample points S = s, fs, ffs, fffs,...
• Therefore, the range for X is Rx = {1,2,3,…}
• Now consider the event X = k, that is, the first success occurs on the
k-th trial. This means that the first k – 1 trials resulted in failure.
• Thus k −1
P( X = k ) = (1 − p) p
Cont.
• A discrete random variable 𝑋 is said to have a Geometric distribution
with parameter 𝑝 (0 ≤ 𝑝 ≤ 1) if it assumes nonnegative values and
its probability mass function is given as:
 p (1 − p ) x −1 , x = 1, 2,3...
P[ X = x] = PX ( x) = 
0, Otherwise

• If X is a geometric rv with parameter p, we write X ~ Geom( p).

• The above pmf is proper probability function for a discrete rv because
PX (k )  o for all k ;
Example
• A large consignment of items contains 40% defective items. Items are
drawn until a defective item is found. What is the probability that:
• (a) three draws will be required to find a defective item?
• (b) fewer than three items will be required to find a defective item?
• c) more than two draws will be required to find a defective items.
Exercise
• Find the mean and variance of a Geometric distribution.
Exercise
• Find the moment generating function of a geometric distribution.
• Use the mgf to find E(X) and Var(X)
Theorem
• Let X be the geometric distribution with parameter 𝑝 . Then,
1
E( X ) =
p
q
Var ( X ) = 2
p
pet
M X (t ) =
(1 − qe t
)
Cont.
• The direct method is the use of the probability mass function while
the indirect method is the use of the moment generating function.
Example
• A fair coin is flipped until a head occurs. What is the probability that
less than 3 flips are required; that less than 4 flips are required?
Example
• A box of billard balls is inspected. The inspector draws a ball randomly
and independently until he finds a ball with a certain brand.It is
known that 10% of all balls have that specific brand.
• 1. What is the pdf of 𝑋?
• 2. What is 𝐸 𝑋 , 𝑉𝑎𝑟 𝑋 𝑎𝑛𝑑 𝑀𝑋 𝑡 ?
• 3. What is the probability that 10 𝑏𝑎𝑙𝑙𝑠 will be drawn before finding
one ball with the required brand?
Memoryless Property
• The geometric distribution possesses what is called a “memoryless
property” which makes it an inappropriate model for some real-life
problems.
• If X is a geometric rv, then the memoryless property states that

P( X  i + k X  i ) = P( X  k )

• That is, the probability that more than k additional trials will be
required to observe the first success, given no success in the first i
trials, does not depend on i.
Cont.
• For instance, if X is a geometric rv, then P(X > 15/X > 10) is simply
P(X > 5).
• The generalization of the geometric probability model is the Negative
Binomial Probability Model.
• Suppose that a sequence of independent Bernoulli trials, each with
probability 𝑝 of success are performed until there are 𝑟 successes.
• Define the rv X as the number of trials needed to achieve this.
• Then X is called the negative binomial rv with parameters r and p,
and we write X ~ NegBin(r , p).
Cont.
• The geometric distribution is the distribution of the number of trials
needed to get the first success in repeated independent Bernoulli
trials.
• The binomial distribution is the distribution of the number of
successes (x) in a fixed number of independent Bernoulli trials (n).

• The negative binomial distribution is the distribution of the number

of trials needed to get a fixed number of successes.
Probability Distribution Function
• The probability distribution function of the rv X is

 x − 1 r x − r
P( X = x) =  p q
 r −1 
• Where
0  p  1, q = 1 − p, r  0

• Remark: If r = 1, the negative binomial distribution reduces to the

geometric distribution.
The Moment Generating Function
• Let X be a rv having a negative binomial distribution. Then

t
rpe
M X (t ) =
1 − qe t

• Using the moment generation function, derive 𝐸 𝑋 𝑎𝑛𝑑 𝐸 𝑋 2 .

Theorem
• Let X be a rv having a negative binomial distribution. Then
r
E( X ) =
p
rq
Var ( X ) = 2
p
rpet
M X (t ) =
1 − qe t
Example
• A bag of bolts is inspected. A bolt is randomly drawn and inspected
until 5 defective bolts are found. It is known that 5% of all bolts in
the bag are defective. What is the probability that 40 bolts will be
examined before finding 5 defective bolts?
• Find E(X) and var(X).
• Let 𝑋 be the number of bolts drawn in order to find 5 defective bolts
Example
• An oil company conducts a geological study that indicates that an
exploratory oil well should have a 20% chance of striking oil.
• A) What is the probability that the first strike comes on the third well
drilled?
• B) What is the probability that the third strike comes on the seventh
well drilled?
Hypergeometric Distribution
• The hypergeometric probability distribution is closely related to the
binomial distribution but differs in two key ways.
• With the hypergeometric distribution,
• 1) The trials are not independent, and
• 2) The probability of success changes from trial to trial
Cont.
• The hypergeometric probability function is used to compute the
probability that in a random selection of 𝑛 elements, selected without
replacement, we obtain 𝑥 elements labelled success and 𝑛 − 𝑥
elements labelled failure.
• For this outcome to occur, we must obtain 𝑥 successes from the 𝑟
successes in the population and 𝑛 − 𝑥 failures from the 𝑁 − 𝑟
failures.
• The following hypergeometric pdf provides 𝑓(𝑥), the probability of
obtaining 𝑥 successes in 𝑛 trials
Cont.

 r  N − r 
 x  n − x 
P[ X = x] =    I ( x)
N 0,1,...,n
• Where n 
 
• 𝑥 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑠𝑢𝑐𝑐𝑒𝑠𝑠𝑒𝑠
• 𝑛 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑡𝑟𝑖𝑎𝑙𝑠 𝑜𝑟 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑖𝑧𝑒
• 𝑁 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛
• 𝑟 = 𝑛𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑒𝑙𝑒𝑚𝑒𝑛𝑡𝑠 𝑖𝑛 𝑡ℎ𝑒 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑙𝑎𝑏𝑒𝑙𝑙𝑒𝑑 𝑠𝑢𝑐𝑐𝑒𝑠𝑠
Cont.
𝑁
• Note that represents the number of ways 𝑛 elements can be
𝑛
selected from a population of size 𝑁;
𝑟
• represents the number of ways that 𝑥 successes can be selected
𝑥
from a total of 𝑟 successes in the population;
𝑁−𝑟
represents the number of ways that 𝑛 − 𝑥 failures can be
𝑛−𝑥
selected from a total of 𝑁 − 𝑟 failures in the population.
Restrictions
• N is a positive integer
•x≤N
•n≤N
• x is a nonnegative integer
• n is a positive integer
Theorem

• Let X be a rv with the hyper geometric distribution. Then

nr
E(X ) =
N
nr N − r N − n
Var ( X ) =  
N N N −1
M X (t )is not used
Example 1
• A box contains 7 red balls and 3 white balls. 5 balls are chosen at
random without replacement. What is the probability that:
• (a) 4 balls will be red?
• (b) at most 2 balls will be white?
Example
• Electric fuses produced by PEM Electric are packaged in boxes of 12
units each. Suppose an inspector randomly selects three of the 12
fuses in a box for testing. If the box contains exactly five defective
fuses, what is the probability that the inspector will find exactly one
of the three fuses defective?
• Find 𝐸 𝑋 𝑎𝑛𝑑 𝑉𝑎𝑟(𝑋)
• What is the probability of finding at least 1 defective fuse?
Example
• There are 20 television sets in a village at a given time. 9 of them are
tuned to channel 1, while 11 of them are tuned to channel 2. A
sample of size 5 of the 20 sets is randomly selected at this specific
time, and it is observed for each of the 5, whether the set is tuned to
channel 1 or 2. Let X be the number of sets in the sample that are
tuned to channel 1.
• A) What is the pdf of X?
• B) What is the probability that 3 or more TV sets in the sample are
tuned to channel 1?
• C) What is the expected value and variance of X?
Summary – Discrete Probability Distributions
• Bernoulli - Two outcomes, success/failure
• Binomial -Repeating an experiment n number of times
• Poisson – Expected number of occurrences in an interval
• Geometric – Number of trials required to obtain the 1st success
• Negative Binomial – Number of trials performed until there are r
successes
• Hypergeometric – Trials are not independent and probability of
success changes from trial to trial
Continuous Random Variables
• A discrete random variable is described by probabilities for each of its
possible values. This sort of distribution does not work for a
continuous random variable, because the probability that such a
variable takes on any given value equal zero, i.e. P(X = a) = 0 for any
value of a.
Cont.
• To illustrate

• A function whose graph is like this is called a probability density

function, denoted by 𝑓𝑥 𝑥 𝑜𝑟 𝑓 𝑥 .
Cont.
• The height of the curve over any point on the horizontal axis is the
probability density at that point.
• Unlike the height of the columns in a graph
of a probability mass function, it is not a probability.
• It is always the area under the curve which
represent the probabilities.
• We note that for a continuous rv P(X = a) = 0, this makes sense now
because the area under the curve from a to a is zero.
• Notice also that
Cont.
•𝑃 1<𝑋<3 =𝑃 1≤𝑋≤3
• Now the area under a graph can be represented by an integral; thus
b
P(a  X  b) =  f ( x)dx
a

• Each probability density must have the properties:

f ( x)  0 for all x

P(−  X  ) = 
−
f ( x)dx = 1
Example
• Let X be a continuous rv such that 𝑓 𝑥 = 1, 𝑓𝑜𝑟 0 < 𝑋 < 1
• Find the following probabilities:
• 𝑃 𝑋 > 0.5 ,
• 𝑃(0.5 < 𝑋 < 0.8)
Cumulative Distribution Function
• The cumulative distribution function of a continuous rv X is defined in
the same way as in discrete case. Thus, the probability that a rv X
assumes a value less than or equal to X is
FX ( x) = P( X  x).

• We can find the cumulative distribution function simply by

integration.
u
F (u ) = P( X  u ) = 
−
f ( x)dx
Cont.
• From this it follows that probability density function can also be
derived from the cumulative distribution function by differentiation.
dF ( x)
f ( x) =
dx
Mean, Variance and Moment Generating
Function
• The mean and the variance of a continuous rv X, denoted by 𝐸 𝑋 is

defined as
E( X ) =  xf ( x)dx
−


• The variance is defined as Var (X) =  ( x − E( X ))

2
f ( x)dx
−


 x 2 f ( x)dx − ( E ( X ) )
2
=
−

• And the moment generating function as M ( t ) = etx f ( x)dx
X  X −
Example
• Let the pdf of a continuous random variable be given by
f X ( x,  ) =  e − x
I ( x)
0,

1: Show that X is a probability density function.

2. Show that 
M X (t ) =
 −t
Cont.

• Using, M X (t ) =
 −t
• Derive E( X )
Var ( X )
Expectation and variance

1
E( X ) =

1
Var ( X ) =
 2
 Properties of f
• Let f be the density function of a random variable X
• Properties of f
t
a ) F (t ) = 
−
f ( x)dx


b)  f ( x)dx = 1
−

c) If f is continuous, then F'(x) = f(x)

Cont.

b
d) For real numbers, a  b, P(a  X  b) =  f (t )dt
a
b
e) P ( a  X  b ) = P ( a  X  b ) = P ( a  X  b ) = P ( a  X  b ) =  f (t )dt
a
Important Continuous Distributions
• The following continuous distributions will be discussed:
• - Uniform Distribution
• - Exponential Distribution
• - Gamma Distribution
• - Beta Distribution
• - Normal Distribution
The Uniform Distribution
• The probability density function of a random variable X with
Continuous Uniform Distribution is given by:
1
fX ( x) = ( x)
Ia ,b
b−a
a  x  b, −   a  b  

• Find 𝐸 𝑋 , 𝑉𝑎𝑟 𝑋 𝑎𝑛𝑑 𝑀𝑋 (𝑡)

Theorem
• Let X be a rv with a continuous uniform distribution. Then
a+b
1.E ( X ) =
2
( b − a)
2

2.Var ( X ) =
12
ebt − e at
3.M x (t ) =
(b − a )t
Moment Generating Function
• Theorem:
• If X is a random variable having a Uniform Distribution over the
interval [a, b], then the moment generating function of X is:

𝑒 𝑏𝑡 −𝑒 𝑎𝑡
𝑀𝑥 𝑡 =
𝑏−𝑎 𝑡

• Proof the above theorem

Example 1
• The height of trees at a school compound is uniformly distributed
with minimum and maximum heights of 160cms and 190cms
respectively.
• Let X be the height of a tree randomly selected for measurement.
What is the expected value of X?
Example
• You arrive into a building and are about to take an elevator to your
floor. Once you call the elevator, it will take between 0 and 40
seconds to arrive to you. We will assume that the elevator arrives
uniformly between 0 and 40 seconds after you press the button.
• 1. Calculate the probability that the elevator takes less than 15
seconds to arrive.
• 2. Find the expected value and variance of X.
Exponential Distribution
• A continuous rv X is said to have an exponential distribution with a
positive parameter  if its probability density function is given by:
− x
 e , x0
f X ( x) = 
o, Otherwise

• The exponential distribution is denoted by Exp(x; λ).

Cont.
• The exponential distribution is the model for elapsed time from the
observation of a process until the first event occurs.
• It is mainly used to measure the life times of events, e.g., the life
times of electric bulbs.
• The process is assumed to be a Poison process with parameter λ > 0,
the rate of occurrence in a unit interval.
Cont.
• We can proof that the exponential distribution is a probability density
function.
• a) f(x) ≥ 0
 
b)  f X ( x)dx =   e −  x dx
− 0

=   e −  x dx
0

 − x  1 
=  e  


 −  
0

= −1(0 − 1)
Probabilities Associated with an Exponential
Distribution
− x
i ) P( X  x) = e
− x
ii ) P( X  x) = 1 − e
−  x1 −  x2
iii ) P( x1  X  x2 ) = e −e
Example
• Let 𝑋~𝐸𝑥𝑝(0.1) Let’s compute the following probabilities
• a ) P( X  5)
b) P(10  X  15)
c) P( X  0.5)
The cumulative distribution function of X
• The cumulative distribution function of X is

t t
FX (t) =  f X (x)dx =   e − x
dx
0 0

• =?
Example
• Let 𝑋 be an exponential r.v. with mean 6. Find
• (a) 𝑃 𝑋 > 4
• (b) 𝑃(𝑋 > 6|𝑋 > 2)
Memoryless Property
• We can show that
P( x  s + t x  s ) = P ( x  t ) for all s, t  0

• If we think of 𝑥 being a lifetime of some instrument, then this

equation states that the probability that the instrument lives for at
least s+t hours, given that it has survived for at least s hours is the
same as the initial probability that it will live for at least t hours.
Theorem
• If X is a rv having an exponential distribution, then
1
a) E ( X ) =

1
b)Var ( X ) =
2

c) M X (t ) = for t  
 −t
• Proof the above theorems
Example
• Suppose that the amount of time one spends in a bank is
exponentially distributed with mean 10 minutes. What is the
probability that a customer will spend more than 15 minutes in the
bank given that she is still in the bank after 10 minutes.
Examples
• 1. The time elapsed, in minutes, between the placement of an order
of pizza and its delivery is random with the density function
1 / 15, 25  X  40
f ( x) = 
0, Otherwise

• Determine the mean and standard deviation of the time it takes for
the pizza shop to deliver pizza
Cont.
• The probability density function of a continuous distribution is given
as:  27

2
f ( x) =  490
( 3x − 2 x ) ,
2/3 x 3
0, elsewhere

• Find the mean

Example
• Suppose that the time until failure of an electric component is an
exponential r.v. with the rate of 𝜆 = 0.0625 per month. Calculate the
probability that the device lasts:
• a) not more than 45 months
• b) longer than 40 months
• c) Find the expected value and variance of X.