Moment Generating Functions

Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

UNIT 6 : PROBABILITY FUNCTIONS

Gabriel Asare Okyere (PhD)

August 25, 2018

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Outline of Presentation

1 Moment Generating Functions

2 Characteristic Function
3 TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
4 Special Probability Distributions
5 TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
6 Useful Continuous Distributions
7 Distributions of Functions of Random Variables

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Moment Generating Functions

The Moment Generating Function (MGF) is a function that
generates moments. It is defined as follows.

Mathematical Definition of MGF

The Moment Generating Function(MGF) of a random variable X,
denoted by MX (t), is defined by
MX (t) = E (e tX ) =
(P
∞
e tx f (x) if X is discrete
f (x) = R ∞x=0 tx
−∞ e f (x)dx if X is continuous

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Example
A discrete random variable Y has probability mass function given
by  
12
P(Y = y ) = (0.6)y (0.4)12−y , y = 0, 1, ..., 12.
y
Find the moment generating function of Y

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Solution

MY (t) = E (e tY )
12  
X 12
= e ty (0.6)y (0.4)12−y
y
y =0
12  
X 12
= (0.6e t )y (0.4)12−y
y
y =0

Recognizing the sum as a binomial expansion of (0.6e t + 0.4)12 ,

we obtain
MY (t) = (0.6e t + 0.4)12

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Example
A continuous random variable Y has a p.d.f given by
(
4e −4y , y > 0
f (y ) =
0 elsewhere
Find the moment generating function if Y

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Solution
Z ∞
tY
MY (t) = E (e ) = e ty f (y )dy
−∞
Z ∞
= e ty 4e −4y dy
Z0 ∞
= 4e −4y +ty dy
Z0 ∞
= 4e −(4−t)y dy
0
" #∞
e −(4−t)y 4
= −4 = ,t < 4
4−t 4−t
0

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Theorem
If Mx (t) exists, then for any positive integer k,

d k MX (t) (k)
|t=0 = MX (0) = E (X k )
dt k
d k MX (t) (k)
, where dt k
= MX is the k th derivative of MX (t) with respect
to t.

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Proof

MX (t) = E (e tX )
MX0 (t) = E (Xe tX ) ⇒ MX0 (0) = E (X )
MX00 (t) = E (X 2 e tX ) ⇒ MX00 (0) = E (X 2 )
. .
. = .
. .
MXn (t) = E (X n e tX ) ⇒ MXn (0) = E (X n )

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Example
A random variable Y has a moment generating function,
MY (t) = (0.6e t + 0.4)12 Find the E (Y ) and Var (Y )

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Solution

MY (t) = (0.6e t + 0.4)12

MY0 (t) = 12(0.6e t + 0.4)11 (0.6e t )
E (Y ) = MY0 (0) = 12(0.6e 0 + 0.4)11 (0.60 )
= 7.2
MY00 (t) = 12 × 11(0.6e t + 0.4)10 (0.6e t )2
+12(0.6e t + 0.4)11 (0.6e t )
E (Y 2 ) = MY00 (0) = 12 × 11 × 0.62 + 12 × 0.6
= 54.72
Var (Y ) = E (Y 2 ) − (E (Y ))2 = 54.72 − 7.22
= 2.88
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Example
A continuous random variable Y has moment generating function
given by

MY (t) = 3(3 − t)−1

Find the E (Y ) and Var (Y ).

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Solution

MY (t) = 3(3 − t)−1

MY0 (t) = 3(3 − t)−2
MY00 (t) = 6(3 − t)−3
1
E (Y ) = MY0 (0) = 3(3)−2 =
3
2 00 −3 2
E (Y ) = MY (0) = 6(3) =
9
Var (Y ) = E (Y 2 ) − (E (Y ))2
 2
2 1
= −
9 3
1
=
9
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Theorem
A moment generating function always exists at t = 0 and equals to
1.

Proof
MX (0) = E (e 0X ) = E (1) = 1

Theorem
Let X be a random variable with moment generating function
MX (t). If Y = aX , where a is a constant, then MY (t) = MX (at)

Proof
MY (t) = E (e tY ) = E (E taX ) = E (e (at)X ) = MX (at)
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Example
4
Given that X has the moment generating function, MX (t) = 4−t
for t < 4, find the moment generating function of Y = 2X

Solution

4
MX (t) =
4−t
MY (t) = M2X (t) = MX (2t)
4
= ,t < 2
4 − (2t)

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Theorem
If Y = a + bX , and MX (t) exists, then MY (t) = e at MX (bt)

Proof

MY (t) = E (e t(a+bX ) ) = E (e at e btX ) = e at E (e btX ) = e at MX (bt)

Example
4
Given that X has the moment generating function, MX (t) = 4−t
for t < 4, find the moment generating function of Y = 4 + 3X

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Solution

4
MX (t) =
4−t
MY (t) = M4+3X (t) = e 4t MX (3t)
 
4t 4 4
= e ,t <
4 − (3t) 3

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

The Uniqueness theorem

Corresponding to each moment generating function M(t), there is
a unique distribution function having that M(t) as moment
generating function.

Independence
Let X1 , X2 , ...Xn be independent random variables with moment
generating functions MXi (t)(i = 1, 2, ..., n). If
Y = X1 + X2 + ... + Xn , then MY (t) = MX1 (t)MX2 (t)...MXn (t).

Example
The random variables X and Y are independent with respective
moment generating functions MX (t) = (pe t + q)n and
MY (t) = (pe t + q)m , where p + q = 1. Find the moment
generating function of X + Y
Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION
 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Solution
Since X and Y are independent,

MX +Y (t) = MX (t)MY (t)

= (pe t + q)n (pe t + q)m
= (pe t + q)n+m

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Outline of Presentation

1 Moment Generating Functions

2 Characteristic Function
3 TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
4 Special Probability Distributions
5 TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
6 Useful Continuous Distributions
7 Distributions of Functions of Random Variables

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Characteristic function
Unfortunately, not all distributions have moment generating
functions and so we consider another function which exists for all
distributions and has more applications than moment generating
functions. This function is called the characteristic function.

Mathematical Definition
The characteristic function of a random variable X, denoted by
φX (t), is defined for all real t by φX (t) = E (e itX ), where i 2 = −1
Thus,
(P
∞
e itx f (x) if X is discrete
φX (t) = R ∞x=0 itx
−∞ e f (x)dx if X is continuous

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Example
Given
(
3e −3x , x ≥0
f (x) =
0 otherwise
Find the characteristic function of X .

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Solution
Z ∞
φX (t) = E (e itX ) = e itx f (x)dx
−∞
Z ∞ Z ∞
−3x
= itx
e (3e )dx = 3e −(3−it)x dx
0 0
" #∞
e −(3−it)x 3
= 3 = , t < 3.
−(3 − it) 3 − it
0

Remark
It can be seen that, if MX (t) exists, then φY (t) = MY (it)

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Example
Given the discrete random variable Y as below, find the
characteristic function of Y
 
12
P(Y = y ) = (0.6)y (0.4)12−y , y = 0, 1, ..., 12.
y

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Solution

12  
itY
X
ity 12
φY (t) = E (e )= e (0.6)y (0.4)12−y
y
y =0
12  
X 12
= (0.6e it )y (0.4)12−y = (0.6e it + 0.4)12
y
y =0

Notice again that φY (t) = MY (it), if MY (t) exists.

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Theorem
φX (0) = E (e 0 ) = 1.

Theorem
If Y = a + bX , where a and b are constants, then
φY (t) = e ita φbX (t) = e ita φX (bt)

Proof

φY (t) = E (e it(a+bX ) ) = E (e ita e itbX )

= e ita E (e itbX ) = e ita φX (tb)

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION

 Moment Generating Functions
Characteristic Function
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Special Probability Distributions
TRIAL QUESTIONS FROM SPECIFIC TEXTBOOK(S)
Useful Continuous Distributions
Distributions of Functions of Random Variables

Theorem
(n)
If the nth moment of X exists, then the nth derivation φX of
φX (t) exists, and
(k)
φX (0) = i k E (X k ), k = 1, 2, ..., n

Gabriel Asare Okyere (PhD) PROBABILITY & PROBABILITY DISTRIBUTION