Moment Generating Functions

The Normal distribution

(mean , standard deviation )
( x −  )2
1 −
f ( x) = e 2 2

2
Expectation of functions of
Random Variables
X is discrete

E  g ( X ) =  g ( x ) p ( x ) =  g ( xi ) p ( xi )
x i

X is continuous


E  g ( X ) =  g ( x ) f ( x ) dx
−
Moments of Random Variables
The kth moment of X.

k = E ( X k )

  xk p ( x ) if X is discrete
 x
= 
  x k f ( x ) dx if X is continuous
-
the kth central moment of X

 = E ( X − ) 

0 k
k
 
  ( x −  )k p ( x ) if X is discrete
 x
= 
  ( x −  )k f ( x ) dx if X is continuous
-

where  = 1 = E(X) = the first moment of X .

Rules for expectation
Rules:
1. E  c  = c where c is a constant

2. E  aX + b  = aE  X  + b where a, b are constants

3. var ( X ) =  = E ( X −  ) 

0 2
2
 
( )
= E X −  E ( X ) = 2 − 12
2 2

4. var ( aX + b ) = a 2 var ( X )
Moment generating functions
Moment Generating function of a R.V. X

  etx p ( x ) if X is discrete
 x
mX ( t ) = E e  =  
tX

  etx f ( x ) dx if X is continuous
 −
Examples
1. The Binomial distribution (parameters p, n)

mX ( t ) = E etX  =  etx p ( x )
x

n
n x
=  e   p (1 − p )
tx n− x

x =0  x
n t x  n  x n− x
( )
n n
=    e p (1 − p ) =    a b
n− x

x =0  x  x =0  x 

(
= ( a + b) = e p +1− p )
n n
t
2. The Poisson distribution (parameter )
x
p ( x) = e−  x = 0,1, 2,
x!
The moment generating function of X , mX(t) is:

mX ( t ) = E etX  =  etx p ( x ) =  e
n
 x
tx
e− 
x x =0 x!

( e )
x
 t 
x
u
= e−  = e −  ee using eu = 
t

x =0 x! x =0 x !

=e
(
 et −1 )
4. The Standard Normal distribution ( = 0,  = 1)
1 − x22
f ( x) = e
2
The moment generating function of X , mX(t) is:

mX ( t ) = E etX  =  f ( x ) dx
e tx

−

1 − x22
= e
tx
e dx
− 2

1 − x2 −22 tx
= 
− 2
e dx
We will now use the fact that
 ( x−b )2
1 − 2
− 2 a e 2a
dx = 1 for all a  0, b

We have
completed
the square
 
1 − x2 −22 tx 1
mX ( t ) =
t2 x2 −2 tx +t 2

 dx = e 2 
−
e e 2 dx
− 2 − 2

t2 1 − ( x−2t )2 t2
=e 2

− 2
e dx = e 2







This is 1
 Properties of
Moment Generating Functions
 1. mX(0) = 1
( ) ( )
mX ( t ) = E etX , hence mX ( 0 ) = E e0 X = E (1) = 1

Note: the moment generating functions of the following

distributions satisfy the property mX(0) = 1
(
Binomial Dist'n mX ( t ) = e p + 1 − p )
n
t
i)

ii) Poisson Dist'n mX ( t ) = e

(
 et −1 )
  
iii) Exponential Dist'n mX ( t ) =  
 t2 − t 
iv) Std Normal Dist'n mX ( t ) = e 2

  
v) Gamma Dist'n mX ( t ) =  
  − t 
 2 3 k
2. mX ( t ) = 1 + 1t + t +
2
t +
3
+ tk +
2! 3! k!
We use the expansion of the exponential function:
2 3
u u uk
eu = 1 + u + + + + +
2! 3! k!
( )
mX ( t ) = E etX
 t2 2 t3 3 tk k 
= E 1 + tX + X + X + + X + 
 2!
2
3!
3
k!
k

t
( ) t
= 1 + tE ( X ) + E X + E X + + E X k +
2!
2

3!
3
( )
t
k!
( )
t2 t3 tk
= 1 + t 1 + 2 + 3 + + k +
2! 3! k!
 k
d
3. mX ( 0 ) = k mX ( t ) = k
(k )
dt t =0

Now 2 2 3 3 k k
mX ( t ) = 1 + 1t + t + t + + t +
2! 3! k!
2 3 2 k k −1
mX ( t ) = 1 + 2t + 3t + + kt +
2! 3! k!
3 2 k k −1
= 1 + 2t + t + + t +
2! ( k − 1)!
and mX ( 0 ) = 1
4 k
mX ( t ) = 2 + 3t + t + + t k −2 +
2! ( k − 2 )!
and mX ( 0 ) = 2
continuing we find mX ( 0 ) = k
(k )
 Property 3 is very useful in determining the moments of a
random variable X.
Examples
i) Binomial Dist'n mX ( t ) = ( e p + 1 − p )
n
t

( ) ( )
n −1
mX ( t ) = n e p + 1 − p
t
pet
m ( 0 ) = n ( e p + 1 − p ) ( pe ) = np =  = 
n −1
0 0
X 1

m ( t ) = np ( n − 1) ( e p + 1 − p ) ( e p ) e + ( e p + 1 − p ) et 
n−2 n −1
t t t t
X  
= npe ( e p + 1 − p ) ( n − 1) ( e p ) + ( e p + 1 − p ) 
n−2
t t t t

= npe ( e p + 1 − p )  ne p + 1 − p 
n−2
t t t

= np  np + 1 − p  = np  np + q  = n 2 p 2 + npq = 2
ii) Poisson Dist'n mX ( t ) = e
(  et −1 )

mX ( t ) = e
(
 et −1 ) et  = e (e −1)+t t

 

mX ( t ) = e
 ( ) et + 1 =  2e (e −1)+ 2t + e (e −1)+t
 et −1 +t t t

 

mX ( t ) =  e
 (
2  e −1 + 2t
t
) e + 2 + e (
t  et −1 +t) et + 1
 

= e
(
2  e −1 + 2t
t
) e + 3 + e (
t  et −1 +t)

= e
( 3  e −1 +3t
t
) + 3 e
( )
2  e −1 + 2t
t

+ e
( )
 et −1 +t
To find the moments we set t = 0.

1 = mX ( 0 ) =  e ( ) =
 e0 −1 + 0

2 = mX ( 0 ) =  e ( ) + e ( e −1)+0 =  2 + 

2  e −1 + 0
0 0

3 = mX ( 0 ) =  3e0 + 3 2e0t +  e0 =  3 + 3 2 + 

The moments for the standard normal distribution

mX ( t ) = e
t2
2

We use the expansion of eu.

 k 2 3
u u u uk
eu =  = 1 + u + + + + +
k =0 k ! 2! 3! k!
( ) ( ) ( )
2 3 k
t2 t2 t2

mX ( t ) = e = 1 + ( )
t2 2 2 2
2 t2
2 + + + + +
2! 3! k!
1 4 1 6 1 2k
= 1+ 2 t + 2 t + 3 t +
1 2
+ k t +
2 2! 2 3! 2 k!
We now equate the coefficients tk in:
2 k 2 k
mX ( t ) = 1 + 1t + t +
2
+ t + k
+ t +
2k

2! k! ( 2k ) !
If k is odd: k = 0.

2 k 1
For even 2k: = k
( 2k ) ! 2 k !
or 2 k =
( 2k )!
k
2 k!

2! 4!
Thus 1 = 0, 2 = = 1, 3 = 0, 4 = 2 =3
2 2 ( 2!)