Conditional Expectations

Martingales
Markov Processes

Martingales and Markov

FE543 Intro to Stochastic Calculus

Papa Momar Ndiaye

Division of Financial Engineering

Stevens Institute of Technology

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Conditional Expectations
Martingales
Conditional Expectation Properties
Markov Processes

Recall the risk neutral probabilities

1+r −d u−1−r
p̃ = ; q̃ = (1)
u−d u−d
We previously determined that the stock at time n could be
expressed as the discounted weighted average of the two
possible stock prices at time n + 1 and expressed this as

1
Sn (ω1 . . . ωn ) = [p̃Sn+1 (ω1 . . . ωn H) + q̃Sn+1 (ω1 . . . ωn T )]
1+r

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Conditional Expectations
Martingales
Conditional Expectation Properties
Markov Processes

To simplify this expression we use

E˜n [Sn+1 ](ω1 . . . ωn ) = p̃Sn+1 (ω1 . . . ωn H) + q̃Sn+1 (ω1 . . . ωn T )

which is the expected value of the stock at time n + 1

conditioned on the value of the stock price at time n. We can
also suppress the ω notation and have

1
Sn = Ẽn [Sn+1 ]
1+r

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Conditional Expectations
Martingales
Conditional Expectation Properties
Markov Processes

Conditional Expectation

I The conditional expectation of a random variable that

depends on information up to time m conditioned on
information up to time n where n < m can be written
En [Xm ](ω1 . . . ωn ).
I The conditional expectation depends on the sequence of
tosses (ω1 . . . ωn ) and can take different values depending
on the outcome of the tosses. So it is a random variable.
I E0 [X ], the conditional expectation based on no information,
is simply E[X ] so E0 [X ] = E[X ]
I EN [X ] the conditional expectation based on knowledge of
all coin tosses is simply X so ẼN [X ] = X .[? ]

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Conditional Expectations
Martingales
Conditional Expectation Properties
Markov Processes

Definition:
I Let n satisfy 1 ≤ n ≤ N, and let ω1 . . . ωn be given and, for
the moment, fixed. There are 2N−n possible continuations
ωn+1 . . . ωN of the sequence ω1 . . . ωn .
I Denote by #H(ωn+1 . . . ωN ) the number of heads in the
continuation and similarly define #T (ωn+1 . . . ωN ) as the
number of tails.
I Ẽn [X ] the conditional expectation of X based on the
information at time n is defined as
#H(ωn+1 ...ωN ) #T (ωn+1 ...ωN )
X
Ẽn [X ](ω1 . . . ωn ) = p̃ q̃ X (ω1 . . . ωn ωn+1 . . . ωN ) (2)
ωn+1 ...ωN

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Conditional Expectations
Martingales
Conditional Expectation Properties
Markov Processes

Fundamental Properties of Conditional Expectations

Let N be a positive integer, and let X and Y be random variables
depending on the first N coin tosses. Let 0 ≤ n ≤ N be given. The
following properties hold.
1. Linearity of conditional expectations: For all constants c1 and
c2 , we have En [c1 X + c2 Y ] = c1 En [X ] + c2 En [Y ]
2. Taking out what is known: If X actually depends only on the
first n tosses, then En [XY ] = X En [Y ]
3. Iterated Conditioning: If 0 ≤ n ≤ m ≤ N, then
En [Em [X ]] = En [X ]
Consequently, we have E[En [X ]] = E[X ]
4. Independence: If X depends only on tosses n + 1 through N,
then En [X ] = E[X ]
5. Conditional Jensen’s Inequality: If ϕ(x) is a convex function of
the dummy variable x, then En [ϕ(X )] ≥ ϕ(En [X ])
FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
[? ]
 Conditional Expectations
Martingales
Markov Processes

Martingales

Definition 2.4.1: Consider the binomial asset-pricing model.

Let M0 , M1 , . . . , MN be a sequence of random variables, with
each Mn depending only on the first n tosses (and M0 is a
constant). Such a sequence of random variables is called an
adapted stochastic process.
1. If Mn = En [Mn+1 ], ∀n = 0, 1, . . . , N − 1 we say this process
is a martingale
2. If Mn ≤ En [Mn+1 ], ∀n = 0, 1, . . . , N − 1 we say this process
is a submartingale
3. If Mn ≥ En [Mn+1 ], ∀n = 0, 1, . . . , N − 1 we say this process
is a supermartingale
[? ]

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Discounted Price Process

Theorem 2.4.4: Consider the general binomial model with

0 < d < 1 + r < u. Let the risk neutral probabilities be given by

1+r −d u−1−r
p̃ = , q̃ =
u−d u−d
Then, under the risk-neutral measure, the discounted stock
price is a martingale, i.e.
 
Sn Sn+1
= Ẽn
(1 + r )n (1 + r )n+1

holds at every time n and for every sequence of coin tosses.[? ]

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Proof 1: Let n and ω1 . . . ωn be given. Then

 
Sn+1 1 1
Ẽn n+1
= n
∗ [p̃Sn+1 (H) + q̃Sn+1 (T )]
(1 + r ) (1 + r ) 1+r
1 1
n
∗ [p̃uSn + q̃dSn ]
(1 + r ) 1+r
Sn p̃u + q̃d
= n
∗
(1 + r ) 1+r
Sn
=
(1 + r )n

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

S
Proof 2: (fancier) Note that Sn+1
n
depends only on the (n + 1)st
coin toss. Using Theorem 2.3.2:
   
Sn+1 Sn Sn+1
Ẽn = Ẽn ∗
(1 + r )n+1 (1 + r )n+1 Sn
 
Sn 1 Sn+1
= n
∗ Ẽn ∗ , (Taking out what is known)
(1 + r ) 1+r Sn
 
Sn 1 Sn+1
= n
∗ Ẽ , (Independence)
(1 + r ) 1+r Sn
Sn 1
= ∗ (p̃u + q̃d)
(1 + r )n 1 + r
Sn
=
(1 + r )n

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Discounted Wealth Process

Theorem Consider the binomial model with N periods. Let

∆0 , ∆1 , . . . , ∆N−1 be an adapted portfolio process, let X0 be a
real number, and let the wealth process X1 , . . . , XN be
generated recursively by the wealth equation.
Xn
I The discounted wealth process (1+r )n , n = 0, 1, . . . , N is a
martingale under the risk-neutral measure. i.e.
 
Xn Xn+1
= Ẽn , n = 0, 1, . . . , N − 1
(1 + r )n (1 + r )n+1

[? ]
h i
Xn
I As a corollary, we have Ẽ (1+r )n = X0 , ∀n = 0, 1, . . . , N

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Theorem 2.4.7: (Risk-Neutral Pricing Formula

Take a N-period BAPM with 0 < d < 1 + r < u and with
risk-neutral probability measure P̃. Let VN be a random variable
(a derivative security paying off at time N) depending on the
coin tosses. Then, for n between 0 and N, the price of the
derivative security at time n is given by
 
VN
Vn = Ẽ
(1 + r )N−n

Furthermore, the discounted price of the derivative security is a

martingale under P̃, i.e.
 
Vn Vn+1
= Ẽ , n = 0, 1, . . . , N − 1
(1 + r )n (1 + r )n+1

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Markov Processes

Definition Consider the binomial asset pricing model. Let

X0 , X1 , . . . , XN be an adapted process.
If, for every n between 0 and N − 1 and for every function f (x),
there is another function g(x) (depending on n and f ) such that

En [f (Xn+1 )] = g(Xn )

we say that X0 , X1 , . . . , XN is a Markov process.[? ]

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

By definition, En [f (Xn+1 )] is a random variable. If the process is

Markov, then the information needed to calculate the value of
this random variable is contained entirely in Xn . Take for
example
(
uSn (ω1 . . . ωn ), if ωn+1 = H
Sn+1 (ω1 . . . ωn+1 ) =
dSn (ω1 . . . ωn ), if ωn+1 = T

I We have

En [f (Sn+1 )] = g(Sn ), where g(x) = pf (ux) + qf (dx)

so the stock price is Markov under P or P̃.

I Not every martingale is Markov and not every Markov
process is a martingale. In fact, the condition for stock
price to be both a martingale and Markov is if pu + qd = 1.
FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Independence

In the N-period binomial asset pricing model, let n be an

integer between 0 and N. Suppose the random variables
X 1 , . . . , X k depend only on coin tosses 1 through n and the
random variables Y 1 , . . . , Y l depend only on coin tosses n + 1
through N. Let f (x 1 , . . . , x k , y 1 , . . . , y l ) be a function of the
dummy variables x 1 , . . . , x k , y 1 , . . . , y l , and define

g(x 1 , . . . , x k ) = E[f (x 1 , . . . , x k , Y 1 , . . . , Y l )]

then
En [f (X 1 , . . . , X k , Y 1 , . . . , Y l )] = g(X 1 , . . . , X k )
[? ]

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Max-to-date

Consider the maximum-to-date process Mn = max0≤k ≤n Sk for

our BAPM with S0 = 4, u = 2, d = 21 , and r = 41 :

Maximum to Date Process

n=3 n=2 n=1 n=0
M3 (HHH) = 32 M2 (HH) = 16 M1 (H) = 8 M0 = 4
M3 (HHT ) = 16
M3 (HTH) = 8 M2 (HT ) = 8
M3 (HTT ) = 8
M3 (THH) = 8 M2 (TH) = 4 M1 (T ) = 4
M3 (THT ) = 4
M3 (TTH) = 4 M2 (TT ) = 4
M3 (TTT ) = 4

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Max-to-date

2
For this process if p = 3 and q = 13 , we have:

2 1 20
E2 [M3 ](TH) = M3 (THH) + M3 (THT ) =
3 3 3
2 1
E2 [M3 ](TT ) = M3 (TTH) + M3 (TTT ) = 4
3 3

Since, M2 (TH) = M2 (TT ) = 4, there cannot be a function g that

satisfies the Markov property for this maximum-to-date process.
As a result this is an example of a non-Markovian process.

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

K-dimensional Markov

Definition 2.5.5: Consider the binomial asset-pricing model.

Let {(Xn1 , . . . , Xnk ); n = 0, 1, . . . , N} be a k -dimensional adapted
process; i.e. k one-dimensional adapted processes.
If, for every n between 0 and N − 1 and for every function
f (x 1 , . . . , x k ), there is another function g(x 1 , . . . , x k ), depending
only on f and n, such that
1 k
En [f (Xn+1 , . . . , Xn+1 )] = g(Xn1 , . . . , Xnk )

we say that {(Xn1 , . . . , Xnk ); n = 0, 1, . . . , N} is a k -dimensional

Markov Process.[? ]

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 Conditional Expectations
Martingales
Markov Processes

Max-to-date again
The 2-dimensional process {(Sn , Mn ); n = 0, 1, . . . , N}, where
Sn is the stock price and Mn is the maximum-to-date is Markov.
S
1. Define Y = Sn+1 n
, (depends only on the n + 1st toss), and so

Sn+1 = Sn Y
Mn+1 = Mn ∨ Sn+1 = Mn ∨ Sn Y
2. Therefore, we have
En [f (Sn+1 , Mn+1 )] = En [f (Sn Y , Mn ∨ Sn Y )]
3. Define the function g by
g(s, m) = E[f (sY , m∨sY )] = pf (us, m∨us)+qf (ds, m∨ds)
4. Consequently, we have
En [f (Sn+1 , Mn+1 )] = g(Sn , Mn )
FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov
 [1] S.E. Shreve. Stochastic Calculus for Finance I: The
Binomial Asset Pricing Model. Number v. 1 in Springer
Finance.

FE543 Intro to Stochastic Calculus Papa Momar Ndiaye Martingales and Markov