Chapter 10

Diusions and Stochastic Dierential

Equations
A diusion process is a stochastic process X = (X(t) : t 0) satisfying a stochastic dierential equations
(SDE). Such diusions are Markov processes that evolve in continuous time and take values in a continuous
state space.
10.1 Stochastic Dierential Equations
A common approach to modeling deterministic dynamical systems is to postulate that the state x = (x(t) :
t 0) satises a deterministic ordinary dierential equation (ODE).

d
dt
x(t) = (x(t))
x(0) = x
0
(10.1)
A natural stochastic analog to (10.1) is

d
dt
X(t) = (X(t)) +(X(t))(t)
X(0) = x
0
(10.2)
where ((t) : t 0) is a unit variance white noise process for which E [(t)] = 0 and
cov ((s), (t), =)
st
where
st
is one if s = t and is zero otherwise. Note that for t
1
< t
2
< t
3
,
cov

t
2
t
1
(s)ds,

t
3
t
2
(u)du, =

0, (10.3)
where as
var

t
0
(s)ds

= t. (10.4)
Such a process ((t) : t 0) has highly irregular sample paths and is very dicult to work with directly.
(Imagine trying to simulate !) As a consequence, it is mathematically easier to work with its (smoother)
integral. This suggests writing (10.2) in the form
X(t) X(0) =

t
0
(X(s))ds +

t
0
(X(s))(s)ds (10.5)
147
In this integrated version, we must make mathematical sense of the stochastic integral involving the inte-
grator (s)ds. From a notational standpoint, it is standard to write
dX(t) = (X(t))dt +(X(t))(t)dt (10.6)
in place of (10.5). The equation (10.6) is what is known as a stochastic dierential equation (SDE). The
rigorous meaning of (10.6) is the integral equation (10.5).
10.2 Brownian Motion
Brownian motion plays a key role in the theory of stochastic integration. A standard Brownian motion is a
Gaussian process B = (B(t) : t 0) satisfying:
E [B(t)] = 0 for t 0
cov ((, B) (s), B(t)) = min(s, t) for s, t 0
B has continuous sample paths
A Brownian motion with drift and variance
2
is a process Z = (Z(t) : t 0) taking the form
Z(t) = t +B(t)
for t 0. Note that
Z(t)
D
= N(t,
2
t)
A Brownian motion has stationary independent increments:
For t
1
< t
2
< . . . < t
n
, Z(t
1
) Z(0), Z(t
2
) Z(t
1
), . . . Z(t
n
) Z(t
n1
) are independent random
variables (i.e. independent increments)
Z(t +s) Z(t)
D
= Z(s) Z(0) (i.e. stationary increments)
As a consequence, it is easily veried that
cov (B(t
2
) B(t
1
), B(t
3
) B(t
2
), =) 0
and
var (B(t) B(0)) = t
Given the similarity with (10.3) and (10.4), this suggests that B can be viewed as integrated white noise,
so that we can rigorously dene

t
0
(s)ds
to be B(T) B(0) (= B(t)).
Remark 10.1: This is something of an over simplication. To write
B(t) =

t
0
(s)ds
would require that B is dierentiable almost everywhere (in time). But
B(t +h) B(t)
h
D
= N

0, h

1
2

148
so no limit exists as h 0. Hence, B is non-dierentiable at t. This over simplication comes from to the
fact that white noise does not exist as a well-dene stochastic process. On the other hand, Brownian motion
is well-dened, so this suggests that mathematically, we should replace (10.5) with
X(t) X(0) =

t
0
(X(s))ds +

t
0
(X(s))dB(s) (10.7)
and (10.6) by
dX(t) = (X(t))dt +(X(t))dB(t) (10.8)
10.3 Stochastic Integrals
The integral

t
0
(X(s))ds
can be dened via a standard Riemann approximation. On the other hand,

t
0
(X(s))dB(s)
must be dened dierently, since the integrator here is a non-dierentiable stochastic process (namely, B).
The most commonly accepted denition of the stochastic integral is to dene it as a limit of approximations
of the form
n1

k=0

kt
n

(k + 1)t
n

kt
n

as n . This leads to the so-called It o integral denition for

t
0
(X(s))dB(s)
Remark 10.2: Because of the non-dierentiability of B, it turns out that the approximation
n1

k=0

(k + 1)t
n

(k + 1)t
n

kt
n

converges to a dierent limit as n . Hence, care must be taken in working with stochastic integrals.
10.4 The Innitesimal Drift and Variance of a Diusion
Under modest conditions on () and (), there exists a solution X = (X(t) : t 0) to the SDE
dX(t) = (X(t))dt +(X(t))dB(t)
The diusion X is a Markov process with continuous sample paths and is time-homogeneous in the sense
that
P
x
{X(t +h) |X(u) : 0 u t} = P(h, X(t), }
where
P{h, x, B} = P
x
{X(h) B}
149
Note that when h > 0 is small,
X(t) X(0) =

h
0
(X(s))ds +

h
0
(X(s))dB(s) (X(0))h +(X(0))[B(h) B(0)] (10.9)
So
E
x
[X(h) x] = (x)h +o(h)
and
E
x

(X(h) x)
2

=
2
(x)h +o(h)
as h . As a consequence, (x) is called the innitesimal drift of the diusion X at x and
2
(x) is the
innitesimal variance of X at x. Hence, a diusion / SDE is formulated (from a modeling viewpoint) by
specifying its innitesimal mean and variance functions.
10.5 Computing Expectations for Diusions
Expectations and probabilities can be computed in the diusion setting by solving ordinary or partial dif-
ferential equations. To determine the appropriate dierential equation, we use an analog to rst transition
analysis in the discrete time Markov chain setting. We illustrate this idea via several examples.
Example 10.1: Computing Exit Probabilities from an Interval
For a < x < b, compute
u(x) = P
x
{X(T) = a}
where T = inf{t 0 : X(t) / (a, b)}, is the exit time from (a, b). Note that u(a) = 1 and u(b) = 0. For
h > 0 and small,
u(x) = E
x
[u(X(h))] +o(h) (10.10)
Assuming u() is twice continuously dierentiable,
E
x
[u(X(h))] = u(x) +u

(x)E
x
[X(h) x] +
u

(x)
2
E
x

(X(h) x)
2

+o(h)
= u(x) +(x)u

(x)h +

2
(x)
2
u

(x)h +o(h)
as h . Plugging this into (10.10), we get
0 = (x)u

(x)h +

2
(x)
2
u

(x)h +o(h)
Dividing by h and letting h 0 we nd that:
0 = (x)u

(x) +

2
(x)
2
u

(x)
subject to u(a) = 1 and u(b) = 0. For example, if = 0 and
2
= 1 (so X is just standard Brownian
motion),
u(x) =
b x
b a
150
Example 10.2: Computing the Mean Exit Time from an Interval
Let u(x) = E
x
[T] where T is as in Example 1. For h > 0 and small,
u(x) = h +E
x
[u(X(h))] +o(h) (10.11)
Assuming u() is twice continuously dierentiable,
E
x
[u(X(h))] = u(x) +(x)u

(x)h +

2
(x)
2
u

(x)h +o(h)
as h 0. Plugging this into (10.11), subtracting u(x) from each side, dividing by h and sending h 0, we
get
1 = (x)u

(x) +

2
(x)
2
u

(x)
subject to the (obvious) boundary conditions that u(a) = u(b) = 0.
Example 10.3: Computing the Mean Reward up to the Exit From an Interval
Let
u(x) = E
x

T
0
r(X(s))ds

For h > 0 and small,

u(x) = r(x)h +E
x
[u(X(h))] +o(h)
Assuming u() is twice continuously dierentiable,
E
x
[u(X(h))] = u(x) +(x)u

(x)h +

2
(x)
2
u

(x)h +o(h)
as h 0. This leads to the ordinary dierential equation (ODE)
r(x) = (x)u

(x) +

2
(x)
2
u

(x)
subject to u(a) = u(b) = 0.
Example 10.4: Computing the Innite Horizon Discounted Reward
Let
u(x) = E
x

0
e
t
r(X(t))dt

for > 0. For h > 0 and small,

u(x) = E
x

h
0
e
s
r(X(s))ds +e
h

0
e
s
r(X(s +h))dx

= r(x)h +e
h
E
x
[u(X(h))] +o(h)
= r(x)h + (1 h)E
x
[u(X(h))] +o(h)
Assuming u() is twice continuously dierentiable,
E
x
[u(X(h))] = u(x) +(x)u

(x)h +

2
(x)
2
u

(x)h +o(h)
151
as h 0. This leads to the ODE:
r(x) = (x)u

(x) +

2
(x)
2
u

(x) u(x)
If r() is bounded, the solution u() must be bounded.
Example 10.5: Computing a Transient Expectation
Let
u(x, t) = E
x
[r(X(t))]
For h > 0 and small,
u(x, t) = E
x
[r(X(t))|X(u) : 0 u h]
= E
x
[r(X(t))|X(h)]
= E
x
[u(t h, X(h))]
Assuming that u() is smooth,
E
x
[u(t h, X(h))] u(t, x) =

t
u(t, x)h +

x
u(t, x)(x)h +
1
2

2
x
2
u(t, x)
2
(x)h +o(h)
Hence, we arrive at the partial dierential equation (PDE)
u
t
(t, x) = (x)u
x
(t, x) +

2
(x)
2
u
xx
(t, x)
subject to u(0, x) = r(x).
10.6 Multi-dimensional Diusions
Suppose that X
1
and X
2
jointly satisfy a coupled system of SDEs (B
1
, B
2
independent standard Brownian
motion).
dX
1
(t) =
1
(X
1
(t), X
2
(t))dt +
11
(X
1
(t), X
2
(t))dB
1
(t) +
12
(X
1
(t), X
2
(t))dB
2
(t)
dX
2
(t) =
2
(X
1
(t), X
2
(t))dt +
21
(X
1
(t), X
2
(t))dB
1
(t) +
22
(X
1
(t), X
2
(t))dB
2
(t)
The same analysis as followed above shows that
E
x,y
[X
1
(h) x] =
1
(x, y)h +o(h)
E
x,y

(X
1
(h) x)
2

2
11
(x, y) +
2
12
(x, y)

h +o(h)
E
x,y
[X
2
(h) y] =
2
(x, y)h +o(h)
E
x,y

(X
2
(h) x)
2

2
21
(x, y) +
2
22
(x, y)

h +o(h)
E
x,y
[(X
1
(h) x) (X
2
(h) y)] = (
11
(x, y)
21
(x, y) +
22
(x, y)
12
(x, y)) h +o(h)
Let K R
2
and let (x, y) K. To compute:
u(x, y) = E
x,y
[T]
152
where T = inf{t 0 : (X
1
(t), X
2
(t)) / K}, we solve the PDE

1
(x, y)u
x
(x, y) +
2
(x, y)u
y
(x, y)
+

2
11
(x, y) +
2
12
(x, y)
2
u
xx
(x, y) +

2
22
(x, y) +
2
21
(x, y)
2
u
yy
(x, y)
+ (
11
(x, y)
21
(x, y) +
22
(x, y)
12
(x, y)) u
xy
(x, y) = 1
Subject to u(x, y) = 0 on the boundary of K. Examples 1 through 4 lead to elliptic PDEs in two variables;
Example 5 leads to a parabolic PDE in two spacial variables.
More generally, if X
1
, . . . , X
d
jointly satisfy a coupled system of d SDEs, Example 1 through 4 lead to ellip-
tic PDEs in d variables; Example 5 leads to a parabolic PDE in d spacial variables. Thus, the full force of
numerical PDEs can be brought to bear on solving such problems.
Conversely, in solving elliptic and parabolic PDEs, we can represent the solutions to such PDEs as expecta-
tions of diusion processes. Hence, one means of solving such PDEs is via the Monte Carlo method. This is
an attractive solution methodology when dealing with high-dimensional elliptic and parabolic PDEs.
153
154