Handout 9 15/08/02 1

Lecture 7: Numerical solution of ODEs I

The model ODE
In this lecture we will be learning how to solve the first order ODE
dy
= f (t, y) . (1)
dt
The reason we analyze such a simplified equation is because, as we saw in the previous
lecture, all higher ODEs can be written in the form of a system of first order ODEs, which
we write as
dy
= F(t, y) . (2)
dt
These higher order systems can be solved using the same methods we develop to solve the
model ODE (1).

Forward and backward Euler: explicit vs. implicit methods

Discretization
The model ODE (1) is written discretely by choosing a time step (or space step) at which
we would like to evaluate both sides of the equation. Let’s say we want to evaluate both
sides of (1) at time step n. In this case, the model ODE would be written as

dy 
 = fn , (3)
dt n

where fn = f (tn , yn ). So far the discretization is exact. We have not made any approxima-
tions yet because we are assuming that we can evaluate everything exactly. If we approximate
the left hand side with the forward discrete derivative with

dy  yn+1 − yn
 = + O (∆t) , (4)
dt n ∆t

then we have the first order accurate approximate to the model equation (1) as
yn+1 − yn
= fn + O (∆t) , (5)
∆t
or  
yn+1 = yn + ∆tfn + O ∆t2 . (6)
This equation is known as the forward Euler method because it uses the forward discrete
derivative in time to evaluate the left hand side. Since in order to evaluate yn+1 , we use
information from time step n, this is known as an explicit method.
Handout 9 15/08/02 2

If we choose to write the model equation (1) at time step n + 1


dy 
 = fn+1 , (7)
dt n+1

then this can be approximated using the backward discrete derivative to yield
yn+1 − yn
= fn+1 + O (∆t) , (8)
∆t
or  
yn+1 = yn + ∆tfn+1 + O ∆t2 . (9)
This is known as the backward Euler method because is uses the backward finite difference
to evaluate the first derivative. If you were to evaluate y at time step n + 1 you would see
that you need information at time step n + 1 in order to compute fn+1 . When you need
information at the next time step, the method is known as an implicit method.

An example
Let’s say you want to numerically determine the evolution of the ODE
dy
= y cos y , (10)
dt
with y(0) = 1. If we use the forward Euler method, we have

yn+1 = yn + ∆tyn cos yn ,

yn+1 = yn (1 + ∆t cos yn ) . (11)

We can easily obtain y1 if y0 is known because everything on the right hand side is known
explicitly. If we use the backward Euler method, however, we have

yn+1 = yn + ∆tyn+1 cos yn+1 ,

yn+1 (1 − ∆t cos yn+1 ) = yn . (12)

Now, instead of having the solution of y1 in terms of y0 , we have a horrendous nonlinear

equation for y1 that must be solved using a nonlinear equation solver, such as Newton’s
method. Clearly, then, in this case, the explicit method is much faster than the implicit
method because we do not have to iterate at every time step to find the solution. The next
section shows the advantages of using implicit methods.

The linearized ODE

In the preceding example we saw how the forward Euler method was much easier and faster
to use than the backward Euler method. Any time something seems too good to be true
in numerical methods, it really is too good to be true. Which leads us to the first law of
numerical methods: There is no free lunch!. The problem with the forward Euler method,
Handout 9 15/08/02 3

despite its simplicity, is that it can be unstable, while the implicit backward Euler method
is unconditionally stable.
In order to study the stability of numerical methods for ODEs, we first need a model
equation that we can use to apply each method to and analyze its stability properties. This
model equation is the linear ODE
dy
= −λy , (13)
dt
where λ is some characteristic value of the ODE that arises from assuming that the ODE
behaves in this linear manner. We need to do this because we would like to analyze the
linear stability characteristics of numerical methods applied to all ODEs in general. Take
for example, the ODE used in the previous example,
dy
= y cos y . (14)
dt
In order to analyze the stability properties of this nonlinear ODE, we need to linearize it.
When we linearize an ODE, we analyze its behavior in the vicinity of some point t0 , y0 to
determine its stability properties. To analyze the behavior of an ODE in the vicinity of y0
and t0 , we make the subsitution y = y0 + y 0 and t = t0 + t0 , and assume that y 0 = y − y0 and
t0 = t − t0 represent very small quantities. Substituting these values into equation (14), we
have
dy dt0 d(y0 + y 0 )
= = (y0 + y 0 ) cos(y0 + y 0 ) . (15)
dt dt dt0
In order to linearize this, we need to use the Taylor Series approximation of the cosine
function  
cos(y0 + y 0 ) = cos(y0 ) − y 0 sin(y0 ) + O (y 0 )2 . (16)
Substitution into equation (15) yields
dy 0 0

0 2

+ (cos y 0 − y 0 sin y 0 ) y = y 0 cos y 0 + O (y ) . (17)
dt0
If we assume that y 0 is very small, then the second order term is negligible, and we have
dy 0
0
+ (cos y0 − y0 sin y0 ) y 0 = y0 cos y0 , (18)
dt
which is a linear inhomogeneous ODE in terms of y 0 and t0 that represents the behavior of
the original nonlinear ODE in equation (14) in the vicinity of y0 , t0 . If we substitute back
in the values for y 0 = y − y0 and t0 = t − t0 we have
dy
+ (cos y0 − y0 sin y0 ) y = 2y0 cos y0 − y02 sin y0 . (19)
dt
If we split the linearized solution into its homogeneous and particular parts with y = yh + yp ,
then the homogenous solution satisfies
dyh
= −λyh , (20)
dt
where λ = (cos y0 − y0 sin y0 ). If we analyze the stability properties of this linearized ODE,
then we can apply that analysis to the nonlinear problem by seeing if it remains stable at
all values of t0 and y0 .
Handout 9 15/08/02 4

Stability
If we apply the forward Euler method to the model linearized ODE
dy
= −λy , (21)
dt
then we have

yn+1 = yn − hλyn ,
= yn (1 − hλ) , (22)

where h = ∆t. If we write the amplification factor at each time step as

 
y 
 n+1 
Gn =   , (23)
 yn 

then, for the forward Euler method, we have

Gn = |1 − hλ| , (24)

where the vertical bars imply the modulus, to account for the possibility that λ may not
necessarily be real. If the amplification is less than 1, then we are guaranteed that the
solution will not grow without bound, and hence it will be stable. If we assume that λ is
real, then for stability we must have

−1 < 1 − hλ < +1 , (25)

which implies that, for stability, 0 < λh < 2, if λ is real. This translates to a time step
restriction for stability, for which 0 < ∆t < 2/λ.
Now consider the backward Euler method applied to the model linearized ODE. This
yields

yn+1 = yn − hλyn+1 ,
(1 + hλ) yn+1 = yn , (26)

and the amplification factor is given by

1 
 

Gn =  . (27)
1 + hλ 
If λ is real, then we must have λh > 0, or ∆t > 0. The backward Euler method is hence
stable in the linear sense for all ∆t! While it may be more expensive to use the implicit
method, as in the example discretization of equation (14), it is guaranteed to be stable.
The greatest drawback to the Euler methods is that they are first order accurate. In the
next sections, we derive more accurate methods to solve ODEs.
Handout 9 15/08/02 5

Euler predictor-corrector method

The improved Euler method is derived by integrating the model ODE from tn to tn+1 to
obtain Z tn+1 Z tn+1
dy
dt = f (y) dt . (28)
tn dt tn
Using the trapezoidal rule, we can approximate the above integral to third order accuracy
with
∆t  
yn+1 − yn = (fn + fn+1 ) + O ∆t3 . (29)
2
to obtain the second order accurate approximation to the model ODE as
yn+1 − yn 1  
= (fn + fn+1 ) + O ∆t2 . (30)
∆t 2
As it is, this method is an implicit method because we need information at time step n + 1
in order to evaluate the right hand side. Instead of using fn+1 , we will use a predicted value,
f∗ = f (y∗ ), where y∗ is obtained with the forward Euler predictor step
y∗ = yn + ∆tfn . (31)
The Euler predictor-corrector method is then given in two steps:
 
Predictor: y∗ = yn + ∆tfn + O ∆t2 ,
∆t  
Corrector: yn+1 = yn + (fn + f∗ ) + O ∆t3 . (32)
2
This method is second order accurate, since y∗ approximates yn+1 to second order accuracy.
Substituting y∗ = yn+1 + O (∆t2 ) into f∗ yields
f∗ = f (y∗ ) ,
 
= f (yn+1 + O ∆t2 ) ,
 
= f (yn+1 ) + O ∆t3 .
Substituting this result into the corrector yields
yn+1 − yn 1  
= (fn + fn+1 ) + O ∆t2 , (33)
∆t 2
which is identical in accuracy to equation (30).

Runge-Kutta methods
The Runge-Kutta methods are the most popular methods of solving ODEs numerically.
They can be derived for any order of accuracy, but we will derive the second order method
first. The second order Runge-Kutta method is derived by taking two steps to get from n
to n + 1 with
yn+1 = yn + ak1 + bk2 ,
k1 = hf (tn , yn ) ,
k2 = hf (tn + αh, yn + βk1 ) , (34)
Handout 9 15/08/02 6

where h = ∆t is the time step. In order to determine what the constants a, b, α, and β
are, we must use the Taylor series to match the terms and make the method second order
accurate. By substituting in for k1 and k2 , we have

yn+1 = yn + ahf (tn , yn ) + bhf [tn + αh, yn + βhf (tn , yn )] . (35)

In order to expand the third term in equation (35), we need to use the Taylor series expansion
of a function of more than one variable, which is given by
∂f ∂f
f (t + ∆t, y + ∆y) = f (t, y) + ∆t + ∆y + O (∆t∆y) , (36)
∂t ∂y
which, when applied to the third term in equation (35), results in
∂f ∂f
f [tn + αh, yn + βhf (tn , yn )] = f + αh + βhf , (37)
∂t ∂y
where all functions and derivatives are evaluated at time step n, and we have left off the
truncation error. Substituting this into equation (35) results in
∂f ∂f
yn+1 = yn + h(a + b)f + αbh2 + βbh2 f . (38)
∂t ∂y
Since y is only dependant on the variable t, then the Taylor series expansion about yn+1 is
given by the ordinary derivatives with
dy h2 d2 y  
yn+1 = yn + h + 2
+ O h3 . (39)
dt 2 dt
But since the ODE we are trying to solve is given by
dy
=f, (40)
dt
then we know that
d2 y df
2
= , (41)
dt dt
so equation (39) becomes
h2 df
yn+1 = yn + hf + , (42)
2 dt
where we have left off the truncation error. Since from the chain rule, if f is a function of t
and y, then
∂f ∂f
df = dt + dy , (43)
∂t ∂y
then
df ∂f ∂f dy ∂f ∂f
= + = + f. (44)
dt ∂t ∂y dt ∂t ∂y
Substitution into equation (42) yields
h2 ∂f h2 ∂f
yn+1 = yn + hf + + f . (45)
2 ∂t 2 ∂y
Handout 9 15/08/02 7

Comparing this equation to equation (38),

h2 ∂f h2 ∂f
yn+1 = yn + hf + + f ,
2 ∂t 2 ∂y
∂f ∂f
yn+1 = yn + h(a + b)f + αbh2 + βbh2 f , (46)
∂t ∂y
in order for the terms to match, we must have

a + b = 1,
1
αb = ,
2
1
βb = . (47)
2
This is a system of three equations in four unknowns. Therefore, we are free to choose one
independantly and the others will then be determined, and the method will still be a second
order method. If we let a = 1/2, then the other parameters must be b = 1/2, α = 1, and
β = 1, so that the second order Runge-Kutta method is given by
1 1
yn+1 = yn + k1 + k2 ,
2 2
k1 = hf (tn , yn ) ,
k2 = hf (tn + h, yn + hf (tn , yn )) , (48)

which is just the Euler predictor-corrector scheme, since k2 = hf∗ , and substitution results
in
h
yn+1 = yn + (fn + f∗ ) . (49)
2
Higher order Runge-Kutta methods can be derived using the same technique. The most
popular method is the fourth order Runge-Kutta method, or RK4 method, which is given
by
1
yn+1 = yn + (k1 + 2k2 + 2k3 + k4 ) ,
6
k1 = hf (tn , yn ) ,
!
h 1
k2 = hf tn + , yn + k1 ,
2 2
!
h 1
k3 = hf tn + , yn + k2 ,
2 2
k4 = hf (tn + h, yn + k3 ) .

Although this method is a fourth order accurate approximation to the model ODE, it requires
four function evaluations at each time step. Again, there is never any free lunch!