MAT 461/561: 12.

4 Convergence Analysis

James V. Lambers

April 8-13-15, 2020

Announcements

Convergence Analysis
So far, we’ve seen convergence analysis for Euler’s Method–proving that as h → 0, the error

en = yn − y(tn )

across all time steps 0 ≤ n ≤ (T − t0 )/h, goes to 0.

To help show that a one-step or multistep method is convergent, we use the related concepts of

• local truncation error: the error incurred a single time step, measured by substitute the
exact solution into the numerical method, and see what is “left over” (after dividing by h!)
Example: Euler’s Method yn+1 = yn + hf (tn , yn ) has local truncation error
2
y(tn+1 ) − y(tn ) hy 0 (tn ) + h2 y 00 (ξ) h
τn (h) = − f (tn , y(tn )) = − f (tn , y(tn )) = y 00 (ξ) = O(h)
h h 2

• consistency: A method is consistent if local truncation error → 0 as h → 0

• stability: A method is stable if a change in the initial data y0 leads to a comparable change
in the solution at any time tn , in a way that is independent of h

Consistency: One-Step Methods

A general one-step method has the form

yn+1 = yn + hΦ(tn , yn , yn+1 , h)

Example: Euler’s Method, Φ(tn , yn , yn+1 , h) = f (tn , yn ). The local truncation error (LTE) is

y(tn+1 ) − y(tn )
τn (h) = − Φ(tn , y(tn ), y(tn+1 ), h)
h
As h → 0, we get
τn (h) = y 0 (tn ) − Φ(tn , y(tn ), y(tn ), 0)
so the method is consistent if Φ(t, y, y, 0) = f (t, y).

1
Example: Midpoint Method
 
h h
yn+1 = yn + hf tn + , yn + f (tn , yn )
2 2
has  
h h
Φ(tn , yn , yn+1 , h) = f tn + , yn + f (tn , yn )
2 2
Let h = 0:
Φ(tn , yn , yn , 0) = f (tn , yn )
therefore it is consistent.
Example: (implicit) Trapezoidal Method:
h
yn+1 = yn + [f (tn , yn ) + f (tn + h, yn+1 )]
2
Let h = 0:
1 1
Φ(tn , yn , yn+1 , h) = [f (tn , yn ) + f (tn + h, yn+1 )] → [f (tn , yn ) + f (tn , yn )] = f (tn , yn )
2 2
What about the order of accuracy? Use Taylor expansion on LTE. Example 12.4.2, Midpoint
Method

Consistency: Multistep Methods

Recall general multistep method:
m
X m
X
αi yn+1−i = h βi fn+1−i .
i=0 i=0
Local truncation error:
Pm
−h m
P
i=0 αi y(tn+1−i ) i=0 βi f (tn+1−i , y(tn+1−i ))
τn (h) = Pm
h i=0 βi
Adams Methods:
m
y(tn+1 ) − y(tn ) X
τn (h) = − βi f (tn+1−i , y(tn+1−i ))
h
i=0
Keep in mind that
Z tn+1 m
X Z tn+1
y(tn+1 ) − y(tn ) = f (s, y(s)) ds, h βi fn+1−i = p(s) ds
tn i=0 tn

where p(t) is a polynomial that interpolates f (s, y(s)) at tn , tn−1 , . . . , tn+1−m (and tn+1 for the
implicit case). Therefore, in the explicit case,
1 tn+1 f (m) (ξ(s), y(ξ(s)))
Z
τn (h) = (s − tn )(s − tn−1 ) · · · (s − tn−m+1 ) ds
h tn m!
so it’s just 1/h times the integral of the polynomial interpolation error (Theorem 7.4.1). In the
implicit case, m → m+1, include (s−tn+1 ). Using the Weighted Mean Value Theorem for Integrals:
Z 1
f (m) (ξ(s), y(ξ(s))) m m
τn (h) = h (−1) (u − 1)(u − 2) · · · (u − m) du
m! 0
in the explicit case; in the implicit case, add factor of u to integrand.

2
For a general multistep method, by Taylor expansion around tn+1 , it can be seen that the local
truncation error is given by the formula under equation (12.29), except it must be negated.

Stability
Stability for One-Step Methods
The problem y 0 = f (t, y) is well-posed (solution depends continuously on the data) if f is Lips-
chitz continuous (and problem has a unique solution) in y, meaning
|f (t, y1 ) − f (t, y2 )| ≤ L|y1 − y2 |
Similarly, a general one-step method
yn+1 = yn + hΦ(tn , yn , yn+1 , h)
is stable if Φ satisfies a Lipschitz condition (is Lipschitz continuous) in the yn and yn+1 variables.
If so, then given two initial values y0 , ỹ0 for the same ODE,
|yn+1 − ỹn+1 | ≤ |yn − ỹn | + h|Φ(tn , yn , yn+1 , h) − Φ(tn , ỹn , ỹn+1 , h)|
If Φ is Lipschitz continuous in yn and yn+1 , meaning
|Φ(tn , yn , yn+1 , h) − Φ(tn , ỹn , ỹn+1 , h)| ≤ L1 |yn − ỹn | + L2 |yn+1 − ỹn+1 |,
then
   
1 + hL1 h(L1 + L2 ) h(L1 + L2 )
|yn+1 − ỹn+1 | ≤ |yn − ỹn | ≤ 1+ |yn − ỹn | ≤ 1 + |yn − ỹn |
1 − hL2 1 − hL2 δ
We assume 1 − hL2 ≥ δ > 0, so let h0 = (1 − δ)/L2 . We require above for h ≤ h0 . We then have
h(L1 + L2 ) n
 
|yn − ỹn | ≤ 1 + |y0 − ỹ0 | ≤ e(T −t0 )(L1 +L2 )/δ |y0 − ỹ0 |
δ
Therefore the method is stable, with K = (T − t0 )(L1 + L2 )/δ.
Example: show Midpoint Method
 
h
yn+1 = yn + hf tn , yn + f (tn , yn )
2
is stable. This requires showing that
 
h
Φ(t, y, h) = f t, y + f (t, y)
2
is Lipschitz continuous in y. We assume f has a Lipschitz constant of L. We have
    
 h h 
|Φ(t, y2 , h) − Φ(t, y1 , h)| = f t, y2 + f (t, y2 ) − f t, y1 + f (t, y1 ) 
2 2
 
 h h 
≤ L y2 + f (t, y2 ) − y1 + f (t, y1 )
2 2
hL
≤ L|y2 − y1 | + |f (t, y2 ) − f (t, y1 )|
2
hL2
 
≤ L + |y2 − y1 |.
2
h0 L2
Therefore Φ is Lipschitz continuous in y with Lipschitz constant LΦ = L + 2 for h ≤ h0 .

3
Stability for Multistep Methods
Recall a general multistep method
m
X m
X
αi yn+1−i = h βi fn+1−i
i=0 i=0

How to show this is stable? We check if it is zero stable. Consider the ODE y 0 = 0, y(t0 ) = y0 .
The solution is y(t) = y0 . The multistep method applied to this problem is
m
X
αi yn+1−i = 0
i=0

This is a (m + 1)-term recurrence relation. We must ensure the solution of this recurrence relation
does not grow with n. The characteristic equation of the recurrence relation is

α0 λm + α1 λm−1 + · · · + αm−1 λ + αm = 0

and the solution has the form

m
X
yn = ci npi λni
i=1

where λ1 , . . . , λm are the roots of the characteristic equation and pi indicate multiplicity (that is,
pi = 0, 1, . . . , mi − 1 where mi is the multiplicity of λi . To ensure the solution does not grow with
n, the roots must satisfy the root condition:

• |λi | ≤ 1

• If |λi | = 1, then it must be a simple root (multiplicity 1)

A multistep method is zero-stable if and only if it satisfies the root condition. It is strongly zero-
stable if λ = 1 is the only root on the unit circle (that is, |λ| = 1), and weakly stable otherwise.
Note: for consistency, 1 is always a root!
Example: An Adams method has the characteristic equation λm − λm−1 = 0. Therefore the roots
are λ = 1 (simple root) and λ = 0 (repeated m − 1 times). Therefore all Adams methods are
strongly stable.
Example: Simpson’s Method
h
yn+1 = yn−1 + (fn+1 + 4fn + fn−1 )
3
Has the characteristic equation λ2 − 1 = 0, with roots λ = ±1. Therefore it is weakly stable.
A m-step BDF is zero-unstable for m > 6.

4
Convergence
If a one-step or multistep method is consistent and stable, it is convergent. For multistep methods, if
it is consistent, then it is convergent if and only if it is zero-stable (Dahlquist equivalence theorem).
For one-step methods, if Φ has a Lipschitz constant LΦ , meaning

|Φ(t, yn , yn+1 , h) − Φ(t, ỹn , ỹn+1 , h)| ≤ LΦ [|yn − ỹn | + |yn+1 − ỹn+1 |]

then we have
|yn − ỹn | ≤ e2LΦ (T −t0 )/δ |y0 − ỹ0 |
from stability, and for h ≤ h0 = (1 − δ)/LΦ , we have
 
2hLΦ
|yn+1 − ỹn+1 | ≤ 1 + |yn − ỹn |.
δ

Following the convergence proof for Euler’s Method, we let en = yn − y(tn ) be the global error,
then taking into account

y(tn+1 ) = y(tn ) + hΦ(tn , y(tn ), y(tn+1 ), h) + hτn (h)

we have  
2hLΦ
|en+1 | ≤ 1+ |en | + h|τn (h)|
δ
Therefore !
e2LΦ (T −t0 )/δ − 1
|en | ≤ δ max |τn (h)|
2LΦ n

Therefore the one-step method is convergent.

For both one-step and multistep methods, if local truncation error is O(hp ), and if method is stable,
then global error is O(hp ).

Stiff Differential Equations

What does it mean for an ODE to be stiff? An ODE y 0 = f (t, y) is stiff if its exact solution y(t) has
terms that decay to zero as t → ∞ but have very large derivatives. Example: a term of the form
e−ct where c is large, for the kth derivative is (−c)k e−ct . Keep in mind a time-stepping method of
pth-order accuracy involves y (p+1) in its error formula. Therefore the error can be large when the
ODE is stiff.
A consequence of this is that the time step h necessary to compute the solution correctly is far
smaller than the time step h needed to depict (represent) the solution (e.g. solution is mostly very
smooth, but has small terms with large derivatives)
Stiff ODEs are problematic for numerical methods because they require a small h, which increases
overall computation time.
Example: y 0 = −100y, y(0) = 1. Exact: y(t) = e−100t , decays to zero rapidly. With Euler’s Method:

yn+1 = yn + h(−100yn ) = (1 − 100h)yn .

Let h = 0.1, then yn+1 = (−9)yn so yn = (−9)n . Let h = 10−3 , yn = 0.9n . Need even smaller h!

5
To analyze methods for the purpose of determining a suitable h, we use the test equation

y 0 = λy, y(0) = 1, Re λ < 0

Then y(t) = eλt , but if |Re λ| is large, this ODE is stiff.

One-step methods: when applied to the test equation, we have

yn+1 = Q(hλ)yn ,

where Q(hλ) is a polynomial in hλ if method is explicit, or rational function if implicit. It is an

approximation of eλh . We require exponential decay, so we require

|Q(hλ)| < 1

for a qualitatively correct solution.

Example: Modified Euler Method (or: Explicit Trapezoidal Method)
h
yn+1 = yn + [f (tn , yn ) + f (tn + h, yn + hf (tn , yn ))]
2
Apply to test equation:
 
h 1 2 2
yn+1 = yn + [λyn + λ(yn + hλyn )] = 1 + hλ + h λ yn
2 2

To ensure Q(hλ) = 1 + hλ + 12 (hλ)2 satisfies |Q(hλ)| < 1, this is equivalent to finding the interval
in x on which
1
0 < 2 + x + x2 < 2
2
2 2
so we must have x + x /2 < 0, and x + x /2 > −2. First part: x(1 + x/2) < 0 so 1 + x/2 > 0 or
x < −2. So we must have 0 < |hλ| < 2.
Multistep methods: apply a general multistep method to test equation to get
m
X
(αi − hλβi )yn+1−i = 0.
i=0

This recurrence relation has the characteristic equation involving the stability polynomial

Q(µ, hλ) = (α0 − hλβ0 )µm + (α1 − hλβ1 )µm−1 + · · · + (αm − hλβm ) = 0.

To ensure exponential decay, must have roots µi of |Q(µ, hλ)| satisfy |µi | < 1 for i = 1, . . . , m.
Example: 3-step Adams-Bashforth Method
h
yn+1 = yn + [23fn − 16fn−1 + 5fn−2 ].
12
The stability polynomial is
 
3 23 4 5
Q(µ, hλ) = µ + −1 − hλ µ2 + hλµ − hλ.
12 3 12
Let λ = −100. For h = 0.1, largest root is µ ≈ −18.884. Choose h = 0.005, then all roots < 1 in
magnitude.

6
Region of Absolute Stability
The region of absolute stability of a one-step or multistep method is the region R of the complex
plane such that hλ ∈ R “works”, meaning |Q(hλ)| < 1 for a one-step method, or |Q(µ, hλ)| has
roots < 1 in magnitude for a multistep method. If R includes the entire left-half plane (real part
negative) then the method is A-stable.
Example: Backward Euler on the test equation:

yn+1 = yn + hλyn+1

or  
1
yn+1 = yn .
1 − hλ
For any λ with Re λ < 0, |Q(hλ)| = |1/(1 − hλ)| < 1 so it is A-stable. Also: Implicit Trapezoidal
Method (2nd-order) is A-stable.

Dahlquist’s Barrier Theorems

For multistep methods:

• First Barrier Theorem: if a m-step multistep method is zero-stable, its order of accuracy is
at most m + 1 if m is odd, and m + 2 if m is even.

• Second Barrier Theorem: NO explicit A-stable method. NO A-stable method of order greater
than 2. The A-stable second-order method with the smallest asymptotic error constant is the
Implicit Trapezoidal Method.