Finite Differences for Differential Equations 28

PART II

Finite Difference Methods for Differential

Equations
Finite Differences for Differential Equations 29

B OUNDARY VALUE P ROBLEMS (I)

• Solving a TWO – POINT BOUNDARY VALUE PROBLEM with D IRICHLET
B OUNDARY C ONDITIONS :
d2y
=g for x ∈ (0, 2π)
dx2
y(0) = y(2π) = 0

• Finite–difference approximation:
– Second–order center difference formula for the interior nodes:
y j+1 − 2y j + y j−1
= g j for j = 1, . . . , N
h2
2π
where h = N+1 and x j = jh
– Endpoint nodes:
y0 = 0 =⇒ y2 − 2y1 = h2 g1
yN+1 = 0 =⇒ −2yN + yN−1 = h2 gN

– Tridiagonal algebraic system — solved very efficiently with the

T HOMAS ALGORITHM (a version of the Gaussian elimination)
Finite Differences for Differential Equations 30

B OUNDARY VALUE P ROBLEMS (II)

• Solving a TWO – POINT BOUNDARY VALUE PROBLEM with N EUMANN
B OUNDARY C ONDITIONS :
d2y
=g for x ∈ (0, 2π)
dx2
dy dy
(0) = (2π) = 0
dx dx
• Finite–difference approximation:
– Second–order center difference formula for the interior nodes:
y j+1 − 2y j + y j−1
= g j for j = 1, . . . , N
h2
– First–order Forward/Backward Difference formulae to re–express
endpoint values: y1 − y0
= 0 =⇒ y0 = y1
h
yN+1 − yN
= 0 =⇒ yN+1 = yN
h
First–order only — DEGRADED ACCURACY !

– Tridiagonal algebraic system — Is there any problem? Where?

Finite Differences for Differential Equations 31

B OUNDARY VALUE P ROBLEMS (III)

• In order to retain the SECOND – ORDER ACCURACY in the approximation of

the Neumann problem need to use higher-order formulae at endpoints, e.g.
−y2 + 4y1 − 3y0 1
y′0 = = 0 =⇒ y0 = (−y2 + 4y1 )
2h 3

• The first row thus becomes

2 2
y2 − y1 = h2 g1
3 3
S ECOND – ORDER ACCURACY RECOVERED!
Finite Differences for Differential Equations 32

B OUNDARY VALUE P ROBLEMS (IV)

• C OMPACT S TENCILS — stencils based on three grid points (in every
direction) only: {x j+1 , x j , x j−1 } at the j − th node
• Is is possible to obtain higher (then second) order of accuracy on compact
stencils? — YES !
d2 y
• Consider the central difference approximation to the equation dy2
=g

y j+1 − 2y j + y j−1 h2 (iv)

2
− y j + O (h4 ) = g j
h 12
h 2 (iv)
• Re-express the error term 12 y j using the equation in question:
h2 (iv) h2 ′′ h2 g j+1 − 2g j + g j−1 h2 (iv)
 
yj = gj = 2
− g j + O (h4 )
12 12 12 h 12

• Inserting into the original finite–difference equation:

y j+1 − 2y j + y j−1 g j+1 − 2g j + g j−1
2
= g j + + O (h4 )
h 12
• Slight modification of the RHS =⇒ FOURTH — ORDER ACCURACY !!!
Finite Differences for Differential Equations 33

B OUNDARY VALUE P ROBLEMS (V)

• C OMPACT F INITE D IFFERENCE S CHEMES —

– A DVANTAGES :
∗ Increased accuracy on compact grids
– DRAWBACKS:
∗ need to be tailored to the specific equation solved
∗ can get fairly complicated for more complex equations
Finite Differences for Differential Equations 34

I NITIAL VALUE P ROBLEMS —

G ENERAL R EMARKS

• Consider the following C AUCHY PROBLEM :

dy
= f (y,t) with y(t0 ) = y0
dt
The independent variable t is usually referred to as TIME .

• Equations with higher–order derivatives can be reduced to systems of

first–order equations

• Generalizations to systems of ODEs straightforward

• When the RHS function does not depend on y, i.e., f (y,t) = f (t),
solution obtained via a QUADRATURE

• Assume uniform time–steps ( h is constant )

Finite Differences for Differential Equations 35

I NITIAL VALUE P ROBLEMS —

C HARACTERIZATION OF I NTEGRATION
M ETHODS

• ACCURACY — unlike in the Boundary Value Problems, there is no terminal

condition and approximation errors may accumulate in time; consequently, a
relevant characterization of accuracy is provided by the GLOBAL ERROR
(global error) = (local error) × (# of time steps),

rather than the LOCAL ERROR .

• STABILITY — unlike in the Boundary Value Problems, where boundedness

of the solution at final time is enforced via a suitable terminal condition , in
Initial Value Problems there is a priori no guarantee that the solution will
remain bounded.
Finite Differences for Differential Equations 36

I NITIAL VALUE P ROBLEMS —

M ODEL P ROBLEM
• S TABILITY of various numerical schemes is usually analyzed by applying
these schemes to the following LINEAR MODEL :
dy
= λy = (λr + iλi )y with y(t0 ) = y0 ,
dt
which is stable when λr <= 0 .
λ 2 h2 λ 3 h3
 
y(t): = y0 eλt = 1 + λh +
• E XACT SOLUTION + + . . . y0
2 6

• M OTIVATION — consider the following ADVECTION – DIFFUSION PDE :

∂u ∂u ∂2 u
+c −a 2 = 0
∂t ∂x ∂x
Taking Fourier transform yields (k is the wavenumber):
d ûk
+ c i k ûk + a k2 ûk = 0
dt
where
– the real term a k2 ûk represents DIFFUSION
– the imaginary term c i k ûk represents ADVECTION
Finite Differences for Differential Equations 37

I NITIAL VALUE P ROBLEMS —

E XPLICIT E ULER S CHEME (I)

• Consider a Taylor series expansion

′ h2 ′′
y(tn+1 ) = y(tn ) + hy (tn ) + y (tn ) + . . .
2
Using the ODE we obtain
dy
y′ = =f
dt
′′ dy′ df
y = = = ft + f fy
dt dt
• Neglecting terms proportional to second and higher powers of h yields the
E XPLICIT E ULER M ETHOD
yn+1 = yn + h f (yn ,tn )

• Retaining higher–order terms is inconvenient, as it requires differentiation of

f and does not lead to schemes with desirable stability properties.
Finite Differences for Differential Equations 38

I NITIAL VALUE P ROBLEMS —

E XPLICIT E ULER S CHEME (II)

• L OCAL ERROR analysis:

yn+1 = (1 + λh) yn + [O (h2 )]

• G LOBAL ERROR analysis:

T
(global error) = Ch2 · N = Ch2 · = C′ h
h
Thus, the scheme is
– locally second–order accurate
– globally (over the interval [t0 ,t0 + Nh]) first–order accurate
Finite Differences for Differential Equations 39

I NITIAL VALUE P ROBLEMS —

E XPLICIT E ULER S CHEME (III)

• Stability (for the model problem)

yn+1 = yn + λhyn = (1 + λh)yn

Thus, the solution after n time steps

yn = (1 + λh)n y0 , σn y0 =⇒ σ = 1 + λh

For large n, the numerical solution remains stable iff

|σ| ≤ 1 =⇒ (1 + λr h)2 + (λi h)2 ≤ 1

– CONDITIONALLY STABLE for real λ

– UNSTABLE for imaginary λ
Finite Differences for Differential Equations 40

I NITIAL VALUE P ROBLEMS —

I MPLICIT E ULER S CHEME (I)

• I MPLICIT S CHEMES — based on approximation of the RHS that involve

f (yn+1 ,t), where yn+1 is the unknown to be determined
• I MPLICIT E ULER S CHEME — obtained by neglecting second and
higher–order terms in the expansion:

′ h2 ′′
y(tn ) = y(tn+1 ) − hy (tn+1 ) + y (tn+1 ) − . . .
2
dy
Upon substitution dt t = f (yn+1 ,tn+1 ) we obtain
n+1

yn+1 = yn + h f (yn+1 ,tn+1 )

The scheme is
– locally SECOND – ORDER accurate
– globally (over the interval [t0 ,t0 + Nh]) FIRST– ORDER accurate
Finite Differences for Differential Equations 41

I NITIAL VALUE P ROBLEMS —

I MPLICIT E ULER S CHEME (II)

• Stability (for the model problem):

yn+1 = yn + λhyn+1 =⇒ yn+1 = (1 − λh)−1 yn
 n
1 1
yn+1 = y0 , σn y0 =⇒ σ =
1 − λh 1 − λh
|σ| ≤ 1 =⇒ (1 − λr h)2 + (λi h)2 ≥ 1

Implicit Euler scheme is thus stable for

– all stable model problems
– most unstable model problems

• R EMARK : When solving systems of ODEs of the form y = A (t)y, each

implicit step requires solution of an algebraic system: yn+1 = (I − hA )−1 yn

• Implicit schemes are generally hard to implement for nonlinear problems

Finite Differences for Differential Equations 42

I NITIAL VALUE P ROBLEMS —

C RANK –N ICHOLSON S CHEME (I)
• Obtained by approximating the formal solution of the ODE
R tn+1
yn+1 = yn + tn f (y,t) dt using the TRAPEZOIDAL QUADRATURE :
h
yn+1 = yn + [ f (yn ,tn ) + f (yn+1 ,tn+1 )]
2
The scheme is
– locally THIRD – ORDER accurate
– globally (over the interval [t0 ,t0 + Nh]) SECOND – ORDER accurate
• Stability (for the model problem):
λh
!
λh 1+ 2
yn+1 = yn + (yn+1 + yn ) =⇒ yn+1 = λh
yn
2 1− 2
λh n
1 + λh
!
1+ 2
yn+1 = y0 , σ n
y0 =⇒ σ = 2
1 − λh
2 1 − λh
2
|σ| ≤ 1 =⇒ ℜ(λh) ≤ 0

S TABLE for all model ODEs with stable solutions

Finite Differences for Differential Equations 43

I NITIAL VALUE P ROBLEMS —

L EAPFROG S CHEME (I)

• L EAPFROG as an example of a TWO – STEP METHOD :

yn+1 = yn−1 + 2 h λ yn

• C HARACTERISTIC EQUATION for the AMPLIFICATION FACTOR (yn = σn y0 )

σ2 − 2 h λσ − 1 = 0

where roots give the amplification factors:

λ 2 h2
+ . . . = eλh + O (h3 )
p
σ1 = λh + 1 + λ2 h2 ≃ 1 + λh +
2
p λ 2 h2
σ2 = λh − 1 + λ h ≃ −(1 − λh +
2 2 − . . . ) = −e−λh + O (h3 )
2
Thus, the scheme is
– locally THIRD – ORDER accurate
– globally (over the interval [t0 ,t0 + Nh]) SECOND – ORDER accurate
Finite Differences for Differential Equations 44

I NITIAL VALUE P ROBLEMS —

L EAPFROG S CHEME (II)

• Stability for diffusion problems ( λ = λr ):

q
σ1 = λh + 1 + λ2r h2 > 1 for all h > 0

Thus the scheme is UNCONDITIONALLY UNSTABLE for diffusion problems!

• Stability for advection problems ( λ = iλi ):
1
σ21/2 = 1 (!!!) for h <
|λi |
Thus, the scheme is CONDITIONALLY UNSTABLE and NON – DIFFUSIVE for
advection problems!

• Q UESTION — analyze dispersive (i.e., related to arg(σ)) errors of the

leapfrog scheme.
Finite Differences for Differential Equations 45

I NITIAL VALUE P ROBLEMS —

M ULTISTEP P ROCEDURES
• General form of a MULTISTEP PROCEDURE :
p q
∑ α j yn+ j = h ∑ β j f (yn+ j ,tn+ j )
j=1 j=1

with characteristic polynomials

ξ p (z) = α p z p + α p−1 z p−1 + · · · + α0
ζq (z) = βq zq + βq−1 zq−1 + · · · + β0

– if p > q — EXPLICIT SCHEME

– if p ≤ q — IMPLICIT SCHEME

• A (ξ, ζ) –procedure converges uniformly in [a, b], i.e.,

limh→0 maxtn ∈[a,b] |yn − y(tn )| = 0 if:
– the following consistency conditions are verified: ξ(1) = 0 and
ξ′ (1) = ζ(1) ( CONSISTENCY CONDITION )
– all roots of the polynomial ξ(z) are such that |zi | ≤ 1 and the roots with
|zk | = 1 are simple ( STABILITY CONDITION )
Finite Differences for Differential Equations 46

I NITIAL VALUE P ROBLEMS —

RUNGE –K UTTA M ETHODS (I)
• General form of a FRACTIONAL STEP METHOD :
yn+1 = yn + γ1 h k1 + γ2 h k2 + γ3 h k3 + . . .
where
k1 = f (yn ,tn )
k2 = f (yn + β1 hk1 ,tn + α1 h)
k3 = f (yn + β2 hk1 + β3 hk2 ,tn + α2 h)
.
.
.
• Choose γi , βi and αi to match as many expansion coefficients as possible in
′ h2 ′′ h3 ′′′
y(tn+1 ) = y(tn ) + hy (tn ) + y (tn ) + y (tn ) . . .
2 6
y′ = f
y′′ = ft + f fy
y′′′ = ftt + ft fy 2 f fyt + f 2 fyt + f 2 fyy

• Runge—Kutta methods are SELF – STARTING with fairly good stability and
accuracy properties.
Finite Differences for Differential Equations 47

I NITIAL VALUE P ROBLEMS —

RUNGE –K UTTA M ETHODS (II)
• RK4 — an ODE “workhorse”:
h h h
yn+1 = yn + k1 + (k2 + k3 ) + k4
6 3 6
h
k1 = f (yn ,tn ) k2 = f (yn + k1 ,tn+1/2 )
2
h
k3 = f (yn + k2 ,tn+1/2 ) k4 = f (yn + hk3 ,tn+1 )
2

• The amplification factor:

λ 2 h2 λ 3 h3 λ 4 h4
σ = 1 + λh + + +
2 6 24
Thus, stability iff |σ| ≤ 1
• ACCURACY:
eλh = σ + O (h5 )
Thus, the scheme is
– locally FIFTH – ORDER accurate
– globally (over the interval [t0 ,t0 + Nh]) FOURTH – ORDER accurate
Finite Differences for Differential Equations 48

I NITIAL VALUE P ROBLEMS —

RUNGE ’ S P RINCIPLE

• Let (k + 1) be the order of the local truncation error; denote Y (t, h) an

approximation of the exact solution y(t) computed with the step size h; then
at t = t0 + 2nh:
y(t) −Y (t, h) ≃ C 2 n hk+1 = C(t − t0 )hk
y(t) −Y (t, 2h) ≃ C n (2h)k+1 = C(t − t0 )2k hk

Subtracting:
Y (t, 2h) −Y (t, h) ≃ C(t − t0 )(1 − 2k )hk

Thus, we can obtain an estimate of the ABSOLUTE ERROR based on solution

with two step–sizes only:
Y (t, h) −Y (t, 2h)
y(t) −Y (t, h) ≃
2k − 1
• Runge’s principle is very useful for ADAPTIVE STEP SIZE REFINEMENT
Finite Differences for Differential Equations 49

I NITIAL VALUE P ROBLEMS —

L AX E QUIVALENCE T HEOREMa
• Consider an I NITIAL VALUE P ROBLEM
du
= L u with u(t0 ) = u0
dt
and assume that it is well–posed, i.e., it admits solutions which are unique
and stable
• Consider a numerical method defined by a finite–difference operator C(h)
such that the approximate solution is given by

uh (nh) = C (h)n u0 , n = 1, 2, . . .
C (h)−I
• The above method is CONSISTENT iff h is a convergent approximation
of the operator L
• L AX T HEOREM — For a CONSISTENT difference method STABILITY is
equivalent to CONVERGENCE
a For a more technical discussion, see $ 5.2 in Atkinson & Han (2001)
Finite Differences for Differential Equations 50

I NITIAL VALUE P ROBLEMS —

C ONSERVATION P ROPERTIES (I)
• Is ACCURACY and S TABILITY all that matters?
• C ONSERVATION P ROPERTIES — conservation by the numerical method
(i.e., in the discrete sense) of various invariants the original equation may
possess
– R EMARK — conservation properties are particularly relevant for solution
of Hamiltonian / hyperbolic systems
• Example — conservation of the solution norm:
– In the continuous setting (assume u = |u|eiϕ )

d|u|
= 0 =⇒ |u(t)| = |u0 |,


du 
dt
= iλi u ⇐⇒
dt dϕ
= λi ,



dt
– In the discrete setting: |uh (nh)| = |uh ((n − 1)h)| = · · · = |uh (0)|
Necessary and sufficient condition for discrete conservation: ∃h, |σ(h)| = 1
Finite Differences for Differential Equations 51

I NITIAL VALUE P ROBLEMS —

C ONSERVATION P ROPERTIES (II)
• Implicit Euler —
1 1 1
|σ| = =q = 1 − λ2i h2 + · · · < 1 for all h
1 − iλi h 1 + λ2i h2 2

The scheme is thus DISSIPATIVE (i.e., not conservative)

• Fourth–Order Runge–Kutta —
λ2i h2 λ3i h3 λ4i h4 1
q
|σ| = 1 + iλi h − −i + = 576 − 8λ6i h6 + λ8i h8
2 6 24 24
1 6 6
= 1− λ h + · · · < 1 for small h
144 i
The scheme is thus DISSIPATIVE (i.e., not conservative)
• Leapfrog — |σ1/2 | ≡ 1 for all h < |λ1 |
i
The scheme is thus CONSERVATIVE for all time–steps for which it is
stable!!! Leapfrog is an example of a SYMPLECTIC INTEGRATOR which are
designed to have good conservation properties.
Finite Differences for Differential Equations 52

F INITE D IFFERENCES FOR PDE S

R EVIEW
• Classification of linear PDEs in 2D: consider u : Ω2 → R and A, B,C ∈ R
such that
∂2 u ∂2 u ∂2 u
A 2 +B +C 2 + f (x, y, u) = 0
∂x ∂x ∂y ∂y
• E LLIPTIC P ROBLEMS : B2 − 4AC < 0
– Poisson equation:
∂2 u ∂2 u
+ = g(x, y)
∂x2 ∂y2
• PARABOLIC P ROBLEMS : B2 − 4AC = 0
– Heat equation:
∂u ∂2 u ∂2 u
 
=a + + g(x, y)
∂t ∂x2 ∂y2
• H YPERBOLIC P ROBLEMS : B2 − 4AC > 0
– Wave equation:
∂2 u ∂2 u ∂2 u
 
=a + + g(x, y)
∂t 2 ∂x2 ∂y2
Finite Differences for Differential Equations 53

F INITE D IFFERENCES FOR PDE S

E LLIPTIC P ROBLEMS
• See Homework Assignment # 1 ...
Finite Differences for Differential Equations 54

F INITE D IFFERENCES FOR PDE S

∂u ∂2 u
PARABOLIC P ROBLEMS ∂t = ∂x2 (I)
• C RANK –N ICHOLSON M ETHOD (x j = j∆x, j = 1, . . . , M, t = n∆t,
n = 1, . . . , N):
 2 n un −2un +un
– spatial derivative: ∂∂xu2 = j+1 (∆x)j 2 j−1 + O ((∆x)2 )
j
– time derivative:
un+1
" n+1 n #
∂u − unj ∂2 u ∂2 u
 n+1 
j 1
= + O (∆t) = + + O ((∆t)2 )
∂t j ∆t 2 ∂x2 j ∂x2 j
∆t  n+1 
un+1 − unj n+1 n+1 n n n 2 2


j = u j+1 − 2u j + u j−1 + u j+1 − 2u j + u j−1 + O (∆x) + (∆t)
2(∆x)2
∆t
– thus, defining r = (∆x)2
,we have at every time step n

−run+1 n+1
j+1 + 2(1 + r)u j − run+1 n n n
j−1 = ru j+1 + 2(1 − r)u j + ru j−1

which for U n = [un1 , . . . , unM ]T can be written as an algebraic system

(2I − A)U n+1 = (2I + A)U n , where A is a tridiagonal matrix
Finite Differences for Differential Equations 55

F INITE D IFFERENCES FOR PDE S

∂u ∂2 u
PARABOLIC P ROBLEMS ∂t = ∂x2 (II)
• θ M ETHOD
– allow for a more general approximation in time of the RHS (θ ∈ [0, 1])
un+1
"  n+1  2 n #
∂u − unj ∂ u ∂ u
 n+1 2
j 1
= + O (∆t) = θ + (1 − θ) + O (∆t)
∂t j ∆t 2 ∂x2 j ∂x2 j

– special cases
∗ θ = 0 =⇒ E XPLICIT METHOD : U n+1 = A0U n
∗ θ = 12 =⇒ C RANK –N ICHOLSON METHOD (see previous slide)
∗ θ = 1 =⇒ I MPLICIT METHOD: A1U n+1 = U n

• Stability:
∆t 1
– The E XPLICIT S CHEME is STABLE for r = 2(∆x)2
< 2
– The C RANK –N ICHOLSON and I MPLICIT S CHEME are STABLE for all r
Finite Differences for Differential Equations 56

F INITE D IFFERENCES FOR PDE S

∂2 u ∂2 u
H YPERBOLIC P ROBLEMS ∂t 2 = ∂x2 (I)
n unj+1 −2unj +unj−1
∂2 u

• Spatial derivative: ∂x2
= (∆x)2
+ O ((∆x)2 )
j

• Time derivative:
un+1 − 2unj + un−1
n n
∂2 u ∂2 u
 
j j
= + O ((∆t)2 ) = + O ((∆t)3 )
∂t 2 j (∆t)2 ∂x2 j

(∆t)2 2

(∆t)
un+1 unj+1 + unj−1 − un−1 n 2 4



j = j + 2 1 − u j + O (∆x) + (∆t)
(∆x)2 (∆x)2

(∆t)2
• Stability for (∆x)2
≤1

• R EMARK : need two initial conditions!