Non-standard finite-difference modelling

Introduction to non-standard finite-difference modelling

Peter M. Manning and Gary F. Margrave

ABSTRACT

This report introduces some of the principles and uses of non-standard finite-difference
modelling. It begins with a review of the elements of finite-differences, and how continu-
ous derivatives are replaced with differences, and incorporated into a ‘scheme’. It shows
examples of scheme selection by testing. Finally, it demonstrates some more advanced
strategies of standard finite-differencing for stability or higher accuracy. Then the non-
standard ideas are introduced. A set of standard finite-differences is compared with their
equivalent ‘exact’ finite-differences. It finishes with some examples where exact difference
forms are incorporated into non-standard schemes.

INTRODUCTION

The laws of nature are usually best described by differential equations which, for the
macroscopic world, apply to continuous materials, and usually have continuous solutions.
Analytic results (from calculus) give elegant solutions for these cases, but for a quite lim-
ited set of problems. That leaves many problems which must have their solutions estimated
by using arithmetic operations on arrays of numbers. The point of the finite-difference lit-
erature is to minimize the difficulties that occur when continuous functions are represented
by arrays of samples spaced at finite intervals. Most of the effort within finite-difference
studies is focused on ‘schemes’ which solve problems posed in the form of the differen-
tial equations.These schemes are then evaluated for efficiency and accuracy by theoretical
means, and then with test runs. This study concentrates on these ‘schemes’, and how they
are derived.

CLASSIC FINITE-DIFFERENCE MODELLING

First derivatives

The essential difficulty of finite-difference procedures becomes obvious when a finite-

difference option must be chosen to represent a first derivative. A first derivative, or slope,
can only be defined as a difference in values between two points. Within calculus the slope
at a point is obtained by using the definition of limits to shrink the difference between
the sample points, and the differences of the values at those points, down to infinitesimal
amounts (see equation 1).
df f (x + ∆x) − f (x)
= lim . (1)
dx ∆x↓0 ∆x

This procedure is not allowed with finite differences, because only the samples as given
are available. An equivalent finite-difference equation is given in equation 2, where the
difference is made as small as possible, or one fixed sample length.
f (x + ∆x) − f (x)
Dx f = . (2)
∆x

CREWES Research Report — Volume 18 (2006) 1

Manning and Margrave

This same equation is given in 3, where the input and output values are given in terms
of sample numbers instead of their locations along the x axis.
fn+1 − fn
D x fn = . (3)
∆x

One of the finite-difference first derivative options available is the forward difference
(to the point ahead) in equation 3. Also valid is a backward difference (to the point behind,
equation 4), or a central difference (from the point ahead to the point behind, equation 5).
The central difference clearly has less bias, but depends on values that are further (in total)
from the point where it will be used.
fn − fn−1
D x fn = . (4)
∆x

fn+1 − fn−1
D x fn = . (5)
2∆x

The result of using the forward and backward finite-differences are shown graphically in
Figure 1. The analytic curve which has been sampled is shown in black, and the derivative
of this curve at the same point is shown in blue. It is obvious that the finite-differences

FIG. 1. The backward finite-difference slope is shown in red, and the forward finite-difference in
magenta. These may be compared with the true analytic derivative plotted in blue.

shown are biased toward the slopes in the direction from which they were calculated.

The central-difference calculation of equation 5 is much closer to the analytic result, as

shown in Figure 2.

2 CREWES Research Report — Volume 18 (2006)

 Non-standard finite-difference modelling

FIG. 2. The central finite-difference slope is shown in red, but is shifted to go through the point
at which it applies. For comparison, the forward finite-difference in magenta is also shown. The
central finite-difference is obviously much closer to the true analytic derivative plotted in blue.

Second derivative

A second derivative does not have a similar range of options, because the difference of
differences falls naturally at the positions of the original samples, see equation 6.
fn+1 − 2fn + fn−1
Dx2 fn = . (6)
(∆x)2

Elementary schemes

The most simple finite-difference ‘scheme’ then has a selected type of estimate for
each term of the differential equation, which makes it into a difference equation. The
desired output is usually the function throughout space projected to an advanced time (a
time-stepped result). In many cases (the explicit cases), the difference equation may be
manipulated to solve for an advanced time term. In some cases the advanced time terms
may be implicit, which means the advanced time appears in ways that are impossible to
solve for directly. Special techniques may then be required for the solution.

In either case, the functions must be successively stepped through further increments in
time to get a full picture through space and time.

The final requirement is to test the scheme. The most important test is for stability.
Here amplitudes at a range of frequencies are examined to make sure that these amplitudes
never become infinite. In linear stability analysis this amounts to the exponential growth of
at least some frequencies, which soon swamp all other features of the model.

Another test is for consistency, to make sure that a decrease in the step size of all
variables leads to reduced errors.

CREWES Research Report — Volume 18 (2006) 3

Manning and Margrave

Further tests usually entail comparisons with certain known solutions, usually analytical
solutions, to see if the results are close enough to what is expected.

An example of this methodology is given in Aki and Richards (1980) where, for an
initial attempt, the wave equation represented by the simultaneous equation pair
∂ u̇ 1 ∂τ
= , (7)
∂t ρ(x) ∂x
∂τ ∂ u̇
= E(x) (8)
∂t ∂x
is translated to a scheme with forward finite differences for t-derivatives, and central dif-
ferences for x-derivatives. Although this appears to be a very natural choice for an explicit
scheme, it turns out to be unstable. A scheme with central differences for both t and x is
then tried and found to be stable. The details of this scheme are laid out as follows,
j j
u̇j+1
n − u̇j−1
n 1 τn+1 − τn−1
= , (9)
2∆t ρn 2∆x
τnj+1 − τnj−1 u̇jn+1 − u̇jn−1
= En . (10)
2∆t 2∆x
Here the time sample variations are annotated as superscripts instead of subscripts. This
explicit scheme may be solved for the advanced time terms u̇j+1 or τ j+1 as a function of
past states and the material parameters.

Smoothing

Several additional operations may be utilized to supplement the basic finite-difference

choices. One which may be incorporated is smoothing in the space domain, a technique
introduced by Lax. An example of its use is given in Press et al. (1992), where the one way
wave-equation
∂u ∂u
= −v (11)
∂t ∂x
is represented by a forward finite-difference in time, and a central finite-difference in space.
This equation is
µ j ¶
uj+1
n − ujn un+1 − ujn−1
= −v . (12)
∆t 2∆x

This may be solved for the advanced time term uj+1n , but as with the Aki and Richards
example, it is shown to be unstable. A smoother applied to the ujn term on the time side (a
gapped running average) gives a result called the Lax-Friedrichs scheme
µ j ¶
uj+1
n − 1/2(ujn+1 + ujn−1 ) un+1 − ujn−1
= −v , (13)
∆t 2∆x

and with appropriate sample rates this is shown to be stable.

4 CREWES Research Report — Volume 18 (2006)

 Non-standard finite-difference modelling

High accuracy derivatives

A second technique used is to improve the accuracy of the differentiation, usually of

the spatial co-ordinates. This is done by using a range of samples that extend further than
normal from the estimation point. For example, an estimated second derivative may make
use of 5 samples rather than the minimal number of 3. Strikwerda (2004) shows how the
‘difference calculus’ may be used to derive an appropriate weighting for these samples.

Dablain (1986) also shows how spatial derivatives may be calculated to a higher or-
der by incorporating samples further from the estimation position. He uses a more direct
technique, and so the principle may be a little easier to see.

Dablain starts with the formulation of the Taylor series given as

(∆x)2 (∆x)3
f (x + ∆x) = f (x) + f 0 (x)∆x + f 00 (x) + f 000 (x) + ... . (14)
2! 3!
If this is added to a similar expansion for −∆x

(∆x)2 (∆x)3
f (x − ∆x) = f (x) − f 0 (x)∆x + f 00 (x) − f 000 (x) + ... (15)
2! 3!
the result is
(∆x)2 (∆x)4
f (x + ∆x) + f (x − ∆x) = 2f (x) + 2f 00 (x) + 2f 0000 (x) + ... . (16)
2! 4!
This may be rearranged so that

f (x + ∆x) − 2f (x) + f (x − ∆x) (∆x)2

f 00 (x) = − 2f 0000
(x) + ... . (17)
(∆x)2 4!

Comparing this with equation 6 shows that the usual expression for the second derivative
is really the first term of an infinite series, and is accurate to second order (fourth order
derivatives are required for higher accuracy).

The next stage of accuracy requires that larger ∆x’s of the Taylor series be utilized, and
so doubling the difference used in equation 16 gives

(2∆x)2 (2∆x)4
f (x + 2∆x) + f (x − 2∆x) = 2f (x) + 2f 00 (x) + 2f 0000 (x) + . . . . (18)
2! 4!
Equation 18 may then be divided by 16 and subtracted from equation 16 to eliminate the
f 0000 term. This equation is then solved for the f 00 term, as for equation 17, but the result
will be accurate to fourth order. In terms of sample numbers, the equation becomes
−fn+2 + 16fn+1 − 30fn + 16fn−1 − fn−2
Dx2 fn = . (19)
12(∆x)2

CREWES Research Report — Volume 18 (2006) 5

Manning and Margrave

High accuracy time-stepping

The final finite-difference extension shown here demonstrates how the time stepping
side of the equation may be enhanced. This can be done if the problem equation itself
can be differentiated to give time derivatives of the unknown function in terms of spatial
derivatives. An example of this is given in Strikwerda (2004), where the one way equation
with a source function term,
∂u ∂u
= −a + f, (20)
∂t ∂x
is differentiated in time to get coefficients for a Taylor series expanded in time. The steps
to derive the second derivative with respect to the t term are as follows:
∂ 2u ∂ 2u ∂f
2
= −a + , (21)
∂t ∂t∂x ∂t
∂ 2u ∂ ³ ∂u ´ ∂f
= −a + , (22)
∂t2 ∂x ∂t ∂t
∂ 2u 2
2∂ u ∂f ∂f
2
= a 2
−a + . (23)
∂t ∂x ∂x ∂t

The Taylor series in time may be written as

∂u(t, x) (∆t)2 ∂ 2 u(t, x)
u(t + ∆t, x) = u(t, x) + ∆t + + ... . (24)
∂t 2! ∂t2
Then equation 23, and the original equation 20 are used to get
h ∂u i (∆t)2 h ∂ 2 u ∂f ∂f i
2
u(t + ∆t, x) = u + ∆t −a +f + a −a + + ... , (25)
∂x 2! ∂x2 ∂x ∂t
∂u (∆t)2 ∂ 2 u (∆t)2 ∂f (∆t)2 ∂f
u(t + ∆t, x) = u − a∆t + ∆tf + a2 − a + + . . . . (26)
∂x 2! ∂x2 2! ∂x 2! ∂t
The third and sixth terms above may be combined, and with λ = ∆t/∆x, the finite-
difference equation is

aλ j (aλ)2 j
uj+1
n = ujn − (un+1 − ujn−1 ) + (un+1 − 2ujn + ujn−1 )−
2 2
a∆tλ j j ∆t j+1
(fn+1 − fn−1 )+ (f + fnj ). (27)
4 2 n

This results in the Lax-Wendorf scheme, which is often much better than the equiv-
alent Lax-Friedrichs scheme. Dablain (1986) uses the same technique to obtain a two-
dimensional fourth order accurate time step from fourth order spatial derivatives in x and
z.

NON-STANDARD FINITE-DIFFERENCE MODELLING

The basic theory of non-standard finite-difference (NSFD) modelling was set up by

Mickens, with some of his papers from the eighties. An important reference is the book
(Mickens, 2000) which he edited, and where he provided the first chapter.

6 CREWES Research Report — Volume 18 (2006)

 Non-standard finite-difference modelling

Exact finite-difference schemes

Mickens shows that an ordinary differential equation with an analytical solution may be
matched exactly by a difference equation with a sampled solution. This equation is what he
calls an exact finite-difference solution. This polynomial, with difference and other terms,
comes from the determinant of a matrix which includes a minimal set of samples from all
the independent analytic solutions of the differential equation.

Use of the analytic solution as a guide for the finite-difference solution is nothing new.
Standard finite-difference (SFD) texts always plot their solutions against analytical solu-
tions as a final quality check. Mickens’ ‘exact schemes’, however, use the analytic solu-
tions directly. As it turns out, these schemes have close parallels, but significant deviations
from the SFD equations.

An incomplete set of Mickens’ SFD and exact solutions is shown in Table

Differential equa- Finite-difference equation Exact FD equation

Eq. tion

uk+1 −uk
1 du
dt
= −λu h
= −λuk ³uk+1 −uk´ = −λuk
1−e−λh
λ

du uk+1 −uk uk+1 −uk

2 dt
= −u2 h
= −u2k h
= −uk+1 uk

du uk+1 −uk
³ ´
3 dt
= −u3 h
= −u3k uk+1 −uk 2uk+1
= − uk+1 uk+1 u2k
h +uk

uk+1 −uk
4 du
dt
= λ 1 u − λ 2 u2 h
= λ1 uk − λ2 u2k ³uk+1 −uk´ = λ1 uk − λ2 uk+1 uk
λ h
e 1 −1
λ

5 d2 u du uk+1 −2uk +uk−1

= λ uk −uh k−1 uk+1 −2uk +uk−1
³ ´ = λ uk −uh k−1
dt2
=λ h2
eλh −1
dt λ
h

uk+1 −2uk +uk−1

6 d2 u
+ ω2u = 0 h2
+ ω 2 uk = 0 ³uk+1´−2uk +u
³ k−1´ + ω 2 uk = 0
dt2 4 ωh
sin2
ω2 2

du uk+1 −uk uk+1 −cos(ωh)uk

7 dt
=w h
= wk ³ ´ = wk
sin(ωh)
ω

dw wk+1 −wk
dt
= −ω 2 u h
= −ω 2 uk wk+1 −cos(ωh)wk
³ ´ = −ω 2 uk
sin(ωh)
ω

Table 1. Exact finite-difference schemes for ODEs, compared with standard finite-difference
schemes.
For all these examples, the two solutions have the same number of terms in roughly the
same forms. The deviations tend to be of three types:

CREWES Research Report — Volume 18 (2006) 7

Manning and Margrave

• The denominators of the derivative terms are usually not h (h = ∆t), but instead
some function of h, depending on the original equation. This is the case for all of
these examples except for the second and third. The function is close to h (to the
order of h2 ). In other words, the smaller h is set, the less the modified function is
required.
• Sometimes one of the terms in the numerator of a derivative is multiplied by a func-
tion. This is shown in the seventh example. The modification is close to 1 (to the
order of h2 ).
• Nonlinear terms are represented by non-local expressions, or non-local difference
expressions. This may be seen in examples 2, 3, and 4, where x2 and x3 do not
appear as x2k and x3k , but as more complex expressions.

Non-standard finite-difference schemes

The rationale of the NSFD method, then, is to improve general finite-difference schemes
by including some of the ‘exact’ forms. Exact results would not be expected from the new
schemes, but higher efficiency and better accuracy would be. As with standard schemes,
testing would determine whether the new forms were better. This testing has been encour-
aging for many cases.

Mickens encapsulates his experiences with NSFD schemes into 6 rules which he has
found to be useful. He then goes on to show examples of ordinary and partial differential-
equations where NSFD based schemes have given superior results, and gives the rationale
for his choice of particular schemes.

He usually modifies the denominators of the derivative terms according to the expected
form of the solution. For harmonic type equations, denominators of the type in example
6 are used; and for exponential type equations, denominators of the type in example 1 are
used.

Mickens discovered that important benefits can be found with the use of appropriate
‘non-local’ terms. The obvious symmetry of example 2 shows how the correct choice
of the non-derivative expression makes the usual adjustment of the derivative expression
unnecessary. However, example 3 shows that the expression may become quite complex.

A successful example of a NSFD equation is one set up for a combustion model

du
= u2 (1 − u). (28)
dt

The SFD equation would be

uk+1 − uk
= u2k − u3k , (29)
h
whereas Mickens’ NSFD equation is
u − uk
³ k+1 ´ = 2(uk )2 − uk+1 uk − uk+1 (uk )2 . (30)
1−e −h

8 CREWES Research Report — Volume 18 (2006)

 Non-standard finite-difference modelling

To the right of the equal sign, the net of the first two terms makes up a square with amplitude
1, and the last term is a cube with amplitude 1, just as the SFD scheme has. The exact
formulation for these terms is designed to give a finite-difference solution with the same
zeros as the analytic solution. In particular, when u reaches 1, the slope is exactly zero.
Note that this is an explicit scheme, it is easily solved for uk+1 .

The second example of a NSFD scheme is a solution for a linear advection-diffusion

equation (normalized). This partial differential equation is

∂u ∂u ∂ 2u
+ = b 2, (31)
∂t ∂x ∂x
and the scheme is

uj+1 j
uj − ujm−1 h uj − 2uj + ujm−1 i
m − um
+ m = b m+1 ∆x/b m . (32)
∆t ∆x b(e − 1)∆x

This was constructed quite simply, because

∂u ∂u
+ =0 (33)
∂t ∂x
has an already exact solution

uj+1 j
m − um uj − ujm−1
+ m = 0, (34)
∆t ∆x
and
∂u ∂ 2u
= b 2, (35)
∂x ∂x
has an exact solution
um − um−1 hu i
m+1 − 2um + um−1
=b , (36)
∆x b(e∆x/b − 1)∆x

and the two solutions were just combined.

Non-standard finite-difference wave equations

The lead in using NSFD schemes for the wave equation has been taken by Cole. Chapter
3 in Mickens’ book was written by Cole, and shows how he has used NSFD techniques to
model the wave equation in one, two, and three dimensions.

The starting point for wave equation NSFD modelling is the one-dimensional one,
where it is shown that for a given wavenumber and frequency, the sample rate ∆t may
be replaced by sin(ω∆t/2), and the sample rate ∆x by sin(k∆x/2). The full equation is

d¯2t ψ(x, t) 2 d¯2x ψ(x, t)

− v(x) = 0, (37)
4 sin2 (ω∆t/2)/ω 2 4 sin2 (k∆x/2)/k 2

CREWES Research Report — Volume 18 (2006) 9

Manning and Margrave

where d¯t is the central finite difference. This conforms with Mickens’ first observation
of his exact scheme changes. Cole shows how a truncated single frequency wave train
propagated through a narrow high velocity barrier appears much closer to expectations
with the NSFD algorithm compared to the SFD algorithm.

Cole continues to two dimensions by combining the normal Laplacian with another,
oriented at 45 degrees. The combination begins with a variable weighting, but is then
optimized. The accuracy of this combination he calculates to be of order 8, as compared
to order 2 for the SFD scheme. In a similar fashion he extends the method to the full three
dimensions.

At the end of the chapter, Cole shows how the SFD Yee algorithm for Maxwell’s equa-
tions may be improved by using a NSFD version. The discussion here is for an harmonic
(single frequency) case, but apparently works very well for the limited range of frequencies
that are required to make up a signal with a finite length in time. He also claims that NSFD
techniques may be used to improve the propagation of broad band signals as well.

CONCLUSIONS

• Mickens and Cole have proved that, at least in some cases, NSFD modelling provides
stability, accuracy, and efficiency benefits over SFD modelling.
• These techniques could have wide applicability.

• At this stage, NSFD techniques are as much an art as science, requiring even more
of the ‘experiments’ characteristic of FD scheme design.
• The ideas are comparatively recent, and improvements can be expected.

REFERENCES
Aki, K., and Richards, P. G., 1980, Quantitative Seismology: W.H. Freeman and Co., first edn.

Dablain, M. A., 1986, The application of high-order differencing to the scalar wave equation: Geophysics,
51, No. 01, 54–66.

Mickens, R. E., 2000, Application of nonstandard finite difference schemes: World Scientific Publishing Co.,
Singapore, first edn.

Press, W. H., Teukolsky, S. A., Vetterling, W. T., and Flannery, B. P., 1992, Numerical recipies in C: Cam-
bridge University Press, first edn.

Strikwerda, J. C., 2004, Finite difference schemes and partial differential equations: Society for Industrial
and Applied Mathematics, Philadelphia, second edn.

10 CREWES Research Report — Volume 18 (2006)