Numerical method lecture note Introduction

1 Introduction
1.1 Motivation

In the pre computer era there were generally three different ways in which engineers
approached problem solving:

I. Analytical, or exact, methods: it is applicable for linear models and those that have
simple geometry and low dimensionality. Analytical solutions are of limited practical
value because most real problems are nonlinear and involve complex shapes and
processes.
II. Graphical methods: graphical solutions usually took the form of plots or nomographs.
However, the results are not very precise. Furthermore, they are extremely tedious and
awkward to implement. And they are limited to problems that can be described using
three or fewer dimensions.
III. Calculator and slide rules: they were used to implement numerical methods manually.
Manual calculations are slow and tedious.

Today, computers and numerical methods provide an alternative for such complicated
calculations. As a result, more time is available for the use of your creative skills. Thus, more
emphasis can be placed on problem formulation and solution interpretation and the
incorporation of total system, or “holistic,” awareness.

There are several additional reasons why you should study numerical methods:

1. Numerical methods are extremely powerful problem solving tools. They are capable
of handling large systems of equations, nonlinearities, and complicated geometries
that are not uncommon in engineering practice and that are often impossible to solve
analytically.
2. During your career, you may often have occasions to use commercially available
prepackaged, or “canned,” computer programs that involve numerical methods. The
intelligent use of these programs is often predicted on knowledge of the basic theory
underlying the methods.
3. Many problems cannot be approached using canned programs. If you are conversant
with numerical methods and are adept at computer programming, you can design your
own programs to solve problems without having to buy or commission expensive
software.
4. Numerical methods are an efficient vehicle for learning to use computers.
5. Numerical methods provide a vehicle for you to reinforce your understanding of
mathematics.

Bahir Dar University; Water Resource Eng. Dep. 1

Numerical method lecture note Introduction

1.2 Mathematical modeling and Engineering problem solving

Knowing and understanding are prerequisites for the effective implementation of any tool. This
is particularly true when using computers to solve engineering problems. Although they have
great potential utility, computers are practically useless without a fundamental understanding
of how engineering systems work.

Mathematical modelings have a rule in engineering problem solving. A mathematical model is

defined as a formulation or equation that expresses the essential features of a physical system
or process in mathematical terms. In a very general sense, it can be represented as a functional
relationship of the form

Dependent Variable = f (independent variable, parameters, forcing functions)

Where the dependent variable is a characteristic that usually reflects the behavior or state of
the system; the independent variables are usually dimensions, such as time and space, along
which the system’s behavior is being determined; the parameters are reflective of the system’s
properties or composition; and forcing functions are external influences acting upon it.

The engineering problem solving process is described by the following diagram.

Problem
solving

Mathematical
Theory model Data

Problem solving tools

Numeric or
Graphic result

Implementation

Bahir Dar University; Water Resource Eng. Dep. 2

Numerical method lecture note Introduction

1.3 Computers and software

Numerical methods combine two of the most important tools in engineering collection:
mathematics and computers. In fact, numerical methods can be loosely defined as computer
math.

No mater what type of computers you use, it has utility only if you provide it with careful
instructions. Theses instructions are the software. The five steps required to produce high
quality software to implement numerical methods are

1. Algorithm design: developing the underlying logic of the

program
2. program composition: writing the program in computer language
3. Debugging and Testing: ensuring that the program is error-free
and reliable
4. Documentation: making the program easy to use and understand
5. Storage and Maintenance: saving the program and improving it
in the light of experience

An algorithm is the sequence of logical steps required to perform a specific task such as
solving a problem. However, for detailed representations of complicated programs, they
rapidly become inadequate. For this reason, more versatile alternatives, called flowcharts and
pseudocodes, have been developed.

A flowchart is a visual or graphical representation of an algorithm. The flowchart employs a

series of blocks and arrows, each of which represents a particular operation or steps in the
algorithm. An alternative way to express an algorithm that bridges the gap between flowcharts
and computer code is called pseudocode. This technique uses codelike statements in place of
the graphical symbol of the flowchart.

After concocting the algorithm, the next step to express it as a code. Ideally you would use the
language which is most appropriate for a given application. For most projects, the language
will be selected from the intersection of the subset of languages you know and the set of
languages supported on the specific platform you require. Only for large projects is it worth
investing the time and money required to utilize the optimal language.

For many applications, the current trend is towards object oriented languages such as C++.
Procedural languages are considered behind the times by many computer scientists, yet they
remain the language of choice for many scientific programs. The main procedural contender is,
and has always been FORTRAN. Both C++ and FORTRAN have their advantages, depending
on the task at hand.

The origins of FORTRAN as a scientific computer language are reflected in the name which
stands for Formula Translation. The language is procedural oriented with the algorithm (or
“formula”) playing the central role controlling the way the program is written and structured.
In contrast, the object oriented C++ focuses on the data. For the majority of scientific programs

Bahir Dar University; Water Resource Eng. Dep. 3

Numerical method lecture note Introduction

using the types of algorithms introduced in this course, the procedural oriented approach is
more appropriate. The program generates data from input parameters. The underlying
numerical algorithms process one type of data in a consistent manner.

As the most computationally intensive programs have traditionally followed the procedural
model, and have been written in FORTRAN, there has been a tremendous effort put into
developing optimization strategies for FORTRAN compilers. As a result the code produced by
FORTRAN compilers is normally more efficient for a numerical program than that produced
by other language compilers. In contrast, if you were to develop a word processor, for example,
in FORTRAN, the end result would be less efficient (in terms of speed and size) and more
cumbersome than something with the same functionality developed in C++.

Note: C possesses many of the features of high level programming language and hence can
be employed by engineers for numerical calculations. It should be noted that it is generally
inferior to FORTRAN 90 in this regard. Some universities choose to teach their engineer
and science students C as their first computer languages. Thus, it must be acknowledged
that C will be used by a significant number of engineers and scientists over the next decade.

1.4 Approximations and Round –off errors

Numerical technique yield usually estimates that are close to the exact analytical solution.
There is a discrepancy, or error, because the numerical method involved an approximation. For
many applied engineering problems we cannot compute exactly the error associated with our
numerical methods. In these cases we must settle for approximations or estimates of the errors.

Numerical errors arise from the use of approximations to represent exact mathematical
operations and quantities. These include truncation errors, which is the discrepancy introduced
by the fact that numerical methods may employ approximations to represent exact
mathematical operations and quantities, and round-off errors, which result when numbers
having limited significant figures are used to represent exact numbers.

For both types, the relationship between the exact, or true, result and the approximation can be
formulated as
True value = approximation + error; by rearranging

Et = true vale – approximation

Where Et is used to designate the exact value of the error.

A short coming of this definition is that it takes no account of the order of the magnitude of the
value under examination. One way to account for the magnitudes of the quantities being
evaluated is to normalize the error to the true value, as in

True fractional relative error = true error/ true value

true error
εt = *100%
true value

Bahir Dar University; Water Resource Eng. Dep. 4

Numerical method lecture note Introduction

Where ε t designates the true percent relative error.

However, in real-world applications, we will obviously not know the true answer a prior. For
these situations, an alternative is to normalize the error using the best available estimate of the
true value, that is, to approximate itself, as in

approximate error
εa = 100%
approximate
Where the subscript a signifies the error is normalized to an approximate value. If the
approximation is greater than the true value, the error is negative. Often, when performing
computations, we may not be concerned with the sign of the error, but we are interested in
whether the percent absolute value is lower than a prespecified percent tolerance ε s .

1.5 Truncation errors and the Taylor series

Truncation errors are those that result from using an approximation in place of an exact
mathematical procedure. Mathematical formulation that is used widely in numerical methods
to express functions in an approximate fashion- the Taylor series.

Taylor series provide a means to predict a function value at one point in terms of the function value and
its derivatives at another point. In particular, the theorem states that any smooth function can be
approximated as a polynomial.

The complete Taylor series expansion is expressed as

f ' ' ( xi ) f ' ' ' ( xi )

f ( xi +1 ) = f ( xi ) + f ' ( xi )( xi +1 − xi ) + ( x i +1 − xi ) 2 + ( x i +1 − xi ) 3 + .... + Rn Eq 1-1
2! 3!

Note that Eq.1-1 is an infinite series, the reminder term is included to account all the terms from n+1 to
infinity

f ( n +1) (ξ )
Rn = ( xi +1 − x) n +1 Eq 1-2
(n + 1)!
In general an nth-order Taylor series expansion will be exact for an nth order polynomial. For other
differentiable and continuous functions, such as exponentials and sinusoids, a finite number of terms
will not yield an exact estimate. The inclusion of few more terms will result in an approximation that is
close enough to the true value for practical purposes. The assessment of how many terms are required to
get “close enough” is based on the remainder term of the expansion. Equation 1-2 is useful to get
insight to truncation errors. We can chose how far away from x we want to evaluate f(x), and we can
control the number of terms we include in the expansion. Accordingly Eqn.1-2 is usually expressed as

Rn = O(h n +1 ) Eq 1-3
where h = xi+1 –xi ,the nomenclature O(h ) means that the truncation error is of the order of hn+1. That
n+1

is, the error is proportional to the step size h raised to the (n+1)th power.

Bahir Dar University; Water Resource Eng. Dep. 5

Numerical method lecture note Roots of Equation

2 Roots of Equations
2.1 Motivation

Before the advent of digital computers, there were several ways to solve for roots of algebraic 1
and transcendental2 equations. But for functions that cannot be solved analytically, the only
alternative is an approximate solution technique.

One method to obtain an approximate solution is to plot the function and determine where it
crosses the x axis. An alternative approach is to use trial and error. The process is repeated
until a guess is obtained that result in an f(x) that is close to zero. Such haphazard methods are
obviously inefficient and inadequate for the requirements of engineering practice.

Roots of equation frequently occur in the area of engineering. Mathematical equations or

models derived from fundamental principles are employed to predict dependent variables as a
function of independent variables, forcing functions, and parameters. An example of such a
model is equation, derived from Newton’s second law, for the parachutist’s velocity:

gm
(1 − e − ( c / m )t )
=v EQ 2-1
c
V is expressed explicitly as a function of time. But if we want to determine, for example, the
drag coefficient C, there is no way to rearrange the equation so that c is located on one side of
the equal sign. In such cases, c is said to be implicit. To solve the problem using numerical
methods, it is conventional to re-express as

gm
f (c) = (1 − e − ( c / m )t ) − v Eq 2-2
c
The value of c that makes f(c) = 0 is, therefore, the roots of the equation.

Bracketing methods for finding roots exploit the fact that a function typically changes sign in
the vicinity of a root. Bracketing methods need two initial guesses which must “bracket”, i.e.,
be on either side of the root. Two of the most used bracketing methods are the bisection
method and the false-position method.

Open methods are based on formulas that require only a single starting value of x or two
starting values that do not necessarily bracket the root. They sometimes diverge or move away
from the true root as computation progresses. However, when the open methods converge they
usually do so much more quickly than the bracketing methods. One point iteration, Newton-
Raphson, and Secant methods are examples of open method.
1
By definition, a function given by y = f(x) is algebraic if it can be expressed in the form
f n y n + f n −1 y n −1 + ... + f1 y + f o =
0 , where fi = an ith order polynomial in x. polynomials are a simple algebraic functions that are
represented generally by f n ( x) = ao + a1 x + a2 x 2 + ... + an x n
2
A transcendental function is one that is nonalgebraic. These include trigonometric, exponential, logarithmic, and other, less familiar,
functions.

Bahir Dar University; Water Resource Eng. Dep. 6

Numerical method lecture note Roots of Equation

2.2 Bisection
This is the simplest method for finding a root to an equation. As we shall see, it is also the most robust.
One of the main drawbacks is that we need two initial guesses xa and xb which bracket the root: let
fa = f(xa) and fb = f(xb) such that fa fb <= 0. An example of this is shown graphically in figure 1 . Clearly,
if fa fb = 0 then one or both of xa and xb must be a root of f(x) = 0.

Fig 2-1Graphical representation of the bisection method

showing two initial guesses (xa and xb bracketting the root).

The basic algorithm for the bisection method relies on repeated application of

• Let xc = (xa+xb)/2,
• if fc = f(c) = 0 then x = xc is an exact solution,
• elseif fa fc < 0 then the root lies in the interval (xa,xc),
• else the root lies in the interval (xc,xb).

By replacing the interval (xa,xb) with either (xa,xc) or (xc,xb) (whichever brackets the root), the error in
our estimate of the solution to f(x) = 0 is, on average, halved. We repeat this interval halving until
either the exact root has been found or the interval is smaller than some specified tolerance.

2.2.1 Convergence

Since the interval (xa,xb) always brackets the root, we know that the error in using either xa or xb as an
estimate for root at the nth iteration must be en < |xa - xb|. Now since the interval (xa,xb) is halved for
each iteration, then

en+1 ~ en/2. EQ 2-3

More generally, if xn is the estimate for the root x* at the nth iteration, then the error in this estimate is

εn = xn - x*. Eq 2-4
In many cases we may express the error at the n+1th time step in terms of the error at the nth time step
as
|εn+1| ~ C|εn|p. Eq2-5

Bahir Dar University; Water Resource Eng. Dep. 7

Numerical method lecture note Roots of Equation

Indeed this criterion applies to all techniques discussed in this course, but in many cases it applies only
asymptotically as our estimate xn converges on the exact solution. The exponent p in equation 2-5 gives
the order of the convergence. The larger the value of p, the faster the scheme converges on the solution,
at least provided εn+1 < εn. For first order schemes (i.e. p = 1), |C| < 1 for convergence.

For the bisection method we may estimate εn as en. The form of equation 2-3 then suggests p = 1 and
C = 1/2, showing the scheme is first order and converges linearly. Indeed convergence is guaranteed a
root to f(x) = 0 will always be found provided f(x) is continuous over the initial interval.

The drawback of this kind of error estimation is that it is based on the knowledge of the true root of the
function. This would not be the case in actual situations because there would be no point in using the
method if the root is already known. An approximate percentage relative error εa can be calculated as
x rnew − x rold
εa = 100% Eq 2-6
x rnew

Where xrnew is the root of the present iteration and xrold is the root from the previous iteration. The
absolute value is used because we are usually concerned the magnitude of εa rather than its sign. When
εa becomes less than a pre specified stopping criteria εs, say 0.1 percent, the computation is terminated.

2.3 Linear interpolation (False-position)

This method is similar to the bisection method in that it requires two initial guesses to bracket the root.
However, instead of simply dividing the region in two, a linear interpolation is used to obtain a new
point which is (hopefully, but not necessarily) closer to the root than the equivalent estimate for the
bisection method. A graphical interpretation of this method is shown in figure 2-2.

Fig 2-2 Root finding by the linear interpolation

(False-position) method. The two initial gueses xa
and xb must bracket the root.

The basic algorithm for the linear interpolation method is

• Let

, then
• if fc = f(xc) = 0 then x = xc is an exact solution,

Bahir Dar University; Water Resource Eng. Dep. 8

Numerical method lecture note Roots of Equation

• elseif fa fc < 0 then the root lies in the interval (xa ,xc),
• else the root lies in the interval (xc,xb).

Because the solution remains bracketed at each step, convergence is guaranteed as was the case for the
bisection method. The method is first order and is exact for linear f.

2.4 Newton-Raphson
Consider the Taylor Series expansion of f(x) about some point x = x0:

f(x) = f(x0) + (x-x0)f'(x0) + ½(x-x0)2f"(x0) + O(|x-x0|3). Eq 2-7

Setting the quadratic and higher terms to zero and solving the linear approximation of f(x) = 0 for x
gives

. Eq 2-8
Subsequent iterations are defined in a similar manner as

Eq 2-9
.
Geometrically, xn+1 can be interpreted as the value of x at which a line, passing through the point
(xn,f(xn)) and tangent to the curve f(x) at that point, crosses the y axis. Figure 2-3 provides a graphical
interpretation of this.

Fig 2-3 Graphical interpretation of the Newton Raphson

algorithm.

When it works, Newton-Raphson converges much more rapidly than the bisection or linear
interpolation. However, if f' vanishes at an iteration point, or indeed even between the current estimate
and the root, then the method will fail to converge. A graphical interpretation of this is given in figure
2-4.

Bahir Dar University; Water Resource Eng. Dep. 9

Numerical method lecture note Roots of Equation

Fig 2-4 Divergence of the Newton Raphson algorithm due to

the presence of a turning point close to the root.

2.4.1 Convergence

To study how the Newton-Raphson scheme converges, expand f(x) around the root x = xr, substituting
this value along with f(xr) = 0

0 = f(xi) + (xr – xi)f'(xi) + 1/2(xr – xi)2f''(ξ) , Eq 2-10

Rearranging eqn 2-9 and equating it to zero we have

0 = f(xi )+f’(xi) (xi+1-xi) which can be subtracted from equation 2-10 give

0=f’(xi)(xr-xi+1)+1/2 f’’(ξ)(xr -xi)2 Eq 2-11

The error is the discrepancy between xi+1 and the true value xr thus eqn 2-11 can be expressed as

0= f’(xi)Et,i+1+1/2 f’’(ξ)Et,i2 Eq 2-12

If we assume convergence, both xi and ξ should eventually be approximated by the root xr

f ' ' ( x r ) Et2,i

Et ,i +1 = − Eq 2-13
2 f ' ( xr )
Thus, by comparison with equation 2-4, there is second order (quadratic) convergence. The presence of
the f' term in the denominator shows that the scheme will not converge if f' vanishes in the
neighborhood of the root.

Bahir Dar University; Water Resource Eng. Dep. 10

Numerical Method Lecture note System of Linear Eqn.

3. Linear Algebraic Equations

3.1. INTRODUCTION

In this chapter, we will deal with the case of determining the values of x1 , x 2 ,..., x n that
simultaneously satisfy the set of equations:

f 1 (x1 , x 2 ,..., x n ) = 0
f 2 ( x1 , x 2 ,..., x n ) = 0
(2.1)
......
f n (x1 , x 2 ,..., x n ) = 0

In particular we will consider linear algebraic equations are equations of the form:

a11 x1 + a12 x 2 + ... + a1n x n = b1

a 21 x1 + a 22 x 2 + ... + a 2 n x n = b2
(2.2)
....
a n1 x1 + a n 2 x 2 + ... + a nn x n = bn

where the a 's are constant coefficients, the b 's are constants, and n is the number of equations.

The above system of linear equations may also be written in matrix form as:

[A]{X } = {B} (2.3)

Where [ A] is an n by n square matrix of coefficients (called the coefficient matrix),

 a11 a12 ... a1n 

a a 22 ... a 2 n 
[A] =  21
 ... ... ... ... 
 
a n1 an2 ... a nn 

{B} is an n by 1 column vector of constants,

{B}T = b1 b2 ... bn 

and {X } is an n by 1 column vector of unknowns:

{X } = x1 x2 ... x n 

Bahir Dar University; Water Resource Eng. Dep. 11

Numerical Method Lecture note System of Linear Eqn.

Non-Computer Methods

There are some non-computer methods which are used to solve small ( n ≤ 3 ) sets of
simultaneous equations that do not require a computer.

The Graphical Method

Plotting the graphs and finding the point of intersection.

Cramer's Rule

Cramer's rule states that the solution of a set of linear equations given in Eq. (2.3) can be give as:

Di
xi = (2.4)
D

where D is the determinant of the coefficient matrix [ A] , and Di is the determinant of the
matrix obtained by replacing the coefficients of the unknown xi in the coefficient matrix by the
constants b1 , b2 , ..., bn .

For more than three equations, Cramer's rule becomes impractical because, as the number of
equations increase, the determinants are time-consuming to evaluate by hand (or by computer).
Consequently, more efficient alternatives are used.

Elimination of Unknowns

The basic strategy is to multiply the equations by constants so that one of the unknowns will be
eliminated when the equations are combined. The result is a single equation that can be solved for
the remaining unknown. This can then be substituted into either of the original equations to
compute the other variable.

The elimination of unknowns can be extended to systems with more that two or three equations.
However, the numerous calculations that are required for larger systems make the method
extremely tedious to implement by hand. However, the technique can be formalized and readily
programmed for the computer.

3.2. GAUSS ELIMINATION

3.2.1. Description of Method

The approach is designed to solve a general set of n equations:

a11 x1 + a12 x 2 + ... + a1n x n = b1 (2.5a)

a 21 x1 + a 22 x 2 + ... + a 2 n x n = b2 (2.5b)
....
a n1 x1 + a n 2 x 2 + ... + a nn x n = bn (2.5c)

Bahir Dar University; Water Resource Eng. Dep. 12

Numerical Method Lecture note System of Linear Eqn.

The naive Gauss elimination method consists of two phases:

1. Forward Elimination: The first step is designed to reduce the set of equations to an upper
triangular system. The initial step will be to eliminate the first unknown x1 , from the second
through the n th equations. To do this, multiply Eq. (2.5a) by a 21 a11 to give:

a 21 a a
a 21 x1 + a12 x 2 + ... + 21 a1n x n = 21 b1 (2.6)
a11 a11 a11

Now this equation can be subtracted from Eq. 2.6b to give:

 a   a  a
 a 22 − 21 a12  x 2 + ... +  a 2 n − 21 a1n x n = b2 − 21 b1
 a 11   a11  a11
or
′ x 2 + ..... + a 2′ n x n = b2′
a 22

where the prime indicates that the elements have been changed from their original values.

The procedure is then repeated for the remaining equations. For instance, Eq. (2.5a) can be
multiplied by a31 a11 and the result subtracted from the third equation. Repeating the procedure
for the remaining equations results in the following modified system:

a11 x1 + a12 x 2 + a13 x3 + ... + a1n x n = b1 (2.7a)

a ′22 x 2 + a 23
′ x3 + ... + a 2′ n x n = b2′ (2.7b)
a32′ x 2 + a33′ x3 + ... + a3′ n x n = b3′ (2.7c)
.....

a ′n 2 x 2 + a n′ 3 x3 + ... + a nn
′ x n = bn′ (2.7d)

For the foregoing steps, Eq.(2.5a) is called the pivot equation, and a11 is the pivot coefficient or
element. Now repeat the above to eliminate the second unknown from Eq. (2.7c) through Eq.
′ a 22
(2.7d). To do this multiply Eq. (2.7b) by a32 ′ , and subtract the result from Eq. (2.7c). Perform
a similar elimination for the remaining equations to yield

a11 x1 + a12 x 2 + a13 x3 + ... + a13 x n = b1

′ x 2 + a 23
a 22 ′ x3 + ... + a 2′ n x n = b2′
′′ x3 + ... + a3′′n x n = b3′′
a33
.....

a n′′3 x3 + ... + a nn
′′ x n = bn′′

where the double prime indicates that the element have been modified twice.

Bahir Dar University; Water Resource Eng. Dep. 13

Numerical Method Lecture note System of Linear Eqn.

The procedure can be continued using the remaining pivot equations. The final manipulation in
the sequence is to use the (n − 1)th equation to eliminate the x n −1 term from the nth equation.
At this point, the system will have been transformed to an upper triangular system:

a11 x1 + a12 x 2 + a13 x3 + ... + a1n x n = b1n (2.8a)

a 22 ′ x3 + ... + a 2′ n x n = b2′
′ x 2 + a 23 (2.8b)
′′ x3 + ... + a3′′n x n = b3′′
a33 (2.8c)
.
.
.
( n −1)
a nn x n = bn(n −1) (2.8d)

2. Back Substitution: Eq. (2.8d) can now be solved for x n :

bn(n −1)
x n = (n −1) (2.9)
an

This result can be back substituted into the (n − 1)th equation to solve for x n −1 . The procedure,
which is repeated to evaluate the remaining x 's, can be represented by the following formula:

n
bi(i −1) − ∑ a(
j =i +1
ij
i −1)
xj
xi = for i = n − 1, n − 2,...,1 (2.10)
aii(i −1)

3.2.2. Pitfalls of Gauss Elimination

Whereas there are many systems of equations that can be solved with naive Gauss elimination,
there are some pitfalls that must be explored before writing general computer program to
implement the method.

Division by Zero

The primary reason that the foregoing technique is called "naive" is that during both elimination
and back-substitution phases, it is possible that a division by zero can occur. Problems also arise
when the coefficient is very close to zero. The technique of pivoting (to be discussed later) has
been developed to partially avoid these problems.

Round-off Errors

The problem of round-off errors can become particularly important when large numbers of
equations are to be solved. A rough rule of thumb is that round-off errors may be important when
dealing with 100 or more equations. In any event, one should always substitute the answers back
into the original equations to check whether a substantial error has occurred.

Bahir Dar University; Water Resource Eng. Dep. 14

Numerical Method Lecture note System of Linear Eqn.

Ill-Conditioned Systems

Ill-conditioned systems are those where a small change in coefficients in large changes in the
solution. An alternative interpretation of ill-conditioning is that a wide range of answers can
approximately satisfy the equations.

An ill-conditioned system is one with a determinant of the coefficient matrix close to zero. It is
difficult to specify how close to zero the determinant must be to indicate ill-conditioning. This is
complicated by the fact that the determinant can be changed by multiplying one or more of the
equations by a scale factor without changing the solution. One way to avoid this difficulty is to
scale the equations so that the maximum element in any row is equal to 1 (This process is called
scaling).

Singular Systems

The system is singular when at least two of the equations are identical. In such cases, we would
lose one degree of freedom, and would be dealing with impossible case of n − 1 equations in n
unknowns. Such cases might not be obvious particularly when dealing with large equation sets.
Consequently, it would be nice to have some way of automatically detecting singularity. The
answer to this problem is neatly offered by the fact that the determinant of a singular system is
zero. This idea can, in turn, be connected to Gauss elimination by recognizing that after the
elimination step, the determinant can be evaluated as the product of the diagonal elements. Thus,
a computer algorithm can test to discern whether a zero diagonal element is created during the
elimination stage.

3.2.3. Techniques for Improving Solutions

1. Use of more significant figures.

2. Pivoting: can be partial or complete.

Partial Pivoting: Determine the largest available coefficient in the column below the pivot
element. The rows are then switched so that the largest element is the pivot element.

Complete Pivoting: When columns as well as rows are switched.

3. Scaling: Scaling is the process by which the maximum element in a row is made to be 1 by
dividing the equation by the largest coefficient.

3.3. GAUSS-JORDAN

Gauss-Jordan is a variation of the Gauss elimination. The major difference is that when an
unknown is eliminated in the Gauss-Jordan method, it is eliminated from all other equations
rather than just the subsequent ones. In addition, all rows are normalized by dividing them by
their pivot elements. Thus, the elimination step results in an identity matrix rather than a
triangular matrix. Thus, back-substitution is not necessary. All the material regarding the pitfalls
and improvements in Gauss elimination also applies to the Gauss-Jordan method. Gauss-Jordan
method involves approximately 50 percent more operations than Gauss elimination.

Bahir Dar University; Water Resource Eng. Dep. 15

Numerical Method Lecture note System of Linear Eqn.

3.4. LU-DECOMPOSITION

Gauss elimination is a sound way to solve systems of algebraic equations of the form

[A]{X } = {B} (2.11)

However, it becomes inefficient when solving equations with the same coefficients [ A] , but with
different right-hand side constants.

LU decomposition methods separate the time-consuming elimination of the matrix [ A] from the
manipulations of the right-hand side {B}. Thus, once [ A] has been "decomposed", multiple right-
hand side vectors can be evaluated in an efficient manner.

3.4.1. Overview of LU Decomposition Method

Eq. (2.11) can be rearranged to give:

[A]{X } − {B} = 0 (2.12)

Suppose that Eq. (2.12) could be expressed as an upper triangular system:

u11 u12 u13   x1   d1 

   
0
 u 22 u 23   x 2  = d 2  (2.13)
 0 0 u 33   x3  d 3 

Eq. (2.13) can also be expressed in matrix notation and rearranged to give:

[U ]{X } − {D} = 0 (2.14)

Now, assume that there is a lower diagonal matrix with 1's on the diagonal,

1 0 0
[L] = l 21 1 0 (2.15)
l31 l32 1

that has the property that when Eq. (2.14) is pre-multiplied by it, Eq. (2.12) is the result. That is,

[L]{[U ]{X } − {D}} = [A]{X } − [B] (2.16)

If this equation holds, it follows that

[L][U ] = [A] (2.17)

and

[L]{D} = {B} (2.18)

Bahir Dar University; Water Resource Eng. Dep. 16

Numerical Method Lecture note System of Linear Eqn.

A two-step strategy (see Fig. 2.1) for obtaining solutions can be based on Eqs. (2.14), (2.17) and
(2.18):

1. LU decomposition step. [ A] is factored or decomposed into the lower [L ] and upper [U ]

triangular matrices.

2. Substitution step. [L ] and [U ] are used to determine a solution {X } for a right-hand side
{B}. This step itself consists of two steps. First, Eq. (2.18) is used to generate an intermediate
vector by forward substitution. Then, the result is substituted into Eq. (2.14) which can be
solved by back substitution for {X }.

[A] {X } = {B}
(a) Decomposition
[U ] [L]

[L] {D} = {B}

(b) Forward

[D]
Substitution
[U ] {X } = {D}
(c) Backward
[X ]

Fig. 2.1 Steps in LU Decomposition

Bahir Dar University; Water Resource Eng. Dep. 17

Numerical Method Lecture note Curve Fitting

4. Curve fitting
Data is often given for discrete values along a continuum. However estimates of points
between these discrete values may be required. One way to do this is to formulate a
function to fit these values approximately. This application is called curve fitting. There
are two general approaches to curve fitting.
The first is to derive a single curve that represents the general trend of the data. One
method of this nature is the least-squares regression.
The second approach is more precise. The basic idea is to fit a curve or a series of curves
that pass directly through each of the points.

4.1. Least squares regression

The simplest example of the least squares approximation is fitting a straight line to a set
of paired observations: (x1,y1),(x2,y2)……(xn, yn). The mathematical expression for the
straight line is
y = ao + a1x+ e
where ao, and a1 are coefficients representing the y-intercept and the slope of the line
respectively while e is the error or residual between the model and the observations,
which can be represented as
e = y - ao - a1x
Thus the error is the discrepancy between the true value of y(observed value) and the
approximate value ao + a1x, predicted by the linear equation. Any strategy of
approximating a set of data by a linear equation (best fit) should minimize the sum of
residuals. The least squares fit of straight line minimizes the sum of the squares of the
residuals.
n n n
S r = ∑ e = ∑ ( y i ,measured − y i ,mod el ) = ∑ ( y i − a o − a1 xi ) 2 …………….(1)
2
i
2

i =1 i =1 i =1

To determine the values of ao and a1, differentiate (*) with respect to each coefficient
∂S r
= −2∑ ( y i − a o − a1 xi )
∂a o
…………………………………………….(2)
∂S r
= −2∑ [( y i − a o − a1 xi ) xi ]
∂a1

Setting the derivatives equal to zero will result in a minimum Sr. The equations can then be
expressed as
∑ y − ∑ a − ∑ a x = 0 ………………………………………………(3)
i o 1 i

∑ y x − ∑a x − ∑a x = 0
i i o i
2
1 i

Solving for ao and a1 simultaneously

Bahir Dar University; Water Resource Eng. Dep. 18

Numerical Method Lecture note Curve Fitting

n∑ xi y i − ∑ xi ∑ y i
a1 = ……………………………………………………(4)
n∑ xi2 − (∑ xi )
2

_ _
a o = y − a1 x ……………………………………………………………………(5)
_ _
where y and x are the means of y and x, respectively

Test of goodness of fit

Any line other than the one derived above gives larger sum of the squares of the
residuals. The square of the residual represents the square of the vertical distance between
the data and the straight line. The standard deviation of the regression line can be
determined by the standard error of the estimate
Sr
Sy/x = …………………………………………………………………..(6)
n−2
Sx/y quantifies the spread around the regression line. This concept can be used to quantify
the goodness of fit. This is particularly necessary when comparing several regressions. To
do this, we return to the original data and determine the total sum of squares around the
mean of the dependent variable i.e., y. This quantity is designated St and represents the
magnitude of the residual error associated with the dependent variable prior to the
regression. Recall that Sr characterizes the residual error that remains after the regression.
The difference between the two quantities, St – Sr, quantifies the improvement or error
reduction due to describing the data in terms of a straight line reather than an average
value. The normalized difference yields
S − Sr
r2 = t …………………………………………………………………….(7)
St
where r2 is called the coefficient of determination and r is the correlation coefficient.
For linear regression r is given by
n∑ xi y i − (∑ xi )(∑ y i )
r= …………………………………….(8)
n∑ xi2 − (∑ xi ) 2 n∑ y i2 − (∑ y i ) 2

For a perfect fit r =1 and r should be close to 1 for good fit.

Caution!
Although the correlation coefficient provides a handy measure of goodness of fit, care
should be taken not to take for granted that the value of r close to one necessarily mean
'"good fit". It is always advised to plot the data along with the regression curve because
it is possible to obtain relatively high value of r when the relationship between y and x is
not linear.

Linearization of non-linear relationships

Bahir Dar University; Water Resource Eng. Dep. 19

Numerical Method Lecture note Curve Fitting

Linear regression provides a powerful technique for fitting a “best” line to a data.
However it is predicated on the fact that the relationship between the independent and
dependent variables is linear. But usually this is not the case. Visual inspection of the plot
of the data will provide useful information whether linear regression is acceptable. In
situations where linear regression is inadequate other methods such as polynomial
regression are appropriate. For others, transformations can be used to express the data in
a form that is compatible with linear regression.
The followings are examples of functions which can be linearized
1. Exponential functions
y = a1e b1x
where a1 and b1 are constants
This function can be linearized by taking the natural logarithm of both sides of the
equation
ln y = ln a1 + b1 x ln e
ln y = ln a1 + b1 x
the plot of ln y versus ln x will yield a straight line with a slope of b1 and an intercept of
ln a1.
2. Power functions
y = a2 x b2
where a2 and b2 are constant coefficients.
This equation can be linearized by taking its base 10 logarithm to give
log y = b2 log x + log a 2
the plot of logy versus logx will yield a straight line with a slope of b2 and an intercept of log a2.
3. Saturation growth rate equation
x
y = a3
b3 + x
this equation can be linearized by inverting to give
1 b3 1 1
= +
y a3 x a3
the plot of 1/y versus 1/x will be linear, with a slope of b3/a3 and an intercept of 1/a3.
In their transformed forms, these models are fit using linear regression in order to
evaluate the constant coefficients. Then they can be transformed back to their original
state and used for predictive purposes.

Polynomial regression

Some engineering data, although exhibiting marked pattern, is poorly represented by

straight line. For these cases a curve would be better suited to fit the data. One of the
possible ways of solving this kind of problems is to fit the data by a polynomial function.
This is called polynomial regression
The least squares method can be extended to fit data to a higher-order polynomial.
Suppose we want to fit a second order polynomial:
y = a o + a1 x + a 2 x 2 + e
For this care the sum of the squares of the residuals is

Bahir Dar University; Water Resource Eng. Dep. 20

Numerical Method Lecture note Curve Fitting

S r = ∑ ( y i − a o − a1 xi −a 2 xi2 ) 2

Taking the derivative of the above equation with respect each unknown coefficients of the
polynomial gives
∂S r
= −2∑ ( y i − a o − a1 xi − a 2 xi2 )
∂a o
∂S r
= −2∑ xi ( y i − a o − a1 xi − a 2 xi2 )
∂a1
∂S r
= −2∑ xi2 ( y i − a o − a1 xi − a 2 xi2 )
∂a o
Setting these equations equal to zero and rearranging to develop the following set of equations
na o + (∑ xi ) a1 + (∑ xi2 ) a 2 = ∑ y i
(∑ xi )a o + (∑ xi2 )a1 + (∑ xi3 )a 2 = ∑ xi y i
(∑ xi2 )a o + (∑ xi3 )a1 + (∑ xi4 )a 2 = ∑ xi2 y i
Solving for the coefficients of the quadratic regression is equivalent to solving three
simultaneous linear equations. The techniques for solving these problems are discussed in
chapter two.
This discussion can easily be extended to an mth-order polynomial as
y = a o + a1 x + a 2 x 2 + ... + a m x m + e
Thus determination of the coefficients of an mth-order polynomial is equivalent to
solving a system of m+1 simultaneous linear equations. For this case the standard error is
formulated as
Sr
Sy/x =
n − (m + 1)
The coefficient of determination can also be computed for polynomial regression with equation
(7).

Multiple linear regression

A useful extension of linear regression is the case where y is a linear function of more
than one variable, say x1 and x2,
y = a o + a1 x1 + a 2 x 2 + e
Such an equation is useful when fitting experimental data where the variable being
studied is a function of two other variables.
in the same manner as the previous cases the best values of the coefficients are
determined by setting the sum of the squares of the residuals to a minimum.
S r = ∑ ( y i − a o − a1 x1i − a 2 x 2i ) 2

Differentiating with respect to the unknown coefficients , we have

Bahir Dar University; Water Resource Eng. Dep. 21

Numerical Method Lecture note Curve Fitting

∂S r
= −2∑ ( yi − ao − a1 x1i − a2 x2i )
∂ao
∂S r
= −2∑ x1i ( yi − ao − a1 x1i − a2 x2i )
∂a1
∂S r
= −2∑ x2i ( yi − ao − a1 x1i − a2 x2i )
∂ao
The coefficients yielding the minimum sum of the residuals are obtained by setting the
partial derivatives equal to zero and expressing the result in a matrix form as
n

∑ x1i ∑ x2i  ao  ∑ yi 
   
∑ x1i ∑ x1i ∑ x1i x2i  a1  = ∑ x1i yi 
2

  a   
∑ x 2i ∑ x1i x 2i ∑ x 2i   2  ∑ x 2i y i 
2

The above case can be extended to m dimension, an in

y = a o + a1 x1 + a 2 x 2 + ... + a m x m + e
where the standard error is formulated as
Sr
Sy/x =
n − (m − 1)
The coefficient of determination is computed as in equation(7).
Multiple linear regression has utility in the derivation of power equations of the general
form
y = a o x1a1 x 2a 2 ....x mam
Transformation of this form of equation can be achieved by taking the logarithm of the
equation.

General linear least squares

In the preceding discussions we have seen three types of regression: linear, polynomial
and multiple linear. All these belong to the general linear least squares model given by
y = a o z o + a1 z1 + a 2 z 2 + ... + a m z m + e
where zo, z1, z2, .....,zm are m+1 different functions. For multiple linear regression zo = 1, z1 =x1,
z2 = x2, ...., zm = xm. For polynomial regression, the z's are simple monomials as in zo= 1, z1 = x,
z2 = x2 , ..., zm = xm.
The terminology linear refers only to the model's dependence on its parameters i.e., the a's.
Equation ( ) can be expressed in a matrix form as
{y} = [z ]{A} + {E}
where [z] is a matrix of calculated values of the z functions at the measured values of the
independent variables.

Bahir Dar University; Water Resource Eng. Dep. 22

Numerical Method Lecture note Curve Fitting

 z o1 z11 .. .. z m1 
z z12 .. .. z m 2 
 o2
 . . . . . 
 
 . . . . . 
 z on z1n . . z mn 
where m is the number of variables in the model and n is the number of data points. Because n
≥m+1 , most of the time [z] is not a square matrix.
The column vector {y} contains the observed values of the dependent variable.
{y}T =  y1 , y 2 , ..., y n 
The column vector {A} contains the unknown coefficients
{A}T= a o , a1 ,..., a m 
the column vector {E} contains the residuals
{E}T = e1\ , e2 ,..., en 
The sum of the squares of the residuals can be defined as
2
n  m 
S r = ∑  y i − ∑ a j z ji 
i =1  j =0 
This quantity can be minimized by taking its partial derivative with respect to each of the
coefficients and setting the resulting equation equal to zero. The outcome of this process
can be expressed in matrix form as
[ ] {
[Z ]T [Z ] {A} = [Z ]T {y} }

Bahir Dar University; Water Resource Eng. Dep. 23

Numerical Method Lecture note Curve Fitting

Bahir Dar University; Water Resource Eng. Dep. 24

Numerical Method Lecture note Curve Fitting

Bahir Dar University; Water Resource Eng. Dep. 25

Numerical Method Lecture note Curve Fitting

4.2. Interpolation
In many engineering applications it is required to estimate intermediate values between precise
data points. The most common method used is the polynomial interpolation. For n+1 data points ,
there is a unique polynomial of order n that passes through all the points.
f(x) = ao + a1 x + a2 x2 + ... +an xn
Polynomial interpolation involves the determination of that unique n-th order polynomial that fits
n+1 data points. This formula can then be used to estimate intermediate values. Although there is
only one n-th order polynomial that fits n+1 points, there are a variety of mathematical formats in
which this polynomial can be expressed. The most widely used alternatives are the Newton and
Lagrange interpolating polynomials.

4.2.1.Newton's Divided Difference Interpolating Polynomials

The simplest form of interpolation is to connect two data points with a straight line. This
technique is called linear interpolation. Using the concept of similar triangles,
f1 ( x) − f ( xo ) f ( x1 ) − f ( xo )
=
x − xo x1 − xo
which can be rearranged to yield
f ( x1 ) − f ( xo )
f1 ( x) = f ( xo ) + ( x − xo )
x1 − xo
this is the linear interpolation formula. The notation f1(x) indicates that this is a first order
polynomial. The term [f(x1)-f(xo)]/(x1-x0) is the finite difference approximation of the first
derivative. The smaller the interval the between the data points, the better the approximation.
Approximation of a curve with a straight line can result in significant errors. A better
approximation can be achieved if some curvature is introduced into the line connecting the points.
If three points are available, a second order polynomial can be plotted through the data points.
This is called a quadratic interpolating polynomial. A convenient form for this purpose is
f2(x) = bo + b1(x-xo) + b2(x-x0) (x-x1)

bo can be solved for by setting x=x0, in which case bo = f2(xo)

then f2(x) can be evaluated at x=x1 to solve for b1
f ( x1 ) − f ( xo )
b1 =
x1 − xo
evaluating f2(x) at x=x2 gives
f ( x 2 ) − f ( x1 ) f ( x1 ) − f ( xo )
−
x 2 − x1 x1 − xo
b2 =
x 2 − xo
Note that b1 represents the slope of the line between xo and x1 and b2 is similar to the finite
difference approximation of the second derivative.

Bahir Dar University; Water Resource Eng. Dep. 26

Numerical Method Lecture note Curve Fitting

General form of Newton's Interpolating Polynomials

The preceding analysis can be generalized to fit an n-th order polynomial to n+1 data points. The
n-th order polynomial is
fn(x) = b0 + b1(x-xo) + b2(x-xo)(x-x1)+ .... + bn(x-xo)(x-x1)....(x-xn-1)
The data points can be used to evaluate the coefficients bo, b1, b2, ...., bn.
b0 = f(xo)
b1 = f[x1,xo]
b2 = f[x2,x1,xo]
.
.
.
bn = f[xn,xn-1, ..., x1,xo]
where the bracketed function evaluations are finite divided differences. For example the first
finite divided difference is given by
f ( xi ) − f ( x j )
f [ xi , x j ] =
xi − x j
The second finite divided difference is expressed by
f [ xi , x j ] − f [ x j , x k ]
f [ xi , x j , x k ] =
xi − x k
Similarly the n-th finite divided difference is
f [ x n , x n −1 ,..., x1 ] − f [ x n −1 , x n − 2 ,..., xo ]
f [ x n , x n −1 ,..., x1 , xo ] =
x n − xo
These differences can be used to evaluate the coefficients and then can be substituted to yield the
interpolating polynomial

fn(x) = f(xo)+(x-xo)f[x1,xo]+(x-xo)(x-x1)f[x2,x1,xo]+... + (x-xo)(x-x1)...(x-xn-1)f[xn,xn-1,..., xo]

which is called the Newton's divided difference interpolating polynomial. It does not require that
the data be put in ascending order or the interval between data points to be equal.

Errors in Newton's Interpolating polynomials

The structure of the general form of Newton's divided difference interpolating polynomial is
similar to the Taylor series expansion in the sense that terms are added sequentially to capture the
higher order behavior of the underlying function. These terms are finite difference
approximations of the higher order derivatives. Thus if the true underlying function is n-th order
polynomial, the n-th order interpolating polynomial based on n+1 data points will yield exact
results.

As in the Taylor series, a formulation of the truncation error can be obtained. Recall that the
truncation error in Taylor series is generally expressed as
f ( n +1) (ξ )
Rn = ( xi +1 − xi ) n +1
(n + 1)!
where ξ is somewhere in the interval xi to xi+1. For the n-th order interpolating polynomial, an
analogous relationship for error is

Bahir Dar University; Water Resource Eng. Dep. 27

Numerical Method Lecture note Curve Fitting

f ( n +1) (ξ )
Rn = ( x − xo )( x − x1 )...( x − x n )
(n + 1)!
where ξ is somewhere in the interval containing the unknown and the data. An alternative
formulation which uses a finite difference to approximate the (n+1)th derivative is
Rn = f[x,xn,xn-1, ..., xo](x-xo)(x-x1)...(x-xn)
where f[x,xn,xn-1,...,xo] is the (n+1)th finite divided difference. The above equation can be solved
if an additional point f(xn+1) is available in which case
Rn ≅ f[xn+1,xn,xn-1,....,xo](x-xo)(x-x1)...(x-xn)
The error estimate for the n-th order polynomial is equivalent to the difference between the
(n+1)th order and the n-th order prediction.
Rn = fn+1 (x) - fn(x)
The validity of this approach lies on the fact that the series is strongly convergent. For such
situations the (n+1)th order prediction would be closer to the true value than the n-th order
prediction.

4.2.2.Lagrange Interpolation Polynomials

The Lagrange interpolating polynomial is a reformulation of the Newton polynomial that avoids
the computation of divided differences. It can be represented as
n
f n ( x) = ∑ Li ( x) f ( xi )
i =0
where
n x − xj
Li ( x) = ∏
j =0 xi − x j
j ≠0
where Π designates the "product of."
for linear interpolation
x − x1 x − xo
f1 ( x) = f ( x0 ) + f ( x1 )
x0 − x1 x1 − xo
For second order interpolating polynomials
( x − x1 )( x − x 2 ) ( x − xo )( x − x 2 ) ( x − xo )( x − x1 )
f 2 ( x) = f ( xo ) + f ( x1 ) + f ( x2 )
( xo − x1 )( xo − x 2 ) ( x1 − xo )( x1 − x 2 ) ( x 2 − xo )( x 2 − x1 )
As in the Newton's method the Lagrange version has an estimated error of
n
Rn = f [ x, x n , x n −1 ,..., xo ]∏ ( x − xi )
i =0
Thus if an additional point at x=x+1 is available an error estimate can be obtained. But because
the finite differences are not employed as part of the Lagrange operation, this is rarely done. For
cases where the order of the polynomial is not determined prior to interpolation, the Newton
method has advantages because of the insight it provides into the behavior of the different order
formulas.

Higher order polynomials tend to be ill conditioned; i.e. they are highly sensitive to round-off
error. The same order goes for higher order polynomial regression.

Bahir Dar University; Water Resource Eng. Dep. 28

Numerical Method Lecture note Curve Fitting

Special care should be taken when interpolating polynomials are to be used for extrapolation.
This arises because the point lies outside the base points used of the interpolation. The best results
of interpolation are obtained when the unknown lies near the center of the base points.
Subsequently when the point lies outside the range of the data set, the error in extrapolation can
be very large.

Bahir Dar University; Water Resource Eng. Dep. 29

Numerical Method Lecture note Differentiation & Integration

5. NUMERICAL DIFFERENTIATION & INTEGRATION

5.1. INTRODUCTION
Calculus is mathematics of change. Because engineers must continuously deal with systems and
processes that change, calculus is an essential tool of the engineering profession. Standing at the
heart of calculus are the related mathematical concepts of differentiation and integration.
Mathematically, the derivative represents the rate of change of a dependent variable with respect
to an independent variable. If y is the dependent variable and x is the independent variable, the
first derivative of y = f(x) w.r.t. to x, represented by dy/dx, is given by

dy f ( xi + ∆x ) − f ( xi )
= lim (4.1)
dx ∆x →0 ∆x

The inverse process to differentiation in calculus is integration. Mathematically, integration is

represented by

b
I = ∫ f ( x )dx (4.2)
a

which stands for the integral of the function f(x) w.r.t. the independent variable x, evaluated
between the limits x = a to x = b. The function f(x) is referred to as the integrand.

Non-Computer Methods for Differentiation and Integration

The function to be differentiated or integrated will typically be in one of the following three
forms:
1. A simple continuous function such as a polynomial, an exponential, or a trigonometric
function.
2. A complicated continuous function that is difficult or impossible to differentiate or
integrate directly.
3. A tabulated function where the values of x and f(x) are given at a number of discrete
points, as is often the case with experimental or field data.

In the first case, the derivative or integral of a simple function may be evaluated analytically
using calculus. For the second case, analytical solutions are often impractical, and sometimes
impossible, to obtain. In these instances, as well as in the third case of discrete date, approximate
methods must be employed.

A non-computer method for determining derivatives from data is called equal-area graphical
differentiation. In this method, the (x, y) data are tabulated and, for each interval, a simple divided
difference Δy/Δx is employed to estimate the slope. Then these values are plotted as a stepped
curve versus x. Next a smooth curve is drawn that attempts to approximate the area under the
stepped curve. That is, it is drawn so that visually, the positive and the negative areas are
balanced. The rates at given values of x can then be read from the curve.

Bahir Dar University; Water Resource Eng. Dep. 30

Numerical Method Lecture note Differentiation & Integration

In the same spirit, visually oriented approaches were employed to integrate tabulated data and
complicated functions in the precomputer era. A simple intuitive approach is to plot the function
on a grid and count the number of boxes that approximate the area. This number multiplied by the
area of each box provides a rough estimate of the total area under the curve. This estimate can be
refined, at the expense of additional effort, by using a finer grid.

Another commonsense approach is to divide the area into vertical segments, or strips, with a
height equal to the function value at the midpoint of each strip. The area of the rectangles can be
then calculated and summed to estimate the total area. In this approach, it is assumed that the
value at the midpoint provides a valid approximation of the average height of the function for
each strip. As with the grid method, refined estimates are possible by using more (and thinner)
strips to approximate the integral.

5.2. NUMERICAL DIFFERENTIATION

5.2.1. Lower Order Methods

By truncating the second- and higher-derivatives in the Taylor series

f ′′(xi ) 2
f ( xi +1 ) = f ( xi ) + f ′( xi )h + h + .... (4.3)
2

the following formula for approximating the first derivative, called the first forward difference, is
obtained

f (xi +1 ) − f (xi ) ∆f
f ′(x ) = + O(h ) = i + O(h ) (4.4)
h h

where ∆f i is referred to as the first forward difference and h is called the step size. The entire
term ∆f h is referred to as a first finite divided difference.

Similarly, by truncating the Taylor series between xi −1 and xi ,

f ′′( xi ) 2
f ( xi ) = f ( xi −1 ) − f ′( xi )h + h − ... (4.5)
2!

the following formula for approximating the first derivative is obtained

f ( xi −1 ) − f ( xi ) ∇f
f ′( xi ) = + O(h ) = i + O(h ) (4.6)
h h

where ∇f i is referred to as the first backward difference.

A third way to approximate the first derivative is to subtract backward Taylor series exapnsion
(between xi −1 and xi ) from the forward Taylor series expansion (between xi and xi +1 ) resulting
in

Bahir Dar University; Water Resource Eng. Dep. 31

Numerical Method Lecture note Differentiation & Integration

f 3 ( xi ) 3
f ( xi +1 ) = f ( xi −1 ) + 2 f ( xi )h + h + ... (4.7)
3!
from which we obtain

f ( xi +1 ) − f ( xi −1 )
f ′( xi ) =
2h
( )
− O h2 (4.8)

The above equation is a centered difference representation of the first derivative.

5.2.2. High-Accuracy Differentiation Formulas

High-accuracy divided difference formulas can be generated by including additional terms from
the Taylor series expansion. For example, the forward Taylor series expansion can be written as

f ′′(xi ) 2
f ( xi +1 ) = f ( xi ) + f ′( xi )h + h + ... (4.3)
2

which can be solved for

f ( xi +1 ) − f ( xi ) f ′′( xi ) 2
f ′( xi ) =
h
−
2
h + O h2 ( ) (4.4)

The result can be truncated by excluding the second- and higher-derivative terms and were thus
left with a final result of

f ( xi +1 ) − f ( xi )
f ′( xi ) = + O(h ) (4.5)
h

We can retain the second-derivative term by substituting the following approximation of the
second derivative

f ( xi + 2 ) − 2 f ( xi +1 ) + f ( xi )
f ′′( xi ) = + O(h ) (4.6)
h2

into Eq. 4.4 to yield

f ( xi +1 ) − f ( xi ) f ( xi + 2 ) − 2 f ( xi +1 ) + f ( xi )
f ′( xi ) = − ( )
h + O h2 (4.7)
h 2h 2

or, collecting terms,

− f ( xi + 2 ) + 4 f ( xi +1 ) − 3 f ( xi )
f ′( xi ) =
2h
( )
+ O h2 (4.8)

Bahir Dar University; Water Resource Eng. Dep. 32

Numerical Method Lecture note Differentiation & Integration

Notice that the inclusion of the second-derivative term has improved the accuracy to O h 2 . ( )
Similar improved versions can be developed for the backward and centered formulas as well as
for the approximations of the higher derivatives. The formulas are given below.

Forward Finite-divided Difference Formulas

First Derivative Error

f ( xi +1 ) − f ( xi )
f ′( xi ) = O(h )
h
− f ( xi + 2 ) + 4 f ( xi +1 ) − 3 f ( xi )
f ′( xi ) =
2h
( )
O h2

Second Derivative
f ( xi + 2 ) − 2 f ( xi +1 ) + f ( xi )
f ′′( xi ) =
h2
( )
O h2
− f ( xi +3 ) + 4 f ( xi + 2 ) − 5 f ( xi +1 ) + 2 f (xi )
f ′′( xi ) = ( )
O h2
h2

Third Derivative
f ( xi +3 ) − 3 f ( xi + 2 ) + 3 f ( xi +1 ) − f ( xi )
f ′′′( xi ) = O(h )
h3
− 3 f ( xi + 4 ) + 14 f ( xi +3 ) − 24 f ( xi + 2 ) + 18 f ( xi +1 ) − 5 f ( xi )
f ′′′( xi ) =
2h 3
( )
O h2

Fourth Derivative
f ( xi + 4 ) − 4 f ( xi +3 ) + 6 f ( xi + 2 ) − 4 f ( xi +1 ) + f ( xi )
f ( 4 ) ( xi ) = O(h )
h4
− 2 f ( xi +5 ) + 11 f ( xi + 4 ) − 24 f ( xi +3 ) + 26 f ( xi + 2 ) − 14 f ( xi +1 ) + 3 f ( xi )
f ( 4 ) ( xi ) =
h4
( )
O h2

Backward Finite-divided Difference Formulas

First Derivative Error

f ( xi ) − f ( xi −1 )
f ′( xi ) = O(h )
h
− 3 f ( xi ) − 4 f ( xi −1 ) + f ( xi − 2 )
f ′( xi ) =
2h
( )
O h2

Second Derivative
f ( xi ) − 2 f ( xi −1 ) + f ( xi − 2 )
f ′′( xi ) = O(h )
h2
2 f ( xi ) − 5 f ( xi −1 ) + 4 f (xi − 2 ) − f ( xi −3 )
f ′′( xi ) =
h2
( )
O h2

Bahir Dar University; Water Resource Eng. Dep. 33

Numerical Method Lecture note Differentiation & Integration

Third Derivative
f ( xi ) − 3 f ( xi −1 ) + 3 f ( xi − 2 ) − f ( xi −3 )
f ′′′( xi ) = O(h )
h3
5 f ( xi ) − 18 f ( xi −1 ) + 24 f ( xi − 2 ) − 14 f (xi −3 ) + 3 f (xi − 4 )
f ′′′( xi ) =
2h 3
( )
O h2

Fourth Derivative
f ( xi ) − 4 f ( xi −1 ) + 6 f ( xi − 2 ) − 4 f ( xi −.3 ) + f ( xi − 4 )
f ( 4 ) ( xi ) = O(h )
h4
3 f ( xi ) − 14 f ( xi −1 ) + 26 f ( xi − 2 ) − 24 f ( xi −3 ) + 11 f ( x x − 41 ) − 2 f ( xi −5 )
f ( 4 ) ( xi ) =
h4
( )
O h2

Centered Finite-divided Difference Formulas

First Derivative
f ( xi +1 ) − f ( xi −1 )
f ′( xi ) =
2h
( )
O h2

− f ( xi + 2 ) + 8 f ( xi +1 ) − 8 f ( xi −1 ) + f ( xi − 2 )
f ′( xi ) =
12h
( )
O h4

Second Derivative
f ( xi +1 ) − 2 f ( xi ) + f ( xi −1 )
f ′′( xi ) =
h2
( )
O h2
− f ( xi + 2 ) + 16 f ( xi +1 ) − 30 f ( xi ) + 16 f ( xi −1 ) − f ( xi − 2 )
f ′′( xi ) =
12h 2
( )
O h4

Third Derivative
f ( xi + 2 ) − 2 f ( xi +1 ) + 2 f ( xi −1 ) − f ( xi − 2 )
f ′′′( xi ) =
2h 3
( )
O h2
− f ( xi +3 ) + 8 f ( xi + 2 ) − 13 f ( xi +1 ) + 13 f ( xi −1 ) − 8 f ( xi − 2 ) + f ( xi −3 )
f ′′′( xi ) =
8h 3
( )
O h4

Fourth Derivative
f ( xi + 2 ) − 4 f ( xi +1 ) + 6 f ( xi ) − 4 f ( xi −1 ) + f ( xi − 2 )
f ( 4 ) ( xi ) =
h 4
O h2 ( )
− f ( xi +3 ) + 12 f ( xi + 2 ) − 39 f ( xi +1 ) + 56 f ( xi ) − 39 f ( xi −1 ) + 12 f ( xi − 2 ) + f ( xi −3 )
f ( 4 ) ( xi ) =
6h 4
O h4 ( )

Bahir Dar University; Water Resource Eng. Dep. 34

Numerical Method Lecture note Differentiation & Integration

5.3. NEWTON-COTES INTEGRATION FORMULAS

The Newton-Cotes formulas are the most common numerical integration schemes. They are
based on the strategy of replacing a complicated function or tabulated data with an approximating
function that is easy to integrate:

b b
I = ∫ f ( x )dx ≅ ∫ f n (x )dx (4.9)
a a

where fn(x) = a polynomial of the form

f n ( x ) = a 0 + a1 x + ... + a n −1 x n −1 + a n x n (4.10)

where n is the order of the polynomial.

The integral can be approximated by one polynomial or using a series of polynomials applied
piecewise to the function or date over segments of constant length.

Closed and open forms of the Newton-Cotes formulas are available. The closed forms are those
where the data points at the beginning and end of the limits of integration are known. The open
forms have integration limits that extend beyond the range of the data. In this sense, they are akin
to extrapolation. Open Newton-Cotes formulas are not generally used for definite integration.
However, they are utilized for evaluating improper integrals and for the solution of ordinary
differential equations.

5.3.1. The Trapezoidal Rule

The trapezoidal rule is the first of the Newton-Cotes closed integration formulas. It corresponds
to the case where the polynomial in Eq. 4.9 is first-order:

b b
I = ∫ f ( x )dx ≅ ∫ f1 ( x )dx (4.11)
a a

The straight line passing through the two points (a, f(a)) and (b, f(b)) is given by

f (b ) − f (a )
f1 ( x ) = f (a ) + (x − a ) (4.12)
b−a

The area under this straight line is an estimate of the integral of f(x) between the limits a and b:

 f (b ) − f (a )
(x − a )dx
b
I = ∫  f (a ) + (4.13)
a 
b−a 

The result of the integration is

Bahir Dar University; Water Resource Eng. Dep. 35

Numerical Method Lecture note Differentiation & Integration

f (a ) + f (b )
I = (b − a ) (4.14)
2

which is called the trapezoidal rule.

Error of the Trapezoidal Rule

When we employ the integral under a straight line to approximate the integral under the curve, we
obviously can incur an error that may be substantial. An estimate of the local truncation error of a
single application of the trapezoidal rule is

f ′′(ξ )(b − a )
1
Et = −
3
(4.15)
12

where ξ lies somewhere in the interval from a to b.

Multiple-Application of Trapezoidal Rule

One way to improve the accuracy of the trapezoidal rule is to divide the integration interval from
a to b into a number of segments and apply the method to each segment. The areas of the
individual segments can then be added to yield the integral for the entire interval. The resulting
equations are called multiple-application, or composite, integration formulas.

Considering n + 1 equally spaced base points (x0, x1, x2, ..., xn), and n segments of equal width:

b−a
h= (4.16)
n

If a and b are designated as x0 and xn, respectively, the total integral can be represented as

x1 x2 xn

I= ∫ f ( x )dx + ∫ f ( x )dx + ... + ∫ f (x )dx (4.17)

x0 x1 xn −1

Substituting the trapezoidal rule for each interval yields

f ( x0 ) + f ( x1 ) f ( x1 ) + f ( x 2 ) f ( x n −1 ) + f ( x n )
I =h +h + ... + h (4.18)
2 2 2

or, grouping terms,

h n −1

I=  f ( x 0 ) + 2 ∑ f ( xi ) + f (x n ) (4.19)
2 i =1 

or, using Eq. 4.16 to express Eq. 4.19 in the following form,

Bahir Dar University; Water Resource Eng. Dep. 36

Numerical Method Lecture note Differentiation & Integration

n −1
f ( x 0 ) + 2∑ f ( x i ) + f ( x n )
I = (b − a ) i =1
(4.20)
2n
Width Average Height

Because the summation of the coefficients of f(x) in the numerator divided by 2n is equal to 1, the
average height represents a weighted average of the function values. According to Eq. 4.20, the
interior points are given twice the weight of the two end points f(x0) and f(xn).

An error for the multiple-application trapezoidal rule can be obtained by summing the individual
errors for each segment to give

(b − a ) n
Et = −
12n 3
∑ f ′′(ξ )
i =1
i (4.21)

where f"(ξi) is the second derivative at a point ξi located in segment i. This result can be
simplified by estimating the mean or the average value of the second derivative for the entire
interval as

∑ f ′′(ξ ) i
f ′′ ≅ i =1
(4.22)
n

Therefore ∑ f ′′(ξ ) ≅ nf ′′ and Eq. 4.21 can be rewritten as

i

Ea = −
(b − a )3 f ′′ (4.23)
12n 2

Thus, if the number in the bracket is doubled, the truncation error will be quartered. Note that Eq.
4.23 is an approximate error because the approximate nature of Eq. 4.22.

NB: In the multiple-application of the trapezoidal formula, the error decreases as the number of
segments n increases. However, the rate of decrease of error is gradual. (Doubling the number of
segments quarters the error.) In the subsequent sections, we will develop higher-order formulas
that are more accurate and that converge more quickly on the true integral as the segments are
increased.

5.3.2. Simpson's Rules

Aside from applying the trapezoidal rule with finer segmentation, another way to obtain a more
accurate estimate of an integral is to use higher-order polynomial to connect the points. For
example, if there is an extra point midway between f(a) and f(b), the three points can be
connected with a parabola. If there are two points equally spaced between f(a) and f(b), the four
points can be connected with a third-order polynomial. The formulas that result from taking the
integrals under these polynomials are called Simpson's rules.

Bahir Dar University; Water Resource Eng. Dep. 37

Numerical Method Lecture note Differentiation & Integration

Simpson's 1/3 Rule

Simpson's 1/3 rule results when a second-order interpolating polynomial is substituted into
Eq. 4.9:

b b
I = ∫ f ( x )dx ≅ ∫ f 2 ( x )dx (4.24)
a a

If a and b are designated as x0 and x2 and f2(x) is represented by a second-order Lagrange

polynomial, the intergral becomes

 (x − x1 )(x − x 2 )
x2
(x − x0 )(x − x2 ) (x − x1 )(x − x2 ) ( )
I = ∫ f (x0 ) + f (x1 ) +
(x0 − x1 )(x0 − x2 ) 2 
f x dx
x 
(x0 − x1 )(x0 − x2 ) (x0 − x1 )(x0 − x2 )
(4.25)

After integration and algebraic manipulation, the following formula results

I≅
h
[ f (x0 ) + 4 f (x1 ) + f (x2 )] (4.26)
3

where, for this case, h = (b − a ) 2 . This equation is known as Simpson's 1/3 rule.

Simpson's 1/3 rule can also be expressed in the following format

f ( x0 ) + 4 f ( x1 ) + f ( x 2 )
I ≅ (b − a ) `(4.21)
6
Width Average Height

where a = x0 , b = x 2 , and x1 = the point midway between a and b, which is given by

(b + a ) 2 . Notice that, according to the above equation, the middle point is weighted two-thirds
and the two end points by one-sixth.

It can be shown that a single-segment application of Simpson's 1/3 rule has a truncation error of

h f (ξ )
1 5 (4 )
Et = − (4.27)
90

or, because h = (b − a ) 2 ,

Et = −
(b − a )5 f (4 ) (ξ ) (4.28)
2880

where ξ lies somewhere in the interval from a to b. Thus, Simpson's 1/3 rule is more accurate that
the trapezoidal rule.

Bahir Dar University; Water Resource Eng. Dep. 38

Numerical Method Lecture note Differentiation & Integration

Multiple Application of Simpson's 1/3 Rule

Just as with the trapezoidal rule, Simpson's rule can be improved by dividing the integration
interval into a number of segments of equal width:

b−a
h= (4.29)
n

The total integration can be represented as

x2 x4 xn

I= ∫ f (x )dx + ∫ f (x )dx + ... + ∫ f (x )dx

x0 x2 xn − 2
(4.30)

Substituting Simpson's 1/3 rule for the individual integral yields

f ( x0 ) + 4 f ( x1 ) + f ( x 2 ) f ( x 2 ) + 4 f ( x3 ) + f ( x 4 )
I ≅ 2h + 2h + ...
6 6
(4.31)
f ( x n − 2 ) + 4 f ( x n −1 ) + f ( x n )
+ 2h
6

or, combining terms, and using Eq. 4.29,

n −1 n−2
f (x0 ) + 4 ∑ f ( xi ) + 2 ∑ f (x ) + f (x )
j n

I ≅ (b − a )
i =1, 3, 5 j = 2, 4, 6
(4.32)
3n
Width Average Height

An error estimate for the multiple-application Simpson's rule is obtained in the same fashion as
for the trapezoidal rule by summing the individual errors for the segments and averaging the
derivative to yield

Ea = −
(b − a )5 f (4 ) (4.33)
4
180n

where f (4 ) is the average fourth derivative for the interval.

Simpson's 3/8 Rule

In a similar manner to the derivation of the trapezoidal and Simpson's 1/3 rule, a third-order
Lagrange polynomial can be fit to four points and integrated:

b b
I = ∫ f ( x )dx ≅ ∫ f 3 ( x )dx (4.34)
a a

Bahir Dar University; Water Resource Eng. Dep. 39

Numerical Method Lecture note Differentiation & Integration

to yield

3h f ( x0 ) + 3 f ( x1 ) + 3 f ( x 2 ) + f ( x3 )
I≅ (4.35)
8 8

where h = (b − a ) 3 . This equation is called Simpson's 3/8 rule because h is multiplied by 3/8. It
is the third Newton-Cotes closed integration formula. The 3/8 rule can also be expressed in the
following form:

f ( x0 ) + 3 f ( x1 ) + 3 f ( x 2 ) + f ( x3 )
I ≅ (b − a ) (4.36)
8
Width Average Height
Thus, the two interior points are given weight's are given weights of three-eighths, whereas the
end points are weighted with one-eighth. Simpson's 3/8 rule has an error of

Et = −
(b − a )5 f (4 ) (ξ ) (4.37)
6480

Simpson's 1/3 rule is usually the method of preference becasue it attains thord-order accuracy
with three points rather than the four points required for the 3/8 version. However, the 3/8 rule
has utility when the number of segments is odd.

Summary of the Newton-Cotes integration formulas is given below.

Segments Points Name Formula Truncation Error

(n)
(x0 ) + f (x1 )
1 2 Trapezoidal rule (b − a ) f − (1 12 )h 3 f ′′(ξ )
2
f ( x0 ) + 4 f ( x1 ) + f ( x 2 )
2 3 Simpson's 1/3 rule (b − a ) − (1 90 )h 5 f (4 ) (ξ )
6
f ( x0 ) + 3 f ( x1 ) + 3 f ( x 2 ) + f ( x3 )
3 4 Simpson's 3/8 rule (b − a ) − (3 80 )h 5 f (4 ) (ξ )
8

Bahir Dar University; Water Resource Eng. Dep. 40

Numerical Method Lecture note Ordinary Differential Eqn.

6. Numerical Solution of Ordinary Differential Equations

6.1 INTRODUCTION

Differential equations are equations composed of an unknown function and its derivatives. The
following are examples of differential equations:

dv c
=g− v (6.1a)
dt m

d 2x dx
m 2
+ c + kx = 0 (6.1b)
dt dt

Differential equations can be classified as ordinary differential equations (ODE) or partial

differential equations (PDE). ODEs are equations in which the function involves one independent
variable. PDE's involve functions two or more variables. Both Eq.(6.1a) and Eq.(6.1b) are
ordinary differential equations. Differential equations are also classified as to their order. For
Eq.(6.1a) is a first-order equation because the highest derivative is a first derivative. A second
order equation includes a second derivative, as in Eq. (6.1b) above. Similarly, an nth-order
equation includes an nth derivative.

Non-computer Methods for Solving ODEs

Without computers, ODEs are usually solved with analytical integration techniques. For example,
Eq. (6.1a) could be multiplied by dt and integrated to yield

 c 
v = ∫  g − v dt (6.2)
 m 

The right-hand side of the above equation is called an indefinite integral because the limits of
integration are unspecified. An analytic solution for the above equation is obtained if the
indefinite integral can be evaluated exactly in equation form. For example, Eq. (6.2) can be
solved analytically (assuming v = 0 at t = 0 ):

v(t ) =
gm
c
(
1 − e −(c m )t ) (6.3)

Analytical solutions of ODEs such as Eq.(6.30) are very useful. However, the exact solutions for
many ODEs of practical importance are not available. Thus, numerical methods offer the only
viable alternative for these cases.

One important method that engineers and applied mathematicians developed to overcome the
problem is linearization. A linear ordinary differential equation is one that fits the general form

a n (x ) y (n ) + ... + a1 (x ) y ′ + a 0 ( x ) y = f (x ) (6.4)

Bahir Dar University; Water Resource Eng. Dep. 41

Numerical Method Lecture note Ordinary Differential Eqn.

where y (n ) is the nth derivative of y with respect to x and a's and f's are specified functions of x.
The practical importance of linear ODEs is that they can be solved analytically. In contrast, most
nonlinear equations cannot be solved exactly.

6.2 RUNGE-KUTTA METHODS

This chapter is devoted to solving ordinary differential equations of the form

= f ( x, y )
dy
(6.5)
dx

One-step Runge-Kutta (RK) methods can be generally expressed as

y i +1 = y i + φh (6.6)

According to this equation, the slope estimate of φ is used to extrapolate from an old value y i to
new value y i +1 over a distance h . This formula can be applied step by step to compute values of
y . The simplest approach is to use the differential equation to estimate the slope in the form of
first derivative at xi .

6.2.1 Euler's Method

The first derivative provides a direct estimate of the slope at xi . That is,

φ = f ( xi , y i ) (6.7)

where f ( xi , y i ) is the differential equation evaluated at xi and y i . This estimate can be

substituted into Eq. (6.6):

y i +1 = y i + f ( xi , y i )h (6.8)

This formula is referred to as Euler's method.

Error Analysis for Euler's Method

As in any other numerical procedure, the numerical solution of ODEs involves two types of errors
(1) Truncation error, and (2) Round-off errors. The truncation errors are composed of two parts.
The first is a local truncation error that results from the application of the method over a single
step. The second is a propagated truncation error that results from the approximation produced
during the previous steps. The sum of the two is the total or global truncation error.

Insight into the magnitude and properties of the truncation error can be gained by deriving Euler's
method directly from the Taylor series expansion. Remember that the differential equation to be
solved will be of the general form
y ′ = f ( x, y ) (6.9)

Bahir Dar University; Water Resource Eng. Dep. 42

Numerical Method Lecture note Ordinary Differential Eqn.

where y ′ = dy dx , and x and y are the independent and the dependent variables, respectively.
If the solution has continuous derivatives, it can be represented by a Taylor series expansion
about a starting value ( xi , y i ) ,

y i′′ 2 y (n )
y i +1 = y i′ + y i′h + h + ... + i h n + Rn (6.10)
2! n!

where h = xi +1 − xi , and Rn = the remainder term, defined by

y (n +1) (ξ ) n +1
Rn = h (6.11)
(n + 1)!
where ξ lies somewhere in the interval from xi to xi +1 . An alternative form can be developed
bu substititing Eq.(6.9) into Eq.(6.10) and (6.11) to yield

f ′(xi , y i ) 2 f (n −1) (xi , y i )

y i +1 = y i + f (xi , y i )h +
2!
h + ... +
n!
( )
+ O h n +1 (6.12)

( )
where O h n +1 specifies that the local truncation error is proportional to the step size raised to the
(n + 1)th power.

Thus we can see that Euler's method corresponds to the Taylor series up to and including the term
f ( xi , y i )h . The true local truncation error, Et , in the Euler method is thus given as

f ′( xi , y i ) 2
Et =
2!
( )
h + ... + O h n +1 (6.13)

For sufficiently small h , the errors in the terms in Eq.(6.13) usually decrease as the order
increases, and the result is often represented as

f ′(xi , y i )
Ea = h2 (6.14)
2!

or

Ea = O h 2( ) (6.15)

where E a = the approximate local truncation error.

6.2.2 Runge-Kutta Methods

Runge-Kutta (RK) methods achieve the accuracy of a Taylor series approach without having
requiring the calculation of higher derivatives. Many variations exist but all can be cast in the
generalized form:

Bahir Dar University; Water Resource Eng. Dep. 43

Numerical Method Lecture note Ordinary Differential Eqn.

y i +1 = y i + φ ( xi , y i , h )h (6.16)

where φ ( xi , y i , h ) is called an increment function, which can be interpreted as a representative

slope over the interval. The increment function can be written in the general form as

φ = a1 k1 + a 2 k 2 + ... + a n k n (6.17)

where the a 's are constants and the k 's are

k1 = f ( x i , y i ) (6.18a)
k 2 = f ( xi + p1 h,. y i + q11 k1 h ) (6.18b)
k 3 = f ( xi + p 2 h, y i + q 22 k 2 h ) (6.18c)
.
.
.
k n = f (xi + p n −1 h, y i + q n −1, 2 k 2 h + ... + q n −1,n −1 k n −1 h ) (6.18d)

Notice that the k's are recurrence relationships.

Various types of Runge-Kutta methods can be devised by employing different numbers in terms
in the increment function as specified by n. The first-order Runge-Kutta method with n = 1 is, in
fact, Euler's method. Once n is chosen, values for the a's, p's and q's are evaluated by setting Eq.
6.16 equal to terms in a Taylor series expansion. The fourth order Runge-Kutta is one of the
relatively accurate and widely used. Thus it will be discussed in detail next.

Fourth-Order Runge-Kutta Methods

There are an infinite number of versions of the fourth-order RK method. The following, called the
classical fourth-order RK method, is the most commonly used:

y i +1 = y i +
1
(k1 + 2k 2 + 2k 3 + k 4 )h (6.19)
6

where
k1 = f ( x i , y i )
 1 1 
k 2 = f  xi + h, y i + k1 h 
 2 2 
 1 1 
k 3 = f  xi + h, y i + k 2 h 
 2 2 
k 4 = f ( x i + h, y i + k 3 h )

Bahir Dar University; Water Resource Eng. Dep. 44