The Islamic University of Gaza

Faculty of Engineering
Civil Engineering Department

Numerical Analysis
ECIV 3306

Chapter 6
Open Methods
Open Methods
Bracketing methods are based on assuming an
interval of the function which brackets the root.
The bracketing methods always converge to the
root.
Open methods are based on formulas that
require only a single starting value of x or two
starting values that do not necessarily bracket
the root.
These method sometimes diverge from the true
root.
Open Methods-
Convergence and Divergence Concepts
Converging increments
f(x)
x
i
x
i+1

x
f(x)
x
i
x
i+1

x
Diverging increments
1. Simple Fixed-Point Iteration
Rearrange the function so that x is on
the left side of the equation:
) (
) ( 0 ) (
1 i i
x g x
x x g x f
=
= =
+
Bracketing methods are convergent.
Fixed-point methods may sometime diverge,
depending on the stating point (initial guess)
and how the function behaves.

Simple Fixed-Point Iteration
Examples:
1.







2. f(x) = x
2
-2x+3 x = g(x)=(x
2
+3)/2
3. f(x) = sin x x = g(x)= sin x + x
3. f(x) = e
-x
- x x = g(x)= e
-x



x
x g
or
x x g
or
x x g
x x x x f
2
1 ) (
2 ) (
2 ) (
0 2 ) (
2
2
+ =
+ =
=
> =
Simple Fixed-Point Iteration Convergence
x = g(x) can be
expressed as a pair of
equations:
y
1
= x
y
2
= g(x). (component
equations)
Plot them separately.


Simple Fixed-Point Iteration Convergence
( )
( )
( ) ( )
1
1
( )
( ) 1
Suppose that the true root:
Subtracting 1 from 2
( ) ( ) (3)
2
r
i i
r i r i
r
x g x
x x x
x
x
g
g
x
g
+
+
=
=
=

Simple Fixed-Point Iteration Convergence

Derivative mean value theorem:
If g(x) are continuous in [a,b] then there exist at
least one value of x=c within the interval such
that:


i.e. there exist one point where the slope parallel to the line
joining (a & b)

( )
( ) ( )
'
g b g a
g
b a
c

=

Simple Fixed-Point Iteration Convergence

( )
( ) ( )
( ) ( ) ( ) ( )
( ) ( )
( )
( )
( )
1
'
'
'
1
'
, 1 ,
'
'
( ) ( )
Let and

1.0 theerror decreases with eachiteration
1.0 theerror increases with eachiteration
r i r i
i r
r i
r i
r i r i
r i r i
t i t i
x x g x g x
a x b x
g x g x
g
x x
g x g x x x g
then x x x x g
E g E
If g
If g
c
c
c
c
c
c
+
+
+
=
= =

=
=
=
<
>
Simple Fixed-Point Iteration Convergence
Fixed-point iteration converges if :

( ) 1 (slope of the line ( ) ) g x f x x
'
< =
When the method converges, the error
is roughly proportional to or less than the
error of the previous step, therefore it is
called linearly convergent.
Simple Fixed-Point Iteration-Convergence
Example: Simple Fixed-Point Iteration
1. f(x) is manipulated so that we get
x=g(x) g(x) = e
-x

2. Thus, the formula predicting the
new value of x is: x
i+1
= e
-
x
i
3. Guess xo = 0
4. The iterations continues till the
approx. error reaches a certain
limiting value
f(x)
Root x
f(x)
x
f(x)=e
-x
- x
g(x) = e
-x
f
1
(x) = x
f(x) = e
-x
- x
Example: Simple Fixed-Point Iteration
i x
i
g(x
i
) c
o
% c
t
%

0 0 1.0
1 1.0 0.367879 100 76.3
2 0.367879 0.692201 171.8 35.1
3 0.692201 0.500473 46.9 22.1
4 0.500473 0.606244 38.3 11.8
5 0.606244 0.545396 17.4 6.89
6 0.545396 0.579612 11.2 3.83
7 0.579612 0.560115 5.90 2.2
8 0.560115 0.571143 3.48 1.24
9 0.571143 0.564879 1.93 0.705
10 0.564879 1.11 0.399
Example: Simple Fixed-Point Iteration
i x
i
g(x
i
) c
o
% c
t
%

0 0 1.0
1 1.0 0.367879 100 76.3
2 0.367879 0.692201 171.8 35.1
3 0.692201 0.500473 46.9 22.1
4 0.500473 0.606244 38.3 11.8
5 0.606244 0.545396 17.4 6.89
6 0.545396 0.579612 11.2 3.83
7 0.579612 0.560115 5.90 2.2
8 0.560115 0.571143 3.48 1.24
9 0.571143 0.564879 1.93 0.705
10 0.564879 1.11 0.399
Flow Chart Fixed Point
Start
Input: x
o
, c
s
, maxi

i=0
c
a
=1.1c
s

1
Stop
1
while
c
a
< c
s
&
i >maxi
i=1
or
x
n
=0
x
0
=x
n

100%
n o
a
n
x x
x
c

=
Print: x
o
, f(x
o
) ,c
a
, i
( )
0
1
n
x g x
i i
=
= +
False
True
2. The Newton-Raphson Method
Most widely used method.
Based on Taylor series expansion:

) (
) (
) ( 0
g, Rearrangin
0 ) f(x when x of value the is root The
...
! 2
) ( ) ( ) ( ) (
1
1
1 i 1 i
2
1
i
i
i i
i i i i
i i i i
x f
x f
x x
x x ) (x f ) f (x
x
x f x x f x f x f
'
=

'
+ =
=
+
A
' '
+ A
'
+ =
+
+
+ +
+
Solve for
Newton-Raphson formula
The Newton-Raphson Method
A tangent to f(x) at the
initial point x
i
is extended
till it meets the x-axis at
the improved estimate of
the root x
i+1
.
The iterations continues
till the approx. error
reaches a certain limiting
value.

f(x)
Root
x
x
i

x
i+1

f(x)
Slope f
/
(x
i
)
f(x
i
)
) (
) (
) (
) (
/
/
i
i
i 1 i
1 i i
i
i
x f
x f
x x
x x
0 x f
x f
=

=
+
+
Example: The Newton Raphson Method
1 1 ) (
) (
/
1
+

+ =


= =

+
x
x
i
x
x
i
i
i
i i
e
x e
x
e
x e
x
x f
x f
x x
Use the Newton-Raphson method to find the
root of e
-x
-x= 0 f(x) = e
-x
-x and f`(x)= -e
-x
-1;
thus

Iter. x
i
c
t
%
0 0 100
1 0.5 11.8
2 0.566311003 0.147
3 0.567143165 0.00002
4 0.567143290 <10
-8
Flow Chart Newton Raphson
Start
Input: x
o
, c
s
, maxi

i=0
c
a
=1.1c
s

1
Stop
1
while
c
a
>c
s
&
i <maxi
i=1
or
x
n
=0
x
0
=x
n

100%
n o
a
n
x x
x
c

=
Print: x
o
, f(x
o
) ,c
a
, i

( )
( )
0
0
'
0
1
n
f x
x x
f x
i i
=
= +
False
True
Pitfalls of The Newton Raphson Method
3. The Secant Method
The derivative is
replaced by a backward
finite divided difference
) ( ) (
) )( (
i 1 i
i 1 i i
i 1 i
x f x f
x x x f
x x

+
Thus, the formula
predicting the x
i+1
is:
/
1
1
( ) ( )
( )
i i
i
i i
f x f x
f x
x x

/
( )
i
f x
The Secant Method
Requires two initial estimates of x , e.g, x
o
, x
1
.
However, because f(x) is not required to change
signs between estimates, it is not classified as a
bracketing method.
The scant method has the same properties as
Newtons method. Convergence is not
guaranteed for all x
o
, x
1
, f(x).

Secant Method: Example
Use the Secant method to find the root of e
-x
-x=0;
f(x) = e
-x
-x and x
i-1
=0, x
0
=1 to get x
1
of the first
iteration using:
Iter x
i-1
f(x
i-1
) x
i
f(x
i
) x
i+1
c
t
%
1 0 1.0 1.0 -0.632 0.613 8.0
2 1.0 -0.632 0.613 -0.0708 0.5638 0.58
3 0.613 -0.0708 0.5638 0.00518 0.5672 0.0048

) ( ) (
) )( (
i 1 i
i 1 i i
i 1 i
x f x f
x x x f
x x

+
Comparison of convergence of False Position and
Secant Methods


False Position Secant Method


Use two estimate x
l
and x
u

Use two estimate x
i
and x
i-1
f(x) must changes signs between x
l

and x
u
f(x) is not required to change signs
between x
i
and x
i-1
X
r
replaces whichever of the original
values yielded a function value with
the same sign as f(x
r
)
X
i+1
replace x
i
X
i
replace x
i-1

Always converge May be diverge
1
1
1
( )( )
( ) ( )
i i i
i i
i i
f x x x
x x
f x f x

( )( )
( ) ( )
u l u
r u
l u
f x x x
x x
f x f x

Comparison of convergence of False Position and

Secant Methods
Use the false-position and secant method to find the root of
f(x)=lnx. Start computation with x
l
= x
i-1
=0.5, x
u
=x
i
= 5.
1. False position method







2. Secant method
Iter x
i-1
x
i
x
i+1

1 0.5 5.0 1.8546
2 5 1.8546 -0.10438


Iter x
l
x
u
x
r

1 0.5 5.0 1.8546
2 0.5 1.8546 1.2163
3 0.5 1.2163 1.0585
False Position and Secant Methods
x
i-1
x
i
x
u
x
l
Although the secant
method may be
divergent, when it
converges it usually
does so at a quicker
rate than the false
position method

See the next figure
Comparison of
the true percent
relative Errors E
t

for the methods
to the determine
the root of
f(x)=e
-x
-x

Flow Chart Secant Method
Start
Input: x
-1
, x
0
,c
s
, maxi

i=0
c
a
=1.1c
s

1
Stop
1
while
c
a
>c
s
&
i < maxi
i=1
or
X
i+1
=0
X
i-1
=x
i
X
i
=x
i+1

1
1
100%
i i
a
i
x x
x
c
+
+

=
Print: x
i
, f(x
i
) ,c
a
, i

1
1
1
( )( )
( ) ( )
1
i i i
i i
i i
f x x x
x x
f x f x
i i

= +
False
True
Modified Secant Method
Rather than using two initial values, an alternative
approach is using a fractional perturbation of the
independent variable to estimate
1
( )
( ) ( )
i i
i i
i i i
x f x
x x
f x x f x
o
o
+

=
+
o is a small perturbation fraction
/
( ) ( )
( )
i i i
i
i
f x x f x
f x
x
o
o
+
~
/
( )
i
f x
Modified Secant Method: Example
Use the modified secant method to find the root of
f(x) = e
-x
-x and, x
0
=1 and o=0.01
( )
( )
( )
( )
0 0
0 0 0 0
1 1
1 1
1 1 1 1
2 1
1 0.63212
1.01 0.64578
( )
0.537263
First Iteration
Second Iteration
5.3%
( ) ( )
0.537263 0.047083
0.542635 0.038579
( )
(
i i
i i t
i i i
i i
i i
x f x
x x f x x
x f x
x x x
f x x f x
x f x
x x f x x
x f x
x x x
f x
o o
o
c
o
o o
o
+
+
= =
+ = + =

= = = =
+
= =
+ = + =

= = 0.56701 0.0236%
) ( )
t
i i i
x f x
c
o
= =
+
Multiple Roots
x
f(x)= (x-3)(x-1)(x-1)
= x
3
- 5x
2
+7x -3
f(x)
1
x
3
Double roots
f(x)= (x-3)(x-1)(x-1)(x-1)
= x
4
- 6x
3
+ 125 x
2
- 10x+3
f(x)
1 3
triple roots
Multiple Roots
Multiple root corresponds to a point
where a function is tangent to the x axis.

Difficulties
- Function does not change sign with double
(or even number of multiple root), therefore,
cannot use bracketing methods.
- Both f(x) and f(x)=0, division by zero with
Newtons and Secant methods which may
diverge around this root.
4. The Modified Newton Raphson Method
Another u(x) is introduced such that u(x)=f(x)/f
/
(x);
Getting the roots of u(x) using Newton Raphson
technique:
| | ) ( ) ( ) (
) ( ) (
)] ( [
) ( ) ( ) ( ) (
) (
) (
) (
//
2
/
/
1
2 /
// / /
/
/
1
i i i
i i
i i
i
i i i i
i
i
i
i i
x f x f x f
x f x f
x x
x f
x f x f x f x f
x u
x u
x u
x x

=
=
+
+
This function has roots
at all the same locations
as the original function
Modified Newton Raphson Method: Example
Using the Newton Raphson and Modified Newton
Raphson evaluate the multiple roots of
f(x)= x
3
-5x
2
+7x-3 with an initial guess of x
0
=0
| |
) 10 6 )( 3 7 5 ( ) 7 10 3 (
) 7 10 3 )( 3 7 5 (
) ( ) ( ) (
) ( ) (
2 3 2 2
2 2 3
//
2
/
/
1
+ +
+ +
=

=
+
i i i i i i
i i i i i
i
i i i
i i
i i
x x x x x x
x x x x x
x
x f x f x f
x f x f
x x
7 x 10 x 3
3 x 7 x 5 x
x
x f
x f
x x
2
i
i
2
i
3
i
i
i
i
i 1 i
+
+
= =
+
) (
) (
/
Newton Raphson formula:
Modified Newton Raphson formula:
Newton Raphson Modified Newton-Raphson
Iter x
i
c
t
% iter x
i
c
t
%
0 0 100 0 0 100
1 0.4286 57 1 1.10526 11
2 0.6857 31 2 1.00308 0.31
3 0.83286 17 3 1.000002 00024
4 0.91332 8.7
5 0.95578 4.4
6 0.97766 2.2
Newton Raphson technique is linearly converging towards the
true value of 1.0 while the Modified Newton Raphson is
quadratically converging.
For simple roots, modified Newton Raphson is less efficient
and requires more computational effort than the standard
Newton Raphson method
Modified Newton Raphson Method: Example
Systems of Nonlinear Equations
Roots of a set of simultaneous equations:
f
1
(x
1
,x
2
,.,x
n
)=0
f
2
(x
1
,x
2
,.,x
n
)=0

f
n
(x
1
,x
2
,.,x
n
)=0
The solution is a set of x values that
simultaneously get the equations to zero.
Systems of Nonlinear Equations
Example: x
2
+ xy = 10 & y + 3xy
2
= 57
u(x,y) = x
2
+ xy -10 = 0
v(x,y) = y+ 3xy
2
-57 = 0
The solution will be the value of x and y which makes
u(x,y)=0 and v(x,y)=0
These are x=2 and y=3
Numerical methods used are extension of the open
methods for solving single equation; Fixed point
iteration and Newton-Raphson.
Systems of Nonlinear Equations:
1.Fixed Point Iteration
1. Use an initial guess x =1.5 and y =3.5
2. The iteration formulae:
x
i+1
=(10-x
i
2
)/y
i
and y
i+1
=57-3x
i
y
i
2

3. First iteration,
x=(10-(1.5)
2
)/3.5=2.21429
y=(57-3(2.21429)(3.5)
2
=-24.37516
4. Second iteration:
x=(10-2.21429
2
)/-24.37516=-0.209
y=57-3(-0.209)(-24.37516)
2
=429.709
5. Solution is diverging so try another iteration formula
Systems of Nonlinear Equations:
1.Fixed Point Iteration
1. Using iteration formula:
x
i+1
=(10-x
i
y
i
)
1/2
and y
i+1
=[(57-y
i
)/3x
i
]
1/2
First guess: x=1.5 and y=3.5
2. 1st iteration:
x=(10-(1.5)(3.5))
1/2
=2.17945
y=((57-(3.5))/3(2.17945))
1/2
=2.86051
3. 2nd iteration:
x=(10-(2.17945)(2.86051))
1/2
=1.94053
y=((57-(2.86051))/3(1.94053))
1/2
=3.04955
4. The approach is converging to the true root, x=2 and y=3

Systems of Nonlinear Equations:
1.Fixed Point Iteration
The sufficient condition for convergence for the
two-equation case (u(x,y)=0 and v(x,y)=0) are:
1
1
u v
x x
and
u v
y y
c c
+ <
c c
c c
+ <
c c
Systems of Nonlinear Equations:
2. Newton Raphson Method
Recall the standard Newton Raphson formula:
1
( )
'( )
i
i i
i
f x
x x
f x
+
=
which can be written as the following formula
1
( )
'( )
'( ) ( )
i i i
i
i
i
i i i
x x x
f x
where x
f x
f x x f x
+
= + A
A =
A =
By multi-equation version (in this section we deal only with
two equation) the formula can be derived in an identical
fashion:
u(x,y)=0 and v(x,y)=0
1
i i
i i
i i i i
i i
i i
i i i i
u u
x u x y
y v v v
x y
u u
x u x y
y v v v
x y

c c


A c c

=
` `
A c c
) )

c c

c c


A c c

=
` `
A c c
) )

c c

Systems of Nonlinear Equations:
2. Newton Raphson Method
And thus
1
1
i i
i i
i i i i
i i
i i
u u
v u
x y
y y
u v v u
v v
v u
x y x y
x y
x x

c c
c c

c c
c c

=
c c c c
c c
c c




c c c c
c c
c c

1
i i
i i
i i
i i i i
v u
u v
y y
x x
u v v u
x y x y
+
c c

c c
=
c c c c

c c c c
1
i i
i i
i i
i i i i
v u
u v
x x
y y
u v v u
x y x y
+
c c

c c
= +
c c c c

c c c c
Systems of Nonlinear Equations:
2. Newton Raphson Method
x
2
+ xy =10 and y + 3xy
2
= 57
are two nonlinear simultaneous equations with two unknown x
and y they can be expressed in the form:
2
2 ,
3 , 1 6
u u
x y x
x y
v v
y xy
x y
c c
= + =
c c
c c
= = +
c c
i x
i
y
i
U
i
v
i
U
i,x
u
i,y
V
i,x
V
i,y c
a,x
c
a,y
0 1.5 3.5 -2.5 1.625 6.5 1.5 36.75 32.5
1 2.03603 2.84388 -.06435 -4.7560 6.91594 2.03603 24.26296 35.74135 26.3 23.1
2 1.9987 3.00229 1.87 5.27
Systems of Nonlinear Equations:
2. Newton Raphson Method