The Islamic University of Gaza

Faculty of Engineering
Civil Engineering Department

Numerical Analysis
ECIV 3306

Chapter 6

Open Methods
 Open Methods
• Bracketing methods are based on assuming an
interval of the function which brackets the root.
• The bracketing methods always converge to the
root.
• Open methods are based on formulas that
require only a single starting value of x or two
starting values that do not necessarily bracket
the root.
• These method sometimes diverge from the true
root.
 Open Methods-
Convergence and Divergence Concepts

f(x) f(x)

x xi xi+1 x
xi xi+1

Diverging increments Converging increments

 1. Simple Fixed-Point Iteration

• Rearrange the function so that x is on

the left side of the equation:
f ( x)  0  g ( x)  x
xi 1  g ( xi )

• Bracketing methods are “convergent”.

• Fixed-point methods may sometime “diverge”,
depending on the stating point (initial guess)
and how the function behaves.
 Simple Fixed-Point Iteration
Examples:
1. f ( x)  x 2  x  2 x0
g ( x)  x 2  2
or
g ( x)  x  2
or
2
g ( x)  1 
x
2. f(x) = x 2-2x+3  x = g(x)=(x2+3)/2
3. f(x) = sin x  x = g(x)= sin x + x
3. f(x) = e-x- x  x = g(x)= e-x
 Simple Fixed-Point Iteration Convergence

• x = g(x) can be
expressed as a pair of
equations:
y1= x
y2= g(x)…. (component
equations)
• Plot them separately.
Simple Fixed-Point Iteration Convergence

x i 1  g (x i )    1
Suppose that the true root:
x r  g (x r )     2
Subtracting 1 from  2 
x r  x i 1  g (x r )  g (x i )   (3)
 Simple Fixed-Point Iteration Convergence

Derivative mean value theorem:

If g(x) are continuous in [a,b] then there exist at
least one value of x= within the interval such
that:
g b   g a 
g   
'

b a

i.e. there exist one point where the slope parallel to the line
joining (a & b)
 Simple Fixed-Point Iteration Convergence
x r  x i 1  g (x r )  g (x i )
Let a  x i and b  x r
g x r   g x i 
g   
'

xr xi
g  x r   g  x i    x r  x i  g '  
then x r  x i 1   x r  x i  g '  
E t ,i 1  g '   E t ,i
If g '    1.0 the error decreases with each iteration
If g '    1.0 the error increases with each iteration
 Simple Fixed-Point Iteration Convergence

• Fixed-point iteration converges if :

g (x )  1 (slope of the line f (x )  x )
• When the method converges, the error
is roughly proportional to or less than the
error of the previous step, therefore it is
called “linearly convergent.”
Simple Fixed-Point Iteration-Convergence
 Example: Simple Fixed-Point Iteration

f(x) = e-x - x f(x)

f(x)=e-x - x
1. f(x) is manipulated so that we get
x=g(x) g(x) = e-x
Root x
2. Thus, the formula predicting the
new value of x is: xi+1 = e-xi
f(x)
3. Guess xo = 0 f1(x) = x

4. The iterations continues till the g(x) = e-x

approx. error reaches a certain
limiting value
x
 Example: Simple Fixed-Point Iteration

i xi g(xi) ea% et%

0 0 1.0
1 1.0 0.367879 100 76.3
2 0.367879 0.692201 171.8 35.1
3 0.692201 0.500473 46.9 22.1
4 0.500473 0.606244 38.3 11.8
5 0.606244 0.545396 17.4 6.89
6 0.545396 0.579612 11.2 3.83
7 0.579612 0.560115 5.90 2.2
8 0.560115 0.571143 3.48 1.24
9 0.571143 0.564879 1.93 0.705
10 0.564879 1.11 0.399
 Example: Simple Fixed-Point Iteration

i xi g(xi) ea% et%

0 0 1.0
1 1.0 0.367879 100 76.3
2 0.367879 0.692201 171.8 35.1
3 0.692201 0.500473 46.9 22.1
4 0.500473 0.606244 38.3 11.8
5 0.606244 0.545396 17.4 6.89
6 0.545396 0.579612 11.2 3.83
7 0.579612 0.560115 5.90 2.2
8 0.560115 0.571143 3.48 1.24
9 0.571143 0.564879 1.93 0.705
10 0.564879 1.11 0.399
Flow Chart – Fixed Point

Start

Input: xo , es, maxi

i=0
ea=1.1es

1
 1

while False
ea< es &
i >maxi

x n  g x 0  Print: xo, f(xo) ,ea , i

i  i 1

i=1
or Stop
xn=0

True x n  xo
ea  100%
xn

x0=xn
 2. The Newton-Raphson Method

• Most widely used method.

• Based on Taylor series expansion:
x 2
f ( xi 1 )  f ( xi )  f ( xi )x  f ( xi )  ...
2!
The root is the value of x i 1 when f(x i 1 )  0
Rearrangin g, Solve for

0  f(xi )  f (xi )( xi 1  xi )
f ( xi ) Newton-Raphson formula
xi 1  xi 
f ( xi )
 The Newton-Raphson Method

• A tangent to f(x) at the

initial point xi is extended f(x)
till it meets the x-axis at f(xi)
the improved estimate of Slope f /(xi)
f(x)
the root xi+1. x
Root
• The iterations continues xi+1 xi
till the approx. error f ( xi )  0
f ( xi ) 
/
reaches a certain limiting xi  xi 1
value. f ( xi )
xi 1  xi 
f / ( xi )
 Example: The Newton Raphson Method
• Use the Newton-Raphson method to find the
root of e-x-x= 0  f(x) = e-x-x and f`(x)= -e-x-1;
thus x x
f ( xi ) e x e x
xi 1  xi  /  xi   x  xi   x
f ( xi )  e 1 e 1

Iter. xi e t%
0 0 100
1 0.5 11.8
2 0.566311003 0.147
3 0.567143165 0.00002
4 0.567143290 <10-8
Flow Chart – Newton Raphson

Start

Input: xo , es, maxi

i=0
ea=1.1es

1
 1

while False
ea >es &
i <maxi

f x 0 
xn  x0  Print: xo, f(xo) ,ea , i
f '
x 0 
i  i 1

i=1
or Stop
xn=0

True x n  xo
ea  100%
xn

x0=xn
Pitfalls of The Newton Raphson Method
 3. The Secant Method

The derivative f / (x i ) is
replaced by a backward
finite divided difference
f (x i )  f (x i 1 )
f (x i ) 
/

x i  x i 1
Thus, the formula
predicting the xi+1 is:
f ( xi )( xi 1  xi )
xi 1  xi 
f ( xi 1 )  f ( xi )
 The Secant Method

• Requires two initial estimates of x , e.g,xo, x1.

However, because f(x) is not required to change
signs between estimates, it is not classified as a
“bracketing” method.
• The scant method has the same properties as
Newton’s method. Convergence is not
guaranteed for all xo, x1, f(x).
 Secant Method: Example

• Use the Secant method to find the root of e-x-x=0;

f(x) = e-x-x and xi-1=0, x0=1 to get x1 of the first
iteration using:

f ( xi )( xi 1  xi )
xi 1  xi 
f ( xi 1 )  f ( xi )
Iter xi-1 f(xi-1) xi f(xi) xi+1 et%
1 0 1.0 1.0 -0.632 0.613 8.0
2 1.0 -0.632 0.613 -0.0708 0.5638 0.58
3 0.613 -0.0708 0.5638 0.00518 0.5672 0.0048
Comparison of convergence of False Position and
Secant Methods
False Position Secant Method
f (x u )(x l  x u ) f (x i )(x i 1  x i )
x r  xu  x i 1  x i 
f (x l )  f (x u ) f (x i 1 )  f (x i )
Use two estimate xl and xu Use two estimate xi and xi-1

f(x) must changes signs between xl f(x) is not required to change signs
and xu between xi and xi-1

Xr replaces whichever of the original Xi+1 replace xi

values yielded a function value with Xi replace xi-1
the same sign as f(xr)

Always converge May be diverge

 Comparison of convergence of False Position and
Secant Methods
• Use the false-position and secant method to find the root of
f(x)=lnx. Start computation with xl= xi-1=0.5, xu=xi = 5.
1. False position method

Iter xl xu xr
1 0.5 5.0 1.8546
2 0.5 1.8546 1.2163
3 0.5 1.2163 1.0585
2. Secant method
Iter xi-1 xi xi+1
1 0.5 5.0 1.8546
2 5 1.8546 -0.10438
 False Position and Secant Methods
Although the secant
method may be
divergent, when it
converges it usually xl xi-1
does so at a quicker
xu xi
rate than the false
position method

See the next figure

• Comparison of
the true percent
relative Errors Et
for the methods
to the determine
the root of
f(x)=e-x-x
Flow Chart – Secant Method

Start

Input: x-1 , x0,es, maxi

i=0
ea=1.1es

1
 1

while False
ea >es &
i < maxi

f (x i )(x i 1  x i )
x i 1  x i  Print: xi , f(xi) ,ea , i
f (x i 1 )  f (x i )
i  i 1

i=1
or Stop
Xi+1=0

True x i 1  x i
ea  100%
x i 1

Xi-1=xi
Xi=xi+1
 Modified Secant Method
Rather than using two initial values, an alternative
approach is using a fractional perturbation of the
independent variable to estimate f / (x )
i

f (x i   x i )  f (x i )
f (x i ) 
/

xi
 is a small perturbation fraction

 x i  f (x i )
x i 1  xi 
f (x i   x i )  f (x i )
 Modified Secant Method: Example
• Use the modified secant method to find the root of
f(x) = e-x-x and, x0=1 and =0.01
First Iteration
x 0 1 f  x 0   0.63212
x 0   x 0  1.01 f  x 0   x 0   0.64578
 x i  f (x i )
x 1  x i 1  x i   0.537263 e t  5.3%
f (x i   x i )  f (x i )
Second Iteration
x 1  0.537263 f  x 1   0.047083
x 1   x 1  0.542635 f  x 1   x 1   0.038579
 x i  f (x i )
x 2  x i 1  x i   0.56701 e t  0.0236%
f (x i   x i )  f (x i )
 Multiple Roots

f(x)= (x-3)(x-1)(x-1) f(x)= (x-3)(x-1)(x-1)(x-1)

= x3- 5x2+7x -3 = x4- 6x3+ 125 x2- 10x+3

f(x)
f(x)

Double roots
3 triple roots
1
x 1 3 x
 Multiple Roots

•“Multiple root” corresponds to a point

where a function is tangent to the x axis.

•Difficulties
- Function does not change sign with double
(or even number of multiple root), therefore,
cannot use bracketing methods.
- Both f(x) and f′(x)=0, division by zero with
Newton’s and Secant methods which may
diverge around this root.
 4. The Modified Newton Raphson Method

• Another u(x) is introduced such that u(x)=f(x)/f /(x);

• Getting the roots of u(x) using Newton Raphson
technique:
This function has roots
u ( xi )
xi 1  xi  / at all the same locations
u ( xi ) as the original function

f /
( x ) f /
( x )  f ( x ) f //
( xi )
u ( xi ) 
/ i i i
[ f / ( xi )]2
/
f ( xi ) f ( xi )
xi 1  xi 
f /
2
( xi )  f ( xi ) f // ( xi )
 Modified Newton Raphson Method: Example
Using the Newton Raphson and Modified Newton
Raphson evaluate the multiple roots of
f(x)= x3-5x2+7x-3 with an initial guess of x0=0
•Newton Raphson formula:
f ( xi ) xi3  5 xi2  7 xi  3
xi 1  xi  /  xi 
f ( xi ) 3 xi  10 x  7
2

•Modified Newton Raphson formula:

f ( xi ) f / ( xi )
xi 1  xi 
f /
( x )  f ( x ) f
i
2
i
//
( xi )
( xi3  5 xi2  7 xi  3)(3xi2  10 xi  7)
 xi 
(3xi2  10 xi  7) 2  ( xi3  5 xi2  7 xi  3)(6 xi  10)
 Modified Newton Raphson Method: Example
Newton Raphson Modified Newton-Raphson
Iter xi et% iter xi et%
0 0 100 0 0 100
1 0.4286 57 1 1.10526 11
2 0.6857 31 2 1.00308 0.31
3 0.83286 17 3 1.000002 00024
4 0.91332 8.7
5 0.95578 4.4
6 0.97766 2.2

•Newton Raphson technique is linearly converging towards the

true value of 1.0 while the Modified Newton Raphson is
quadratically converging.
•For simple roots, modified Newton Raphson is less efficient
and requires more computational effort than the standard
Newton Raphson method
 Systems of Nonlinear Equations

• Roots of a set of simultaneous equations:

f1(x1,x2,…….,xn)=0
f2 (x1,x2,…….,xn)=0

fn (x1,x2,…….,xn)=0
• The solution is a set of x values that
simultaneously get the equations to zero.
 Systems of Nonlinear Equations

Example: x2 + xy = 10 & y + 3xy2 = 57

u(x,y) = x2+ xy -10 = 0
v(x,y) = y+ 3xy2 -57 = 0
• The solution will be the value of x and y which makes
u(x,y)=0 and v(x,y)=0
• These are x=2 and y=3
• Numerical methods used are extension of the open
methods for solving single equation; Fixed point
iteration and Newton-Raphson.
 Systems of Nonlinear Equations:
1.Fixed Point Iteration
1. Use an initial guess x =1.5 and y =3.5
2. The iteration formulae:
xi+1=(10-xi2)/yi and yi+1=57-3xiyi2
3. First iteration,
x=(10-(1.5)2)/3.5=2.21429
y=(57-3(2.21429)(3.5)2=-24.37516
4. Second iteration:
x=(10-2.214292)/-24.37516=-0.209
y=57-3(-0.209)(-24.37516)2=429.709
5. Solution is diverging so try another iteration formula
 Systems of Nonlinear Equations:
1.Fixed Point Iteration
1. Using iteration formula:
xi+1=(10-xiyi)1/2 and yi+1=[(57-yi)/3xi]1/2
First guess: x=1.5 and y=3.5
2. 1st iteration:
x=(10-(1.5)(3.5))1/2=2.17945
y=((57-(3.5))/3(2.17945))1/2=2.86051
3. 2nd iteration:
x=(10-(2.17945)(2.86051))1/2=1.94053
y=((57-(2.86051))/3(1.94053))1/2=3.04955
4. The approach is converging to the true root, x=2 and y=3
 Systems of Nonlinear Equations:
1.Fixed Point Iteration

The sufficient condition for convergence for the

two-equation case (u(x,y)=0 and v(x,y)=0) are:

u v
 1
x x
and
u v
 1
y y
 Systems of Nonlinear Equations:
2. Newton Raphson Method
• Recall the standard Newton Raphson formula:
f ( xi )
xi 1  xi 
f '( xi )
• which can be written as the following formula
xi 1  xi  xi
f ( xi )
where xi  
f '( xi )
f '( xi )  xi   f ( xi )
 Systems of Nonlinear Equations:
2. Newton Raphson Method
• By multi-equation version (in this section we deal only with
two equation) the formula can be derived in an identical
fashion:
• u(x,y)=0 and v(x,y)=0
 ui ui 
 x y   x  u 
  i   i
 vi vi  yi   vi 
 x y 
 
1
 ui ui 
 xi   x y  ui 
      
yi   vi vi   vi 
 x y 
 
 Systems of Nonlinear Equations:
2. Newton Raphson Method
1
 ui ui   vi ui 
 x 
y  1  y y 
    
 vi vi  ui vi vi ui  vi ui 
  
 x  x y x y  x
 y  x 
• And thus

vi ui vi ui

ui   vi ui   vi
y y x x
xi 1  xi  yi 1  yi 
ui vi vi ui ui vi vi ui
   
x y x y x y x y
 Systems of Nonlinear Equations:
2. Newton Raphson Method
• x 2+ xy =10 and y + 3xy 2 = 57
are two nonlinear simultaneous equations with two unknown x
and y they can be expressed in the form:
u u
 2 x  y, x
x y
v v
 3y2 ,  1  6 xy
x y
i xi yi Ui vi Ui,x ui,y Vi,x Vi,y ea,x ea,y
0 1.5 3.5 -2.5 1.625 6.5 1.5 36.75 32.5

1 2.03603 2.84388 -.06435 -4.7560 6.91594 2.03603 24.26296 35.74135 26.3 23.1

2 1.9987 3.00229 1.87 5.27