II.

Root Finding
Objectives:
Develop various methods for solving r=x that satisfies f(x)=0
in which y=f(x) is a nonlinear function.
2.1 Bisection Method
2.1.1 Procedure
Step 1: updating
Set x0=a, x1=b; make sure f0*f1<0
Make own convention: f0>0, f1<0.
f 0 >0 Get x2=(x0+x1)/2
y Check sign of f2=f(x2).
f 2 >0 SetIfx0f=a,
2>0,x1replace
=b; make x0 by x2f0*f1<0
sure
a Make own convention:
b f0>0, f1<0.
fx00 >0 x2 Get x2=(x0+x1x)/2 1 x
y Check sign of f2=f(x2).
f 2 >0 f 1 <0
Since f2>0, replace x0 by x2
a Comments on how to start: b
i) One x 1)/2.
x0 must guarantee x2 f(a)*f(b)<0
**Getin order
x2=(x 0+xto1 begin x the iteration
Check: is |f2| < tolerance?
ii) If f(a)*f(b)<0, Intermediate Value Theorem
if yes, f 1 <0the
accept =>root
a root r in2.(a, b)
as x=x
y
if not, check sign of f2
Step 2: checking convergence if f2<0 replace x1 by x2,
f0 >0 **Get x =(x +x1replace
2 if f20>0 )/2. x0 by x2
Check:goto is |f**2 | < tolerance?
if yes, accept the root as x=x2.
y x2 if not, check x1 sign of x f2
x0
if f2<0 replace x1 by x2,
f0 >0 f <0 if f2>0 replace x0 by x2
2
goto **f 1 <0
y x0 x2 x1 x

f 0 >0 f2 <0
x2 f 1 <0 x
y x1 of Florida
x0 Mei, University
Notes organized by Prof. Renwei 1

f 0 >0 f 1 <0
x2 x
 x0 x2 x1 x

f2 <0
f 1 <0
y

f 0 >0
x2 x
x0 x1 (=old x2)

f 1 <0
Comments on checking convergence
(for all methods of root finding):
i) It is easier to use the magnitude of |f(xn)| to see if xn can be
accepted as a root.
ii) Sometimes, however, it is inadequate. Consider f(x)=(x-1)3.
When xn =1+0.01 with an error of | xn –r|=0.01, f(xn)=10-6
which may be accepted by an inexperienced user.

0.03
f(x)
0.02

0.01

-0.01

-0.02

-0.03
0.7 0.8 0.9 1 1.1 1.2 x

iii) It is thus more consistent to use |xn+1-xn| <  = tolerance

Notes organized by Prof. Renwei Mei, University of Florida 2

Example: solve the root of f(x) = x-1 = 0 (so we know r = 1)
Soln.: Start from x0 = 0 and x1 = 3
with f0 = -1 and f1 =2 (=< f0f1 = -2<0).
Hence r(0, 3). That is, the root is bracketed.
1
Now find x2 = ( x0 + x1 ) = 1.5 => f2 = 0.5.
2
(check convergence: | f2 | = 0.5 obviously too big!)
Since f0 f2 = -0.5<0, x2→ x1 (x2 =1.5 will be new x1).
1
=> New x2 = (0 + 1.5) = 0.75 => f2 = -0.25.
2
(check convergence: | f2 | = 0.25 is still too big!
Or |New x2 – Old x2|= |0.75-0.5|=0.25 is too big)
Since f1 f2 <0, x2→ x0.
1
=> new x2 = (0.75 + 1.5) = 1.125…
2
... eventually, x2 = 1 after a large number of iterations.

Observations & comments:

* This method is slow.
 Initial interval length: b - a.
 After every iteration, the interval is reduced by a factor of 2.
 After n iterations, | x1- x0| = (b-a) / 2n.
 If we use the stopping criterion:
| x1- x0| < 1 = 2n ≥(b-a)/ 1
or n ≥ log [(b-a)/1] /log 2.
= we know how many iterations are needed
to reach a desired accuracy regardless the behavior of f(x).

Notes organized by Prof. Renwei Mei, University of Florida 3

2.1.2 On the Convergence speed
 Definition:
Let x = r = actual root,
xn = nth iteration (obtained using any method)
=> en = r - xn = error of nth iteration
How is en+1 related to en?
If en+1 = k1 en with k1 < 1 = linear convergence.
k1  rate of convergence;
it DOES NOT VARY MUCH from one
iteration to next iteration.
If en+1 = k2 en2 , = quadratic convergence.

If en+1 = kp enp , = order of convergence = p.

* For Bisection method: linear convergence (p=1); & k1 ~ 1/2

Example: f(x) = x 2 -2x -3 = 0

Solution: Since f(x) = (x+1)(x-3), it is seen that there are two root:
x = -1 and x = 3.
We consider the one at r = 3.
Use (x0, x1) = (a, b) = (2.33, 3.51)

Notes organized by Prof. Renwei Mei, University of Florida 4

 estimate:
n xn f(x2) |en| = |r-xn| (b-a)/2n
1 2.92 -0.3136 0.08 0.59
2 3.215 0.906225 0.215 0.295
3 3.0675 0.2745563 0.0675 0.1475
4 2.99375 -0.0249609 0.00625 0.07375
5 3.030625 0.1234379 0.030625 0.036875
6 3.0121875 0.0488985 0.0121875 0.018438
7 3.00296875 0.0118838 0.0029687 0.009219
8 2.998359375 -0.0065598 0.0016406 0.004609
9 3.000664063 0.0026567 0.0006641 0.002305
10 2.999511719 -0.0019529 0.0004883 0.001152
11 3.000087891 0.0003516 8.789E-05 0.000576
12 2.999799805 -0.0008007 0.0002002 0.000288
13 2.999943848 -0.0002246 5.615E-05 0.000144
14 3.000015869 6.348E-05 1.587E-05 7.2E-05
15 2.999979858 -8.057E-05 2.014E-05 3.6E-05
16 2.999997864 -8.545E-06 2.136E-06 1.8E-05
17 3.000006866 2.747E-05 6.866E-06 9E-06
18 3.000002365 9.46E-06 2.365E-06 4.5E-06
19 3.000000114 4.578E-07 1.144E-07 2.25E-06
20 2.999998989 -4.044E-06 1.011E-06 1.13E-06
21 2.999999552 -1.793E-06 4.482E-07 5.63E-07
22 2.999999833 -6.676E-07 1.669E-07 2.81E-07
23 2.999999974 -1.049E-07 2.623E-08 1.41E-07
24 3.000000044 1.764E-07 4.411E-08 7.03E-08
25 3.000000009 3.576E-08 8.941E-09 3.52E-08
26 2.999999991 -3.457E-08 8.643E-09 1.76E-08
27 3.00000000015 5.96E-10 1.49E-10 8.79E-09
28 2.99999999575 -1.699E-08 4.247E-09 4.4E-09

Notes organized by Prof. Renwei Mei, University of Florida 5

 1.E+00
|r-xn|
1.E-01
estimate
1.E-02
1.E-03
1.E-04
1.E-05
1.E-06
1.E-07
Convergence of
1.E-08 Bisection method
1.E-09
1.E-10
0 5 10 15 20 25 30 n 35
Observations:
i. Estimate en ~(b-a)/2n, gives an upper bound; agree with data.
ii. the rate of the convergence can be estimated from the above as 1/2.

Notes organized by Prof. Renwei Mei, University of Florida 6

 Another pitfall of bisection method:

f(x) Cannot use bisection method

if a & b are not properly
selected.
It will not find double root either.

a b x

f(x)
Bisection method in this case
will only give one root.

a b x

Notes organized by Prof. Renwei Mei, University of Florida 7

2.2 Newton's Method
2.2.1 General idea:
* Linearly approximating the curve near the root.
=> Use the tangent to approximate f(x) locally.
 Most widely used because of its fast convergence.
f0
y tangent

x1
x2
x0 x

f1

 General procedure: (Suppose x is close to the root r).

i) Compute the slope of the curve at x0: k = f ( x0 ) = f0'
approximate the curve by the tangent: y = f0 + k(x-x0)
which can be also viewed as a two-term TS expansion.
ii) Set y=0, solve for x: x1 = x0 - f0/f0'
iii) Check for convergence: is | x1 - x0 | < tolerance?

If not, set x0 = x1, compute f0 and f0', go to step ii).

Notes organized by Prof. Renwei Mei, University of Florida 8

Example 1: f(x) = 3x + sin(x) - ex
Solution: Derivatives : f (x) = 3 + cos(x) – ex

f ( x) = − sin( x) − e x .
Start with x0 = 0 → f(0) = 0 + 0 -1 = -1
& f (0) = 3 +1-1 = 3

 x1 = x0 - f0/f0' = 0 - (-1)/3 = 0.3333333

→ f(0.3333333) = -0.068417728
& f  (0.3333333) =2.5493445

 x2 = x1 - f1/f1' = 0.3333333-(-0.068417728)/2.5493445
= 0.360171
Iter. # xn f(xn) en =|r-xn| en/ en2−1

0 0 -1 0.36042
1 0.3333333333 -0.6841772829E-01 2.71E-02 0.209
2 0.3601707136 -0.6279850706E-03 2.52E-04 0.346
3 0.3604216805 -0.5625155319E-07 2.25E-08 0.354
4 0.3604217030 -0.6661338148E-15
(treated as exact root r)
Note: f (r ) = 2.501814, f (r ) = -1.786603
1 f (r )
=> − = 0.357
2 f (r )
 Comments:
* It may not converge if x0 is too far from x = r.
* If x0 is close to r, Newton's method converges very fast.
Notes organized by Prof. Renwei Mei, University of Florida 9
Example 2: Find the root for: f(x) = exp(x) –2 = 0
using Newton’s method: xn+1 = xn - f(xn)/f′(xn)
Sol. * Exact result: r = ln(2)
* f'(x)= exp(x)  xn+1 = xn – [exp(xn)-2] /exp(xn)

* x0 = 3  f(3) = 18.085537 & f' = 20.085537

=> x1 = x0 -f0/f0' = 3 -18.085537/20.085537 = 2.0995741…

N xn f(x) f'(x) en = x n - r en/ en2−1

0 3 18.085537 20.085537 2.3068528
1 2.09957414 6.162693 8.162693 1.406427 0.264288
2 1.34459131 1.8366183 3.8366183 0.6514441 0.329339
3 0.86588373 0.3771059 2.3771059 0.1727366 0.407033
4 0.70724297 0.0283912 2.0283912 0.0140958 0.472412
5 0.69324606 0.0001978 2.0001978 9.888E-05 0.497659
6 0.69314719 9.777E-09 2 4.889E-09 0.499984
7 0.69314718 0 2

* Quadratic convergence: en / en2−1 = constant

Notes organized by Prof. Renwei Mei, University of Florida 10

 Newton's method for root finding
1.E+01
1.E+00
1.E-01
1.E-02
parabola
Error

1.E-03
xn-x_exact
1.E-04
1.E-05
1.E-06
f(x)=exp(x)-2
1.E-07
1.E-08
1.E-09
0 1 2 3 4 5 6 7 8 9 n

(This graph shows the quadratic behavior of the error curve in the
Log-linear scale)

Notes organized by Prof. Renwei Mei, University of Florida 11

2.2.1 Quadratic convergence
 Why en / en2−1 = constant?
Scheme: xn+1 = xn – f ( xn ) / f ( xn )
-r: xn+1 –r = xn –r – f ( xn ) / f ( xn ) (*)

Taylor series expansion for f(xn) near x=r:

( xn − r ) 2
f ( xn ) = f (r ) + ( xn − r ) f (r ) + f (r ) + ...
2
1
= 0 − en f (r ) + en2 f (r ) + ... (**)
2
f ( xn ) = f (r ) + ( xn − r ) f (r ) + ... (***)

(** & ***) into (*) =>

1
0 − en f (r ) + en2 f (r ) + ...
-en+1 = -en – 2
f (r ) − en f (r ) + ...

1
1 − en f (r ) / f (r ) + ...
2
= -en + en
1 − en f (r ) / f (r ) + ...

(for the denominator, let   en f (r) / f (r)

& use 1/(1-)=1+  + 2… )

1 f (r ) 1 f (r )
= -en + en [1 + en + ...] = en2 + ...
2 f (r ) 2 f (r )
1 f (r )
=> en+1 / en2 = − + ...
2 f (r )

Notes organized by Prof. Renwei Mei, University of Florida 12

2.2.3Failure of Newton’s method
The following graphs show some typical cases when
Newton's method fails to find the root when initial guess x0 is too far
from the root.
y f0

x2 x3 x1
x0 x

f3
f1
f2

* Instead of finding r, it tries

to find the false solution in y

(x2, x3)➔ divergence

* Iteration diverges when x0
is a little too far x1 x0 x2 x

Notes organized by Prof. Renwei Mei, University of Florida 13

 2.3 Secant Method
 Motivation:
Newton’s method converges very rapidly when it works.
But in some applications, it is difficult or expensive to compute f (x) .
=> An alternative strategy must be developed to take advantage
of the rapid convergence of Newton’s method.
2.3.1 Idea and procedure
Suppose x0 and x1 are close to x = r.

f1 f0 x0 & x1 are near the root x=r.

y f0 and f1 do not have to be of
opposite signs.
x2
x1 x0 x
f2
i. Approximate the curve of y = f(x) near the root by a line as
f1 − f 0
y = f1 + (x-x1)
x1 − x0
ii. Set y = 0 in the above to solve for x2:
x1 − x0 x1 − x0
x2 = x1 - f1 = x0 - f0
f1 − f 0 f1 − f 0

iii. If f2= f(x2) < tolerance , accept r = x2. Exit computation.

If f2 > , set x0= x1, f0= f1;
then set x1= x2, f1= f2,
Go to step ii to solve for new x2.

Notes organized by Prof. Renwei Mei, University of Florida 14

 Example: f(x) = 3x + sin x - ex
Note, for 0<x<2, there are two roots.
2
f(x)
1

0
x2
-1

-2

-3

-4
-2 -1 0 1 2 3 x
If we choose x0 = 1, x1 =0, then f0 =1.123189, f1 = -1
x1 − x0 0 −1
=> x2 = x0 - f0 =1 - 1.233189 = 0.4709896;
f1 − f 0 − 1 − 1.123189

f(x2)= 0.2651588…
Iter. #. x0 x1 x2 f2
0 1 0 => 0.4709896 0.26515588
1 0 0.4709896 0.37227705 0.02953367
2 0.4709896 0.37227705 0.359904249 -0.001294787
3 0.37227705 0.359904249 0.3604239134 5.496E-6
4 0.359904249 0.360423913 0.3604217034 1.02E-9
Take r ≈ x4 (4th iteration of x2 column), we note r - x3 ≈ 2.2E-6

* Comparison with the bisection method for this problem:

1− 0
If bisection method is used, n = log( ) / log 2  19 .
2 .2 E − 6
=> Secant method is much more efficient than bisection.
Notes organized by Prof. Renwei Mei, University of Florida 15
 2.3.2 On the (superlinear) convergence of the secant method:
xn − xn −1
* Scheme: xn+1 –r = xn –r - fn (&)
f n − f n −1

* Taylor series expansion for f(x) near the root for f(xn) & f(xn-1)

( xn − r ) 2
f ( xn ) = f (r ) + ( xn − r ) f (r ) + f (r ) + ...
2
1
= 0 − en f (r ) + en2 f (r ) + ... (&&)
2

( xn −1 − r ) 2
f ( xn −1 ) = f (r ) + ( xn −1 − r ) f (r ) + f (r ) + ...
2
1
= 0 − en −1 f (r ) + en2−1 f (r ) + ... (&&&)
2
xn − xn −1 = xn − r − ( xn −1 − r ) = −en + en −1

1
f n − f n −1 = − (en − en −1 ) f (r ) + (en2 − en2−1 ) f (r ) + ...
2
(&) =>:
1 − (en − en −1)
-en+1 = -en + en [ f (r ) − en f (r )...]
2 1
− (en − en −1 ) f (r ) + (en2 − en2−1 ) f (r ) + ...
2
1 1
= -en + en [ f (r ) − en f (r )...]
2 1
f (r ) − (en + en −1 ) f (r ) + ...
2
1 1
f (r ) − (en + en −1 ) f (r ) + ... − [ f (r ) − en f (r )...]
en+1 = en 2 2
1
f (r ) − (en + en −1 ) f (r ) + ...
2
1
− en −1 f (r ) + ...
f (r )
= en 2  - en en-1
f (r ) + ... 2 f (r )

Notes organized by Prof. Renwei Mei, University of Florida 16

 That is en+1  - en en-1 f (r ) /[2 f (r )] = A en en-1

Let en+1 = k enq

then en-1 = (en / k )1 / q

 k enq = A en (en / k )1 / q

 q = 1+ 1/q & k = A k −1 / q
 q = ( 5 + 1) / 2 =1.618 or q= (− 5 + 1) / 2 = -0.618
Expect the error to decrease as n increases  take q=1.618.
 k 1+1 / q = A

 k = A1 / q = A0.618 = f (r ) /[2 f (r )] 0.618

Finally, en+1 = k e1n.618 ; => p =1.618; k = f (r ) /[2 f (r )] 0.618 .

* p =1.618 >1  superlinear.
Better than linear; slow than quadratic (p=2)
* Comparison with Newton’s method:
Slower, but no need to evaluate f (x) .
* Apply secant method only when x0 and x1 are close to the root x = r.
Otherwise, the method may fail.
Caution:
|f 0 | > |f 1 |
y

x0 x1 x2 x3 x
Iterat ion does not
converge
Notes organized by Prof. Renwei Mei, University of Florida 17
2.4 Method of False Position (Regula Falsi)
2.4.1 Idea and procedure
To avoid the pathology of the secant method, we can take
advantages of both the secant method and the bisection method
i) Ensure f0f1 <0 so that x0 & x1 bracket the root r. (Bisection!)
ii) Compute x2 = x0 - f0 (x1-x0)/(f1 -f0) -- ( Secant!)
iii) Check the stopping criterion (e.g. |f(x2)| < )
iv) If not converged,
if f0 f2 <0 set x1 = x2 & f 1 = f2 --(bisection)
if f1 f2 <0 set x0 = x2 & f 0 = f2
go to step ii) to compute new x2.
The method of False position guarantees the solution as long as
f0 f1 <0 to start with, but it is slower than the secant method.

y
f1

x3

x1 x2 x0 x

Note: x3 is based on x0 and x2,

not on x1 and x2! f0

Notes organized by Prof. Renwei Mei, University of Florida 18

* Example: Compare the performance of three methods
for solving the root of
f(x) = 3x + sin(x) - exp(x) = 0;
x0 = 0, x1 = 1
Bisection False Position Secant
Iter. # x2 f2 x2 f2 x2 f2
1 0.5 0.330704 0.470990 2.65E-1 0.470990 2.65E-1
2 0.25 -0.28662 0.372277 2.95E-2 0.372277 2.95E-2
3 0.375 0.036281 0.316598 2.94E-3 0.359904 -1.29E-3
4 0.3125 -0.12190 0.360538 2.90E-4 0.360424 5.53E-6
5 0.34375 -0.04196 0.360433 2.83E-5 0.360422 7.43E-7
As expected, false position is faster than bisection
but it is slower than secant method.

Notes organized by Prof. Renwei Mei, University of Florida 19

2.4.2 Comparison with bisection and secant methods
 Typical comparison of convergence histories (solving x3 -3x +1=0 )

10
f
5

0
f
-5

-10
-3 -2 -1 0 1 2 x

Convergence criterion 10 -14

Bisection -- 47 iterations
False position -- 15 iterations
Secant -- 10 iterations
Newton’s -- 6 iterations

Bisection
False
position

Secant

Newton’s

Notes organized by Prof. Renwei Mei, University of Florida 20

* Example: Find the smallest positive root: f(x) = x2 |sinx| - 4=0
using false position method
Solution: Sketch of the function: (poorly behaved!)
10
0.0
9 y = x*x*|sinx| - 4
8
7
-0.1
6
5
4 -0.2
3
2 -0.3
1 2.1 2.2 2.3 2.4 2.5 x
0
-1
-2
-3
-4
-5
0 1 2 3 4 5
x
a=0, b=4, => x1=1.321349…

* Clearly, the root is at 3<r<4, NOT near x=2.3 as seen from the inset.
* The derivative f (x) has a discontinuity at x= for x<4.
* Secant method is essentially a derivative based method;
it may not work. 8

6 f
N xn f(xn) f(xn)
1 1.321349 -2.308077 4
2 1.914858 -0.548215
2
3 2.046901 -0.276157
4 2.111226 -0.177942 0
5 2.151783 -0.129541 0 1 2 3 4
6 2.180845 -0.101817 -2
7 2.203403 -8.45E-02
8 2.221934 -7.31E-02 -4
x
9 2.237829 -6.55E-02 -6
10 2.25195 -6.04E-02

Notes organized by Prof. Renwei Mei, University of Florida 21

 11 2.2649 -5.71E-02 25 2.5098 -0.27981
12 2.2770 -5.53E-02 26 2.5595 -0.39844
13 2.2887 -5.47E-02 27 2.6270 -0.60331
14 2.3001 -5.52E-02 28 2.7221 -0.98179
15 2.3116 -5.69E-02 29 2.8601 -1.72745
16 2.3234 -5.98E-02 30 3.06027 -3.23919
17 2.3357 -6.41E-02 31 3.328504 -1.94126
18 2.3487 -7.02E-02 32 3.458209 -0.27647
19 2.3629 -7.86E-02 33 3.476072 -3.34E-02
20 2.3786 -9.00E-02 34 3.478221 -3.94E-03
21 2.3964 -0.10582 35 3.478475 -4.65E-04
22 2.4171 -0.12794 36 3.478504 -5.63E-05
23 2.4417 -0.15971 37 3.478508 -7.15E-06
24 2.4718 -0.20687 38 3.478508 -4.77E-07

* To use derivative-based methods 4

xn
in the presence of discontinuity,
3.5
we must be very careful.
* The slow convergence results 3

from the fact that this algorithm is 2.5

first trying to find an illusive root
2
near x=2.3 which actually is a local
maximum. 1.5

* The eventual success of finding n

1
the root is ensured because the root is 0 10 20 30 40
bracketed in the False position
method.
* The faster convergence near the end results from the secant-like iteration.

Notes organized by Prof. Renwei Mei, University of Florida 22

2.5 Successive Iteration Method-- fixed point iteration method
2.5.1 General procedure
i. Rewrite the original eqn. f(x) = 0
into a different form x = g(x)
For x = r  r = g(r)
ii. start with initial guess x0, compute g(x0)
iii. get x1 = g(x0)
iv. check for convergence to see if |x1 -x0 |≤.
If not, set x0 = x1 go to step iii.
* Example: f(x) = x3 - x -1 = 0
i. Rearrange x = (x+1)1/3  g(x) = (x+1)1/3
initial guess: x0 = 0

n xn n xn
1 1 7 1.3247018
2 1.2599211 8 1.3247149
3 1.3122939 9 1.32471737
4 1.3223538 10 1.32471785
5 1.3242688 11 1.32471794
6 1.3246326 12 1.32471795

ii. Rearrange the equation as x = x3 - 1  g(x) = x3 - 1

Initial guess: x0 = 0
 x1 = 03 – 1 = -1
 x2 = (−)3 – 1 = -2
 x3 = (−)3 – 1 = -9  diverges !
Notes organized by Prof. Renwei Mei, University of Florida 23
 1 1
iii. Rearrange x( x 2 − 1) = 1  x = , where g(x) 
x2 −1 x2 −1
* initial guess: x0 = 0  x1 = -1, x2→ ∞
* initial guess: x0 = 0.5

n xn n xn
1 -1.333333 8 7.277602
2 1.285714 9 0.019244
3 1.531252 10 -1.000371
4 0.743643 11 1349.3016
5 -2.237161 12 0.000001
6 0.249695 13 -1
7 -1.066493 14 1/0 → ∞

2.5.2 Convergence Requirements

We need: |g'(x)| < 1 near the root x = r
→ Hence, how f(x) = 0 is rearranged into x=g(x) is critical to the
convergence.
→ Properly rearranged the equation leads to fast convergence.

Notes organized by Prof. Renwei Mei, University of Florida 24

* Geometrical interpretation of the convergence requirements:
|g'(x)| > 1 |g'(x)| < 1

y y=g(x) y y=x
y=x
g(x 1)
g(x 2) y=g(x)
g(x 1)
g(x 0)
g(x 0)

r
x0 x1 x2 x x0 x1 x2 x3 x
r
Diverges Converges

Notes organized by Prof. Renwei Mei, University of Florida 25

* Example: Find the root for x = ln(x)+2
using fixed point iteration method
Soln. Start with x0 = 1
n xn |en|=|r-xn| |en/ en-1|
0 1
1 2 1.1461932
2 2.693147181 0.453046 0.3953
3 2.990710465 0.1554828 0.3432
4 3.095510973 0.0506822 0.3260
5 3.129952989 0.0162402 0.3204
6 3.141017985 0.0051752 0.3187
7 3.144546946 0.0016463 0.3181
8 3.145669825 0.0005234 0.3179
9 3.146026848 0.0001664 0.3179
10 3.146140339 5.288E-05 0.3178
11 3.146176412 1.681E-05 0.3179
12 3.146187878 5.342E-06 0.3178
13 3.146191523 1.698E-06 0.3179
14 3.146192681 5.397E-07 0.3178
15 3.146193049 1.715E-07 0.3178
16 3.146193166 5.453E-08 0.3180
17 3.146193203 1.733E-08 0.3178
(r = 3.1461932206205800…)

Notes organized by Prof. Renwei Mei, University of Florida 26

* Graphical illustration of convergence history
3.5
g(x)
3
2.5
2
1.5
g(x)
1
xn
0.5
0
0 1 2 3 4
x

* Note the linear convergence because |en / en-1| ~ constant

Fixed point method for root finding

1.E+02

1.E+00 xn-x_exact
1.E-02
linear
1.E-04
Error

1.E-06

1.E-08
x=Ln(x)+2
1.E-10

1.E-12

1.E-14

1.E-16
n
0 5 10 15 20 25 30 35

(This graph shows the linear convergence of error in log-linear scale)

Notes organized by Prof. Renwei Mei, University of Florida 27

2.6 Muller's Method
◊ Secant method fits f(x) locally using a line based on 2 points.
◊ Consider 3 points: x0, x1, & x2, with f0, f1, & f2;
Convention: place x0 in the middle between x1, & x2
◊ Expect it to be better than secant method by fitting 3 points
with a parabola.

y f0 f1 y=f(x)

f2

x3 x2 x0 x1 x

◊ Procedure:
i. Let h1 = x1- x0, h2 = x2- x0,
s = x - x0;  = h2/h1
parabola: y = as2 + bs + c
 c = f0,
f1 − f 0 (1 +  ) + f 2
a= ,
h1 (1 +  )
2

f1 − f 0 − ah12
b=
h1

1
ii. Set y = 0  s = (−b  b 2 − 4ac )
2a

Note: -b and b 2 − 4ac may be close in magnitude.

Notes organized by Prof. Renwei Mei, University of Florida 28

 subtraction  loss of accuracy.
u2 − v u2 − v
Note: u+ v= , u- v= ,
u− v u+ v

1 (−b) 2 − (b 2 − 4ac )
so that s=
2a − b  b 2 − 4ac

2c
= = x - x0
2
− b  b − 4ac
2c
 x3 = x0 + s = x0 - .
2
b  b − 4ac

iii. Now, which sign (or root of the quadratic equation) to take?
Take "+" if b>0; take " - " if b<0
 possibly largest denominator so that x3 is closer to x0.
iv. Which three points to use to construct next parabola?
If x3>x0, use current (x0, x3, x1) as NEW (x2, x0, x1).
If x3<x0, use current (x2, x3, x0) as NEW (x2, x0, x1).
v. Go to step i. to evaluate a, b, c,...
◊ Example:
f(x) = 3x+sin(x) - exp(x)
using x1=0, x2=1.0, x0=0.5
Iter. # x f(x)
1 0.3549138905 -1.380655671E-02
2 0.3604646779 1.075137341E-04
3 0.3604216977 -1.325225929E-08
4 0.3604217030 4.440892099E-16

Notes organized by Prof. Renwei Mei, University of Florida 29

◊ For the SAME equation: f(x) = 3x+sin(x) - exp(x)
but with x1=0.5, x2=1.5, x0=1
Iter. # x f(x)
1 0.3777378070 4.305176629E-02
2 0.3606852070 6.591760593E-04
3 0.3604218053 2.560045698E-07
4 0.3604217030 1.847411113E-13
◊ Comments:
* Convergence is faster than secant method.
* Convergence is compatible with Newton's method,
BUT no need to evaluate derivative f (x)
* We need THREE points to start; but it is ok in general
(in the sense that once you know two of them, you can always get
the third one by using the average of them).
* If b2 - 4ac<0  you FIND THE COMPLEX ROOT if complex
arithmetic is used.
This is an ADVANTAGE in comparison with the secant method.

Notes organized by Prof. Renwei Mei, University of Florida 30

◊ Going through an example of a FORTRAN program for root finding
to see how an efficient program is written.

C PROGRAM PNEWTN
C
C --------------------------------------------------------------------
C
C DRIVER FOR NEWTON'S METHOD SUBROUTINE, NEWTN
C
C --------------------------------------------------------------------
C
IMPLICIT NONE
REAL*8 FCN, FDER, X, XTOL, FTOL
INTEGER I, NLIM
EXTERNAL FCN, FDER
C
C ---------------------------------------------------------------------
C
C INITIALIZE VARIABLES FOR SUBROUTINE NEWTN
C
DATA X, XTOL, FTOL, I, NLIM / 0.5, 1.0E-14, 1.0E-17, 0, 50/
write(6,*) 'Input x0='
read(5,*) x
C
C --------------------------------------------------------------------
C
CALL NEWTN(FCN,FDER,X,XTOL,FTOL,NLIM,I)
C
C THE ROOT HAS BEEN FOUND AT X IF I EQUALS 1 OR 2
C
STOP
END
C
C -------------------------------------------------------------------
C
DOUBLE PRECISION FUNCTION FCN(X)
REAL*8 X, SIN, EXP
FCN = 3.0*X + SIN(X) - EXP(X)
C FCN= X-EXP(-X*X)
RETURN
END
C
C -------------------------------------------------------------------
C
DOUBLE PRECISION FUNCTION FDER(X)

Notes organized by Prof. Renwei Mei, University of Florida 31

 REAL*8 X, COS, EXP
FDER = 3.0 + COS(X) - EXP(X)
C FDER = 1.0 + 2*X*EXP(-X*X)
RETURN
END
C
C -------------------------------------------------------------------
C
SUBROUTINE NEWTN(FCN,FDER,X,XTOL,FTOL,NLIM,I)
C
C --------------------------------------------------------------------
C
C SUBROUTINE NEWTN :
C THIS SUBROUTINE FINDS THE ROOT OF A FUNCTION,
C F(X) = 0, BY NEWTON'S METHOD.
C
C ------------------------------------------------------------------
C
C PARAMETERS ARE :
C
C FCN -FUNCTION THAT COMPUTES VALUES FOR F(X). MUST BE DECLARED
C EXTERNAL IN CALLING PROGRAM. IT HAS ONE ARGUMENT, X.
C FDER -FUNCTION THAT COMPUTES THE DERIVATIVE OF F, ALSO
C DECLARED EXTERNAL.
C X -INITIAL VALUE OF X, SHOULD BE NEAR ROOT. ALSO RETURNS
C VALUE OF THE ROOT TO THE CALLER.
C XTOL,FTOL -TOLERANCE VALUES FOR X, F(X) TO TERMINATE ITERATIONS.
C NLIM -LIMIT TO NUMBER OF ITERATIONS.
C I -A SIGNAL FOR HOW ROUTINE TERMINATED.
C I = 1 MEETS TOLERANCE FOR X VALUES.
C I = 2 MEETS TOLERANCE FOR F(X).
C I = -1 NLIM EXCEEDED.
C
C WHEN THE SUBROUTINE IS CALLED, THE VALUE OF I INDICATES WHETHER
C TO PRINT EACH VALUE OR NOT. I=0 MEANS PRINT THEM, .NE. 0 MEANS
C DON'T.
C
C --------------------------------------------------------------------
C
IMPLICIT NONE
REAL*8 FCN, FDER, X, XTOL, FTOL
INTEGER NLIM, I, J
REAL*8 FX, DELX
C
C -------------------------------------------------------------------
C

Notes organized by Prof. Renwei Mei, University of Florida 32

C COMPUTE SEQUENCE OF POINTS CONVERGING TO THE ROOT
C
FX = FCN(X)
DO 20 J=1,NLIM
DELX = FX / FDER(X)
X = X - DELX
FX = FCN(X)
IF ( I .EQ. 0 ) THEN
write(8, 199) J, X, FX
END IF
C -----------------------------------------------------------------
C
C CHECK ON STOPPING CRITERIA
C
IF ( ABS(DELX) .LE. XTOL ) THEN
I=1
write(8, 202) J, X, FX
RETURN
END IF
IF ( ABS(FX) .LE. FTOL ) THEN
I=2
write(8, 203) J,X,FX
RETURN
END IF
20 CONTINUE
C
C -----------------------------------------------------------------
C
C WHEN LOOP IS NORMALLY COMPLETED, NLIM IS EXCEEDED
C
I = -1
write(8, 200) NLIM, X, FX
RETURN
C
C -------------------------------------------------------------------
C
199 FORMAT(' AT ITER',I3,3X,' X =',E20.12,X,' F(X) =',E20.11)
200 FORMAT(/' TOLERANCE NOT MET AFTER ',I4,' ITERATIONS X = ',
+ E12.5,' F(X) = ',E19.10)
202 FORMAT(/' TOL. MET IN',I3,' ITER.S: X =',E20.11,
+ X,' F(X) =',E19.10)
203 FORMAT(/' F TOL MET IN',I3,' ITER.S X =',E20.12,
+ X,' F(X) =',E19.10)
C
END

Notes organized by Prof. Renwei Mei, University of Florida 33

2.7 Repeated Roots
y

x
A double root A trip le root
2 3
f(x) ~ (x-r) f(x) ~ (x-r)
 For double root, bisection method & false position method
DO NOT WORK because we cannot bracket the root as
both require that f0f1<0.
 Newton's method or Muller's method works but converges slowly.
 Example: solving f(x) = (x-1)2 = 0 starting from x0 = 0.5
xn f(xn) en=r-x en/en-1
0.5 0.25 0.5
0.75 0.0625 0.25 0.5
0.875 0.015625 0.125 0.5
0.9375 0.00390625 0.0625 0.5
0.96875 0.00097656 0.03125 0.5
0.984375 0.00024414 0.01563 0.5
0.9921875 6.1035E-05 0.00781 0.5
0.99609375 1.5259E-05 0.00391 0.5
0.99804688 3.8147E-06 0.00195 0.5
0.99902344 9.5367E-07 0.00098 0.5
0.99951172 2.3842E-07 0.00049 0.5
0.99975586 5.9605E-08 0.00024 0.5
0.99987793 1.4901E-08 0.00012 0.5
0.99993896 3.7253E-09 6.1E-05 0.5

 We observe that for the double root (p=2), the error, en = r-xn,
decreases linearly with n,
en+1 / en =1/2 (c.f. en+1 / en2 = constant for single root)

Notes organized by Prof. Renwei Mei, University of Florida 34

 Reason for linear convergence with repeated roots
f ( xn )
Recall xn+1 = xn –
f ( xn )

1 f (r )
 en+1 / en2 = − + ... if f (r )  0
2 f (r )

* Q: what happens if f (r ) = 0 for the present case?

1
0 − en f (r ) + en2 f (r ) + ...
* A: -en+1 = -en – 2
f (r ) − en f (r ) + ...

1 2
en f (r ) + ...
e
= -en – 2 =− n
− en f (r ) + ... 2

 en+1 / en = 1/2
 For a triple root (p=3) or higher,
en+1 / en = (p-1)/p
(for p=1, en/ en-1 = 0 & it simply means the convergence is much
faster than "linear"; )

Notes organized by Prof. Renwei Mei, University of Florida 35

 Example: Newton's method for a double root problem:
f(x)=(x-1)*[exp(x-1)-1]
with x0 = 2; exact solution r=1
Iter. # x f(x)
1 1.612699837 0.5179807167E+00
2 1.350575458 0.1472011647E+00
3 1.190166373 0.3983049892E-01
4 1.099528735 0.1041570320E-01
5 1.050992095 0.2667629770E-02
6 1.025819673 0.6753364934E-03
7 1.012992988 0.1699192292E-03
8 1.006517573 0.4261749291E-04
9 1.003264094 0.1067171452E-04
10 1.001633378 0.2670104537E-05
11 1.000817023 0.6677986408E-06
12 1.000408595 0.1669837560E-06
13 1.000204318 0.4175020299E-07
14 1.000102164 0.1043808388E-07
15 1.000051083 0.2609587618E-08
16 1.000025542 0.6524052360E-09
17 1.000012771 0.1631023505E-09
18 1.000006386 0.4077571781E-10
19 1.000003193 0.1019394573E-10
20 1.000001596 0.2548488466E-11
Note: en+1 / en = 0.5 with p=2 for the double root problem.

Error en is reduced by a factor of 2 at every iteration.

Notes organized by Prof. Renwei Mei, University of Florida 36

 Newton's method for a triple root problem:
f(x)=(x-1) 2*[exp(x-1)-1]
with x0=2; exact solution r =1
Iter. # x f(x)
1 1.720824539 0.5487513833E+00
2 1.509025438 0.1719612296E+00
3 1.353645851 0.5305907820E-01
4 1.242688504 0.1617752269E-01
5 1.165059174 0.4889382819E-02
6 1.111551903 0.1468518051E-02
7 1.075059022 0.4391465770E-03
8 1.050352291 0.1309294817E-03
9 1.033709037 0.3895645499E-04
10 1.022535817 0.1157504567E-04
11 1.015052092 0.3436079803E-05
12 1.010047315 0.1019374214E-05
13 1.006703818 0.3022896208E-06
14 1.004471709 0.8961732255E-07
15 1.002982250 0.2656317439E-07
16 1.001988661 0.7872526029E-08
17 1.001325994 0.2332986846E-08
18 1.000884093 0.6913317423E-09
19 1.000589439 0.2048541258E-09
20 1.000392979 0.6070050016E-10
21 1.000261994 0.1798592236E-10
22 1.000174667 0.5329278532E-11
23 1.000116446 0.1579068475E-11
24 1.000077632 0.4678766808E-12
25 1.000051755 0.1386310245E-12
26 1.000034503 0.4107603626E-13
27 1.000023002 0.1217071240E-13
28 1.000015335 0.3606143921E-14

Note: the error | en | = |r- xn| is reduced by a factor of 2/3 every iteration;

rate of convergence en+1 / en =0.6667 = (p-1)/p;

Notes organized by Prof. Renwei Mei, University of Florida 37

 with p=3 = multiplicity of the root.
Iter. # en en / en-1
24 0.000077632
25 0.000051755 0.6667
26 0.000034503 0.6667
27 0.000023002 0.6667
28 0.000015335 0.6667
 Remedies for slow convergence of repeated roots:
If we know there are m roots at x=r, we can use
f ( xn )
xn+1 = xn - m
f ( xn )
 en+1 / en2 = constant just like in the single root case.
Example: Modified Newton's method for the
double root problem:
f(x)=(x-1)*[exp(x-1)-1],
xn+1 = xn - m f(xn)/f'(xn), m=2; x0=2.0

Iter. # x f(x)
1 0.1225399674E+01 0.5698629752E-01
2 0.1012450098E+01 0.1559738635E-03
3 0.1000038711E+01 0.1498563087E-08
4 0.1000000000E+01 0.1403493791E-18

Practically, you need to know the multiplicity m in order to use

the Newton's method.
=> need to look for alternative more robust method.

Notes organized by Prof. Renwei Mei, University of Florida 38

 Modified Newton’s Method with u = f / f

* Consider f ( x) = ( x − r ) m & f ( x) = m( x − r ) m −1

with unknown m.
f(x) x−r
* Let u(x) =  = which is a linear function with only
f (x) m

ONE ROOT now; but we still do not know m.

u ( xn )
* Apply Newton’s method to u(x), we get xn +1 = xn −
u ( xn )

f (xn ) f (xn )
Or xn +1 = xn −
 f (xn )2 − f (xn ) f (xn )
* Example: Solve f(x)=(x-1)*[exp(x-1)-1]= 0; x0 =2
Solution: (this is double root case; we do not need to know m)
n x f f' f" |r-xn|
0 2 1.718282 4.436564 8.15485 1
1 0.65569209 0.100293 -0.535304 1.17341 0.344308
2 0.97362152 0.000687 -0.051725 1.92224 0.026378
3 0.99982757 2.97E-08 -0.000345 1.99948 0.000172
4 0.99999999 5.52E-17 -1.49E-08 2 7.43E-09

* Example: Solve f(x)=(x-1)2*[exp(x-1)-1] = 0; x0 =2

Solution: (this is triple root case; we do not need to know m)
n x f f' f" |r-xn|
0 2 1.71828 6.15485 17.028 1
1 0.77357856 -0.01039 0.13263 -1.0865 0.2264214
2 0.9920881 -4.9E-07 0.00019 -0.047097 0.0079119
3 0.99998959 -1.1E-15 3.2E-10 -6.24E-05 1.041E-05
4 1 -1.9E-32 1.4E-21 -1.59E-10 2.651E-11
* Clearly, the convergence in both cases has greatly improved.

Notes organized by Prof. Renwei Mei, University of Florida 39

2.8 Brent's Method
2.8.1 Basic features & ideas
Basic features:
* Brent's method is a complicated but popular root-finding
algorithm.
* It combines bisection method, secant method, & inverse
quadratic interpolation.
* It is as reliable as bisection.
* It can be as quick as secant.
Basic ideas:
• use the secant method or inverse quadratic interpolation if
possible, because they converge faster
• fall back to the more robust bisection method if necessary.
• Brent's method (1971) builds on an earlier algorithm of
Dekker (1969).

2.8.2Dekker's method

• combine the bisection method with the secant method

• start with two points, say a0 and b0, such that f(a0) and f(b0)
have opposite signs.
• bk is the current iterate, i.e., the current guess for the root of f.

Notes organized by Prof. Renwei Mei, University of Florida 40

 • ak is the "contrapoint," i.e., a point such that f(ak) and f(bk) have
opposite signs, so the interval [ak, bk] contains the solution.
Furthermore, |f(bk)| should be less than or equal to |f(ak)|, so that
bk is a better guess for the unknown solution than ak.
• bk−1 is the previous iterate (for the first iteration, we set b−1 = a0).
• Two provisional values for the next iterate are computed. The
first one is given by the secant method:

and the second one is given by the bisection method

• If the result of the secant method, s, lies between bk and m, then

it becomes the next iterate (bk+1 = s), otherwise the midpoint is
used (bk+1 = m).
• Then, the value of the new contrapoint is chosen such that f(ak+1)
and f(bk+1) have opposite signs.
• If f(ak) and f(bk+1) have opposite signs, then the contrapoint
remains the same: ak+1 = ak.
• Otherwise, f(bk+1) and f(bk) have opposite signs, so the new
contrapoint becomes ak+1 = bk.

Notes organized by Prof. Renwei Mei, University of Florida 41

 • Finally, if |f(ak+1)| < |f(bk+1)|, then ak+1 is probably a better guess
for the solution than bk+1, and hence the values of ak+1 and bk+1
are exchanged.

2.8.3Brent's method

• Dekker's method performs well if f(x) is reasonably well-

behaved.
• In some cases, every iteration employs the secant method, but
the iterates bk converge very slowly (in particular, |bk − bk−1|
may be arbitrarily small). Dekker's method requires far more
iterations than the bisection method in this case.

Basic ideas of Brent:

• Brent inserted an additional test which must be satisfied before

the result of the secant method is accepted as the next iterate.
• If the previous step used the bisection method, the inequality

(comparing the progress)

must hold; otherwise the midpoint m is used for the next iterate.

• If the previous step used interpolation, then the inequality

is used instead.

Notes organized by Prof. Renwei Mei, University of Florida 42

 This modification ensures that a bisection step is done if the secant
method does not progress fast enough.

* In addition, Brent used inverse quadratic interpolation instead of

linear interpolation (as used by the secant method) if f(bk), f(ak)
and f(bk−1) have all different values.

* This slightly increases the efficiency. As a consequence, the

condition for accepting s (the value proposed by either linear
interpolation or inverse quadratic interpolation) has to be
changed: s has to lie between (3ak + bk)/4 and bk.

Example: Find the smallest positive root: f(x) = x2 |sinx| - 4=0.

K ak bk f(bk)
1 0 1.321348709 -2.31E+00
2 1.321348709 2.724506991 -9.93E-01
3 2.724506991 3.362253496 -1.53E+00
4 3.362253496 3.681126748 2.96E+00
5 3.3622534957031 3.4706734998653 -1.07E-01
6 3.4706734998653 3.4785803371277 9.85E-04
7 3.4785803371277 3.4785083135104 -2.77E-06
8 3.4785083135104 3.4785085155695 -7.11E-11
9 3.4785085155695 3.4785085155746 8.88E-16

Notes organized by Prof. Renwei Mei, University of Florida 43

 8 4
f 3.5
b
6 k
bk
3
4
2.5
2 2
1.5
0
0 1 2 3 4 1
-2
0.5
-4 0
x 0 2 4 6 8 10
-6
k

Clearly, it converges much faster than the false position method.

Notes organized by Prof. Renwei Mei, University of Florida 44