CSC475: Numerical Computing

Department of Computer Science

COMSATS University, Islamabad
Recap
Numerical Methods
for Solving
Nonlinear Equations
Bracketing Methods

Department of Computer Science 2

 Root of an equation

In graph, the root (or zero) of Various numerical methods for

a function is the x-intercept root-finding
The Bisection Method
The False Position Method
root Fixed point iterations
The Newton-Raphson Method
The Secant Method

Department of Computer Science

Department of Computer Science 4
 Lecture No 4

Numerical Methods
for Solving
Nonlinear Equations
Open Methods

Department of Computer Science 5

 Newton Raphson Method

 Newton's method (also called the Newton-

Raphson method) is a scheme for finding a
numerical solution of an equation of the form
f(x) = 0
 where f(x) is continuous and
differentiable
 the equation is known to have a solution near a
given point.

Department of Computer Science 6

 Newton Raphson Method

 The method is illustrated in Fig.

Department of Computer Science 7

 Newton Raphson Method

 The solution process starts by choosing point x1

as the first estimate of the solution.
 The second estimate x2 is obtained by taking
the tangent line to f(x) at the point (x1, f(x1))
and finding the intersection point of the tangent
line with the x-axis.
 The next estimate x3 is the intersection of the
tangent line to f(x) at the point (x2, f(x2)) with
the x-axis, and so on.

Department of Computer Science 8

 Newton Raphson Method

 Mathematically, for the first iteration, the slope, f

'(x1), of the tangent at point (x1, f(x1)) is given by:

Department of Computer Science 9

 Algorithm for Newton's method

Department of Computer Science 10

 Algorithm

Department of Computer Science 11

 Example: NR

Department of Computer Science 12

 Example: True Error

Department of Computer Science 13

Newton’s Method

THE NEWTON-RAPHSON METHOD is a method for finding successively better

approximations to the roots (or zeroes) of a function.

Algorithm Example

To approximate the roots of

1 0.000000000000000
2 0.500000000000000
3 0.566311003197218
4 0.567143165034862 The true value of the root: 0.56714329.Thus,
5 0.567143290409781
Department of Computer Science the approach rapidly converges on the true root.
 Comparison with previous two methods :
 In previous methods, we were given an interval.
 Here we are required an initial guess value of root.
 The previous two methods are guaranteed to
converge.
 Newton Rahhson may not converge in some cases.
 For many problems, Newton Raphson method converges
faster than the above two methods.
 The previous two methods do not identify repeated
roots.
 It can identify repeated roots, since it does not look
for changes in the sign of f(x) explicitly.

Department of Computer Science 15

 Comparison with previous two methods :
 Newton Raphson method requires derivative.
 Some functions may be difficult to differentiate.

Department of Computer Science 16

 Notes on Newton's method

 Illustrations of two cases where Newton's method

diverges are shown in following fig.

 When it does not converge, it is usually because the

starting point is not close enough to the solution.

Department of Computer Science 17

 Notes on Newton's method

Department of Computer Science 18

 Notes on Newton's method

Department of Computer Science 19

Convergence of Newton-Raphson Method

 The method, when successful, works well and

converges fast.
 Convergence problems typically occur when the
value of f '(x) is close to zero in the vicinity of the
solution (where f(x) = 0).
 It is possible to show that Newton's method
converges:
 if the function f(x) and its first and second derivatives f '(x)
and f "(x) are all continuous,
 if f '(x) is not zero at the solution
 and if the starting value x1 is near the actual solution.

Department of Computer Science 20

 Advantages and Drawbacks
of Newton Raphson Method

Department of Computer Science 21

 Advantages

 Converges fast (quadratic convergence), if it

converges.
 Requires only one guess

Department of Computer Science 22

 Secant Method

Department of Computer Science

 Secant Method -Derivation

f(x ) Newton’s Method

f(xi )
xi 1 = xi - (1)
f (xi )
f ( x i)
x f x 
i, i

Approximate the derivative

f ( xi )  f ( xi 1 )
f ( xi )  (2)
f ( x i-1 )
xi  xi 1

X
Substituting Equation (2)
x i+ 2 x i+ 1 xi
into Equation (1) gives the
Secant method
Figure 1 Geometrical illustration of f ( xi )( xi  xi 1 )
the Newton-Raphson method. xi 1  xi 
f ( xi )  f ( xi 1 )
24
 Algorithm for Secant Method

Department of Computer Science 26

 Step 1

Calculate the next estimate of the root from two initial guesses
f ( xi )( xi  xi 1 )
xi 1  xi 
f ( xi )  f ( xi 1 )
Find the absolute relative approximate error

xi 1- xi
a =  100
xi 1

27
 Step 2

 Find if the absolute relative approximate error is

greater than the prespecified relative error tolerance.

 If so, go back to step 1, else stop the algorithm.

 Also check if the number of iterations has exceeded

the maximum number of iterations.

28
29
 Advantages
 Converges fast, if it converges
 Requires two guesses that do not need to
bracket the root

Department of Computer Science 30

 Secant Method

 The method uses two points in the

neighborhood of the solution to determine a
new estimate for the solution.
 the two points can be on one side of the solution
 the solution can be between the two points

Department of Computer Science 31

 Secant Method

 The two points (marked as x1 and x2 in the

figure) are used to define a straight line (secant
line), and the point where the line intersects the
x-axis (marked as x3 in the figure) is the new
estimate for the solution.

Department of Computer Science 32

 Secant Method

 The slope of the secant line is given by:

 which can be solved for x3 :

Department of Computer Science 33

 Secant Method

 Once point x3 is determined, it is used together

with point x2 to calculate the next estimate of
the solution, x4.
 The previous equation can be generalized to an
iteration formula in which a new estimate of the
solution xi+ 1 is determined from the previous
two solutions xi and xi-1.

Department of Computer Science 34

 Relationship to Newton's method

 This equation is almost identical to Eq. of

Newton's method.(How?)

Department of Computer Science 35

 Relationship to Newton's method

 In Eq. the denominator of the

second term on the right-hand side of the
equation is an approximation of the value of the
derivative of f(x) at xi.
 In Newtons formula , the denominator is
actually the derivative f'(x).

 In the secant method (unlike Newton’s

method), it is not necessary to know the
analytical form of f'(x).

Department of Computer Science 36

 Relationship to Newton's method

 Examination of the secant method shows that

when the two points that define the secant line
are close to each other, the method is actually
an approximated form of Newton's method.
 This can be seen by rewriting Eq.
in the form:

 This equation is almost identical to Eq. of

Newton's method

Department of Computer Science 37

 Example

Department of Computer Science

Department of Computer Science 39
 Your Turn!

• Find a root of the equation x3 – 8x – 5 = 0

using the secant method.

• Find a root of an equation f(x)=2cos(x)-x

between 0 and 1, using Secant method.

Department of Computer Science

Fixed- Point Iteration

Example:

solution

Department of Computer Science

Fixed- Point Iteration
simple fixed-point iteration

Step
1

Step
2
Note
Convert the problem from
root-finding to finding fixed-
Department of Computer Science point
Fixed- Point Iteration
Step Example1
1

Step
1 0.0000000000000
2 2 1.0000000000000
3 0.3678794411714
4 0.6922006275553
Starting with an initial guess of x = 0 0
5 0.5004735005636
6 0.6062435350856
7 0.5453957859750
8 0.5796123355034
9 0.5601154613611
10 0.5711431150802
11 0.5648793473910
12 0.5684287250291
Thus, each iteration brings the estimate closer
Department of Computer Science to the true value of the root: 0.56714329
 When does it
converge?

Department of Computer Science 44

 Example
Find a root of an equation f(x)=2x3-2x-5 using Fixed Point
Iteration method.
Solution:
Let f(x)=2x3-2x-5
Here 2x3-2x-5=0
∴2x3=2x+5
∴x3=(2x+5)/2
∴x=3√(2x+5)/2
∴ϕ(x)=3√(2x+5)/2

Here,
x 0 1 2 Here, f(1)=-5<0 and f(2)=7>0
f(x) -5 -5 7 ∴ Root lies between 1 and 2
x0=(1+2)/2=1.5

Department of Computer Science 45

Department of Computer Science 46
 Conclusion

Department of Computer Science

 Summary

Department of Computer Science 48

 Comparison of the methods

Department of Computer Science 49