Scribd
Upload
141 views17 pages

1.3: Newton's Method: MA385 - Numerical Analysis September 2019

Newton's method is an iterative method for finding the roots, or zeros, of a nonlinear function. It uses the tangent line of the function at the current point to estimate where the next improved solution is. The method converges quadratically under certain conditions on the function and initial guess. Exercises are provided to derive Newton's method geometrically and algebraically, prove its convergence rate, and consider variations like Steffenson's method.

Uploaded by

Bashar Jawad
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
141 views17 pages

1.3: Newton's Method: MA385 - Numerical Analysis September 2019

Newton's method is an iterative method for finding the roots, or zeros, of a nonlinear function. It uses the tangent line of the function at the current point to estimate where the next improved solution is. The method converges quadratically under certain conditions on the function and initial guess. Exercises are provided to derive Newton's method geometrically and algebraically, prove its convergence rate, and consider variations like Steffenson's method.

Uploaded by

Bashar Jawad
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 17

§1.

3 Newton’s Method (31/65)

Solving nonlinear equations


§1.3: Newton’s Method
MA385 – Numerical Analysis
September 2019

Sir Isaac Newton, 1643 - 1727, England. Easily one


of the greatest scientist of all time. The method
we are studying appeared in his celebrated
Principia Mathematica in 1687, but it is believed
he had used it as early as 1669.
Motivation (32/65)

Secant method can be written as

xk+1 = xk − f (xk )φ(xk , xk−1 ),

where the function φ is chosen so that xk+1 is the root of the 
secant line joining the points xk−1 , f (xk−1 ) and xk , f (xk ) .
A closely related idea leads to Newton’s Method: set
xk+1 = xk − f (xk )λ(xk ), where we choose λ so that xk+1 is the
zero of the tangent to f at (xk , f (xk )).
Motivation (33/65)

(x0, f(x0))
2

0
x1 x0
−1

−2 f(x)= x2 −2
tangent line
x−axis
−3
0.8 1 1.2 1.4 1.6 1.8 2 2.2
√
Figure: Estimating 2 by solving x 2 − 2 = 0 using Newton’s Method
Motivation (33/65)

1
2
f(x)= x −2
0.8 secant line
x−axis
0.6

0.4 (x1, f(x1))


0.2

−0.2 x2 x1

−0.4

−0.6

−0.8

−1
1.2 1.3 1.4 1.5 1.6 1.7
√
Figure: Estimating 2 by solving x 2 − 2 = 0 using Newton’s Method
Motivation (34/65)

The formula for Newton’s method may be deduced writing down


the equation for the line at xk , f (xk ) with slope f 0 (xk ), and


setting xk+1 to be its zero; see Exercise 1.8-(i)).

Newton’s Method
1. Choose any x0 in [a, b],
2. For k = 0, 1, . . . , set

f (xk )
xk+1 = xk − . (5)
f 0 (xk )
Motivation (35/65)
Example 1.8
Use bisection, secant, and Newton’s Method to solve x 2 − 2 = 0 in
[0, 2].

For this case, Newton’s method is

f (xk ) xk2 − 2
xk+1 = xk − = x k − ,
f 0 (xk ) 2xk

which simplifies as
1 1
xk+1 = xk + .
2 xk

Taking x0 = 2, we get x1 = 3/2.


Then x2 = 2x1 + 1/x1 = 17/12 = 1.46667.
Then x3 = 1.4142, etc.
Motivation (36/65)

Iter Bisection Secant Newton


k |xk − τ | |xk − τ | |xk − τ |
0 1.41 1.41 5.86e-01
1 5.86e-01 5.86e-01 8.58e-02
2 4.14e-01 4.14e-01 2.45e-03
3 8.58e-02 8.09e-02 2.12e-06
4 1.64e-01 1.44e-02 1.59e-12
5 3.92e-02 4.20e-04 2.34e-16
6 2.33e-02 2.12e-06 —
7 7.96e-03 3.16e-10 —
8 7.66e-03 4.44e-16 —
9 1.51e-04 — —
10 3.76e-03 — —
11 1.80e-03 — —
.. .. .. ..
. . . .
22 5.72e-07 — —
Motivation (37/65)

Deriving Newton’s method geometrically certainly has an intuitive


appeal. However, to analyse the method, we need a more abstract
derivation based on a Truncated Taylor Series.

(x − xk )2 00
f (x ) = f (xk ) + (x − xk )f 0 (xk ) + f (xk ) + . . .
2!
(x − xk )n (n) (x − xk )n+1 (n+1)
+ f (xk ) + f (ηk )
n! (n + 1)!

where ηk ∈ (x , xk ). Now truncate at the second term (i.e., take


n = 1):
Newton Error Formula (38/65)

We now want to show that Newton’s converges quadratically,


that is, with at least order q = 2. To do this, we need to

1. Write down a recursive formula for the error.


2. Show that it converges.
|τ − xk+1 |
3. Then find the limit of .
|τ − xk |2
Step 2 is usually the crucial part.
There are two parts to the proof. The first involves deriving the
so-called “Newton Error formula”.
We’ll assume that the functions f , f 0 and f 00 are defined and
continuous on the an interval Iδ = [τ − δ, τ + δ] around the root τ .
The following proof is essentially the same as the above derivation
(see also Theorem 3.2 in Epperson).
Newton Error Formula (39/65)

Theorem 1.9 (Newton Error Formula)


If f (τ ) = 0 and
f (xk )
xk+1 = xk − ,
f 0 (xk )
then there is a point ηk between τ and xk such that

(τ − xk )2 f 00 (ηk )
τ − xk+1 = − , (6)
2 f 0 (xk )
Applying the Newton Error Formula (40/65)

The Newton Error Formula is important in theory (for proving that


Newton’s Method converges) and practice (to estimate the error
when it is applied to specific problems).
In practical applications, we can use the (6) as follows. So suppose
we are applying Newton’s Method to solving f (x ) = 0 on [a, b].
Denote the error at Step k by εk = |τ − xk |. Then we can deduce
that
max |f 00 (x )|
2 a≤x ≤b
εk+1 ≤ εk . (7)
2|f 0 (xk )|
Then, using that ε0 ≤ |b − a|, (7) can be used repeatedly to bound
ε1 , ε2 , etc.
Convergence of Newton’s Method (41/65)

We’ll now complete our analysis of this section by proving the


convergence of Newton’s method.
Theorem 1.10
Let us suppose that f is a function such that
f is continuous and real-valued, with continuous f 00 , defined
on some closed interval Iδ = [τ − δ, τ + δ],
f (τ ) = 0 and f 00 (τ ) 6= 0,
there is some positive constant A such that

|f 00 (x )|
≤A for all x , y ∈ Iδ .
|f 0 (y )|

Let h = min{δ, 1/A}. If |τ − x0 | ≤ h then Newton’s Method


converges quadratically.
Convergence of Newton’s Method (42/65)
Exercises (43/65)

Exercise 1.8 (? Homework problem)


Write down the equation of the line that is tangential to the function f
at the point xk . Give an expression for its zero. Hence show how to
derive Newton’s method.

Exercise 1.9
(i) Is it possible to construct a problem for which the bisection method
will work, but Newton’s method will fail? If so, give an example.
(ii) Is it possible to construct a problem for which Newton’s method will
work, but bisection will fail? If so, give an example.
Exercises (44/65)
Exercise 1.10 (? Homework problem)

(i) Let q be your student ID number. Find k and m where k − 2 is the


remainder on dividing q by 4, and m − 2 is the remainder on
dividing q by 6.
(ii) Show how Newton’s
√ method can be applied to estimate the postive
real number k m. That is, state the nonlinear equation you would
solve, and give the formula for Newton’s method, simplified as
much as possible.
(iii) Do three iterations by hand of Newton’s Method for this problem.

Exercise 1.11 (? Homework problem)


Suppose we want apply to Newton’s method to solving f (x ) = 0 where f
is such that |f 00 (x )| ≤ 10 and |f 0 (x )| ≥ 2 for all x . How close must x0 be
to τ for the method to converge?
Exercises (45/65)
Exercise 1.12
Here is (yet) another scheme called Steffenson’s Method: Choose
x0 ∈ [a, b] and set
2
f (xk )
xk+1 = xk −  for k = 0, 1, 2, . . . .
f xk + f (xk ) − f (xk )

It is remarkable because its convergence is quadratic, like Newton’s, but


does not require derivatives of f .
Show how the method can be derived from Newton’s Method, using the
formal definition of the derivative.
Exercises (46/65)
Exercise 1.13
(This is Exercise 1.6 from Süli and Mayers) The proof of the convergence
of Newton’s method given in Theorem 1.10 uses that f 0 (τ ) 6= 0. Suppose
that it is the case that f 0 (τ ) = 0.
(i) Starting from the Newton Error formula, show that

(τ − xk ) f 00 (ηk )
τ − xk+1 = ,
2 f 00 (µk )

for some µk between τ and xk . (Hint: try using the MVT).


(ii) What does the above error formula tell us about the convergence of
Newton’s method in this case?

You might also like