10.

2 Newton's Method 611

and

In this exercise, we would like to consider the problem of determining equilibrium populations
of the two species. The mathematical criteria that must be satisfied in order for the populations
to be at equilibrium is that, simultaneously,

dX2(t)
dt = O.
This occurs when the first species is extinct and the second species has a population of 20,000
or when the second species is extinct and the first species has a population of 13,333. Can an
equilibrium occur in any other situation?

10.2 Newton's Method

The problem in Example 2 of the previous section is transformed into a convergent fixed-
point problem by algebraically solving the three equations for the three variables Xl, X2,
and X3. It is, however, rather unusual for this technique to be successful. In this section,
we consider an algorithmic procedure to perfOlIll the transformation in a more general
situation.
To construct the algorithm that led to an appropriate fixed-point method in the one-
dimensional case, we found a function ¢ with the property that

g(X) = X - ¢(x)f(x)

gives quadratic convergence to the fixed point p of the function g (see Section 2.4). From
this condition Newton's method evolved by choosing ¢(x) = l/!'(x), assuming that
!,(x) t= O.
Using a similar approach in the n-dimensional case involves a matrix

all (x) adx) • • • aln(x)

a21 (x) a22(x) • • • a2n (x)
A(x) = • • •
, (10.5)
• • •
• • •

ani (x) an2(x) • • • ann (x)

where each of the entries aij (x) is a function from ~n into R This requires that A (x) be
found so that

G(x) = x - A(x)-'F(x)

gives quadratic convergence to the solution ofF(x) = 0, assuming that A (x) is nonsingular
at the fixed point p of G.
The following theorem parallels Theorem 2.8 in Section 2.4. Its proof requires being
able to express G in tenlls of its Taylor series in n variables about p.
612 C HAP T E R 10 • Numet1cal Solutions of Nonlinear Systems of Equations

Theorem 70.7 Let p be a solution of G(x) = x, Suppose a number 8 > 0 exists with

(i) agdaxj is continuous on No = (x I Ilx - pll < 8}, for each i = 1,2, ... ,n
andj=I,2, ... ,n;
(ii) a2gj(x)/(aXjaXk) is continuous, and la2gj(x)/(aXjaXk)1 < M for some con-
stant M, whenever x E No, for each i = 1,2, ... , n, j = 1,2, ... , n, and
k=I,2, ... ,n;
(iii) agj(p)/aXk = 0, for each i = 1,2, ... , nand k = 1,2, ... , n.
~

Then a number 8 < 8 exists such that the sequence generated by X(k) = G(X(k-I» con-
verges quadratically to p for any choice of x(O) , provided that x(O) - p < 8. Moreover,

for each k >

- 1. •
,

To use Theorem to.7, suppose that A(x) is an n x n matrix of functions from lRn
into lR in the form of Eq. (10.5), where the specific entries will be chosen later. Assume,
moreover, that A (x) is nonsingular near a solution p of F(x) = 0, and let bi} (x) denote the
entry of A(X)-I in the ith row and jth column.
Since G(x) = x - A(X)-IF(x), we have gj(x) = Xj - I:j=1 bij (x)/j (x) and

ago
I-I:
j=1
if i = k,
--=-1 (x) =
aXk ~ at- ab··
- ~ bij(x) a J (x) + a IJ (x)/j(x) , if i i= k.
j=1 Xk Xk

Theorem 10.7 implies that we need agj(p)/aXk = 0, for each i 1, 2, ... , n and k =
1,2, ... , n. This means that for i = k,

so

(10.6)

When k i= i,

so

(10.7)
 10.2 Newton's Method 613

Defining the matrix J (x) by

a/J (x) afl (x) • • • a/J (x)

aXI aX2 aXn
ah (x) ah (x) • • • ah (x)
J(x) = aXI aX2 aXn , (10.8)
• • •
• • •
• • •

afn (x) afn (x) • • • Jfn (x)

aXI aX2 JXn
we see that conditions (10.6) and (10.7) require

A (p) -I J (p) = I, the identity matrix,

so

A(p) = J(p).

An appropriate choice for A (x) is, consequently, A(x) = J (x) since this satisfies condition
(iii) in Theorem 10.7.
The function G is defined by

G(x) = x - J(x)-IF(x),

and the functional iteration procedure evolves from selecting xeD) and generating, for k > I,

(10.9)

This is called Newton's method for nonlinear systems, and it is generally expected
to give quadratic convergence, provided that a sufficiently accurate starting value is known
and J(p)-I exists.
The matrix J (x) is called the Jacobian matrix and has a number of applications in
analysis. It might, in particular, be familiar to the reader due to its application in the mul-
tiple integration of a function of several variables over a region that requires a change of
variables to be performed.
The weakness in Newton's method arises from the need to compute and invert the ma-
trix J (x) at each step. In practice, explicit computation of J (x) -I is avoided by perfonning
the operation in a two-step manner. First, a vector y is found that satisfies J(X(k-I»y =
-F(X(k-I». Then the new approximation, X(k), is obtained by adding y to X(k-I). Algorithm
10.1 uses this two-step procedure.

Newton's Method for Systems

To approximate the solution of the nonlinear system F(x) = 0 given an initial approxima-
tion x:

I N PUT number n of equations and unknowns; initial approximation x = (XI, ... , Xn f ,

tolerance TOL; maximum number of iterations N.
614 C HAP T E R 10 • Numerical Solutions oj Nonlinear Systems oj Equations

OUTPUT approximate solution x = (XI, ... , Xn)1 or a message that the number of
iterations was exceeded.
Step 7 Set k = I.
Step 2 While (k < N) do Steps 3-7.
Step3 CakulateF(x) and lex), where l(x)i,j = (af;(x)/axj) for I <i,j< n.
Step 4 Solve the n x n linear system 1 (x)y = -F(x).
Step 5 Set x = x + y.
Step 6 If iiYii < TOL then OUTPUT (x);
(The procedure was successful.)
STOP.
Step 7 Set k = k + I.
Step 8 OUTPUT (,Maximum number of iterations exceeded');
(The procedure was unsuccessful.)
STOP. •
EXAMPLE 1 The nonlinear system
I
3XI - COS(X2X3) - 2 = 0,

xt - 81(X2 + 0.1)2 + sinx3 + 1.06 = 0,

e-
X1X2
+ 20X3 +
lOJT - 3
3
= °
was shown in Example 2 of Section 10.1 to have an approximate solution at (0.5,0,
-0.52359877)1. Newton's method will be used to obtain this approximation when the ini-
tial approximation is x(O) = (0.1,0.1, -0.1)' and

F(XI, X2, X3) = (fI (XI, X2, X3), h(xI, X2, X3), h(xI, X2, X3»I,

where
I
/1 (Xl, X2, X3) = 3XI - COS(X2X3) - 2'

h(xI, X2, X3) = x~ - 81(X2 + 0.1)2 + sinx3 + 1.06,

and

x x
12
lOJT - 3
h(XI,X2,x3)=e- +20X3+ 3 .

The Jacobian matrix lex) for this system is

•
X2 SlllX2X3

-162(X2 + 0.1) cosx3

20
 10.2 Newton's Method 615

and

(k) (k-I) (k-i)

xl XI YI
(k) (k-J) (k-I)
X2 - X2 + Y2
,
(k) (k-J) (k-l)
X3 X3 Y3

where

(k-J)
YI
(k-I)
Y2
__ (1
-
(k-I)
xI
(k-l)
,X2
(k-l)))-I
,X3
F (k-l)
xl
(k-l)
,X2
(k-l»)
,X3 .
(k-I)
Y3

Thus, at the kth step, the linear system 1 (X(k-l)) y(k-I) - -F (x(k-I») must be solved,
where

(k-I). (k-IJ (k-l) (k-I). (k-I) (k-I)

3 x3 smx 2 x3 x2 smx 2 x3
2XI
(k-I)
-162 (XY-I) + 0.1) (k-I)
COSX 3 •

( k-I) _ (k-I) (k-I) (k-I) _ (k-I) (k-I)

-x2 e XI X2 -x I e Xl x2 20
(k-l)
Y1
(k-I) = (k-I)
Y Y2 '
(k-l)
Y3

and

(k-I) (k-l) (k-l) I

3X I - cos X z x3 - 2

F (X(k-I)) = (xi k- ' )/ _ 81 (xik- 1) + 0.1)2 + sinx~k-I) + 1.06 .

_x(k-l)x(k-l) 20 (k-I) 1011"-3
e I 2 + x3 + 3

The results using this iterative procedure are shown in Table 10.3.
•
n.ble 103 k X(k) (k)
X2
X(k) IIx(k) - X(k-l) 1100
1 3

0 0.10000000 0.10000000 -0.10000000

1 0.50003702 0.01946686 -0.52152047 0.422
2 0.50004593 0.00158859 -0.52355711 1.79 x 10-2
3 0.50000034 0.00001244 -0.52359845 1.58 x 10- 3
4 0.50000000 0.00000000 -0.52359877 1.24 x 10-5
5 0.50000000 0.00000000 -0.52359877 0
616 C HAP T E R 10 • Numerical Solutions of Nonlinear Systems of Equations

The previous example illustrates that Newton's method can converge very rapidly once
an approximation is obtained that is near the true solution. However, it is not always easy
to determine starting values that will lead to a solution, and the method is comparatively
expensive to employ. In the next section, we consider a method for overcoming the latter
weakness. Good starting values can usually be found by the method that will be discussed
in Section 10.4.
Initial approximation to the solutions of 2 x 2 and often 3 x 3 nonlinear systems can
also be obtained using the graphing facilities of Maple. The nonlinear system
xl - xi + 2X2 = 0,

2xI +xi - 6 = 0
has two solutions, (0.625204094, 2.17935582W and (2.109511920, -1.334532188)1. To
use Maple we first define the two equations
>eql:=xl-2-x2-2+2*x2=O;
>eq2:=2*xl+x2-2-6=O;
To obtain a graph of the two equations for - 3 < XI, X2 < 3, enter the commands
>with(plots);
>implicitplot({eql,eq2},xl=-3 .. 3,x2=-3 .. 3);
From the graph shown in Figure 10.2, we are able to estimate that there are solutions near
(0.64,2.2)1 and (2.1, -1.3)1. This gives us good starting values for Newton's method.

Figure 10.2
3

x2
1

-3 -2 -1 o 1
xl
2 3

-1

-2

-3

The problem is more difficult in three dimensions. Consider the nonlinear system
2xI - 3X2 + X3 - 4 = 0,

2xI + X2 - X3 + 4 = 0,
xl + xi + xj - 4 = O.
 10.2 Newton's Method 617

Define three equations using the Maple commands

>eql:=2*xl-3*x2+x3-4=O;
>eq2:=2*xl+x2-x3+4=O;
>eq3:=xl-2+x2-2+x3-2-4=O;

The third equation describes a sphere of radius 2 and center (0,0,0), so xl, x2, and x3
are in [-2,2]. The Maple commands to obtain the graph in this case are

>with(plots) ;
>implicitplot3d({eql,eq2,eq3},xl=-2 .. 2,x2=-2 .. 2,x3=-2 .. 2);

Various three-dimensional plotting options are available in Maple for isolating a so-
lution to the nonlinear system. For example, we can rotate the graph to better view the
sections of the surfaces. Then we can zoom into regions where the intersections lie and al-
ter the display form of the axes for a more accurate view of the intersection's coordinates.
For this problem, a reasonable initial approximation is (Xl, X2, X3)' = (-0.5, -1.5, 1.5)'.

EXERCISE SET 10.2

1. Use Newton's method with x(O) - 0 to compute X(2) for each of the following nonlinear
systems.

a. b.
4n - 1
(e 2X1 - e) + 4exi - 2exi = 0.
4n

1
c. 3Xl - COS(X2 X 3) - 2 = 0, d. Xf + X2 - 37 = 0,

4x; - 625xi + 2X2 - 1 = 0, Xl - xi - 5 = 0,

IOn - 3 Xl + X2 + X3 - 3 = 0.
e- X1X2
+ 20X3 +
3
= 0.

2. Use the graphing facilities of Maple to approximate solutions to the following nonlinear sys-
tems.

a. xl(1-xl)+4x2=12, b. 5xf - xl = 0,
(XI - 2)2 + (2X2 - 3)2 = 25. X2 - O.25(sin Xl + cos X2) = 0.

C. 15xI + xi - 4X3 = 13, d. lOx I - 2xi + X2 = 0,

- 2X3 - 5
xf + lOx2 - X3 = 11, 8xi + 4x; - 9 = 0.
xi - 25x3 = -22. 8X2X3 +4 = 0.
618 C HAP T E R 10 • Numerical Solutions of Nonlinear Systems of Equations

3. Use Newton's method to find a solution to the following nonlinear systems in the given do-
main. Iterate untilllx(k) - X(k-l) 1100 < 10-6 .

a. 3xr - xi = 0, b. In(xr + xi} - sin(xlx2) = In2 + In Jr,

3XlXi - xi -1 = O. eX) -X2 + COS(XlX2) = O.
Use x(O) = (1, 1)'. Use x(O) = (2, 2)'.

c. xi + XlX2 - XlX3 +6 = 0,
eX) + e
X2
- X3 = 0,
xi - 2XlX3 = 4.
Use x(O) = (-1, -2, 1)'.

d. 6Xl - 2 COS(X2X3) - 1 = 0,

9X2 + Jxr + sinx3 + 1.06 + 0.9 = 0,

6OX3 + 3e- + IOJr - 3 = X
)X2 O.
Use x(O) = (0, 0, 0)'.

4. Use the answers obtained in Exercise 2 as initial approximations to Newton's method. Iterate
until II X(k) - x(k-I) 1100 < 10- 6
5. The nonlinear system

1
3Xl - COS(X2X3) - 2 = 0,

221
Xl - 625x 2 - 4 = 0,

IOJr - 3
e- X
)X2 + 20X3 + = 0
3

has a singular Jacobian matrix at the solution. Apply Newton's method with x(O) = (1, 1 - 1)'.
Note that convergence may be slow or may not occur within a reasonable number of iterations.
6. The nonlinear system

-Xl + 3X2 - 2x3 = X2X4,

Xl - 2X2 + 3X3 = X3X4,

xf +X; +X~ = 1
-

has six solutions.

a. Show that if (Xl, X2, X3, X4)' is a solution then (-Xl, -X2, -X3, -X4)' is a solution. ,,
b. Use Newton's method three times to approximate all solutions. Iterate until II X(k)_ -
•
X(k-I) 1100 < 10- 5 •
-•
7. Show that when n = 1, Newton's method given by Eq. (10.9) reduces to the familiar Newton's (

method given by Eq. (2.5).

 10.2 Newton's Method 619

8. What does Newton's method reduce to for the linear system Ax = b given by

where A is a nonsingular matrix?

9. In calculating the shape of a gravity-flow discharge chute that will minimize transit time of
discharged granular particles, C. Chiarella, W. Charlton, and A. W. Roberts [eCR] solve the
following equations by Newton's method:
(i) /n(eJ, ... ,eN) = sinen+J(1 - f-i,Wn+J) - sinen(l - f-i,W n ) 0, for each n
Vn+J Vn
I. 2, ... , N - I.
(ii) /N(e J, ... , 8N ) = fly L~ J tan8i - X = 0, where
a. 2 _
vn - Vo
2
+ 2gnfly -
. "n
2f-i,fly L-j=J
I
' for each n = I, 2, .... N, and
cose j
N 1
b. Wn = -flyvn Li=J 3 (j' for each n = 1,2, ... , N.
Vi cos i

The constant Vo is the initial velocity of the granular material, X is the x-coordinate of the end
of the chute, f-i, is the friction force, N is the number of chute segments, and g = 32.17ftls2
e
is the gravitational constant. The variable i is the angle of the ith chute segment from the
vertical, as shown in the following figure, and Vi is the particle velocity in the ith chute seg-
r
ment. Solve (i) and (ii) for e = (e J, ... , eN with f-i, = 0, X = 2, fly = 0.2, N = 20, and
Vo = 0, where the values for Vn and Wn can be obtained directly from (a) and (b). Iterate until
IleCk) - eCk-J)lloo < 10-2 .

(0,0)
x

yz

10. The amount of pressure required to sink a large, heavy object in a soft homogeneous soil that
lies above a hard base soil can be predicted by the amount of pressure required to sink smaller
objects in the same soil. Specifically, the amount of pressure p required to sink a circular plate
of radius r a distance d in the soft soil, where the hard base soil lies a distance D > d below
the surface, can be approximated by an equation of the form