Solution of Equations of One Variable

Chapter 1: Solution of Equations of One Variable

Introduction.
One of the most basic problems of numerical approximation is the root-finding
problem, i.e. finding a root, or solution, of an equation of the form 𝑓(𝑥) = 0.

1.1 The Bisection Method.

The first and most elementary technique we consider is the Bisection, or Binary-
Search, method. The Bisection method is used to determine, to any specified
accuracy that your computer will permit, a solution to f(x) = 0 on an interval [a,
b], provided that f is continuous on the interval and that f(a) and f(b) are of
opposite sign.

Principle of the method

To begin the Bisection method, set 𝑎1 = 𝑎 and 𝑏1 = 𝑏, and let 𝑝1 be the midpoint
of the interval [a, b]:

𝑏1 − 𝑎1
𝑝1 = 𝑎1 +
2

If 𝑓(𝑝1 ) = 0, then the root 𝑝 is given by 𝑝 = 𝑝1 ; if 𝑓(𝑝1 ) ≠ 0, then 𝑓(𝑝1 ) has the
same sign as either 𝑓(𝑎1 ) or 𝑓(𝑏1 ).
If 𝑓(𝑝1 ) and 𝑓(𝑎1 ) have the same sign, then 𝑝 is in the interval (𝑝1 , 𝑏1 ), and we
set
𝑎2 = 𝑝1 and 𝑏2 = 𝑏1

If, on the other hand, 𝑓(𝑝1 ) and 𝑓(𝑎1 ) have opposite signs, then 𝑝 is in the
interval (𝑎1 , 𝑝1 ), and we set
𝑎2 = 𝑎1 and 𝑏2 = 𝑝1

We reapply the process to the interval [𝑎2 , 𝑏2 ], and continue forming [𝑎3 , 𝑏3 ],
[𝑎4 , 𝑏4 ],… Each new interval will contain 𝑝 and have length one half of the
length of the preceding interval.

In summary, An interval [𝑎𝑛+1 , 𝑏𝑛+1 ], containing an approximation to

a root of 𝑓(𝑥) = 0 is constructed from an interval [𝑎𝑛 , 𝑏𝑛 ], containing the root
by first letting
𝑏𝑛 − 𝑎𝑛
𝑝𝑛 = 𝑎𝑛 +
2
Page 1 of 13
 Solution of Equations of One Variable

Then, if 𝑓(𝑎𝑛 )𝑓(𝑝𝑛 ) < 0, set 𝑎𝑛+1 = 𝑎𝑛 and 𝑏𝑛+1 = 𝑝𝑛

Otherwise, if 𝑓(𝑎𝑛 )𝑓(𝑝𝑛 ) > 0 set 𝑎𝑛+1 = 𝑝𝑛 and 𝑏𝑛+1 = 𝑏𝑛

There are three stopping criteria commonly incorporated in the Bisection

method:
(i) First, the method stops if one of the midpoints happens to coincide with the
root. In this case, a test of the form
|𝑓(𝑝𝑛 )| < 𝜖 (𝑬𝟏)
is used, where 𝜖 is a small number related to the tolerance.
(ii) The method also stops when the length of the search interval is less than
some prescribed tolerance 𝑇𝑂𝐿. The test is of the form
|𝑎𝑛 − 𝑏𝑛 | ≤ 𝑇𝑂𝐿 (𝑬𝟐)
(iii) The procedure also stops if the number of iterations exceeds a preset bound
𝑁𝑚𝑎𝑥 . To start the Bisection method, an interval [𝑎, 𝑏] must be found
with 𝑓(𝑎)𝑓(𝑏) < 0. At each step, the length of the interval known to
contain a zero of f is reduced by a factor of 2. Since the midpoint 𝑝1
must be within (𝑏 − 𝑎)/2 of the root p, and each succeeding iteration
divides the interval under consideration by 2, we have
𝑏−𝑎
|𝑝𝑛 − 𝑝| ≤ 𝑛
2
As a consequence, it is easy to determine a bound for the number of
iterations needed to ensure a given tolerance. If the root needs to be
determined within the tolerance 𝑇𝑂𝐿, we need to determine the number of
iterations, n, so that
𝑏−𝑎
< 𝑇𝑂𝐿
2𝑛
Solving for n in this inequality gives
𝑏−𝑎
𝑛 > log 2 ( )
𝑇𝑂𝐿
Since the number of required iterations to guarantee a given accuracy depends on
the length of the initial interval [a, b], we have to choose this interval as small as
possible.
To avoid the possibility of overflow or underflow in the multiplication of 𝑓(𝑎𝑛 )
and 𝑓(𝑝𝑛 ) the test 𝑓(𝑎𝑛 )𝑓(𝑝𝑛 ) < 0 is performed using the signum function
which is defined as:
1 if 𝑥 > 0
sgn(𝑥) = { −1 if 𝑥 < 0
0 if 𝑥 = 0
Page 2 of 13
 Solution of Equations of One Variable

and the test sgn(𝑓(𝑎𝑛 ))sgn(𝑓(𝑝𝑛 )) < 0 is used instead of 𝑓(𝑎𝑛 )𝑓(𝑝𝑛 ) < 0.
The, following Algorithm can be used to solve equation 𝑓(𝑥) = 0 using the
bisection method:
Algorithm
INPUT: endpoints 𝑎, 𝑏; tolerance TOL; Maximum number of iterations 𝑁0
Set 𝑖 = 1
𝐹𝐴 = 𝑓(𝑎)
𝐖𝐡𝐢𝐥𝐞 (𝑖 ≤ 𝑁0 ) do
𝑝 = 𝑎 + (𝑏 − 𝑎)/2; (compute 𝑝𝑖 )
𝐹𝑃 = 𝑓(𝑝)
If (𝐹𝑃 = 0 or (𝑏 − 𝑎)/2 < 𝑇𝑂𝐿 ) Then
𝐎𝐮𝐭𝐩𝐮𝐭 (𝑝); (Procedure completed successfully)
STOP
𝑖 = 𝑖 + 1.
IF ( sign(𝐹𝐴) × sign(𝐹𝑃) > 0 ) Then
𝑎 = 𝑝; (Compute 𝑎𝑖 and 𝑏𝑖 )
𝐹𝐴 = 𝐹𝑃
else
𝑏=𝑝
endif
end do

OUTPUT (“The procedure was unsuccessful”);

END.

Problem: 1) Translate the above algorithm in PASCAL, and run the programs
to find the roots of the function 𝑓(𝑥) = 𝑥(𝑥 + 2)(𝑥 + 1)2 (𝑥 − 1)3 (𝑥 − 2) in
the intervals (i) [−1.5, 2.5], (ii) [−0.5, 2.4], (iii) [−0.5, 3.0], and (iv) [−3, −0.5].

1.2 Newton’s Method and its extensions.

Newton’s (or the Newton-Raphson) method is one of the most powerful and well-
known numerical methods for solving a root-finding problem. There are many
ways of introducing Newton’s method.

Page 3 of 13
 Solution of Equations of One Variable

1.2.1 Newton-Raphson Method

This method is distinguished from other methods by the fact that it requires the
evaluation of both the function 𝑓(𝑥), and the derivative 𝑓′(𝑥), at arbitrary points
𝑥. The Newton-Raphson formula consists geometrically of extending the tangent
line at a current point 𝑥𝑖 until it crosses zero, then setting the next guess 𝑥𝑖+1 to
the abscissa of that zero-crossing. Algebraically, the method derives from the
familiar Taylor series expansion of a function in the neighborhood of a point.

Suppose 𝑓 ∈ 𝐶 2 [𝑎, 𝑏]. Let 𝑥0 ∈ [𝑎, 𝑏] be an approximation to 𝑥 such that

𝑓′(𝑥0 ) ≠ 0 and |𝑥 − 𝑥0 | is small. The first Taylor polynomial for 𝑓(𝑥) expanded
about 𝑥0 and evaluated at 𝑥 as
𝑓 ′′ (𝜉(𝑥))
𝑓(𝑥) = 𝑓(𝑥0 ) + (𝑥 − 𝑥0 )𝑓 ′ (𝑥0 ) + (𝑥 − 𝑥0 )2
2
Where 𝜉(𝑥) lies between 𝑥 and 𝑥0 . Since 𝑓(𝑥) = 0, this equation gives
𝑓 ′′ (𝜉(𝑥))
𝑓(𝑥0 ) + (𝑥 − 𝑥0 )𝑓 ′ (𝑥0 ) + (𝑥 − 𝑥0 )2 = 0
2
Newton’s method is derived by assuming that since |𝑥 − 𝑥0 | is small, the term
involving (𝑥 − 𝑥0 )2 is much smaller, so

Page 4 of 13
 Solution of Equations of One Variable

𝑓(𝑥0 ) + (𝑥 − 𝑥0 )𝑓 ′ (𝑥0 ) ≅ 0
And solving for 𝑥 gives
𝑓(𝑥0 )
𝑥 = 𝑥0 − ≡ 𝑥1
𝑓 ′ (𝑥0 )
This sets the stage for Newton’s method, which starts with an initial
approximation 𝑥0 and generates the sequence {𝑥𝑛 }∞
𝑛=0 , by

𝑓(𝑥𝑛−1 )
𝑥𝑛 = 𝑥𝑛−1 − , 𝑛≥1
𝑓 ′ (𝑥𝑛−1 )
Graphically, the successive approximation for 𝑥1 , 𝑥2 , 𝑥3 , … are obtained using
successive tangents (as illustrated). Starting with the initial approximation 𝑥0 , the
approximation 𝑥1 is the x-intercept of the tangent line to the graph of 𝑓 at
(𝑥0 , 𝑓(𝑥0 )). The approximation 𝑥2 is the x-intercept of the tangent line to the
graph of 𝑓 at (𝑥1 , 𝑓(𝑥1 )) and so on.
The stopping-technique inequalities given with the Bisection method are
applicable to Newton’s method. That is, select a tolerance 𝜀 > 0, and construct 𝑥1 ,
𝑥2 , …, 𝑥𝑁 until
|𝑥𝑛 − 𝑥𝑛−1 | < 𝜀,
|𝑥𝑛 − 𝑥𝑛−1 |
< 𝜀, 𝑜𝑟
|𝑥𝑛 |
|𝑓(𝑥𝑛 )| < 𝜀
Problem: (i) Propose an algorithm to compute the root of a function 𝑓
following the Newton-Raphson procedure. (ii) Give the translation of this
algorithm into TURBO PASCAL. (iii) Run the above program taking as the
trial function 𝑓(𝑥) = 4𝑥 cos(2𝑥) − (𝑥 − 2)2 in the interval [0, 8] with
an accuracy of 10−5 .
1.2.2 Secant Method
Newton’s method is an extremely powerful technique, but it has a major
weakness: the need to know the value of the derivative of 𝑓 at each
approximation. Frequently, 𝑓′(𝑥) is far more difficult and needs more arithmetic
operations to calculate than 𝑓(𝑥). To circumvent the problem of the derivative
evaluation in Newton’s method, we introduce a slight variation. By definition

Page 5 of 13
 Solution of Equations of One Variable

𝑓(𝑥) − 𝑓(𝑥𝑛−1 )
𝑓′(𝑥𝑛−1 ) = lim
𝑥→𝑥𝑛−1 𝑥 − 𝑥𝑛−1
If 𝑥𝑛−2 is closed to 𝑥𝑛−1 , then
𝑓(𝑥𝑛−2 ) − 𝑓(𝑥𝑛−1 ) 𝑓(𝑥𝑛−1 ) − 𝑓(𝑥𝑛−2 )
𝑓 ′ (𝑥𝑛−1 ) ≈ =
𝑥𝑛−2 − 𝑥𝑛−1 𝑥𝑛−1 − 𝑥𝑛−2
Using this approximation for 𝑓 ′ (𝑥𝑛−1 ) in Newton’s formula gives
𝑓(𝑥𝑛−1 )(𝑥𝑛−1 − 𝑥𝑛−2 )
𝑥𝑛 = 𝑥𝑛−1 − , 𝑛≥2
𝑓(𝑥𝑛−1 ) − 𝑓(𝑥𝑛−2 )
This technique is called the Secant method. Graphically, the technique consists in
choosing two initial approximations 𝑥0 and 𝑥1 and determine 𝑥2 as the x-
intercept of the secant line to the curve, the line through (𝑥0 , 𝑓(𝑥0 )) and
(𝑥1 , 𝑓(𝑥1 )). This places the approximation 𝑥2 of the root 𝑥 closer to the endpoint
of the interval for which 𝑓 has smaller absolute value. The approximation 𝑥3 is
obtained as the x-intercept of the line joining (𝑥1 , 𝑓(𝑥1 )) and (𝑥2 , 𝑓(𝑥2 )), and so
on.

Because the Secant method does not have the root-bracketing property of the
Bisection method, the method does not always converge, but when it does
converge, it generally does so much faster than the Bisection method.

Page 6 of 13
 Solution of Equations of One Variable

Two stopping conditions are used in the Secant method:

(i) First, we assume that 𝑥𝑛 is sufficiently accurate when |𝑥𝑛 − 𝑥𝑛−1 | is
within a given tolerance.
(ii) A safeguard exit based upon a maximum number of iterations is given in
case the method fails to converge as expected.

Problem: Propose an algorithm that solves an equation of the form 𝑓(𝑥) = 0

using the secant method. Translate it into PASCAL.

1.2.3 Method of False position

It is possible to derive a method based on the combination of the Bisection and
the Secant methods. The method of False Position (or Regula Falsi) is a hybrid
bisection-secant method that constructs approximating lines similar to those of
the Secant method but always brackets the root in the manner of the Bisection
method. As with the Bisection method, the method of False Position requires that
an initial interval [𝑎, 𝑏] first be found, with 𝑓(𝑎) and 𝑓(𝑏) of opposite sign. With
𝑥0 = 𝑎 and 𝑥1 = 𝑏, the approximation, 𝑥2 , is given by
𝑓(𝑥1 )(𝑥1 − 𝑥0 )
𝑥2 = 𝑥1 −
𝑓(𝑥1 ) − 𝑓(𝑥0 )
 If 𝑓(𝑥2 ) and 𝑓(𝑥1 ) are of opposite signs, then 𝑥1 and 𝑥2 bracket a root and
we can determine 𝑥3 as the x-intercept of the line joining (𝑥1 , 𝑓(𝑥1 )) and
(𝑥2 , 𝑓(𝑥2 )), giving
𝑓(𝑥2 )(𝑥2 − 𝑥1 )
𝑥3 = 𝑥2 −
𝑓(𝑥2 ) − 𝑓(𝑥1 )
 Alternatively, if 𝑓(𝑥2 ) and 𝑓(𝑥1 ) have the same sign, the root is bracketed
by 𝑥0 and 𝑥2 . And the expression for 𝑥3 is given by the relation above after
the updating 𝑥1 = 𝑥0 .
In a similar manner, once 𝑥3 is found, the sign of 𝑓(𝑥3 ) 𝑓(𝑥2 ) determines
whether we use 𝑥2 and 𝑥3 or 𝑥3 and 𝑥1 to compute 𝑥4 .

Although the method of False Position may appear superior to the Secant
method, it generally converges more slowly. It converges even more slowly than
the Bisection method.

Problem: Propose an algorithm to compute the rote of a function 𝑓 using the

False position method. Give the translation of this algorithm into PASCAL.

Page 7 of 13
 Solution of Equations of One Variable

1.3 Zeros of polynomials

A polynomial of degree 𝑛 has the form

𝑃(𝑥) = 𝑎0 + 𝑎1 𝑥 + 𝑎2 𝑥 2 + ⋯ + 𝑎𝑛 𝑥 𝑛
where the 𝑎𝑖 , 𝑖 = 0,1, … , 𝑛 are the coefficients of 𝑃.

Theorem: Fundamental Theorem of Algebra

If 𝑃(𝑥) is a polynomial of degree 𝑛 ≥ 1 with real or complex coefficients, then
𝑃(𝑥) = 0 has at least one (possibly complex) root.

1.3.1 Horner’s Method

To use Newton’s method to locate approximate zeros of a polynomial 𝑃(𝑥), we
need to evaluate 𝑃(𝑥) and 𝑃′(𝑥) at specified values. Since 𝑃(𝑥) and 𝑃′(𝑥) are
both polynomials, computational efficiency requires that the evaluation of these
functions be done in the nested manner. Horner’s method incorporates this
nesting technique, and, as a consequence, requires only n multiplications and n
additions to evaluate an arbitrary nth-degree polynomial.
Let
𝑃(𝑥) = 𝑎𝑛 𝑥 𝑛 + 𝑎𝑛−1 𝑥 𝑛−1 + ⋯ + 𝑎1 𝑥 + 𝑎0
Define 𝑏𝑛 = 𝑎𝑛 and
𝑏𝑘 = 𝑎𝑘 + 𝑏𝑘+1 𝑥0 , 𝑘 = 𝑛 − 1, 𝑛 − 2, … ,1,0
Then 𝑏0 = 𝑃(𝑥0 ). Moreover, if
𝑄(𝑥) = 𝑏𝑛 𝑥 𝑛−1 + 𝑏𝑛−1 𝑥 𝑛−2 + ⋯ + 𝑏2 𝑥 + 𝑏1 ,
Then 𝑃(𝑥) = (𝑥 − 𝑥0 )𝑄(𝑥) + 𝑏0 .
An additional advantage of using the Horner (or synthetic-division) procedure is
that, from
𝑃(𝑥) = (𝑥 − 𝑥0 )𝑄(𝑥) + 𝑏0
The derivative 𝑃′(𝑥) of 𝑃(𝑥) is obtained as
𝑃′ (𝑥) = 𝑄(𝑥) + (𝑥 − 𝑥0 )𝑄′(𝑥)
Hence 𝑃′ (𝑥0 ) = 𝑄(𝑥0 )
As a consequence, when the Newton-Raphson method is used to find an
approximate zero of a polynomial, 𝑃′(𝑥) and 𝑃(𝑥) are evaluated in a similar
manner.

Page 8 of 13
 Solution of Equations of One Variable

Problem:
1. Propose an algorithm to compute the value of a polynomial 𝑃(𝑥) and
its derivative 𝑃′(𝑥) at a given point 𝑥0 using the Horner method.
2. Propose an algorithm that uses the synthetic division procedure of
part 1 to find a root of the polynomial 𝑃(𝑥).
3. Give the translation of the above algorithm into PASCAL.

1.3.2 Complex Zeros: Müller’s Method

There are a number of root-finding problems for which the Secant, False Position,
and Newton’s methods will not give satisfactory results. This is the case e.g. when
finding the complex roots of polynomials because this methods require real
numbers as initial approximations, hence subsequent approximations are also real
numbers.
Müller’s method is a generalization of the Secant method to complex roots of
polynomial functions. The Secant method finds the zero using the line passing
through points on the graph of the function that corresponds to the two
immediately previous approximations. Müller’s method uses the parabola passing
through the three immediately previous points to find the new approximation.

Method
Suppose that three initial approximations, 𝑝0 , 𝑝1 , and 𝑝2 , are given for a solution
of f(x) = 0. The derivation of Müller’s method for determining the next
approximation 𝑝3 begins by considering the quadratic polynomial
𝑃(𝑥) = 𝑎(𝑥 − 𝑝2 )2 + 𝑏(𝑥 − 𝑝2 ) + 𝑐
that passes through (𝑝0 , 𝑓(𝑝0 )), (𝑝1 , 𝑓(𝑝1 )), and (𝑝2 , 𝑓(𝑝2 )). The constants 𝑎, 𝑏,
and 𝑐 can be determined from the conditions

𝑓(𝑝0 ) = 𝑎(𝑝0 − 𝑝2 )2 + 𝑏(𝑝0 − 𝑝2 ) + 𝑐

{ 𝑓(𝑝1 ) = 𝑎(𝑝1 − 𝑝2 )2 + 𝑏(𝑝1 − 𝑝2 ) + 𝑐
𝑓(𝑝2 ) = 𝑎02 + 𝑏0 + 𝑐

Giving

Page 9 of 13
 Solution of Equations of One Variable

𝑐 = 𝑓(𝑝2 )
(𝑝0 − 𝑝2 )2 [𝑓(𝑝1 ) − 𝑓(𝑝2 )] − (𝑝1 − 𝑝2 )2 [𝑓(𝑝0 ) − 𝑓(𝑝2 )]
𝑏=
(𝑝0 − 𝑝2 )(𝑝1 − 𝑝2 )(𝑝0 − 𝑝1 )
(𝑝1 − 𝑝2 ) [𝑓(𝑝0 ) − 𝑓(𝑝2 )] − (𝑝0 − 𝑝2 ) [𝑓(𝑝1 ) − 𝑓(𝑝2 )]
𝑎=
{ (𝑝0 − 𝑝2 )(𝑝1 − 𝑝2 )(𝑝0 − 𝑝1 )
The next approximate root 𝑝3 is given by solving 𝑃(𝑥) = 0, hence
−𝑏 ± √𝑏 2 − 4𝑎𝑐
𝑝3 − 𝑝2 =
2𝑎
To obtain a more accurate four-digit rounding approximation, we can change the
form of the quadratic formula by rationalizing the numerator and the above relation
transforms to
−2𝑐
𝑝3 − 𝑝2 =
𝑏 ± √𝑏 2 − 4𝑎𝑐
Among the two solutions above given, Müller’s method considers the solution for
which the sign in front of the radical agree with the sign of b.
2𝑐
𝑝3 = 𝑝2 −
𝑏 + sgn(𝑏)√𝑏 2 − 4𝑎𝑐
Chosen in this manner, the denominator will be the largest in magnitude, which
avoids the possibility of subtracting nearly equal numbers and results in 𝑝3 being
selected as the closest root of 𝑃(𝑥) = 0 to 𝑝2 .
The same routine applies if the approximate solution 𝑝3 is not within the desired
accuracy, and 𝑝4 is determined upon replacing, 𝑝0 , 𝑝1 and 𝑝2 by 𝑝1 , 𝑝2 and 𝑝3
respectively.
The method continues until a satisfactory approximation is obtained. Since the
method involves the radical √𝑏 2 − 4𝑎𝑐 at each step, the method approximates
complex roots when 𝑏 2 − 4𝑎𝑐 < 0, provided, of course, that complex arithmetic
is used. The following algorithm implements this procedure
Algorithm
INPUT: initial approximations 𝑝0 , 𝑝1 and 𝑝2 ; tolerance TOL; Maximum number
of iterations 𝑁0
Step1: Set ℎ1 = 𝑝1 − 𝑝0 ;
ℎ2 = 𝑝2 − 𝑝1 ;
𝛿1 = (𝑓(𝑝1 ) − 𝑓(𝑝0 ))/ℎ1 ;
𝛿2 = (𝑓(𝑝2 ) − 𝑓(𝑝1 ))/ℎ2 ;

Page 10 of 13
 Solution of Equations of One Variable

𝑑 = (𝛿2 − 𝛿1 )/(ℎ1 + ℎ2 );
𝑖 = 3;
Step 2: 𝐖𝐡𝐢𝐥𝐞 (𝑖 ≤ 𝑁0 ) do Steps 3-7
Step 3: 𝑏 = 𝛿2 + ℎ2 𝑑;
𝐷 = √𝑏 2 − 4𝑓(𝑝2 )𝑑; (May require complex arithmetic)
Step 4: if |𝑏 − 𝐷| < |𝑏 + 𝐷| then ;
set 𝐸 = 𝑏 + 𝐷;
else
set 𝐸 = 𝑏 − 𝐷
end if
Step 5: set ℎ = −2𝑓(𝑝2 )/𝐸;
𝑝 = 𝑝2 + ℎ;
Step 6: if ℎ < 𝑇𝑂𝐿 then ;
OUTPUT(p); (“the procedure is successful”)
STOP
end if
Step 7: set 𝑝0 = 𝑝1 ; (“preparing for the next iteration”)
𝑝1 = 𝑝2 ;
𝑝2 = 𝑝;
ℎ1 = 𝑝1 − 𝑝0 ;
ℎ2 = 𝑝2 − 𝑝1 ;
𝛿1 = (𝑓(𝑝1 ) − 𝑓(𝑝0 ))/ℎ1 ;
𝛿2 = (𝑓(𝑝2 ) − 𝑓(𝑝1 ))/ℎ2 ;
𝑑 = (𝛿2 − 𝛿1 )/(ℎ1 + ℎ2 );
𝑖 = 𝑖 + 1.
Step 8: OUTPUT(“Method failed after 𝑁0 iterations”)
END.
Problem: (i) Propose an algorithm to compute the rote of a function 𝑓 using
the Müller method. (ii) Give the translation of this algorithm into PASCAL.
(iii) Run the above program and find all the taking as the trial function
𝑓(𝑥) = 𝑥 5 − 𝑥 4 + 2𝑥 3 − 3𝑥 2 + 𝑥 − 4 in the interval [0, 8] with an accuracy of
10−5 .

Page 11 of 13
 Solution of Equations of One Variable

41.4. Iterative Method For Nonlinear systems of Equations

Except in linear problems, root finding invariably proceeds by iteration, and this
is equally true in one or in many dimensions. Starting from some approximate
trial solution, a useful algorithm will improve the solution until some
predetermined convergence criterion is satisfied. For smoothly varying functions,
good algorithms will always converge, provided that the initial guess is good
enough.
It should be emphasized how crucially success depends on having a good first
guess for the solution, especially for multidimensional problems. Newton’s
method, a technique that is generally quadratically convergent.
4.2.1 Newton’s Method for systems of equations.
Consider a system of nonlinear equations of the form

𝑓1 (𝑥1 , 𝑥2 , … , 𝑥𝑛 ) = 0
𝑓 (𝑥 , 𝑥 , … , 𝑥𝑛 ) = 0
{ 2 1 2
⋮
𝑓𝑛 (𝑥1 , 𝑥2 , … , 𝑥𝑛 ) = 0

where each function 𝑓𝑖 can be thought of as mapping a vector 𝑥 = (𝑥1 , 𝑥2 , … , 𝑥𝑛 )𝑇

of the n-dimensional space ℝ𝑛 into the real line ℝ .
A general system of n nonlinear equations in n unknowns can be alternatively
represented by defining a function 𝑭, mapping ℝ𝑛 into ℝ𝑛 , by
𝑭(𝑥1 , 𝑥2 , … , 𝑥𝑛 ) = [𝑓1 (𝑥1 , 𝑥2 , … , 𝑥𝑛 ), 𝑓2 (𝑥1 , 𝑥2 , … , 𝑥𝑛 ), . . . , 𝑓𝑛 (𝑥1 , 𝑥2 , … , 𝑥𝑛 ) ]
If vector notation is used to represent the variables 𝑥1 , 𝑥2 , … , 𝑥𝑛 , the nonlinear
system assumes the form
𝑭(𝐱) = 𝟎
The functions 𝑓1 , 𝑓2 , … , 𝑓𝑛 are the coordinate functions of 𝑭.
The method starts by giving a rough initial guess 𝐱 (𝟎) or first order approximate.
If the exact solution of the problem is 𝐱 = 𝐱 (𝟎) + 𝛿𝐱, then in the neighborhoods
of 𝐱 (𝟎) , each of the functions 𝑓𝑖 can be expanded in Taylor series as
𝒏
𝜕𝑓𝑖
𝑓𝑖 (𝐱 (𝟎) + 𝛅𝐱) = 𝑓𝑖 (𝐱 (𝟎) ) + ∑ | 𝛿𝑥𝒋 + 𝑂(𝛿𝐱 2 )
𝜕𝑥𝑗 𝐱(𝟎)
𝒋=𝟏
Putting all these relations together, we obtain the general equation in its vector-
matrix form as
𝑭(𝐱 (0) + 𝛅𝐱) = 𝑭(𝐱 (0) ) + [𝕵(𝐱 (0) )](𝛿𝐱) + 𝑂(𝛿𝐱 2 )

Page 12 of 13
 Solution of Equations of One Variable

Where 𝕵 is an 𝑛 × 𝑛 matrix known as the Jacobian matrix. The elements of the

matrix 𝕵 are given by
𝜕𝑓1 (𝐱 (0) ) 𝜕𝑓1 (𝐱 (0) ) 𝜕𝑓1 (𝐱 (0) )
⋯
𝜕𝑥1 𝜕𝑥2 𝜕𝑥𝑛
(0) (0)
𝜕𝑓 𝜕𝑓2 (𝐱 ) 𝜕𝑓2 (𝐱 ) 𝜕𝑓2 (𝐱 (0) )
𝕵𝒊𝒋 (𝐱 (0) ) =
𝑖
| = ⋯
𝜕𝑥𝑗 𝐱(𝟎) 𝜕𝑥 1 𝜕𝑥 2 𝜕𝑥𝑛
⋮ ⋮ ⋱ ⋮
(0)
𝜕𝑓𝑛 (𝐱 ) 𝜕𝑓2 (𝐱 ) (0)
𝜕𝑓𝑛 (𝐱 (0) )
⋯
[ 𝜕𝑥1 𝜕𝑥2 𝜕𝑥𝑛 ]
By neglecting terms in 𝛿𝐱 2 and higher, and by letting 𝑭(𝐱 (0) + 𝛿𝐱) = 𝟎, we
obtain a set of linear equations for the corrections 𝛿𝐱 that move each function
closer to zero simultaneously, namely

[𝕵(𝐱 (0) )](𝛿𝐱) = −𝑭(𝐱 (0) )

This equation is solved for 𝛿𝐱 using any direct method and the new approximated
solution is calculated as
𝐱 (1) = 𝐱 (0) + 𝛿𝐱
If 𝐱 (𝟏) does not approximate the solution to the given tolerance, a new correction
term is calculated using the same procedure described above. Repeating this
procedure yields a sequence of vectors 𝐱 (𝟎) , 𝐱 (𝟏) , 𝐱 (𝟑) , …
The approximation 𝐱 (𝑘+1) of the solution is obtained from the previously
approximated solution 𝐱 (𝑘) by solving the linear system

[𝕵(𝐱 (𝑘) )](𝛿𝐱) = −𝑭(𝐱 (𝑘) )

and letting
𝐱 (𝑘+1) = 𝐱 (𝑘) + 𝛿𝐱
Newton’s method has a quadratic convergence, hence therefore it converges very
rapidly once an approximation is obtained that is near the true solution. However,
it is not always easy to determine starting values that will lead to a solution, and
the method is computationally expensive. To overcome this difficulty, a graphical
method can be used to choose the first approximation close to the real solution.

Page 13 of 13