MODULE IV

INTRODUCTION

Lesson 1 Golden Search Method

Lesson 2 Quadratic Interpolation

Lesson 3 Newton’s Method

EEPC102 Module IV
 MODULE IV

OPTIMIZATION

 INTRODUCTION

Root finding and optimization are related in the sense that both involve
guessing and searching for a point on a function. Root location involves searching
for zeros of a function or functions. In contrast, optimization involves searching
for either the minimum or the maximum. The optimum is the point where the
curve is flat. In mathematical terms, this corresponds
to the x value where the derivative f’(x) is equal to zero. Additionally, the second
derivative, f ”(x), indicates whether the optimum is a minimum or a maximum:
if f”’(x) <0, the point is a maximum; if f”’(x) >0, the point is a minimum.

Common applications: Minimal cost, maximal profit, minimal error, optimal

design, optimal management, variational principles.

OBJECTIVES

After studying the module, you should be able to:

1. Understand why and where optimization occurs in engineering problem

solving.
2. Locate the optimum of a single variable function with the golden-section
search, quadratic interpolation, and Newton’s method.
3. Recognize the trade-offs among these approaches, with particular attention
to initial guesses and convergence.
4. Implement optimization with the use of Microsoft Excel

 DIRECTIONS/ MODULE ORGANIZER

EEPC102 Module IV
 There are three lessons in the module. Read each lesson carefully then
answer the exercises/activities to find out how much you have benefited from
it. Work on these exercises carefully and submit your output to your instructor.
In case you encounter difficulty, discuss this with your instructor during the face-
to-face meeting.
Lesson 1

 Golden Section Search Method

The Golden Section Search method is used to find the maximum or

minimum of a unimodal function. (A unimodal function contains only one
minimum or maximum on the interval [a,b].)

As shown in Figure 1 below, choose three points xl, x1, xu ( xl < x1 < xu )along
the x-axis with corresponding values of the function f(xl) , f(x1) ,and f(xu)
respectively. Since, f(x1) > f(xl) and f(x1) > f(xu) the maximum must lie between
xl and xu . Now a fourth point denoted by x2 is chosen to be between the larger
of the two intervals of [xl , x1] and [x1, x2] . Assuming that the interval [xl , x1] is
larger than [x1, x2] , we would chose [xl , x1] l as the interval in which x2 is chosen.
If f(x2) > f(x1) then the new three points would be xl < x2 < x1 else if f(x2) < f(x1)
then the new three points are x2 < x1 < xu. This process is continued until the
distance between the outer points is sufficiently small.

FIGURE 1

Intermediate Points in the Golden Section Search

EEPC102 Module IV
We chose the first intermediate point xl to equalize the ratio of the lengths as
shown in Eq. (1) where a and b are distance as shown in Figure 2. Note that a +
b is equal to the distance between the lower and upper boundary points xl and
xu .
𝑎 𝑏
= (1)
𝑎+𝑏 𝑎

FIGURE 2 Determining the first intermediate point

The second intermediate point x2, is chosen similarly in the interval a to satisfy
the following ratio in Eq. (2) where the distances of a and b are shown in Figure
3.
𝑏 𝑎−𝑏
= (2)
𝑎 𝑏

FIGURE 3 Determining the second intermediate point

EEPC102 Module IV
Does the Golden Section Search have anything to do with the Golden Ratio?

The ratios in Equations (1) and (2) are equal and have a special value known as
the Golden Ratio. The Golden Ratio has been used since ancient times in
various fields such as architecture, design, art and engineering. To determine
the value of the Golden Ratio let
R = a / b , then Eq. (1) can be written as

1
1+𝑅 =
𝑅
Or

R2 + R – 1 = 0 (3)

Using the quadratic formula, the positive root of Eq. (3) is

−1 + √1 − 4(−1)
𝑅=
2

√5 − 1
=
2
= 0.61803

In other words, the intermediate points x1 and x2 are chosen such that, the
ratio of the distance from these points to the boundaries of the search region is
equal to the golden ratio as shown in Figure 4.

FIGURE 4 Intermediate points and their relation to boundary

points

EEPC102 Module IV
The Golden Section Search Algorithm

The following algorithm can be used to determine the maximum of a function

f(x) .

Initialization:
Determine xl and xu which is known to contain the maximum of the function f(x).

Step 1
Determine two intermediate points x1 and x2 such that
x1 = xl + d
x2 = xu - d

where:

√5 − 1
𝑑= (𝑥𝑢 − 𝑥𝑙 )
2

Step 2
Evaluate f(x1) and f(x2).
If f(x1) > f(x2), then determine new xl, x1, x2 and xu as shown in Equation set (5).
Note that the only new calculation is done to determine the new x1 .
xl = x2
x2 = x1
xu = xu (5)
√5 − 1
𝑥1 = 𝑥𝑙 + (𝑥𝑢 − 𝑥𝑙 )
2
If f(x1) < f(x2),, then determine new xl, x1, x2 and xu as shown in Equation set (6).
Note that the only new calculation is done to determine the new x2 .
xl = xl
x2 = x1
x1 = x2 (6)
√5 − 1
𝑥2 = 𝑥𝑢 − (𝑥𝑢 − 𝑥𝑙 )
2
Step 3
𝑥 +𝑥
If xu - xl < ε (a sufficiently small number), then the maximum occurs at 𝑢 2 𝑙 and
stop iterating, else go to Step 2.

Example:

Use the golden-section search to find the maximum of

𝑥2
( )
𝑓 𝑥 = 2 sin 𝑥 −
10

EEPC102 Module IV
 within the interval xl = 0 and xu = 4.
Solution:
First, the golden ratio is used to create the two interior points
√5−1
𝑑 = 2 (4 − 0 ) = 2.472
x1 = 0 + 2.472 = 2.472
x2 = 4 – 2.472 = 1.528
The function can be evaluated at the interior points
1.5282
𝑓 (𝑥2 ) = 𝑓 (1.528) = 2 sin(1.528) − = 1.765
10
𝑓 (𝑥1 ) = 𝑓 (2.472) = 0.63

Because f(x2) > f(x1), the maximum is in the interval defined by xl, x2, and x1.
Thus, for the new interval, the lower bound remains xl = 0, and x1 becomes the
upper bound, that is, xu = 2.472. In addition, the former x2 value becomes the
new x1, that is, x1 = 1.528. Further, we do not have to recalculate f(x1) because
it was determined on the previous iteration as f(1.528) = 1.765.
All that remains is to compute the new values of 𝑑 and x2,
√5−1
𝑑= (2.472 − 0 ) = 1.528
2
x2 = 2.4721 – 1.528 = 0.944
The function evaluation at x2 is f(0.994) = 1.531. Since this value is less than the
function value at x1, the maximum is in the interval prescribed by x2, x1, and xu.
The process can be repeated, with the results tabulated below:
i 𝑥𝑙 𝑓(𝑥𝑙 ) 𝑥2 𝑓(𝑥2 ) 𝑥1 𝑓(𝑥1 ) 𝑥𝑢 𝑓(𝑥𝑢 ) d
1 0 0 1.5279 1.7647 2.4721 0.6300 4.000 -3.1136 2.4721
2 0 0 0.9443 1.5310 1.5279 1.7647 2.4721 0.6300 1.5279
3 0.9443 1.5310 1.5279 1.7647 1.8885 1.5432 2.4721 0.6300 0.9443
4 0.9443 1.5310 1.3050 1.7595 1.5279 1.7647 1.8885 1.5432 0.5836
5 1.3050 1.7595 1.5279 1.7647 1.6656 1.7136 1.8885 1.5432 0.3607
6 1.3050 1.7595 1.4427 1.7755 1.5279 1.7647 1.6656 1.7136 0.2229
7 1.3050 1.7595 1.3901 1.7742 1.4427 1.7755 1.5279 1.7647 0.1378
8 1.3901 1.7742 1.4427 1.7755 1.4752 1.7732 1.5279 1.7647 0.0851

Note that the current maximum is highlighted for every iteration. After the
eighth iteration, the maximum occurs at x = 1.4427 with a function value of
1.7755. Thus, the result is converging on the true value of 1.7757 at x = 1.4276.

EEPC102 Module IV