CH3133 - Modelling, Simulation and Optimisation for Chemical Engineering

LECTURE 11 – WEEK 11
NUMERICAL TOOLS FOR
OPTIMIZATIONS
Ho Chi Minh City University of technology

Lecturer: Aqsha, Khoa Ta Dang

Ho Chi Minh City University of technology
Institut Teknologi Bandung
2024
Image Souurce: Licdn.com
Convex Optimization
Convex Optimization Problem
One-dimensional convex function: Significance of
Convexity

▪ A strictly convex or concave function provides a

single optimum
▪ A non-convex function may have multiple local
optimum
▪ No method can guarantee a global optimum for a
general non-convex function

Image:
Convex Optimization
Convexity of a one-dimensional function:
Given the function f = x2, determine its’ convexity

f ‘(x) > 0 → a strictly convex function

f ‘(x) ≥ 0 → a convex function (not strict)

Image:
CONVEXITY & CONCAVITY
For the following functions, determine if f(x) is convex, concave,
strictly convex, strictly concave, all, or none these classes (indefinite)
in the range of -4 ≤ x ≤ +4

i. f(x) = 3x2
ii. f(x) = 2x
iii. f(x) = -5x2
iv. f(x) = 2x2 - x3

Image:
CONVEXITY & CONCAVITY
Solution

1. f "(x) = 6, always positive, hence f(x) is both strictly convex and

convex.
2. f "(x) = 0 for all values of x, hence f(x) is convex and concave. Note
straight lines are both convex and concave simultaneously.
3. f "(x) = - 10, always negative, hence f(x) is both strictly concave and
concave.
4. f "(x) = 6 - 3x; may be positive or negative depending on the value of
x, hence f(x) is not convex or concave over the entire range of x.

Image:
CONVEXITY & CONCAVITY
How to determine the convexity (concavity) of a multivariable function,
f(x), where x is a vector of variables?

Answer:
Use Hessian matrix to represent second order derivative for a multi-
variable function

Image:
CONVEXITY & CONCAVITY
Classify the function 𝑓 𝑥 = 2𝑥12 − 3𝑥1 𝑥2 + 2𝑥22 using the categories in
Table below, or state that it does not belong in any of the categories.

Image:
CONVEXITY & CONCAVITY

The eigenvalues of H are 7 and 1, hence H(x) is positive-definite.

Consequently, f(x) is strictly convex (as well as convex)

Image:
CONVEXITY & CONCAVITY
previous one

the Table

Image:
CONVEXITY & CONCAVITY

Image:
CONVEXITY & CONCAVITY

Image:
1D Unconstrained Optimisation

Image:
1D Unconstrained Optimisation

Image:
Convex Optimisation
Unconstrained Non-Linear Optimisation

We need more convenient methods for searching optimal

solutions for a set of non-linear equations:

Image:
Convex Optimisation
Newton Method
▪ The necessary condition for f(x) to have an optimum is that f ’(x) = 0
▪ Applying Newtons method to solve f ’(x) = 0

Image:
NEWTON’S METHOD
Minimise the following function using the Newton method:
f(x) = x2 - x

• Assume initial search point (xk).

• Calculate f ‘(xk)
• Calculate f ‘‘(xk)
• Calculate (xk+1)
• Iterate until converge

Image:
NEWTON’S METHOD
Minimise the following function using the Newton method:
f(x) = x2 – x

f'(x) = 2x - 1
f"(x) = 2.
Note f"(x) is always positive-definite.
Let say we pick a bracket on the minimum, say x = -10 to x = 10. Assume
x0 = 10 is the starting point for the minimization.
x1= 10 – [19/2] = 1.5
x2= 1.5 – [2/2] = 0.5
x3= 0.5 – [0/2] = 0.5 (converged)
Note: if our initial guess is 3, then we only need 1 guess to obtain the solution
NEWTON’S METHOD

Image:
NEWTON’S METHOD
How to solve this in Matlab?
We can use subroutine fminsearch

>> fun = @(x)x^2-x;

x0 = [10];
x = fminsearch(fun,x0)

x=

0.5000

Image:
NEWTON’S METHOD
How to solve this in
Wolfram?

We can use subroutine

minimize

Image:
NEWTON’S METHOD
Example:
Minimise a nonquadratic function f(x) = x4 - x + 1

Solution:
For Newton's method, f' = 4x3 - 1 and f " = 12x2, and the sequence of
steps is

Image:
fun = @(x)x^4-x+1;
x0 = [10];
options = optimset('Display','iter','PlotFcns',@optimplotfval);
[x,fval,exitflag,output] = fminsearch(fun,x0,options)
NEWTON’S METHOD

Image:
NEWTON’S METHOD
How to solve this in Matlab?
We can use subroutine fminsearch

>> fun = @(x)x^4-x-1;

x0 = [10];
x = fminsearch(fun,x0)

x=

0.6299

Image:
NEWTON’S METHOD
Example – Do it yourself:
Minimize this function
− −

Image:
Using Matlab

Fungsi FMINBND
FMINBND Single-variable bounded nonlinear function minimization.
X = FMINBND(FUN,x1,x2) attempts to find a local minimizer X of the function
FUN in the interval x1 < X < x2. FUN is a function handle. FUN accepts
scalar input X and returns a scalar function value F evaluated at X.
………..

FUN can also be an anonymous function:

x = fminbnd(@(x) sin(x)+3,2,5)

If FUN is parameterized, you can use anonymous functions to capture the

problem-dependent parameters. Suppose you want to minimize the objective
given in the function myfun, which is parameterized by its second argument c.

FMINBND uses the Golden Section algorithm and Quadratic Interpolation. (FMINBND
Matlab documentation)
Example
Determine the value of x that minimizes the function :

f(x) = 5 x6 – 36 x5 + (165/2) x4 + 60 x3 + 36

In the interval [0,1] ; [1,2] and [2,3].

Multiple independent variable nonlinear optimization

• Nelder – Mead Algorithm

• Finding local minima of a function of multiple independent variables
• Slow but effective and compact.
Simplex Algorithm (Nelder-Mead)
zi = f (xi , yi )
Suppose a function with two independent variables:
dengan i = 1, 2 dan 3
The steps of the simplex algorithm are as follows:

1. Form an initial triangle from three points:

Use the following terms: B (Best); G (Good); and W (worst)

2. Determine the midpoint between B and G to
reflect point W.
B+G x + xG y B + yG
M = = f( B , )
2 2 2

3. The function value decreases

from W to B, and likewise from
W to G. A better (smaller)
value may lie at the reflection
of W on the line BG (point R).

R = M + ( M − W ) = 2M − W
4. Expansion to point E
If the function value of point R is smaller than W, then the direction to
the minimum point is correct. Maybe the minimum point is a little
further from point R. Therefore, the search point is expanded to point
E by increasing the distance d from point R.
E = R + (R − W ) = 2R − W
5. Shrinking to point C
If the function value of point R is greater than G, then the minimum
point is between WR. Determine point C:
C = ( M + R) / 2 = 2M − R
6. Shrinking to point B
If the function value of point C is not smaller than the function value of
W, then points G and W must shrink towards B. Point G is replaced by
point M while point W is replaced by point S which is the midpoint of
lines B and W.

7. Repeat step 1 until all three points are close to each other.
Example steps:
Rosenbroc Banana Function
Himmelblau Function
Matlab® Subroutines
• Nelder-Mead Subroutine in Matlab ® :
x = fminsearch(fun,x0)
• x is a vector that represents the optimum point of the objective function
• ‘fun’ is a subroutine of the objective function
• x0 is a vector that represents the starting point of the search
Example:
Determine the minimum point of the function:
f (x) = x12 − x1 x2 + 4 x1 + x22 − x2
function simpleks1;

x_star = fminsearch(@fun,[0;0])

function z = fun(x)
z = x(1).^2 - x(1)*x(2) - 4*x(1) + x(2).^2 - x(2);
Example:
Determine the radius and height of the cylinder that give the minimum surface
area of ​the cylinder for a given volume.

Surface area of ​the cylinder:

𝐹 𝑥 = 2𝜋𝑟 2 + 2𝜋𝑟ℎ

Cylinder volume limitations:

𝜋𝑟 2 ℎ = 𝑉

Equation manipulation:
2
𝑉
𝐹 𝑥 = 2𝜋𝑟 + 2𝜋𝑟
𝜋𝑟 2
2𝑉
= 2𝜋𝑟 2 +
𝑟
PLEASE CONTACT ME!

AQSHA
Assistant (Research) Professor
Dept. of Bioenergy Engineering & Chemurgy
Dept. of Chemical Engineering
Institut Teknologi Bandung, Indonesia

Waste to Energy Conversion | Biofuel & Bio-based Product Development |

CCS/BECCS | Biomimicry Catalyst | Thermochemical Conversion

Cell/WA: +62 813 888 70350 | [email protected] | [email protected]

Program Studi Teknik Kimia, Teknik Pangan, Teknik Bioenergi dan Kemurgi, Institut Teknologi Bandung 42