Scribd
Upload
39 views

Static Optimization

The document discusses optimization techniques for minimizing objective functions subject to constraints. It begins by describing optimization without constraints, where the goal is to minimize a scalar performance index by finding the stationary point where its gradient is zero. It then covers optimization with equality constraints using a Lagrangian approach, where a Hamiltonian function is constructed using Lagrange multipliers to transform the constrained optimization into an unconstrained problem. Examples are provided to demonstrate finding the optimal solution and minimum value for quadratic performance indices subject to linear equality constraints.

Uploaded by

Faiz Ramadhan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
39 views

Static Optimization

The document discusses optimization techniques for minimizing objective functions subject to constraints. It begins by describing optimization without constraints, where the goal is to minimize a scalar performance index by finding the stationary point where its gradient is zero. It then covers optimization with equality constraints using a Lagrangian approach, where a Hamiltonian function is constructed using Lagrange multipliers to transform the constrained optimization into an unconstrained problem. Examples are provided to demonstrate finding the optimal solution and minimum value for quadratic performance indices subject to linear equality constraints.

Uploaded by

Faiz Ramadhan
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 14

Static

Optimization
Trihastuti Agustinah
Electrical Engineering Department
Institut Teknologi Sepuluh Nopember
1.1 Optimization without constraints

A scalar performance index 𝐿(𝑢) is a function of control vector 𝑢 ∈ 𝑅𝑚

Objective: determine the value of u that minimizes 𝐿(𝑢)

– Taylor series expansion of an increment in L:


1
𝑑𝐿 = 𝐿𝑇𝑢 𝑑𝑢 + 𝑑𝑢𝑇 𝐿𝑢𝑢 𝑑𝑢 + 𝑂(3) …(1)
2

– Gradient of L:
𝜕𝐿
𝐿𝑢 ≜
𝜕𝑢
– Hessian matrix:

𝜕2 𝐿
𝐿𝑢𝑢 = → curvature matrix
𝜕𝑢2
1.1 Optimization without constraints

Critical or stationary point: zero increment dL for all increment du

𝐿𝑢 = 0

For the critical point to be a local minimum:

𝐿𝑢𝑢 > 0 → positive definite

– If Luu is negative definite, the critical point is a local maximum

– If Luu is indefinite, the critical point is a saddle point

– If Luu is semidefinite, then the higher terms of the expansion (1) must be
examined to determine the type of critical point
Example 1: Quadratic surfaces
Let 𝑢 ∈ 𝑅2 and

1 𝑇
𝐿 𝑢 = 𝑢 𝑄𝑢 + 𝑆 𝑇 𝑢
2

𝑞11 𝑞12 𝑠1
where 𝑄 = 𝑞 𝑞22 ; 𝑆 = 𝑠2
12

Determine the type of the critical point.

– Critical point:

𝑇 𝑇
1 𝜕𝑢𝑇 𝜕 𝑢 𝑄𝑢 𝜕(𝑆 𝑇 𝑢)𝑇
𝐿𝑢 = 𝑄𝑢 + +
2 𝜕𝑢 𝜕𝑢 𝜕𝑢

1
= 𝑄𝑢 + 𝑄 𝑇 𝑢 + 𝑆 = 𝑄𝑢 + 𝑆 = 0
2
Example 1: Quadratic surfaces

– The optimal control: – The point u* is a minimum if 𝐿𝑢𝑢 > 0


𝑢∗ = −𝑄 −1 𝑆 – Maximum if 𝐿𝑢𝑢 < 0
– Saddle point if 𝑄 < 0
– Hessian matrix:
𝜕𝐿𝑢 – The extremal value of L(u):
𝐿𝑢𝑢 = =𝑄
𝜕𝑢 1 ∗ 𝑇 ∗
𝐿(𝑢∗ ) = 𝑢 𝑄𝑢 + 𝑆 𝑇 𝑢∗
2
1 𝑇
= −𝑄 −1 𝑆 𝑄(−𝑄 −1 𝑆) + 𝑆 𝑇 (−𝑄 −1 𝑆)
2
1 𝑇 −1 1
= 𝑆 𝑄 𝑆 + 𝑆 𝑇 𝑄 −1 𝑆 = − 𝑆 𝑇 𝑄 −1 𝑆
2 2
Example 1: Quadratic surfaces
Let 𝑢 ∈ 𝑅2 and
1 𝑇 – Definiteness test via leading minors:
𝐿 𝑢 = 𝑢 𝑄𝑢 + 𝑆 𝑇 𝑢 𝑚1 = 1 = 1 → positive
2
1 1 0 1 1
𝑄= ;𝑆= 𝑚2 = = 1 → positive
1 2 1 1 2
Determine the type of the critical point. – The point u* is a minimum, since 𝐿𝑢𝑢 > 0
– The minimum value of L(u*):
– The optimal control: 𝐿 𝑢∗ =
1 1 1 1
+ 0 1
1
1 −1
2 −1 0 1 2 1 2 −1 −1
𝑢∗ = −𝑄−1 𝑆 = − =
−1 1 1 −1 1
=−
2
– Hessian matrix:
1 1
𝐿𝑢𝑢 = 𝑄 =
1 2
1.2 Optimization with equality constraints
Let the scalar performance index 𝐿(𝑥, 𝑢) is a function of the control vector
𝑢 ∈ 𝑅𝑚 and a state vector 𝑥 ∈ 𝑅𝑛
– Objective: determine the control vector u that minimizes 𝐿(𝑥, 𝑢) and the
same time satisfies the constraint equation 𝑓 𝑥, 𝑢 = 0
– Necessary conditions:
At a stationary point, dL is equal to zero in the first order
approximation with respect to du when df is zero

𝑑𝐿 = 𝐿𝑇𝑢 𝑑𝑢 + 𝐿𝑇𝑥 𝑑𝑥 = 0 and 𝑑𝑓 = 𝑓𝑢 𝑑𝑢 + 𝑓𝑥 𝑑𝑥 = 0


Hamiltonian approach
Hamiltonian function:
𝐻 𝑥, 𝑢, 𝜆 = 𝐿 𝑥, 𝑢 + 𝜆𝑇 𝑓(𝑥, 𝑢)

where 𝜆 ∈ 𝑅𝑛 is a Lagrange multiplier

– To determine x, u, and , the increments in H:


𝑑𝐻 = 𝐻𝑥𝑇 𝑑𝑥 + 𝐻𝑢𝑇 𝑑𝑢 + 𝐻𝜆𝑇 𝑑𝜆

– Note:

𝜕𝐻
𝐻𝜆 = = 𝑓(𝑥, 𝑢)
𝜕𝜆
Hamiltonian approach
Necessary conditions for a minimum point of 𝐿(𝑥, 𝑢) that also
satisfies the constraint 𝑓 𝑥, 𝑢 = 0

𝜕𝐻
=𝑓=0
𝜕𝜆

𝜕𝐻
= 𝐿𝑥 + 𝑓𝑥𝑇 𝜆 = 0
𝜕𝑥

𝜕𝐻
= 𝐿𝑢 + 𝑓𝑢𝑇 𝜆 = 0
𝜕𝑢

By introducing Lagrange multipliers, the problem of minimizing 𝐿(𝑥, 𝑢)


subject to the constraint 𝑓 𝑥, 𝑢 = 0 is replaced with the problem of
minimizing the Hamiltonian 𝐻 𝑥, 𝑢, 𝜆 without constraints.
Hamiltonian approach
Sufficient conditions to ensure a minimum, the curvature
matrix with constant f equal to zero is positive definite

The curvature matrix can be obtained using


𝑓
𝐿𝑢𝑢 = 𝐻𝑢𝑢 − 𝑓𝑥𝑇 𝑓𝑥−𝑇 𝐻𝑥𝑢 − 𝐻𝑢𝑥 𝑓𝑥−1 𝑓𝑢 + 𝑓𝑢𝑇 𝑓𝑥−𝑇 𝐻𝑥𝑥 𝑓𝑥−1 𝑓𝑢
Example 2: Quadratic surface with linear constraint
The performance index is as given in Example 1:
1 1 1 𝑥 𝑥
𝐿 𝑥, 𝑢 = 𝑥 𝑢 + 0 1
2 1 2 𝑢 𝑢
Renamed the scalar components 𝑢1 , 𝑢2 as 𝑥, 𝑢 respectively.
The constraint:
𝑓 𝑥, 𝑢 = 𝑥 − 3 = 0
Find the stationary point 𝑢∗ and minimum value of 𝐿 𝑥, 𝑢
Example 2: Quadratic surface with linear constraint
The Hamiltonian is
𝑇 1 2
𝐻 𝑥, 𝑢, 𝜆 = 𝐿 𝑥, 𝑢 + 𝜆 𝑓 𝑥, 𝑢 = 𝑥 + 𝑥𝑢 + 𝑢2 + 𝑢 + 𝜆(𝑥 − 3)
2
Necessary conditions for stationary point:
𝜕𝐻
𝐻𝜆 = =𝑥−3=0 …(i)
𝜕𝜆
𝜕𝐻
𝐻𝑥 = =𝑥+𝑢+𝜆 =0 …(ii)
𝜕𝑥
𝜕𝐻
𝐻𝑢 = = 𝑥 + 2𝑢 + 1 = 0 …(iii)
𝜕𝑢
Solving in the order (i), (iii), (ii) yields 𝑥 = 3, 𝑢 = −2 and 𝜆 = −1
The stationary point: (𝑥, 𝑢)∗ = (3, −2)
Example 3: Quadratic performance index with linear constraint

Consider the quadratic performance index – The conditions for a stationary point:
1 𝑇 1 𝑇
𝐿 𝑥, 𝑢 = 𝑥 𝑄𝑥 + 𝑢 𝑅𝑢 𝐻𝜆 = 𝑥 + 𝐵𝑢 + 𝑐 = 0 …(i)
2 2
with linear constraint 𝐻𝑥 = 𝑄𝑥 + 𝜆 = 0 …(ii)
𝑓 𝑥, 𝑢 = 𝑥 + 𝐵𝑢 + 𝑐 = 0
𝐻𝑢 = 𝑅𝑢 + 𝐵𝑇 𝜆 = 0 …(iii)
where 𝑥 ∈ 𝑅𝑛 , 𝑢 ∈ 𝑅𝑚 , 𝑓 ∈ 𝑅𝑛 , 𝜆 ∈ 𝑅𝑛 ; 𝑄, 𝑅, and 𝐵 are
matrices The stationary point u* in term of :
Assume that 𝑄 > 0 and 𝑅 > 0 (both symmetric) 𝑢∗ = −𝑅−1 𝐵𝑇 𝜆
Determine the critical point. According to (ii)
𝜆 = −𝑄𝑥
– The Hamiltonian:
Taking into account (4) results in
𝐻 𝑥, 𝑢, 𝜆 = 𝐿 𝑥, 𝑢 + 𝜆𝑇 𝑓(𝑥, 𝑢)
1 𝑇 1 𝑇 λ = 𝑄𝐵𝑢 + 𝑄𝑐
= 𝑥 𝑄𝑥 + 𝑢 𝑅𝑢 + 𝜆𝑇 (𝑥 + 𝐵𝑢 + 𝑐)
2 2
Thank you

You might also like