Daniel Abebe Modern Control Systems Lecture Note

having rank 3 hence the system is completely observable.

Let us arrange the dynamic equation as follows, to simplify the problem

      
ẋ3 −1 −4 −4 x3 1
      
ẋ1  =  0 1  x1  + 0 u
      
0
      
ẋ2 1 0 0 x2 0
 
x
h i  3
y = 1 0 0 x1 
 
 
x1
h iT
Thus the measurable state is η1 = x3 and unmeasurable state is η2 = x1 x2 ,
which is straight forward (similar way as above)
..
   
−1 . −4 −4 1
..
      
   
η̇  . . . . . . . . . .  η1 . . .
 1 =    +  u
.
 0 .. 1
   
η̇2 0  η2 0
..
   
1 . 0 0 0
 
i η1
.
h
y = 1 .. 0 0  
η2
h i
A11 = −1 A12 = −4 −4 B1 = 1 C1 = 1
     
0 1 0 0 h i
A21 =   A22 =   B2 =   C2 = 0 0
1 0 0 0
So we have
η̇1 = A11 η1 + A12 η2 + B1 u
η̇2 = A21 η1 + A22 η2 + B2 u
With estimation error η̃2 = η2 − η̂2 , state observer output equation η̂2 =
Lc y + ω = η̂2 = Lc η1 + ω and minimum order state observer dynamics ω̇ =
Jω + N y + M u, then the estimation error dynamics is
η̂˙ 2 = Lc η̇1 + ω̇
η̂˙ 2 = Lc (A21 η1 + A22 η2 + B2 u) + Jω + N η1 + M u
Which implies
η̃˙ 2 = η̇2 − η̂˙ 2 = η̇2 − Lc η̇1 + ω̇
η̃˙ 2 = A21 η1 + A22 η2 + B2 u − Lc (A11 η1 + A12 η2 + B1 u) − Jω − N η1 − M u
η̃˙ 2 = A21 η1 + A22 η2 + B2 u − Lc (A11 η1 + A12 η2 + B1 u) − J(η2 − η̃2 − Lc η1 ) − N η1 − M u
η̃˙ 2 = (A21 − Lc A11 + JLc − N )η1 + (A22 − Lc A12 − J)η2 + J η̃2 (B2 − Lc B1 − M )u

Chapter 4 101
Daniel Abebe Modern Control Systems Lecture Note

η̃2 should tend to zero at t → ∞ irrespective of η1 , η2 , u and J should have

strictly negative real part eigenvalues, hence
A22 − Lc A12 − J = 0 ⇒ J = A22 − Lc A12
A21 − Lc A11 + JLc − N = 0 ⇒ N = A21 − Lc A11 + JLc
B2 − Lc B1 − M = 0 ⇒ M = B2 − Lc B1
The value of the reduced observer matrix gain Lc that place the poles of
estimation reduced order state observer error dynamics at the desired location
is
∥sI − J∥ = s2 + 10s + 25
∥sI − A22 + Lc A12 ∥ = s2 + 10s + 25
     
s 0 1 0 l h i
 −  +  1  −4 −4 = s2 + 10s + 25
0 s 0 0 l2
 
s + 1 − 4l1 −4l1
  = s2 + 10s + 25
−4l2 s − 4l2
s2 + (1 − 4l1 − 4l2 )s − 4l2 = s2 + 10s + 25
 
4
1 − 4l1 − 4l2 = 10 ⇒ l2 = 4 − 4l2 = 25 ⇒ l2 = −6.25 Lc =  
−6.25
then
 
17 16
J = A22 − Lc A12 =  
−25 −25
 
−28
N = A21 − Lc A11 + JLc =  
49
 
−4
M = B2 − Lc B1 =  
6.25
The reduced order state observer output equation is
 
x
  1
.

      0 0 4 .. 1 0 x2 


x̂1 4y + ω1 4x3 + ω1 
.
 
 = = 0 0 −6.25 .. 0 1 x3 
=  
x̂2 −6.25y + ω2 −6.25x3 + ω2 
..
 
 
0 0 1 . 0 0 ω1 
 
ω2
The minimum order state observer dynamics is
        
ω̇1 17 16 ω1 −28 −4
 =   +   x3 +  u
ω̇2 −25 −25 ω2 49 6.25

Chapter 4 102
Daniel Abebe Modern Control Systems Lecture Note

The open loop system dynamics combined with minimum state observer dy-
namics is
..
      
ẋ1 0 1 0 . 0 0 x 0
 1  
..
  
 ẋ2   0 0 1 . 0 0   x2   1 
      
..
      
 ẋ3  −4 −4 −1
      
. 0 0   x3   0 
 =   +  u
      
. . .  . . . . . . . . . . . . . . . . . .  . . . . . .
..
      
0 −28 .
      
 ω̇1   0 17 16   ω1   0 
..
      
ω̇2 0 0 49 . −25 −25 ω2 1
h i
Since matrix B is not changed Kf = 12 12 5 , the closed loop dynamics of
the system with reduced order observer is
    
η̇ A − B1 Kf S1 A12 − B1 Kf S2 0 B1 Kf S2 η
 1   11   1
η̇2  A21 − B2 Kf S1 A22 − B2 Kf S2
    
0 B2 Kf S2  η2 
 =   (4.43)
˙    
η̃1   0 0 0 0  η̃1 
    
η̃˙ 2 0 0 0 A22 − Lc A12 η̃2
h iT
Let S1 = 1 1 1 , with this

A11 − B1 Kf S1 = −30
 T
1 1 1
Let S2 =   we have,
1 1 1
h i
A12 − B1 Kf S2 = −33 −33

h i
B1 Kf S2 = 29 29

 
0
A21 − B2 Kf S1 =  
−1

 
1 0
A22 − B2 Kf S2 =  
0 0

 
0 0
B2 Kf S2 =  
0 0

Chapter 4 103
Daniel Abebe Modern Control Systems Lecture Note
 
17 16
A22 − Lc A12 =  
−25 −25

Finally the close loop system with reduced state observer is

.. .. ..
 
−30 . −33 −33 . 0 . 29 29
 
 ... ... ... ... ... ... ... ...  ...
 
.. .. ..
 
    0 . 1 0 . 0 . 0 0  

η̇1 
.. .. ..
 η1
   −1 . 0 0 . 0 . 0 0 
 
    
η̇2    η2 
 = ... ... ... ... ... ... ... ... ...  
˙    
η̃1    .. .. .. η̃
  1

   0 . 0 0 . 0 . 0 0  
η̃˙ 2 

 η̃
 2
 ... ... ... ... ... ... ... ... ... 
.. .. ..
 
 
 0 . 0 0 . 0 . 17 16 
.. .. ..
 
0 . 0 0 . 0 . −25 −25
h iT
Replacing η1 = x3 (directly measured states)and η2 = x1 x2 (unmeasured
states) we have,
. . ..
 
−30 .. −33 −33 .. 0 . 29 29
    
ẋ3  . . . . . . . . . . . . . . . . . . . . . . . . . . .  x3
 
.. .. ..
    
 ẋ1   0 . 1 0 . 0 . 0 0   x1 
    
.. .. ..
    
 ẋ2   −1 . 0 0 . 0 . 0 0   x2 
    
    
. . . =  . . . . . . . . . . . . . . . . . . . . . . . . . . .  . . .
    
.. .. ..
    
˙    
 x̃3   0 . 0 0 . 0 . 0 0   x̃3 
    
˙    
 x̃1   . . . . . . . . . . . . . . . . . . . . . . . . . . .   x̃1 
. . .
    
x̃˙ 2

 0 .
. 0 0 .
. 0 .
. 17

16  x̃2
.. .. ..
 
0 . 0 0 . 0 . −25 −25

Chapter 4 104
Daniel Abebe Modern Control Systems Lecture Note

6. For the system below design a minimum order state observer.

       
ẋ 0 1 0 0 x
 1     h i  1
ẋ2  =  0 1  x +  1  u y = 1 0 0 x2 
       
0
       
ẋ3 −6 −11 −6 −3 x3
Solution
The observability matrix of the system is
 
1 0 0
 
 
0 1 0
 
0 0 1
which has full rank, implying that the system is completely observable. But
from the system output matrix only one state can be directly measured. The
two state are not available. Hence the reduced order observer has order 2. If
the observer is required to gave a pole at p = −2 ± 2i, then we have
..
   
0 . 1 0 0
   
   
. . . . . . . . . . . . . . .
ẋ = 
 ..
x +  u
  
0 . 0 1 1
..
   
−6 . −11 −6 −3
.
h i
y = 1 .. 0 0

Implying that
   
0 1 h 0
i
A22 =   A11 = 0 A12 = 1 0
A21 =  
−11 −6 −6
 
1 h i
B1 = 0 B2 =   C1 = 1 C2 = 0 0
−3
So J = A22 − LA12 is the error dynamics matrix which need to be stabilized.
   
s −1 l h i
  +  1  1 0 = s2 + 4s + 8
11 s + 6 l2
 
s + l1 −1
  = s2 + 4s + 8
11 + l2 s + 6
s2 + (6 + l1 )s + 6l1 + l2 + 11 = s2 + 4s + 8
6 + l1 = 4 ⇒ l1 = −2 6l1 + l2 + 11 = 8 ⇒ l2 = 9
   
−2 2 1
L= ⇒ J = 
9 −20 −6

Chapter 4 105
Daniel Abebe Modern Control Systems Lecture Note

Finally
 
   
0 13 13
N = A21 + JLc + Lc A11 =   +   =  
−6 4 −2
 
1
M = B2 − Lc B1 =  
−3

The minimum order state observer dynamic equation is

        
ω̇ −2 1 ω 13 1
 1 =    1  +   x1 +   u
ω̇2 −20 −6 ω2 −2 −3

The reduced order state observer output equation is then

 
x
  1
.

1 0 0 .. 0 0 x2 
     
x̂ −2y + ω1 
.
 
 2 =  −2 0 0 .. 1 0 x3 
=  
x̂3 9y + ω2 
..
 
 
9 0 0 . 0 1 ω1 
 
ω2

The open loop dynamics of the system combined with reduced order state
observer dynamics is
..
      
ẋ1 0 1 0 . 0 0 x 0
 1  
..
  
 ẋ2   0 0 1 . 0 0   x2   1 
      
..
      
 ẋ3  −6 −11 −6 . 0   x3  −3
      
0
 =   +  u
      
. . .  . . . . . . . . . . . . . . . . . .  . . .  . . . 
..
      
−2 1   ω1   1 
      
 ω̇1   13 0 0 .
..
      
ω̇2 −2 0 0 . −20 −6 ω2 −3

Chapter 4 106
Daniel Abebe Modern Control Systems Lecture Note

7. For system in chapter 2 on example 2 design a minimum order observer.

Solution
As shown in chapter 2 on example 2 the system has one unobservable mode.
So we cannot design an observer for unobservable mode of the system, but
can design for the observable modes of the system. As we can see from the
output matrix of the decomposed system one state can be directly measured
while one cannot. Thus we design a state observer of order 1. The dynamics
look of the observable mode of the system is
..
   
2 . 1 1
   
ẋ =  . . . . . . . . . x + . . . u
   
..
   
−4 . 0 1
.
h i
y = 1 .. 0 x

From this A11 = 2, A12 = 1, A21 = −4 A22 = 0, B1 = 1, B2 = 1. If the

observer is required to have pole at p = −1, then

s − A22 + lA12 = s + 1
s − 0 + l = s + 1 ⇒ l = 1 ⇒ Lc = 1
J = A22 − lA12 = 0 − 1 = −1
N = A21 + JLc + Lc A11 = −4 − 1 + 2 = −3
M = B2 − Lc B1 = 1 − 1 = 0

The minimum order state observer dynamic equation is

ω̇ = −ω − 3x1

The reduced order state observer output equation is then

 
x
i  1
.
h i h i h
x̂2 = Lc y + ω = 1 0 .. 1 x2 
 
 
ω

The open loop system state equation combined with reduced order state ob-
server equation is
..
      
ẋ1 2 1 . 0 x 1
 1  
..
  
 ẋ2  −4 0
      
. 0   x2   1 
 =   +  u
      
. . .  . . . . . . . . . . . .  . . . . . .
..
      
ω̇ −3 0 . 1 ω 0

Chapter 4 107
Daniel Abebe Modern Control Systems Lecture Note

4.7 Exercises
1. Consider a system
 
 
0.16 2.16 −1 h i
ẋ =   x+   u y= 1 1 x
−0.16 −1.16 1

find it open-loop eigenvalues, corresponding eigenvectors, check stability, con-

trollability, and observability of the system. Determine feedback gain matrix
Kf such that both eigenvalues of the closed-loop system are at −1 and finally
design a minimum order observer for this system that has eigenvalue at −3.

2. Design a full state observer for IPC dynamics on 1.3.4 that has poles at
−1, −1, −1 ± i. use any valid parameter.

3. Design a full state observer for dc motor dynamics having poles at −1 ± 2i.
Use any valid parameters.

4. Design a reduced-order state observer for the dynamics of dc motor on 1.3.3,

where the speed is directly measured and the armature current ia is required
to be estimated. The pole of the observer must be at −1, use any valid
parameters.

5. Design a reduced state observer for IPC dynamics on 1.3.4, where the position
of the cart and position of the pendulum is directly measured and the angular
velocity of the pendulum and linear velocity of the cart is not. The poles of
the observer are at−1 ± 2i.

6. Design a Kalman-Bucy filter for IPC dynamics and DC motor dynamics. Use
any valid parameters.

7. For a dynamic system

     
0 0 1 1 0
ẋ =  x +  u +  ν
1 −1 0 0 1
h i
y = 0 1 x+e

design a Kalman filter.

8. For a system described by

   
−1 −2 −2 2
    h i
ẋ =  0 −1 1  x + 0 u y= 1 1 0 x
   
   
1 0 −1 1

check controllability, observability, stabilizability, and detectability. Finally,

design an observer-based feedback control system, that places all closed-loop
eigenvalues of the system at −2. The observer has all poles at −4.

Chapter 4 108
Daniel Abebe Modern Control Systems Lecture Note

9. Design a full state observer having poles at −1, −1±2i, for a system described
by
   
0 3 0 2
    h i
ẋ = 0 2 1 x +  1  u y = 0 2 1 x
   
   
1 1 0 −1

10. For dynamic system

   
0 1 0 0
   
ẋ =  0 1  x + 1 u
   
0
   
−4 −4 −1 1
h i
y= 0 1 0 x

determine minimum order state observer having pole at −2, −2 ± 2i.

11. For dynamic system

   
0 1 0 1
   
ẋ =  0 1  x + 0 u
   
0
   
−4 −4 −1 1
h i
y= 0 1 0 x

determine minimum order state observer having pole at −2, −2 ± 2i.

12. For dynamic system

   
0 1 0 0
   
ẋ =  0 1  x + 1 u
   
0
   
−4 −4 −1 0
h i
y= 1 0 1 x

determine minimum order state observer having pole at −2, −2 ± 2i.

13. For examples 6 and 7 determine the closed loop dynamics of the system with
full state feedback using reduced order state observer, if the feedback gain Kf
required to place the closed loop poles of the system at s = −1, s = −1 ± 1i

Chapter 4 109
Chapter 5

Introduction to Optimal Control

Optimal control deals with the most celebrated problem of modern control theory,
”determination of best control strategy, selecting optimum control strategy, select-
ing optimum control vector u, that minimize cost function or performance index”.
Consider an LTI system with initial condition x0 :
ẋ = Ax + Bu
y = Cx + Du (5.1)
The objective of optimal control is to determine the control vector u that forces
the behavior of the dynamic system at the required final state by maximizing the
required cost function and at the same time fulfills the physical constraints.
The cost function has a general representation of:
Z tf
t=tf
J = θ[x(t), t] |t=t0 + φ(x(t), u(t), t)dt (5.2)
t0
t=t
The first term θ[x(t), t] |t=t0f
is called terminal error optimization problem (Termi-
Z tf
nal cost function) and the second term φ(x(t), u(t), t)dt is called optimal cost
t0
function over the entire trajectory (the function to be optimized over the entire
trajectory).

5.1 Performance Indices

The cost function (performance index) may be a function that optimizes energy
consumed, optimize the time required to reach the desired state, or minimize the
tracking or terminal value error. Depending on the desired optimization problem
we have five performance indices.

5.1.1 Minimum time control problem

The aim is to determine a control vector that derives the state of the dynamic system
from its initial value x0 to its final value xf in minimum time. The best example
is car racing where finishing time is the only required parameter to be minimized.
The cost function for minimum time is given as
Z tf
J= dt (5.3)
t0

110
Daniel Abebe Modern Control Systems Lecture Note

5.1.2 Minimum terminal error control problem

The aim is to determine a control vector u that minimizes terminal error. Consider
a system with state x we want to reach xd after a certain time t, in terminal control
problem we need to determine control signal u that minimize ξ = xf − xd . The best
example is rocket launching, where the emphasis is put on target only. The cost
function of this control problem is:

J = (xf − xd )T S(xf − xd ) (5.4)

where S is n × n, symmetric positive definite matrix.

5.1.3 Minimum effort/energy control problem

The aim is to determine the minimum amount of energy that is required to derive
the system from x0 to xf . The best example is car braking where stopping time is
required to be achieved with minimum energy. The cost function is:
Z tf
J= uT Ru dt (5.5)
t0

where R is m × m, symmetric positive definite matrix.

5.1.4 Optimal servo-mechanism or tracking problem

This control problem aims to determine a control vector u that minimizes the track-
ing error when driving the system from any initial state x0 to the desired state (final
state). If the desired trajectory is xd over the course from t0 to tf , the control vector
u is required to minimize e = x − xd . The best example is space shuttle tracking
and control, where tracking error, energy consumption, and the difference between
desired terminal point and the shuttle final point are all required to be minimized.
The cost function for this control problem is:
Z tf
J= ((x − xd )T Q(x − xd ))dt (5.6)
t0

where Q is n × n, symmetric positive definite matrix. If we want to minimize the

energy required by the system, in addition to minimizing tracking error the cost
function becomes the sum of tracking error and minimum effort cost function.
Z tf Z tf
T
J= ((x − xd ) Q(x − xd ))dt + uT Ru dt
t t0
Z 0tf
= ((x − xd )T Q(x − xd ) + uT Ru) dt (5.7)
t0

If we add terminal error minimization to the problem in addition to to minimizing

tracking error and energy optimization, the cost function takes general form,
Z tf
T
J = (xf − xd ) S(xf − xd ) + ((x − xd )T Q(x − xd ) + uT Ru) dt (5.8)
t0

Chapter 5 111
Daniel Abebe Modern Control Systems Lecture Note

5.1.5 Optimal regulator problem

This control problem is a special case of optimal tracking problem where the desired
final state is a point in space, particularly an origin xd = 0. The best example
is restoring the equilibrium point of any dynamic system. The tracking optimal
problem cost function is:
Z tf
T
J = xf Sxf + (xT Qx + uT Ru) dt (5.9)
t0

5.2 Elements of Calculus of Variation

5.2.1 Increment of a function and a functional
Consider a function f (t), an increment of function f (t) is defined as

∆f = f (t + ∆t) − f (t) (5.10)

∆f depends both on t and ∆t, hence it is written as ∆f (t, ∆t). In similar way for
a functional J(x(t), t) the incremental of a functional is defined as:

∆J = J((x(t) + δx(t)), t) − J(x(t), t) (5.11)

δx(t) is called variation of the function x(t). Since the increment of a functional
depends on x(t) and δx(t) it is written as J((x(t), δx(t)).

5.2.2 Differential and variation

For a function f (t), the Taylor series expansion about a point t∗ is given as

1 d2 f
   
df
∆f = ∆t + ∆t2 + . . . − f (t∗ ) (5.12)
dt t∗ 2! dt2 t∗
neglecting higher order terms in ∆t,
 
∗ df
∆f = f (t ) + ∆t − f (t∗ )
dt t∗
∆f = f˙(t∗ )∆t = df (5.13)

df is called differential of f at point t∗ , and f˙(t) is the slope of f at t∗ .

Variation of a functional: Expanding a functional J(x(t), t) using Taylor series about
a function x(t), gives us:

∂J 1 ∂ 2J 2
∆J = δx + δx + . . . − J(x(t)) (5.14)
∂x 2! ∂x2
∂J 1 ∂ 2J
is called first variation and is called second variation of a functional.
∂x 2! ∂x2
neglecting higher order terms in δx,
∂J
∆J = δx (5.15)
∂x

Chapter 5 112
Daniel Abebe Modern Control Systems Lecture Note

5.2.3 Optimum of a function and unconstrained functional

For a function to have an optimum value at a point the following two conditions
should be met.
• Necessary condition: For a function to have an optimum value at a point t∗
df
it’s first derivative must vanish at the point(i.e. = 0 at t∗ ).
dt
• Sufficient Condition: For a function to have an optimum value at a point t∗ ,
df > 0 for minimum and df < 0 for maximum.

Similar conditions hold for functional to have optimum value, the necessary con-
dition is that it’s first variation must vanish (δJ = 0) and sufficient condition is
δ 2 J < 0 for local maximum and δ 2 J > 0 for local minimum.

A functional maybe a function of one or more functions, like function of several vari-
able V (x, y, z), in similar manner we have functional which can be J(x(t), y(t), z(t)).
The only difference is in function the variable x, y and z are independent variables
while in functional they are a function of another independent variable.
In control system we deal with functional of the form

J(x(t), ẋ(t), u(t), t) (5.16)

If the system is time invariant the functional will be

J(x(t), ẋ(t), u(t)) (5.17)

Consider a functional φ(x, ẋ) if we take δx and δ ẋ as deviation from x and ẋ respec-
tively, and if we define our cost function as
Z tf
J(x, ẋ) = φ(x, ẋ)dt (5.18)
t0

The increment of J(x, ẋ) will be

Z tf
∆J = (φ(x + δx, ẋ + δ ẋ) − φ(x, ẋ)) dt (5.19)
t0

Using Taylor series expansion:

Z tf
∆J = (φ(x + δx, ẋ + δ ẋ) − φ(x, ẋ)) dt
t0
Z tf  
∂φ ∂φ
= φ(x, ẋ) + δx + δ ẋ + . . . + hot − φ(x, ẋ) dt
t0 ∂φ ∂ ẋ
Z tf  
∂J ∂φ
= δx + δ ẋ + . . . + hot dt (5.20)
t0 ∂φ ∂ ẋ
Lt δJ be linear part of ∆J with respect to δx and δ ẋ, then
Z tf  
∂φ ∂φ
δJ = δx + δ ẋ dt (5.21)
t0 ∂φ ∂ ẋ

Chapter 5 113
Daniel Abebe Modern Control Systems Lecture Note

For the cost function to have optimum value δJ = 0, so

Z tf  
∂φ ∂φ
δx + δ ẋ dt = 0
t0 ∂φ ∂ ẋ
Z tf   Z tf  
∂φ ∂φ
δx dt + δ ẋ dt = 0 (5.22)
t0 ∂φ t0 ∂ ẋ
Z b Z b
∂φ
Using integration by part udv = uv|ba − vdu to solve equation 5.22, let u =
a a ∂x
∂φ
and dv = δxdt for the first term and u = and dv = δ ẋdt for the second term,
∂ ẋ
we have;
Z tf    
∂φ d ∂φ ∂φ t
− δx dt + δx |tf0 = 0 (5.23)
t0 ∂x dt ∂ ẋ ∂ ẋ
     
∂φ d ∂φ ∂φ ∂φ d ∂φ
Which means both − and δx must be zero. −
∂x dt ∂ ẋ ∂ ẋ ∂x dt ∂ ẋ
∂φ
is called Euler-Lagrangian equation while δx is called boundary condition equa-
∂ ẋ
tion.
The sufficient condition is derived the same way from δ 2 J, which lead to:

∂ 2φ
 2 
d ∂ φ ∂ 2φ
− ≥ 0 and ≥0
∂x2 dt ∂x∂ ẋ ∂ ẋ2
for local minimum and
∂ 2φ ∂ 2φ ∂ 2φ
 
d
2
− ≤ 0 and ≤0
∂x dt ∂x∂ ẋ ∂ ẋ2
for local maximum.
With regard to boundary condition there are four cases:
1. x is fixed at t0 and fixed at tf (fixed end trajectory). This means x(t0 ) = C1
and x(tf ) = C2 where C1 and C2 are constants. Then there is no restriction
∂φ
posed on since δx = 0 for both t = t0 and t = tf .
∂ ẋ
∂φ
2. x is fixed at t0 and free at tf . This means x(t0 ) = C1 and = 0 for t = tf
∂ ẋ
∂φ
since we don’t know about δx at tf . Then there is no restriction posed on
∂ ẋ
for t = t0 .
∂φ
3. x is fixed at tf and free at t0 . This means x(tf ) = C2 and = 0 for t = t0
∂ ẋ
∂φ
since we don’t know about δx at t0 . Then there is no restriction posed on
∂ ẋ
for t = tf .
∂φ
4. x is free at tf and free at t0 . This means = 0 for both t = t0 and t = tf
∂ ẋ
since we don’t know about δx at t0 and tf .

Chapter 5 114
Daniel Abebe Modern Control Systems Lecture Note

5.2.4 Optimum of constrained functional

Real-world problems, especially control problems are restricted by different natural
laws or the incapability of technology. The optimization of such kinds of problems
under different constraints is the mainstream of optimal control. For example, the
tracking control of a satellite is restricted by the trajectory selected and the maxi-
mum input allowed. The objective of such kind of control problem is to determine
a control signal u that optimizes a cost function subjected to the constraints. If we
want to optimize a cost function J at the same time required to full fill the state
dynamics of the system, then optimal control problem formulation is to determine
a control signal u that minimize
Z tf
Υ(x, ẋ, u)dt (5.24)
t0

subjected to

ẋ = Ax + Bu (5.25)

Let us define general function which include the constraint and the function to be
optimized as

φ(x, ẋ, u) = Υ(x, ẋ, u) + λT (Ax + Bu − ẋ) (5.26)

The the optimization problem reduced to unconstrained optimization of the form

Z tf
Υ(x, ẋ, u) + λT (Ax + Bu − ẋ) dt


t
Z 0tf
φ(x, ẋ, u)dt (5.27)
t0

By Euler-Lagrangian equation for the function to have an optimum value

 
∂φ d ∂φ
− =0 (5.28)
∂x dt ∂ ẋ
 
∂φ d ∂φ
− =0 (5.29)
∂u dt ∂ u̇
 
∂φ d ∂φ
− =0 (5.30)
∂λT dt ∂ λ̇T

Chapter 5 115
Daniel Abebe Modern Control Systems Lecture Note

5.3 Solution to Optimal Linear Quadratic Regu-

lator Problem
The problem is determining control law u that minimize cost function on equation
5.9 subjected to ẋ = Ax + Bu. Assume we need to bring the final state to zero
(xf = 0).
Which means we need to find u that minimize
Z tf
xT Qx + uT Ru + λT (Ax + Bu − ẋ) dt


J= (5.31)
t0

Matrix Q is a weighing matrix for state error and R is a weighing matrix for input.
The Choice depended on the optimization problem and design specifications. If
state error is required to be minimized matrix Q should be high and if input to
the system is required to be minimized more than the state tracking error matrix
R should be high. In state feedback controller design penalizing input is same as
penalizing state, since the control signal is linear combination of the state of the
system.

The possible starting of guessing the values of the matrices can be one of the following

• Simplest choice: Q = I, R = ρI ⇒ L = x2 + ρu2 then vary ρ to get something

that has good response.

• Diagonal weights

 
q 0 0 0
 1 
 
 0 q2 0 0 
Q=



 0 0 q3 0 
 
0 0 0 q4

1
Choose each qi = , where x̃i is the tolerable error of each state of the system.
x̃2i
• Output weighting
Let z = Hx be the output you want to keep small. Assume (A, H) observable.
Use Q = xT x, R = ρI ⇒ trade off z 2 vs ρu2 .
Where H = xT Qx + uT Ru + λT and λ(t) = P (t)x(t)

• Trial and error (on weights)

Most of the time Q = C T C and the weight of the matrix is adjusted based on the
performance requirement of the system.

Chapter 5 116
Daniel Abebe Modern Control Systems Lecture Note

Using Euler-Lagrangian equations:

 
∂φ d ∂φ
− = 0 ⇒ xT Q + xT QT + λT A − λ̇T = 0 (5.32)
∂x dt ∂ ẋ
 
∂φ d ∂φ
− = 0 ⇒ uT R + uT RT + λT B = 0 (5.33)
∂u dt ∂ u̇
 
∂φ d ∂φ
− = 0 ⇒ ẋ = Ax + Bu (5.34)
∂λT dt ∂ λ̇T
Let λ = P x, where P is positive definite symmetric matrix, So λT = xT P and
Q + QT
λ̇T = ẋT P = (Ax + Bu)T P = (xT AT + uT B T )P , replacing for Q = ,
2
R + RT T
R= , λ and λ̇T , the Euler Lagrangian equation become:
2
xT Q + xT P A + (xT AT + uT B T )P = 0
xT (Q + P A + AT P ) + uT B T P = 0 (5.35)
T T
u R + x PB = 0 (5.36)
ẋ = Ax + Bu (5.37)
If the control law is design based on state feedback u = −Kf x, equation 5.35 becomes
xT (Q + P A + AT P )xT KfT B T P = 0
xT (Q + P A + AT P − KfT B T P ) = 0
Q + P A + AT P − KfT B T P = 0 (5.38)
and from equation 5.36.
− xT KfT R + xT P B = 0
xT (−KfT R + P B) = 0
Kf = R−1 B T P (5.39)
Kf is the optimal feedback gain matrix that stabilize the system ẋ = Ax+Bu under
state feedback control law
u = −Kf x(t) = −R−1 B T P x (5.40)

Replacing for Kf in equation 5.38 result in Riccati equation in which we solve for
matrix P .
Q + P A + AT P − P BR−1 B T P = 0 (5.41)
given that (A, B) is stabilizable, the other method of finding solution for linear
d
quadratic optimal regulator is by letting xT (Q + KfT RKf )x = − (xT P x) after
dt
replacing for u in equation 5.9 and final state xf = 0 (same control objective bring
the system to zero final state). which result in
xT (Q + KfT RKf )x = −ẋT P x − xT P ẋ
xT (Q + KfT RKf )x = −xT (AT − KfT B T )P x − xT P (A − BKf )x
AT P − KfT B T P + P A − P BKf = −(Q + KfT RKf ) (5.42)

Chapter 5 117
Daniel Abebe Modern Control Systems Lecture Note

Finding u that minimize J is same as finding Kf that minimize J, which in-turn

minimizes equation 5.41. Derivative of equation 5.41 with respect to Kf to get Kf
that minimize the equation result in:

(B T P )T + P B − (RKf )T + KfT R = 0
Kf = R−1 B T P (5.43)

Since R = RT are P = P T symmetric positive definite matrices.

So the control signal that minimize the optimal quadratic regulator function is

u = −Kf x = −R−1 B T P x (5.44)

With this control law the close loop system becomes

ẋ = Ax + Bu ⇒ ẋ = (A − BKf )x

P is determined from Riccati equation after replacing for Kf = R−1 B T P in equation

5.42 which is the same as equation 5.41. The Block diagram representation of system
with optimal state feedback controller same as full state feedback controller is shown
in Figure 5.1.

xd + u + x y
- B ʃ C
+

Kf

Figure 5.1: Block diagram representation of LTI system with optimal state feedback
controller

Chapter 5 118
Daniel Abebe Modern Control Systems Lecture Note

5.4 Optimal Steady State Reference Tracking

In control systems, it is often desired to achieve fast convergence of the controlled
output y to a specific nonzero constant set-point value r. The set-point values (xd ,
ud ), xd , represents the desired state of the system and while ud represent the desired
control signal that the output of system require to follow the reference input.

ẋ = Ax + Bu
y = Cx + Du (5.45)

at this point the rate of change of the states variables will be zero and the output is
expected to be equal to the reference values. Hence for a pair (xd ,ud ) equation 5.45
will be

Axd + Bxd = 0
r = Cxd + Dud (5.46)
(5.47)

    
−A B −x 0
   d =   (5.48)
−C D ud r

This corresponds to an LQR criterion of the form

Z tf
J= (ỹ T Qỹ + ũT Rũ) dt (5.49)
t0

where ỹ = y − r and ũ = u − ud . To understand when this is possible, three distinct

cases should be considered:

• Case-I: When the number of inputs m, is strictly smaller than the number
of controlled outputs p, we have an under-actuated system. In this case, the
system of equation 5.48 generally does not have a solution because it presents
more equations than unknowns.

• Case-II: When the number of inputs m is equal to the number of controlled

outputs p, equation 5.48 always has a solution as long as the Rosenbrock’s 1
system matrix
 
sI − A B
P (s) =   (5.50)
−C D

is non-singular for s = 0. This means that s = 0 should not be an invariant

zero of the system and therefore it cannot also be a transmission zero of the
transfer function C(sI − A)−1 B + D.

1
Rosenbrock’s matrix is obtained by regarding the controller output as the only output of the
system.

Chapter 5 119
Daniel Abebe Modern Control Systems Lecture Note

Intuitively, one should expect problems when s = 0 is an invariant zero of the

system because when the state converges to an equilibrium point, the control
input u(t) must converge to a constant. By the zero-blocking property one
should then expect the controlled output y(t) to converge to zero and not to
r.
• Case-III: When the number of inputs m is strictly larger than the number
of controlled outputs p, we have an over-actuated system and equation 5.48
generally has multiple solutions.
When P (0) is full row-rank, i.e., when it has n + p linearly independent rows,
the (n + p) × (n + p) matrix P (0)P (0)T is non-singular and one solution to
equation 5.48 is given by
   
xd 0
  = P (0)T (P (0)P (0)T )−1   (5.51)
ud r

Also in this case, s = 0 should not be an invariant zero of the system because
otherwise P (0) cannot be full rank.
The optimal set-point problem can be reduced to that of optimal regulation by
considering an auxiliary system with state x̃ = x − xd , ũ = u − ud whose dynamics
are:
x̃˙ = A(x − xd ) + B(u − ud )
x̃˙ = Ax̃ + B ũ (5.52)
We can then regard equation 5.49 and equation 5.52 as an optimal regulation prob-
lem for which the optimal solution is given by:
ũ = −Kf x̃ (5.53)
Going back to the original input and state variables u and x, we conclude that the
optimal control for the set-point problem defined by equation 5.45 and equation 5.49
is given by
u = −Kf (x − xd ) + ud (5.54)
Since the solution to equation 5.48 can be written in form
xd = F r (5.55)
ud = N r (5.56)
The values of F and N are determined using equation 5.46, by replacing for xeq = F r
and xd = N r:
Axd + Bud = 0 ⇒ AF r + BN r = 0 ⇒ (AF + BN ) = 0 (5.57)
Cxd + Dud = r ⇒ CF r + DN r = r ⇒ (CF + DN ) = I (5.58)
Solving for F from equation 5.57 and replacing in equation 5.58, the values for
matrices N and F is:
N = (D − CA−1 B)−1 (5.59)
F = −A−1 B(D − CA−1 B)−1 (5.60)

Chapter 5 120
Daniel Abebe Modern Control Systems Lecture Note

ud
N

r xd + u + x y
F + -
Kf + B ʃ C
+

Figure 5.2: Steady state reference tracking

The other method is, at steady state

u = −Kf x + Ko r (5.61)

replacing for u in the state space equation;

ẋ = Ax − Bu ⇒ ẋ = (A − BKf )x + BKo r (5.62)

as time goes to infinity (t → ∞) the state x, goes to the desired state xd and the
input u, tends toward desired input ud , having dynamic equation:

ẋd = Axd + Bud ⇒ ẋd = (A − BKf )xd + BKo r (5.63)

ud = −Kf xd + Ko r (5.64)

the tracking error x̃ = x − xd is then

x̃˙ = (A − BKf )x̃ (5.65)

since ẋd is zero at t → ∞, then the dynamic equation 5.63, can be written as

xd = −(A − BKf )−1 BKo r (5.66)

replacing equation 5.66 in equation 5.64, we have

ud = Kf (A − BKf )−1 BKo r + Ko r

= (Kf (A − BKf )−1 BKo + Ko )r (5.67)

and the output of the dynamic system y, goes to r as t → ∞,

y = Cxd + Dud = r (5.68)

Form equation 5.68, by replacing for ud and xd ,the value of matrix Ko is:

Ko = −(C(A − BKf )−1 B − D((Kf (A − BKf )−1 B + I))−1 (5.69)

In cases where the states of the dynamic system are not directly accessible, a state
observer can be utilized to estimate the state of the system, thereby allowing the
use of the estimated state for control purposes.

Chapter 5 121
Daniel Abebe Modern Control Systems Lecture Note

r u + x y
Ko
+ B ʃ C
- +

Kf

Figure 5.3: Steady state reference tracking

5.5 Optimal State Estimation

The state feedback controller gain matrix design problem involves finding a gain
matrix that can stabilize a given linear system by controlling its states. On the
other hand, the state estimator gain matrix design problem involves finding a gain
matrix that can estimate the states of a given linear system based on its input and
output signals. Interestingly, these two problems obey a duality property. This
means that the solution to one problem can be used to solve the other problem. In
this context, the problem of finding the state feedback controller gain matrix using
a linear quadratic regulator (LQR) can be used to design an optimal state estimator
gain matrix for linear time-invariant systems.

The LQR approach is a widely used method for designing state feedback controllers.
It involves minimizing a quadratic cost function that captures the system’s perfor-
mance and control effort. The optimal state feedback controller gain matrix is
obtained by solving the associated algebraic Riccati equation. Once the controller
gain matrix is obtained, it can be used to stabilize the system and achieve desired
performance.

On the other hand, the problem of designing an optimal state estimator gain matrix
involves minimizing a similar quadratic cost function that captures the difference
between the actual state and the estimated state. The optimal estimator gain ma-
trix is obtained by solving another algebraic Riccati equation. Once the estimator
gain matrix is obtained, it can be used to estimate the system’s state accurately.

The duality property between these two problems arises because the same cost func-
tion is used in both cases, but with different weighting matrices. In other words, the
solution to one problem can be used to solve the other problem by simply swapping
the weighting matrices.

consider a linear time-invariant system described by the following state-space model:

ẋ = Ax + Bu
y = Cx + Du (5.70)

where x is the state vector, u is the control input, y is the output, and A, B, C, and
D are matrices that describe the system’s dynamics.

Chapter 5 122
Daniel Abebe Modern Control Systems Lecture Note

To design an optimal state feedback controller gain matrix using LQR, we first define
the cost function as follows:
Z tf
J= ((x̂ − x)T Q(x̂ − x) + uT Ru) dt (5.71)
t0

where Q and R are weighting matrices that capture the relative importance of the
state and control effort, respectively. The optimal controller gain matrix Kf is
obtained by solving the following algebraic Riccati equation:

Q + P A + AT P − P BR−1 B T P = 0 (5.72)

where P is any symmetric positive definite matrix.

Once Kf = R−1 B T P is obtained, it can be used to stabilize the system and achieve
desired performance.

To design an optimal state estimator gain matrix using LQR, we define a similar
cost function as follows:
Z tf
J= ((x̂ − x)T Q(x̂ − x) + (y T Ry y)) dt (5.73)
t0

where Q and Ry are weighting matrices that capture the relative importance of the
output and state estimation error, respectively. The optimal estimator gain matrix
Lc is obtained by solving the following algebraic Riccati equation:

Q + P A + AT P − P C T Ry−1 CP = 0 (5.74)

where P is any symmetric positive definite matrix.

Once Lc = P C T Ry−1 is obtained, it can be used to estimate the system’s state ac-
curately.

The duality property between the two problems arises from the fact that they share
the same mathematical structure. Specifically, the state feedback controller gain
matrix design problem can be formulated as a quadratic optimization problem us-
ing the linear quadratic regulator (LQR) technique. Similarly, the state estimator
gain matrix design problem can be formulated as a quadratic optimization problem
using the linear quadratic estimator (LQE) technique.

The LQR and LQE techniques are based on minimizing a cost function that mea-
sures the deviation of the system’s states from a desired trajectory and the deviation
of the estimated states from the actual states, respectively. The cost function is for-
mulated as a quadratic function of the system’s states and inputs, and its coefficients
are determined by the system’s dynamics and performance specifications.

The duality property between the two problems implies that the optimal gain ma-
trices obtained from the LQR and LQE techniques are related to each other. Specif-
ically, the optimal state feedback controller gain matrix is equal to the transpose of
the optimal state estimator gain matrix, up to a scaling factor. This means that if
we solve the state feedback controller gain matrix design problem using LQR, we
can obtain the optimal state estimator gain matrix by transposing the solution and
scaling it appropriately.

Chapter 5 123
Daniel Abebe Modern Control Systems Lecture Note

5.6 Examples
1. Determine the incremental of the following functional

Z tf
J= (2x2 (t) + 1)dt (5.75)
t0

Solution

∆J = J(x(t) + δx(t)) − J(x(t))

Z tf Z tf
2
= (2(x(t) + δx(t)) + 1) dt − (2x2 (t) + 1) dt
t t0
Z 0tf
= (2x2 (t) + 4x(t)δx(t) + 2(δx(t))2 + 1 − 2x2 (t) − 1)dt
t
Z 0tf
= (4x(t)δx(t) + 2(δx(t))2 )dt
t0

2. Find the first variation of the following function

Z tf
J= (2x2 (t) + 1) dt
t0

Solution

∂J
δJ = δx(t)
Z∂xtf
∂
= (4x(t)δx(t) + 2(δx(t))2 )δx(t) dt
t ∂x
Z 0tf
= (4δx(t) + 4δx(t))δx(t) dt
t0
Z tf
= 8(δx(t))2 dt
t0

3. Find x that minimize the following cost function

Z π
2
J= (x2 − ẋ2 )dt (5.76)
0

π
with boundary condition x(0) = 0, x( ) = 1
2
Solution:
From the cost function we know that φ(x, ẋ) = x2 − ẋ2 , from Euler Lagrangian
equations
 
∂φ d ∂φ
− =0
∂x dt ∂ ẋ

Chapter 5 124
Daniel Abebe Modern Control Systems Lecture Note

which result in homogeneous second order system

ẍ + x = 0

The characteristics equation of this system is

λ2 + λ = 0 ⇒ λ1 = −i, λ2 = i

the solution to this system has a form

x = C1 eλ1 t + C2 eλ2 t
= C1 e−it + C2 eit
= (C1 + C2 ) cos t + i(C2 − C1 ) sin t

Using boundary condition to solve for the constants C1 and C2 , at t = 0, x = 0

(C1 + C2 ) cos 0 + i(C2 − C1 ) sin 0 = 0

(C1 + C2 ) = 0
π
at t = ,x = 1
2
π π
(C1 + C2 ) cos + i(C2 − C1 ) sin = 1
2 2
(C2 − C1 ) = 1

Solving the two equations simultaneously result in C1 = 0.5 and C2 = −0.5.

Hence the value of x that minimize the cost function is

x(t) = sin(t)

4. Find u that minimize the following cost function

Z 1
1 2
J= u dt (5.77)
0 2

subjected to
   
0 1 0
ẋ =  x +  
0 0 1

with initial and final states x(0) = [1 1]T and x(1) = [0 0]T .

Chapter 5 125
Daniel Abebe Modern Control Systems Lecture Note

Solution:
1
From constrained optimization we know that φ(x, ẋ, u) = u2 +λT (Ax+Bu−
2
ẋ). The control law we determine should minimize
Z 1 
1 2
J= u + λ1 (x2 − ẋ1 ) + λ2 (u − ẋ2 ) dt
0 2
Using the Euler Lagrangian equations
 
∂φ d ∂φ
− = 0 ⇒ 0 + λ̇1 = 0 ⇒ λ̇1 = 0
∂x1 dt ∂ ẋ1

 
∂φ d ∂φ
− = 0 ⇒ x2 − ẋ1 = 0 ⇒ ẋ1 = x2
∂λ1 dt ∂ λ̇1
result in state equation.
 
∂φ d ∂φ
− = 0 ⇒ λ1 + λ̇2 = 0 ⇒ λ̇2 = −λ1
∂x2 dt ∂ ẋ2

 
∂φ d ∂φ
− = 0 ⇒ u − ẋ2 = 0 ⇒ ẋ2 = u
∂λ2 dt ∂ λ̇2
result in state equation.
 
∂φ d ∂φ
− = 0 ⇒ u − λ2 = 0 ⇒ λ2 = −u
∂u dt ∂ u̇

Solving these equations

λ̇1 = 0 ⇒ λ1 = C1 ,
λ̇2 = −λ1 ⇒ λ2 = −C1 t + C2
u(t) = −λ2 ⇒ u(t) = C1 t + C2

from state equations

C1 2
ẋ2 = u ⇒ ẋ2 = C1 t − C2 ⇒ x2 (t) = t − C2 t + C3
2
C1 2 C1 3 C2 2
ẋ1 = x2 ⇒ ẋ1 = t − C2 t + C3 ⇒ x1 (t) = t − t + C3 t + C4
2 6 2
Using the boundary conditions to solve for the constants C1 , C2 , C3 , C4 , we
have the following equations

x2 (0) ⇒ C3 = 1, x1 (0) ⇒ C1 − 2C2 + 2 = 0

x2 (0) ⇒ C3 = 1, x1 (0) ⇒ C1 − 3C2 + 12 = 0

solving this equations C1 = 18 and C2 = 10. Hence the control law that
optimize the constrained cost function is

u(t) = 18t − 10

Chapter 5 126