Lecture Notes

Optimal and Robust

Control

Herbert Werner

ICS
Institute
of Control
Systems
© 2021 Herbert Werner ([email protected])
Hamburg University of Technology

About this version:

Build date: December 13th , 2021
Build info: commit 41ba89b5 on branch master .
Contents

I LINEAR QUADRATIC OPTIMAL CONTROL 1

Introduction 3

1 The Linear Optimal Regulator Problem 5

2 The Principle of Optimality 7

3 The Hamilton-Jacobi Equation 9

4 The Matrix Riccati Equation 13

5 Solution of the Riccati Equation 19

6 The Infinite-Time Regulator Problem 25

7 Properties of the Optimal Regulator 33

8 The Euler-Lagrange Equations 45

9 State Estimate Feedback 49

10 Loss of Robustness and Loop Transfer Recovery 61

Bibliography - Linear Quadratic Optimal Control 67

II OPTIMAL AND ROBUST CONTROL 69

Introduction 71
iv CONTENTS

11 The Concept of a Generalized Plant - A Review of LQG Control 73

12 Vector Norms and Induced Norms 79

13 The Singular Value Decomposition 85

14 System Norms 89

15 Computing System Norms 95

16 Design Objectives and Sensitivity Functions 101

17 Mixed Sensitivity - A Design Example 115

18 Design Specifications as LMI Constraints 135

19 Controller Design Using Linear Matrix Inequalities 143

20 LMI Approach to Multi-Objective Design 153

21 Design Example: Robust Control of a Power System Stabilizer 157

22 Model Order Reduction 169

Bibliography - Optimal and Robust Control 177

III APPENDICES 179

A Solutions to Exercises - Linear Quadratic Optimal Control 181

B Solutions to Exercises - Optimal and Robust Control 199

 Part I

LINEAR QUADRATIC OPTIMAL

CONTROL
Introduction

In the first chapter, the linear regulator problem with finite time horizon is introduced.
This is a basic problem, and from its solution the solution to other problems - e.g. the
tracking problem - can be derived. In order to solve the regulator problem, we employ the
principle of optimality and the Hamilton-Jacobi equation, which for the linear regulator
problem takes the form of the Riccati equation. In this case, the optimal control law takes
the form of linear state feedback, and it is initially assumed that all states are available for
feedback. In chapter 6, we turn to the important case of infinite time horizon problems,
where the solution is particularly simple. It will be shown that the optimal controller
possesses attractive engineering properties such as good robustness margins.
The assumption that all states are available for feedback is removed in chapter 9 and
replaced by the more realistic assumption that only a linear combination of state variables
- the output variables - are measured. In order to be able to use the results derived in the
first part, the state variables have to be estimated and state feedback has to be replaced
by state estimate feedback. If the measurement of the output variables is corrupted by
noise (which is always the case in real life problems), state estimation involves a trade-off
between a fast decay rate of estimation errors and a low noise sensitivity of the closed-
loop system. The Kalman Filter is shown to be the “optimal solution” to the estimation
problem, optimal in the sense of a probabilistic problem description and minimal variance
of the estimation error.
Unfortunately, the attractive robustness properties of the optimal state feedback solution
derived in the first part may be lost when state feedback is replaced by state estimate
feedback. A technique known as loop transfer recovery is presented, that can be used to
asymptotically “recover” these properties, at the expense of increased noise sensitivity.
Chapter 1

The Linear Optimal Regulator

Problem

Consider the following control problem. The plant to be controlled is represented by a

linear state space model
ẋ(t) = Ax(t) + Bu(t) (1.1)
y(t) = Cx(t). (1.2)
The objective is to keep the state variables at desired values, and we can assume without
loss of generality that the desired state vector is the zero vector (see Exercise 1.1). Assume
that at time t = t0 an external disturbance drives the system into a state x(t0 ) = x0 6= 0,
and that the controller is required to bring the system back to the zero state as quickly
as possible. One possible criterion that expresses how fast an initial state is reduced to
the zero state is the quadratic integral criterion
Z T
xT (t)Qx(t) dt, (1.3)
t0

where Q is a symmetric, positive semi-definite matrix (Q = QT ≥ 0). The term xT (t)Qx(t)

is a measure of the extent to which the state at time t deviates from the zero state, and the
weighting matrix Q determines how much weight is attached to each of the components
of the state. The integral is a measure for the deviation of x from zero during the interval
[t0 , T ].
In many cases the state variables themselves do not have any physical significance, and it
may be desirable to penalize instead the deviation of a vector
z(t) = Dx(t)
whose entries are linear combinations of the state variables and do have some significance
(a possible choice is z(t) = y(t)). It is then easy to see that
Z T
z T (t)Q̃z(t) dt, (1.4)
t0
6

is equivalent to (1.3) with Q = DT Q̃D.

Trying to find an optimal control input u(t) that minimizes (1.3) or (1.4) leads to the
problem that the cost can be made arbitrarily small when the control amplitudes are
allowed to become infinitely large. A trade-off between fast disturbance rejection and
reasonable control effort is possible when a second term, reflecting control energy, is
included in the cost function
Z T
xT (t)Qx(t) + uT (t)Ru(t) dt, R = RT > 0 (1.5)
t0

With a control input that makes (1.5) small, the final state x(T ) will in general not be the
zero state, and with the above formulation of the control objective one has no influence
on how close the final state is to zero. For this reason, it is convenient to add a third term
to the performance index that penalizes the “distance” of the final state from zero
Z T
xT (t)Qx(t) + uT (t)Ru(t) dt + xT (T )Sx(T ), S = S T ≥ 0. (1.6)
t0

The problem of finding a control input u(t), t0 ≤ t ≤ T that minimizes (1.6) under
the constraint (1.1) is called the linear optimal regulator problem. This problem and its
solution will be studied in the following chapters. At this point, one comment is in order.
The optimal solution to the control problem described here is “optimal” only in the sense
of the performance index (1.6) specified by the quantities Q, R and S, and one can view
these quantities as convenient “tuning parameters” which can be used in an interactive
procedure of trial and error to improve the behaviour of the control system.

Exercises

Problem 1.1
Consider a DC motor with shaft angular velocity ω that is controlled by the input voltage
u(t). The system can be described by the scalar differential equation

ω̇(t) = −aω(t) + bu(t),

where a and b are given constants. Consider the problem of stabilizing the angular
velocity at a desired value ω0 . Assume that u0 is the steady state input that keeps
the plant in equilibrium at ω = ω0 . Formulate the problem of finding a control input
that brings the system from an initial state ω(t0 ) = ω1 back to ω0 quickly and with
reasonable control effort, as a linear optimal regulator problem in the form of (1.6). For
this purpose, transform the problem into an equivalent problem with zero equilibrium
state by introducing a new state vector ω̃ = ω − ω0 .
Chapter 2

The Principle of Optimality

In order to solve the linear optimal regulator problem, we will first study a wider class
of optimization problems, and derive a partial differential equation (the Hamilton-Jacobi
equation) which is satisfied by the optimal performance index.
Consider the following optimal control problem. For the system

ẋ = f (x, u, t), x(t0 ) = x0 (2.1)

find the optimal control u∗ (t), t ∈ [t0 , T ], that minimizes

Z T
V (x(t0 ), u(·), t0 ) = l(x(τ ), u(τ ), τ ) dτ + m(x(T )). (2.2)
t0

Here f, l and m are sufficiently smooth functions which can be essentially arbitrary (they
will later be chosen to match the linear optimal regulator problem). The performance
index V depends on initial state x(t0 ) and time t0 and on the control input u(t) for
t0 ≤ t ≤ T .
To simplify notation, we will write u[a, b] to denote the function u(·) restricted to the
interval [a, b], and we sometimes write xT for x(T ). Now we define the optimal performance
index
V ∗ (x(t), t) = min V (x(t), u(·), t). (2.3)
u[t,T ]
∗
Notice that V is independent of the control input, because it is defined to be the cost
incurred by the optimal control input u∗ . Notice also that we have replaced the initial
time t0 with t, because we want to study the optimal cost for different initial times.
To solve the optimal control problem, we will use the principle of optimality:

“An optimal trajectory has the property that at an intermediate point, no

matter how it was reached, the rest of the trajectory must coincide with an
optimal trajectory as computed from this intermediate point as the initial
point”.
8

x1 (T )
x1 (t1 ) x2 (T )
x(t)
x2 (t1 ) x3 (T )

x3 (t1 )

t t1 T

Figure 2.1: Principle of optimality

This may seem almost self-evident, but it is nevertheless a very powerful idea. The
principle of optimality was first formulated in 1697 by James Bernoulli and restated by
Euler in 1744, but then largely ignored until it was revived in the 1930s and formulated
in the above form by Richard Bellman in 1957.
We will now discuss how this idea can be applied to the optimal control problem. First
we cut the time interval [t, T ] in two subintervals, and consider the two control inputs
u[t, t1 ] and u[t1 , T ]. Since u[t, T ] is the concatenation of u[t, t1 ] and u[t1 , T ], minimizing
over u[t, T ] is equivalent to minimizing over u[t, t1 ] and u[t1 , T ]. Thus
Z T
∗
V (x(t), t) = min l(x, u, τ ) dτ + m(xT )
u[t,T ] t
!
Z t1 Z T
= min min l(x, u, τ ) dτ + l(x, u, τ ) dτ + m(xT ) .
u[t,t1 ] u[t1 ,T ] t t1

The inner minimum depends on u[t, t1 ], but the first summand is independent of u[t1 , T ],
so the cost can be rearranged as
!!
Z t1 Z T
∗
V (x(t), t) = min l(x, u, τ ) dτ + min l(x, u, τ ) dτ + m(xT )
u[t,t1 ] t u[t1 ,T ] t1

or Z t1 
∗ ∗
V (x(t), t) = min l(x, u, τ ) dτ + V (x(t1 ), t1 ) . (2.4)
u[t,t1 ] t

The minimization required in (2.4) is illustrated in Fig. 2.1. All trajectories from t1 to
T are optimal. The value of the cost function V (x(t), t) for the whole interval [t, T ] is
obtained by adding the cost of the trajectory form t to t1 , to the optimal cost from t1 to
T . Both depend on u[t, t1 ], which must be chosen to minimize the sum.
Chapter 3

The Hamilton-Jacobi Equation

To solve the optimal control problem posed in chapter 1, the principle of optimality is
applied with setting t1 = t + ∆t and taking the limit as ∆t → 0. This leads to a partial
differential equation for the optimal performance index, the Hamilton-Jacobi equation.
Replacing t1 by t + ∆t in (2.4) gives
!
Z t+∆t
∗ ∗
V (x(t), t) = min l(x, u, τ ) dτ + V (x(t + ∆t), t + ∆t) . (3.1)
u[t,t+∆t] t

The first term on the right hand side can be written as

Z t+∆t
l(x, u, τ ) dτ = ∆t l(x(t + α∆t), u(t + α∆t), t + α∆t)
t

with α some constant between 0 and 1. The second term can be expanded into a Taylor
series

V ∗ (x(t + ∆t), t + ∆t) =

#T
∂V ∗ ∂V ∗
"
∗
V (x(t), t) + (x(t), t) ẋ(t)∆t + (x(t), t)∆t + O(∆t2 ).
∂x ∂t
Substituting these expressions in (3.1) and writing lα for the l(. . .) term at time t + α∆t,
we obtain

V ∗ (x(t), t) =
 #T 
∂V ∗ ∂V ∗
"
∗
min ∆t lα + V (x(t), t) + (x(t), t) ẋ(t)∆t + (x(t), t)∆t + O(∆t2 ) .
u[t,t+∆t] ∂x ∂t

The term V ∗ (x(t), t) can now be eliminated on both sides. Moreover, the fourth summand
is independent of u[t, t+∆t] and can be pulled out of the minimization; thus, after dividing
by ∆t, we obtain
 #T 
∂V ∗ ∂V ∗
"
(x(t), t) = − min lα + f (x, u, t) + O(∆t) .
∂t u[t,t+∆t] ∂x
10

Finally, after taking the limit as ∆t → 0 we have

 #T 
∂V ∗ ∂V ∗
"
(x(t), t) = − min l(x, u, t) + f (x, u, t) . (3.2)
∂t u(t) ∂x

Here, f and l are known functions of their arguments x, u, t, whereas the function V ∗ is
unknown. Equation (3.2) is already a form of the Hamilton-Jacobi equation, even though
it does not yet look like a partial differential equation.
To bring (3.2) in the form of a first order partial differential equation, we define ū to be
the control input that minimizes the right hand side. The minimizing input depends on
the values of x(t), ∂V ∗ /∂x and t, so we write ū(x, ∂V ∗ /∂x, t). Note that the input ū that
minimizes the right hand side of (3.2), minimizes at the same time the performance index
V (x(t), u(·), t), because from the expanded form of V , only terms independent of u have
been removed.
With this definition of ū, (3.2) becomes
#T
∂V ∗ ∂V ∗ ∂V ∗ ∂V ∗
"
(x(t), t) = −l(x, ū(x, , t), t) − f (x, ū(x, , t), t). (3.3)
∂t ∂x ∂x ∂x

This is a first order partial differential equation with independent variable V ∗ and depen-
dent variables x and t, because l, f and ū are known functions of their arguments. From
the definition (2.2) of the performance index follows the boundary condition

V ∗ (x(T ), T ) = m(x(T )). (3.4)

Equations (3.3) and (3.4) together form the Hamilton-Jacobi equation.

Let us summarize what we have achieved so far. In order to solve the optimal control
problem (2.1), (2.2), we used the principle of optimality to derive a partial differential
equation for the optimal cost V ∗ , the Hamilton-Jacobi equation. The Hamilton-Jacobi
equation contains the control input ū that minimizes the right hand side of (3.2), and ū
∗
is a function of x, ∂V
∂x
and t. The optimal control problem can then be solved in three
steps:

∂V ∗
1. find the minimizing ū as a function of x, ∂x
and t

2. substitute this expression in (3.3) and solve the Hamilton-Jacobi equation

for V ∗
∂V ∗ ∗
3. compute the gradient ∂x
and the optimal control u∗ (t) = ū(x(t), ∂V
∂x
, t).

An important observation at this point is that the optimal control input at time t is given
as a function of the state vector; this means that the optimal control law has the form of
3. The Hamilton-Jacobi Equation 11

state feedback. The optimal control law can be obtained via the solution to the Hamilton-
Jacobi equation: if such a solution exists, it is the optimal performance index. A problem
with this approach is that a solution to the Hamilton-Jacobi equation may not exist,
and that not every optimal performance index satisfies the Hamilton-Jacobi equation,
so the equation represents only a sufficient condition for the optimal performance index.
Moreover, it is in general difficult to solve this partial differential equation. The situation
is different however when the above approach is applied to the linear regulator problem:
in that case the Hamilton-Jacobi equation represents a necessary and sufficient condition,
and the solution can be obtained via the solution to a nonlinear differential equation, the
Riccati Differential Equation. This is shown in the following chapter.

Exercises

Problem 3.1
Consider the system
ẋ = u
and the cost function
Z T 1
V (x(0), u(·), 0) = (x2 + u2 + x4 ) dt.
0 2
Write down the Hamilton-Jacobi equation for this problem.
Chapter 4

The Matrix Riccati Equation

In this chapter, we shall use the results of the previous chapter to derive the solution to
the linear optimal regulator problem posed in chapter 1. This will be done in two steps:

1. We show that if V ∗ (x(t), t) exists, it must be of the form xT (t)P (t)x(t), where P (t)
is a positive semidefinite, symmetric matrix

2. Using the above result, we show that if V ∗ (x(t), t) exists, it satisfies the so called
Riccati differential equation.

Finally we outline a proof of existence of V ∗ , and find the optimal control law in terms
of the solution to the Riccati differential equation.
Proof
Step 1
The first step is to show that V ∗ must be of the form xT P x. This is called a quadratic
form, and we will use the following result: the function V ∗ (x) is a quadratic form if and
only if it is continuous in x and

V ∗ (λx) = λ2 V ∗ (x) for all real λ (4.1)

1
V ∗ (x1 ) + V ∗ (x2 ) = (V ∗ (x1 + x2 ) + V ∗ (x1 − x2 )). (4.2)
2
To show that these two conditions are necessary for V ∗ to be a quadratic form is left
as an exercise; the proof of sufficiency is more involved and will not be discussed here.
Now we have to show that V ∗ satisfies (4.1) and (4.2). Let u∗x denote the optimal control
when the initial state is x. Because the plant model is linear and the performance index
quadratic in x, we have
V (λx, λu∗x , t) = λ2 V ∗ (x, t)
and
1
V ∗ (λx, t) = λ2 V (x, u∗λx , t).
λ
14

Because the optimum is minimal, we also have

V ∗ (λx, t) ≤ V (λx, λu∗x , t)

and
1
λ2 V ∗ (x, t) ≤ λ2 V (x, u∗λx , t).
λ
Putting the equalities and inequalities together, we obtain

V ∗ (λx, t) ≤ λ2 V ∗ (x, t)
λ2 V ∗ (x, t) ≤ V ∗ (λx, t).

These two inequalities imply that V ∗ (x, t) satisfies the first condition (4.1). That it also
satisfies the second condition can be shown in a similar manner. Because V ∗ (x, t) is
continuous in x, it follows that it has the form

V ∗ (x(t), t) = xT (t)P (t)x(t)

for some matrix P (t). Positive semidefiniteness follows from the fact that V ∗ (x(t), t)
cannot be negative. To justify the claim that P is symmetric, assume it is not, then one
can replace it by the symmetric matrix 1/2(P + P T ) without altering the value of V ∗ .

Step 2
Next we show that the matrix P (t) that appeared in the last step, satisfies a particular
matrix differential equation, the Riccati differential equation (RDE). We start with the
first form of the Hamilton-Jacobi equation (3.2)
 #T 
∂V ∗ ∂V ∗
"
(x(t), t) = − min l(x, u, t) + f (x, u, t) .
∂t u(t) ∂x

When applied to the linear regulator problem, we have

l(x, u, t) = xT Qx + uT Ru;

and we know that V ∗ = xT P x for some P (t), thus

#T
∂V ∗
"
= 2xT P.
∂x
Furthermore
f (x, u, t) = Ax + Bu,
and we have
∂V ∗
= xT Ṗ x.
∂t
Substitution of these expressions in the Hamilton-Jacobi equation yields

xT Ṗ x = − min
u
(xT Qx + uT Ru + 2xT P Ax + 2xT P Bu). (4.3)
4. The Matrix Riccati Equation 15

The minimizing control input ū can be found by setting the partial derivative of the right
hand side expression in (4.3) to zero. An alternative way - completing the squares - is
shown in Exercise 4.3. The solution is

ū(t) = −R−1 B T P (t)x(t). (4.4)

If this is substituted for ū in (4.3), we obtain

xT Ṗ x = −xT (P A + AT P − P BR−1 B T P + Q)x.

Since this equation holds for all x, we have

− Ṗ = P A + AT P − P BR−1 B T P + Q. (4.5)

This is the celebrated matrix Riccati Equation (or Riccati differential equation, to distin-
guish it from the algebraic Riccati equation discussed in chapter 6). A boundary condition
for (4.5) follows from the boundary condition (3.4) for the Hamilton-Jacobi equation

V ∗ (x(T ), T ) = xT (T )P (T )x(T ) = xT (T )Sx(T )

or
P (T ) = S, (4.6)
where S is the penalty on the final state vector introduced in (1.6).
From the discussion in the previous chapter we know that ū is the optimal control input
at time t, thus solving the Riccati equation (4.5) and substituting the solution P (t) in
(4.4) yields the optimal controller in the form of linear, time-varying state feedback

u∗ (t) = F (t)x(t) (4.7)

F (t) = −R−1 B T P (t). (4.8)

The above results have been derived under the assumption that a solution V ∗ to the
minimization problem exists. We now briefly indicate how one can prove the existence of
V ∗ . From the boundary condition (4.6) we know that there exists a matrix P (t) at t = T
and (by continuity) for t < T when t is sufficiently close to T . The only way a solution
can fail to exist is that there is a finite time T1 < T at which some entries of P become
infinite (in that case T1 is called a finite escape time). However, V ∗ can never be greater
than any V resulting from an arbitrary control input; so we can choose the zero control
input and use the fact that the state vector cannot become unbounded in finite time to
show that elements of P cannot become infinite in finite time.

Exercises

Problem 4.1
16

Consider the problem of minimizing

Z T
V = (x2 + u2 )dt
0

for the first order system

ẋ = u.
Determine the solution P (t) of the matrix Riccati equation using separation of variables
and hence compute the optimal controller for this problem. Plot the time response of
P (t).

Problem 4.2
Define the matrix X as " #
M L
X= T
L N
Show that:

X > 0 is equivalent to N > 0 and M − LN −1 LT > 0

or M > 0 and N − LT M −1 L > 0

where M = M T and N = N T , and L is a matrix with appropriate dimension.

Problem 4.3 Completing squares

a) Consider the function W (x, u) defined as

" #" #
h
T T
i M L x
W (x, u) = x u
LT N u

and assume that N = N T > 0. Show that

min W (x, u) = xT (M − LN −1 LT )x
u

and
ū = arg min W (x, u) = −N −1 LT x.
u

Hint: Find a nonsingular matrix T such that

" #
T M L
T T
LT N

is block diagonal, and use this matrix to write W (x, u) as a sum of two terms of
which one is independent of u and the other one is nonnegative.

b) Use the above result to show that ū in (4.4) minimizes the right hand side of (4.3).
4. The Matrix Riccati Equation 17

Problem 4.4 Riccati Differential Equation

Consider the system
ẋ = Ax + Bu, x(t0 ) = x0 6= 0.
Prove that:
the optimal control input u∗ (t), t0 ≥ t ≥ T that minimizes the performance index V in
(2.3) is
u∗ (t) = −R−1 B T P (t)x(t),
where P (t) = P T (t) ≥ 0 ∀t ≥ 0 satisfies

−Ṗ = P A + AT P − P BR−1 B T P + Q
P (T ) = S.

Hint: Start with an arbitrary u(t) = −R−1 B T P (t)x(t) + v(t), where the choice of P (t)
and v(t) are arbitrary.
Make use of the fact that
Z T d T Z T
(x P x)dt = (ẋT P x + xT Ṗ x + xT P ẋ)dt
t0 dt t0
Chapter 5

Solution of the Riccati Equation

In this chapter, we show how the Riccati differential equation

−Ṗ = P A + AT P − P BR−1 B T P + Q.

with boundary condition

P (T ) = S,
can be solved in cases more general than the simple example at the end of the last chapter.
First, we show that the Riccati equation can be solved by solving the linear system

A −BR−1 B T
" # " #" #
Ẋ X
= (5.1)
Ẏ −Q −AT Y

with boundary condition " # " #

X(T ) I
= , (5.2)
Y (T ) S
where X(t) and Y (t) are square matrices of the same size as A. Having solved the above
system for X and Y , one can compute the solution of the Riccati equation as

P (t) = Y (t)X −1 (t). (5.3)

To prove that (5.3) is indeed the solution to the Riccati equation, we compute its derivative

dP dY X −1 dX −1 dY −1
= =Y + X . (5.4)
dt dt dt dt
Next, differentiate both sides of
X(t)X −1 (t) = I
to get
dX −1 dX −1
= −X −1 X .
dt dt
20

Substituting this and the derivatives of X and Y given in (5.1) in (5.4) gives
dY X −1
= −AT Y X −1 − Y X −1 A − Q + Y X −1 BR−1 B T Y X −1 ,
dt

and it is clear that P = Y X −1 satisfies the Riccati equation (4.5) and (4.6).
When the matrices A, B, Q and R are constant, one can compute the solution to the
Riccati equation in terms of the transition matrix of system (5.1). So far we have made
no statement about these matrices being constant or time-varying; in fact the results we
have seen so far hold for both cases. From now on, we assume that these matrices are
constant.
The matrix
A −BR−1 B T
" #
H=
−Q −AT
is called the Hamiltonian matrix and plays an important role in linear quadratic optimi-
sation. It has the property that if λ is an eigenvalue of H, then so is −λ. To see this,
consider the nonsingular transformation matrix
" #
0 I
J= .
−I 0

It is straightforward to verify that −H = JH T J −1 . We assume that H has no eigenvalues

on the imaginary axis (we will see in chapter 7 under which conditions this is true). Then
there exists a nonsingular transformation U such that
" #
Λ 0
U −1
HU = s
0 Λu
where Λs is a matrix whose eigenvalues are the stable eigenvalues of H, and Λu is a matrix
with the unstable eigenvalues of H. When U is partitioned as
" #
U U12
U = 11 ,
U21 U22
then the columns of " #
U11
U21
span the eigenspace corresponding to the stable eigenvalues of H, whereas the columns of
" #
U12
U22
span the eigenspace corresponding to the unstable eigenvalues. (When the matrices Λs and
Λu are diagonal, the columns of U are the eigenvectors of H.) Applying the transformation
" # " #
X X̃
=U (5.5)
Y Ỹ
5. Solution of the Riccati Equation 21

to (5.1) yields
˙
 " #" #
X̃
 =
Λ s 0 X̃
˙ . (5.6)
Ỹ 0 Λu Ỹ
From the decoupled equations, the solution at time T can be computed in terms of the
solution at time t as

X̃(T ) = eΛs (T −t) X̃(t)

Ỹ (T ) = eΛu (T −t) Ỹ (t).

From (5.5) evaluated at t = T we have

I = U11 X̃(T ) + U12 Ỹ (T )

S = U21 X̃(T ) + U22 Ỹ (T )

which can be used to solve for Ỹ (T ): defining

G = −(U22 − SU12 )−1 (U21 − SU11 )

we obtain
Ỹ (T ) = GX̃(T ).
Then again using (5.5), this time evaluated at t, and substituting the decoupled solutions
of (5.6) yields

X(t) =(U11 + U12 e−Λu (T −t) GeΛs (T −t) )e−Λs (T −t) X̃(T )
Y (t) =(U21 + U22 e−Λu (T −t) GeΛs (T −t) )e−Λs (T −t) X̃(T ).

These expressions can be substituted for X(t) and Y (t) in (5.3) to compute the solution
to the Riccati differential equation

P (t) = Y (t)X −1 (t)

= (U21 + U22 e−Λu (T −t) GeΛs (T −t) )(U11 + U12 e−Λu (T −t) GeΛs (T −t) )−1 . (5.7)

Example 5.1
Solve the problem of the exercise 4.1 using (5.7).
Solution: From
ẋ = u
and Z T
V (x(0), u(·), 0) = (x2 + u2 ) dt.
0
we have the Hamilton matrix " #
0 −1
H= .
−1 0
22

The eigenvalues are λs = −1 and λu = 1, and the transformation matrix is

" #
1 1
U= .
1 −1
With S = 0, we have G = 1, and (5.7) becomes
1 − e−2(T −t)
P (t) =
1 + e−2(T −t)
as before.

Exercises

Problem 5.1
Consider the system shown in Figure 5.1.

r=0 ẋ R x y
B C

R-1 A
G(s)

v λ λ̇
-B T -C T
R

-AT

Figure 5.1: System Block diagram

a) What is the open-loop transfer function from y to v, and how is it related to G(s)?

b) Find a state space model of the closed loop

"
system.
#
x(t)
Hint: choose a state vector of the form: .
λ(t)

Problem 5.2 Consider again the speed control problem of Exercise 1.1, with plant
equation
ω̇(t) = −aω(t) + bu(t)
5. Solution of the Riccati Equation 23

and cost function Z T

(ω 2 (t) + ρu2 (t)) dt + sω 2 (T ).
0

The numerical values are a = 0.5 sec−1 , b = 150 rad/(Vsec2 ) and T = 1 sec. Use (5.7) to
compute the optimal control law and plot the optimal state feedback gain f (t) and the
speed ω(t) in the interval 0 sec ≤ t ≤ 1 sec for the following values of ρ and s

a) ρ = 102 , s = 0

b) ρ = 103 , s = 0

c) ρ = 104 , s = 0

d) ρ = 103 , s = 0.19.
Chapter 6

The Infinite-Time Regulator

Problem

The optimal state feedback gain even for a time-invariant plant is time varying because
the solution P (t) of the Riccati equation is time-varying. Fig. 6.1 shows the solution of
the Riccati equation for the example in Exercise 5.2 with ρ = 1000, s = 0 and different
time horizons T , ranging from T = 1 to T = 10.

0.20
T = 1sec
0.18 T = 2sec
T = 5sec
0.16 T = 10sec
0.14

0.12
P(t)

0.10

0.08

0.06

0.04

0.02

0.00
0 2 4 6 8 10 12 14
Time [sec]

Figure 6.1: Solutions of the Riccati equation for the DC motor example in Exercise 5.2,
with ρ = 1000, s = 0 and different time horizons T

The solution is constant during most of the time interval and changes only when approach-
ing the final time. From a practical point of view, constant feedback gains are much more
26

suitable for implementation in controllers for steady state operation than time-varying
gains; and the fact that the time horizon T is a design parameter that can be chosen by
the designer, suggests to take the limit as T → ∞ to obtain constant gain feedback. From
(5.7) we have
−1
lim P = U21 U11 , (6.1)
T →∞

i.e. as T → ∞ the solution to the Riccati equation becomes time-invariant, which leads
to a constant, steady state optimal state feedback gain

F ∗ = −R−1 B T U21 U11

−1
.

With a constant solution P , we have Ṗ = 0, and the Riccati differential equation turns
into the algebraic Riccati equation

0 = P A + AT P + Q − P BR−1 B T P, (6.2)

which is satisfied by P as given by (6.1). From (5.7) it is also clear that a penalty S on the
final state has no influence on the solution as T → ∞. Changing the problem description
accordingly, we have thus obtained the solution to the infinite-time regulator problem.
The infinite-time regulator problem is also called the steady state regulator problem, it
is: for the system
ẋ(t) = Ax(t) + Bu(t)
and the performance index
Z ∞
V (x0 , u, t0 ) = (xT Qx + uT Ru) dt, (6.3)
0

find the optimal control input u∗ (t) that minimizes V . The steady state solution is

u∗ (t) = −R−1 B T U21 U11

−1
x = F ∗x (6.4)

where U21 and U11 are submatrices of the transformation matrix U that decouples the
Hamiltonian system into a stable and an unstable subsystem.
Compared with the finite-time problem, the steady state regulator problem is easier to
solve, and the solution is easier to implement. But with an infinite time horizon, new
questions concerning the asymptotic behaviour of the solution have to be addressed. The
following example illustrates possible problems.

Example 6.1
Consider the system " # " # " # " # " #
ẋ1 x 0 x10 1
= 1 + , =
ẋ2 x2 1 x20 0
and the performance index
Z ∞
V = (x21 + x22 + u2 ) dt.
0
6. The Infinite-Time Regulator Problem 27

From ẋ1 = x1 , we have x1 = et independent of the control input, so the integrand will
contain the term e2t no matter what control is chosen, and the performance index will be
infinite.

There are two reasons why there exists no solution in the above example. The first one
is the fact that the unstable mode x1 is uncontrollable. The second reason is that this
unstable and uncontrollable mode is reflected in the performance index. Two questions
must be addressed: a) does a solution to the problem exist, and b) is the solution stable.

Existence of a Solution

A solution P to the algebraic Riccati equation for the infinite time regulator problem
exists, if the system (A, B) is controllable. By “existence of a solution”, we mean existence
of a bounded solution P < ∞.
To see that this is true, we note that controllability implies the existence of a state feedback
gain F such that (A + BF ) is stable. Then the control input u(t) = F x(t), which is not
necessarily optimal, stabilizes the system, and the resulting performance index V (x0 , u, t0 )
will be finite. But the optimal performance index cannot be greater than the performance
index resulting from any control input, so we have

V ∗ (x0 ) ≤ V (x0 , u) < ∞.

Recalling that V ∗ (x0 ) = xT0 P x0 , we conclude that xT0 P x0 < ∞; and because this holds
for any x0 , it follows that P is bounded.

Stability

Controllability ensures the existence of a bounded solution P to the algebraic Riccati

equation, and of a state feedback gain that minimizes the performance index. But it does
not imply closed-loop stability, as the following example illustrates.

Example 6.2
Consider the system
ẋ = x + u
and the performance index Z ∞
V = u2 dt.
0
The system is controllable, and the optimal control is u∗ (t) ≡ 0, resulting in V = 0 and
an unstable closed-loop system.

The instability here is due to the fact that the unstable mode is not reflected in the
performance index. A bounded solution P guarantees stability if all modes are reflected
28

in the performance index. To formulate that requirement, we write the weight matrix for
the states as Q = C T C. Thus, the term xT Qx can be interpreted as the cost resulting
from a fictitious output z = Cx. Then stability of the closed-loop system is guaranteed if
(A, C) is observable.
To show this, we need a result from Lyapunov stability theory: the system ẋ = Āx is
stable if there exists a Lyapunov function V = xT P x such that P > 0 and V̇ ≤ 0, and
where V̇ ≡ 0 implies x(t) ≡ 0. Now let Ā denote the optimal closed-loop state matrix

Ā = A + BF ∗ = A − BR−1 B T P.

We first show (A, C) observable ⇒ P > 0 by showing that P ≥ 0 leads to a contradiction.

Assume P ≥ 0, then there exists a nonzero initial state x0 6= 0 such that
Z ∞
xT0 P x0 = (xT C T Cx + uT Ru) dt = 0.
0

But this can only be true if Cx(t) = CeAt x0 ≡ 0 for 0 ≤ t < ∞, and (A, C) observable
6 0.
then implies that x0 = 0, which contradicts the assumption x0 =
Next we prove that V̇ = dtd (xT P x) ≤ 0, and that V̇ ≡ 0 implies x(t) ≡ 0. Observe that
from the algebraic Riccati equation and from the definition of Ā we have

ĀT P + P Ā = −P BR−1 B T P − C T C.

Substituting the right hand side in

V̇ = ẋT P x + xT P ẋ = xT (ĀT P + P Ā)x

gives
V̇ = −xT P BR−1 B T P x − xT C T Cx,
so clearly V̇ ≤ 0, and V̇ ≡ 0 can only be true if Cx(t) ≡ 0, which by observability implies
x0 = 0 and thus x(t) ≡ 0. Invoking the Lyaponov stability result quoted above, this
proves that observability of (A, C) guarantees stability of the optimal closed-loop system.
Controllability and observability are only sufficient, not necessary conditions for the exis-
tence of a stabilizing solution. One can extend the above results and show the following:
a necessary and sufficient condition for the existence of a stabilizing solution is that the
unstable modes are controllable (i.e. the system is stabilizable) and that there are no
unobservable modes on the imaginary axis. If (A, C) has stable but unobservable modes,
the solution P will be positive semidefinite (instead of positive definite).
6. The Infinite-Time Regulator Problem 29

Closed Loop Eigenvalues

The derivation of the solution (5.7) to the Riccati differential equation, which led to
the steady state solution (6.1), was based on decoupling the stable and the unstable
eigenvalues of the Hamiltonian matrix. Using the algebraic Riccati equation, one can show
that the optimal closed-loop eigenvalues are the stable eigenvalues of the Hamiltonian
matrix: apply the similarity transformation
" #
−1 I 0
T HT = H̃, T =
P I

to the Hamiltonian matrix. The result is

Ā −BR−1 B T
" #
H̃ = ,
0 −ĀT

so the eigenvalues of H are the eigenvalues of Ā together with those of −Ā.

For single input, single output systems, this result makes it possible to solve the infi-
nite time regulator problem as a pole assignment problem: form the Hamiltonian and
compute the optimal eigenvalues, and then find the unique state feedback gain that as-
signs the optimal eigenvalues. For multivariable systems, the optimal eigenvalues alone
do not uniquely determine the optimal state feedback gain, and one must consider also
the optimal eigenvectors.
Exercise 6.3 illustrates the procedure for a single input, single output system.

Uniqueness of the Solution

We have shown that controllability and observability guarantee the existence of a stable
solution to the steady state regulator problem, and that the solution can be obtained via
the solution to the algebraic Riccati equation. However, unlike the Riccati differential
equation, the algebraic Riccati equation has in general many solutions, as the following
example illustrates.

Example 6.3
Consider the scalar system
ẋ = 3x + u
and the performance index Z ∞
V = (7x2 + u2 ) dt.
0

The algebraic Riccati equation is

0 = 6P − P 2 + 7 = (P − 7)(P + 1),
30

and there are two solutions

P1 = 7, P2 = −1

corresponding to two different state feedback gains f1 = −7 and f2 = 1. However, only

f1 leads to a stable closed-loop system ẋ = −4x, whereas f2 leads to the unstable system
ẋ = 4x, so here the optimal solution is obviously P1 , which is positive definite.
One can show that under the assumption of controllability and observability, there is
only one stabilizing solution, and that the stabilizing solution is also the only one that
−1
is positive definite. The solution P = U21 U11 is in fact stabilizing when the first half of
the columns of U are the stable eigenvectors of H (or a basis for the stable eigenspace).
On the other hand, when H is 2n × 2n, any combination of n eigenvectors forming the
matrices U21 and U11 leads to a solution P of the algebraic Riccati equation, but the
closed-loop eigenvalues are the eigenvalues corresponding to these eigenvectors, so any
choice other than the n stable eigenvectors results in an unstable closed-loop system.

Exercises

Problem 6.1
For the plant of Problem 5.2, find the optimal state feedback u = f ω that minimizes the
cost function
Z ∞
(ω 2 (t) + ρu2 (t))dt
0

For this purpose, use the MATLAB command

f = −lqr(a, b, 1, ρ)

where ρ can be used as a tuning parameter.

a) Compute the optimal state feedback gain for different values of ρ and compare the
resulting responses of the closed-loop system to an initial speed ω(0) = 1;

R1 R1
b) Plot 0 ω 2 dt versus 0 u2 dt for different values of ρ in the range 0.1 ≤ ρ ≤ 10.
6. The Infinite-Time Regulator Problem 31

Problem 6.2
Consider the control system shown in Figure 6.2.

r e u y = x1
−f1 ẋ = Ax + bu y = cx
-

x2
f2

x3
f3

Figure 6.2: System block diagram

A state space model of the plant is

ẋ = Ax + bu
y = cx

where    
0 1 0 0 h i
   
A=
0 0 1; b=
0 ;
 c= 1 0 0
0 −2 −3 1

(a) The performance index to be minimized is

Z ∞  
V = xT Qx + uT Ru dt
0

Assume that initially

 
q 0 0
 11 
Q =  0 q22 0 

 = I; R=1
0 0 q33

Design an optimal state feedback control law for the above system using MATLAB
by tuning Q and R. Assume that the reference input is zero (or r = 0) for designing
the controller.

(b) Plot the step response of the closed loop system with the resulting controller, and
use the cost weights Q and R to tune the response.
32

Problem 6.3
Consider the system
ẋ(t) = Ax(t) + bu(t)
where " # " #
0 1 0
A= b= .
5 0 3
A state feedback control law u(t) = f x(t) is to be determined that minimizes the perfor-
mance index Z ∞
V = xT (t)Qx(t) + ρu2 (t) dt
0
with weights " #
1 0
Q= , ρ = 1.
0 0
Use the Hamiltonian matrix to compute the optimal closed-loop eigenvalues and the
optimal state feedback gain.
Chapter 7

Properties of the Optimal Regulator

In this chapter, we study properties of infinite-time linear optimal regulator systems from
an engineering point of view. In the previous chapter it was shown how for a given
problem - given in terms of the matrices A, B, Q and R - the optimal state feedback gain
can readily be computed numerically. The matrices A and B represent the plant to be
controlled, and the matrices Q and R express the design objectives. In practice, however,
it is very often not clear what values for Q and R should be chosen to achieve a desired
performance; and it turns out that - rather than being a natural part of the problem
description - these matrices are in fact design parameters, and that the main use of linear
quadratic theory is to provide a convenient parametrization of trial solutions that can be
improved by iteratively changing Q and R.
Provided the system is stablizable and detectable (detectable from a fictitious output, in
the sense discussed in the previous chapter), the existence of a solution is guaranteed for
any choice of R > 0 and Q ≥ 0. Moreover, the optimal closed-loop system is guaranteed
to be stable. Another attractive feature of LQ-optimal design is the fact that there is
no additional complexity involved when working with multivariable systems compared to
single-input single-output systems. We will now explore some further properties of linear
optimal regulators.

Kalman’s Identity

An important result for studying the properties of linear optimal regulators is the following
identity, due to Kalman. Introduce the notation

Φ(jω) = (jωI − A)−1 , ΦH (jω) = [(−jωI − A)−1 ]T ,

then the following equation - referred to as Kalman’s identity - holds true for all ω

(I − B T ΦH (jω)F T )R(I − F Φ(jω)B) = R + B T ΦH (jω)QΦ(jω)B. (7.1)

34

The proof is deferred to the end of this chapter. Because B T ΦH QΦB ≥ 0, an immediate
consequence of Kalman’s identity is the following inequality, also referred to as Kalman’s
inequality
(I − B T ΦH (jω)F T )R(I − F Φ(jω)B) ≥ R ∀ω. (7.2)

For single-input, single-output systems, Kalmans’s identity simplifies to

ρ|1 − f Φ(jω)b|2 = ρ + bT ΦH (jω)QΦ(jω)b ∀ω. (7.3)

If in addition the weight matrix Q has rank one and can be written as Q = cT c (i.e. only
one fictitious output y = cx is being penalized), then the inequality (7.2) becomes

|1 − f Φ(jω)b| ≥ 1 ∀ω. (7.4)

Optimal State Feedback Loop Gain

We shall study some of the implications of inequality (7.4) for single-input, single-output
systems. The importance of this inequality is due to the fact that the term −f Φ(jω)b is
the loop gain of the optimal closed-loop system. To see this, draw the state feedback loop
as shown in Fig. 7.1, where the negative feedback convention used in the classical control
literature is adopted.

(sI − A)−1 B x −F
-

Figure 7.1: Loop gain of the optimal closed-loop system

Because L(jω) = −f Φ(jω)b is the loop gain, the left hand side of (7.4) is equal to the
distance |L(jω) + 1| between a point at frequency ω on the Nyquist plot of the loop gain
L(jω) and the critical point -1 in the complex plane. Thus, Kalman’s inequality states
that the Nyquist plot of the loop gain under optimal state feedback never enters a disc of
radius 1 around the critical point.

Phase and Gain Margin

Provided the plant is controllable and observable, the optimal closed-loop system is guar-
anteed stable. Now assume the nominal loop gain L(jω) is replaced by kL(jω), with k
a static gain. Then from the fact that the Nyquist plot of the loop gain never enters a
disc of radius 1 around the point -1, it is straightforward to show that the number of
encirclements of the critical point cannot change as long as 0.5 < k < ∞. Thus optimal
state feedback guarantees an infinite upper gain margin and a 50% lower gain margin.
7. Properties of the Optimal Regulator 35

To investigate the phase margin, consider the Nyquist plot shown in Fig. 7.2.

Im

Re
−1 60◦

L(jω)

Figure 7.2: Phase margin under optimal state feedback

The phase margin is the amount of negative phase shift that must be introduced into the
loop to make the Nyquist plot touch the critical point. The point of the Nyquist plot
L(jω) that will hit the point -1 is the point at the intersection with a circle of radius 1
around the origin. From the Figure, it is clear that the smallest phase shift that could
make any Nyquist plot touch the critical point is 60o . Thus, optimal state feedback
guarantees a phase margin of at least 60o .
Unfortunately, the above phase and gain margins are guaranteed only when the full state
vector is available and used for feedback. In practice, this is normally not the case, and
when the state variables are estimated and state feedback is replaced by state estimate
feedback, these margins can disappear. Techniques to partially recover these properties
(Loop Transfer Recovery) will be discussed in chapter 10.

Sensitivity and Complementary Sensitivity

The sensitivity function S(jω) is defined as

1
S(jω) = .
1 + L(jω)
Therefore, inequality (7.4) implies that |S(jω)| ≤ 1 for all frequencies, which is an attrac-
tive property as far as disturbance rejection and tracking is concerned.
The complementary sensitivity T (jω) is
L(jω)
T (jω) = 1 − S(jω) = .
1 + L(jω)
36

Observing that this is at the same time the closed-loop transfer function, we find from
Fig. 7.1
T (jω) = −f (jωI − A − bf )−1 b.
For rejection of measurement noise, as well as for robustness against unstructured per-
turbations, the roll-off of T (jω) at high frequencies is of interest. Taking the limit as
ω → ∞, we have
1 T
lim jωT (jω) = −f b = b P b,
ω→∞ ρ
which means that at high frequencies we have
1
|T (jω)| ∼
ω
or a roll-off of 20dB per decade. Therefore, noise performance will be rather poor. How-
ever, whereas replacement of state feedback by state estimate feedback can destroy the
gain and phase margins, using a dynamic state observer will increase the roll-off and thus
improve the noise performance of the closed-loop system.

Spectral Factorisation

It was shown earlier that the optimal closed-loop eigenvalues are the stable eigenvalues
of the Hamiltonian matrix. Kalman’s identity provides a similar characterisation of the
optimal eigenvalues in frequency domain, using spectral factorisation. For single-input,
single-output systems, this characterisation becomes particularly simple.
We first state the result. Consider the system

ẋ = Ax + bu
y = cx.

Here y is again a fictitious output, corresponding to the choice Q = cT c. The performance

index is Z ∞
V = (y 2 + ρu2 ) dt.
0
Introduce the transfer function
b(s)
= c(sI − A)−1 b, (7.5)
a(s)
then the optimal closed-loop eigenvalues are the stable roots of the polynomial
1
p(s) = a(−s)a(s) + b(s)b(−s). (7.6)
ρ

To prove this, we use the fact that for two matrices X ∈ IRm×n and Y ∈ IRn×m , we have

det (Im − XY ) = det (In − Y X)

7. Properties of the Optimal Regulator 37

(this can be shown by considering the determinant of the matrix

" #
Im X
Y In

and making use of the Schur-Complement.) Notice that for column vectors x and y this
implies
det (Im − xy T ) = 1 − y T x.
Now let ac (s) denote the closed-loop characteristic polynomial, which can be factored as
 
ac (s) = det (sI − A − bf ) = det (sI − A) det I − (sI − A)−1 bf . (7.7)

Using the above determinant identity and the notation Φ(s) = (sI − A)−1 , the last factor
in (7.7) is
det (I − Φ(s)bf ) = 1 − f Φ(s)b.
Thus, equation (7.7) can be written as

ac (s) = a(s)(1 − f Φ(s)b)

and we have
ac (s)
1 − f Φ(s)b = .
a(s)
Substituting this and (7.5) in the Kalman identity

ρ(1 − bT ΦH f T )(1 − f Φb) = ρ + bT ΦH cT cΦb

yields
ac (−s)ac (s) b(−s)b(s)
ρ =ρ+
a(−s)a(s) a(−s)a(s)
Rearrangement leads to the result (7.6).
This result provides for single-input, single-output systems a simple way of computing
the optimal state feedback gain: from the given data, form the polynomial p(s) in (7.6),
find the stable roots, and solve a pole-assignment problem.

Eigenvalues on the Imaginary Axis

At this point, we come back to a question that arose when the eigenvalues of the Hamil-
tonian matrix were discussed: under what conditions can we be sure that the polynomial
p(s) has no roots on the imaginary axis?
Assume p(s) has a purely imaginary eigenvalue, i.e. p(jωo ) = 0 for some ωo . Then (7.6)
implies
1
p(jωo ) = |a(jωo )|2 + |b(jωo )|2 = 0,
ρ
38

which can only be true if

a(jωo ) = b(jωo ) = 0.
But then both a(s) and b(s) can be factored as

a(s) = ã(s)(s − jωo )

b(s) = b̃(s)(s − jωo ).

Thus
a(s) ã(s)(s − jωo )
= ,
b(s) b̃(s)(s − jωo )
the transfer function (7.5) must have a pole-zero cancellation. A sufficient condition to
exclude this is that the state space representation (A, b, c) is stabilizable and detectable.
Thus, stabilizability and detectability guarantee that p(s) has no roots on the imaginary
axis (similarly that the Hamiltonian matrix has no eigenvalues on the imaginary axis).
The spectral factorisation result (7.6) provides a characterisation of the optimal closed-
loop poles in terms of the open-loop poles and zeros. We use this result now to investigate
two limiting cases: the case when the cost of control is very high and the case when it is
very low.

High Cost of Control

First we study the optimal solution when the cost of control is infinite, i.e. ρ → ∞. From
1
p(s) = ac (−s)ac (s) = a(−s)a(s) + b(s)b(−s).
ρ
it is clear that in this case we have

ac (−s)ac (s) → a(−s)a(s).

Because all roots of ac (s) must be in the left half plane, this means that

a) Stable open-loop roots remain where they are, and

b) Unstable open-loop roots are reflected about the imaginary axis.

The first point makes intuitively sense: when control is expensive, one should not spend
control effort to move stable poles around. For the second point, it is clear that unstable
roots must be moved into the left half plane, but it is not immediately clear how far. The
spectral factorisation result gives the answer: the optimal strategy is to move an unstable
pole to its mirror image in the left half plane. A simple example serves to illustrate this
result.
7. Properties of the Optimal Regulator 39

Example 7.1
Consider the scalar system

ẋ = ax + u, x(0) = x0
y=x

and the performance index Z ∞

V = (y 2 + ρu2 ) dt.
0

The objective is to find the state feedback gain f such that u = f x minimizes V . The
closed-loop system is ẋ = (a + f )x, and the closed-loop state trajectory is therefore

x(t) = x0 e(a+f )t .

Substituting this and the control law into the cost function gives
∞
2
Z ∞ 1 + ρf 2 2 2(a+f )t 
V = (1 + ρf )x20 e 2(a+f )t
dt = x e  .
0 2(a + f ) 0 
0

Obviously, if a+f is nonnegative (if the closed-loop system is unstable), the cost is infinite.
So we have 
 − 1+ρf 2 x2 , a + f < 0
V = 2(a+f ) 0
 ∞ else
Taking the limit as ρ → ∞, this shows that if the open-loop system is stable (a < 0), the
choice f = 0 leads to a finite cost V = −1/(2a); this is in fact the optimal solution for a
stable plant. If the plant is unstable (a > 0), we need f < a to stabilize the closed-loop
system; this excludes the choice f = 0. To find out how far the closed-loop pole must be
shifted into the left half plane in order to minimize V , we take the derivative
!
dV d 1 + ρf 2
= = 0.
df df 2(a + f )

This leads to
1
f 2 + 2af − =0
ρ
or s
1
f1,2 = −a ± a2 + .
ρ
From the two solutions, f1 = 0 leaves the closed-loop system unstable. When ρ → ∞,
the stabilizing and thus optimal solution is f2 = −2a. The optimal closed-loop system is
therefore
ẋ = (a − 2a)x = −ax,
and the optimal pole location is the mirror image of the unstable pole.
40

Low Cost of Control

Now we consider the optimal solution in the limiting case ρ → 0. When control is very
cheap, it is intuitively clear that the closed-loop poles should be moved far into the left
half plane in order to bring the states quickly to zero. In this case we have
1
ac (−s)ac (s) → b(−s)b(s).
ρ
Let n = deg a(s) and m = deg b(s), then a condition for a system to be physically
realizable is n > m. Among the 2n roots of ac (−s)ac (s) there must be the 2m roots
of b(−s)b(s). These are the finite roots, i.e. roots at finite values of s. The remaining
2(n−m) roots are roots at infinity. To investigate the behaviour of these infinite roots, we
consider equation (7.6) at large values of s. In this case, the polynomials are dominated
by the highest power of s and we can ignore the lower order terms. With a(s) = sn +
an−1 sn−1 + . . . a0 and b(s) = bm sm + bm−1 sm−1 + . . . b0 , we can thus use the approximation

a(−s)a(s) = (−1)n s2n , b(−s)b(s) = (−1)m b2m s2m .

For large values of s, (7.6) can thus be written as

1
(−1)n s2n + (−1)m b2m s2m = 0
ρ
or
1
s2(n−m) = (−1)m−1−n b2m .
ρ
The magnitude of the solution is
! 1
b2m 2(n−m)
|s| = ,
ρ
and the right hand side is the radius of a circle on which the roots are located. Fig. 7.3
shows some examples; this pole configuration is known in network theory and filter design
as Butterworth configuration.
Proof of Kalman’s Identity
We conclude this chapter with the proof of Kalman’s identity

(I − B T ΦH (jω)F T )R(I − F Φ(jω)B) = R + B T ΦH (jω)QΦ(jω)B.

which was claimed to hold for all ω if F = −R−1 B T P and P is the positive (semi-) definite
solution of the algebraic Riccati equation

−P A − AT P + P BR−1 B T P = Q.

associated with this problem. Using F T R = −P B and adding and subtracting P jω, we
have
−P A − AT P + F T RF + P jω − P jω = Q
7. Properties of the Optimal Regulator 41

n−m=1 n−m=2 n−m=3

Figure 7.3: Butterworth configurations

or
P (jωI − A) + (−jωI − A)T P + F T RF = Q.
Substituting Φ = (jωI − A)−1 , and multiplying B T ΦH from left and ΦB from right yields

B T ΦH P B + B T P ΦB + B T ΦH F T RF ΦB = B T ΦH QΦB.

Using again F T R = −P B and adding R on both sides, and introducing Ψ = F ΦB yields

R + B T ΦH QΦB = R − ΨH R − RΨ + ΨH RΨ
= (I − ΨH )R(I − Ψ),

which completes the proof.

Exercises

Problem 7.1
Consider the spring-mass system shown in Figure 7.4.

a) Derive a state space model for the above system (Hint: Use Newton’s laws) with
h iT
x = q q̇ .

b*) A performance index V is given by

Z ∞
V = [z 2 (t) + ρu2 (t)]dt,
0
42

q(t)

u(t)

Figure 7.4: Mass spring system

where u(t) is the control signal, ρ is the control weightingh parameter

i and ρ > 0.
The variable of interest z(t) is defined as z(t) = cx(t), c = 1 0 .
By solving the Riccati equation show that the optimal state feedback gain vector f
as a function of the tuning parameter ρ is
s v s
u
1 u 1
f1 = − 1 + + 1; f2 = −t2( 1 + − 1)
ρ ρ

and the closed-loop eigenvalues are

v s v s
u u
1u 1 1u 1
λ1,2 = − t2( 1 + − 1) ± j t2( 1 + + 1)
2 ρ 2 ρ

c) Discuss the effect of ρ on the feedback gains and the closed-loop eigenvalues?
Hint: Check for ρ → 0 and ρ → ∞.

d) Simulate this system using MATLAB/SIMULINK and plot the locus of the closed
loop pole locations as ρ varies over 0 < ρ < ∞.

Problem 7.2
Consider the system

ẋ = Ax + bu
y = cx

where " # " #

0 1 0
A= , b= , c = [1 0].
5 0 3
The performance index is Z ∞
V = (y 2 + u2 ) dt.
0
7. Properties of the Optimal Regulator 43

Compute the transfer function from u to y and use spectral factorisation to find the
optimal closed-loop eigenvalues and the optimal state feedback gain.

Problem 7.3
Consider a single input-system ẋ = Ax + bu, where
" # " #
0 1 0
A= ; b=
0 0 1
h i
For what values of q is the state feedback gain vector f = −1 −q optimal in the sense
that there exist Q ≥ 0, ρ > 0 such that f minimizes
Z ∞
V = (xT Qx + ρu2 )dt
0

Problem 7.4
Consider a nearly vertical rod with an attached pendulum, as shown in Figure 7.5. The
input u to the system is a torque. The equations of motion are

θ
Ψ
u

Figure 7.5: Vertical rod with a pendulum

3
ψ̈ − ψ − (ψ + θ) = u
2
θ̈ + θ + ψ̈ = 0

a) Derive a state-space model by defining the state variable as

x1 = ψ, x2 = ψ̇, x3 = θ, x4 = θ̇

b) Use the spectral factorization theorem and the root locus technique to plot the
optimal root loci for regulation of the quantity y = ψ + θ, and select an optimal
44

pole configuration such that the dominant time constant is 13 .

Hint: Use the command rltool in MATLAB.
h iT
c) Assume that at time t = 0, the initial condition is x(0) = 1 0 1 0 . Design a
state-feedback controller using MATLAB so that the mass m will swing out with a
time constant of about 13 . Also plot the time response of the system for the given
initial conditions.
Chapter 8

The Euler-Lagrange Equations

The solution to the linear optimal regulator problem was derived in chapter 1 using the
principle of optimality and the Hamilton-Jacobi equation. This approach appears to be
the simplest self-contained way to derive the optimal control law, i.e. without having to
quote results from elsewhere. An alternative approach to the derivation of the optimal con-
trol law is to use the Minimum Principle of Pontryagin together with the Euler-Lagrange
equations. In this chapter, a brief outline of this approach is presented.
As in the derivation of the Hamilton-Jacobi equation, we consider a more general class
of problems with a nonlinear, time-varying plant and cost function. For simplicity we
restrict the discussion to single-input, single-output systems; the model is
ẋ = f (x, u, t), x(0) = x0
and the cost function Z T
V = l(x, u, t) dt + m(x(T )).
0
The objective is to find the control input u∗ (t), 0 ≤ t ≤ T that minimizes V , subject to
the constraint ẋ = f (x, u, t).
The first step is to adjoin the constraint ẋ = f to the cost by rewriting the cost function
as Z T
V = (l + ψ T (f − ẋ)) dt + m.
0
Since f − ẋ = 0, this does not change the cost function. Because f and ẋ are column
vectors, they need to be multiplied by a row vector ψ T . This vector is called a Lagrange
multiplier, it can be freely chosen, for example as a time-varying vector ψ = ψ(t). Next
we introduce the Hamilton function
H = l + ψ T f.
With this definition, the cost function becomes
Z T
V = (H − ψ T ẋ) dt + m.
0
46

The term containing ẋ can be eliminated using integration by parts

Z T T Z T
T T
ψ ẋ dt = ψ x − ψ̇ T x dt,

0 0 0

and substituting this in the cost function yields

T Z T
V = − ψ T x + (H + ψ̇ T x) dt + m(xT ). (8.1)

0 0

To find the control input that minimizes (8.1), we use the following result from calculus
of variations. If u∗ is the optimal control input, and x∗ the optimal state trajectory, then
an infinitely small but arbitrary variation δu will lead to a small variation δx in the state
trajectory. The optimal solution has the property that the variation of the optimal cost
is zero for any δu. Therefore, we apply the condition

δV = 0 ∀δu

to (8.1) in order to find the optimal control input.

From (8.1) we have
 !T  !T 
T ∂H ∂H ∂m
  Z 
ψ T δx ψ T δx δu + ψ̇ T δx dt +

δV = − + + δx + δx .
  
T 0 0 ∂x ∂u ∂x 
T

Because the initial state is given, the second term on the right hand side is zero. Collecting
terms with common factors yields
 !T    !T  
∂m T ∂H ∂H 
 Z
T
+ ψ̇ T  δx +

δV =  − ψ δx +  δu dt (8.2)
∂x  0 ∂x ∂u
T

This equation offers a convenient choice for the Lagrange multiplier ψ(t): if we choose

∂m  ∂H
ψ(T ) =  , ψ̇ = − , (8.3)
∂x T ∂x

then (8.2) simplifies to

∂H Z T
δu dt. δV =
0 ∂u
Optimality requires δV = 0 ∀δu(t), thus a necessary condition for u to be optimal is
∂H
= 0. (8.4)
∂u
Equations (8.3) and (8.4) are known as the Euler-Lagrange equations.
Returning to the linear optimal regulator problem, we replace the functions f (x, u, t) and
l(x, u, t) in the general problem description by
1
f = Ax + bu, l = (xT Qx + ρu2 ).
2
8. The Euler-Lagrange Equations 47

For simplicity we choose m(xT ) = 0. The Hamilton function is then

1
H = l + ψ T f = (xT Qx + ρu2 ) + ψ T (Ax + bu).
2
Equations (8.3) are now

ψ̇ = −Qx − AT ψ, ψ(T ) = 0. (8.5)

For the third equation, we need the derivative of the Hamilton function H
∂H
= ρu + bT ψ = 0.
∂u
Solving for u yields the optimal control input as a linear function of ψ
1
u∗ (t) = − bT ψ(t).
ρ
Substituting this in the plant equation gives
1 T
ẋ = Ax − b b ψ,
ρ
which can be combined with (8.5) into the Hamiltonian system

A −b ρ1 bT
" # " #" #
ẋ x
=
ψ̇ −Q −AT ψ

The optimal control law can be obtained from the solution to the Hamiltonian system
with boundary conditions x(0) = x0 , ψ(T ) = 0, but unlike the Hamilton-Jacobi equation,
the Euler-Lagrange approach does not show directly that the optimal control law has the
form of linear state feedback. This must be shown separately, in fact it can be shown
that as T → ∞, and under the usual controllability and observability assumptions there
exists a constant (but so far unknown) matrix P that satisfies

ψ(t) = P x(t).

Substituting P x for ψ in the control law leads to linear state feedback

1
u∗ (t) = − bT P x(t).
ρ
To find the matrix P , use this control in the plant equation and multiply from the left by
P to get
1
P ẋ = P Ax − P b bT P x.
ρ
Finally, equating this with
P ẋ = ψ̇ = −Qx − AT P x
shows that P is the solution to the algebraic Riccati equation.
Chapter 9

State Estimate Feedback

We now remove the assumption on which the optimal control law discussed in the pre-
vious chapters is based, namely that the full state is available for feedback. In practical
problems, this is rarely the case; instead of the state variables, only estimates can be used
for feedback. In this chapter, we will study the estimation problem and the solution to
the optimal regulator problem when state feedback is replaced by state estimate feedback.
In the next chapter we will investigate some properties of the closed-loop system when
optimal state feedback is replaced by state estimate feedback.
We assume that the estimator has the structure of an identity observer as shown in Fig.
9.1.

u 1 x y
B C
s

- r=0
A

F Fe

u 1 x̂ ŷ
B C
s

Figure 9.1: State estimate feedback

50

Subtracting the estimator equation

x̂˙ = Ax̂ + Bu − Fe C(x − x̂)

from the plant equation

ẋ = Ax + Bu
gives
˙ = (A + Fe C).
The state feedback gain F and the estimator gain Fe can be designed to place the closed-
loop plant eigenvalues (the eigenvalues of A + BF ) and the estimator eigenvalues (the
eigenvalues of A + Fe C) independently in desired locations. The estimator eigenvalues
should be faster than the closed-loop plant eigenvalues, but an upper limit on the estimator
bandwidth is determined by the level of measurement noise present in the output channels:
increasing the estimator bandwidth leads to increased amplification of high frequency
noise.
Whereas in the previous chapters the regulator problem was studied in a purely deter-
ministic framework, the study of the estimation problem makes it necessary to take noise
effects into account, and a stochastic problem formulation is more appropriate.

Stochastic Plant Model

We now extend the plant model to include the effect of stochastic disturbances. Consider
the system

ẋ(t) = Ax(t) + Bu(t) + w(t)

y(t) = Cx(t) + v(t), (9.1)

where w(t) is process noise and v(t) is measurement noise. Both noise processes are
assumed to be white, zero mean, Gaussian and uncorrelated, and satisfy

E[w(t)wT (t + τ )] = Qe δ(τ ), E[v(t)v T (t + τ )] = Re δ(τ ).

A block diagram of the closed-loop with the stochastic plant model is shown in Fig. 9.2.

Optimal Estimation Problem

Let ε(t) = x(t) − x̂(t) denote the estimation error, and let q be a weighting vector such
that q T ε is a linear combination of the errors where the elements of q reflect the relative
weight placed on the estimation of individual state variables. With the above definitions,
we pose the estimation problem as follows. Given the estimator structure in Fig. 9.1, find
the estimator gain Fe that minimizes the stochastic cost function

Ve = lim E[εT qq T ε]. (9.2)

t→∞
9. State Estimate Feedback 51

w
v

u y
Plant

- r=0

x̂
F Observer

Figure 9.2: Closed-loop system with stochastic plant model

Here the limit is taken as t → ∞ because we are interested in the steady state.
Subtracting the estimator equation

x̂˙ = Ax̂ + Bu + Fe (ŷ − y)

from the state equation in (9.1) yields

ε̇ = (A + Fe C)ε + ξ,

where ξ is the white, zero mean, Gaussian noise process

ξ = w + Fe v, E[ξ(t)ξ T (t + τ )] = (Qe + Fe Re FeT )δ(τ ).

Stochastic-Deterministic Dualism

The estimation problem as formulated above is a stochastic optimization problem. We

will not solve it directly, but we will show that this problem has the same structure as a
deterministic problem the solution of which we know: the linear regulator problem. When
a stochastic problem has the same structure and can be solved by the same methods as
an equivalent deterministic problem, we speak of a stochastic-deterministic dualism.
To establish this dualism, we first consider a simple deterministic and a simple stochastic
problem.

Deterministic Problem

Given the autonomous plant model

ẋ = Ax, x(0) = x0 ,
52

assume we are interested in the value of the cost function

Z ∞
V = xT Qx dt, Q = QT > 0.
0

One can show that the value of the cost function is

V = xT0 P x0 (9.3)

where P is the positive definite matrix satisfying

P A + AT P + Q = 0. (9.4)

This can be shown in the same way as the following result, see Exercise 9.1.

Stochastic Problem

Now consider the process

ẋ = Ax + ξ, (9.5)
where ξ is a white, zero mean, Gaussian noise process that satisfies E[ξ(t)ξ T (t + τ )] =
Qs δ(τ ), and assume we want to find the value of the stochastic cost function

Vs = lim E[xT qq T x],

t→∞

where q is a weighting vector as before.

We will show that the solution to this problem is

Vs = q T Ps q, (9.6)

where Ps is a positive definite matrix satisfying

Ps AT + APs + Qs = 0. (9.7)

Notice that this problem and its solution have the same structure as the deterministic
problem; A is replaced by AT , x0 by q, Q by Qs and P by Ps .
To prove that (9.6) is indeed the value of the cost, we start with the solution to the state
equation (9.5) Z t
x(t) = eAt x0 + eA(t−τ ) ξ(τ ) dτ.
0
Using the properties of ξ, we have

Tt
Z t Z t T (t−τ

E[xxT ] = E eAt x0 xT0 eA + eA(t−τ1 ) ξ(τ1 )ξ T (τ2 )eA 2)
dτ1 dτ2
0 0
Z t Z t
AT t T (t−τ
= eAt x0 xT0 e + eA(t−τ1 ) Qs δ(τ1 − τ2 )eA 2)
dτ1 dτ2
0 0
Tt
Z t T (t−τ )
= eAt x0 xT0 eA + eA(t−τ ) Qs eA dτ
0
T
Z t T
At
=e x0 xT0 eA t + eAτ Qs eA τ dτ.
0
9. State Estimate Feedback 53

Assuming that A is stable and taking the limit as t → ∞

Z ∞ T
T
lim E[xx ] = 0 + eAτ Qs eA τ dτ
t→∞ 0

where in the last term the variable of integration has been changed. Multiplying the
above equation from left and right by q T and q respectively yields
Z ∞ T
T T T T T
lim E[q xx q] = lim E[x qq x] = q eAτ Qs eA τ dτ q,
t→∞ t→∞ 0

because q T xxT q is scalar. The left hand side is the stochastic cost function Vs , and the
above equation can be written as
V s = q T Ps q
when the positive definite matrix Ps is defined as
Z ∞ T
Ps = eAτ Qs eA τ dτ.
0

Thus we have shown that the value of the stochastic cost is given by (9.6). It remains to
prove that Ps is the solution to (9.7). Substituting the above expression for Ps in (9.7)
yields Z ∞
T T
(eAτ Qs eA τ AT + AeAτ Qs eA τ ) dτ + Qs = 0.
0
But this is the same as Z ∞ d  Aτ T

e Qs eA τ dτ + Qs = 0
0 dτ
or
T ∞ 
eAτ Qs eA τ  + Qs = 0 − Qs + Qs = 0,
0
which proves that Ps satisfies (9.7). (That Ps is the unique solution follows from the fact
that a matrix equation M P + P N + Q = 0 has a unique solution P if and only if M and
−N have no common eigenvalues.)

Solution to the Optimal Estimation Problem

We return to the optimal estimation problem

min lim E[εT qq T ε]

Fe t→∞

where the estimation error is governed by

ε̇ = (A + Fe C)ε + ξ, E[ξ(t)ξ T (t + τ )] = (Qe + Fe Re FeT )δ(τ ).

Applying the above result with the replacements

A → A + Fe C, Qs → Qe + Fe Re FeT ,
54

the value of the stochastic cost function Ve is

Ve = q T Pe q,

where Pe is the positive definite solution to

Pe (A + Fe C)T + (A + Fe C)Pe + Qe + Fe Re FeT = 0.

We will find the optimal estimator gain Fe by comparing the above with the solution to
the linear regulator problem.

Linear Regulator Problem

Because we know that the optimal solution to the linear regulator problem has the form
of linear state feedback u = F x, we can pose the linear regulator problem in the following
form: given the closed-loop system

ẋ = (A + BF )x, x(0) = x0 ,

find the state feedback gain F that minimizes

Z ∞ Z ∞
V = (xT Qx + uT Ru) dt = xT (Q + F T RF )x dt.
0 0

The optimal solution was shown to be

F = −R−1 B T P,

where P is the positive semidefinite solution to the algebraic Riccati equation

P A + AT P − P BR−1 B T P + Q = 0,

and the value of the optimal cost function is

V = xT0 P x0 .

With the substitution P B = −F T R, it is straightforward to show that the Riccati equa-

tion can also be written as

P (A + BF ) + (A + BF )T P + Q + F T RF = 0.

Comparing this with the optimal estimation problem, we find that both problems are
equivalent with the replacements

Q → Qe , R → Re , A → AT , B → CT ,
x0 → q, F → FeT , P → Pe .
9. State Estimate Feedback 55

Thus, we can establish the following result by duality, i.e. by making the above replace-
ments in the solution to the regulator problem: the optimal estimator gain is

Fe = −Pe C T Re−1 , (9.8)

where Pe is the positive definite solution to

Pe AT + APe − Pe C T Re−1 CPe + Qe = 0. (9.9)

This equation is known as the filter algebraic Riccati equation (FARE).

The following example illustrates the result.

Example 9.1
Consider the first order system

ẋ = ax + w
y = x + v,

where w and v are white, zero mean, Gaussian noise processes that satisfy

E[w(t)w(t + τ )] = qe δ(τ ), E[v(t)v(t + τ )] = re δ(τ ).

For the estimator

x̂˙ = ax̂ + fe (x̂ − y) = (a + fe )x̂ − fe y,
find the optimal estimator gain fe .
Solution: the FARE is
1 2
2ape − p + qe = 0
re e
or
p2e − 2are pe − re qe = 0
with positive solution q
pe = are + a2 re2 + re qe .
The optimal estimator gain is therefore
s
pe qe
fe = − = −a − a2 + .
re re
Substituting the optimal gain in the estimator equation yields
s
qe
x̂˙ = − a2 + x̂ − fe y.
re

Note that the solution depends only on the ratio qe /re of the intensities of process and
measurement noise. It is instructive to consider the two limiting cases qe /re → 0 and
qe /re → ∞.
56

The case qe /re → 0 corresponds to a situation with very large measurement noise. In this
case, the estimator equation becomes

x̂˙ = −|a|x̂ − fe y,

because the square root is positive no matter whether a is positive or negative. For the
estimator gain, we have to distinguish between these cases, thus
(
0, a < 0
fe =
−2a, a ≥ 0

If measurement noise is very large and the plant is stable, the optimal solution is to use
no measurements at all. If the plant is unstable, this is not possible because it would
result in an unstable estimator. In this case the optimal solution is to place the estimator
pole at the stable mirror image of the unstable plant pole; this is the dual to the result
on high cost on control for the regulator problem.
The assumption qe /re → ∞ corresponds to a situation with no measurement noise. In
this case, we have fe → −∞, i.e. the optimal solution is to make the estimator dynamics
infinitely fast.

A final comment on the noise matrices Qe and Re is in order. These parameters play the
same role as the weight matrices Q and R in the dual regulator problem. Just as in real
life problems these weight matrices are not uniquely defined by the problem description,
but are rather used as convenient tuning parameters, the noise intensities are in many
cases not precisely known, and the matrices Qe and Re are used as tuning parameters in
the same way. In the above example it is easy to see that the parameter qe /re can be
used to tune the bandwidth of the estimator; if the bandwidth is too low, then qe /re can
be increased and vice versa.

LQG Control

Now that we have established the solution to the optimal estimation problem, we turn to
the problem of finding an optimal controller for the plant (9.1) when the full state is not
available for feedback and state estimate feedback must be used. First we must clarify
what we mean by optimal in this case. The closed-loop system in Fig. 9.2 is redrawn
in Fig. 9.3 in a form that clearly shows which parts of the loop form the controller. We
let K(s) denote the transfer function from the state error r − y to u, and define the
optimal control problem as the problem of finding the controller K(s) that minimizes the
stochastic cost function
VLQG = lim E[xT Qx + uT Ru]. (9.10)
t→∞

This problem is known as the LQG (Linear Quadratic Gaussian) control problem. The
LQG problem represents all aspects of the optimal control problem we have discussed
9. State Estimate Feedback 57

so far: the cost function has the structure of the cost for the linear regulator problem
with two terms penalizing state error and control energy respectively; it reflects the fact
that the state is not available and state estimate feedback must be used instead, and the
presence of process and measurement noise is taken into account by the stochastic plant
model (9.1).

w
K(s) v
r=0
x̂ u y
- Observer F Plant

Figure 9.3: Closed-loop system with LQG controller K(s)

The Separation Principle

The solution to the LQG problem is based on the separation principle: the controller
K(s) that minimizes (9.10) is obtained by combining the optimal state feedback gain F
and the optimal estimator gain Fe . Even though it may seem natural that the optimal
controller should be the combination of optimal state feedback with optimal estimation,
it is non-trivial to prove this fact. Here, we give only a brief outline of the proof.
Proof of Separation Principle
The proof is based on the following two properties of the optimal estimator:

1) State estimate and estimation error are uncorrelated, i.e. E[x̂εT ] = 0, and

2) the output error y − C x̂ is white and zero mean, its covariance is

E[(y − C x̂)(y − C x̂)T ] = Re (this can be proved by using Kalman’s identity).

Because x = x̂ + ε, we can use the first property above and rewrite the LQG cost as

VLQG = lim E[x̂T Qx̂ + uT Ru + εT Qε].

t→∞

Thus, the LQG cost can be split into two terms that can be minimized independently: a
term that represents the cost of estimation

Ve = lim E[εT Qε],

t→∞
58

and a term that represents the cost of control

Vc = lim E[x̂T Qx̂ + uT Ru].

t→∞

The cost Ve is independent of the state feedback gain F , it depends only on the estimator
gain Fe . The estimator gain that minimizes Ve is clearly the solution to the optimal
estimation problem Fe = −Pe C T Re−1 , where Pe is the solution to the FARE. The cost
Vc is independent of the estimator gain and depends only on the state feedback gain
F ; from property (2) of the optimal estimator we know that the state estimate x̂ is
the state x perturbed by an additive white, zero mean noise process. It can be shown
that under this condition, the solution to the problem of minimizing Vc is the same as
that to minimizing the deterministic cost function for the linear regulator problem: it is
F = −R−1 B T P , where P is the solution to the ARE. When a stochastic problem can
be solved by replacing the stochastic variables with deterministic ones, we say that the
certainty equivalence principle holds.
In summary, the controller that minimizes the LQG cost (the LQG controller) is obtained
by solving two Riccati equations: the ARE to find the optimal state feedback gain, and
the FARE to find the optimal estimator gain.

Exercises

Problem 9.1
Given the autonomous plant model

ẋ = Ax, A is stable x(0) = x0

and the cost function Z ∞

V = xT Qx dt.
0

Prove that the value of the cost function is given by

V = xT0 P x0 , (9.11)

where P is the positive definite matrix satisfying

P A + AT P + Q = 0. (9.12)

(That is, show that (9.3) and (9.4) are indeed the solution to the deterministic problem.)

Problem 9.2 LQG Control

9. State Estimate Feedback 59

Consider the system

ẋ(t) = x(t) + u(t) + w(t),

y(t) = x(t) + v(t)

where the process noise w(t) and the measurement noise v(t) are white, zero mean, Gaus-
sian and uncorrelated, and satisfy

E[w(t)w(t + τ )] = qe δ(τ ) = 1δ(τ ), E[v(t)v(t + τ )] = re δ(τ ).

A controller K(s) that minimizes

VLQG = lim E[x2 (t) + ρu2 (t)].

t→∞

can be constructed from an optimal state feedback gain f and an optimal estimator gain
fe .

a) Draw the block diagram of the controller K(s) and give its transfer function.

b) Find the optimal estimator gain fe as a function of re .

c) The optimal state feedback gain was found to be f = −3. Find the corresponding
value of ρ.

d) When re = 0.125, compare the error dynamics with the plant dynamics. Replace
re by a value r̃e such that the error dynamics is at least five times faster than the
plant.

e) The LQG cost can be written as

VLQG = lim E[x̂2 (t) + ρu2 (t) + ε2 (t)]

t→∞

where ε̇(t) = αe ε(t) + ξ(t). With your result from (d), what is αe ? Express the
process ξ(t) in terms of w(t) and v(t) and compute E[ξ(t)ξ(t + τ )].

Problem 9.3
Consider again the LQG loop of Problem 9.2. What is the transfer function from d to y
with the designed controller (see Figure 9.4)? How does that transfer function change if
the disturbance d enters directly at plant input without being feedback to the estimator
(see Figure 9.5)?
60

w
d v

x̂ u y
- Estimator F Plant

Figure 9.4: LQG Loop

w
d v

x̂ u y
- Estimator F Plant

Figure 9.5: Disturbance at plant input

Chapter 10

Loss of Robustness and Loop

Transfer Recovery

In chapter 7, we studied the robustness properties of the closed-loop system with optimal
state feedback. It turned out that the optimal closed-loop system has attractive robustness
margins. In this chapter we will review these results for the state estimate feedback case.
We first present an example which shows that the robustness margins for optimal state
feedback control can completely disappear when state estimate feedback is used; and
we will introduce a design technique that allows to partially recover these properties by
manipulating the optimal estimator.
The following well known example was published in 1978 in the IEEE Transactions on
Automatic Control by John Doyle.

Example 10.1
Consider an LQG design problem with the following data. The system matrices are
" # " #
1 1 0 h i
A= , b= , c= 1 0 ,
0 1 1
the cost weights are " #
1 1
Q=γ , ρ = 1,
1 1
and the noise intensities " #
1 1
Qe = σ , re = 1.
1 1
The optimal state feedback and estimator gains can be computed as
h i √
f =α 1 1 , α = −2 − 4 + γ
√
" #
1
fe = β , β = −2 − 4+σ
1
62

To investigate the robustness of the optimal closed-loop system, we introduce a gain

perturbation k. Assume the plant equation is

ẋ = Ax + kbu + w,

where k = 1 represents the nominal model for which the controller has been designed,
and values k 6= 1 represent perturbations. With the closed-loop plant equation

ẋ = Ax + kbf x̂ + w,

and the estimator equation

x̂˙ = (A + fe c)x̂ + bf x̂ − fe cx,

the closed-loop system matrix is

 
1 1 0 0
" #
A kbf  0 1 kα kα 
 
Ā = = .
−fe c A + bf + fe c −β

0 1+β 1 

−β 0 α+β 1+α

A necessary condition for closed-loop stability is that all coefficients of the closed-loop
characteristic polynomial

det(sI − Ā) = s4 + ā3 s3 + ā2 s2 + ā1 s + ā0

are positive. We check only the absolute coefficient ā0 , it is

ā0 = det Ā = 1 + (1 − k)αβ.

A necessary condition for stability is therefore

1
ā0 > 0 ⇒ k <1+ .
αβ

The last inequality shows that there is no infinite gain margin in this case, in fact for
large values of α or β the gain margin can become arbitrarily small.

Loop Transfer Function

The fact - illustrated by the above example (10.1) - that under state estimate feedback
the robustness margins can become arbitrarily small, gave rise to research into methods
to improve the robustness of LQG controllers. Here we present a method known as LQG
Loop Transfer Recovery, which consists in manipulating the optimal estimator in order to
recover the loop transfer function of state feedback.
10. Loss of Robustness and Loop Transfer Recovery 63

1 2 u y
F Plant

x̂ - r=0

Observer

Figure 10.1: Loop break points

It was shown in chapter 7 that optimal state feedback has attractive robustness properties
because the loop gain, or loop transfer function, satisfies Kalman’s identity. To study the
closed-loop properties of LQG control, we first need to find the loop transfer function for
the state estimate feedback case. The closed-loop system is shown in Fig. 10.1. In the
state feedback case, the loop gain was obtained by cutting the loop at plant input. In
Fig. 10.1, two possible loop break points are marked by 1 and 2, and it makes a crucial
difference at which of these points the loop is cut. When cut at point 1, the inputs to
plant and observer are identical. A transfer function describes a system in steady state
and initial states are ignored, therefore in this case we have x̂ ≡ x, and the loop gain is

L1 (s) = F (sI − A)−1 B.

However, it is not the loop gain seen from point 1 that determines the robustness properties
of the control system, because the loop break point is inside the controller. The loop break
point is the point where a disturbance or perturbation is expected to enter the loop, and it
is unlikely that a perturbation is injected inside the controller. For a realistic assessment of
the closed-loop robustness properties, it is necessary to cut the loop outside the controller,
at point 2 just before the plant input.

r=0 K(s)
u y
Plant - x̂ u
Observer F

Figure 10.2: Loop broken at point 2

Fig. 10.2 shows the loop cut open at point 2; the loop transfer function L2 (s) is the
64

negative product of plant transfer function and controller transfer function

L2 (s) = −K(s)C(sI − A)−1 B.

A block diagram of the controller is shown in Fig. 10.3. The state space representation of
the controller is

x̂˙ = (A + BF + Fe C)x̂ + Fe (−y)

u = F x̂

and the controller transfer function (from −y to u) is therefore

K(s) = F (sI − A − BF − Fe C)−1 Fe .

−y 1 x̂ u
Fe F
s

Figure 10.3: LQG controller

Loop Transfer Recovery

We present now a technique to recover the optimal state feedback loop gain L1 (s) with
an LQG controller. For simplicity, we consider the single-input, single-output case. If an
LQG design turns out to show poor robustness margins, then the idea is to pretend that
there is white noise entering the loop at point 2, and to redesign the optimal estimator
such as to cope with this fictitious noise. (Loosely speaking, the fictitious white noise
at plant input can be viewed as representing plant uncertainty.) Therefore, we make the
replacement
Qe → Qe + σbbT
for the true process noise covariance, where σ is a tuning parameter that determines the
intensity of the fictitious white noise process.
10. Loss of Robustness and Loop Transfer Recovery 65

For the case when the estimator is designed for the fictitious noise intensity, we will prove
the following: provided the plant is minimum phase, we have

lim L2 (s) = f (sI − A)−1 b = L1 (s),

σ→∞
(10.1)

which means the optimal state feedback loop gain can be asymptotically recovered by
letting the fictitious noise intensity approach infinity.
Proof The proof is based on the following asymptotic result for the optimal estimator
gain: as σ → ∞, we have s
σ
fe = ± b, (10.2)
re
where the sign is determined by the requirement that the estimator is stable. We will
prove this fact later and first use it to show the main result (10.1). At this point we make
the observation that as σ becomes large, the estimator gain will become large and thus
the closed-loop system becomes sensitive against measurement noise. This indicates that
LTR design involves a trade-off between closed-loop robustness and noise sensitivity.
We now prove (10.1). We will make repeated use of the matrix identity

(I − XY )−1 X = X(I − Y X)−1 ,

which is particularly useful when X and Y are vectors x and y T . Using the notation

Φ(s) = (sI − A)−1 , Φcl (s) = (sI − A − bf )−1 ,

the controller transfer function can be written as

K(s) = f (Φ−1 −1
cl − fe c) fe
= f [Φ−1 −1
cl (I − Φcl fe c)] fe
= f (I − Φcl fe c)−1 Φcl fe
= f Φcl fe (1 − cΦcl fe )−1 .

Note that the denominator in the last expression is a scalar. Taking the limit as σ → ∞
and using (10.2), the controller transfer function becomes

f Φcl (s)fe f Φcl (s)b

K(s) = =− .
1 − cΦcl (s)fe cΦcl (s)b

The common factor Φcl (s)b of numerator and denominator can be written as

Φcl (s)b = (Φ−1 − bf )−1 b = (I − Φbf )−1 Φb = Φb(1 − f Φb)−1 .

Thus we have
f Φ(s)b
K(s) = − .
cΦ(s)b
66

Substituting the last expression for K(s) in the loop gain leads to

f Φ(s)b
L2 (s) = −K(s)cΦ(s)b = cΦ(s)b = f Φ(s)b
cΦ(s)b
or
L2 (s) = L1 (s) = f Φ(s)b.
This shows that as σ → ∞, the controller cancels the plant zeros and replaces them with
the zeros of f Φ(s)b; this explains why the plant must have no unstable zeros.
To complete the proof, we have to show that (10.2) holds. We will show this by duality.
Setting Q = γcT c in Kalman’s identity and considering the regulator problem, we first
prove that s
γ
γ→∞ ⇒ f =± c. (10.3)
ρ
We have
ρ|1 − f Φb|2 = ρ + γ|cΦb|2 .
Dividing by ρ and taking the limit as γ → ∞, we obtain
s
γ
|1 − f Φb| = |cΦb|.
ρ

From this equation, we can draw two conclusions. The first one is that as γ → ∞, the
state feedback gain f must also become infinitely large, in fact it can be expressed as
√
f = γ f˜, where f˜ is some vector that does not depend on γ. The second observation
here is that because the zeros of 1−f Φ(s)b are the closed-loop poles under state feedback,
by root locus arguments they approach the zeros of f Φ(s)b as γ → ∞. The fact that the
closed-loop poles are stable then implies that the zeros of f Φ(s)b are also stable.
If f is infinitely large, we can make a further simplification to get
s
γ
|f Φ(s)b| = |cΦ(s)b|.
ρ

This equality forces the zeros of f Φ(s)b and cΦ(s)b to be either equal or reflections across
the imaginary axis. The claim (10.3) holds if all zeros are equal. We know that the zeros
of f Φ(s)b are stable; and because we assumed that the same is true for the plant transfer
function cΦ(s)b we conclude that (10.3) holds. Now making the replacements

γ → σ, ρ → re , f → f eT , c → bT

shows that (10.2) holds by duality. This completes the proof of (10.1).
Bibliography - Linear Quadratic
Optimal Control

[1] H. Kwakernaak and R. Sivan, Linear Optimal Control Systems. Wiley-interscience

New York, 1972, vol. 1.
[2] B. D. Anderson and J. B. Moore, Optimal Control: Linear Quadratic Methods.
Courier Corporation, 2007.
[3] P. Dorato, V. Cerone, and C. Abdallah, Linear Quadratic Control: An Introduction.
Simon & Schuster, Inc., 1994.
[4] T. Kailath, Linear Systems. Prentice-Hall Englewood Cliffs, NJ, 1980, vol. 156.
 Part II

OPTIMAL AND ROBUST

CONTROL
Introduction

This course introduces modern design techniques for optimal and robust multivariable
control systems. The state space methods for optimal controller design developed in the
1960s and 1970s (LQR/LQG) were found to suffer from being sensitive to modelling errors
and parameter uncertainty. In the 1980s research activities turned to a new approach,
where design objectives are achieved by minimizing the H2 norm or H∞ norm of suitable
closed-loop transfer functions. LQG control can be interpreted as a special case of H2
optimal control, while constraints on the H∞ norm can be used to include robustness
against modelling uncertainty into the controller design.
The new approach turned out to be closely related to the familiar LQG methods - the
computation of both H2 and H∞ optimal controllers involves the solution of two algebraic
Riccati equations. Moreover, H2 optimal control can be shown to be a special case of H∞
optimal control.
Conditions on the H2 norm and the H∞ norm represent different aspects of the design. The
H2 norm can be used to achieve an optimal trade-off between regulation and control effort,
and to deal with noisy measurements. Constraints on the H∞ norm on the other hand
can guarantee closed-loop stability for a given range of uncertain parameters. However,
neither H2 nor H∞ norm constraints give direct control of the closed-loop pole locations
and thus of dynamic properties like the damping ratio.
In many practical applications it would be desirable to combine the three design aspects
mentioned above: for example, one might want to design a controller that achieves a
given damping ratio and speed of response (a condition on the pole locations), while
at the same time guaranteeing robust stability in the face of model uncertainty (a H∞
constraint). And of all controllers that satisfy these conditions one would like to pick the
best one in the sense of a quadratic performance index (H2 norm). Efficient methods for
such a design have been developed in the 1990s. Instead of solving Riccati equations, one
can express H2 and H∞ constraints as linear matrix inequalities (LMI). The advantage
over the Riccati approach is that one can combine different constraints into a single convex
optimization problem, including constraints on admissible pole regions which can also be
expressed as LMI constraints.
A major problem with modern H2 and H∞ optimal control is the fact that the controllers
72

have the same dynamic order as the plant. If the plant to be controlled is of high dynamic
order, the optimal design results in controllers that are difficult to implement. Moreover,
the high order may cause numerical problems. For practical applications, it is therefore
necessary to approximate the plant by a simplified, reduced order model. This can be
done for example by using the method of balanced truncation or balanced residualization.
This course provides an introduction to the topics outlined above. First, the notion of a
generalized plant as a way of expressing design specifications is introduced and a review
of signal and system norms and of the singular value decomposition is given. Next, the
role of the sensitivity and complementary sensitivity function in H∞ optimal design is
discussed and illustrated in a case study. It is shown how one can express constraints on
the H2 and H∞ norm and pole region constraints as LMI constraints, and how one can
compute controllers that satisfy such constraints on the closed-loop system. Finally the
design of robust controllers is discussed. It is shown how parameter uncertainty can be
included in the plant model and taken into account when designing the controller. Again,
the approach is illustrated with a case study.
The emphasis in this course is on practical application, and theoretical aspects are dis-
cussed in so far as their understanding is important for using the design tools. On com-
pletion, students should be able to apply the relevant MATLAB and Simulink tools to
solve optimal and robust design problems. The course includes two design exercises: one
on mixed sensitivity design and one on robust controller design.
Chapter 11

The Concept of a Generalized Plant

- A Review of LQG Control

In modern control, almost any design problem is represented in the form shown in Figure
11.1. Here, P is referred to as the generalized plant and K is the controller. The signal
vector w represents external inputs to the control system, such as command inputs, dis-
turbances and noise, u is the control input, v contains the measured plant signals that are
used for feedback control, and z is a fictitious output vector that is used to express design
specifications. We will model the dynamic systems P and K both as transfer function
matrices P (s) and K(s) and in the form of state space realizations.

w z
P
u v

Figure 11.1: Control loop with generalized plant

To illustrate the concept of a generalized plant, we will express the problem of designing
an LQG controller in the form of Figure 11.1. The control system is shown in Figure
11.2. The plant with transfer function model G(s) is corrupted by process noise wx and
measurement noise wy . We consider a state space realization of G

ẋ = Ax + Bu + wx
y = Cx + wy (11.1)
74

where wx and wy are white noise processes

E [wx (t)wxT (t + τ )] = Qe δ(τ ), E [wy (t)wyT (t + τ )] = Re δ(τ ), E [wx (t)wyT (t + τ )] = 0

(11.2)

wx wy

r K(s) G(s) y
-

Figure 11.2: LQG control loop

Initially, we consider a regulation problem with zero reference input r = 0. The objective
is to find a controller K(s) that minimizes the LQG performance index
" #
1ZT T T
V = lim E (x Qx + u Ru) dt (11.3)
T →∞ T 0
To express this design problem in the form of Figure 11.1, we start with a state space
realization of P . With two inputs w and u, and two outputs z and v, we have

ẋ = Ap x + Bw w + Bu u
z = Cz x + Dzw w + Dzu u
v = Cv x + Dvw w + Dvu u (11.4)

The first step is to relate the inputs and outputs of the generalized plant to the LQG
problem under consideration. We take the control input u of the generalized plant to be
the control input of the LQG problem, and the measured output v of the generalized plant
to be the control error e = −y in the LQG problem. Accordingly, we relate the plant
model (A, B, C) in (11.1) to the subsystem (Ap , Bu , Cv ) of the generalized plant model
(11.4) by defining
Ap = A, Bu = B, Cv = −C, Dvu = 0
where the definition of Dvu reflects the fact that the plant in (11.1) is strictly proper.
To express the cost function (11.3) in terms of the generalized plant, choose
" # " #
Q1/2 0
Cz = , Dzu =
0 R1/2
Assume first that w = 0, then the squared integral of the fictitious output z has the same
form as the integral in the LQG cost
Z ∞ Z ∞
z T z dt = (xT Qx + uT Ru) dt
0 0
11. The Concept of a Generalized Plant - A Review of LQG Control 75

We have not yet taken the noise processes into account. The external input w in (11.4)
can be used to represent process and measurement noise as follows. Assume that w is a
white noise process satisfying

E [w(t)wT (t + τ )] = δ(τ )I

and choose h i h i
Bw = Q1/2
e 0 , Dvw = 0 Re1/2 , Dzw = 0
We then have " #
h i w1
wx = Bw w = Q1/2
e 0 = Q1/2
e w1
w2
and " #
h i w1
wy = Dvw w = 0 Re1/2 = Re1/2 w2
w2
where w is partitioned to be compatible with the dimensions of Qe and Re . It is now easy
to see that minimizing " Z #
1 T T
lim E z (t)z(t) dt
T →∞ T 0
is equivalent to minimizing the LQG performance index V in (11.3).
For the transfer function matrix G(s) of a system

ẋ = Ax + Bu
y = Cx + Du

we will use the following notation in terms of its state space realization
" #
A B
G(s) =
C D
Thus, the transfer function of a generalized plant that represents the LQG problem is
 h i 

A Q1/2
e 0 B 
" #  
Ap Bp  " 1/2 #
 Q
"
0
# 
P (s) = =  0


Cp Dp 
 0 R 1/2 
 h i 
−C 0 −Re1/2 0
where 0 stands for zero matrix blocks of appropriate dimensions.
So far, the LQG problem was considered as a regulator problem, i.e. we assumed that
r = 0 in Figure 11.2. How a reference input can be introduced into the above model to
make it equivalent to the control system shown in Figure 11.2 is indicated in Figure 11.3.
The external input w and the fictitious output z of the generalized plant are
 
r " #
z
z= 1
 
w =  w1 
 and
 z2
w2
76

w1 Q1/2
e

wy
w2 Re1/2

wx Q1/2 z1

R1/2 z2

u B 1 x C e
s -

A
P (s)

K(s)

Figure 11.3: LQG problem represented in a generalized plant configuration

The input vector u of P is the same as that in Figure 11.2, and the measured output v is
equal to the control error e.

Example 11.1
The following state space model represents a linearized model of the vertical-plane dy-
namics of an aircraft. It has three inputs, three outputs and five states. The inputs
are
u1 spoiler angle (in 0.1 deg)
u2 forward acceleration (in m s−2 )
u3 elevator angle (in deg)
x1 relative altitude (in m)
x2 forward speed (in m s−1 )
The states are x3 pitch angle (in deg)
x4 pitch rate (in deg s−1 )
x5 vertical speed (in m s−1 )
11. The Concept of a Generalized Plant - A Review of LQG Control 77

The matrices A, B, C and D are

 
0 0 1.132 0 −1
0 −0.0538 −0.1712
 
 0 0.0705 

A=
0 0 0 1 0


 
0 0.0485
 0 −0.8556 −1.013 

0 −0.2909 0 1.0532 −0.6859
 
0 0 0    
−0.12
 
 1 0 
 1 0 0 0 0 0 0 0
   
B= 0 0 0 , C = 0 1 0 0 0, D = 0 0 0
   
  
 4.419
 0 −1.665 
 0 0 1 0 0 0 0 0
1.575 0 −0.0732

Exercises

Problem 11.1
The function aircraft_model() generates the aircraft model from Example 11.1 and the
Simulink model aircraft_sim.mdl provides a closed-loop for simulations.

a) Design an LQG controller for the aircraft. For this purpose, construct a state space
representation of the generalised plant in Figure 11.3 and use the MATLAB function
h2syn to design the LQG controller. The usage is

K = h2syn( Gplant, nmeas, ncont )

where the input argument Gplant is the generalized plant, nmeas is the number
of measured outputs and ncont is the number of controller inputs. The returned
output K is the LQG controller.
h iT
b) Design a controller for tracking a reference step input r(t) = σ(t) 0 0 ("altitude
acquire") subject to the constraint |u3 | ≤ 20 on the elevator angle.
Chapter 12

Vector Norms and Induced Norms

In the previous chapter it was shown how the LQG problem can be transformed into
an equivalent problem in terms of the generalised plant in Figure 11.3. The objective
of finding the controller K(s) in Figure 11.2 that minimises the performance index V in
(11.3), is then equivalent to finding the controller K(s) in Figure 11.3 that minimises
" #
1ZT T
lim E z (t)z(t) dt
T →∞ T 0

where the input w is a white noise process with E [w(t)wT (t + τ )] = δ(τ )I.
When a design problem is expressed in terms of a generalised plant, the objective is
usually to minimise the "size" of the fictitious output signal z, under the condition that
a specified type of signal is applied to the external input w. This leads to the concept of
the signal norm as a measure of the "size" of a signal, and the system norm as a measure
of the system "gain" (in terms of a given signal norm).

Vector Norms

To introduce signal and system norms, we start with a review of vector norms and induced
norms. Consider a vector space X . A norm kxk is function mapping a vector x into a
real number, that satisfies the following four properties for any x, y ∈ X

1) kxk ≥ 0

2) kxk = 0 ⇔ x=0

3) kαxk = αkxk for any scalar α > 0

4) kx + yk ≤ kxk + kyk
80

We first consider finite dimensional real or complex vector spaces, X = IRn or X = C | n.

Let x = [x1 x2 . . . xn ]T be a vector with xi ∈ IR, i = 1, . . . , n or xi ∈ C,

| i = 1, . . . , n .
A frequently used norm on x is the vector-p-norm
n
!1
p
p
X
kxkp = |xi |
i=1

where p is a positive integer. Of practical importance are the three cases

n
X
kxk1 = |xi |
i=1
v
u n
uX
kxk2 = t |x i|
2
i=1

kxk∞ = max |xi |

i

The vector-2-norm can also be written as

√ √
kxk2 = xT x if x ∈ IRn or kxk2 = | n
xH x if x ∈ C

where AH denotes the Hermitian of a matrix A: AH = ĀT where Ā is the complex

conjugate of A. In this course we will only use the 2-norm for vectors, and we will usually
drop the subscript and write kxk for the vector-2-norm of x.

Signal Norms

Now consider the space of scalar, real or complex valued signals x(t). The p-norm of a
signal is defined in a way similar to the vector p-norm as
Z ∞ 1
p
p
kx(t)kp = |x(τ )| dτ
−∞

Here we are interested in the signal-2-norm

sZ
∞
kx(t)k2 = |x(τ )|2 dτ
−∞

We will usually drop the subscript on the signal-2-norm. Note that taking the magnitude
in the vector-2 and signal-2-norm is only required for complex vectors.
There are signals for which the signal-2-norm as defined above does not exist. As an
example, consider the signals shown in Figure 12.1: the signal x1 (t) has a finite 2-norm,
whereas the 2-norm of the sinusoidal signal x2 (t) is infinite. These two types of signals
are referred to as energy signals and power signals, respectively. Energy signals have a
finite 2-norm. For power signals, the 2-norm does not exist, but we can define the root
mean square (rms) value as
s
1 ZT
kx(t)krms = lim |x(τ )|2 dτ
T →∞ 2T −T
12. Vector Norms and Induced Norms 81

0.6

0.4

x1 (t)
0.2

−0.2
0 1 2 3 4 5 6 7 8 9 10 11 12
Time [sec]
0.4

0.2
x2 (t)

−0.2

−0.4
0 1 2 3 4 5 6 7 8 9 10 11 12
Time [sec]

Figure 12.1: Energy signal x1 (t) and power signal x2 (t)

Note that this is strictly speaking not a norm, because it does not satisfy property (2) of
a norm.
Now consider a real or complex valued signal vector x(t) = [x1 (t) x2 (t) . . . xn (t)]T . Its
signal-2-norm is defined as
sZ
∞
kx(t)k2 = kx(τ )k2 dτ
−∞

The integral is now taken over the square of the vector-2-norm of x(t). Note that at a
| n , and
given time τ , x(τ ) is simply a real or complex vector, i.e. x(τ ) ∈ IRn or x(τ ) ∈ C
that the same norm symbol denotes on the left hand side in the above equation a signal
norm and on the right hand side a finite dimensional vector norm.

The Matrix-2-Norm

Consider two complex vectors x ∈ C | n and y ∈ C

| m , and a linear mapping y = Ax. The

complex matrix A ∈ C | m×n can be interpreted as an operator that maps x into y, or more

| n into C
generally C | m . One can compare the vector norms of x and y, and associate a "gain"

with A as the ratio of these vector norms. This ratio depends on x, and an important
property of the matrix A is the maximum value of kyk/kxk over all nonzero x ∈ C | n (the

"maximum gain" of A). This positive real number can be defined as a norm of the matrix
A; since it depends also on the choice of vector norm, it is called an induced norm. The
82

matrix-2-norm induced by the vector-2-norm is defined as

kAxk2
kAk2 = max (12.1)
x6=0 kxk2

Again, we will usually drop the subscript and write kAk for the matrix-2-norm. It is
straightforward to verify that the matrix-2-norm - and indeed all induced matrix-p-norms
- satisfy the four properties of a norm.
To find the value of kAk, we take squares on both sides of (12.1) to get

kAxk2 xH AH Ax xH M x
kAk2 = max = max = max
x6=0 kxk2 x6=0 xH x x6=0 xH x

where we introduced the Hermitian matrix M = AH A. In order to find the maximum

value of the last term, we diagonalize M , i.e. we compute M = V ΛV −1 , where Λ is a
diagonal matrix containing the eigenvalues of M , and V is a matrix with right eigenvectors
of M as columns.
We will first establish the following useful facts about the Hermitian matrix M = AH A

1) | n)
M is positive semi-definite (xH M x ≥ 0 ∀x ∈ C

2) the eigenvalues of M are real

3) the eigenvectors of M are orthogonal (two vectors x and y are orthogonal if xH y = 0)

With y = Ax, property (1) follows immediately from

xH M x = xH AH Ax = y H y ≥ 0

Note that this implies that xH M x is real even if x is complex. That the eigenvalues of
M are real can be shown as follows. Let λ be an eigenvalue and v be an eigenvector of
M , and consider
M v = λv
Multiplying with v H from the left yields v H M v = λv H v. We established already that the
left hand side of this equation is real, and on the right hand side v H v is also real. Thus,
λ must be real.
To show that two eigenvectors of M belonging to different eigenvalues are orthogonal,
consider
M v1 = λ1 v1 , M v2 = λ2 v2 , λ1 6= λ2
We have
(λ1 v1 )H v2 = (M v1 )H v2 = v1H M v2 = v1H λ2 v2
thus λ1 v1H v2 = λ2 v1H v2 , and from the assumption λ1 6= λ2 it then follows that v1H v2 = 0.
12. Vector Norms and Induced Norms 83

A consequence of property (3) is that if all eigenvectors vi of M are normalized such that
kvi k = 1, i = 1, . . . , n, the eigenvector matrix V is unitary, i.e. V H V = I, or V −1 = V H .
(Strictly speaking, we have shown this only for matrices with distinct eigenvalues. It can
be shown however that even a matrix with repeated eigenvalues has a full set of orthogonal
eigenvectors.)
Note that properties (2) and (3) are true for any Hermitian matrix even when it is not
positive semidefinite.
We now return to finding the value of kAk by solving

xH AH Ax
max
x6=0 xH x

With the diagonalization AH A = V ΛV H this becomes

xH V ΛV H x
max
x6=0 xH x

and introducing y = V H x and thus x = V y (using orthonormality of V ), we obtain

y H Λy y H Λy λ1 |y1 |2 + λ2 |y2 |2 + . . . + λn |yn |2

max = max = max
y6=0 yH V H V y y6=0 y H y y6=0 |y1 |2 + |y2 |2 + . . . + |yn |2

where λ1 , . . . , λn are the eigenvalues of AH A. Assume that the eigenvalues are ordered
such that λ1 ≥ λ2 ≥ . . . ≥ λn . Then it is easy to see that the maximum value of the above
h iT
expression is λ1 , which is achieved if we choose y = 1 0 . . . 0 , and the minimum
h iT
value is λn , achieved by choosing y = 0 . . . 0 1 .
Because the above expression is the square of the matrix-2-norm of A, we have thus
established that
kAxk q
kAk = max = λmax (AH A)
x6=0 kxk

and we also found that

kAxk q
min = λmin (AH A)
x6=0 kxk
The square roots of the eigenvalues of AH A are called the singular values of A. This
leads us to the most important and useful of matrix factorizations, the singular value
decomposition (SVD), which is discussed in the next chapter.

Exercises

Problem 12.1 Show that kABk ≤ kAkkBk

Chapter 13

The Singular Value Decomposition

In the last chapter we used the fact that any Hermitian matrix M can be factored into

M = V ΛV H

where V is the eigenvector matrix of M and unitary, and Λ is the diagonal eigenvalue
matrix of M . The same factorization is obviously not possible for non-Hermitian or even
non-square matrices. A similar factorization is however possible in these cases, if we do
not insist on the same matrix V on both sides, but allow different unitary matrices U and
V as left and right factors.
| m×n there
Theorem 13.1 (Singular Value Decomposition) For every matrix A ∈ C
| m×m and V ∈ C
exist unitary matrices U ∈ C | n×n such that

A = U ΣV H (13.1)

and Σ is real and diagonal with non-negative entries.

The matrix Σ has the same size as A. For example, if A is a 3 × 2 or 2 × 3 matrix, then
 
σ 0
 1
" #
σ 0 0
Σ= 1

Σ =  0 σ2 
 or
 0 σ2 0
0 0
respectively, where σ1,2 ≥ 0. The diagonal entries σi are called the singular values of A.
From (13.1) we obtain AV = U Σ and thus

Avi = σi ui , i = 1, . . . , n

where vi and ui are the columns of V and U , respectively. Compare this with M vi = λi vi -
an eigenvector vi is transformed into λi vi , whereas A transforms vi into σi ui . From (13.1)
we also have
AAH = U ΣV H V ΣT U H = U ΣΣT U H (13.2)
86

and
AH A = V ΣT U H U ΣV H = V ΣT ΣV H (13.3)
Equation (13.2) shows that U is the eigenvector matrix of AAH , and (13.3) shows that V is
the eigenvector matrix of AH A. The eigenvalue matrices are ΣΣT and ΣT Σ, respectively.
Again, if A is 3 × 2 then
 
σ2 0 0
 1
" #
T Tσ2 0
2
Σ Σ= 1

ΣΣ = 
 0 σ2 0 ,

0 σ22
0 0 0

This shows that the singular values of A are the square roots of the eigenvalues of AAH
and AH A.
Proof of Theorem 13.1
To prove Theorem 13.1, we show how to construct U, V and Σ that satisfy (13.1) for a
given matrix A. We start with the diagonalization of AH A: we established already that
there exists a unitary matrix V such that

AH A = V ΛV H

where Λ = diag(λ1 , λ2 , . . . , λn ) is the diagonal eigenvalue matrix of AH A, and the columns

vi of V are the corresponding eigenvectors. Thus

AH Avi = λi vi and viH AH Avi = λi viH vi = λi

because V is unitary, and therefore

kAvi k2 = λi (13.4)

This implies that λi ≥ 0. Assume that the eigenvalues λ1 , . . . , λr are positive and the
remaining n − r eigenvalues λi and vectors Avi are zero. Note that r ≤ min(n, m). Define
q 1
σi = λi , ui = Avi , i = 1, . . . , r
σi
It follows from (13.4) that kui k = 1. Moreover, we have

viH AH Avj λi viH vj

uH
i uj = = = 0, i 6= j
σi σj σi σj
This shows that the vectors u1 , . . . , ur defined above have the properties required of column
vectors for U to be unitary. If r < m, one can fill up the matrix U with m − r further
orthogonal vectors (by using Gram-Schmidt orthogonalization) to make it into a m × m
unitary matrix.
Now it remains to show that the matrices U, V as defined above satisfy

U H AV = Σ
13. The Singular Value Decomposition 87

where Σ is diagonal with σi as diagonal entries. The (i, j) entry of U H AV is

(
H σj uH
i uj , j≤r
(U AV )i,j = uH
i Avj =
0, j>r
Because σj uH H
i uj is zero if i 6= j and σj if i = j, the above shows that the entries of U AV
are all zero except for the first r entries on the main diagonal, which are the singular
values of A. This completes the proof.

Exercises

Problem 13.1 Calculate the singular value decomposition of the matrices

     
2 1 1 2 0 −1
     
4 2 3 ; 0 −3 ;  2 
     
6 3 4 0 0 2

Problem 13.2 Use the singular value decomposition to calculate the rank of the ma-
trices " # " # " #
 2  1  1
; ;
1 2 0 0  1+
where  is very small number. Discuss the result.

Problem 13.3
Suppose a satellite takes the following picture and wants to send it to earth:
 
38 66 20 47 40 49 55 97 75 69
 
90 164 54 121 97 126 135 235 186 171
 
77 141 45 104 83 108 113 201 160 145
 
 
77 140 49 103 78 113 112 201 158 142
 
96 172 61 128 90 146 132 249 189 175
Image = 
 

96 176 30 117 153 76 167 243 218 194
 
55 98 33 72 56 80 79 144 112 103
 
 
81 146 56 113 69 135 110 214 160 145
 
41 77 23 55 50 53 63 107 88 80 
 
83 151 57 116 68 137 111 217 164 149
(8-Bit Image, gray-level (0-255), Resolution (10 × 10) pixels.)
The picture contains 10 · 10 · 8 = 800 bits. Try to reduce the number of transmitted Bits
such that the image can still be recovered at earth.
Note: the image matrix Image can be downloaded from the web site of this course.
Hint: Find the essential information in this image using SVD.
Chapter 14

System Norms

In this chapter, we introduce two system norms: the H∞ norm and the H2 norm. The
H∞ norm is a measure for the "gain" of a system in terms of the signal-2-norm - it is an
induced norm. The H2 norm on the other hand is not induced by a signal norm. We
will first discuss both system norms for single-input single-output systems and then for
multivariable systems.

The H∞ Norm for Single-Input Single-Output Systems

For a stable, proper single-input single-output (SISO) system with transfer function G(s),
the H∞ norm is defined as
kG(s)k∞ = sup |G(jω)|
ω

Here "sup" stands for supremum or least upper bound, and means the same as "max", but
in addition allows for the possibility that its value is only approached asymptotically.

u(t) G(s) y(t)

Figure 14.1: Single-input single-output system

A graphical interpretation of this definition is shown in Figure 14.2. The H∞ norm of a

SISO system is simply the maximum gain over all frequencies, and can be read off the
Bode magnitude plot of the frequency response.
For a system with input u(t) and output y(t) (see Figure 14.1), the H∞ norm is equal to
the norm induced by the signal-2-norm:

ky(t)k
kG(s)k∞ = max
u6=0 ku(t)k
90

|G|

kGk∞

ω0 ω

Figure 14.2: H∞ norm as peak value in the frequency response

if u(t) is an energy signal, and

ky(t)k rms
kG(s)k∞ = max
u6=0 ku(t)krms
if u(t) is a power signal. We will not prove this fact here; referring to Figure 14.2 it
should however be clear that the power signal leading to the largest "gain" in terms of
rms values is a sinusoidal input at frequency ω0 . In fact, the steady state response to an
input u(t) = sin ω0 t is
y(t) = kG(s)k∞ sin(ω0 t + φ)
where φ = arg G(jω0 ) is the phase shift at that frequency. Note that the phase response
has no effect on the H∞ norm of the system.

The H2 Norm for SISO Systems

The H2 norm for a SISO system with a stable, strictly proper transfer function G(s) is
defined as s
1 Z∞
kG(s)k2 = |G(jω)|2 dω
2π −∞
The restriction to strictly proper systems is necessary because otherwise |G(jω)| > 0 as
ω → ∞ and the integral does not exist. This norm is not induced by a signal norm, but
there are two important interpretations of this norm in terms of certain input signals -
one stochastic and one deterministic.
For a stochastic interpretation of the H2 norm, assume that the input u(t) is white noise
with E[u(t)u(t + τ )] = δ(τ ). In that case, the rms value of the output signal is equal to
the H2 norm of the system
ky(t)krms = kG(s)k2
This fact makes it possible to express the LQG problem as the problem of minimizing the
H2 norm of the generalized plant introduced in chapter 11.
14. System Norms 91

A deterministic interpretation goes as follows. Let

ẋ = Ax + bu, y = cx

be a state space realization of G(s); note that D = 0 because G(s) is strictly proper. Let

g(t) = ceAt b

denote the impulse response of the system. By Parseval’s theorem, the above definition of
the H2 norm is then equivalent to the following definition in terms of the impulse response
sZ
∞
kG(s)k2 = |g(t)|2 dt = kg(t)k2
0

The H2 norm of the system is equal to the signal-2-norm of its impulse response. This
makes it possible to express the deterministic LQR problem as a H2 optimization problem.

The H∞ Norm for MIMO Systems

Consider the system shown in Figure 14.3, where input and output are now assumed to
be vector signals. For SISO systems, the H∞ norm was defined as the maximum gain over
frequency. Now we are dealing with a transfer function matrix G(s), and the definition
of the H∞ norm should be consistent with the interpretation as a system norm induced
by the signal-2-norm or the rms value of power signals.

u(t) G(s) y(t)

Figure 14.3: Multi-input multi-output system

What is different now is the fact that it is not only the “size" of the signals that matters,
but also the “direction" of the signal vectors. To illustrate this point, consider the following
2 by 2 system consisting only of constant gains.

Example 14.1
For a system with transfer function matrix
" #
3 0
G(s) =
4 1
h iT h iT
the response to a constant input u(t) = 1 0 σ(t) is y(t) = 3 4 σ(t), and ky(t)krms =
q h iT h iT
5/ (2). The response to u(t) = 0 1 σ(t) is y(t) = 0 1 σ(t), and ky(t)krms =
92

q q
1/ (2). Since in both cases ku(t)krms = 1/ (2), the "gain" in terms of rms values is 5 for
the first input signal and 1 for the second input signal.

To introduce the definition of the H∞ norm for MIMO systems, we will again consider
sinusoidal input signals, and we will find the combination of inputs that maximizes the
output signal. The above example showed that we need to take the direction of the input
signal into account. With a vector of sinusoidal input signals, it is not only the direction
in terms of the magnitude (amplitude) of the individual signals, but also their phase that
has an effect on the output. To account for both magnitude and phase, we will introduce
complex signals and complex "directions".
A sinusoidal input with amplitude u0 and phase ϕ can be interpreted as the imaginary
part of a complex signal
h i
u(t) = u0 sin(ωt + ϕ) = Im u0 ej(ωt+ϕ)

Now define the complex signal

ũ(t) = u0 ej(ωt+ϕ) = ûejωt

where the complex amplitude

û = u0 ejϕ
contains both magnitude and phase information about the signal. The steady state re-
sponse to the complex input signal ũ(t) = ûejωt is

ỹ(t) = ŷejωt

where the complex amplitude of the output signal is the amplitude of the input signal
multiplied by the transfer function evaluated at s = jω

ŷ = G(jω)û

The physical signals u(t) and y(t) can be recovered by taking the imaginary parts of the
complex signals. We now apply the above signal representation with complex amplitudes
to a multivariable system with m inputs and l outputs. At a given frequency ω, the input
and output signals are
   
û1 ŷ1
 .  jωt  .  jωt
ũ(t) =  .. 

e = ûejωt , ỹ(t) =  .. 

e = ŷejωt
ûm ŷl

where ũ(t) is applied to the system input and ỹ(t) is the response in steady state. To find
the induced 2-norm of the system, it is enough to consider the amplitude vectors û and ŷ
because
kỹ(t)krms kŷk
=
kũ(t)krms kûk
14. System Norms 93

The output amplitude vector at a given frequency ω is obtained by multiplying the input
amplitude vector with G(jω). For example if the system is 2 by 2, we have
" # " #" #
ŷ1 G11 (jω) G12 (jω) û1
=
ŷ2 G21 (jω) G22 (jω) û2

Note that for a given frequency, the transfer function matrix is just a complex matrix
G(jω) ∈ C| l×m . The maximum value of kŷk/kûk at that frequency is given by the maximum

singular value of the transfer function matrix

kŷk
max = σ̄(G(jω))
û6=0 kûk

We will use the notation σ̄(G) and σ(G) for the maximum and minimum singular value of
G. The H∞ norm of the system is defined as the maximum value of this induced matrix
norm over all frequencies
kG(s)k∞ = sup σ̄(G(jω))
ω

kGk∞
σ̄(G)
σ(G)

ω0 ω

Figure 14.4: H∞ norm as peak value in the singular value plot

A graphical interpretation of the H∞ norm is shown in Figure 14.4: it is the peak value of
the maximum singular value over frequency. Singular value plots of multivariable systems
usually show the minimum and the maximum singular value - indicating the minimum
and maximum gain at a given frequency over all input directions.

The H2 Norm for MIMO Systems

To extend the definition of the H∞ norm to MIMO systems, we used the induced 2-norm
for matrices, which is equal to the maximum singular value. For the H2 norm, we need
to introduce a new matrix norm. The Frobenius norm of a matrix A is defined as
q
kAkF = trace(AH A)
94

The definition of the H2 norm of a multivariable system is

s
1 Z∞
kG(s)k2 = kG(jω)k2F dω
2π −∞
Again, using Parseval’s theorem, one can show that an equivalent definition in time do-
main is sZ
∞
kG(s)k2 = kg(t)k2F dt
0

where g(t) = CeAt B is the impulse response matrix of the system.

Exercises

Problem 14.1
For the aircraft model in Exercise 11.1, plot the singular values of the closed-loop system
with an LQG controller over frequency (you can use the MATLAB command sigma).
What do the singular values at low frequencies tell you about the systems tracking ca-
pability? What does the increasing difference between minimum and maximum singular
values at high frequencies indicate about the systems sensitivity against measurement
noise?
Chapter 15

Computing System Norms

In this chapter we show how to compute the H2 norm and the H∞ norm of a given transfer
function. The H2 norm can be computed by solving a single Lyapunov equation, while
the computation of the H∞ norm is an iterative procedure. The computation of both
norms is based on a state space realization

ẋ = Ax + Bu, y = Cx + Du

of the transfer function G(s).

Computing the H2 norm

The H2 norm is defined only for plants with a strictly proper transfer function, thus we
assume D = 0. From the definition we have
s Z ∞
kG(s)k2 = trace (g T (t)g(t)) dt
0

where
g(t) = CeAt B
is the impulse response matrix. Substituting this expression in the above equation and
taking squares yields
Z ∞ T
kG(s)k22 = trace B T eA t C T CeAt B dt (15.1)
0
Z ∞ T
= trace B T eA t C T CeAt dtB (15.2)
0

Defining Z ∞ T
Wo = eA t C T CeAt dt (15.3)
0
we have q
kG(s)k2 = trace B T Wo B (15.4)
96

and it is straightforward to show (compare Exercise 9.1 in Part I, page 58) that Wo is the
solution to the Lyapunov equation

AT Wo + Wo A + C T C = 0 (15.5)

Using the fact that trace M N = trace N M for two matrices M and N with compatible
dimensions, we can rewrite (15.1) as
Z ∞ T
kG(s)k22 = trace CeAt BB T eA t C T dt
0

and obtain as an alternative expression for the value of the H2 norm

q
kG(s)k2 = trace CWc C T (15.6)

where Z ∞ Tt
Wc = eAt BB T eA dt (15.7)
0
is the solution to
AWc + Wc AT + BB T = 0 (15.8)
Thus, the H2 norm can be computed by solving a single Lyapunov equation. Note that
equations (15.5) and (15.8) are linear in the elements of Wo and Wc , respectively.
The matrices Wc and Wo are called the Controllability Gramian and the Observability
Gramian, respectively. Their singular value decompositions reveal important information
about the properties of a system, and we will study Gramians in more detail later in the
context of model order reduction. The terms controllability Gramian and observability
Gramian are justified by the fact that

• a system (A, B) is controllable if and only if Wc is positive definite;

• a system (A, C) is observable if and only if Wo is positive definite.

Computing the H∞ norm

The H∞ norm is defined for systems with stable, proper transfer functions - there is no
need to assume D = 0. However, to simplify the derivation of the result given below, we
will consider strictly proper systems first.
Because the H∞ norm of G(s) is the maximum of σ̄(G(jω)) over frequency, one could
try to compute σ̄(G(jω)) for many values of ω and then search for the maximum. A
more efficient method that avoids the problem of determining the range and spacing of
frequencies to be checked, is to check the imaginary eigenvalues of a certain Hamiltonian
matrix.
The method for computing the H∞ norm described below is an iterative procedure - in
each step we can only check whether kG(s)k∞ is less than a given positive constant γ.
15. Computing System Norms 97

Consider a stable plant with transfer function G(s) = C(sI − A)−1 B. For a given γ > 0,
define the Hamiltonian matrix
 
1
A γ
BB T
Mγ =  1 T T

−γ C C −A

The key for computing kG(s)k∞ is the following Theorem.

Theorem 15.1 Given a positive constant γ > 0, the following two statements are equiv-
alent

i) γ is a singular value of G(jω0 ) at some frequency ω0

ii) Mγ has at least one eigenvalue on the imaginary axis.

This Theorem allows the following strategy for computing kG(s)k∞ . We want to find the
largest singular value over all frequencies, and we can start with a sufficiently large value
of γ and use the above Theorem to check whether it is a singular value of G(jω) at some
frequency. Note that we do not need to know at which frequency. If Mγ has no imaginary
eigenvalue, γ was too large and we try with a smaller value of γ. If Mγ does have an
imaginary eigenvalue, γ was too small and we try with a larger value. A bisection method
can be used as an efficient way of finding a value of γ that is equal to kG(s)k∞ within a
guaranteed accuracy.
Proof of Theorem 15.1
We first show (i) ⇒ (ii). Assume that γ is a singular value of G(jω0 ); to simplify notation
we write G0 = G(jω0 ). Let G0 = U ΣV H be the singular value decomposition of G0 .
Then, from G0 V = U Σ and GH T
0 U = V Σ , there exist nonzero vectors u and v such that

G0 v = γu, GH
0 u = γv

Substituting G(s) = C(sI − A)−1 B at s = jω0 for G0 yields

C(jω0 I − A)−1 Bv = γu, and B T (−jω0 I − AT )−1 C T u = γv (15.9)

Introducing the vectors

p = (jω0 I − A)−1 Bv and q = (−jω0 I − AT )−1 C T u (15.10)

this becomes
Cp = γu and B T q = γv
or " #" # " #" #
C 0 p 0 γI v
=
0 BT q γI 0 u
98

Solving for v and u yields

" # " #−1 " #" #
v 0 γI C 0 p
= (15.11)
u γI 0 0 BT q
Note that (15.11) guarantees that " # " #
p 0
6=
q 0
From (15.10), the vectors p and q satisfy

(jω0 I − A)p = Bv and (−jω0 I − AT )q = C T u

or " #" # " #" # " #

A 0 p B 0 v p
+ = jω0
0 −AT q 0 −C T u q
Substituting from (15.11) yields
" # " #" #−1 " # " # " #
A 0 B 0 0 γI C 0  p p
T + = jω0 (15.12)

0 −A 0 −C T γI 0 0 B T
q q

It is straightforward to verify that the matrix expression in parentheses is Mγ and thus

" # " #
p p
Mγ = jω0
q q
which shows that jω0 is an eigenvalue of Mγ . This proves (i) ⇒ (ii).
To prove (ii) ⇒ (i), assume that jω0 is an eigenvalue of Mγ . Then there exists a nonzero
h iT
vector pT q T that satisfies (15.12). Now use (15.11) to define [v T uT ]T 6= 0. Then
from (15.11) and (15.12) it follows that γ is a singular value of G(jω0 ). This completes
the proof.
If D 6= 0, the associated Hamiltonian matrix to check is
A − BR−1 DT C −γBR−1 B T
" #
Mγ =
−γC T S −1 C −(A − BR−1 DT C)T
where R and S are given by

R = DT D − γ 2 I and S = DDT − γ 2 I

H2 and H∞ Spaces

Why are the two system norms introduced in the previous chapter called the H2 norm and
the H∞ norm? The H in these norms refers to the British mathematician G.H. Hardy,
after whom a certain type of vector spaces - the Hardy spaces - are named. The space of
strictly proper, stable transfer functions is the Hardy space H2 , and the space of proper,
stable transfer functions is the Hardy space H∞ .
15. Computing System Norms 99

Exercises

Problem 15.1
Use the MATLAB functions norm(T,2) and norm(T,inf) to compute the H2 norm and
the H∞ norm of the closed-loop system T in Exercise 11.1. Compare the MATLAB code
used in these functions with the results presented in this chapter.

Problem 15.2
For the aircraft model from Example 11.1 design an LQG controller using the following
set of weighting matrices

Ap = A; Bu = B; Cv = −C; Dvu = 0; Dzw = 0;

Design 1:
" # " #
C 0 h i h i
Cz = 2 · ; Dzu = ; Bw = 2 · B 0 ; Dvw = 0 I3
0 I3

Design 2:
" # " #
C 0 h i h i
Cz = 50 · ; Dzu = ; Bw = 50 · B 0 ; Dvw = 0 I3
0 I3

• Plot the singular values of the closed-loop system with both controllers. (Use the
MATLAB command sigma).

• What do the singular values at low frequencies tell you about the system tracking
capability?

• Give some comments on the noise rejection capabilities of these systems. (Look at
the singular values at high frequencies)
100

Problem 15.3
For the closed loop system of Problem 15.2 (design 2)

a) Calculate the H2 norm of the closed loop system, using the controllability and
observability Gramian.
Hint: Use the MATLAB function gram.

b) Estimate the H2 norm of the closed loop system T (s) using

!1/2
Z Tf
kT (s)k2 = kg(t)k2F dt
0

where g(t) is the impulse response matrix of the system and Tf is a sufficiently large
value.
Hint: Use the MATLAB function impulse, and use the trapezoidal approximation
to integrate.

c) Calculate the H2 norm of the closed loop system, using the MATLAB function norm.
Compare with previous results.

Problem 15.4
For the closed loop system of Problem 15.2 (design 2)

a) Calculate the H∞ norm of the closed loop system by iterating on γ, using Theorem
15.1.

b) Read the H∞ norm of the closed loop system from the singular value plots obtained
in Problem 15.2.

c) Calculate the H∞ norm of the closed loop system, using the MATLAB function
norm. Compare with previous results.
Chapter 16

Design Objectives and Sensitivity

Functions

In this chapter we will discuss how the H∞ norm can be used to express design specifica-
tions on control systems. For this purpose, we will first introduce the sensitivity function
and the complementary sensitivity function of a feedback system.

e u
r K(s) G(s) y
-

Figure 16.1: Closed-loop system

Consider the closed-loop system shown in Figure 16.1. External inputs are a reference
input r(t), an output disturbance d(t) and measurement noise n(t). The controlled output
is y(t), and we have
y = Gu + d
Substituting
u = K(r − y − n)
yields
y = GK(r − y − n) + d or (I + GK)y = GK(r − n) + d
so the controlled output is
y = (I + GK)−1 GK(r − n) + (I + GK)−1 d (16.1)
102

Now define the transfer functions

S(s) = (I + G(s)K(s))−1 and T (s) = (I + G(s)K(s))−1 G(s)K(s)

The function S(s) is called the sensitivity function of the feedback system. From (16.1),
it is the transfer function from the output disturbance d to the controlled output y. It is
easy to check that the functions S(s) and T (s) are related by

S+T =I (16.2)

For this reason, the function T (s) is called the complementary sensitivity of the system.
From (16.1), it is the closed-loop transfer function from r to y, and it is also the transfer
function from measurement noise to controlled output. In terms of S and T , we have

y = T (r − n) + Sd (16.3)

There is a further interpretation of the sensitivity function. The control error e(t) is

e = r − n − y = r − n − (GKe + d)

Solving for e yields

e = S(r − n − d) (16.4)
which shows that the sensitivity S is also the transfer function from reference input,
measurement noise and output disturbance, to the control error.
For the control system in Figure 16.1, we consider the following design objectives

• tracking: the controlled output should track the reference input, ideally y(t) = r(t)

• disturbance rejection: the controller should keep the controlled output at its desired
value in the presence of a disturbance d(t) 6= 0. Ideally the transfer function from
d to y would be zero.

• noise rejection: the controller should suppress the effect of measurement noise on
the control input

• reasonable control effort: the above design objectives must be achieved within given
constraints on the actuators, which means the magnitude of the control input must
not exceed given limits.

In terms of sensitivity and complementary sensitivity, it is clear from (16.3) that perfect
tracking and disturbance rejection require T (s) = I. From (16.2), this implies S(s) = 0,
which is also consistent with (16.4) - the control error should be zero.
Turning to the third design objective, (16.3) shows that perfect noise rejection requires
T = 0 (and consequently S = I). Clearly, tracking and disturbance rejection on one
16. Design Objectives and Sensitivity Functions 103

hand and the suppression of measurement noise on the other hand are conflicting design
objectives.
The control effort is related to the sensitivity because

u = Ke = KS(r − n − d) (16.5)

To achieve the design objectives with reasonable control effort, the transfer function
K(s)S(s) should not become "too large". The function KS is called the control sen-
sitivity.
The discussion so far showed that tracking and noise rejection are conflicting design
objectives. On the other hand, it is clear that for physical reasons alone perfect tracking
is not possible for signals at arbitrary high frequencies. One important decision that has to
be made when designing a control system is to determine the closed-loop bandwidth. The
bandwidth is the frequency range in which control is effective, i.e. where the reference input
is tracked and disturbances are rejected. In practice, the design objectives are interpreted
such that good tracking and disturbance rejection properties should be achieved for signals
within the closed-loop bandwidth, and noise rejection for high frequency measurement
noise beyond the bandwidth.

Mixed Sensitivity Design for SISO Systems

In the preceding discussion and the notation used so far, we assumed that the plant is
a multi-input multi-output system. We will discuss how the design objectives can be
expressed and achieved by constraints on S, T and KS for single-input single-output
systems first. Figure 16.2 shows a typical shape of the magnitude of S(s) and T (s) for the
control system in Figure 16.1 when the plant is a SISO system. At low frequencies, the
sensitivity S is close to zero, which means good tracking and disturbance rejection. At
higher frequencies, the magnitude of S increases, and at the frequency ω = ωb , it is 0.71
(-3dB). We will consider control to be effective if |S| < 0.71, and define the closed-loop
bandwidth as this frequency.
The complementary sensitivity T at low frequencies is approximately 1, which is consistent
with |S| ≈ 0 and indicates good tracking properties, because T is the transfer function
from r to y. At high frequencies, |T | rolls off and approaches zero. This is required for
the rejection of high frequency measurement noise, and implies that |S| approaches 1. An
alternative way of defining the closed-loop bandwidth is to take it as the frequency ωbT
where |T | = 0.71. Figure 16.2 illustrates that the two frequencies ωb and ωbT are usually
not equal, but also not far from each other.
104

|T (jω)| |S(jω)|
1

0, 71

0
ωb ωbT ω

Figure 16.2: Sensitivity and complementary sensitivity of a SISO system

The "Waterbed Effect"

In Figure 16.2 both |S| and |T | are shown to have peak values greater than 1. This happens
often and is in many cases unavoidable. A peak of |T | greater than 1 indicates a resonant
peak with overshoot in the step response, and |S| > 1 means that disturbances in this
frequency range are not suppressed but actually amplified. These peaks are unavoidable
if

• the pole excess of the plant is greater than one, or

• if the plant has one or more zeros in the right half plane

If the plant is minimum-phase and if the pole excess of the plant is greater than one, one
can show that Z ∞
ln |S(jω)| dω = 0
0

A consequence of this is that the area below 1 and above |S(jω)| is equal to the area above
1 and below |S(jω)|, when |S(jω)| is plotted on a logarithmic scale. In other words, if
we push |S(jω)| down at low frequencies, it must pop up somewhere else (the "waterbed
effect") - usually at high frequencies.
We will not prove this fact here but try to make it plausible with the help of a Nyquist plot.
Consider Figure 16.3 where the Nyquist plot of the loop transfer function L(s) = G(s)K(s)
is shown for the control system in Figure 16.1 with a SISO plant. If L(s) has a pole excess
of 2, then the Nyquist plot of L will penetrate a disc of radius 1 around the critical point
-1. Since the distance between L(jω) and -1 is |1 + L(jω)|, we have inside the disc

|1 + L(jω)| < 1 and therefore |S(jω)| > 1

If the plant has right half plane zeros, the resulting phase lag will lead to L(jω) penetrating
the disc even if the pole excess is only 1.
16. Design Objectives and Sensitivity Functions 105

Im

−1 Re
L(jω)

Figure 16.3: Penetration of the −1 disc by the Nyquist plot of L(s)

Low Frequency Design - Shaping the Sensitivity

In chapter 19 we will study methods for designing a controller such that the H∞ norm
of the closed-loop transfer function is less than a specified value. Here we will show how
the H∞ norm can be used to express constraints on the sensitivity that are required to
achieve the design objectives. Properties of the closed-loop system that depend on the
shape of |S(jω)| include

• the bandwidth ωb

• peak overshoot

• the system type (capability of tracking step or ramp inputs with zero steady state
error)

To see how the sensitivity is related to the system type, consider Figure 16.4. The
magnitude response of a type 1 system - a system with integral action in the loop - has
at low frequencies a slope of -20 dB/dec. Because at low frequencies (ω  ωb ) we have
|L(jω)|  1, we can approximate the sensitivity by

1 1
S(jω) = ≈
1 + L(jω) L(jω)

Thus, at low frequencies the graph of |S(jω)| in dB is approximately equal to the mirror
image of |L(jω)| about the 0 dB line. In other words, a feedback system has integral
action in the loop if the magnitude of the sensitivity has at low frequencies a positive
slope of 20 dB/dec. Similarly, for a type 2 system we need |S(jω)| to have a slope of 40
dB/dec etc.
106

−20 dB/dec
|L(jω)|
0 dB
|S(jω)|
20 dB/dec

Figure 16.4: Relation between loop gain and sensitivity at low frequencies

We have seen that we need to keep |S(jω)| small at low frequencies. One could use the H∞
norm to express constraints on the magnitude of the sensitivity, for example by imposing

kS(s)k∞ < MS

for some small constant MS > 0. Because the H∞ norm is the peak value of the magnitude
of S(s), this would enforce
|S(jω)| < MS ∀ω
Such a constraint would be unrealistic however, since we know that we cannot keep the
sensitivity small at all frequencies. To express a constraint on a selected frequency range,
one can instead introduce a weighting function WS (s) and impose

kWS (s)S(s)k∞ < 1

This enforces
|WS (jω)S(jω)| < 1 ∀ω
or equivalently
1
|S(jω)| < ∀ω
|WS (jω)|
This shows that one can use the weighting function WS (s) - together with a constraint
on the H∞ norm - to shape the magnitude of S(jω). If the H∞ norm is less than 1, the
inverse of the weighting function is an upper bound on the sensitivity. A typical shape of
1/|WS (jω)| is shown in Figure 16.5 - the weighting function enforces integral action and
an upper bound MS on the peak value of |S(jω|).
16. Design Objectives and Sensitivity Functions 107

MS

0 dB
1
|WS |
|S|

Figure 16.5: Shaping the sensitivity with a weighting function WS (s)

High Frequency Design - Shaping |T (s)| and |K(s)S(s)|

At high frequencies (ω  ωb ) the complementary sensitivity is required to roll off in order

to suppress measurement noise. This requirement can be expressed by introducing a
weighting function WT (s) for the complementary sensitivity and imposing

kWT (s)T (s)k∞ < 1

The function WT (s) can be used to shape |T (jω)| just as WS (s) can be used to shape
|S(jω)|.
It was mentioned earlier that in order to maintain a reasonable control effort, the control
sensitivity K(s)S(s) should not become "too large". Now we can be more specific and
impose
kWK (s)K(s)S(s)k∞ < 1
where a further weighting function WK (s) has been introduced. This function can be
chosen to impose limits on the control effort in a selected frequency range - typically at
high frequencies.

Weighting Filters and Generalized Plant

The constraints on sensitivity, complementary sensitivity and control sensitivity discussed

so far can be interpreted in the form of the block diagram in Figure 16.6. In this block
diagram, the functions S, T and KS represent the following transfer functions:

• sensitivity S: r→e

• complementary sensitivity T : r→y

• control sensitivity KS: r→u (because e = Sr and u = Ke)

108

WS (s) zS

WT (s) zT

WK (s) zK

r K(s) G(s) y
- e u

Figure 16.6: Constraints on S, T and KS as weighting filters

The weighting functions are here used as filters to generate fictitious output signals zS ,
zT and zK . This is done in a way such that the weighted sensitivity functions - on which
a H∞ constraint is to be imposed - can now be seen as the transfer functions from r to
the fictitious signals

• weighted sensitivity WS S: r → zS

• weighted complementary sensitivity WT T : r → zT

• weighted control sensitivity WK KS: r → zK

In Figure 16.7, the block diagram has been redrawn to bring it into the form of the
generalized plant shown in Figure 11.1.
Referring to this generalized plant, each of the three constraints

kWS (s)S(s)k∞ < 1 (16.6)

kWT (s)T (s)k∞ < 1 (16.7)
kWK (s)K(s)S(s)k∞ < 1 (16.8)

can be expressed as a constraint on the H∞ norm of the closed-loop transfer function

from the external input r to one of the outputs zS , zT or zK .
Of these three constraints, the first one is active at low frequencies, while the second and
third are active at high frequencies. Usually the low frequency constraint is combined
with one of the high frequency constraints to shape the overall loop transfer function.
One can approximate such a combined constraint by considering the H∞ norm of the
transfer function from r to an output vector z that contains two (or possibly three)
16. Design Objectives and Sensitivity Functions 109

u y e
G(s) - WS (s) zS

WT (s) zT

WK (s) zK

K(s)

Figure 16.7: Constraints on S, T and KS in terms of a generalized plant

output signals. As an illustration, assume we want to design a controller with integral

action and given actuator constraints. We choose zS and zK as outputs and (ignoring zT
in this case) define the fictitious output vector z of the generalized plant as
" #
z
z= S
zK

Let Tzr (s) denote the closed-loop transfer function from r to z such that z = Tzr r. From
Figure 16.7 we have
" # " # " #
zS WS S WS (s)S(s)
= r, therefore Tzr (s) =
zK WK KS WK (s)K(s)S(s)

When WS (s) and WK (s) have been chosen to express the design specifications, the task is
to find a controller K(s) such that the closed-loop transfer function satisfies kTzr (s)k∞ < 1
or " #
W S
S
<1 (16.9)


WK KS
∞

We will discuss in chapter 19 how a controller that satisfies this condition can be computed.
Here we assume that a controller has been found that satisfies (16.9). From the definition
of the H∞ norm, (16.9) is equivalent to
" #!
WS (jω)S(jω)
sup σ̄ <1
ω WK (jω)K(jω)S(jω)
110

We are still considering SISO systems, and at a given frequency, we are now looking for
the maximum singular value of a two-dimensional complex vector. For a complex vector
x = [x1 x2 ]T we have
q √ q
σ̄(x) = λmax (xH x) H
= x x = |x1 |2 + |x2 |2 = kxk

The condition (16.9) is therefore

q
sup |WS S|2 + |WK KS|2 < 1 (16.10)
ω

If |WS S|  1, this is equivalent to

sup |WK KS| < 1

ω

On the other hand, if |WK KS|  1, (16.10) is the same as

sup |WS S| < 1

ω

If either of the two terms |WS S| or |WK KS| is small, (16.9) is a close approximation
of the H∞ constraint on the remaining term. The largest approximation error occurs if
|WS S| = |WK KS|, in this case the approximation of (16.6) and (16.8) by (16.9) becomes

sup |WS S| < 0.71 and sup |WK KS| < 0.71
ω ω

respectively. Thus, the largest approximation error is -3dB - which means that (16.9) is
a reasonable approximation of the constraints (16.6) and (16.8) on S and KS.

Mixed Sensitivity Design for MIMO Systems

For SISO plants, we used the fact that the condition

kWS (s)S(s)k∞ < 1 (16.11)

is equivalent to
1
|S(jω)| < ∀ω (16.12)
|WS (jω)|
For MIMO plants, the sensitivity S(s) is a l by l transfer function matrix (where l is the
number of plant outputs). Accordingly, we need a l by l matrix WS (s) of shaping filters.
Usually WS (s) is chosen diagonal, and we will begin with the choice
 
wS (s) 0
WS (s) = 
 ... 

 
0 wS (s)
16. Design Objectives and Sensitivity Functions 111

where the same scalar shaping filter wS (s) is used for each output of S(s). With this
choice, (16.11) can be written as

sup σ̄(WS (jω)S(jω)) = sup σ̄(wS (jω)S(jω)) = sup |wS (jω)| σ̄(S(jω)) < 1
ω ω ω

This is equivalent to
1
σ̄(S(jω)) < ∀ω
|wS (jω)|
While for a SISO plant the shaping filter acts as the upper bound in (16.12) on the
magnitude of the sensitivity, it acts as an upper bound on the maximum singular value
of the sensitivity for a MIMO plant. Figure 16.8 shows a typical pattern of the maximum
and minimum singular value plots of S(jω) and T (jω).
Similarly, the constraints kWT T k∞ < 1 and kWK KSk∞ < 1 on complementary sensitivity
and control sensitivity are equivalent to
1 1
σ̄(T (jω)) < ∀ω and σ̄(K(jω)S(jω)) < ∀ω
|wT (jω)| |wK (jω)|

where wT (s) and wK (s) are scalar shaping filters.

σ̄(T ) σ̄(S)
1
σ(T ) σ(S)
0, 71

0
ωb ω

Figure 16.8: Sensitivity and complementary sensitivity of a MIMO system

High Gain and Low Gain Directions

At a given frequency ω0 , the maximum and minimum singular values σ̄(S(jω0 )) and
σ(S(jω0 )) represent the maximum and minimum gain of the transfer function S(s) over
all input directions - which are the maximum and minimum gains from a vector d of
sinusoidal disturbances to the output vector y. The singular value decomposition of
S(jω0 ) provides not only the gains but also the input directions associated with the
largest and smallest gain. From S(jω0 ) = U ΣV H follows that SV = U Σ and

S(jω0 )v1 = σ1 u1
112

Here v1 and u1 are the first columns of V and U , respectively. Assuming that the singular
values are ordered by magnitude, we have σ̄ = σ1 . The input direction associated with
the largest gain is therefore v1 , and the corresponding output direction is u1 . Similarly,
the input direction associated with the lowest gain is the column of V corresponding to
the smallest singular value.

Exercises

Problem 16.1
What is the largest approximation error when three weighting filters are used to express
constraints on S, T and KS for a SISO system, and the controller is designed to satisfy
 
WS S

 
 WT T 
  <1
WK KS

∞

Problem 16.2
Confirm the information about the steady state tracking capabilities of the LQG control
loop for the aircraft model obtained in Exercise 14.1: plot the singular values of the closed-
loop sensitivity over frequency. Use a singular value decomposition of the sensitivity at
low frequencies to find the "best" and the "worst" input directions.

Problem 16.3
Consider the following open loop transfer functions and the closed loop system in Figure
16.9, where G(s) is either
2 2−s
G1 (s) = or G2 (s) =
s(s + 1) s(s + 2)

e u
r K(s) G(s) y
-

Figure 16.9: Closed-loop system

a) For G1 (s) plot the sensitivity function |S| for K(s) = 0.1, K(s) = 1.0, K(s) = 10
and compare the bandwidth and the peak value of |S|.
16. Design Objectives and Sensitivity Functions 113

b) Plot the closed-loop step response of G1 (s) with K(s) = 0.1, K(s) = 1.0, K(s) = 10
and compare the maximum overshoot and the rise time.

c) For G2 (s) plot the sensitivity function |S| for K(s) = 0.1, K(s) = 0.5, K(s) = 1.5
and compare the bandwidth and the peak value of |S|.
Chapter 17

Mixed Sensitivity - A Design

Example

In this chapter we will design a multivariable controller for the aircraft model in example
11.1 that has been used in Exercises 11.1, 14.1, 15.1 and 16.2. We will use software
tools for designing controllers that satisfy constraints on the H∞ norm of the closed-loop
transfer function - how these software tools work will be studied in chapter 19. Here
we will discuss how the sensitivity functions can be shaped with the help of weighting
filters, and how available software tools can be used to obtain a controller that satisfies the
constraints. Examples are given to illustrate the effect of different choices for weighting
filters. The actual design is left as an exercise.

e u
r K(s) G(s) y
-

Figure 17.1: Closed-loop system

The closed-loop system is shown in Figure 17.1. Initially we consider the two design
objectives

• fast tracking of step changes for all three reference inputs, with little or no overshoot

• control input must satisfy |u3 | < 20

Scaling the Weighting Filters

It was shown in the previous chapter how weighting filters for loop shaping can be ex-
pressed in the form of the generalized plant in Figure 16.7. In our design we will shape the
116

sensitivity S and the control sensitivity KS, to achieve desired properties of the closed-
loop transfer function Tzw (s) from the reference input r to the fictitious output vector
z = [zST zKT T
] . The design will be carried out by choosing suitable weighting filters WS
and WK , and by imposing
" #
W S
S
kTzw k∞ = <1 (17.1)


WK KS
∞

Of course the closed-loop transfer function Tzw (s) depends on the controller K(s), and the
task is to find a controller that satisfies this constraint. If there are no other constraints on
the closed-loop system, one can in fact compute the controller that minimizes kTzw (s)k∞ .
Let
γo = min kTzw (s)k∞
K

denote the optimal value of the H∞ norm. This value may or may not be less than one. If
γo > 1, the constraints expressed by the weighting functions WS and WK are too strong
and a controller that satisfies them does not exist. In this case, the constraints must be
relaxed. On the other hand, if γo < 1, controllers that satisfy the constraints do exist and
the constraints can actually be strengthened.
A simple way of adjusting the weighting filters is scaling. Assume that the minimum
value of kTzw (s)k∞ is γo 6= 1 and that this value is achieved with the optimal controller
Ko (s). Introduce the scaled weighting functions
1 1
W̃S (s) = WS (s) and W̃K (s) = WK (s)
γo γo

Replace the weighting filters in (17.1) by W̃S and W̃K . The H∞ norm is minimized by
the same controller Ko (s) as before, and we have
" #
W̃ S
S
=1


W̃K KS
∞

This is illustrated in Design 1 below.

Choice of Weighting Functions - Sensitivity

We begin the design with scalar weighting filters wS (s) and wK (s), as discussed in the
previous chapter. For the loop to have integral action, a positive slope of 20 dB/dec of
the sensitivity is required at low frequencies. This could be enforced by including a factor
1/s in the weigthing function wS (s). However, it is clear from Figure 16.7 - and also from
(17.1) - that the weighting filters are factors of the closed-loop transfer function Tzw (s),
and if wS has a pole at the origin then the same is true for Tzw . Because the H∞ norm
is only defined for proper, stable transfer functions, the weighting filters must therefore
also be proper and stable.
17. Mixed Sensitivity - A Design Example 117

1
|wS |

MS

ωS =  ω
|wS |
1
MS

ωS ω

Figure 17.2: Weighting filter wS (s)

To enforce integral action in the loop, one can choose the weighting function to include a
factor
1
s+
where  > 0 is a small constant. Figure 17.2 shows the shape of a first order weighting
filter and its inverse. MS is a small constant that is chosen as an upper bound on the
sensitivity at low frequencies. The transfer function of the weighting filter is
ωS /MS
wS (s) = (17.2)
s + ωS
where ωS = .

Choice of Weighting Functions - Control Sensitivity

The weighting filter wK (s) can be used to impose an upper bound on the control sensi-
tivity. Because the control sensitivity should roll off at high frequencies, the inverse of
this filter should have low-pass behaviour and thus the filter itself should be a high-pass
filter. But, as mentioned before, weighting filters must be stable and proper, and if we
start with a zero at ωK (see Figure 17.3), an additional pole at a frequency well above
the bandwidth is required to make wK (s) proper. In Figure 17.3, this pole is placed at
cωK , where c is a sufficiently large constant.
From Figure 17.3, the filter transfer function is
c s + ωK
wK (s) = (17.3)
MK s + cωK
118

1
|wK |
MK

ωK cωK ω
|wK |

1
MK
ωK cωK ω

Figure 17.3: Weighting filter wK (s)

Loop Shaping

With the filter transfer functions (17.2) and (17.3), we have made choices concerning the
structure and order of the weighting functions WS and WK - we use the same scalar, first
order filters for all outputs. These choices are restrictive and can be changed if required.
They have however the advantage that there is only a small number of design parameters.
Initially we fix c = 103 (placing the pole of wK three decades above the zero). That leaves
us with the design parameters ωS , MS , ωK and MK .
The parameter ωS can be used to determine the closed-loop bandwidth, and MS can be
used to push the steady state error towards zero. The shape of wS according to (17.2)
is different from that discussed in the previous chapter (compare Figure 16.5), in so far
that we are not imposing an upper bound on |S|. The reason for this is that we can use
the upper bound MK on the control sensitivity to impose a limit on a sensitivity peak, as
will be seen below. The corner frequency ωK of WK should be chosen high enough not to
interfere with the bandwidth constraint on |S|.
17. Mixed Sensitivity - A Design Example 119

Following these considerations, we start the design procedure with an initial choice of
parameters

ωS MS ωK MK
Design 1:
10−4 10−4 102 103

The task is now to compute a controller that achieves kTzw k∞ < 1, where Tzw is a
generalized plant of the form shown in Figure 16.7 (without output zT ), with weighting
filters WS and WK based on (17.2) and (17.3) and parameter values as above. Here we
will use the function hinfsyn() of the robust control toolbox for MATLAB to find the
optimal controller - the controller that minimizes the H∞ norm of Tzw .

1.5
y1
1 y2
Output y

y3
0.5

−0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]
50

0
Input u

−50
u1
−100 u2
u3
−150
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]

Figure 17.4: Design 1: response to r(t) = [σ(t) 0 0]T

Since the aircraft model has three inputs and three outputs, the optimal controller Ko (s)
is a three-by-three transfer function matrix. We will later see that the dynamic order
of the optimal controller is equal to the dynamic order of the generalized plant. In the
present case, the plant is of fifth order, and we have first order weighting filters for each
of the three outputs of S and KS. This leaves us with a generalized plant of order 11,
and therefore with an 11th order controller.
The response of the closed-loop system in Figure 17.1 with the optimal controller Ko (s) to
h iT
a reference step input r(t) = σ(t) 0 0 is shown in Figure 17.4. The response is fast,
but the control input u3 violates the actuator constraint - this requires some adjustment.
First we note that the optimal controller Ko (s) achieves a closed-loop H∞ norm of less
120

than one; in fact we have

γo = min kTzw k∞ = 0.1794
K

in this case. Figure 17.5 shows the singular values of the sensitivity together with 1/|WS |,
and Figure 17.6 shows the control sensitivity together with 1/|WK |.
The fact that γo < 1 means that we can tighten the constraints. Introduce the scaled
weighting filters
1 1
W̃S (s) = WS (s) and W̃K (s) = WK (s)
γo γo
The optimal controller with these new filters is the same as before, but now we obtain
γo = 1. The inverse magnitude responses of the scaled filters - together with the sensitivity
functions - are plotted in Figures 17.7 and 17.8. They show that at low frequencies the
active constraint is kWS Sk∞ < 1, and at high frequencies kWK KSk∞ < 1. At frequencies
where a constraint is active, the constrained function (S or KS) fits tightly the shape of
the filter.
Figure 17.8 shows also that at low frequencies, the control sensitivity KS is constant
and slightly greater than one. The corner frequency where - moving from higher to
lower frequencies - the magnitude of KS starts to decrease from its enforced peak value,
is roughly the frequency where the magnitude of S reaches one. It is the closed-loop
bandwidth, which is enforced by the weighting filter WS . The reason for the constant
low-frequency value of |KS| is the near-integral behaviour of the controller K(s). The
constraint on |S| leads to a controller with poles very close to the origin in all controller
channels. At low frequencies, the controller gain is therefore very large, and the inverse
in KS = K(I + GK)−1 is dominated by GK. As a result, the maximum singular value
of KS at low frequencies is approximately equal to the inverse of the smallest singular
value of G. In the present case, the smallest singular value of the aircraft model transfer
function at low frequencies is slightly less than one.
17. Mixed Sensitivity - A Design Example 121

150
σ 1 (S)
σ 2 (S)
100 σ 3 (S)
1/wS
Singular Values [dB]

50

−50

−100 −3
10 10−2 10−1 100 101 102 103 104 105 106 107
Frequency [rad/sec]

Figure 17.5: Design 1: Sensitivity S and constraint

60
σ 1 (KS)
40 σ 2 (KS)
σ 3 (KS)
20
1/wK
0
Singular Values [dB]

−20

−40

−60

−80

−100

−120

−140 −3
10 10−2 10−1 100 101 102 103 104 105 106 107
Frequency [rad/sec]

Figure 17.6: Design 1: Control sensitivity KS and constraint

122

150
σ 1 (S)
σ 2 (S)
100 σ 3 (S)
γ 0 /wS
Singular Values [dB]

50

−50

−100 −3
10 10−2 10−1 100 101 102 103 104 105 106 107
Frequency [rad/sec]

Figure 17.7: Design 1: Sensitivity S and scaled constraint

60
σ 1 (KS)
40 σ 2 (KS)
σ 3 (KS)
20
γ 0 /wK
0
Singular Values [dB]

−20

−40

−60

−80

−100

−120

−140 −3
10 10−2 10−1 100 101 102 103 104 105 106 107
Frequency [rad/sec]

Figure 17.8: Design 1: Control sensitivity KS and scaled constraint

17. Mixed Sensitivity - A Design Example 123

We now return to the problem of re-tuning the controller such that it meets the actuator
constraint |u3 | < 20. We need to reduce the upper bound MK on the control sensitivity
KS. We try MK = 102 instead of 103 and leave the other parameters unchanged, so our
next attempt is

ωS MS ωK MK
Design 2:
10−4 10−4 102 102

1.5
y1
1 y2
Output y

y3
0.5

−0.5
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]
10

0
Input u

−10
u1
−20 u2
u3
−30
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]

Figure 17.9: Design 2: response to r(t) = [σ(t) 0 0]T

The step response with the resulting controller is shown in Figure 17.9. Clearly the peak
value of the elevator angle u3 has been significantly reduced, although it is still slightly
too large. The sensitivity and control sensitivity - together with the scaled weightings -
are shown in Figures 17.10 and 17.11. Notice that the closed-loop bandwidth has been
reduced, even though we did not change the weight WS . This is because the scaled
weightings are shown - the H∞ norm achieved with the unscaled weights is now γo =
0.3805, and the scaled filter W̃S represents now a lower bandwidth. Intuitively it should
be clear that reducing the control effort (reducing MK ) leads to a slower response.
Because we are minimizing kTzw k∞ rather than rigidly enforcing kTzw k∞ < 1, the func-
tions WS and WK act as weights - or "soft constraints" - rather than hard constraints. The
effect of reducing MK is illustrated in Figure 17.12. The curves labelled 1 represent scaled
constraints (dashed) and sensitivity shapes (solid) of the previous design. The dashed line
labelled 2 shows the constraint on KS when MK is reduced. Because the constraints were
scaled for the previous design, the new design will give a controller with γo > 1, which
124

150
σ 1 (S)
σ 2 (S)
100 σ 3 (S)
γ 0 /wS
Singular Values [dB]

50

−50

−100 −3
10 10−2 10−1 100 101 102 103 104 105 106 107
Frequency [rad/sec]

Figure 17.10: Design 2: Sensitivity S and scaled constraint

40
σ 1 (KS)
20 σ 2 (KS)
σ 3 (KS)
0
γ 0 /wK
−20
Singular Values [dB]

−40

−60

−80

−100

−120

−140

−160 −3
10 10−2 10−1 100 101 102 103 104 105 106 107
Frequency [rad/sec]

Figure 17.11: Design 2: Control sensitivity KS and scaled constraint

means the constraints will be violated. The violation of the constraints is spread equally
over S and KS, indicated by the solid lines labelled 2. Rescaling the constraints then
leads to the plots in Figures 17.10 and 17.11.
Returning to the step response in Figure 17.9, we noted that the control effort needs to be
slightly reduced to meet the design specification. Moreover, the response to a reference
17. Mixed Sensitivity - A Design Example 125

|S| |KS|
1 1
0 dB
2
1 2 2

ω ω

Figure 17.12: "Soft constraints" on sensitivity and control sensitivity: effect of reducing
MK

step change in y1 leads to a perturbation in y3 . To improve tracking of r3 , we increase the

weight on the corresponding channel in WS and make the following adjustment to obtain
Design 3:
   
w (s) 0 0 w (s) 0 0
 S   S 
WS (s) =  0
 wS (s) 0   →  0
 wS (s) 0  
0 0 wS (s) 0 0 10wS (s)

1.5
y1
1 y2
Output y

y3
0.5

−0.5
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]
5

0
Input u

−5
u1
−10 u2
u3
−15
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]

Figure 17.13: Design 3: response to r(t) = [σ(t) 0 0]T

The step response with the resulting controller is shown in Figure 17.13. The perturbation
of y3 in the response has disappeared, and |u3 | < 13. This means that the controller is
not using the full range available, and the response can be made faster by increasing the
closed-loop bandwidth. With ωS = 2.5 · 10−4 we obtain
126

1.5
y1
1 y2

Output y
y3
0.5

−0.5
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]
10

0
Input u

−10
u1
−20 u2
u3
−30
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]

Figure 17.14: Design 4: response to r(t) = [σ(t) 0 0]T

ωS MS ωK MK
Design 4:
2.5 · 10−4 10−4 102 102

where WS = diag (wS , wS , 10wS ) and WK = diag (wK , wK , wK ). Figure 17.14 shows that
the tracking performance is satisfactory.

Input Disturbances

The sensitivity S is the transfer function from an output disturbance to the controlled
output, and when |S| is made small at low frequencies, the closed-loop system has a built-
in capability of good rejection of output disturbances. The same is not automatically
guaranteed for disturbances occurring at the plant input, see Figure 17.15. In fact, even
though the response to a reference step achieved with Design 4 is excellent, the closed-loop
system shows poor rejection of input disturbances - the response to a disturbance step
applied at plant input is shown in Figure 17.16.
The surprisingly poor response to an input step disturbance reveals a major weakness
of the loop shaping approach: we are trying to shape the closed-loop dynamics without
taking the open-loop dynamics into account. Often the only way to satisfy the constraints
imposed on the closed-loop transfer function is to cancel undesirable plant poles by con-
troller zeros. Figure 17.17 shows that this is exactly what happened here: shown are
open-loop poles close to the imaginary axis - with a low damping ratio - that have been
cancelled by transmission zeros of the controller.
17. Mixed Sensitivity - A Design Example 127

e
r K(s) G(s) y
-

Figure 17.15: Input disturbance

2
y1
1 y2
Output y

y3
0

−1

−2
0 10 20 30 40 50 60 70 80 90 100
Time [sec]
1
u1
u2
Control u

0.5 u3

−0.5
0 10 20 30 40 50 60 70 80 90 100
Time [sec]

Figure 17.16: Design 4: response to input disturbance d(t) = [σ(t) 0 0]T

Because input disturbances are likely to occur - in the present example they could be
caused by hysteresis and friction at the aircraft control surfaces - the design needs to be
modified to improve its disturbance rejection. One way to deal with this problem is to
consider the transfer function from the input disturbance d to the controlled output y.
We have (assuming r = 0)

y = G(d − Ky) thus (I + GK)y = Gd

and
y = (I + GK)−1 Gd = SGd
Figure 17.18 shows the singular values of the transfer function SG - it clearly displays the
resonant peak responsible for the poor response in Figure 17.16.
The problem is that the pole-zero cancellations are not visible when examining the transfer
functions S = (I +GK)−1 and T = (I +GK)−1 GK, but are visible in SG = (I +GK)−1 G.
This can be seen most easily by considering the SISO version of this problem. Using the
128

1.5

0.5

Imaginary Axis
0

−0.5

−1

−1.5
−1 −0.9 −0.8 −0.7 −0.6 −0.5 −0.4 −0.3 −0.2 −0.1 0 0.1
Real Axis

Figure 17.17: Pole-zero cancellation in Design 4

50
σ 1 (SG)
σ 2 (SG)
0
σ 3 (SG)

−50
Singular Values [dB]

−100

−150

−200

−250

−300 −3
10 10−2 10−1 100 101 102 103 104 105 106 107
Frequency [rad/sec]

Figure 17.18: Design 4: Singular values of SG

notation
ng (s) nk (s)
G(s) = , K(s) =
dg (s) dk (s)
we have
G ng dk
SG = =
1 + GK dg dk + ng nk
A pole-zero cancellation means that the polynomials dg and nk have a common factor, so
17. Mixed Sensitivity - A Design Example 129

we can write
dg = d¯g d˜g and nk = n̄k d˜g
where d˜g contains the plant poles that are cancelled by controller zeros. Substituting this
in the above yields
ng dk
SG = ˜ ¯
dg (dg dk + ng n̄k )
which shows that the cancelled plant poles turn up as poles of the transfer function from
d to y.

W2 (s) z2
d

y
K(s) G(s) W1 (s) z1
- u

Figure 17.19: Generalized plant for input disturbance rejection

To prevent such undesirable pole-zero cancellations, we can try to shape the transfer
function SG. Figure 17.19 shows one way of doing this: the weighting filter W1 shapes
SG, and a second filter W2 shapes the transfer function from d to the controller output
u. It is straightforward to check that the transfer function from d to u is

TK = −(I + KG)−1 KG

This function has the same structure as the complementary sensitivity, only the loop
gain GK is replaced by KG - the loop gain seen from the controller output. Defining
z = [z1T z2T ]T , we can compute the controller that minimizes the H∞ norm of the closed-
loop transfer function from d to z. With the choices

ω1 /M1 c s + ω2
w1 (s) = and w2 (s) =
s + ω1 M2 s + cω2

where we fix again c = 103 , and

ω1 M1 ω2 M2
Design 5:
10−3 10−7 102 2

we obtain the response shown in Figure 17.20 to an input disturbance step. The singular
values of SG and TK together with the scaled constraints are plotted in Figures 17.22
and 17.23. The plots show that the constraint on SG is active at low frequencies and the
constraint on TK near the corner frequency of W2 (s).
130

The setup in Figure 17.19 does not consider the reference input r. Therefore, one cannot
constrain the sensitivity S explicitly, but only as a factor in SG. So we would expect
the tracking performance to be inferior to that in Design 4. This is confirmed by the
response to a reference step input, shown in Figure 17.21. The control effort needs to be

·10−3
1
y1
y2
Output y

0.5 y3

−0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]
1
u1
u2
Control u

0.5 u3

−0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]

Figure 17.20: Design 5: response to input disturbance d(t) = [σ(t) 0 0]T

1.5
y1
1 y2
Output y

y3
0.5

−0.5
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]
·104
0.5

0
Input u

−0.5 u1
u2
u3
−1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]

Figure 17.21: Design 5: response to r(t) = [σ(t) 0 0]T

17. Mixed Sensitivity - A Design Example 131

considerably reduced to meet the design specification. The final design for this problem
is left as an exercise at the end of the chapter. The 4-block design is presented to achieve
both good reference tracking and disturbance rejection with a reasonable control effort.

50

0
Singular Values [dB]

−50

−100

−150

σ 1 (SG)
−200 σ 2 (SG)
σ 3 (SG)
γ 0 /w1
−250 −4
10 10−3 10−2 10−1 100 101 102 103 104 105 106
Frequency [rad/sec]

Figure 17.22: Design 5: Singular values of SG and scaled constraint

50

−50
Singular Values [dB]

−100

−150

−200
σ 1 (Tk )
σ 2 (Tk )
−250 σ 3 (Tk )
γ 0 /w2
−300 −4
10 10−3 10−2 10−1 100 101 102 103 104 105 106
Frequency [rad/sec]

Figure 17.23: Design 5: Singular values of TK and scaled constraint

132

Constructing the Generalized Plant

The actual computation of the controller in the examples in this chapter was carried out
with the function hinfsyn() of the robust control toolbox for MATLAB. The design
constraints are entered in the form of generalized plant models. The construction of the
generalized plant is illustrated here for the first four designs with constraints on S and
KS.
The generalized plant is shown in Figure 17.24. Unlike the LQG example in chapter 11,
the generalized plant P contains not only the dynamics of the physical plant
" #
A B
G(s) =
C 0

but also the dynamics of the weighting filters. It is convenient to construct first a gener-
alized plant without the filters. Define z̃S and z̃K as the signals that enter the weighting
filters, i.e. z̃S = e and z̃K = u, and consider the transfer function P̃ obtained by replacing
z with z̃  
z̃
 S
" #
z̃K  = P̃ r
  u
e

u y e
G(s) - WS (s) zS

WK (s) zK

P (s)

K(s)

Figure 17.24: Generalized plant model for Design 1 - 4

From Figure 17.24, one finds that

 
A 0 B
" # " # " # 
−C I 0 


P̃ (s) =  

 0 0 I 
−C I 0
17. Mixed Sensitivity - A Design Example 133

The generalized plant P is then easily obtained as

 
w (s) 0 0
 S
" #
 AP BP
P (s) = 
 0 wK (s) 0 · P̃ (s) = CP DP
0 0 I
The state space model (AP , BP , CP , DP ) can then be used as input argument to the
function hinfsyn().

Exercises

Problem 17.1
Consider the configuration shown in Figure 17.25, where
200 1 10
G(s) = 2
; Gd (s) =
10s + 1 (0.05s + 1) 10s + 1
are the transfer functions to y from u and d respectively. The control objectives are

0.05

r u 1 200 y
K(s) (0.05s+1)2 10s+1
-

Figure 17.25: System block diagram

1. Command tracking: The rise time should be less than 0.3 sec. and the overshoot
should be less than 5%.

2. Disturbance rejection: The output in response to a unit step disturbance should

remain within the range [−1, 1] at all times, and it should return to 0 as quickly as
possible (|y(t)| should at least be less than 0.1 after 3 sec.).

3. Input constraints: u(t) should remain within the range [−1, 1] at all times to avoid
input saturation (this is easily satisfied for most designs).

a) Try to design a controller that achieves these objectives.

Hint: Minimise " #
W S
S

WK KS
∞
134

b) Repeat the design, but this time with 10 times stronger disturbance:
Gd (s) = 100/(10s + 1) (gain of 0.5 in disturbance gain block in Figure 17.25).

Problem 17.2

a) Design a controller for the aircraft model in example 11.1 that achieves good tracking
properties. Initially, ignore input disturbances and measurement noise, and tune
your controller for a fast response to reference step inputs. What is the fastest
response that can be achieved within the given actuator constraints |u3 | < 20?

b) Check the noise sensitivity of your design. You can do this by injecting a sinusoidal
input at 104 rad/sec into the feedback signals. Improve the noise sensitivity if
required.

c) Check the rejection of input disturbance steps. Modify the design to improve input
disturbance rejection.
Chapter 18

Design Specifications as LMI

Constraints

So far we discussed the use of weighting functions for shaping the closed-loop dynamics
via H∞ constraints, and we assumed the availability of software tools for computing the
controller. Similarly, in chapter 11 we discussed how the LQG problem can be reformu-
lated in terms of a generalized plant, and we used available software tools for computing
a LQG controller in Exercise 11.1. We used in fact the same function hinfsyn() of the
robust control toolbox for MATLAB that is used in Exercise 17.2 for H∞ loop shaping.
Both H∞ constraints and H2 constraints can be expressed in the form of linear matrix
inequalities (LMI), and efficient LMI solvers can be employed to compute controllers that
satisfy these constraints. In this chapter we will study how to express design specifications
in the form of linear matrix inequalities. The next chapter will then introduce methods
for computing controllers by solving LMI problems.
We will discuss three types of design specifications: constraints on the H2 and the H∞
norm, and constraints on the location of closed-loop poles in the complex plane.

Linear Matrix Inequalities

We begin with a brief review of linear matrix inequalities. A linear matrix inequality
(LMI) has the form

M (p) = M0 + p1 M1 + · · · + pN MN < 0 (18.1)

h iT
where M0 , M1 , . . . , MN are given symmetric matrices, p = p1 p2 . . . pN is a column
vector of real scalar variables (the decision variables), and the matrix inequality M (p) < 0
means that the left hand side is negative definite. An important property of LMIs is that
the set of all solutions p is convex.
136

Linear matrix inequalities can be used as constraints for the minimization problem
min
p
cT p subject to M (p) < 0 (18.2)

where the elements of the vector c in the linear cost function are weights on the individual
decision variables. This problem is convex and can be solved by efficient, polynomial-time
interior-point methods. Several LMI constraints can be combined into a single constraint
of type (18.1). For example the constraint
M1 (p) < 0 and M2 (p) < 0
is equivalent to the single LMI constraint
" #
M1 (p) 0
<0 (18.3)
0 M2 (p)
The problem (18.2) is quite general, and a variety of problems can be reduced to this
form. It has been known for a long time that certain control problems can be expressed
in this way. Already in 1971 the question was raised whether linear matrix inequalities
can be exploited for numerical purposes in control engineering, but it took more than
20 years for this to happen. It became possible when interior-point methods for solving
LMI problems were developed in the the 1980s. Control engineers became aware of these
developments in the early 1990s, standard software tools were made available in the mid
1990s, and from then on LMI techniques developed into an active area of research.
Optimization problems of the form (18.2) will be used here to search for the best controller
- in the sense of a given performance index - that satisfies LMI constraints on the closed-
loop system shown in Figure 18.1.

w z
P
u v

Figure 18.1: Generalized plant

w T z

Figure 18.2: Closed-loop system

In this chapter, we will only consider constraints on the transfer function from w to z,
without discussing its dependence on the controller K(s) - so we will consider the system
shown in Figure 18.2. How to construct controllers that satisfy the constraints will be the
subject of the following chapter.
18. Design Specifications as LMI Constraints 137

Im s

Damping
ratio
Admissible β
Re s
αl Pole Region

αr
Speed of Response

Figure 18.3: Desired pole region

Pole Region Constraints

It is well known that the dynamic properties of a linear, time-invariant system are de-
termined by the location of the poles of its transfer function in the complex plane. For
second order systems, the relationship between the pole location and damping ratio, rise
time and settling time is particularly simple. For higher order systems these relations
are still useful because system dynamics are often governed by a dominant pole pair. In
practice the designer often works on specifications that include a minimum damping ratio
and a minimum speed of response, and the objective is to find the best controller (in terms
of a suitable performance index) that satisfies these constraints. These constraints can be
expressed as a region in the complex plane where the closed-loop poles should be located,
as shown in Figure 18.3. A minimum speed of response is represented by a right boundary
on the admissible pole region at −αr , and a minimum damping ratio by a conic sector.
To avoid excessive control action and high noise sensitivity, the closed-loop bandwidth
can be limited by placing a left boundary at −αl on the admissible pole region.
The condition that the poles of a system are located within a given region in the complex
plane can be formulated as an LMI constraint. As an example, consider the dynamic
system
ẋ(t) = Ax(t) (18.4)
This system is stable if and only if the matrix A has all eigenvalues in the left half plane,
which in turn is true if and only if there exists a positive definite, symmetric matrix P
that satisfies the Lyapunov inequality

P AT + AP < 0 (18.5)

This inequality is linear in the matrix variable P , and one can use LMI solvers to search
for solutions. It is straightforward to rewrite (18.5) in the standard form (18.1) of an LMI.
To see this, assume that A is a 2 by 2 matrix and write the symmetric matrix variable P
138

as " # " # " # " #

p p 1 0 0 1 0 0
P = 1 2 = p1 + p2 + p3
p2 p3 0 0 1 0 0 1
Substitution of the right hand side for P in (18.5) yields an LMI in the form of (18.1)
with three decision variables p1 , p2 and p3 . Available software tools operate directly on
matrix variables, so that it is usually not necessary to carry out this transformation.

Im Im

Re Re
αl αr

Figure 18.4: LMI regions

The LMI (18.5) represents a necessary and sufficient condition for the matrix A to have all
eigenvalues in the left half plane. It is possible to generalize this result: one can express
an arbitrary region D in the complex plane such as the one shown in Figure 18.3 (as long
as it is convex and symmetric about the real axis) in terms of two matrices L = LT and
M as the set of all complex numbers that satisfy an LMI constraint
| : L + M s + M T s̄ < 0}
D = {s ∈ C (18.6)
where s̄ denotes the complex conjugate of s. Such a region is called an LMI region. One
can show that a necessary and sufficient condition for a matrix A to have all eigenvalues
in D is the existence of a positive definite, symmetric matrix P that satisfies
L ⊗ P + M ⊗ (AP ) + M T ⊗ (AP )T < 0 (18.7)
The symbol ⊗ stands for the Kronecker product: if M is a 2 by 2 matrix then
" # " #
m11 m12 m11 P m12 P
M ⊗P = ⊗P =
m21 m22 m21 P m22 P
Thus if P is also 2 by 2 the Kronecker product is 4 by 4. Inequality (18.7) is an LMI in
the matrix variable P . It is easy to see that the Lyapunov inequality (18.5) is obtained
as a special case of (18.7) with L = 0 and M = 1 and D as the left half plane. One can
verify that the conic sector shown in Figure 18.4 is an LMI region with
" # " #
0 0 cos β − sin β
Lc = , Mc = (18.8)
0 0 sin β cos β
18. Design Specifications as LMI Constraints 139

and the vertical strip is an LMI region with

" # " #
2αl 0 −1 0
Lv = , Mv = (18.9)
0 −2αr 0 1

The intersection of both regions is the pole region shown in Figure 18.3; it can be repre-
sented as an LMI region by combining the two constraints as in (18.3) with
" # " #
L 0 Mc 0
L= c , M= (18.10)
0 Lv 0 Mv

Constraints on the H2 Norm

Consider a stable system with state space realization

ẋ(t) =Ax(t) + Bw(t)

z(t) =Cx(t) (18.11)

We will later interpret this system as the closed-loop system in Figure 18.2. Let T (s)
denote the strictly proper transfer function matrix from w to z

T (s) = C(sI − A)−1 B

It was shown in chapter 15 that the H2 norm of T (s) can be computed from

kT k22 = trace CP0 C T (18.12)

where P0 is the positive definite matrix that solves the Lyapunov equation

AP0 + P0 AT + BB T = 0 (18.13)

Now assume that we want to impose a constraint on kT k2 , say

kT k2 < ν

for some ν > 0. To see how (18.12) and (18.13) can be used to express this constraint,
consider the solution P to the following modified version of (18.13)

AP + P AT + BB T + Q = 0 (18.14)

where Q > 0 is a positive definite matrix. Subtracting (18.13) from (18.14) gives

A(P − P0 ) + (P − P0 )AT + Q = 0

Because Q > 0 and A is stable, this implies P − P0 > 0 or

P > P0 (18.15)
140

The reverse is also true: if (18.15) holds, then a matrix Q > 0 exists that satisfies (18.14).
Since the above holds for any Q > 0, we can replace (18.14) by the matrix inequality

AP + P AT + BB T < 0 (18.16)

and we have from (18.15) and (18.12) that

kT k22 < trace CP C T (18.17)

if and only if P satisfies (18.16). The above results can be summarized as

Theorem 18.1 For the system (18.11), kT k2 < ν if and only if there exists a positive
definite matrix P that satisfies (18.16) and

trace CP C T < ν 2 (18.18)

In the next chapter it will be shown how to compute controllers that satisfy constraints
on kT k2 . The system matrices A, B and C will then be closed-loop system matrices that
contain controller parameters, and the terms BB T and trace CP C T will be nonlinear in
these parameters. The following equivalent formulation of the above result is then used
instead. Introduce a new symmetric matrix variable W (which is used as a slack variable),
then we have

Theorem 18.2 kT k2 < ν if and only if there exist symmetric matrices P and W that
satisfy
" # " #
AP + P AT B W CP
< 0, >0 and trace W < ν 2 (18.19)
BT −I P CT P

To see that the inequalities in (18.19) are equivalent to (18.18) and (18.16), note that
" #
M L
>0
LT N

where M = M T and N = N T , is equivalent to

N >0 and M − LN −1 LT > 0

This fact is frequently used to convert nonlinear inequalities into LMI form; the term
M − LN −1 LT is the Schur complement with respect to N . In the above case, this leads
to W > CP C T and consequently to trace W > trace CP C T , from which the equivalence
of (18.18) and (18.19) follows.
18. Design Specifications as LMI Constraints 141

Constraints on the H∞ Norm

Consider again the system in Figure 18.2 with transfer function T (s) and state space
realization
ẋ(t) =Ax(t) + Bw(t), x(0) = 0
z(t) =Cx(t) + Dw(t) (18.20)
Assuming that T (s) is stable, the H∞ norm of the system is
R∞
0 z T (t)z(t) dt
kT k2∞ = max R
w6=0 ∞
0 wT (t)w(t) dt
where we also assume that x(0) = 0. From the above it follows that kT k∞ < γ is
equivalent to Z ∞
(z T (t)z(t) − γ 2 wT (t)w(t)) dt < 0
0
holding true for all square integrable, non-zero w(t). Now introduce a Lyapunov function
V (x) = xT P x with P = P T > 0. Because x(0) = x(∞) = 0, the constraint kT k∞ < γ is
then enforced by the existence of a matrix P = P T > 0 such that
dV (x) 1 T
+ z (t)z(t) − γwT (t)w(t) < 0 (18.21)
dt γ
for all x(t), w(t); this can be seen by integrating (18.21) from t = 0 to t = ∞. To turn
(18.21) into a linear matrix inequality, substitute
dV (x)
= xT (AT P + P A)x + xT P Bw + wT B T P x
dt
and z = Cx + Dw in (18.21) to obtain
 " #
h i AT P + P A + γ1 C T C P B + γ1 C T D x
xT w T   <0
B T P + γ1 DT C −γI + γ1 DT D w

For kT k∞ < γ the above must hold for all x and w, thus the block matrix must be
negative definite. This condition can be rewritten as
" # " #
AT P + P A P B 1 CT h i
+ C D <0
BT P −γI γ DT
It can be shown that the existence of a solution to the above LMI is in fact not only a
sufficient but also a necessary condition for kT k∞ < γ. Using the Schur complement, we
can rewrite this condition and summarize the result as follows.

Theorem 18.3 kT k∞ < γ if and only if there exists a positive definite, symmetric
matrix P that satisfies the linear matrix inequality
 
AT P + P A P B C T
BT P −γI DT 
 
<0 (18.22)


C D −γI
142

Exercises

Problem 18.1
Show that the matrices L = LT and M from (18.6) define the LMI region D(αr , αl , β)
shown in Figure 18.3.

Problem 18.2
Write MATLAB programs computing the H∞ norm and the H2 norm of a system using
the LMI conditions (18.22) and (18.19). Use the “LMI lab” toolbox of MATLAB to
express the LMI conditions. Verify your results by computing the H2 and H∞ norms
of the closed-loop system T (s) in Exercise 11.1 and compare the results with the results
obtained by the norm function (see Exercise 15.1).
Hint: To get started see either the help of rctlmi or the documentation of LMI lab in
“Robust Control Toolbox”, “Linear Matrix Inequalities”, “The LMI lab” in the MATLAB
help.
Chapter 19

Controller Design Using Linear

Matrix Inequalities

In the previous chapter it was shown how design specifications can be expressed as linear
matrix inequality constraints. In this chapter we will see how one can compute controllers
that satisfy these constraints.
We will now interpret the plant model on which constraints were placed in the previous
chapter, as the closed loop system in Figure 19.1.

w z
P
u v

Figure 19.1: Generalized plant

Recall that the generalized plant P (s) has a state space realization

ẋ(t) = Ax(t) + Bw w(t) + Bu u(t)

z(t) = Cz x(t) + Dzw w(t) + Dzu u(t)
v(t) = Cv x(t) + Dvw w(t) (19.1)

where w(t) represents external inputs (reference input, disturbances and noise), u(t) rep-
resents the control input, z(t) is a vector of fictitious output signals used to assess the
performance of the control system, and v(t) is the measured output that is used for feed-
back control. The type of controller considered here is dynamic, linear time-invariant
144

output feedback control. The controller dynamics are represented by a state space model

ζ̇(t) = AK ζ(t) + BK v(t)

u(t) = CK ζ(t) + DK v(t) (19.2)

To use the LMI constraints in the previous chapter for controller synthesis, they must
be applied to the closed-loop system matrices. It is straightforward to check that a state
space realization of the closed loop-system in Figure 19.1 is

ẋc (t) = Ac xc (t) + Bc w(t)

z(t) = Cc xc (t) + Dc w(t) (19.3)

where
" # " #
A + Bu DK Cv Bu CK Bw + Bu DK Dvw
Ac = , Bc =
BK Cv AK BK Dvw
h i
Cc = Cz + Dzu DK Cv Dzu CK , Dc = Dzw + Dzu DK Dvw (19.4)

Let the transfer function from w to z of the closed-loop system be

" #
Ac Bc
T (s) = = Cc (sI − Ac )−1 Bc + Dc
Cc Dc

The matrices (Ac , Bc , Cc , Dc ) depend on the controller K(s). The problem considered in
this chapter is to design a controller such that constraints on the eigenvalue locations of
Ac and on kT k2 and kT k∞ are satisfied.

Linearizing Change of Variables - State Feedback

Three LMI constraints were discussed in the previous chapter:

• a pole region constraint in the form of matrix inequality (18.7) which is linear in
the matrix variable P

• a bound on the H2 -norm in the form of matrix inequalities (18.16) and (18.19) which
are linear in P and W

• a bound on the H∞ -norm in the form of matrix inequality (18.22) which is linear
in P .

When the system matrices A, B, C and D in these inequalities are replaced by the
closed-loop matrices Ac , Bc , Cc and Dc , the controller matrices AK , BK , CK and DK
appear as variables, and the resulting inequalities are not linear because they contain
products of variables. As a consequence, they cannot be solved with LMI solvers. One
19. Controller Design Using Linear Matrix Inequalities 145

can however introduce a change of variables to transform the constraints into equivalent
LMI constraints which can be solved. This transformation is much simpler in the special
case where all state variables are measured and available for feedback (i.e. the special case
Cv = I) and where the controller has the form of static state feedback
u(t) = F x(t)
This corresponds to the choice of controller matrices AK = 0, BK = 0, CK = 0 and
DK = F .
In this section, a full discussion of the state feedback case is given to illustrate the lin-
earizing change of variables, and in the following section an outline is given for the output
feedback case.

Pole Region Constraints

Consider first the LMI condition for the homogenous system (18.4): the existence of a
symmetric matrix P that satisfies the Lyapunov inequality
P AT + AP < 0
is a necessary and sufficient condition for the system to be stable. In order to use this
condition to search for stabilizing state feedback controllers, the system matrix A can be
replaced by the system matrix of the closed-loop system
ẋ(t) = (A + Bu F )x(t)
The Lyapunov inequality becomes
P (A + Bu F )T + (A + Bu F )P < 0
and because the variables are now P and F , this inequality is nonlinear in the variables.
It is however easy to make it linear - introduce a new variable Y = F P to obtain
P AT + AP + Y T BuT + Bu Y < 0
which is linear in the variables P and Y and can be solved with LMI solvers. Moreover, the
set of all solution pairs (P, Y ) represents the set of all stabilizing state feedback controllers
F = Y P −1 .
The same variable transformation can be applied to general pole region constraints in the
form of (18.7): a necessary and sufficient condition for a state feedback controller u = F x
to place the closed-loop poles inside an LMI region characterized by two matrices L and
M , is the existence of matrices P = P T > 0 and Y that satisfy the LMI
L ⊗ P + M ⊗ (AP + Bu Y ) + M T ⊗ (AP + Bu Y )T < 0 (19.5)
Moreover, the set of all solution pairs (P, Y ) represents the set of all state feedback
controllers F = Y P −1 that place the closed-loop poles inside the specified region.
146

H2 Performance

It was shown in the previous chapter that the H2 -norm of the system (18.11) is less than
ν if and only if matrices P = P T > 0 and W = W T exist that satisfy
" #
T T W CP
AP + P A + BB < 0, >0 and trace W < ν 2 (19.6)
P CT P
To use these conditions as constraints for controller synthesis, the system matrices A, B
and C must be replaced by the matrices of the closed-loop system. The system (18.11)
does not have a control input; if the system (19.1) is considered instead with Dzw = 0,
Dvw = 0 and Cv = I, the closed-loop system under state feedback u = F x becomes
ẋ(t) = (A + Bu F )x(t) + Bw w(t)
z(t) = (Cz + Dzu F )x(t)
If A and C are replaced in (19.6) by (A + Bu F ) and (Cz + Dzu F ), respectively, the
resulting inequalities are nonlinear in the variables P , W and F , but they can again be
made linear by introducing Y = F P as a new variable. It follows that a necessary and
sufficient condition for a state feedback controller to achieve a H2 -norm less than ν is the
existence of matrices P = P T > 0, W = W T and Y that satisfy the LMIs
P AT + AP + Y T BuT + Bu Y + Bw BwT < 0 (19.7)
" #
W Cz P + Dzu Y
>0 (19.8)
P CzT T T
+ Y Dzu P
and trace W < ν 2
and the set of all solutions (P, W, Y ) represents the set of all state feedback controllers
F = Y P −1 that achieve this norm constraint.

H∞ Performance

The H∞ constraint (18.22) can be used as a constraint for controller synthesis in the same
way as the H2 constraint. It is however convenient to rewrite it as
 
AP + P AT B P CT
BT −γI DT 
 
<0 (19.9)


CP D −γI

To verify that this condition is equivalent to (18.22), introduce P̃ = P −1 and multiply

(18.22) from left and right by diag(P̃ , I, I). In (19.9) the tilde has been dropped.
State feedback u = F x applied to the system (19.1) yields the closed-loop system
ẋ(t) = (A + Bu F )x(t) + Bw w(t)
z(t) = (Cz + Dzu F )x(t) + Dzw w(t)
19. Controller Design Using Linear Matrix Inequalities 147

Replacing the system matrices in (19.9) by the closed-loop matrices and using again the
variable transformation Y = F P leads to the following result: a necessary and sufficient
condition for a state feedback controller to achieve a H∞ -norm less than γ is the existence
of matrices P = P T > 0 and Y that satisfy
 
AP + P AT + Bu Y + Y T BuT Bw P CzT + Y T Dzu
T

BwT T
 

 −γI Dzw <0
 (19.10)
Cz P + Dzu Y Dzw −γI

and the set of all solutions (P, Y ) represents the set of all state feedback controllers
F = Y P −1 that achieve this norm constraint.

Linearizing Change of Variables - Output Feedback

When a dynamic controller (19.2) is used instead of state feedback in the configuration
shown in Figure 11.1, the resulting matrix inequality constraints on the closed-loop system
are again nonlinear in the variables. As in the state feedback case, they can be transformed
into equivalent LMI constraints by a change of variables. However, whereas with state
feedback the introduction of a new variable Y = F P was all that was required to make
the constraints linear, the situation is more complex in the output feedback case, and in
this section only an outline of the approach is presented.

Pole Region Constraints

Consider the pole region constraint

L ⊗ P + M ⊗ (P Ac )T + M T ⊗ (P Ac ) < 0 (19.11)

for the closed-loop system matrix

" #
A + Bu DK Cv Bu CK
Ac =
BK Cv AK

to have its eigenvalues in D, where D is an LMI region characterized by the two matrices
L and M as in (18.6). In (19.11) Ac has been replaced by ATc (which has the same
eigenvalues). The matrix variables are P and the controller matrices AK , BK , CK and
DK ; the inequality is nonlinear because the term P Ac contains products of variables. In
the following it will be assumed that the controller has the same order as the plant, i.e.
AK has the same size as A. Now partition P and P −1 as
" # " #
X R −1 Y S
P = , P = T
RT ∗ S ∗
148

where dimensions are compatible with the block matrix Ac , X and Y are symmetric and
(∗) denotes a matrix block that is of no particular interest. One can verify that
" # " #
Y I I X
P TY = TX where TY = T , TX = (19.12)
S 0 0 RT

The matrices TX and TY can be used to transform the nonlinear constraint (19.11) into
a linear one. This transformation is based on the fact that
" #
AY + Bu C̃K A + Bu D̃K Cv
TYT P Ac TY = TXT Ac TY = (19.13)
ÃK XA + B̃K Cv

where new controller variables

ÃK = RAK S T + RBK Cv Y + XBu CK S T + X(A + Bu DK Cv )Y

B̃K = RBK + XBu DK
C̃K = CK S T + DK Cv Y
D̃K = DK (19.14)

have been introduced. Moreover

" #
Y I
TYT P TY = (19.15)
I X

The important point here is that the block matrices in (19.13) and (19.15) are linear in the
transformed controller variables (19.14) and the auxiliary variables X and Y . Using the
fact that for a nonsingular matrix TY the condition M < 0 is equivalent to TYT M TY < 0,
such a congruence transformation can be applied to (19.11) - where diag(TY , TY , . . . , TY )
is used instead of TY - to obtain

L ⊗ (TYT P TY ) + M ⊗ (TXT Ac TY )T + M T ⊗ (TXT Ac TY ) < 0 (19.16)

This inequality is linear in the variables ÃK , B̃K , C̃K , D̃K , Y and X. It is equiva-
lent to (19.11), therefore the existence of these matrices satisfying (19.16) is a necessary
condition for Ac to have its poles in D. On the other hand, having found solutions
ÃK , B̃K , C̃K , D̃K , Y and X to the LMI (19.16), one can compute via singular value
decomposition nonsingular S and R satisfying

RS T = I − XY

and solve (19.14) for DK , CK , BK and AK . Note that the constraint

" #
Y I
>0
I X

guarantees that I−XY < 0 and therefore that nonsingular matrices S and R satisfying the
above exist. The resulting controller has the property that the closed-loop eigenvalues are
19. Controller Design Using Linear Matrix Inequalities 149

in D. Thus, the existence of matrix variables satisfying (19.15) and (19.16) is a necessary
and sufficient condition for the closed-loop system to have its poles in D.
In summary, a pole region constraint in the form of a matrix inequality with variable terms
P and P Ac can be transformed by a congruence transformation TY into an equivalent
constraint with the terms TYT P TY and TYT P Ac TY . The resulting inequality is linear in
transformed controller matrices and auxiliary variables. Having solved this LMI, one can
compute the desired controller matrices from (19.14).

H2 and H∞ Constraints

The same transformation can be used to make the H2 constraint (18.19) and the H∞
constraint (18.22) linear in the transformed controller variables. In addition to the terms
P and P Ac , the constraints (18.19) and (18.22) applied to the closed-loop system involve
the terms P Bc and Cc Dc , which are transformed into TYT P Bc and Cc Dc TY , respectively.
These terms are again linear in the new controller variables (19.14) and the auxiliary
variables X and Y .

List of LMI Constraints

The constraint P > 0 is shared by pole region, H2 and H∞ constraints; with a congruence
transformation TY , it is transformed into
" #
Y I
TYT P TY = >0 (19.17)
I X

The following conditions refer to the configuration shown in Figure 19.1 with plant model
(19.1) and controller (19.2). The controller has the same dynamic order as the plant. The
transformed controller matrices ÃK , B̃K , C̃K , D̃K are defined by (19.14). (∗) denotes
matrix blocks that can be inferred by symmetry.
Pole Region Constraint
150

The closed-loop matrix Ac has all eigenvalues in D if and only if there exist matrices ÃK , B̃K , C̃K , D̃K , X and Y that satisfy (19.17)
and
" # " # " #T
Y I AY + Bu C̃K A + Bu D̃K Cv AY + Bu C̃K A + Bu D̃K Cv
L⊗ +M ⊗ + MT ⊗ <0 (19.18)
I X ÃK XA + B̃K Cv ÃK XA + B̃K Cv

H2 Constraints
kT k2 < ν if and only if there exist matrices ÃK , B̃K , C̃K , D̃K , X, Y and W that satisfy (19.17) and
 
AY + Y AT + Bu C̃K + (Bu C̃K )T ÃTK + A + Bu D̃K Cv (Cz Y + Dzu C̃K )T
 

 ∗ AT X + XA + B̃K Cv + (B̃K Cv )T (Cz + Dzu D̃K Cv )T 
<0 (19.19)
∗ ∗ −I
 
Y I Bw + Bu D̃K Dvw
 
 ∗ X XBw + B̃K Dvw  > 0
  (19.20)
∗ ∗ W
and trace W < ν 2 and Dc = Dzw + Dzu D̃K Dvw = 0

H∞ Constraints
kT k∞ < γ if and only if there exist matrices ÃK , B̃K , C̃K , D̃K , X and Y that satisfy (19.17) and

 
AY + Y AT + Bu C̃K + (Bu C̃K )T ÃTK + A + Bu D̃K Cv Bw + Bu D̃K Dvw (Cz Y + Dzu C̃K )T
 
 ∗ AT X + XA + B̃K Cv + (B̃K Cv )T XBw + B̃K Dvw (Cz + Dzu D̃K Cv )T 


<0 (19.21)
 ∗ ∗ −γI (Dzw + Dzu D̃K Dvw )T 

∗ ∗ ∗ −γI
19. Controller Design Using Linear Matrix Inequalities 151

Design Algorithm

The design process can be summarised in the following algorithm.

1. Specify the design objective and generate the required matrices

a) Pole region constraint - express the desired pole-region in terms of the matrices
L and M (see for example (18.8), (18.9) and (18.10)).
b) H2 performance - set Bu = B, Cv = −C and construct the generalised plant
by appropriately selecting the matrices Bw , Cz , Dzu and Dvw (see for example
chapter 11).
c) H∞ performance - choose weighting filters WS (s), WK (s) or/and WT (s) and
construct the generalised plant (see chapter 16). Extract the matrices A, Bw ,
Bu , Cz , Cv , Dzw , Dzu , Dvw .

2. Express the design specifications as an LMI problem. The following table sum-
marises the relevant LMI constraints
Feedback type
Design objective state-feedback output feedback LMI objective
P = PT X = XT , Y = Y T
" #
Y I
with any P >0 >0
I X
Pole region constraint (19.5) (19.18) feasibility
H2 performance (19.7), (19.8) (19.19), (19.20) min trace W
H∞ performance (19.10) (19.21) min γ

3. Construct the controller

• State-feedback case: F = Y P −1
• Output-feedback case: perform singular value decomposition of I − XY
√ √
U ΣV T = I − XY ⇒ R = U Σ, ST = ΣV T

Then use the inverse transformation of (19.14):

DK = D̃K
CK = (C̃K − DK Cv Y )(S T )−1
BK = R−1 (B̃K − XBu DK )
 
AK = R−1 ÃK − RBK Cv Y − XBu CK S T − X(A + Bu DK Cv )Y (S T )−1
152

Exercises

Problem 19.1
Design an LQG controller for the aircraft model in Example 11.1 using LMIs and the
weighting matrices from Problem 15.2 [Design 2]
" # " #
C 0 h i h i
Cz = 50 · Dzu = ; Bw = 50 · B 0 ; Dvw = 0 I3 .
0 I3

Hint: Use that the LQG problem can be expressed as an H2 design problem, when the
generalised plant in Fig. 11.3 is used. See the design algorithm on page 151 for details.
Compare the obtained controller with the one obtained using the h2syn function.

Problem 19.2
Consider the state space model
        
x˙1 0 0 1 0 x1 0 0
x˙2   0 0 0 1 x2  0 0
        
     
 =
  +  u + 
 w
x˙3  −1.25 1.25 0 0
 x3  1 −1
   

x˙4 1.25 −1.25 0 0 x4 0 1

 
x1
x2 
i
h 
z = 0.75 −0.75 0 0  
x3 
 

x4

a) Design a state feedback controller that minimises the H∞ norm (i.e. γ = kTzw k∞ )
of the closed loop transfer function from input w to output z, and assigns the
closed-loop poles to the LMI region D with αl = −∞, αr = −10 and β = 45◦ .

b) For αr = [−1, −2 − 5, −10, −20, −30, −50, −100], calculate the H∞ norm and
plot γ versus αr .
Chapter 20

LMI Approach to Multi-Objective

Design

So far it has been shown how the three types of design specifications introduced in Chapter
18, can be expressed as LMI constraints on the synthesis of a dynamic controller (19.2).
The LMI constraints involve transformed controller matrices and auxiliary variables. Let
K̃ denote the collection of matrices

K̃ = (ÃK , B̃K , C̃K , D̃K , X, Y )

These variables in the LMI constraints provide the data from which the controller can be
computed. The three types of design specifications considered here are

• the pole region constraint (18.7) which was shown to be equivalent to (19.16), an
LMI of the form
Mreg (K̃) < 0

• the constraint (18.19) on the H2 -norm in the form of an upper bound ν, which can
be transformed into an LMI of the form

M2 (K̃, W ) < 0

where W is used as a slack variable and satisfies trace W < ν 2

• the constraint (18.22) on the H∞ -norm in the form of an upper bound γ, which can
be transformed into
M∞ (K̃, γ) < 0

Explicit formulae for Mreg (K̃), M2 (K̃, W ) and M∞ (K̃, γ) were given in the previous chap-
ter. The controller design can be carried out by solving a convex optimization problem
of the form
min
p
cT p subject to M (p) < 0
154

where M (p) is a combination of the above constraints. The decision variables - the ele-
ments of the vector p - are the elements of the matrix variables K̃ and auxiliary variables.
The linear cost function cT p can be chosen to represent the search for the "best" controller
that satisfies the given specifications.

Mixed H2 /H∞ Design

To combine H2 and H∞ norm constraints, one can partition the input and output vectors
w(t) and z(t) as
" # " #
w∞ z
w= , z= ∞ (20.1)
w2 z2
Figure 20.1 shows the configuration with separate channels for H2 and H∞ constraints.
A possible choice for a generalized plant is then
 h i 
A Bw∞ Bw2 Bu
 
Cz∞ ∞
" # " #
 Dzu 
P (s) = 0
 
Cz2 2
 

 h i
Dzu 

∞ 2
Cv Dvw Dvw 0

w∞ z∞
w2 P z2

u v
K

Figure 20.1: Generalized plant for mixed H2 /H∞ design

Let T2 (s) denote the transfer function from w2 to z2 and T∞ (s) from w∞ to z∞ , then
the H2 constraint should be applied to T2 (s) and the H∞ constraint to T∞ (s). For
example, one might want to design a controller for a plant with uncertain parameters
such that the closed-loop poles of the nominal system are constrained to be in a given
pole region, and where an additional H∞ constraint (say kT∞ k∞ < γ where γ = 1)
guarantees robust stability for the admissible range of system parameters. One can then
search for a controller that minimizes a quadratic performance index (H2 -norm of T2 (s))
under these constraints by solving the convex optimization problem (18.2) in the form
 
M (K̃) 0 0
 reg 
min trace W subject to 
 0 M2 (K̃, W ) 0 <0
 (20.2)
K̃,W
0 0 M∞ (K̃, γ = 1)
20. LMI Approach to Multi-Objective Design 155

From the solution K̃ to this problem one can compute a controller that satisfies the pole
region and H∞ constraints as well as kT2 k22 < trace W . A design example of this type is
presented in the next chapter to illustrate the approach.
Alternatively, one can minimize the H∞ norm subject to the H2 norm being less than a
specified upper bound ν, in this case the minimization is over K̃, W and γ as variables
subject to trace W < ν 2 . Similarly, it is possible to minimize a linear combination of the
H2 -norm and the H∞ -norm.

Existence of Solutions and Conservatism of Design

An important feature of the LMI approach to controller design is that it allows to combine
different design specifications into a single convex optimization problem. However, this
advantage comes at a price: combining different constraints may introduce conservatism
into the design. Even though each of the LMI constraints considered here represents a
necessary and sufficient condition for a certain property of the system, combining them
into a single convex optimization problem is only possible by relaxing the conditions to
sufficient conditions, i.e. by giving up necessity. This aspect is now briefly discussed.
When the plant is controllable and observable, it is clear that there exist controllers that
place the closed-loop eigenvalues in a given region and at the same time achieve finite
values for the H2 and H∞ -norm. On the other hand, it may turn out that the constraints
for the problem
 
M (K̃) 0 0
 reg 
min c1 ν + c2 γ subject to 
 0 M2 (K̃, W ) 0 <0
 (20.3)
K̃,W,ν,γ
0 0 M∞ (K̃, γ)
and trace W < ν 2 are infeasible, i.e. that a combination of decision variables that satisfy
the constraints does not exist.
To see why this may happen, recall that the pole region constraint involves the term
Ac Preg , whereas the H2 constraint involves the terms Ac P2 and Cc P2 , and the H∞ con-
straint involves P∞ Ac and P∞ Bc . Here different subscripts for the matrix variable P have
been introduced to indicate in which constraint it is used. A necessary and sufficient
condition for the closed-loop system to satisfy all constraints simultaneously is the exis-
tence of three matrices Preg , P2 and P∞ each of which satisfies the corresponding LMI
constraint. However, the linearizing change of variables - and thus the synthesis of a
controller - is only possible by imposing the additional constraint
P = Preg = P2 = P∞ (20.4)
Whereas the search for a controller under just one of the three constraints is a search
over all controllers that satisfy this constraint, the assumption (20.4) reduces the search
under combined constraints to a search over only a subset of all controllers that satisfy
the combined constraints.
Chapter 21

Design Example: Robust Control of

a Power System Stabilizer

In this chapter it is shown how a combination of LMI constraints can be used to design
a robust controller for system with uncertain parameters. The design of a power system
stabilizer for a range of operating conditions is presented as a design example. First a
model is constructed that includes information about the parameter uncertainty. The
small gain theorem is then introduced as an important tool for robust controller design.
With the help of the small gain theorem, a H∞ constraint on the uncertain model is
derived that guarantees robust stability. This H∞ constraint is then combined with pole
region and H2 constraints for shaping the nominal performance.

Generator
Xl

Exciter
Et

Voltage - Eref
Regulator

Figure 21.1: Single machine on infinite busbar

Problem Description

Power system stabilizers (PSS) are used to suppress oscillation in power systems. A power
station connected to a large power system is often represented by a single equivalent ma-
158

chine connected through an impedance to an infinite busbar for the purpose of stability
and controller design studies. Such a model is shown in Figure 21.1. In this representa-
tion, the dynamic interaction between the various machines in the power station is not
considered, but it is still adequate for many types of studies, especially when the machines
are identical and operate at nearly the same load levels.

u ẋ = Ai x + Bu y = ∆ω
PSS
y = Cx

Figure 21.2: PSS control loop

Power systems constantly experience changes in operating conditions due to variations

in generation and load patterns, as well as changes in transmission networks. There is
a corresponding large variation in the small signal dynamic behavior of a power system.
This can be expressed as a parametric uncertainty in the small signal linearized model of
the system. The problem considered here is the robust design of PSS so that adequate
damping can be provided, while guaranteeing stability over a wide range of operating
conditions. In this simulation study a linearized state space model

ẋ(t) = A(P, Q, Xl ) x(t) + Bu(t)

y(t) = Cx(t)

with four state variables is used to model the single-machine-infinite-bus system. The
system matrix A in this model is a function of real power P , reactive power Q and line
impedance Xl . It is assumed that these parameters vary in the intervals

0.4 ≤ P ≤ 1.0, −0.2 ≤ Q ≤ 0.5, 0.2 ≤ Xl ≤ 0.7

Varying the parameters independently in steps of 0.1 over these intervals generates a
family of linearized models

ẋ = Ai x + Bu, y = Cx, i = 0, . . . , N (21.1)

where the system matrices Ai describe the system dynamics in different operating condi-
tions and N = 336. The distribution of open-loop eigenvalues of the family of 336 linear
systems is shown in Figure 21.3.
The PSS control loop is shown in Figure 21.2. The controlled output is the machine
shaft speed ∆ω, and the control input u is a voltage which is added to the terminal
reference voltage. The performance of the controller is assessed in terms of rejection of a
disturbance step d at plant input.
21. Design Example: Robust Control of a Power System Stabilizer 159

10

Imaginary Axis 4

−2

−4

−6

−8

−10
−18 −16 −14 −12 −10 −8 −6 −4 −2 0 2
Real Axis

Figure 21.3: Open-loop eigenvalues for the given range of uncertain parameters (red:
nominal plant)

The objective is to design a controller that maintains a damping ratio of at least 10% and
a real part of all closed-loop poles less than -0.5 at nominal operating conditions, and at
the same time guarantees stability for all admissible operating conditions.
In this chapter it is shown how the LMI constraints introduced in the previous chapter
can be used to design a dynamic output feedback controller of the form (19.2) that meets
the specifications. A H∞ constraint will be used to guarantee robust stability; for this
purpose a suitable representation of the parameter uncertainty is required.

Representing Parameter Uncertainty

The available information about parameter variation can be included in a generalized

plant model in the following way. Replace B in (21.1) by Bu and C by Cy , let A0 denote
the system matrix under nominal operating conditions. Augment the system with an
additional input vector w and output vector z to obtain

ẋ = A0 x + Bw w + Bu u, z = Cz x (21.2)

Now introduce feedback

w = ∆z
through a gain matrix ∆ that is unknown, but satisfies k∆k ≤ 1. The dynamics of the
system (21.2) are then governed by

ẋ = (A0 + Bw ∆Cz )x + Bu u (21.3)

160

and with a suitable choice of the matrices Bw and Cz , all models representing the uncertain
system (21.1) can be "covered" by the model (21.2) in the sense that any admissible system
matrix
Ai = A0 + Bw ∆Cz
can be generated by choosing a matrix ∆ with norm less than 1. A block diagram of this
uncertainty representation is shown in Figure 21.4. The uncertainty has been "isolated" by
mapping it into the matrix ∆ - note that once Bw and Cz have been fixed, all parameters
of the model inside the dashed box are known. The matrices Bw and Cz - together with
the condition that k∆k < 1 - represent a priori knowledge about the possible range of
parameter variations.
The construction of suitable matrices Bw and Cz will be discussed below.

∆
w z

Bw Cz

u Bu 1 Cy y
s

A0

Figure 21.4: Representation of model uncertainty

21. Design Example: Robust Control of a Power System Stabilizer 161

The Small Gain Theorem

The Power System Stabilizer must - apart from providing adequate damping and response
times - guarantee that the closed-loop system is stable for all admissible operating con-
ditions. If we use the representation in Figure 21.4 and map the parameter uncertainty
into an unknown matrix ∆, we therefore need a condition for the system to be stable
with all ∆ that satisfy k∆k < 1. Such a condition is provided by the following Theorem.
Consider the loop in Figure 21.5. We have

L(s)

Figure 21.5: Loop for small gain theorem

Theorem 21.1 Assume L(s) is stable. Then the loop is stable if kL(jω)k < 1 for all
ω.

This Theorem is a simple version of the Small Gain Theorem. Most methods for robust
controller design are based on one or another version of this Theorem. For SISO systems
it simply says that if the Nyquist plot of L(s) does not leave the unit disc, the closed-loop
system is stable (because L(jω) cannot encircle the critical point -1). For MIMO systems,
the small gain condition kL(jω)k < 1 implies σ̄(L(jω)) < 1 for all ω. As a consequence,
det(I − L(jω)) cannot encircle the origin and it follows from the generalized Nyquist
Theorem that the closed-loop system is stable.

Generalized Plant and Model Uncertainty

We will now express the design problem in the form of a generalized plant that includes
not only weighting functions, but also the representation of the model uncertainty. The
configuration is shown in Figure 21.6, where the uncertainty representation in Figure 21.4
has been combined with the generalized plant in Figure 20.1 by connecting the uncertain
block ∆ to the H∞ channels. The generalized plant transfer function is then
 h i 
A Bw∞ Bw2 Bu
 
Cz∞
" # " # " #
 0 0 0 
P (s) =  
(21.4)
Cz2 2
 

 h
0 0 i Dzu 

2
Cv 0 Dvw 0

To obtain a condition on robust closed-loop stability, we consider the loop formed by

∆ and the system inside the dashed box in Figure 21.6. A simplified block diagram
162

w∞ z∞
w2 P z2

u v
K

Figure 21.6: Generalized plant

is shown in Figure 21.7. Assuming that the controller K stabilizes the nominal plant
model (A0 , Bu , Cy ), we can invoke the small gain theorem to establish that the closed-
∞
loop system is stable if the magnitude of the loop transfer function ∆Tzw is less than one
at all frequencies, or if
∞ ∞
kL(jω)k = k∆Tzw (jω)k ≤ k∆kkTzw (jω)k < 1 ∀ω

It follows that the closed-loop system is stable for all ∆ satisfying k∆k < 1 if
∞
kTzw (jω)k∞ < 1

∞
w Tzw z

Figure 21.7: Configuration for robust closed-loop stability

The following problem can now be posed in the form of linear matrix inequalities and
solved with LMI solvers: Design a controller that minimizes the H2 -norm of the transfer
function from w2 to z2 (pick the best controller in terms of a quadratic performance index)
under the constraints that the H∞ -norm of the transfer function from w∞ to z∞ is less
than 1 (guarantee robust stability), and that the closed-loop eigenvalues of the nominal
system are located within a specified region in the complex plane (meet specs on damping
ratio and speed of response).
21. Design Example: Robust Control of a Power System Stabilizer 163

Constructing the Uncertainty Model

To apply the design procedure described above, the model uncertainty - represented by
a family of 336 linear models - must be expressed in terms of the matrices Bw∞ and Cz∞ .
For this purpose, write the uncertain system matrix Ai at a given operating point as

Ai = A0 + Ãi , Ãi = Bw∞ ∆i Cz∞ (21.5)

The matrices Bw∞ and Cz∞ must be chosen such that a matrix ∆i satisfying

Ãi = Bw∞ ∆i Cz∞ and k∆i k ≤ 1

exists for all admissible operating conditions (i = 1, . . . , N ). The first requirement is

equivalent to
Ãi ∈ R (Bw∞ ), ÃTi ∈ R (Cz∞ T ), i = 1, . . . , N
where R(M ) denotes the column space of a matrix M . Define
h i h i
ΨB = Ã1 Ã2 . . . ÃN , ΨC = ÃT1 ÃT2 . . . ÃTN

and introduce the singular value decomposition of these two matrices

ΨB = UB ΣB VBT , ΨC = UC ΣC VCT

Let rB = rank ΨB and rC = rank ΨC , then matrices ∆i satisfying (21.5) exist for i =
1, . . . , N if Bw∞ = B̃w and Cz∞ = C̃z where

B̃w = [ub1 ub2 . . . ubrB ], C̃z = [uc1 uc2 . . . ucrC ]T

and ubi and uci are the columns of UB and UC , respectively. The non-zero elements of the
matrices
ΣB = diag (σ1b , σ2b , . . .), ΣC = diag (σ1c , σ2c , . . .)
are the singular values of ΨB and ΨC ordered by decreasing magnitude; small singular
values indicate that the dimension of the uncertainty (rB or rC ) can be reduced.
To meet the requirement k∆i k ≤ 1, i = 1, . . . , N , the columns of Bw∞ and the rows of Cz∞
need to be scaled. This can be done as follows. Let

Σ̄B = diag (σ1b , . . . , σrbB ), Σ̄C = diag (σ1c , . . . , σrcC )

and take Bw0 = B̃w Σ̄B , Cz0 = Σ̄C C̃z as a first scaling. In (21.5), replace Bw∞ and Cz∞ with
Bw0 and Cz0 , respectively, solve for ∆i in all operating points and find κ = maxi k∆i k.
Then matrices ∆i that satisfy (21.5) and k∆i k ≤ 1 exist for all operating points if Bw∞
and Cz∞ are chosen as
√ √
Bw∞ = κBw0 , Cz∞ = κCz0
164

In order to construct the uncertain model, a nominal system matrix A0 must be chosen
and the matrices Bw∞ and Cz∞ must be computed.
Here, the nominal system matrix is chosen as
1
A0 = (Amax + Amin )
2
where Amax and Amin are matrices whose elements are the maximum and minimum values
of the corresponding elements of Ai over i = 1, . . . , N , respectively.
Computing the singular value decomposition of ΨB and ΨC leads to

Σ̄B = diag (2163.5, 449.3, 0, 0), Σ̄C = diag (2209.6, 14.5, 2.6, 0.4)

Thus rB = 2, and the drop by more than two orders of magnitude after the first singular
value of ΨC suggests that without significant loss of information the uncertainty can be
projected onto one dimension, i.e. the numerical rank of ΨC is taken as rC = 1. Carrying
out the scaling procedure described above yields
 
0 0
" #
 0.130 3.676  δ1i
  h i
Bw∞ =
 , Cz∞ = 18.079 0.095 0.014 −0.024 , ∆i =
 0 0   δ2i
17.701 −0.027

To assess the conservatism of this uncertainty representation, (21.5) is solved for ∆i in all
operating points under consideration, and the result is plotted in Figure 21.8. Stability
is guaranteed inside the unit disc, and the fact that it is largely filled shows that the
representation is not too conservative.
The remaining system matrices are
   
0 0 0 −0.154 0
 4.086 −0.491 −0.099 0.219   0 
   
A0 = 
 , Bu = 
 
 0 1000 −20 0   1000


417.870 1.454 0.346 −0.665 0

h i
and Cy = 1 0 0 0 .

LMI-Based Controller Design

With A0 , Bw∞ , Bu , Cz∞ and Cy as above, the robust design problem can be solved as an
LMI problem in the form of (20.2). Design parameters are (a) the weights for the H2
constraint and (b) the parameters (αl , αr , β) of the LMI region in Figure 18.3 for the
closed-loop poles of the nominal system.
We will treat this problem as a deterministic design problem. This means that in the
H2 performance index no noise inputs are assumed and the input channel w2 is omitted;
21. Design Example: Robust Control of a Power System Stabilizer 165

0.8

0.6

0.4

0.2
δ2

−0.2

−0.4

−0.6

−0.8

−1
−1.2 −1 −0.8−0.6−0.4−0.2 0 0.2 0.4 0.6 0.8 1 1.2
δ1

Figure 21.8: Uncertainty representation after scaling

therefore the matrices Bw2 and Dvw

2
in (21.4) are removed from the model. The remaining
H2 weights are chosen as " # " #
C y 0
Cz2 = , 2
Dzu =
0 1
This H2 constraint can be interpreted in terms of linear quadratic optimal control: the
2-norm of z2 (t) is given by
Z ∞ Z ∞
kz2 (t)k22 = (xT CyT Cy x + uT u) dt = (y T y + uT u) dt
0 0
With the choice αl = −13, αr = −0.5 and β = 6o for the pole region, the solution to the
LMI problem is a controller
(1 + sT1 )(1 + sT2 )(1 + sT3 )
K(s) = k
(1 + sT4 )(1 + sT5 )(s2 + 2ζωn s + ωn2 )
with values given in the table below.

k T1 T2 T3 T4 T5 ζ ωn
4.014 2.032 0.203 0.084 0.503 0.077 0.14 72.6

The closed-loop pole distribution is shown in Figure 21.9, and Figure 21.10 shows the
step response at three different operating points.
The pole region parameters αl , αr and β can be used to tune the performance of the
control system. For example, moving the right boundary αr of the pole region to the left
leads to a smaller peak of the regulation error in response to a disturbance step, and to a
larger peak of the control signal. In this way, the pole region parameter αr can be used
as a “tuning knob” to trade control effort against peak regulation error.
166

80

60

40
Imaginary Axis
20

−20

−40

−60

−80
−14 −12 −10 −8 −6 −4 −2 0
Real Axis

Figure 21.9: Closed-loop eigenvalues (dashed: LMI region, red: nominal plant)

·10−3
1
Output y

−1 a
b
c
−2
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Time [sec]
·10−2
2

0
Input u

−2 a
b
c
−4
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Time [sec]

Figure 21.10: System response to a 5% disturbance step at different operating conditions.

(a) P = 0.8, Q = 0.4, X = 0.2, (b) P = 0.8, Q = 0.0, X = 0.6, (c) P = 1.0, Q = 0.54,
X = 0.7

User-Friendly Design Procedure

The design approach presented in this chapter can be carried out as an automated, in-
teractive procedure with a simple user interface. The data input consists of a family of
21. Design Example: Robust Control of a Power System Stabilizer 167

linear models - which represent parameter uncertainty - and the design specifications,
which are expressed in terms of pole region parameters αl , αr and β. Practical experience
suggests that one can fix the weights in the H2 cost function and use only the pole region
for tuning. The H∞ constraint guarantees that for any choice of pole region parameters
the system is stable in all admissible conditions. This fact may be used to develop an
automated search strategy for tuning the controller.

Exercises

Problem 21.1 ACC Benchmark Problem

Consider the system shown in Figure 21.11. Two bodies with masses m1 and m2 are
connected by a spring with stiffness k. The position of the second body is to be controlled
(desired position y = 0). The controller generates a force acting on the first body, and
only the position y(t) of the second body is available for feedback. A state space model
of the system is
        
ẋ1 0 0 1 0 x1 0 0 0
" #
ẋ2  0 0 0 1 x2   0  0 0  d1
       
      
 =
  +  u + 
 
ẋ3  −k/m1 k/m1 0 0 x3  1/m1  1/m1 0  d2
  

ẋ4 k/m2 −k/m2 0 0 x4 0 0 1/m2

y = x2 (21.6)

Numerical values of the nominal system are m1 = m2 = 1 and k = 1. However, the

stiffness of the spring can take values in the range 0.5 ≤ k ≤ 2.0. A controller is to
be designed that brings the second body back to |y(t)| < 0.1 ∀ t > 15 after a unit
disturbance impulse d1 (t) = δ(t) or d2 (t) = δ(t). The control input must satisfy |u(t)| ≤ 1
at all times.

x1 x2 = y

k
d1
m1 m2 d2
u

Figure 21.11: Two-mass-spring system

a) For ki = 0.5, 0.8, 1.1, 1.4, 1.7, 2.0, compute the system matrices Ai , and construct
168

the matrices (Bw , Cz ) for an uncertainty model. Also, derive these matrices directly
from the physical model (21.6) and compare the results.

b) Design a controller that guarantees stability for 0.5 ≤ k ≤ 2.0, while satisfying the
constraints on actuator and settling time.

c) Modify the controller such that the maximum possible range kmin ≤ k ≤ kmax for
robust stability is achieved. The actuator constraint must still be satisfied, but the
settling time can be longer than 15.

d) Analyze the controller in frequency domain. Why is it not reasonable to choose this
controller for solving the control task? Changing the setup to a four-block design
improves the frequency response of the controller. Design a controller using the
four-block approach and give a reason for the improvement.
Chapter 22

Model Order Reduction

Modern design techniques - like those discussed in this course - produce controllers of the
same dynamic order as the generalized plant. Thus, when the system to be controlled
is of high dynamic order, the controller may be too complex to be acceptable for im-
plementation. This is an important issue when LMI techniques are used for the design
of controllers, because solving a high-order LMI problem takes more computation time
than solving the Riccati equations associated with the same problem. In such cases, the
plant model should be approximated by a simplified model. This chapter gives a brief
introduction to the topic of model order reduction.
Consider a stable system with transfer function G(s) and state space realization

ẋ = Ax + Bu, y = Cx + Du

with n dimensional state vector x. If the number of states n is very large, one could try
to find a model of lower order that behaves "similar" to the original system. For example,
if some of the state variables do not have much effect on the system behaviour, one might
consider removing these states from the model. Thus, we need to know which states are
"important" for the model and which ones are not. The controllability Gramian and the
observability Gramian - discussed in Chapter 5 - turn out to be helpful for answering this
question.

Controllability Gramian

Recall the definition of the controllability Gramian

Z ∞ Tt
Wc = eAt BB T eA dt
0

A useful geometrical interpretation of Wc - given here without proof - is the following.

Define the set Sc as the set of all points in the state space to which the state vector x(t)
of the system can be driven from the origin with an input signal u(τ ), 0 ≤ τ ≤ t, that
170

satisfies ku(τ )k2 ≤ 1 (assume that u(τ ) = 0 for τ > t). Compute the singular value
decomposition of the controllability Gramian

Wc = V ΣV T

where h i
Σ = diag (σ1 , σ2 , . . . σn ) and V = v1 v2 . . . vn

The singular values are ordered such that σ1 ≥ σ2 ≥ . . . ≥ σn as usual. The set Sc of
points in the state space reachable with input energy 1 is then a hyper-ellipsoid centered
at the origin, with semi-axes in the directions of the columns vi of V , and length given
by the square root of the corresponding singular value. This is illustrated in Figure 22.1
for a second order system. Note that v1 is the direction that is most easily controlled,
whereas small singular values indicate directions which are difficult to control.

x2
1/2
σ 1 v1

x1
1/2
σ2 v2
Sc

Figure 22.1: Interpretation of the controllability Gramian

Observability Gramian

A similar interpretation can be given for the observability Gramian

Z ∞ T
Wo = eA t C T CeAt dt
0

Define the set So as the set of all points in the state space which - when taken as initial
conditions x(0) - lead to output signals y(t) that satisfy ky(t)k2 ≤ 1. Again the singular
value decomposition
Wo = V ΣV T

where h i
Σ = diag (σ1 , σ2 , . . . σn ) and V = v1 v2 . . . vn
22. Model Order Reduction 171

determines the set So - it is a hyper-ellipsoid centered at the origin with semi-axes given
√
by vi / σ i . This set is illustrated for a second order system in Figure 22.2. Note that
the axes are long in directions with small singular values, indicating directions that have
little effect on the output. These directions are difficult to observe.

x2

v2
1/2
σ2
v1
1/2
σ1

x1

So

Figure 22.2: Interpretation of the observability Gramian

Balanced Realization

The question posed at the beginning of the chapter was: which state variables are im-
portant for the system and which ones are not? The singular value decomposition of the
Gramians tells us which states show only a weak response to a control input (the ones
associated with small singular values of Wc ), and which ones have only weak influence on
the observed output (the ones where the singular values of Wo are small). Now it would
be unwise to remove a state variable from the model only because it shows little response
to control inputs - the same state variable may have a strong effect on the output. The
reverse may be true for states with small singular values of Wo . To find out which states
have little influence both in terms of controllability and observability, we will use a special
state space realization of the plant model which is known as balanced realization.
Initially we assumed a state space realization of G(s) with system matrices A, B, C and
D. Applying a similarity transformation T leads to a different state space model

(A, B, C, D) → (T −1 AT, T −1 B, CT, D)

for the same plant. The eigenvalues and the input/output behaviour of both state space
models are the same, because they are realizations of the same transfer function. The
controllability and observability Gramians however are different: it is straightforward to
check that if Wc and Wo are the Gramians of the first model, then T −1 Wc T −T and T T Wo T
are the Gramians of the second model.
172

A balanced realization of G(s) has the property that its controllability and observability
Gramians are equal and diagonal
 
σ1 0
Wc = Wo = 
 .. 
 . 

0 σn

Using Cholesky factorization and singular value decomposition, it is always possible to

find a similarity transformation T that brings a given state space model into this form -
in MATLAB one can use the function balreal() for this task. The diagonal entries of
the Gramian are called the Hankel singular values of the system. In the coordinate base
of the state space associated with the balanced realization, a small Hankel singular value
indicates that a state has little influence both in terms of controllability and observability.
Therefore, this realization is well suited for model reduction by removing "unimportant"
state variables. We will discuss two different ways of doing this.
Let (A, B, C, D) be a balanced realization of G(s) with n state variables. Assume the
inspection of the Hankel singular values indicates that only r states are significant and
that the last n − r Hankel singular values are small enough to be neglected. Partition the
state space model as
" # " #
A A12 B1 h i
A = 11 , B= , C = C1 C2 (22.1)
A21 A22 B2

where A11 ∈ IRr×r , A22 ∈ IR(n−r)×(n−r) etc.

Balanced Truncation

The subsystem (A11 , B1 , C1 , D) of the partitioned model (22.1) contains the states with
significantly large Hankel singular values. One approach to model order reduction is to
use this system with r state variables as an approximation of the full order model G(s).
This approach is known as balanced truncation, because the parts of the model associated
with insignificant states are simply ignored.
An important property of the resulting reduced order model
" #
A11 B1
Gtr (s) =
C1 D

is that it satisfies
Gtr (j∞) = G(j∞) = D

The direct feedthrough terms of full order and truncated model are the same. This
indicates that both models will exhibit similar high-frequency behaviour.
22. Model Order Reduction 173

Balanced Residualization

An alternative approach is not to ignore the insignificant states, but to assume that they
are constant and take them into account in the reduced order model. This method is
known as balanced residualization. In the partitioned model
" # " #" # " #
ẋ1 A A12 x1 B
= 11 + 1 u
ẋ2 A21 A22 x2 B2
" #
h i x1
y = C1 C2 + Du
x2
we make the assumption
ẋ2 = 0
and eliminate x2 from the model. It is straightforward to check that the resulting reduced
order model is
A11 − A12 A−1 −1
" #
22 A21 B1 − A12 A22 B2
Gres (s) =
C1 − C2 A−1
22 A21 D − C2 A−122 B2
The feedthrough term is different from that of the full order model, indicating that the
high-frequency behaviour will not be the same. In fact the reduced model was arrived at
by assuming that derivatives of some state variables are zero. This is true in steady state,
so we would expect similar behaviour of full order and reduced model at low frequencies.
One can indeed verify that the steady state gains of both models are the same, i.e.
G(0) = Gres (0) = D − CA−1 B

Example 22.1
A linearized model of a High Voltage DC transmission system with 31 state variables
is approximated by a second order model. Figure 22.3 shows the magnitude frequency
response when balanced truncation is used. Figure 22.4 shows the result of balanced resid-
ualization. The results illustrate that balanced truncation attempts to capture the high
frequency behaviour of the full order model, whereas balanced residualization captures
its low frequency behaviour. The MATLAB routines sresid() and balmr() were used
for residualization and truncation, respectively.

Approximation Error

A useful property shared by both reduction techniques discussed in this chapter - here
stated without proof - is the following. Let G(s) denote the full order plant model and
Ĝ(s) a reduced order approximation obtained either by balanced truncation or by balanced
residualization. The H∞ norm of the approximation error satisfies
n
X
kG(s) − Ĝ(s)k∞ ≤ 2 σi
i=r+1
174

2
10
Full order model
Truncation

0
10

−2
10
Magnitude

−4
10

−6
10

−8
10

−10
10 1 2 3 4 5 6 7 8
10 10 10 10 10 10 10 10
Frequency (rad/sec)

Figure 22.3: Balanced truncation: the reduced model captures - as far as possible with a
second order model - the resonant peak and high-frequency roll-off

2
10
Full order model
Residualization

0
10

−2
10
Magnitude

−4
10

−6
10

−8
10

−10
10 1 2 3 4 5 6 7 8
10 10 10 10 10 10 10 10
Frequency (rad/sec)

Figure 22.4: Balanced residualization: the reduced model captures the low frequency
behaviour

In other words: the maximum error in the magnitude response is less than twice the sum
of the Hankel singular values of the states that are being removed from the model.
22. Model Order Reduction 175

Order Reduction for Unstable Models

So far we assumed that the full order model is stable - a balanced realization is defined
only for stable systems. If the reduced order model is to be used for controller design, it
would be unwise to ignore any unstable pole of the plant. One way of dealing with an
unstable high order system is to split it up into a stable and an anti-stable part, i.e.

G(s) = Gs (s) + Gu (s)

where Gs (s) is stable and Gu (s) has all poles in the right half plane. The stable part of
the system can then be reduced by balanced truncation or residualization to Ĝs (s), and
a reduced model of the plant that retains all unstable modes is

Ĝ(s) = Ĝs (s) + Gu (s)

This approach is implemented in the MATLAB function balmr().

Bibliography - Optimal and Robust
Control

[5] V. Balakrishnan, S. Boyd, L. El Ghaoui, and E. Feron, “Linear matrix inequalities

in systems and control,” Studies in Applied Mathematics, vol. 15, 1994.
[6] P. Gahinet, A. Nemirovskii, A. J. Laub, and M. Chilali, “The lmi control toolbox,”
in Proceedings of 1994 33rd IEEE Conference on Decision and Control, IEEE, vol. 3,
1994, pp. 2038–2041.
[7] J. M. Maciejowski, Multivariable Feedback Design. Amsterdam: Addison-Wesley,
1989, isbn: 978-0-201-18243-9.
[8] S. Skogestad and I. Postlethwaite, Multivariable Feedback Control - Analysis and
Design. New York: Wiley, 2005, isbn: 978-0-470-01168-3.
[9] G. Strang, Linear Algebra and Its Applications. Stanford: Elsevier Science, 2014,
isbn: 978-1-483-26511-7.
[10] K. Zhou and J. C. Doyle, Essentials of Robust Control -. London: Prentice Hall,
1998, isbn: 978-0-135-25833-0.
 Part III

APPENDICES
Appendix A

Solutions to Exercises - Linear

Quadratic Optimal Control

Solution to Problem 1.1 DC motor

Since for the desired steady-state angular velocity ω0 a steady-state input signal u0 is
needed to translate the system’s equilibrium. For the purpose define ω̃ = ω − ω0 and
ũ = u − u0 . Further because there is only one state and one input Q = q ≥ 0, R = ρ > 0
and S = s ≥ 0. Then the problem can be formulated as:
Find control input ũ(t) that minimises the performance index
Z T  
q w̃2 (t) + ρũ2 (t) dt + sw̃2 (T )
t0

Then the control signal that has to be applied to the DC motor is u(t) = u0 + ũ(t). The
coefficients q, ρ and s can be then used to tune the controller to the design requirements.

Solution to Problem 3.1 Hamilton-Jacobi equation

For the problem given f (x, u, t) = u, l(x, u, τ ) = x2 + u2 + x4 /2 and m(x) = 0. Denoting
with ū the minimising input and substituting in (3.3) and (3.4) leads to
#T
∂V ∗ ∂V ∗
"
1
(x(t), t) = −(x2 + ū2 + x4 ) − ū(t)
∂t 2 ∂x
V ∗ (x(T ), T ) = 0

From Z T 1
∗
V (x(t), t) = (x2 + ū2 + x4 ) dt
0 2
follows
∂V ∗ (x(t), t) Z T
= (2x + 2x3 ) dt.
∂x 0
182 Appendices

and finally
∂V ∗ 2 2 1 4 Z T
(x(t), t) = −(x + ū + x ) − (2x + 2x3 ) dt ū(t)
∂t 2 0
V ∗ (x(T ), T ) = 0

Solution to Problem 4.1 Solution of Riccati equation

Taking the standard system description and cost function
Z T  
V = xT Qx + uT Ru dt + xT Sx
t0

leads to A = 0, B = 1, Q = 1, R = 1, S = 0 which reduces the Riccati differential

equation

−Ṗ = Q + P A + AT P − P BR−1 B T P, P (T ) = S
to − Ṗ = 1 − P 2 , P (T ) = 0

The differential equation can be solved using variable separation method

dP dP
Ṗ = = P2 − 1 ⇒ = dt
dt P2 − 1
Which after integration leads to
dP 1 P −1 P −1
Z Z  
= dt ⇒ ln = t + C̃ ⇒ = e2t+2C̃ = Ce2t
P2 − 1 2 P +1 P +1
Then, solving for P leads to
1 + Ce2t
P (t) =
1 − Ce2t
and then using that P (T ) = 0 ⇒ Ce2t = −1 ⇒ C = −e−2T leads to

1 − e2(t−T )
P (t) =
1 + e2(t−T )
Thus the optimal control is

u∗ (t) = −R−1 B T P (t)x(t)

1 − e2(t−T )
u∗ (t) = − x(t)
1 + e2(t−T )

Solution to Problem 4.2 Schur complement

A. Solutions to Exercises - Linear Quadratic Optimal Control 183

Recall that X is positive definite (X > 0), if X = X T and y T Xy > 0 holds for all y 6= 0,
or equivalently if the eigenvalues of X are positive. Let T ∈ IRn×n be any full-column
matrix. Then
T T XT > 0 ⇒ q T T T XT q > 0, q 6= 0

Defining y = T q leads to y T Xy > 0 and thus to X > 0.

Then X > 0 holds if and only if

I −LN −1
" #" #" #
M L I 0
>0
0 I LT N −N −1 LT I

since clearly the selected T matrix is full-column rank (its lower triangular with 1’s on
the diagonal). Performing the multiplication leads to

M − LN −1 LT 0
" #
>0
0 N

which is positive definite only when N > 0 and M − LN −1 LT > 0.

To prove X > 0 ⇔ M > 0, N − LT M −1 L > 0 select

I −M −1 L
" #
T =
0 I

Solution to Problem 4.3 Completing squares

a) Rewrite
" # " #
M L M L
T = T −T T T T T T −1
L N L N

where T is selected from the Schur complement

" #
I 0
T =
−N −1 LT I

Then
M − LN −1 LT 0
" # " #
h i
−T x
W (x, u) = x T
u T
T T −1
0 N u

Using that
" #−1 " #
I 0 I 0
=
X I −X I
184 Appendices

follows
I LN −1 M − LN −1 LT 0
" #" #" #" #
h
T T
i I 0 x
W (x, u) = x u −1 T
0 I 0 N N L I u
M − LN −1 LT 0
" #" #
h
T T −1 T
i x
= x x LN +u −1 T
0 N N L x+u
     
= xT M − LN −1 LT x + xT LN −1 + uT N N −1 LT x + u

    T  
min
u
W (x, u) = min
u
xT M − LN −1 LT x + N −1 LT x + u N N −1 LT x + u

The first term is independent of u and the second is quadratic in u and non-negative
(since N > 0). The control signal that gets the second term to zero (and therefore
minimises W (x, u)) is
ū = −N −1 LT x

b) Rewrite the term to be minimised from Hamilton-Jacobi equation (4.3) as

W (x, u) = xT Qx + uT Ru + xT (P A + AT P )x + xT P Bu + uT B T P x
" #" #
h
T T
i Q + P A + AT P P B x
= x u
BT P R u

Then using the above results follows ū = −R−1 B T P x, i.e. (4.4).

Solution to Problem 4.4 Riccati Differential Equation

Define an arbitrary control input u(t)

u(t) = −R−1 B T P (t)x(t) + v(t),

where P (t) and v(t) are arbitrarily chosen.

Substitution of u(t) in the state equation yields

ẋ = (A − BR−1 B T P )x + Bv. (A.1)

In the cost function

Z T
V = (xT Qx + uT Ru) dt + xT (T )Sx(T ),
t0

we can rewrite the term uT Ru as

uT Ru = (−R−1 B T P x + v)T R(−R−1 B T P x + v)

= xT P BR−1 B T P x + v T Rv − v T B T P x − xT P Bv. (A.2)
A. Solutions to Exercises - Linear Quadratic Optimal Control 185

From equation (A.1) we have

Bv = ẋ − Ax + BR−1 B T P x.
The left multiplication of Bv with xT P yields
xT P Bv = xT P ẋ − xT P Ax + xT P BR−1 B T P x. (A.3)
Replacing the term uT Ru in V with equation (A.2) and (A.3) yields
Z T 
V = xT Qx + xT P BR−1 B T P x + v T Rv − xT P ẋ + xT P Ax
t0

−xT P BR−1 B T P x − ẋT P x + xT AT P x − xT P BR−1 B T P x dt

+ xTT (S − PT )xT + xTT PT xT − xT0 P0 x0 + xT0 P0 x0 ,

where
xT = x(T )
PT = P (T )
x0 = x(t0 )
P0 = P (t0 ),
and
Z T d T
xTT PT xT − xT0 P0 x0 = (x P x)dt
t0 dt
Z T
= (ẋT P x + xT Ṗ x + xT P ẋ)dt.
t0

After the cancellation and rearrangement of V ,

Z T
V = xT (Q + P A + AT P − P BR−1 B T P + Ṗ )xdt
t0
Z T
+ v T Rvdt + xTT (S − PT )xT + xT0 P0 x0 .
t0

When P (t) satisfies

Q + P A + AT P − P BR−1 B T P + Ṗ = 0
S − PT = 0.
the following terms in V remain:
Z T
V = v T Rvdt + xT0 P0 x0
t0

Since R > 0, choosing v(t) ≡ 0 leads to the optimal control input u(t) = u∗ (t). We obtain
the minimal performance index
V ∗ = xT0 P0 x0 .

Solution to Problem 5.1 Hamiltonian state-space model

186 Appendices

a) The transfer function y to v is the transfer function of the lower dashed block. The
state-space matrices of the block are Ā = −AT , B̄ = −C T , C̄ = −B T and D̄ = 0
and hence
Gvy (s) = C̄(sI − Ā)−1 B̄ + D̄ = B T (sI + AT )−1 C T

The eigenvalues of Gvy (s) are the negative eigenvalues of G(s).

b) The state-space model is

−BR−1 B T
" # " #" # " #
ẋ A x B
= + r
λ̇ −C T C −AT λ 0
" #
h i x
y= C 0
λ

Note that by Q = C T C the system’s A matrix is the Hamiltonian matrix.

Solution to Problem 5.2 DC motor Construct the Hamiltonian matrix with A =

−a, B = b, R = ρ and Q = 1:

2
−a −b ρ1 b −0.5 − 150
" # " #
H= = ρ
−1 a −1 0.5

As transformation matrix U can be used the right eigenvectors of H. In MATLAB:

[U,L] = eig(H). MATLAB arranges the real eigenvalues in increasing order, therefore
the first one is the stable and the second the unstable one.
For a) ρ = 100, s = 0
" # " # " #
−0.5 −225 1 1 −15.0083 0
H= ⇒ U= , Λ=
−1 0.5 0.0645 −0.0689 0 15.0083
u21
G = −(u22 − su12 )−1 (u21 − su11 ) = − = 0.9355
u22

Substituting in (5.7) leads to

0.0645 − 0.0689e−15.0083(T −t) 0.9355e−15.0083(T −t) 1 − e−30(1−t)

P (t) = ≈ 0.06
1 + e−15.0083(T −t) 0.9355e−15.0083(T −t) 1 + 0.94e−30(1−t)

1
Then the feedback gain is f (t) = − bP (t).
ρ
A. Solutions to Exercises - Linear Quadratic Optimal Control 187

1 − e−30(1−t) 1 − e−30(1−t)
a) ρ = 100, s = 0 P (t) ≈ 0.06 f (t) ≈ −0.1
1 + 0.94e−30(1−t) 1 + 0.94e−30(1−t)

3 1 − e−10(1−t) 1 − e−10(1−t)
b) ρ = 10 , s = 0 P (t) ≈ 0.2 f (t) ≈ −0.03
1 + 0.8e−10(1−t) 1 + 0.8e−10(1−t)

1 − e−3(1−t) 1 − e−3(1−t)
c) ρ = 104 , s = 0 P (t) ≈ 0.5 f (t) ≈ −0.0075
1 + 0.5e−3(1−t) 1 + 0.5e−3(1−t)

104 + 5e−10(1−t) 104 + 5e−10(1−t)

d) ρ = 103 , s = 0.19 P (t) ≈ 0.2 f (t) ≈ −0.03
104 − 6e−10(1−t) 104 − 6e−10(1−t)

The plots of f (t), u(t) = f (t)ω(t) and ω(t) in the interval under the initial condition
ω(0) = 1 are shown in Fig. A.1 and Fig. A.2-left.

0.02

0.00

−0.02
f(t)

−0.04

−0.06
ρ = 102 ; s = 0
−0.08 ρ = 103 ; s = 0
ρ = 104 ; s = 0
ρ = 103 ; s = 0.19
−0.10
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]

Figure A.1: Feedback gain f (t)

From the expressions for f (t) (and the figures) a), b) and c) is obvious that by increasing
ρ the argument of the exponents in P (t) gets smaller, corresponding to a larger time-
constant and therefore to slower control. The feedback gains in b) and d) are similar for
small t and have difference only at the end of the time period where the later converges to
a finite value (P (1) = s = 0.19). To see a difference in the control signal and the angular
velocity in Fig. A.2-right the signals are showed “zoomed-in” near t = 1 sec.
MATLAB code: orc05_DCmotor.m, orc05_DCmotor_sim.mdl.

Solution to Problem 6.1 DC motor - LQR problem

188 Appendices

·10−3
0
0.00
−0.2
−0.4
102 ;
u(t)

u(t)
ρ = s = 0
−0.05 ρ = 103 ; s = 0 −0.6
ρ = 104 ; s = 0
−0.8
ρ = 103 ; s = 0.19
−0.10 −1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.8 0.85 0.9 0.95 1
Time [sec] Time [sec]
·10−2
1.00
0.80 4
0.60
ω(t)

ω(t)
0.40 2
0.20
0.00 0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.8 0.85 0.9 0.95 1
Time [sec] Time [sec]

Figure A.2: Control signal u(t) and angular velocity ω(t)

Note that now the infinite horizon control problem is considered, for which P (t) and F (t)
are constant matrices. Then the solution can be easily obtained using the LQR function
in MATLAB:
F=-lqr(A,B,Q,R)
See the MATLAB files: orc05_DCmotorLQR.m, orc05_DCmotor_sim.mdl
A. Solutions to Exercises - Linear Quadratic Optimal Control 189

Solution to Problem 6.2 LQR control

From the Figure 6.2 the control signal can be derived

u = −f1 (r − x1 ) + (f2 x2 + f3 x3 )
u = −f1 r + (f1 x1 + f2 x2 + f3 x3 ) = f x − f1 r

a) In determining the optimal control, we assume that the input is zero. Let us deter-
mine the state-feedback gain matrix f , where
h i
f = f1 f2 f3

such that the following performance index is minimized:

Z ∞  
V = xT Qx + uT Ru dt
0

To get the fast response, q11 must be sufficiently large compared with q22 , q33 and R,
because y(t) = x1 (t). The optimal control gain is obtained using the lqr MATLAB
function. The controller can be further tuned with respect to some non-zero initial
conditions x0 . For the purpose of simulation the function initial can be used.

b) The state-space equation for the closed-loop system is

ẋ = Ax + bu = Ax + b(f x − f1 r) = (A + bf )x − bf1 r
y = cx = x1

To obtain the unit step-response, following code can be used.

Gcl = ss(A+b*f, -b*f(1), c, d);

[y,x,t] = step(Gcl, 10);

MATLAB code: orc05_tuneLQR.m

Solution to Problem 6.3 Infinite horizon LQR

Construct the Hamiltonian matrix and find its eigenvalues λ and eigenvectors U . Select
the eigenvectors corresponding to
"
the#
stable (with negative real part) eigenvalues and
U11 −1
stack them together in a matrix . Now P = U21 U11 and
U21
h i
F = −3.6103 −1.5514

This lead to the closed-loop eigenvalues −2.3271 ± j0.6446.

MATLAB files: orc06_inf_horizon_LQR.m

Solution to Problem 7.1 Mass-spring system

190 Appendices

a) Using Newton’s laws, the equation of motion is

mq̈(t) + kq(t) = u(t)
Letting x1 (t) = q(t) denote the position of mass and x2 (t) = q̇(t) its velocity, the
dynamics can be expressed in the state space as
k 1
ẋ1 (t) = x2 (t); ẋ2 (t) = − x1 (t) + u(t)
m m
or
ẋ(t) = Ax(t) + bu(t)
with " # " # " #
x (t) 0 1 0
x(t) = 1 ; A= k ; b= 1
x2 (t) −m 0 m
For simplicity assume that m = 1 and k = 1, where appropriate units are used.
In the absence of control, initial conditions will produce a persistent sinusoidal
motion of mass. As such, we seek a control law that regulates the position of the
mass to its equilibrium value of q(t) = 0. Thus, we define the state of interest, z as
x1 (t). h i
z(t) = cx(t) with c= 1 0

b*) Since the control is scalar, the cost function takes the following form
Z ∞  
V = z 2 (t) + ρu2 (t) dt with ρ>0
0
where ρ is the control weighting parameter. The optimal LQR control h law itakes
the form u(t) = f x(t) where the feedback gain f is a row vector f = f1 f2 . To
determine the value of gain, we must solve the algebraic Riccati equation. Recalling
that we want a symmetric solution to the Riccati equation, letting
" #
P P12
P = 11 (A.4)
P12 P22
and using the values of A, b and c given above, the Riccati equation becomes
" #" # " #" # " #
P P12 0 1 0 −1 P11 P12 1 h i
0 = 11 + + 1 0
P12 P22 −1 0 1 0 P12 P22 0
" #" # " #
1 P11 P12 0 h i P
11 P12
− 0 1
ρ P12 P22 1 P12 P22
Carrying out the matrix multiplication leads to the following three equations in the
three unknown Pij from Equation (A.4)
1 2
0 = 2P12 − 1 + P12
ρ
1
0 = −P11 + P22 + P12 P22 (A.5)
ρ
1 2
0 = −2P12 + P22
ρ
A. Solutions to Exercises - Linear Quadratic Optimal Control 191

s
1
Solving the first equation for P12 and simplifying yields P12 = −ρ ± ρ 1 + .
ρ
Both the positive and negative choices for P12 are valid solution for Riccati equation.
While we are only interested in the positive semidefinite solution, we still need more
information to resolve which choice of P12 leads to the unique choice of P ≥ 0.
Rewriting the 3rd equation
1 2
2P12 = P22
ρ
indicates that P12 must be positive, since the right hand side of the equation must
always be positive. The last equation indicates that there will also be a ± sign am-
biguity in selecting the appropriate P22 . To resolve the ambiguity we use Sylvester’s
Test, which says that for P ≥ 0 both
2
P11 ≥ 0 and P11 P22 − P12 ≥0
Solving Equation (A.5) using the above mentioned relations, which clearly show
that P22 > 0, gives the remaining elements of P
v s s
u
u 1 1 1
P11 = ρt2(1 + )( 1 + − 1); P12 = −ρ + ρ 1 +
ρ ρ ρ
v s
u
u 1
P22 = ρt2( 1 + − 1)
ρ
The final step in computing the controller is to evaluate the control gains f = −bT /ρP
s v s
u
1 u 1
f1 = − 1 + + 1; f2 = −t2( 1 + − 1)
ρ ρ
Evaluating the eigenvalues of the closed-loop dynamics leads to a pair of complex
conjugate closed-loop poles
v s v s
u u
1u 1 1u 1
λ1,2 = − t2( 1 + − 1) ± j t2( 1 + + 1)
2 ρ 2 ρ

c) Given the control gains as a function of the control weighting ρ it is useful to examine
the locus of the closed-loop poles for the system as ρ varies over 0 < ρ < ∞. For
large ρ
ρ → ∞, f1 → 0, f2 → 0, λ1,2 = ±j1
Thus when the control is expensive there is practically no feedback, and the closed-
loop eigenvalues are the same as the open-loop eigenvalues. For small ρ
ρ → 0, f1 → ∞, f2 → ∞, λ1,2 → −∞ ± j∞
This is consistent with how ρ influences the cost. Larger values of ρ place a heavy
penalty on the control and lead to low gains with slow transients (poles close to the
imaginary axis), while small values of ρ allow larger control action, move the poles
further to the left from the imaginary axis, thus leading to fast transients.
192 Appendices

d) MATLAB files: orc05_mass_spring.m

Solution to Problem 7.2 Spectral factorisation

The transfer function u to y is
" #" #
s 1
h i 0
i s −1 −1 0
" # " # 1 0
h 5 s 3 3 b(s)
Gyu = 1 0 = = =
−5 s 3 s2 − 5 s2 −5 a(s)

Then
1 1
p(s) = a(s)a(−s) + b(s)b(−s) = ((−s)2 − 5)(s2 − 5) + 9 = s4 − 10s2 + 34
ρ 1
The four roots are ±2.3271 ± j0.6446, thus the closed-loop eigenvalues are −2.3271 ±
j0.6446 and the closed-loop characteristic polynomial is

ac (s) = (s + 2.3271 + j0.6446)(s + 2.3271 − j0.6446) = s2 + 4.6542s + 5.831

Recall that
" # " # " # "  #
 s 0 0 1 0 h i s −1
ac (s) = det(sI −A−bf ) = det  − − f1 f2  = det
 
 0 s 5 0 3 −5 − 3f1 s − 3f2
or
ac (s) = s2 + 4.6542s + 5.831 = s2 − 3f2 s − (5 + 3f1 )
By comparing the coefficients follows f2 = −1.5514, f1 = −3.6103, which is exactly the
same result obtained in Problem 6.3.

Solution to Problem 7.3 Optimal state feedback

Check the closed-loop characteristic polynomial
" #
s −1
ac (s) = det = s2 + qs + 1
1 s+q

Thus closed-loop stability requires q > 0 (recall that for first and second order system
sufficient condition for stability is that all coefficients are positive).
The optimal state feedback loop gain L(s) is given by
1 + qs
L(s) = −f (sI − A)−1 b =
s2
and Kalman’s identity, a necessary and sufficient condition for LQR optimality for a stable
control law, requires the satisfaction of inequality (7.4) or what is equivalent

|1 + L(jω)| ≥ 1 ∀ω.
A. Solutions to Exercises - Linear Quadratic Optimal Control 193

Substituting L(jω) and squaring both sides (doesn’t change the inequality) leads to
 1 − ω 2 + jqω 2
 
(1 − ω 2 )2 + q 2 ω 2
≥ 1, ⇒ ≥ 1, ⇒ 1 + (q 2 − 2)ω 2 ≥ 0
 
−ω 2 ω4
 
 
√
The last inequality can be hsatisfied ifor all ω if and only if q ≥ 2. Thus, for example,
the stable gain matrix f = −1 −2 is optimal with respect to some LQR problem, but
h i
the stable gain matrix f = −1 −1 is not optimal with respect to any LQR problem.

Solution to Problem 7.4 Pendulum

h i
a) With the state vector ψ ψ̇ θ θ̇ the state-space matrices are
   
0 1 0 0 0
 2.5 0 1 0  1 
    h i
A= ; b= 

; c= 1 0 1 0 .
 0 0 0 1  0 


−2.5 0 −2 0 −1

b) The open-loop characteristic equation

a(s) = det(sI − A) = s4 − 0.5s2 − 2.5
has eigenvalues ±1.361 and ±j1.162. To obtain a time-constant of 1/3 the dominant
closed-loop pole(s) should have a real part of −3. We could choose four poles
consistent with this constraint, but this still leaves a lot of choice. By means of
spectral factorisation theorem, we can make a reasonable choice of the eigenvalues.
The open-loop transfer function is
b(s) 1
= 4
a(s) s − 0.5s2 − 2.5
By spectral factorisation, the 2n degree polynomial is
1 1
p(s) = a(s)a(−s) + b(s)b(−s) = (s4 − 0.5s2 − 2.5)2 +
ρ ρ
Note that the characteristic polynomial of the negative closed-loop of
1 1 b(s)b(−s)
L̃(s) = G(s)G(−s) =
ρ ρ a(s)a(−s)
is equal to p(s). Thus, in order to determine ρ leading to the desired time constant,
construct the system G(s)G(−s) and use standard root-locus software (e.g., rltool
in MATLAB) to find a value of k = 1/ρ leading to the desired closed-loop pole
locations in the left-half plane - see Fig. A.3 (this is known as “symmetric root-
locus” because by the properties of the Hamiltonian each pole in the left half-plane
has a mirror in the right half-plane).
Selecting 1/ρ = k = 1.5 107 leads to the optimal closed-loop poles −3.0238±j7.2742
and −7.3034 ± j3.0117.
194 Appendices

4
Imaginary Axis
2

−2

−4

−6

−8
−8 −6 −4 −2 0 2 4 6 8
Real Axis

Figure A.3: Symmetric root-locus for LQR design for pendulum

c) Knowing the optimal closed-loop locations, the problem now is to find the feedback
gain matrix f . One method is to find ac (s) = det(sI − A − bf ) as function of
f and solve for f (as in Problem 7.2). However, since this is a tedious task one
can instead use a pole-placement - the desired closed-loop poles are known and are
optimal! Thus f = -place(A, b, stabp) where stabps are the stable poles
of p(s) leads to
h i
f = −3875.5 −1283.9 −3662.2 −1263.2 .

MATLAB files: orc07_pendelum.m

Solution to Problem 9.1 Autonomous system

We start with the solution to the state equation

ẋ = Ax; x(t) = eAt x0 ; x(0) = x0

Substituting for x(t) in the cost function, we get

Z ∞ T
Z ∞ T
V = (xT0 eA t QeAt x0 ) dt = xT0 (eA t QeAt dt)x0 = xT0 P x0
0 0

where Z ∞ T
P = (eA t QeAt dt)
0
A. Solutions to Exercises - Linear Quadratic Optimal Control 195

T
To
 show P is the
 solution to P A + A P + Q = 0, substitute for P and use that A is stable
lim eAt → 0 :
t→∞
Z ∞ T T
(eA t QeAt A + AT eA t QeAt ) dt + Q = 0
0
Z ∞ d AT t At
(e Qe )dt + Q = 0
0 dt ∞
T
eA t QeAt  + Q = 0
0
0−Q+Q=0
which proves the result.

Solution to Problem 9.2 LQG control

a) The LQG controller can be seen in Fig. 9.1. Using that A = 1, b = 1 and c = 1
simplifies to the controller in Fig. A.4 with transfer function
f fe
K(s) = F (sI − A − BF − Fe C)−1 Fe =
s − 1 − f − fe

−y 1 x̂ u
fe f
s

Figure A.4: LQG controller

b) Writing the filter algebraic Riccati equation (FARE):

Pe AT + APe − Pe C T Re−1 CPe + Qe = 0.
1 q
2pe − p2e + 1 = 0 → pe 1,2 = re ± re2 + re
re
q
The stable solution is the positive solution: pe = re + re2 + re > 0, therefore
s
1 1
fe = −Pe C T
Re−1 = −pe = −1 − 1+
re re
196 Appendices

c) By duality s
1 1
f = −1 − 1+ = −3 → ρ=
ρ 3

d) From re = 0.125 we have fe = −4, and with f = −3

ẋ = (A + bf )x + w − bf ε = (1 − 3)x + w + 3ε = −2x + w + 3ε;

ε̇ = (A + fe c)ε + w + fe v = (1 − 4)ε + w − 4v = −3ε + w − 4v

The error dynamics is not much faster than the plant. To make it five times faster
than the plant we need

ε̇ = −10ε + w + fe v = (1 + fe ) + w + fe v → fe = −11
s
1
fe = −1 − 1+ = −11 → r̃e = 0.01
r̃e

e) From d), αe = −10 and

ξ(t) = w(t) + fe v(t) = w(t) − 11v(t).

We also have

E[ξ(t)ξ(t + τ )] = E[w(t)w(t + τ )] + fe2 E[v(t)v(t + τ )]

= (qe + re fe2 )δ(τ ) = 2.21δ(τ )
A. Solutions to Exercises - Linear Quadratic Optimal Control 197

Solution to Problem 9.3 LQG and disturbance

Disturbance d enters the loop at a point inside the controller, which is not a realistic
assumption. The error dynamics is not controllable from d, so the transfer function from
d to y is same as that under state feedback. From Figure A.5 we get the transfer function

d
1 x y
F b c
s

Figure A.5: LQG controller

from d to y as
Y (s) cb 1
= c(sI − A − bf )−1 b = = .
D(s) s − A − bf s+2
Next if the disturbance enters the loop at a point outside the controller, at the plant
input, then the estimator dynamics do affect the controlled output. The controller transfer
function (from −y to u) is

K(s) = f (sI − A − bf − fe c)−1 fe

and the plant transfer function (from u to y) is

cb
G(s) = c(sI − A)−1 b =
s−a
Substituting f = −3 and fe = −4 in the transfer function from d to y gives

Y (s) G (bc)(s − a − bf − fe c) s+6 s+6

= = = 2 =
D(s) 1 + GK (s − a)(s − a − bf − fe c) + cbf fe s + 5s + 6 (s + 2)(s + 3)
Appendix B

Solutions to Exercises - Optimal and

Robust Control

Solution to Problem 11.1 LQG control of an aircraft

One way to construct the generalised plant P (s) is (manually) constructing its matrices.
Since the only states of the generalised plant in Fig. 11.3 are the states of the aircraft
AP = A. The rest can be constructed by direct comparison with Fig. 11.3 - note that the
h iT
input to P (s) is rT w1T w2T uT .

0 Q1/2
   
A e 0 B A 0 B 0 B
Q1/2 0 0 0 0 C 0 0 0 0
   
   
P (s) =   =  

 0 0 0 0 R1/2 


 0 0 0 0 R1/2 

−C I 0 −Re1/2 0 −C I 0 −Re1/2 0

where the second expression holds for the initial values Q = C T C and Qe = BB T (which
is often a good initial guess:the disturbance enters the same way as the input signal u and
we are interested in the output signals y). Set initially R = I, Re = I. The generalised
plant can now be constructed by: Gplant = mk_LQG_gplant(Gairc,R,Re).
Next the LQG controller can be designed: K = h2syn ( Gplant, 3, 3 )
(Alternatively one can use the lqg function in MATLAB).
Finally simulate the response closed-loop system to the desired reference step command
and update R and Re (and Q and Qe if necessary) to meet the design requirements.
MATLAB files: orc11_LQG.m, aircraft_model.m, aircraft_sim.mdl, mk_LQG_gplant.m

Solution to Problem 12.1 Matrix norms

200 Appendices

From the definition of the induced norm we get

kABxk kABxkkBxk kAyk kBxk

kABk = max = max ≤ max max = kAkkBk
x6=0 kxk x6=0 kBxkkxk y6=0 kyk x6=0 kxk

In the above proof the inequality

kABxk kAyk
max ≤ max
x6=0 kBxk y6=0 kyk

has been applied. We know that Bx is always a vector in the column space of the matrix
B and the column rank of this matrix might not be full. In this case we cannot produce
any arbitrary vector y through Bx which implies the correctness of this inequality.
B. Solutions to Exercises - Optimal and Robust Control 201

Solution to Problem 13.1 Singular value decomposition

Let A be an n × m matrix with a singular value decomposition U ΣV T = A, and the first r
singular values non-zero. Recall that the columns of U and V define bases for the various
matrix spaces - see Fig. B.1.
r m
r n

basis L.null space

basis null space

basis col. space

basis row space

U= V =

n
m

Figure B.1: Basis of matrix spaces using SVD

For the first matrix σ3 = 0 and thus the matrix has rank=2. The second matrix is diagonal
and the singular values are the magnitudes of the diagonal elements. The third matrix is
a vector, in which case the only singular value is the length of the vector. In this case the
first column of U is a normalised vector in the same direction.
MATLAB file: orc13_SVD.m

Solution to Problem 13.2 Rank of a matrix

The columns (and rows) of the first matrix are linearly dependent and thus it has rank 1.
This is also confirmed, from the SVD, since σ2 = 0. Same is true for the second matrix.
The third matrix has rank=2, but σ2 = 0. This is so because from numerical point of
view the matrix is still singular. Thus SVD provides more reliable computation of the
(numerical) rank of a matrix.
MATLAB file: orc13_SVD.m

Solution to Problem 13.3 SVD of an image

The SVD returns the following singular values
h i
1231.4 104 5.1 3.8 3.1 2.3 2.1 1.8 0.4 0.1

Because σ7  σ8 one can truncate the last 2. This would however not reduce the number
of bits transmitted. Since σ2  σ3 one can leave only the first 2 singular values, in
which case 2 matrices has to be transmitted: U12 and S12 = Σ12 V12T correspondingly with
2 · 10 = 20 and 2 · 10 = 20 elements or all-together (10 + 10) · 2 · 8 = 320 bits. The receiver
has to perform the multiplication U12 S12 to recover the image.
MATLAB file: orc13_SVD.m. See orc13_SVD_photo.m for more realistic example.
202 Appendices

Solution to Problem 14.1 Singular value plot

The desired singular value plot (sigma plot) can be obtained through sigma(Gyr), where
Gyr is the closed-loop transfer function with the LQG controller obtained in Problem 11.1.
For each frequency Gyr (jω) is just a (complex) matrix for which SVD can be performed
and the singular values plotted over ω. The plot is shown in Fig. B.2.

50
σ 1 (Gyr )
σ 2 (Gyr )
0 σ 3 (Gyr )
Singular Values [dB]

−50

−100

−150

−200

−250 −3
10 10−2 10−1 100 101 102 103 104
Frequency [rad/s]

Figure B.2: Singular value plot of the aircraft with LQG controller

The fact that for low frequencies all singular values are close to 0 dB, means that the
closed-loop system has static gain (also low-frequency gain) of about 1, which means that
good reference tracking can be expected. At high frequencies the singular values drop,
which means that noise signals will be suppressed. If the maximum singular value is too
large for high frequencies the controller should be redesigned and if this does not help,
better sensors needs to be selected for the channels corresponding to its input direction.

Solution to Problem 15.1 H2 and H∞ norm

In general the H2 and H∞ norms should be computed between the external inputs w and
the fictitious outputs z of the closed-loop system T (s), where T=lft(G,K), where G is the
generalised plant and K is the controller. However for the purpose of illustration one can
also compute the norms for Gyr (s).
The H2 norm of T (s) is 2.5040 (will vary depending on the chosen matrices Q, R, . . . ). By
using breakpoints in Matlab, one can trace that the function norm(T,2) calls the function
normh2.m, which first solves (15.8) and than uses (15.6) to compute the norm.
B. Solutions to Exercises - Optimal and Robust Control 203

The H∞ norm of T (s) turns out to be 2.4939. Again using breakpoints one can find
that the computation is performed in norminf.m. Two methods for computing H∞ were
discussed in the script: (A) gridding over frequencies and computing the singular value de-
composition for each frequency and (B) using the Hamiltonian matrix Mγ and performing
bisection steps over γ. The method implemented in Matlab uses a fast algorithm, pro-
posed by Bruisma and Steinbuch, that combines the above two.

Solution to Problem 15.2 Two LQG designs

The design procedure is similar to the one in Problem 11.1, but no reference signal is
considered in the generalised plant. Having constructed the two generalised plants corre-
sponding to the two designs the function h2syn.m can be used to design the controllers.
The H2 norm returned by the function is equal to the LQG performance index V . For the
first design it is 9.45 and for the second 792.5. The singular value plots of the closed-loop
transfer function from r to y with the two controllers is given in Fig. B.3.

50

0
Singular Values [dB]

−50

−100

−150 σ1 Design 1
σ2 Design 1
σ3 Design 1
−200 σ1 Design 2
σ2 Design 2
σ3 Design 2
−250 −3
10 10−2 10−1 100 101 102 103 104
Frequency [rad/sec]

Figure B.3: Singular value plot of the aircraft with two LQG controllers

At low frequency the singular values of Gyr2 (s) lie very close to the 0 dB line, where as
with the first design they are on ±5 dB. Thus we expect the second design to have better
reference tracking. At high frequencies the singular values with the second design start
decreasing later than with the first design showing higher bandwidth, but also higher
sensitivity to noise. Further at frequencies between 10 and 100 rad/s the increase above 1
of all singular values with the second design indicate that noise signals in this frequency
range will be actually amplified. In summary we should expect faster system response
and better reference tracking with the second design, but also higher sensitivity to high
204 Appendices

frequency noise. This can be validated by performing closed-loop simulations.

Matlab file: orc15_LQGsigma.m

Solution to Problem 15.3 Computing the H2 norm

The observability and controllability gramians of T (s) can be computed by Wo = gram(T,’o’)
and Wc = gram(T,’c’) respectively. Then the H2 norm can be simply obtained by
H2o = sqrt(trace(B’*Wo*B)) or H2c = sqrt(trace(C*Wc*C’)).
When using the impulse response it is important that the integration is performed cor-
rectly. The easiest way to do so is to use a fixed step size when computing the impulse
response, e.g. [g,t] = impulse(T,0:0.01:2) and later multiply the sum by the step
size (i.e., 0.01).
Matlab file: orc15_H2.m

Solution to Problem 15.4 Computing the H∞ norm

As discussed in the solution of Problem 15.1 there are several ways to compute the H∞
norm using Mγ . The easiest to implement is the “brute-force” approach, where one starts
from high γ and with a fixed step size decreases it until Mγ becomes with imaginary eigen-
values. A much faster results can be obtained using the bisection method, summarised in
the following algorithm:

1. Choose γmax , γmin and tolerance γtol (e.g. γmax = 1000, γmin = 0, γtol = 10−6 )

2. Set γ = (γmax + γmin )/2

3. If γmax − γmin < γtol Stop!

4. Construct Mγ and compute its eigenvalues

5. If Mγ has eigenvalues on the imaginary axis set γmin = γ, else set γmax = γ

6. Return to step 2

The H∞ norm is equal to γ (±γtol /2).

Matlab file: orc15_Hinf.m
B. Solutions to Exercises - Optimal and Robust Control 205

Solution to Problem 16.1 Three block approximation

From the definition of the H∞ norm
   
WS S

WS (jω)S(jω)
   
 WT T 
  <1 is equivalent to sup σ̄  WT (jω)T (jω)   < 1

ω

WK KS WK (jω)K(jω)S(jω)

∞

Consider the SISO case. Then using that for a vector

q √ q
σ̄(x) = λmax (xH x) = xH x = |x1 |2 + · · · + |xn |2

follows q
sup |WS S|2 + |WT T |2 + |WK KS|2 < 1
ω
which has a maximum approximation error when all three terms are equal, i.e.
q
sup 3|WS S|2 < 1
ω
q √
or supω |WS S|2 = kWS Sk < 1/ 3 = 0.577, which results in −4.8 dB approximation
error.

Solution to Problem 16.2 Sensitivity and input directions

The singular value plot of S(s) for the LQG design from Problem 14.1 is shown in Fig. B.4.
One can clearly see that in one of the input directions (corresponding to the minimum
singular value) the low-frequency sensitivity is low, meaning good reference tracking.
However the other two singular values are large (one even close to 2 dB) showing poor
disturbance rejection.
For example for ω = 10−3 rad/s the maximum and minimum singular values are 0.9802
and 0.0002 and the corresponding input directions
h iT
v1 = −0.0007 −0.0015 + 0.5704j −0.0025 + 0.8213j
h iT
v3 = 1.0000 0 + 0.0001j 0 + 0.0008j

Thus disturbances entering through the first input have less effect, than combination of
disturbances acting on all inputs.
Matlab file: orc16_sensitivity.m

Solution to Problem 16.3 Waterbed effect

The two systems visualise the waterbed effect. Note that the integrator in G1 (s) is only
there to give zero-steady state error for the step responses - the waterbed effect will be
2
also evident, for example G(s) =
(s + 1)(s + 2)
Matlab file: orc16_waterbed.m
206 Appendices

10

−10
Singular Values [dB]
−20

−30

−40

−50
σ 1 (S)
−60 σ 2 (S)
σ 3 (S)
−70 −3
10 10−2 10−1 100 101 102 103 104
Frequency [rad/sec]

Figure B.4: Singular value plot of the sensitivity of the aircraft with LQG controller

Solution to Problem 17.1 Mixed sensitivity design

The fact that G(s) and Gd (s) share common dynamics means that the disturbance is
not acting on the plant’s output, but rather enters inside the plant, as apparent from
Fig. 17.25. The block diagram can be re-arranged to the one in Fig. B.5.

Gd (s)

r u y
K(s) G(s)
-

Figure B.5: System block diagram

At a first time we can “neglect” the disturbance input during the design process and
concentrate only on the reference tracking. However the disturbance rejection properties
with the obtained controller have to be analysed and changes to the design might be
needed.
For the purpose of reference tracking the suggested problem formulation with sensitivity
and control sensitivity filters can be used. The generalised plant is shown in Fig. B.6.
To construct the generalised plant one needs to select the WS (s) and WK (s) filters. To
B. Solutions to Exercises - Optimal and Robust Control 207

WK (s) zK
r
WS (s) zS
y
u G(s) - e

Figure B.6: Generalised plant for the H∞ design

meet the design requirements several iterations over/adjustments of the filters might be
needed. From the requirements for reference tracking we know that the sensitivity S(s)
should be low at low frequencies. Since the H∞ design algorithm will aim on achieving
kWS (s)S(s)k∞ < 1 this means that the gain of WS (s) should be high at low frequencies.
Further it is known, that S(s) cannot be kept low at all frequencies, i.e. for high frequen-
cies the gain of WS (s) should become low. A reasonable initial filter WS (s) is a first order
(low-pass) system
wS 1 wS
WS (s) = NS =
s + wS MS s + wS
Notice that NS directly sets the gain of WS (s) for low frequencies, or alternatively MS
sets an upper-boundary on S(s) for low frequencies.
In order to select MS and wS one can have a look the sigma plot of Gd (s), shown in
Fig. B.7. The disturbance is a low-frequency one, has low-frequency gain of 10 and a
crossover frequency of 0.1 rad/s. Thus select MS = 1/10 and wS = 0.1 rad/s. A good
initial guess for the control sensitivity is WK (s) = 1.
The generalised plant can be constructed with the following code

systemnames = ’G Ws Wk’;
inputvar = ’[r; u]’;
input_to_G = ’[u]’;
input_to_Wk = ’[u]’;
input_to_Ws = ’[r-G]’;
outputvar = ’[Wk; Ws; r-G]’;
GSYS = sysic;

The H∞ controller can be designed with the function hinfsyn

[K, Gcl, hinf] = hinfsyn(GSYS, nmeas, ncont)
where nmeas and ncont are the number of measured outputs and controlled inputs (in
this case both equal 1); Gcl is the closed-loop
" of the# generalised plant and the controller;
W S
S
hinf is the H∞ norm of Gcl, i.e. hinf =


WK KS ∞
208 Appendices

20
σ (Gd )
15

10
Singular Values [dB]
5

−5

−10

−15

−20 −3
10 10−2 10−1 100 101
Frequency [rad/sec]

Figure B.7: Sigma plot of Gd (s)

Simulating the closed-loop response with the obtained controller shows that the system
response to a reference step is slow and the control input is not efficiently used u(t) <
0.3. In order to improve these one can further reduce MS to MS = 0.01, construct the
generalised plant and repeat the design. Now the reference response satisfies the design
requirements, but the disturbance rejection is poor. “Pushing down” the sensitivity by
selecting MS = 0.001 and setting wS = 0.01 rad/s yields a controller satisfying the design
requirements posed in the problem statement. The sensitivity and control sensitivity are
shown in Fig. B.8 together with the sigma plots of the inverse filters. The fact that hinf
is greater than 1 indicates that in the filters are “imposing” a performance criteria that
cannot be met. By appropriately re-scaling the filters the same controller can be obtained,
but with norm equal to 1.
Matlab file: orc17_Hinf_design.m
The disturbance rejection of the obtained controller are however not satisfactory for the
stronger disturbance Gd (s) = 100/(10s + 1). By selecting a stronger sensitivity filter
(smaller MS ) the disturbance rejection can be improved by at the price of violating the
actuator constraints.
One approach to simultaneously address both the reference tracking and the disturbance
rejection is to take into account both the transfer functions from reference to output and
control action, and from disturbance input to output and control action. For this purpose,
the generalised plant in Figure B.9 can be used.
Combining all the individual transfer functions to a single one and minimising its H∞
B. Solutions to Exercises - Optimal and Robust Control 209

100

Singular Values [dB]

σ(S)
50 1/Ws

−50

−100 −4
10 10−3 10−2 10−1 100 101 102 103 104 105
Frequency [rad/sec]
20
Singular Values [dB]

σ(KS)
0 1/Wk

−20

−40

−60 −4
10 10−3 10−2 10−1 100 101 102 103 104 105
Frequency [rad/sec]

Figure B.8: Sigma plot of S(s) and K(s)S(s) with the obtained controller

r
d
WK (s) zK

0.05
WS (s) zS
1 200
u (0.05s+1)2 10s+1 e
y -

Figure B.9: Generalised plant with reference and disturbance inputs

norm results in the so called 4-block design

" #
W S WS SGd
S



(B.1)
WK KS WK KSGd
∞

This can be easily seen by closing the loop u = Ke, computing the transfer function and
using that k − G(s)k∞ = kG(s)k∞
" # " #" #
zS WS S −WS SGd r
=
zK WK KS −WK KSGd d

Performing the design with this new generalised plant, without re-configuring the filters
results in a controller that has excellent disturbance rejection, but high overshoot and
control signal to reference tracking - Fig. B.10. The singular value plots of the four
transfer functions in (B.1) are shown in Fig. B.11. As can be seen both S and SGd have
210 Appendices

1.5

Output y
1

0.5
r → y
d → e
0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]
3
r → u
2 d → u
Control u

−1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Time [sec]

1 1
Figure B.10: Step and disturbance responses WK = 1, WS (s) = 0.001 s/0.01+1

peak values over 1 at frequency of approx. 10 rad/s, which is known to be the reason for
the overshoot in the reference step response. One way to reduce this peak values of S
and SGd is by using the WK (s) filter. Another option is to use WS (s) filter and impose
an upper limit for the sensitivity also for high frequencies.

100 100
Singular Values [dB]

Singular Values [dB]

σ(S) σ(SGd )
50 1/Ws 50 1/Ws

0 0

−50 −50

−100 −100
10−3 10−1 101 103 105 10−3 10−1 101 103 105
Frequency [rad/sec] Frequency [rad/sec]
20 0
Singular Values [dB]

Singular Values [dB]

0
−50
−20
−100
−40 σ(KS) σ(KSGd )
1/Wk 1/Wk
−60 −150
10−3 10−1 101 103 105 10−3 10−1 101 103 105
Frequency [rad/sec] Frequency [rad/sec]

Figure B.11: Sigma plots by the 4-block design

B. Solutions to Exercises - Optimal and Robust Control 211

For example using the bi-proper sensitivity filter

1 s/(cωs ) + 1
WS (s) =
MS s/ωs + 1

with ωs = 0.01 rad/s, MS = 0.02 and c = 80 obtains a controller which achieves closed-
loop performance close to the required - see Figures B.12 and B.13.

1.5
Output y

0.5
r → y
d → e
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3
Time [sec]
1
r → u
0.8 d → u
Control u

0.6
0.4
0.2
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3
Time [sec]

Figure B.12: Step and disturbance responses for the second 4-block design

Matlab file: orc17_Hinf_design_2.m

However close, meeting the design requirements remains a difficult task. Partially the
problem is due to the presence of 4 transfer functions that have to be shaped, but only 2
design parameters WS (s) and WK (s). This problem can be easily remedied using prefilters
for the reference signal and the disturbance signal, as shown in Fig. B.14.
212 Appendices

50 50

Singular Values [dB]

Singular Values [dB]

0 0

−50 −50
σ(S) σ(SGd )
1/Ws 1/Ws
−100 −100
10−3 10−1 101 103 105 10−3 10−1 101 103 105
Frequency [rad/sec] Frequency [rad/sec]
20 0
Singular Values [dB]

Singular Values [dB]

0
−50
−20
−100
−40 σ(KS) σ(KSGd )
1/Wk 1/Wk
−60 −150
10−3 10−1 101 103 105 10−3 10−1 101 103 105
Frequency [rad/sec] Frequency [rad/sec]

Figure B.13: Sigma plots for the second 4-block design

In this case, the 4-block design becomes

r Vr (s)

d Vd (s)
WK (s) zK

0.05
WS (s) zS
1 200
u (0.05s+1)2 10s+1 e
y -

Figure B.14: Generalised plant with prefilters for the reference and disturbance inputs

" #
W SV WS SGd Vd
S r



(B.2)
WK KSVr WK KSGd Vd
∞

Matlab file: orc17_Hinf_design_3.m

In order to meet the design requirements one can try using higher order shaping filters
and shaping the complementary sensitivity.
Matlab file: orc17_Hinf_design_4.m
B. Solutions to Exercises - Optimal and Robust Control 213

During the preparation of this solution no controller was found meeting all of the design
requirements. Thus the questions whether such controller exists and how to find it are
still open.
214 Appendices

Solution to Problem 17.2 H∞ control of an aircraft

The H∞ controller designs presented in the lecture and some improved designs are avail-
able in the Matlab solution file. In order to achieve both good reference tracking and
disturbance rejection a 4-block design similar to the one presented in the solution of
Problem 17.1 will be needed. This time, however the inputs to the generalised plant are
the reference signal r and the input disturbance d, see Fig. B.15.

r Vr (s)
WK (s) zK

d Vd (s)
WS (s) zS

u G(s) e
y -

Figure B.15: Generalised plant with reference input and input disturbance

Matlab file: orc17_Hinf_aircraft.m

a) Ignoring the input disturbance and the noise effects, Design 4 from Chapter 17 can
be modified to satisfy the actuator constraints by slightly reducing the bandwidth
requirements (reducing ωS ). The step response is shown in Fig. B.16, with settling
time for the relative altitude (y1 ) of approximately 1.2 seconds.
ωS MS ωK MK
Design 6:
2.5 · 10−4 10−4 102 102
Note, that since the design requirements do not specify actuator constraints for
inputs u1 and u2 one can set the WK (s) as
 
δ 0 0
 
WK (s) = 0 δ
 0 

1 s/ωK +1
0 0 MK s/(cωK )+1

where δ ≥ 0 is a small number. Then the settling time can be reduced to approx.
1 sec, but at the expense of (unrealistically) high control in the first two input
channels.

b) The system response to a sinusoidal measurement noise with amplitude of 0.1 and
frequency of 104 rad/s (injected in all outputs) is shown in Fig. B.17.
Note that since the transfer function from noise to output is S(s), and the sensi-
tivity is close to 1 for large frequencies the noise is not attenuated. This is natural
and cannot be avoided. However, since the system will not respond to this high-
frequency noise, there is not a reason for the controller to attempt to compensate
B. Solutions to Exercises - Optimal and Robust Control 215

1.5
y1
1 y2

Output y
y3
0.5

−0.5
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]
10

0
Input u

−10
u1
−20 u2
u3
−30
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]

Figure B.16: Design 6: response to r(t) = [σ(t) 0 0]T

0.2

0.1
Output y

0
y1
−0.1 y2
y3
−0.2
0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20
Time [sec]
2
Control u

−2 u1
u2
u3
−4
0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20
Time [sec]

Figure B.17: Design 6: response to a sinusoidal noise injected in all outputs

it. Thus the only improvement of interest is a reduction of the magnitude of the
control signals. This can be achieved by reducing ωK , and compensating the result-
ing slower-system response by slight increase in ωS . The tracking performance and
the noise response are shown in Fig. B.18 and Fig. B.19.
216 Appendices

ωS MS ωK MK
Design 7:
3.5 · 10−4 10−4 10 102

1.5
y1
1 y2
Output y
y3
0.5

−0.5
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]
20

10
Input u

0
u1
−10 u2
u3
−20
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
Time [sec]

Figure B.18: Design 7: response to r(t) = [σ(t) 0 0]T

c) The system response to a step input disturbance applied to the spoiler angle (u1 )
is shown in Fig. B.20. It is clear, that whereas the disturbance has no big effect on
the y2 and y3 it leads to significant oscillations in the relative altitude y1 and to a
steady-state error.
To improve the disturbance rejection a generalised plant with only disturbance input
(see Fig. 17.19) can be used, but the resulting controllers will have poor reference
tracking. To achieve both good reference tracking and disturbance rejection a gen-
eralised plant with both r and d as inputs can be constructed, as shown in Fig. B.15,
resulting in the 4-block design (2 inputs (r and d) times 2 outputs (zK and zS )).
The pre-filters Vr (s) and Vd (s) are added to allow for independent shaping of the
four transfer functions
" #" #" # " #
W
S 0 S SG Vr 0 W SV
S r WS SGVd
= .


0 WK KS KSG 0 Vd WK KSVr WK KSGVd
∞ ∞

Designing a controller with this new generalised plant using the parameters from
Design 7 and identity gains as pre-filters (Vr (s) = I3 , Vd (s) = I3 ) leads to closed-loop
system with large overshoot at reference step, but very good disturbance rejection
capabilities, as shown in Fig. B.21 (notice the different time-scale of the disturbance
response plots compared to Fig. B.20).
B. Solutions to Exercises - Optimal and Robust Control 217

0.2

Output y
0.1

0 y1
y2
y3
−0.1
0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20
Time [sec]
0.5
Control u

−0.5 u1
u2
u3
−1
0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20
Time [sec]

Figure B.19: Design 7: response to a sinusoidal noise injected in all outputs

2
y1
1 y2
Output y

y3
0

−1

−2
0 10 20 30 40 50 60 70 80 90 100
Time [sec]
1
u1
u2
Control u

0.5 u3

−0.5
0 10 20 30 40 50 60 70 80 90 100
Time [sec]

Figure B.20: Design 7: response to d(t) = [σ(t) 0 0]T

The following steps were taken to obtain a better controller

– Vd (s) = 0.01 I3 - set smaller priority on the disturbance rejection: reduces the
reference tracking overshoot (at the expense of slower disturbance rejection),
but also results in very slow convergence of y1 to the reference value.
218 Appendices

Reference step [σ 0 0]T Input disturbance [σ 0 0]T

1.5 0.1

1 5 · 10−2

Output y 0.5 0
y1
0 −5 · 10−2 y2
y3
−0.5 −0.1
0 0.5 1 1.5 2 0 2 4 6 8 10
Time [sec] Time [sec]
20 1
u1
u2
0 0.5 u3
Input u

−20 0

−40 −0.5
0 0.5 1 1.5 2 0 2 4 6 8 10
Time [sec] Time [sec]

Figure B.21: Design 7: responses after 4-block design

 
5w 0 0
 S 
– WS (s) =  0 wS
 0  - give higher value to the error in the first channel:
0 0 10wS
results fast tracking, reasonable disturbance rejection (with no steady-state
error), but violates |u3 | ≤ 20.
– ωS = 3 · 10−5 - reduce the bandwidth requirement: makes the system response
slower, but reduces the 3rd input to satisfy the actuator constraints. Shows
cross-coupling between the outputs (reference step in y1 results in some re-
sponse in y2 and y3 ).

With the so-obtained design parameters, shown below, the closed-loop system has
the response shown in Fig. B.22.
Design 8:
ωS MS 
WS (s) 
ωK MK 
WK (s)  Vr Vd
5w 0 0 w 0 0
 S  K
3 · 10−5 10−4 102
 
 10  I3 0.01 I3
 0 wS 0   0 wK 0 
 
0 0 10wS 0 0 wK
where
1 1 1 ωS
wS (s) = s =
MS ωS
+1 MS s + ωS
s
1 ωK + 1 c s + ωK
wK (s) = s = ,
MK cωK + 1 MK s + cωK
B. Solutions to Exercises - Optimal and Robust Control 219

Reference step [σ 0 0]T Input disturbance [σ 0 0]T

1.5 1

1
Output y
0
0.5
−1 y1
0 y2
y3
−0.5 −2
0 0.5 1 1.5 2 0 20 40 60 80 100
Time [sec] Time [sec]
10 1
u1
0 u2
0.5 u3
Input u

−10
0
−20

−30 −0.5
0 0.5 1 1.5 2 0 20 40 60 80 100
Time [sec] Time [sec]

Figure B.22: Design 8: responses after 4-block design

Matlab file: orc17_Hinf_aircraft_2.m

220 Appendices

Solution to Problem 18.1 Pole regions using LMIs

Start from the requirement that the poles have real part between αl and αr , see Fig. B.23
(right). Let s = x + jy be a point inside the region. Then αl < x < αr . This can be
rewritten as 2 inequalities

2αl − 2x < 0 and 2x − 2αr < 0

On another hand taking the 2 × 2 matrices L and M results in

" # " # " #
l11 l12 m11 m12 m11 m21
+ (x + jy) + (x − jy) < 0
l12 l22 m21 m22 m12 m22
" #
l11 + m11 2x l12 + m12 (x + jy) + m21 (x − jy)
<0
l12 + m21 (x + jy) + m12 (x − jy) l22 + m22 2x
The last can be used to express the desired 2 inequalities by selecting l12 = 0, m12 =
m21 = 0 (to get the off-diagonal terms equal to zero), l11 = 2αl , m11 = −1 and l22 = −2αr ,
m22 = 1, resulting in the matrices
" # " #
2αl 0 −1 0
L= , M=
0 −2αr 0 1

Im Im

s0
s
β β
s
y
Re Re
x αl x αr

Figure B.23: LMI regions

Now consider the cone region in Fig. B.23 (left), and a point s = x + jy in it. Let
s0 = x + jy0 be a point at the boundary of the region. Then
|x| |x|
tan β = < , for all s ∈ D
|y0 | |y|
This can be rewritten as
2 sin2 β x2
tan β = < 2 ⇒ x2 cos2 β − y 2 sin2 β > 0
cos2 β y
B. Solutions to Exercises - Optimal and Robust Control 221

h i
Further, x < 0 for all s ∈ D. Since β ∈ 0 90◦ , cos β > 0 and thus x cos β < 0 for all
s ∈ D. The problem is now write the above 2 conditions in the form
" #
k k
K = 11 12 < 0
k21 k22

Using Schur complement, K can be written as

−1
k22 < 0, k11 − k12 k22 k21 < 0

Multiplying the second equation by k22 , which is a negative number, yields k11 k22 −
k12 k21 > 0. The similarity with the desired inequalities becomes noticeable.

k22 < 0 x cos β < 0

k11 k22 − k12 k21 > 0 x2 cos2 β − y 2 sin2 β > 0

To obtain the desired matrices, recall that s + s̄ = 2x and s − s̄ = 2jy. Thus select
" # " # " # " #
k11 k12 x cos β −jy sin β cos β − sin β cos β sin β
=2 = (x + jy) + (x − jy)
k21 k22 jy sin β x cos β sin β cos β − sin β cos β

and therefore " # " #

0 0 cos β − sin β
L= , M= .
0 0 sin β cos β
Here, the Schur complement is applied to the Hermitian matrix K. In problem 4.2, The
Schur complement has been defined for the symmetric matrices and it is straightforward
to extend the result of this problem to the case of Hermitian matrices.

Solution to Problem 18.2 H2 and H∞ norm computation via LMIs

Let’s start from the H∞ norm computation. Theorem 18.3 gives a condition to check,
whether the H∞ norm is smaller than γ. Thus, the smallest γ for which the LMI in
(18.22) holds will be equal to the H∞ norm. Therefore one can compute the norm, by
solving the following problem
 
AT P + P A P B C T
BT P −γI DT  P = P T > 0.
 
minimise γ subject to 
  < 0,
C D −γI

This can be solved using the mincx function of the LMI lab toolbox in MATLAB, which
can solve problems of the type

minimise cT p subject to M (p) < 0,

P
where M (p) = k Mk pk is an LMI in p.
Start by defining the beginning of a set of LMIs: setlmis([]); which is always the first
command. Next, define the decision variables P = P T and γ.
222 Appendices

P = lmivar(1,[nstate 1]); % symmetric, [n x n full block]

g = lmivar(1,[1 0]); % symmetric, [1 x 1 scalar]

where n is the dimension of A. If instead of a symmetric a full-block n × m matrix Q has

to be defines, this can be done via Q = lmivar(2, [n m]).
Next, define the beginning of a new LMI by Mk = newlmi();. One can enumerate the
LMIs manually, but it is recommended to do it automatically to avoid mistakes. Now the
elements of the Mk < 0 can be defined. Since Mk is a symmetric matrix, only one of the
off-diagonal elements should be specified, that is either Mkij or Mkji , but not both.
The LMIs are specified with the command lmiterm, having the following syntax:

lmiterm([Mk i j X],A,B,’s’);

The meaning of the variables is as follows: Mk that the term has to be added to the k-th
LMI; i j indicates the position in the LMI, that is Mkij ; X is variable; A and B are matrices
multiplying X by the left and right, that is Mkij = AXB. If present ’s’ indicates, that
the term is symmetric: Mkij = AXB + B T X T AT . Since lmiterm assumes, that the LMI
is Mk < 0, if 0 < Mk has to be defined this can be done by writing -Mk. Finally a constant
term Mkij = C can be defined as lmiterm([Mk i j 0],C);.
The LMIs for computing the H∞ norm can then be defined by

M1 = newlmi();
lmiterm([M1 1 1 P],1,A,’s’); % M11 = (*) + I*P*A
lmiterm([M1 1 2 P],1,B); % M12 = I*P*B
lmiterm([M1 1 3 0],C’); % M13 = C’
lmiterm([M1 2 2 g],-1,1); % M22 = -I*g*I
lmiterm([M1 2 3 0],D’); % M23 = D’
lmiterm([M1 3 3 g],-1,1); % M33 = -I*g*I
M2 = newlmi();
lmiterm([-M2 1 1 P],1,1); % 0 < P
mylmi = getlmis();

where getlmis() gathers the 2 LMIs in one.

The last task before solving the minimisation problem is to express it in the form cT p.
This can be done by:

n = decnbr(mylmi); % number of decision variables

c = zeros(n,1); % prepare the vector
for i = 1:n % loop over all decision variables
gi = defcx(mylmi,i,g); % g matrix when decision var. i is =1
B. Solutions to Exercises - Optimal and Robust Control 223

% and all other dec. vars = 0

c(i) = gi; % how c’p depends on gi
end

For example, if the objective function is g +xT0 P x0 , the above following code can construct
the corresponding c vector.

n = decnbr(mylmi); % number of decision variables

c = zeros(n,1); % prepare the vector
for i = 1:n % loop over all decision variables
[gi,Pi] = defcx(mylmi,i,g,P); % g and P when only dec. var i is = 1
c(i) = gi + x0’*Pi*x0; % how c’p depends on gi
end

Finally solve the optimisation problem

[cost, par] = mincx(mylmi,c);

The H∞ norm is equal to cost. The decision variables P and g can be extracted as

P = dec2mat(mylmi,par,P);
g = dec2mat(mylmi,par,g);

Similarly the LMI conditions for computing the H2 norm can be constructed.
Matlab files: lmi_norm_hinf.m, lmi_norm_hinf.m, orc18_norms_via_LMIs.m
Besides the LMI Lab of the Robust Control toolbox of MATLAB there are many other
LMI solvers available. Worth mentioning here is the “Yet Another LMI Parser” (YALMIP)
that provides an interface between the definition of the LMIs and many LMI solvers
(including the LMI Lab). Two freeware solvers worth mentioning are the SEDUMI and
SDPT3 toolboxes. A lengthier list of LMI solvers for MATLAB can be found in the
documentation of YALMIP.

Solution to Problem 19.1 H2 synthesis using LMIs

" #
Y I
To synthesise a controller the LMI conditions (19.19), (19.20) and > 0 are needed
I X
as constraints and trace(W ) is minimised. Note that in order to impose Dzw +Dzu D̃K Dvw =
0 the feed-through matrix DK of the controller is set to zero. This accordingly simplifies
the LMI conditions.
Matlab files: orc19_H2_design_LMI.m, lmi_h2syn.m
224 Appendices

Solution to Problem 19.2 H∞ state-feedback design with pole-region con-

straints
Because this is a state-feedback design and furthermore there are pole-region constraints
imposed, the standard MATLAB function hinfsyn cannot be used in this problem. The
synthesis is a direct implementation of LMIs (19.10), (19.5) and P = P T > 0. Expanding
inequality (19.5) in the case of conic sector and vertical strip pole-region constraints leads
to
   
0 0 0 0 cos(β) − sin(β) 0 0
0 0 0 0  sin(β) cos(β) 0 0
   
 
 ⊗P +
  ⊗ (AP + Bu Y )
0

0 2αl 0   0 0 −1 0

0 0 0 −2αr 0 0 0 1
 
cos(β) sin(β) 0 0
− sin(β) cos(β) 0 0
 
+
  ⊗ (AP + Bu Y )T < 0.
 0 0 −1 0

0 0 0 1

Using the short notations s = sin(β), c = cos(β) and Q = AP + Bu Y the above LMI can
be rewritten as

cQ + cQT −sQ + sQT

 
0 0
sQ − sQ T T
cQ + cQ 0 0
 

  < 0.

 0 0 2αl P − Q − QT 0 

T
0 0 0 −2αr P +Q+Q

Matlab files: orc19_Hinf_design_LMI.m, lmi_hinfsyn_SFB.m

B. Solutions to Exercises - Optimal and Robust Control 225

Solution to Problem 21.1 ACC Benchmark Problem

The uncertainty model can be constructed in 3 different ways.

1. Using the physical model of the system. This is possible, only when it is known
which are the varying coefficients and what is their range.
Here only a simplified construction from the physical model is presented. A more
detailed examples can be found, for example, in the documentation of the Matlab
“µ-Analysis and Synthesis Toolbox” from Balas et.al., and in the book “Robust
Control Design with MATLAB” from Gu et.al. (note that both books use older
version of the Robust Control Toolbox than the one discussed in the exercises).
In the provided model the only physical parameter that varies is k, and kmin ≤
k ≤ kmax . It is always possible to rewrite it as k = k0 + k̃δ, where |δ| ≤ 1, where
k0 = (kmax + kmin )/2 and k̃ = (kmax − kmin )/2. Then as signal u multiplied by k can
be equivalently represented in a linear-fractional transformation form, as shown in
Fig. B.24.

" #
ku 0 1 u
k̃ k0

Figure B.24: Linear fractional transformation of an uncertain element.

Using this the system in (21.6) can be augmented with an additional input and
output connected to δ, as shown below.
       
ẋ1 0 0 1 0 x1 0 0 0 0 w
ẋ2  0 0 0 1
 x2  +  0 0 0 0 
u
       
     
 =
ẋ3  −k0 /m1 k0 /m1 0 0 x3  −k̃ 1/m1 1/m1 0  d1 
       

ẋ4 k0 /m2 −k0 /m2 0 0 x4 k̃ 0 0 1/m2 d2

z = x1 − x2
y = x2

where w = δz (perform the substitution to check the validity!).

The advantage of this approach is the clear connection between the physical pa-
rameters and δ. Using such models it becomes also clear, that whenever there are
several independent parameters that vary, combining them to a single uncertainty
matrix ∆, with k∆k ≤ 1 is restrictive, since it includes more variations than the
possible ones. This problem is addressed by the µ-synthesis approach (see the above
references).
226 Appendices

2. Using the uncertain state-space model functions in Matlab (see help uss). These
functions allow defining uncertain gains, matrices and state-space models and using
them in standard Matlab functions.
For the above system, one can define: k = ureal(’k’, 1, ’range’, [0.5 2]);
and use it in the construction of the A matrix (which in return is an uncertain ma-
trix) and the system model G (which turns out as uncertain state-space model).
Then the function lftdata can be used to separate the uncertain model to a fixed
part and an uncertain varying part: [Go, Delta] = lftdata(G).
Note, that the function lftdata appears to work correctly only when the uncertain
parameters are defined with the middle value (that is k0 in the case of k as nominal
one). That is, lftdata will return the correct fixed part Go, only when k is defined
as: k = ureal(’k’, 1.25, ’range’, [0.5 2])

3. Using the uncertainty model construction technique presented on page 163. Note,
that wherever the previous two approaches can be used only in the case, where it
is known what the varying parameter is, this method can be also applied, when
one has several models of the same system. For example, one can linearise a non-
linear model (or identify a system from measurement data) for different operating
conditions, and thus obtain several different models of the same system. Then, using
this approach, an uncertainty model can be constructed that captures the variation
in the system in the form of uncertainty.
A vital issue when using these models is the selection of the nominal system (that
is the nominal A matrix). For simplicity take only the spring constant (instead of
the whole A matrix). It is known that 0.5 ≤ k ≤ 2. Consider two different choices
for the nominal value: k0 = 1 and k0 = 1.25. Since the uncertainty δ is normalised
(recall that k∆k ≤ 1 for the small gain theorem) each of the two choices of k0 will
lead to different scaling factors. When k0 = 1 a scaling k̃ = 1 will be needed, in
order to include kmax = 2, but this will also include, for example k = 0 which is
equivalent to no spring at all - that is the position of the second cart cannot be
influenced by moving the first one. When k0 = 1.25, the tightest (for this problem)
uncertainty description is achieved (no plants included outside the range of interest)
with k̃ = 0.75. This is illustrated in Fig. B.25.

Bad choice
kmin k0 kmax k
Good choice
kmin k0 kmax k

Figure B.25: Uncertainty description depending on the nominal model/value. The filled
area represents the values of k captured by the uncertainty model.
B. Solutions to Exercises - Optimal and Robust Control 227

The last two approaches are demonstrated in the Matlab solution to this problem.
(orc21_ACC1_model.m)
Having once constructed the model with the uncertainty there are different techniques
that one can use for the controller design.

• Nominal Performance [NP] - design a controller for the nominal system. This can be
done with any of the previously presented techniques (LQR, H2 , mixed-sensitivity,
etc.). However such a controller will not guarantee the stability of the two-cart
system under variations in the spring constant.

• Robust Stability [RS] - design a controller that stabilises the closed-loop system
under all plant variations captured by the uncertainty model. To do this one needs
to construct a generalised plant as the one shown in Fig. B.26. Note, that besides
having only the uncertainty input one can add also the disturbance inputs. This
is demonstrated in orc21_ACC2_hinfsyn_uss.m and orc21_ACC2_hinfsyn_unc.m
correspondingly for modelling techniques 2 (uss) and 3 (unc) above.

∆ ∆

w∞ z∞ w∞ z∞
P d P
u v u v

Figure B.26: Generalised plant for robust stability design.

• Robust Stability and Nominal Performance [RS+NP] - design a controller guaran-

teeing the robust stability of the system under all plant variations and achieving
some nominal performance. For example, using LMIs, one can combine the require-
ment for robust stability under the model uncertainty (that is, the H∞ norm from
w∞ to z∞ being less than 1) with LQG, H2 , mixed-sensitivity H∞ or/and pole-
placement requirements for the nominal system. Note, that a controller satisfying
this design requirements will guarantee the robust stability, but only the nominal
performance. This is, however, in many cases sufficient, since the nominal perfor-
mance often provides some “useful” performance under small changes in the plant.
These designs are relatively straightforward, since they require combining the nec-
essary LMI conditions. The pole region constraint approach and the mixed H2 /H∞
approach are presented in the Matlab file orc21_ACC3_othersyn.m.

• Robust Performance [RP] - design a controller that satisfies the specified perfor-
mance criteria for all plants modelled by the uncertainty model. This is a much
stronger requirement than the previous ones and more difficult to satisfy. To per-
form the design a generalised plant P (s) with both uncertainty channels (z∞ , w∞ )
228 Appendices

and “mixed-sensitivity-type” performance channels (zP , wP ) is constructed - see

Fig. B.27. If a controller guarantying H∞ norm smaller than 1 from w to z is ob-
tained it will guarantee robust performance, i.e. no matter how the plant changes
(k∆k ≤ 1) the performance requirements will be satisfied. The solution is presented
both with modelling techniques 2 (uss) and 3 (unc) of the above showed ones.

w w∞ z∞ z
wP P (s) zP

u v
K(s)

Figure B.27: Generalised plant with both stability and performance channels.

Looking at the frequency response of the controller it can be seen that the controller
does not have a sufficient roll-off for high frequencies. The setup used so far weights the
transfer functions SGd and KSGd . Both of them should decrease for high frequencies, so
first order filters seem to be a reasonable choice for shaping the transfer functions. This
choice enforces a 20 dB/dec decrease of the transfer functions. For high frequencies S is
assumed to be constant. The plant’s frequency response to disturbances Gd decreases with
40 dB/dec. Therefore, the requirement of a 20 dB/dec decrease of the transfer functions
is already fulfilled without further roll-off in K. K can even increase with 20 dB/dec or
at least be constant for high frequencies without violating the requirements given by the
shaping filters.
In general, there are two possible improvements to deal with this problem. One possibility
would be a higher order shaping filter to enforce sufficient roll-off in K. Above that, the
four-block design allows additional weighting of the transfer functions S and KS and
leads to an appropriate controller in frequency domain.
For setting up the four-block design a second input channel r acting on the plant’s output
has to be introduced. In a second step, the shaping filters have to be tuned adequately.
As explained before, there should be a sufficient roll-off in the controller transfer func-
tion for high frequencies. Above that, the disturbance responses should meet the given
requirements. Increasing the tuning parameter M2 by a factor of four gives satisfying
results. For further details please consult the Matlab file orc21_ACC4_4block.m.
B. Solutions to Exercises - Optimal and Robust Control 229

Matlab files:
orc21_ACC1_model.m, uncertainty_model.m
orc21_ACC2_hinfsyn_unc.m, orc21_ACC2_hinfsyn_uss.m
orc21_ACC3_othersyn.m
orc21_ACC4_4block.m