Section 11.

7 Optimal Control Systems 859

1
Compensator

Observer Control Law System Model

FIGURE 11.13
State variable I H x = Ax + Bu
k = (A - BK - LC)x + L(.y - r) - K
compensator with y = Cx
reference input and
N = 0 and
M = -L.

compensator is in the forward path. These two implementations are representative of

the possibilities open to control system designers when considering reference inputs.
Depending on the choice of N and M, other implementations are possible. For
example, Section 11.8 presents a method of tracking reference inputs with guaran-
teed steady-state tracking errors using internal model design techniques.

11.7 OPTIMAL CONTROL SYSTEMS

The design of optimal control systems is an important function of control engineer-

ing. The purpose of design is to realize a system with practical components that will
provide the desired operating performance. The desired performance can be readily
stated in terms of time-domain performance indices. For example, the maximum
overshoot and rise time for a step input are valuable time-domain indices. In the case
of steady-state and transient performance, the performance indices are normally
specified in the time domain; therefore, it is natural that we wish to develop design
procedures in the time domain.
The performance of a control system can be represented by integral performance
measures, as we found in Section 5.7. Therefore, the design of a system must be based
on minimizing a performance index, such as the integral of the squared error (ISE), as
in Section 5.7. Systems that are adjusted to provide a minimum performance index are
often called optimal control systems. In this section, we will consider the design of an
optimal control system that is described by a state variable formulation. We will con-
sider the measurement of the state variables and their use in developing a control sig-
nal u(t) so that the performance of the system is optimized.
The performance of a control system, written in terms of the state variables of a
system, can be expressed in general as

J = g(x, u, t) dt, (11.26)

where x equals the state vector, u equals the control vector, and tt equals the final time.1
We are interested in minimizing the error of the system; therefore, when the de-
sired state vector is represented as x(J = 0, we are able to consider the error as iden-
tically equal to the value of the state vector. That is, we intend the system to be at

'Note that to denote the performance index. J is used instead of /, as in Chapter 5. This will enable the
reader to distinguish readily the performance index from the identity matrix, which is represented by the
boldfaced capital I.
860 Chapter 11 The Design of State Variable Feedback Systems

Control signals State variables

Control + x2
FIGURE 11.14 system
A control system in + x,
terms of x and u.

equilibrium, x = x^ = 0, and any deviation from equilibrium is considered an error.

Therefore, in this section, we will consider the design of optimal control systems
using state variable feedback and error-squared performance indices [1-3].
The control system we will consider is shown in Figure 11.14 and can be repre-
sented by the vector differential equation
x = Ax + Bu. (11.27)
We will select a feedback controller so that u is some function of the measured state
variables x and therefore
u = -k(x).
For example, we might use

Mi = ~kiX\, «2 =
~&2*2> •••* ^)11-^111- (11.28)
Alternatively, we might choose the control vector as

"l = ~k\{x\ + x2), u2 = -&2O2 + x3), (11.29)

The choice of the control signals is somewhat arbitrary and depends partially on the
actual desired performance and the complexity of the feedback structure allowable.
Often, we are limited in the number of state variables available for feedback, since
we are only able to use measurable state variables.
In our case, we limit the feedback function to a linear function so that
u = — Kx, where K is an m X n matrix, as in Section 11.3. Therefore, in expanded
form, we have

«1 -11 •In
u2 x2
(11.30)

•m\ k,

Substituting Equation (11.30) into Equation (11.27), we obtain

x = Ax - BKx = Hx, (11.31)
where H is the n X n matrix resulting from the addition of the elements of A and - BK.
Now, returning to the error-squared performance index, we recall from Section
5.7 that the index for a single state variable, Xn, is written as

J [x^t)]2 dt. (11.32)

Section 11.7 Optimal Control Systems 861
A performance index written in terms of two state variables would then be

J = / (jq2 + x22) dt. (11.33)

Jo
Since we wish to define the performance index in terms of an integral of the sum of
the state variables squared, we will use the matrix operation

'/'„ _ [.X * » #3, • - . , X ] X2 — xi

v 2 + *2 + *3 + ••• + V ,
xx h 2 n (11.34)

where \T indicates the transpose of the x matrix.1 Then the specific form of the per-
formance index, in terms of the state vector, is

J = xr\ dt. (11.35)

The general form of the performance index (Equation 11.26) incorporates a term
with u that we have not included at this point, but we will do so later in this section.
Again considering Equation (11.35), we will let the final time of interest be
tf = oo. To obtain the minimum value of/, we postulate the existence of an exact
differential so that

— (x r Px) = - x r x , (11.36)
dt

where P is to be determined. A symmetric P matrix will be used to simplify the alge-

bra without any loss of generality. Then, for a symmetric P matrix, py = pyr Complet-
ing the differentiation indicated on the left-hand side of Equation (11.36), we have

d_
(x'Px) = x'Px + x 7 Px.
dt

Substituting Equation (11.31), we obtain

(x r Px) = (Hx)7"Px + x r P(Hx)

dt
= x r H 7 P x + x r PHx
= x r ( H r P + PH)x, (11.37)

where (Hx) 7 = x 7 H 7 by the definition of the transpose of a product. If we let

H r P + PH = - I , then Equation (11.37) becomes

1(,¾) -x 7 x, (11.38)

*Thc matrix operation xT\ is discussed on the MCS website.

862 Chapter 11 The Design of State Variable Feedback Systems

which is the exact differential we are seeking. Substituting Equation (11.38) into
Equation (11.35), we obtain

J = —(x r Px) dt = - x r P x = xTy (0)Px(0). (11.39)

dt

In the evaluation of the limit at t - oo, we have assumed that the system is stable,
and hence x(oo) = 0, as desired. Therefore, to minimize the performance index / ,
we consider the two equations

(11.40)

and
H ' P + PH = - I . (11.41)

The design steps are then as follows:

1. Determine the matrix P that satisfies Equation (11.41), where H is known.
2. Minimize J by determining the minimum of Equation (11.40) by adjusting one or
more unspecified system parameters.

EXAMPLE 11.11 State variable feedback

Consider the open-loop control system shown in Figure 11.15. The state variables
are identified as X\ and x2. The performance of this system is quite unsatisfactory
because an undamped response results for a step input. The vector differential equa-
tion of this system is

d xi1 0 i X[ 0
+ u(t), (11.42)
dt _ * 2 _ L° oj x2 L'ij
where
0 1
and B =
0 0

We will choose a feedback control system so that

u(t) = ~ki%i - k2x2, (11.43)

FIGURE 11.15
Open-loop control x-,
Ms) •+• tt.v)
system of Example
11.11.
Section 11.7 Optimal Control Systems 863
and therefore the control signal is a linear function of the two state variables. Then
Equation (11.42) becomes

X\ — X2,
X2 — —k\X\ — kjXi, (11.44)
in matrix form, we have

X = Hx
0 1
x. (11.45)
_~~ k\ ~kj

We note that X\ would represent the position of a position control system, and the
transfer function of the system would be G{s) = l/(Ms2), where M = 1 and the fric-
tion is negligible. We will let k\ - 1 and determine a suitable value for k2 so that the
performance index is minimized. Writing Equation (11.41), we have

H r P + PH = - I ,
and in expanded form

0 -1 Pu P12 Pu P12 0 1 0
+ (11.46)
1 -k2 P12 P22 Pl2 P22 -1 "*2 -1

Completing the matrix multiplication and addition, we have

-P12 ~ P12 = ~h
Pu ~ k2pn - P22 = 0,
P12 - k2p22 + P12 ~ k2P22 = - 1 . (11.47)
Solving these simultaneous equations, we obtain
1 1 _ k22 + 2
Pl2 - -v ^22 - "T-. Pll_
Wi *
2 k2 2k2
The integral performance index is then
J = x 7 (0)Px(0), (11.48)

and we will consider the case where each state is initially displaced one unit from
equilibrium so that xT(0) = [1,1]. Therefore Equation (11.48) becomes

7 = [1 1] Pu P12 1
Pn P22_ _1_

Pu + Pu
= [1 1]
Pn + P22_
_P\2 t- P22J
= {Pu + Pn) + (Pu + P22) = P11 + 2/?i2 + p22> (11.49)
864 Chapter 11 The Design of State Variable Feedback Systems

Substituting the values of the elements of P, we have

/<22 + 2 1 k22 + 2k2 + 4

J = — +1 + - = — . (11.50)
2k2 k2 1k2

To minimize as a function of k2, we take the derivative with respect to k2 and set it
equal to zero:

dJ 2k2(2k2 + 2 ) - 2(k22 + 2k2 + 4)

= 0. (11.51)
dk- (2k2f

Therefore, k2 = 4, and k2 = 2 when / is a minimum. Hie minimum value of J is

obtained by substituting k2 = 2 into Equation (11.50). Thus, we obtain

/ - = 3
•'nun •*«

The system matrix H, obtained for the compensated system, is then

0 1
H (11.52)
-1 -2

The characteristic equation of the compensated system is therefore

-1
det[AI - H] = det kl + 2A + 1. (11.53)
1 A+ 2

Because this is a second-order system, we note that the characteristic equation is

of the form s2 + 2(,(ons + a),2 = 0, and therefore the damping ratio of the com-
pensated system is £ = 1.0. This compensated system is considered to be an opti-
mal system in that the compensated system results in a minimum value for the
performance index when kx = 1 is fixed. Of course, we recognize that this system
is optimal only for the specific set of initial conditions that were assumed. The
compensated system is shown in Figure 11.16. A curve of the performance index
as a function of k2 is shown in Figure 11.17. It is clear that this system is not very
sensitive to changes in k2 and will maintain a near-minimum performance index
if the k2 is altered by some percentage. We define the sensitivity of an optimal
system as
, A///
sr-ufc- (11.54)

where k is the design parameter. Then, for this example, we have k = /c2, and con-
sidering k2 = 2.5, for which J = 3.05, we obtain

0.05/3
opt = 0.07. (11.55)
SV 0.5/2
Section 11.7 Optimal Control Systems 865

«(.v) = 0 • n.v)

0 1 2 3 4
h

FIGURE 11.17
FIGURE 11.16 Compensated control system of Performance index versus the
Example 11.11. parameter/c 2.

EXAMPLE 11.12 Determination of an optimal system

Now let us consider again the system of Example 11.11, where both the feedback
gains, k\ and k2, are unspecified. To simplify the algebra without any loss in insight
into the problem, let us set k\ = k2 = k. We can prove that if kx and k2 are unspec-
ified, then kx = k2 when the minimum of the performance index (Equation 11.40) is
obtained. Then, for the system of Example 11.11, Equation (11.45) becomes

x = Hx = (11.56)

To determine the P matrix, we use Equation (11.41), which is

H 7 P + PH = - I . (11.57)
Solving the set of simultaneous equations resulting from Equation (11.57), we find that
1 k + 1 , 1+2*
P\2 = 2k' P22 = -ZTT-, and pu = 2k
2k~
Let us consider the case where the system is initially displaced one unit from equi-
librium so that x r (0) = [1 0], Then the performance index (Equation 11.40)
becomes
/.00

/ = / xrx dt = xT(0)Px(0) = pn. (11.58)

Jo

Thus, the performance index to be minimized is

1 + 2k , 1
./ (11.59)
Ik 2k
The minimum value of J is obtained when k approaches infinity; the result is
•Anin = 1- A plot of 7 versus k, shown in Figure 11.18, illustrates that the perfor-
mance index approaches a minimum asymptotically as k approaches an infinite
866 Chapter 11 The Design of State Variable Feedback Systems

FIGURE 11.18
Performance index
versus the
feedback gain k for
Example 11.12.

value. Now, we recognize that, in providing a very large gain k, we can cause the
feedback signal
u{t) = -k[Xl(t) + x2(t)]

to be very large. However, we are restricted to realizable magnitudes of the control

signal u{i).Therefore, we must introduce a constraint on u{t) so that the gain k is not
made too large. Then, for example, if we establish a constraint on u(t) so that

|K(0I =5 50, (11.60)

we require that the maximum acceptable value of k in this case be

km,, = ^ ^ = 50. (11.61)

*i(0)
Then the minimum value of J is

/ = 1 + = 1.01, (11.62)
2kr
which is sufficiently close to the absolute minimum of J to satisfy our requirements.
Upon examining the performance index (Equation 11.35), we recognize that
the reason the magnitude of the control signal is not accounted for in the original
calculations is that u(t) is not included within the expression for the performance
index. However, in many cases, we are concerned with the expenditure of the con-
trol signal energy. For example, in an electric vehicle control system, [u(t)]2 repre-
sents the expenditure of battery energy and must be restricted to conserve the energy
for long periods of travel. To account for the expenditure of the energy of the control
signal, we will use the performance index

(11.63)
Section 11.7 Optimal Control Systems 867

where A is a scalar weighting factor and I = identity matrix. The weighting factor A
will be chosen so that the relative importance of the state variable performance is
contrasted with the importance of the expenditure of the system energy resource
that is represented by u 7 u. As in the previous paragraphs, we will represent the state
variable feedback by the matrix equation
u = -Kx, (11.64)
and the system with this state variable feedback as
x = Ax + Bu = Hx. (11.65)
Now, substituting Equation (11.64) into Equation (11.63), we have

J = (xTl\ + A(Kx) r (Kx)) dt

Jo
x y (I + AKyK)x dt = / x'Qxdt, (11.66)
Jo

where Q = I + AKTK is an n X n matrix. Following the development of Equations

(11.35) through (11.39), we postulate the existence of an exact differential so that

iyp* — *J
x 7 Qx. (11.67)

Then, in this case, we require that

H r P + PH = - Q , (11.68)
and thus we have, as before, (Equation 11.39):

J = x 7 (0)Px(0). (11.69)

Now, the design steps are exactly as for Equations (11.40) and (11.41), with the
exception that the left side of Equation (11.68) equals Q instead of - I . Of course,
if A = 0, Equation (11.68) reduces to Equation (11.41). Now, let us consider again
Example 11.11 when A is other than zero and account for the expenditure of control
signal energy. •

EXAMPLE 11.13 Optimal system with control energy considerations

Let us consider again the system of Example 11.11, which is shown in Figure 11.15.
For this system, we use a state variable feedback so that

*i
u = -Kx *] (11.70)
x2
Therefore, the matrix

7V =-
1 + kk2 \k2
Q = I + AK'K (11.71)
kk2 1 + kk2
868 Chapter 11 The Design of State Variable Feedback Systems

5
I
J 4 \
3
A= 1
FIGURE 11.19
Performance index A= 0
versus the
feedback gain k for 0.5 1.5 2.5
Example 11.13.

As in Example 11.12, we will let x r (0) = [1, 0] so that J = pn. We evaluate pn

from Equation (11.68), namely,

H 7 P + PH = - Q . (11.72)
Thus, we find that

J = Pn = ( 1 + A*2)[ 1 + 2 £ ) - Xk
^ (11.73)

and we note that the right-hand side of Equation (11.73) reduces to Equation
(11.59) when A = 0. Now, the minimum o f / i s found by taking the derivative of/,
which is
dJ_ 1
0. (11.74)
dk k2

Therefore, the minimum of the performance index occurs when k = kt

where kmin is the solution of Equation (11.74).
VT
Let us complete this example for the case where the control energy and the
state variables squared are equally important, so that A = 1. Then Equation (11.74)
is satisfied when k2 - 1 = 0, and we find that kmio = 1.0. The value of the perfor-
mance index J obtained with kmin is greater than that of the previous example
because the expenditure of energy is equally weighted as a cost. The plot of J versus
k for this case is shown in Figure 11.19. The plot of J versus k for Example 11.12 is
also shown for comparison in Figure 11.19. •
It has become clear from the examples in this chapter that the actual minimum
obtained depends on the initial conditions, the definition of the performance index,
and the value of the scalar factor A.
The design of several parameters can be accomplished in a manner similar to
that illustrated in the examples. Also, the design procedure can be carried out for
higher-order systems. However, we must then consider the use of a digital computer
to determine the solution of Equation (11.41) in order to obtain the P matrix. The
 Section 11.8 Internal Model Design 869
computer may also provide a suitable approach for evaluating the minimum value of
J for one or more parameters. However, the solution of Equation (11.68) may be dif-
ficult, especially when the system order is quite high (« > 3). An alternative method
suitable for computer calculation is stated without proof in the following paragraph.
Consider the uncompensated single-input, single-output system with

x = Ax 4- Bu
and feedback
u = - K x = —[k] k2---krl]x.

The performance index is

/ = / (x 7 Qx + Ru2) dt,

where R is the scalar weighting factor. This index is minimized when

K = R~lBTV.
The n X n matrix P is determined from the solution of the equation

A 7 P + PA - P B i T B ' P + Q = 0. (11 75)

Equation (11.75) can be easily programmed and solved using numerical methods.
Equation (11.75) is often called the Riccati equation. This optimal control is called
the linear quadratic regulator (LQR) [12,19].

11.8 INTERNAL MODEL DESIGN

In this section, we consider the problem of designing a compensator that provides

asymptotic tracking of a reference input with zero steady-state error. The refer-
ence inputs considered can include steps, ramps, and other persistent signals, such
as sinusoids. For a step input, we know that zero steady-state tracking errors can
be achieved with a type-one system. This idea is formalized here by introducing an
internal model of the reference input in the compensator [5,18].
Let us consider a state variable model of the plant given by

x = Ax + BH, y = Cx, (11.76)

where x is the state vector, u is the input, and y is the output. We will consider a ref-
erence input to be generated by a linear system of the form

xr = Arxn r = d,x r , (11.77)

with unknown initial conditions. An equivalent model of the reference input r{t) is

r("> = a „ V ! _ 1 ) + a„- 2 r ( "- 2 > + • • • + a{r + a{)r, (11.78)

(,,)
where r is the nth derivative of r{t).