OXFORD APPLIED MATHEMATICS

AND COMPUTING SCIENCE SERIES 2

Introduction
~to Mathematical
-Control Theory
- STEPHENBARNETT
| a De es
~~,

Oxford Applied Mathematics

and Computing Science Series LE
STEPHEN BARNETT
University of Bradford

Introduction to
mathematical
control theory

CLARENDON PRESS - OXFORD

1975
Oxford University Press, Ely House, London W. 1
GLASGOW NEW YORK TORONTO MELBOURNE WELLINGTON

CAPE TOWN IBADAN NAIROBI DAR ES SALAAM LUSAKA ADDIS ABABA

DELHI BOMBAY CALCUTTA MADRAS KARACHI LAHORE DACCA

KUALA LUMPUR SINGAPORE HONG KONG TOKYO

ISBN 0 19 859619 7 9

©OxfordUniversityPress1975
All rights reserved. No part of this publication may be reproduced,
Stored in a retrieval system, or transmitted, in anyform or by any means,
electronic, mechanical, photocopying, recording or otherwise, without
the prior permission af Oxford University Press

Text setiin 10/12 pt. IBM Press Roman, printed by

photolithography and bound in Great Britain at
The Pitman Press, Bath
 Preface

Control theory has developed rapidly over the past two decades, and is
now established as an important area of contemporary applied mathe-
matics. Some justification for this latter claim is provided in Chapter 1,
which begins with a general discussion on the scope and relevance of
control theory. The aim of this book is to present a concise, readable
account of some basic mathematical aspects of control. The origins of
the book lie in courses of lectures which I have given for a number of
years to final honours mathematics students and to postgraduate control
engineers at the University of Bradford. My approach concentrates on
so-called state space methods, and emphasises points of mathematical
interest, but I have avoided undue abstraction and the contents should
be accessible to anyone with a basic background in calculus and matrix
theory. To help achieve this objective a summary is given in Chapter 2
_of some appropriate topics in linear algebra. The book should therefore
also be useful to qualified engineers seeking an introduction to con-
temporary control theory literature, some of which tends to be expressed
with rather forbidding mathematical formulation.
I have provided problems to be attempted at the end of each section.
Most of these have been class-tested, and so should be within the reader’s
grasp; all answers are given. In an introductory work it is essential to
supply references for further study, and except in a few cases I have
quoted text books rather than journal papers which are usually more
suitable for specialists. It is of interest that most of the references listed
have appeared within the last ten years, giving some measure of the
publishing ‘explosion’ in the control field in recent times.
In any compact treatment many points of detail, refinements and
extensions are inevitably omitted; in particular I regret that it has not
been possible to include any material on statistical concepts which arise
in control. Despite the many texts now available on control theory it is
my hope that this book fills a niche in the literature by its combination
of conciseness and range of topics. I hope also that it will help to pro-
mote interest in this exciting field, thus providing at least a small counter-
balance to the deadweight of British classical applied mathematics.
I havedonemy bestto eliminateerrors,althoughin an entertaining
vi Preface

article P. J.Davis (Amr. Math. Mon., 1972, 79, 252) suggests that such
attempts are doomed to failure! Thanks however are due to P. A. Cook,
A. T. Fuller, J. B. Helliwell, G. T. Joyce, A. G. J. MacFarlane, D. D. Siljak,
C. Storey, and H. K. Wimmer who have made useful comments on various
parts of the manuscript. I should also like to thank Dr. Hedley Martin for
encouraging me to write this book and for his helpful assistance through-
out the project, and the Clarendon Press for their efficient co-operation.
Finally, as on many previous occasions, I am grateful to Mrs. Margaret
Balmforth for her skilful preparation of the typescript.

Bradford S.B.
July 1974
Contents

1. INTRODUCTION TO CONTROL THEORY

1.1. General remarks and examples
1.2. Classical control and transform theory
1.2.1. Continuous-time systems: Laplace transform
1.2.2. Discrete-time systems: z-transform

2. PRELIMINARY MATRIX THEORY

2.1. Definitions
2.2. Linear dependence and rank
2.3. Polynomials
2.4. Characteristic roots
2.5. Polynomial matrices
2.6. Jordan canonical form
2.7. Functions of a matrix
2.8. Quadratic and Hermitian forms

3. MATRIX SOLUTION OF LINEAR SYSTEMS

3.1. Solution of uncontrolled system: spectral form
3,2. Solution of uncontrolled system: exponential matrix
3.3. Solution of uncontrolled system: repeated roots
3.4. Solution of controlled system
3.5. Time varying systems
3.6. Discrete-time systems
3.7. Relationships between state space and classical forms

4. LINEAR CONTROL SYSTEMS

4.1. Controllability
4.2. Observability
4.3.+ Controllability and polynomials
4.4. Linear feedback
4.5. State observers
4.6. Realization of constant systems
4.7. Discrete-time systems
4.8.7 Realization of time varying systems

See OLABIEITY
5.1. Definitions
5.2. Algebraic criteria for linear systems
5.2.1. Continuous-time

+ May be omitted on a first reading.

 Contents

5.2.2. Discrete-time 158

5.2.3. Time varying 161
Nyquist criterion for linear systems

Annn
DPW Liapunov theory
Application
Construction
5.6.1.
of Liapunov theory to linear systems
of Liapunov functions
Variable gradient method
5.6.2 Zubov’s method
5.7. Stability and control
5.7.1. Input—output stability
5.7.2. Linear feedback
5.7.3.+ Nonlinear feedback

6. OPTIMAL CONTROL
6.1. Performance indices
6.1.1. Measures of performance
6.1.2. Evaluation of quadratic indices
6.2. Calculus of variations
6.3. Pontryagin’s principle
6.4. Linear regulator
6.5. Dynamic programming

REFERENCES

ANSWERS TO EXERCISES

INDEX 261

+ May be omitted on a first reading.

1 Introduction to control theory

1.1. General remarks and examples

In most branches of applied mathematics, the aim is to analyse a given
situation. To take a very simple example, if a mass is suspended by a
string from a fixed point then the assumptions might be that air resist-
ance, the mass of the string and the dimensions of the body could all be
neglected, and that gravitational attraction is constant. A familiar mathe-
matical problem would then be to determine the nature of small motions
about the equilibrium position. Thus in order to be able to obtain a mathe-
matical description, or model, of the real-life situation it is necessary to
make certain simplifying assumptions so that established laws from science,
engineering, economic theory, etc., can be used. Mathematical methods
can then be applied to investigate the properties of the model, and the
conclusions reached will reflect reality only insofar as the accuracy of the
model permits. Of course the more realistic the model, the more difficult
in general will it be to solve the resultant mathematical equations. The
‘classical’ areas of applied mathematics such as mechanics (of particles,
solids, fluids), electromagnetic theory, thermodynamics, etc., have been
extensively developed during the last two hundred years and generally
reflect this emphasis on analysis. However, many problems of great im-
portance in the contemporary world require a quite different approach,
the aim being to compel or control a system to behave in some desired
fashion. Here ‘system’ is used to mean a collection of objects which are
related by interactions and produce various outputs in response to diff-
erent inputs. This statement is deliberately very vague so as to embrace a
very wide range of situations. These can include for example industrial
complexes such as chemical plants or steelworks; electro-mechanical
machines such as motor cars, aircraft, or spaceships; biological systems
such as the human body; and economic structures of countries or regions.
Control problems associated with these systems might be production of
some chemical or steel product as efficiently as possible; automatic land-
ing of aircraft, soft landings on the moon, or rendezvous with an arti-
ficial satellite; regulation of body functions such as heartbeat, blood
pressure, temperature, and the ever-present problem of control of econo-
mic inflation. The complexity of many systems in the present-day world
2 Introduction to control theory

is such that it is often desirable for control to be carried out automatically,

without direct human intervention. To take a simple example, the room
thermostat in a domestic central heating system turns the boiler on and
off so as to maintain room temperature at a predetermined level. Nature
provides many other examples of remarkable self-regulation, such as the
way in which body temperature is kept constant despite large variations
in external conditions, or the manner in which it is possible to fix the
eyes on some object whilst engaged in strenuous physical activity.
The main features of a control system can be represented as in Fig. 1.1.

Disturbances

Control
variables
(inputs) Controlled
system Outputs yi,
Controlling:
[5175
Deen)
oy state
variables
Sy i=) Te
devicecnet :
oeFes
|VP) Output
monitoring
Feedback

FIG/1.1

The state variables x; describe the condition or state of the system,

and provide the information which (together with a knowledge of the
equations describing the system) enables us to calculate the future
behaviour from a knowledge of the inputs; the n-dimensional space
containing the x;s is the state space —hence the widespread use of terms
like ‘state space approach’. In practice it is often not possible to determine
the values of the state variables directly, perhaps for reasons of expense
or inaccessibility. Instead only a set of output variables y;, which depend
in some way on the x;, is measured and almost invariably r<n. For ex-
ample, the state of the economy of a country is described by a great
many variables, but it is only practicable to measure a few of these, such
as the volume of production, the number in employment, the value of
gold reserves, and so on. In general the object is to make a system perform
in some required way by suitably manipulating the control variables u;,
this being done by some controlling device, or ‘controller’. Systems are
often subject to external disturbances of an unpredictable nature, for
example wind gusts during aircraft landing, or variations in the cost of
raw materials for a manufacturing process. Such: factors require a statisti-
tical treatment and will not be dealt with in this book. We shall assume
that all our system models have the property that, given an initial state
 Introduction to control theory )

and any input then the resulting state and output at some specified later
time are uniquely determined. Such models are often termed dynamical
systems, although of course they need have nothing to do with Newtonian
mechanics, and can be defined in rigorous formal terms (e.g. Desoer 1970).
If the controller operates according to some pre-set pattern without
taking account of the output or state, the system is called open loop,
because the ‘loop’ in Fig. 1.1 is not completed. If however there is feed-
back of information concerning the outputs to the controller, which then
appropriately modifies its course of action, the system is closed loop.
Simple illustrations of open and closed loop systems are provided by
traffic lights which change at fixed intervals of time, and those which are
controlled by some device which measures traffic flow and reacts accord-
ingly.
Another familiar example is provided by the closed loop feedback
system which controls the water level in the cistern of the domestic toilet.
After flushing is completed the water outlet valve closes, and water flows
into the cistern. A ball float measures the water level, and closes the
supply valve when the water reaches a pre-set level. Thus a knowledge of
the actual water level is ‘fed back’ via the float to obtain a measured level
which is compared with the desired level; when this difference, or ‘error’
e, exceeds a predetermined value the supply valve is open, and when e is
less than this value the valve is closed. This is represented diagrammatically
in Fig. 1.2.

FIG. 1.2 @= actual water level; d = desired water level; m = measured water level; e = d—m.

Both Figs 1.1 and 1.2 are examples of block diagrams, which are widely
used in control engineering as a convenient method of representing the
relationships within a system in a pictorial form. The convention used is
that flows are added with their indicated signs where the junction is shown
as a circle, otherwise signals proceed along branching paths without altera-
tion. The rectangles represent operators of some sort.
4 Introduction to control theory

In the plumbing system it is clear that the control of water level will
be satisfactory even if there are wide variations in the pressure of the in-
flowing water. Similarly, in the central heating system mentioned earlier,
the performance may well be acceptable even if the heat output of the
boiler varies or if outside weather conditions change suddenly. These
examples illustrate a very important property of feedback systems: their
ability to operate satisfactorily even when parameters in various parts of
a system may vary considerably. This insensitivity will be discussed in a
little more detail in Section 1.2. Feedback is thus a crucial concept in
control engineering, and indeed a water level control of the type described
above was known to the Greeks in the first century A.D. Although feed-
back has a long history (see Mayr 1970), it is only in the last forty years
or so that extensive efforts have been given to the study of feedback
theory. The so-called ‘classical’ control engineering methods rely heavily
on transform methods for solving linear differential or difference equa-
tions, and received great impetus from military applications during the
Second World War. Before going into further details in the next section,
we first discuss some aspects of ‘modern’ control theory, which has been
developed mainly over the last twenty years using the state variable
approach, and on which this book concentrates most attention. The
examples presented below will be referred to again in subsequent chap-
ters, and all illustrate the important feature of control theory, mentioned
earlier, namely to synthesize a control strategy which satisfies require-
ments.
Example 1.1. Suppose a car is to be driven along a straight, level road,
and let its distance from an initial point 0 be s(t) at time ¢. For simplicity
assume that the car is controlled only by the throttle, producing an acce-
lerating force of u,(t) per unit mass, and by the brake which produces
a retarding force of u(t) per unit mass. Suppose that the only factors of
interest are the car’s position x, (f) = s(t) and velocity x, (t) = s(t), where
(*) =d/dt. Ignoring other forces such as road friction, wind resistance,
etc., the equations which describe the state of the car at time f are

X1 =X2

Xq =U, — U2

or in matrix notation

%=Ax + Bu (1.1)
 Introduction to control theory 5

where

X41
| head uy 01
Placa (allPg eeden P= 00
xX Uy 00 Pt
In practice there will be limits on the values of u, and u,, for obvious
reasons, and it will also be necessary to impose restrictions on the magni-
tudes of velocity and acceleration to ensure passenger comfort, safety,
tc;
It may then be required to start from rest at 0 and reach some fixed
point in the least possible time, or perhaps with minimum consumption
of fuel. The mathematical problems are firstly to determine whether such
objectives are achievable with the selected control variables, and if so, to
find appropriate expressions for uv, and uw,as functions of time and/or
x, and x2.
The complexity of the model could be increased so as to take into
account factors such as engine speed and temperature, vehicle interior
temperature, and so on. A further aspect of realism could be added by
imagining the car to be travelling in one lane of a motorway, in which
case the objective might be to maintain the distance from the vehicle in
front within certain limits.

Example 1.2. The interior temperature of an electrically heated oven

is to be controlled by varying the heat input u to the jacket, as shown in
Fig, Tio

ot

it

Heating
coil

FIG.-le3
6 Introduction to control theory

Let the heat capacities of the oven interior and of the jacket be c,
and c, respectively; let the interior and exterior jacket surface areas be
a, and a; and let the radiation coefficients of the interior and exterior
jacket surfaces ber; andr. Assume that there is uniform and instan-
taneous distribution of temperature throughout, and that rate of loss
of heat is proportional to area and the excess of temperature over that
of the surroundings. Ignoring other effects, if the external temperature
is J), the jacket temperature is 7, and the oven interior temperature is
T>, then we have:

Forthejacket:
eyT;=—ayro(T,
—To)—a17,
(TyTa)ee (1.2)
Fortheoven
interior:
caT2=440;
aged): (1.3)
Let the state variables be the excesses of temperature over the exterior,
ie.x, =T, —To,X2 =T2 —To. It is easy to verify that (1.2) and (1.3)
can be written in the form (1.1) with

—@or2 tayry [ey ayri/ey Ie,

A— >
Bic e (1.4)
a111/C2 —ayr;|/c2 0
This problem will be studied again in Examples 2.5 and 3.7, and Exercise
3.16. Two further aspects which will be discussed in Chapter 4 (Examples
4.3, 4.5) are firstly whether it is possible to maintain the temperature of
the oven interior at any desired level merely by altering uw;and secondly ,
to determine whether the value of 7, can be determined even if it is not
possible to measure it directly. If the desired objective is attainable then
there may well be many different suitable control schemes, and considera-
tions of economy, practicability of application, and so on will then
determine how control is actually applied.
The preceding two examples involve linear differential equations. Gen-
erally, a system is linear if, when the response to some input u(t) is y(t) =
L(u) then the response to c;u, + C2, is cyL(u,) + cL (uz). Here L is
some operator —differential, integral, probabilistic, etc., the c; are con-
stants; and the wu;in general will be vectors. The system is called time
invariant (or constant) if the response to u(t—7) isy(t—7), i.e. L[u(t—-7)] =
y(t—-7) for any fixed r. Because linear mathematics is very well developed,
linear time invariant sets of state equations are the easiest to manage ana-
Introduction to control theory 7

lytically and numerically, and the first model of a situation is often con-
structed to be linear for this reason. Furthermore, the technique of
linearization of nonlinear systems is an important one, and relies on the
fact that if the perturbation z(t) from some desired state x(t) is small then
a set of linear equations in z can be formed by neglecting all but the first
terms in a Taylor series expansion in z. Chapters 3 and 4 are devoted to
the study of basic properties of linear systems. Of course in many cases
linear descriptions may be inapplicable, as the following example illustrates.

Example 1.3. Consider a simplified space rendezvous problem, in

which a spacecraft S; wishes to join up with a second non-manoeuvering
mothership S,. Suppose that the only control variable is the thrust direc-
tion u, the thrust itself being assumed to provide a constant acceleration
c. Take S, as the origin of the coordinate system, as shown in Fig. 1.4.

FIG. 1.4

The state variables are the velocity x, of S, relative to S,, the distance
Xz between S, and S, and the angle x3. Ignoring any external gravitational
forces, the state equations are

x, =ccos(u+x3)

X2 =—X; cos x3

X3 =(x,/x2) sinx3 —(c/x,) sin (ut+x3).

The problem might then be to find a function wu,if possible, which takes
S; to S, in minimum time T or with minimum consumption of fuel. A
numerical solution to the former problem, with x; ae 0, can be found
in Bryson and Ho (1969, p. 143).
8 Introduction to control theory

Example 1.4. In contrast to the simple pendulum referred to at the

beginning of this section, consider the ‘inverted pendulum’ represented
in Fig. 1.5.

FIG.1.5

A uniform rod AB, mass m, length 22 is smoothly hinged to the centre

of a platform of mass Mwhich runs on smooth straight rails. When the
rod is vertical (6 = 0) it is in equilibrium, but clearly this is an unstable
condition since any small perturbation will cause the rod to fall. The aim
is to apply a control force u to the platform so as to keep the rod upright.
Standard theory of mechanics gives the equations of motion as

me =(Y
sin 0 —X cos 0)
d2
X=m sn (z + sin 6)
a (1.5)
Y—mg=mon (2 cos8)

u X=M
melea2»
where X and Y are the horizontal and vertical components of the force
exerted by the hinge on the rod, and z is the displacement of the centre
of mass of the platform from some fixed point. Again, an initial problem
is to determine whether the desired objective can be achieved.
Introduction to control theory 9

The problem described above is sometimes referred to as that of ‘broom

balancing’, by analogy with the idea of keeping a broom balanced verti-
cally on the end of one finger by moving the finger in a horizontal plane.
Two interesting applications of the inverted pendulum model are to a
one-dimensional version of a rocket on take-off, the vehicle being balanc-
ed by the engine thrust; and to biped locomotion systems relating to
prosthetic limbs for the physically disabled. The theory of stability of
systems is a fundamental one and will be developed in Chapter 5, and
stability aspects of the inverted pendulum are studied in Exercises 5.42
and 5.49.
Applications of control theory to economic problems are becoming
of increasing importance, and we present a very simple example.

Example 1.5. Suppose that the sales S(¢) of a product are affected by
the amount of advertising A(t) in such a way that the rate of change of
sales decreases by an amount proportional to sales, but increases by an
amount proportional to the advertising applied to the share of the market
not already purchasing the product. If the total extent of the market is
M, the state equation is therefore
S =-aS + bA(#)(1 -S/M)

subject to S(O) = So, a and b being positive constants. In practice the

amount of advertising will be limited, i.e. 0 <A(t) <K,where K isa
constant, and the aim would be to find the advertising schedule (i.e. the
function A(t)) which maximizes the sales over some given period of time.
Further details of this and other applications in economics can be found
in Intriligator (1971).
In both this example and in Example 1.3 the objective was to find a
control strategy which was the best possible, according to some criterion.
Such problems of ‘optimal’ control are dealt with in Chapter 6.

Exercise 1.1.
In Example 1.1, list as many factors as you can think of which are present
in reality and can affect the motion of a car being driven along a road.

Exercise 1.2.
Consider the system composed of two masses lying on a smooth hori-
zontal table and connected by two springs to a fixed support, as shown
below.
10 Introduction to control theory

The springs are assumed to obey Hooke’s law (force = k x extension).

A force u(t) is applied to the right hand mass. Obtain the system equa-
tions in the form (1.1) taking the displacements x, , x2 of the masses from
equilibrium and velocities X; =x3,%2 =Xq as state variables.
This problem will be returned to in Example 2.6 and Exercises 2.19,
4.4 and 4.14.

YASpring Spring
constant constant
2

Exercise 1.3.
In the inverted pendulum problem in Example 1.4 take as state variables
x, =0,x, =0,x3 =Z,x4 =Z, and set u = 0. By eliminating X and Y in
eqns (1.5), obtain the system equations in state space form X = f(x, x).

Exercise 1.4.
Consider a controlled environment consisting of rabbits and foxes, the
numbers of each at time ¢ being x, (t) and x2 (ft) respectively. Suppose
that without the presence of foxes the number of rabbits would grow
exponentially, but that the rate of growth of rabbit population is reduced
by an amount proportional to the number of foxes. Furthermore suppose
that, without rabbits to eat, the fox population would decrease exponen-
tially, but that the rate of growth in the number of foxes is increased by
an amount proportional to the number of rabbits present. Show that
under these assumptions the system equations can be written

Xy =X, —A2X%2,X2q
= 3X1 —dgXx2 (1.6)
where the a; are positive constants.
This system will be studied again in Exercises 1.7, 4.15, and 4.33, and
Example 5.2.
Introduction to control theory Nia

1.2. Classical control and transform theory

Classical control theory deals with a linear time invariant systems having
scalar input u(t) and scalar output z(t). The analysis problem is to study
the behaviour of a given system in specified circumstances. The aim of
design is to ensure that the performance of the system satisfies required
specifications. If z(¢) is required to be as close as possible (in some sense)
to a given reference signal r(t) the control system is called a servomecha-
nism, and if r is constant, a regulator. For example, the inverted pendulum
discussed in Example 1.4 is a regulator, since it is required to keep the
angle to the vertical 6 as near to zero as possible. Similarly, a central-
heating system is designed to keep room temperature close to a prede-
termined value.

1.2.1, Continuous-time systems: Laplace transform

If u and z are continuous functions of time then the model of classical
linear control theory is the n-th order differential equation

2) +k,z@-)) aEse«|«+kp—yz) +kyz=Bou™)+p, 1)

+2**+Bmu (1.7)
where superscript (7) denotes jth derivative with respect to time ¢ and the
k; and 8; are constants. It can be assumed that m <n. In fact it is easy to
convert (1.7) into the matrix-vector form (1.1) and this is dealt with in
Section 3.7. However, the standard techniques of Laplace transforms can
be applied to (1.7) using the result
L{2(t)}= sz —s/—12(0)
—51-22D0) —+- -= sz 20) - 29- D0)
(1.8)
where z(s) is the Laplace transform of z(t) defined by

Z(s)=L{
z(t)}=
its2(t)e~*
dt. (1.9)
It is assumed that the reader has a working knowledge of the use of Lap-
lace transforms for solving ordinary differential equations.
Assuming that all the 2(0) and u(0) are zero, application of (1.8)
to (1.7) gives
k(s) 2(s) = B(s)u(s) (1.10)
12 Introduction to control theory

where
K(s)=s" +k, s® 1 400 e4hk, 54K, (1.11)

B(s) = Bo 8 + Bys™—1+90. + +B. 8 +Bp outed

2)
Eqn (1.10) can be written
Z(s) =g(s) u(s) (1.13)
where g(s) = B(s)/k(s) (1.14)
is the transfer function, which is the ratio of the Laplace transform of the
output to that of the input. The zeros of the characteristic polynomial
k(s) are called the poles of the transfer function, and the zeros of B(s)
are the zeros of g(s). In block diagram form eqn (1.13) can be represented
as in Fig. 1.7.

Input u(s) Output z(s)

PIGS 1.7

If u(t) (and hence u(s)) is known then solution for z(t) requires expan-
sion of the right hand side of eqn (1.13) into partial fractions. If the
poles ofg(s) are denoted by A,, A2,..., Ay (these of course may be real
or complex numbers) then this expansion will involve terms of the form
c;/(s —Ay),the c; being constants. These correspond to terms cjexp(Ajt)
in z(t). Solution of linear systems is dealt with systematically in Chapter
3 using matrix methods, which are generally preferable unless n is small.
It is worth noting that if the system is unforced (i.e. u= 0) and is subject
to given initial conditions at ¢ = 0, then the solution for z(f) (i.e. the com-
plementary function for (1.7)) tends to a steady state ast > © only if
all the A; have non-positive real parts. Constant terms in the steady state
solution correspond to zero As, and sinusoidal terms to purely imaginary
As. The question of stability of linear systems is gone into in detail in
Chapter 5.
Classical control theory is based on the study of transfer functions,
and a number of powerful methods have been evolved. However, with
one exception (see Section 5.3) these will be dealt with only very briefly
in this book. This is not to deny the enormous practical utility of the
techniques, but reflects the fact that for the mathematician, at least,
Introduction to control theory 13

greater interest lies in the more recent approaches to the study of control
systems, especially those involving vector inputs (multivariable control
systems). Moreover, transform methods are applicable only to constant
linear systems and are of no help with time varying or nonlinear systems.
Consider a closed loop system represented in the block diagram below
Fig. 1.8, where g(s) is the open loop transfer function and A(s) is the
feedback transfer function.

FIG. 1.8

Since e =u —h(s)z and z = g(s)e it follows by eliminating e that

Z(S) =Sc (s)u(s),

where the closed loop transfer function is

4 se)
&(s) T+esi)’ (1.15)

Suppose that the input is a unit step function, i.e.

u(t)=1,t>0
(1.16)
=0,t<0

and that

IT Ga) IIG -2)

&c(8)=#1 ost (1.17)

IT @~ TT 6-4)
i=1 Jeu
14 Introductionto controltheory
where all the poles 41, ..., My of the closed loop system transfer function
are assumed distinct. Then it is easy to show using u(s) = 1/s that the
system response to the step input is

2(t)=1+ > Ajexp(ujt), (1.18)

i=1
where
Ajon [Stee
(s —M)&c(s) SH (1.19)

so that the response is entirely determined by the poles and zeros of

&c (5).
If the system input is harmonic, say u = ugcoswrt, then if all the u; have
negative real parts this leads to a steady state response which is also har-
monic with the same frequency w. After elementary manipulations this
steady state response can be expressed as

2(t)=Ug|e (iw) | cos[wt+arg{ec(iw)}] (1.20)

[i =V(— 1)] showing that the amplitude is |g¢ (iw) | times that of the
input, and that the phase is increased by arg {gc (iw). The function g, (iw)
is called the frequency transfer function, and because of the multiplicative
effect on the amplitude of the output, | gy (iw) | is called the gain. Infor-
mation about the behaviour of a system is thus provided by a knowledge
of the modulus and argument of the frequency transfer function. For
example, a hi-fi amplifier can be regarded as a control system whose pur-
pose is to magnify the input with as little distortion as possible over
frequencies ranging from 50 Hz to 15 kHz. A large range of values of w
such as this is common in applications, so it is convenient to draw graphs
of logio | Zo (iw) | and arg {Sc (iw)} against log;)@ when studying system
performance. Such diagrams are called Bode plots (after their originator),
and methods for drawing them are straightforward, the actual mathemati-
cal details not being of great interest. It is possible in some cases to apply
the procedure in reverse, so that if the gain is measured experimentally a
corresponding transfer function can be deduced. Because of the way w is
used as a variable the method is often referred to as one in the ‘frequency
domain’. A second such technique, due to Nyquist, will be dealt with in
some detail in Chapter S.
Introduction to control theory 15

We have seen that system response depends on the poles and zeros of
g-(s). Suppose that the open loop transfer function g(s) is multiplied by
a gain parameter K so that (1.15) becomes
8c (8) =Kg(s)/[1 + Kg(s)r(s)]. (1.21)
The root locus method has been devised for investigating how the poles
of g_ (s) in (1.21) vary with K. These poles are the roots of
1 + Kg(s)h(s) = 0 (1.22)
and sets of rules have been drawn up for determining the loci of these
roots graphically as K varies from 0 to -. Again, the mathematical detail
is rather tedious.
There are very many textbooks available on control engineering and
the reader should consult these for comprehensive accounts of the above
and other classical methods. In particular the books by Elgerd (1967),
Eveleigh (1972) and Takahashi, Rabins, and Auslander (1970) can be
recommended for treatments which combine the classical and modern
approaches, and the account by Truxal (1972) is clear and interesting.
It is instructive to return in more detail to the question of sensitivity
of feedback systems, briefly mentioned in Section 1.1. For a transfer
function ¢(s) the sensitivity with respect to small variations in some
parameter 0 is defined by

deft _ dt.
Se) ~a6/6 a0 Ps (1.23)
Consider the systems represented in Fig. 1.9, and
16 Introduction to control theory

suppose for simplicity that g,,g2, and h are constants. For the open loop
system in Fig. 1.9(a) the transfer function is just t; =18 so Sg (t,) = 1.
Thus a small percentage change in g, causes the same percentage change
in t,. For the closed loop system in Fig. 1.9(b) eqn (1.15) gives

ty =8182/(1 + 8182h), (1.24)

and differentiation with respect to g and use of (1.23) leads to )

Sg, (t2) = 1/1 +8182h). (1.25)

Clearly in this case for a given gz, the values of g; and h can be chosen so
as to make the sensitivity to small changes in g2 as small as we please. Such
sensitivity considerations carry over in a fairly general way to more com-
plicated systems, and thus provide a very important practical justification
for the use of feedback. In a similar way it can be demonstrated that feed-
back can reduce the effect of external disturbances on a system output.
As is illustrated by (1.25), the denominator of the transfer function plays
an important role in these discussions. More generally, the denominator
1 + g(s)h(s) in the transfer function (1.15) is termed the return difference
for the following reason. Suppose that in the corresponding block diagram
(Fig. 1.8) uw = 0, but that an input e = 1 is inserted. The returning signal
will be —g(s)h(s), so the difference between them is just 1 + gh.
We shall encounter the ideas of transfer functions in our subsequent
work on linear systems, especially the extension (for multi-input multi-
output systems) to matrices whose elements are transfer functions.

1.2.2. Discrete-time systems: z-transform

We now turn to situations where the variables are measured (or ‘sampled’)
only at discrete intervals of time instead of continuously, producing what
are referred to as sampled-data, or discrete-time systems. Such circum-
stances are common in real life: for example, the interest on savings
accounts is calculated daily or monthly; the temperature of a hospital
patient is recorded at perhaps hourly intervals; a driver of a car glances
at the instruments, rear mirror, etc., only intermittently —the reader can
easily add many instances of his own (see Cadzow 1973, ch. 2).
We shall suppose that our variables are defined at fixed intervals of
time 0, T, 27, 37, . . . where T is a constant, and use X(k), u(k) to denote
the values of the output X (KT), and the input u(kT) respectively (k = 0,
Introduction to control theory 17

1,2,3,...). For linear systems the differential eqn (1.7) is replaced by

the difference equation

X(Kt+n)t+kyX(kKt+n-1) +++++kyiyX(K +1) + kyX(k)

= Bou(k +m) + Byu(kK +m—1)++

++ +Bmu(k). (1.26)

The technique corresponding to Laplace transforms is to define the

z-transform of a scalar (or indeed a vector) function X(k), k = 0, 1, 2,
5) ee

X@)
=2{xo}
=>,xG02* (1.27)
k=0
=x)+707D. . (1.28)
The reader may not have encountered z-transforms previously so it will
be useful to go into a little detail here.

Example 1.6. Obtain the z-transforms of the following functions,

defined fork =0,1,2,...: (a) X(k) = 1; (6) X(k)= a* , where a isa
constant.
(a) From the definition (1.27)
Z{ip=1+1+1 +--+:
Pie
=(1 -1y , by the binomial theorem

=z/(z-1).

(b) Similarly
2
dt a,a tat.
Blab =lt+—+steee

a4
=(lire)
= z/(z —a). (1.29)
18 Introduction to control theory

The result corresponding to L£{dz(1)/ah obtained by taking = 1 in (1.8) is

Z{X(k+ 1)}= > X(k+ 1)z-*,by(1.27)

k=0
=X(1)
+X@)+XG), eee
2 oa
=[1 2) 20h
z ze Zz
=z [X(z) —X(0)] by (1.27) (1.30)
A repetition of this argument produces

Z{X(k
+) }=z!X(z)-z/X(0)—-z!~
1y(1)-z/-?x(2)- 20—z*XG=2)-
zX(j-1) (131)
which is the analogue of the result for the Laplace transform of the jth
derivative of z(t) given in (1.8). Application of (1.31) to the difference
equation (1.26) then transforms this into an algebraic equation for X(z).
The resulting expression is expanded into partial fractions as for Laplace
transforms.

Example 1.7. Solve the difference equation

X(k + 2) + SX(kK-+1) + 6X(k) =0
subject to X(0) = Xo, X11) =X.
Take z-transforms of both sides of the equation and use (1.30) and
(1.31) with j = 2 to obtain
2?X —27X9—2X,+ 52(X —Xo)+ 6X =0
Thus
X(z) =2(zXo+ Xy+5X0)
z*+5z +6

=2ec +X1) , GX 5)
zi £42
Hence using (1.29) the solution of the equation is
X(k)=—(2X9+X1) —3)*+3X0 +X1)C2).
Introduction to control theory 19

As the example illustrates, if all the roots \; of k(z) defined in (1.11)

are distinct, the general solution of (1.26) involves terms of the form
ci(Xi)* , where c; is a constant, so for unforced systems there is a steady
state solution only if all the \; have modulus not greater than unity. In
particular, constant terms in the solution correspond to As equal to unity
Solution and stability of linear discrete-time systems are dealt with in
detail in Chapters 3 and 5, along with their continuous-time equivalents.
Clearly there are close parallels between Laplace and z-transform methods,
and in fact a formal relationship between the two techniques can be est-
ablished. For details of this, together with standard theorems and lists of
transform pairs, reference texts such as that by Jury (1965) should be
consulted.
Application of z-transforms to (1.26) gives

X(z) =g(z)u(z)
where g is defined by (1.14). Some of the continuous-time frequency
domain methods have been extended to deal with sampled-data transfer
functions (see for example Saucedo and Schiring 1968).

Exercise 1.5. ;
' Show that the system represented in Fig. 1.8. is equivalent to that shown
in Fig. 1.10.

g(s)A(s)

FIG, 1.10

Since the feedback is simply equal to the output, it is called unity

feedback.

Exercise 1.6.
Find the transfer function for the system represented by the block dia-
gram in Fig. 1.11.

Exercise 1.7.
Consider the rabbit-fox environment described in Exercise 1.4. If the
experiment begins with initial population sizes x, (0) and x2(0) and if
20 Introduction to control theory

FIG. 1.11

in eqns (1.6) a,/a3 =a2/aq, use Laplace transforms to solve for x; (f).
Hence show that for arbitrary initial conditions the populations will
attain a steady state ast > c only if a; —aq4<0, and give an expression
for the ultimate size of the rabbit population in this case. Finally, deduce
that if the environment is to reach a steady state in which both rabbits
and foxes are present then x, (0) > (a, /a3 )x2(0).
What happens if x, (0) = (a, /a; )x.(0)?

Exercise 1.8.
Deduce by considering eqn (1.22) that the root loci start (i.e. K= 0) and
terminate (i.e. K = °°) at the poles and zeros respectively of the function
8(s)h(s).

Exercise 1.9.
For the transfer function obtained in Exercise 1.6. derive an expression
for the sensitivity with respect to g,. How can this be made equal to zero?

Exercise 1.10.
Obtain the z-transforms of the following functions, defined for k = 0, 1,
2,... (a) X(k) = kT (b) X(k) = e%, where a is a constant.
By replacing a by ia in (b) and considering imaginary parts, deduce that

{ d } z sina
z*—-2zcosatl.
Zysinak ¢=————_

Exercise 1.11.
Prove that

Z{e?*X(k)}=
X(ze)
 Introduction to control theory 21

whereZ{X(k)}=x (z) and b is a constant. Henceshow,usingthe result

of the preceding exercise, that
ze b sing
Z{e°* sinak}= - (1.32)
z? —2ze? cosate

Exercise 1.12.
A very simple model of a national economy can be constructed by assum-
ing that at year k the national income J; is given by

Ie = Cy +P, +G,,k =0, 1,2,."..

where Cx is consumer expenditure, Px is private investment and Gx is
government expenditure. Assuming that consumer spending is propor-
tional to national income in the previous year, i.e.
Cy =alp—-1,0<a<l,
and that private investment is proportional to the change in consumer
spending over the previous year, i.e.

Pe =p(Ce —Ce_7),0< 651;

| show that

Tk +2 -—(1 + BWK+1+ 0B Tp= GK+2°

If a=%, B = 1, government spending is at a constant level G, = G
(all kK)and J) = 2G, 1, = 3G, show by using z-transforms that

Te=2{1+(1/4/2)*sin %k
IG
(the result in (1.32) is needed).
The assumptions of the model therefore lead to a national income
which is oscillatory, and as k > © national income tends to twice govern-
ment expenditure.

Exercise 1.13.
The following simplified model for the buffalo population in the American
West in 1830 has been suggested by Truxal (1972). Let Fx, Mybe the
numbers of female and male buffalo at the start of year k (k = 0 corres-
ponding to 1830). Five per cent of adults die each year. Buffalo reach
maturity at age two years, and the number of new adult females alive
at the beginning of year k + 2, taking into account infant mortality, is
22 Introduction to control theory

12 per cent of Fx; more male calves than female are born, and the corres-
ponding figure is 14 per cent of Fz. Thus
Fy 49 ="0:95F,., + 0-12F, CY.33)

Use z-transforms to find expressions for F; and Mx, and deduce that as
k > © the population increases by 6.3 per cent per year.
This model will be studied further in Exercise 4.53 and Example 5.11.
2 Preliminary matrix theory

A brief review is now given of some mathematical material necessary for

the main part of the book. It has been assumed that the reader possesses
a basic knowledge of matrices and related topics. The aims of this chapter
are twofold: firstly, to supplement this knowledge with some less familiar
results; and secondly to establish notation and techniques so as to provide
a convenient source of reference for the remainder of the book. A gener-
ally non-rigorous presentation is used and most results are given without
proof. Further details can be found in the many standard textbooks avail-
able — see, for example, Cullen (1972) and Gantmacher (1959) for
mathematical treatment of linear algebra, and Director and Rohrer (1972)
for a systems theory approach.
If desired, this chapter can be omitted on a first reading without loss
of continuity, and any unfamiliar points which occur later can then be
referred to here.
2.1. Definitions

A useful compact way of writing a matrix is

ee ag hile ne ple 2h (2.1)
where aj; is the element in the ith row and jth column, and A thus has m
rows and n columns. The transpose A? is obtained by interchanging the
rows and columns of A, so A! = [aj] and isn x m; the conjugate trans-
pose is A* = [a;i] , where the overbar denotes complex conjugate. If
either m orn is equal to 1, the matrix is termed a row n-vector or column
m-vector respectively. When m =n A is said to be square of order n. We
shall write
fad
Ould
Fm larg! & (2.2)

for the unit matrix of order n which has the property that Al, =/,A for
any n X n matrix A. The matrix in (2.2) has all its elements zero except
24 Preliminary matrix theory

those on the principal diagonal, and any square matrix of this form is
called a diagonal matrix, written diag [a11,4@22,...,4nn]. It is assumed
that the reader is familiar with the usual rules of addition, subtraction,
multiplication, inversion, etc. for matrices. If the elements in any kK(<m)
rows and &(<n) columns of A are selected they form ak x 2 submatrix of
A, In particular A can be partitioned into submatrices (or blocks) aj; of
orders mj; x nj, as follows:

ny Na ° ° Ng

O11 | O42 ! oT a Oeaig my,

|
leery| ety fag ited | 2, -
1 |' Q22 | A : ! Qos mM
Ala ea apa ay ' Daal gare
i} 1 1 |
e | e 1e | | ° e
' | | |
---t--4--4----}--
| |
1 & Mr
|
where 2m; =m, Zn; =n. When A is square, a block diagonal matrix is
defined by

my, mM . My

O11 | 0 ; 0 my,
eS ee ee Ler
| 1 |
c iss
diag aes. «ss 0 |1 O22| Neer | 0 mM 3
at Sl ee es 4--- (2.3)
eu,
rr " 1 !
e I e 1 e | e °
| I |
across howe (rae
O- wsiQonk T *? hlebes Mr

A useful form of partitioned matrix is obtained by defining the Kronecker

product of two matrices A(m x n) and B(p x q) by
44,B a12B ; o : a,nB

a7,B a2,B ¢ * i a2nB

AQB= (2.4)
Preliminary matrix theory D>

where A is given by (2.1). Thus A@B is an mp x nq matrix and is parti-

tioned into the mn blocks shown in (2.4). It is easy to verify using the
definition that
(A @B)(C ®D)=AC@BD (2.5)
(A @B)* =A* @B*. (2.6)
If A isn xn and X isn x m then the matrix equation AX =C can be
written in the form

(A ® Im) x =o, (2.7)

where
x =([X11,X12,
s203XimsX215+++sX2amo->>
xa | (2.8)
is the column mn-vector formed from the rows of X taken in order, and
c is formed in an identical fashion from C. Similarly, XB = C can be
written

UT,®B")x =¢. (2.9)

Thus linear equations involving matrices can easily be rewritten in the
more usual matrix —vector form (see (2.18) and (2.19)).
The determinant det A or | A | is a scalar function of a square matrix,
and a knowledge of the basic properties of determinants is also assumed.
For example,

det A’ =det A, det A*=det A,

det (AB)=det A det B (2.10)

for any twon xXn matrices A and B, and

det A=ay, 422 +++Ayn (2.11)

if A is diagonal or triangular, i.e. all the elements above (or below) the
principal diagonal are zero, so that a;,= Oif7 >i (orj <i). If det A =0,
A is singular, otherwise nonsingular; in the latter case the inverse of A is

A= AdjA/det A

where AdjA = [Aj;], the adjoint of A, is defined by

Ay = (A)! *Mig
26 Preliminary matrix theory

and Mj; is the determinant of the submatrix formed by deleting row 7 and
column j of A. In general the determinant of any submatrix is termed a
minor of A.
The following result due to Schur on evaluation of partitioned
determinants is often useful. If W, X, Y, and Z are matrices having
dimensions n x n,n X m,m xXnandm x m respectively, with W
nonsingular, then

det
K7|e
WX Wdet
(Z-YW!
X) (2.1
A similar result holds if Z is nonsingular (see Exercise 2.3).

Exercise 2.1.
What is the condition on the dimensions of A and B for the product AB
to exist? What does this imply for the matrices in eqn (2.5)?

Exercise 2.2.
If A and B are nonsingular matrices, use (2.5) to show that (A @BY ' =
A! @B’}. What is the corresponding expression for (AB) *?

Exercise 2.3.
By considering the product

w' o| |wx

-Yw' Fry oz

and using (2.10), verify Schur’s formula (2.12). Similarly, if Z is non-

singular show that

det u WX]
A =
_ det Z det (W—XZ
NeoSs Y).

Hencededucethat
det U, AY )= det UZ, —FA)

Exercise 2.4.
Show that the matrix equation
AX +XB=C
 Preliminary matrix theory pag!

where A isn x n, Bism x m, and Cisn x m, can be written in the form

(A BIm +In @B')x=c

where x is defined in (2.8) and c is formed similarly from C

2.2. Linear dependence and rank

An important concept which we will find essential in linear control

theory is now introduced. Consider a set of n column m-vectors

ay 12 Gin

a 422 Zon
ay = . ’ a> a > » Ayn = 4 (2513)

4m1 an? inh

with the a;; allowed to be complex in general. The set of all such vectors
is called an m-space, denoted by C” (or R™ if the aj; are restricted to be
real). Both R™ and C”™are examples of a vector-space, and development
of the theory of vector spaces can be found in standard texts on linear
algebra. Notice that the vectors in (2.13) could equally well be written in
row form.
ipa, X;, are scalars then the vector

Ayy Xy FAQ XQ t+... Fan Xp

Keer. + Xp)Ay, = : (2.14)

Gm +1 tama xo ts, Fann kn

is called a linear combination of a,,...,d,.1faset of the xs, with not

all of them zero, can be found so that all the elements of the vector on
the right hand side of (2.14) are zero, then the vectors a,,42,...,@m
are said to be linearly dependent; otherwise they are linearly independent.

Example 2.1. If
1 6 =|
a, = 9) : ay = 0 > a3= ! — (2.15)
3 2 2
28 Preliminary matrix theory

then 22, +a, —4a3 = 0, so we can write

1
a, eG, a4 + 243
or
a, =—2a,+ 4a;
Or
1 1
a3 81 +74:
Thus each vector can be expressed as a linear combination of the other
two, sOa,, 2, a3 are linearly dependent, whereas any two 4; , a; are
linearly independent.
Given a particular set of n column m-vectors in (2.13), then the set of
all linear combinations (2.14) forms a subspace of C™. This subspace is
said to be spanned by a,,...,4y. If every column m-vector can be
expressed as a linear combination of the a; then (2.13) is termed a
spanning set for C™.

Example 2.2. When m = 3 consider the vectors

1 0 G 1
aie 0 Sa 1 qe tiie 6) s @a— 1 (2.16)
0 0 1 1

The subspace spanned by a, and @, consists of all vectors of the form

x}

Xx ,X1,Xq arbitrary.

0
The vectors a,, 42,43, 44 forma spanning set for C*, and for example

I. = 2a, +a, tS availa

In general there will not be a unique spanning set for C”. However, if
the members of a spanning set are linearly independent then the set is
called a basis, and it can be shown that any basis for C” contains exactly
m vectors, so that C™ is said to have dimension m.
Preliminary matrix theory 29

Example 2.3. For the vectors in (2.16)a4 =a, +a, +a3 but

X14, +X2 dz +X303 = |X

#0 unless x; =x, =x3=0.

$0 @1,4, 43 are linearly independent and form a basis for ‘See

In general for any finite value of m the set of unit m-vectors
1 0 0 0
0 1 0 0
0 > 0 > | ee ee b (2.17)

0 0 0

forms a basis for C™ or R™.

Closely related to the idea of dependence is that of the rank r of an
m X n matrix A = [a;;|, defined as the maximum number of linearly
independent rows (or columns) of A. Clearly r cannot exceed
p = min (m,n), and p —r is called the rank defect of A.

Example 2.4. Suppose a matrix is formed by using the three vectors

in (2.15) as columns, i.e.
1 6 -
Hien ad oreo 41
gent ations 9

As was seen in Example 2.1, any two of the columns are linearly inde-
pendent, so rank A = 2. Similarly, any two rows of A are independent.
The rank of A can be shown to be equal to the order of the largest
nonsingular submatrix of A. In particular, if A is square then it is non-
singular if and only if its rows (columns) are linearly independent.
Suppose now that the a;; are the coefficients in a set of m linear
algebraic equations in n unknowns
n
DS iy AB i 1,2, cs Mt (2.18)

jal
These equations can be written in matrix-vector form as
Ay =b (2.19)
30 Preliminary matrix theory

where

x=[x1,X0,0
0bXelt)b=bi, bose-oe (2.20)
Notice that the elements of Ax in (2.19) are precisely the same as those
on the right hand side of (2.14). Thus the problem of solving (2.19) for
the x; is equivalent to finding the coefficients in the expression of 5 as a
linear combination of the columns a; of A, if possible. It then follows that
(2.19) possesses a solution if and only if
Rank
A=Rank[4,5] (2.21)
where [4,b] is the m x (m + 1) matrix obtained by appending b to A as
an extra column. Two particular cases should be mentioned: when A is
square the equations (2.19) have a unique solution if and only if A is
nonsingular, and when the equations are homogeneous (i.e. all b; = 0) then
the necessary and sufficient condition for a nonzero solution to exist is
Rank A <n. Similar remarks apply to the set of equations.

yvA=b, y= [1,)25---.¥ml -

Exercise 2.5.
If a given m-vector v is expressed in terms of a given basis, ie.» =X, a
+ ++++Xp7Qm , show that the coefficients x; are unique.

Exercise 2.6.

Let A be an x m matrix and bda nonzero column n-vector such that

A’b #0,A’*! b = 0 for some positive integer r less than n. By considering
the equation
Cob + cy Ab +c, A*b ++ ++ +c,ATb=0

where the c; are scalars, deduce that the vectors b, Ab, A*b,... ,Ab are
linearly independent.

2.3. Polynomials

A polynomial a (\) of degree 5a = n is an expression of the form

a (A) =aoX" +4; NT 1+... +ay_y A+ an, (2.22)
where the leading coefficient ag is nonzero. When dg = | the polynomial
is called monic, and in this book the a; will be real or complex numbers.
We have seen in Section 1.2 that polynomials arise in linear system theory

.
 Preliminary matrix theory 31

when Laplace or z-transforms are applied to linear differential or differ-

ence equations respectively, and this led us to define a transfer function
in (1.14) as a rational function, namely the ratio of two polynomials. In
this section we record some basic properties of polynomials which are
often required.
If b (A) is a second polynomial with 6b <n then a (A) can be divided
by b (A) to give
a(A) = D(A) +r(A), (2.23)
where the remainder r(A) has 6r < 5b; if r=0 then a(A) is divisible with-
out remainder by b(A). Any polynomial d(A) which divides both a and b
without remainder is acommon divisor of a and b. The greatest common
divisor (g.c.d.) is the unique monic common divisor having greatest
possible degree, and is divisible by all other common divisors. The poly-
nomials a and b are relatively prime if their g.c.d. is equal to unity. A
resultant of a and b is a scalar function of the coefficients of a and b
which is nonzero if and only if a and b are relatively prime. These defi-
nitions carry over in an obvious way to any number of polynomials. For
example, a set of m polynomials p; (A), P2 (A), ... , Pm (A) is a relatively
, prime if there is no polynomial d(A) with 6d > 0 which divides each of
the p; (A) without remainder.
If the number c is such that a(c) = 0 then c is a root or zero of a (A).
This implies that a (A) is divisible by \ —cwithout remainder, and in
general every nth degree polynomial can be written in the form
a(X)=(A-A1)"t (A= Az)? + + = (A-Ap)!P (2.24)
where the A; are complex numbers, all different from each other, and the
t; are positive integers. The root A; has multiplicity t;, and t; +t, +++>
+ tp =n. Any root having unit multiplicity is called simple, and if t; = 1,
i=1,2,...,n, the polynomial is said to have n distinct roots.

Exercise 2.7.
Deduce that if there exist polynomials x(A) and y(A) such that
a(r)x (A) + BAYA) = 1
than a(A) and b(A) are relatively prime.
The converse of this result also holds.

Exercise 2.8.
If the coefficients a; in (2.22) are all real numbers, show that if c is a
root of a (A) so is its complex conjugate ¢
32 Preliminary matrix theory

2.4. Characteristic roots

Polynomials also arise in a fundamental way when matrix methods are

applied to differential equations in the state space form

xX(t) = Ax(t). (2.25)

Eqn (2.25) is the unforced version of (1.1), but now x(f) is an n-dimen-
sional state vector and A is a constant 1 x n matrix. We remarked in
Section 1.2.1 that the solution of linear systems consists of exponential
terms. Substituting x (t) = w exp (AL)into (2.25), where w is a column n-
vector and ) a scalar, gives

dw exp (At) = Aw exp (Af)

which implies Aw = dw, or
(UU, —A)w = 0. (2.26)
It follows from the discussion at the end of Section 2.2 that for the solu-
tion w of the homogeneous linear equations (2.26) to be nonzero we must
have

det (UJ, —A) = 0. bad

When expanded egn (2.27) represents an nth degree polynomial equation
KQQSN +k te ee tk, Athy =O. (2.28)

called the characteristic equation of A. The n roots dy, A2,..-, Ay Of the

characteristic polynomial k (X) are the characteristic roots of A (often
called the eigenvalues, an Anglo—German hybrid term), and may be
written as \;(A).
Since by virtue of (2.24) we can write

K(A)= det (AJ, —A)

= (AA) A~Ad) + + AAAn)
where some of the \; may be repeated, it follows by comparing terms in
d° and A”! that
(-1)'ky, = det A=)yAn °° * Ay

(2.29)
-k,=ta=>Ni i=1
where tr A = ay, +dy) + ++++ Gny the trace of A.
Preliminary matrix theory 33

For each value of \ there is a corresponding solution w; of (2.26), and

these vectors are called the characteristic vectors (or eigenvectors) of A,
and thus satisfy

Aw; =)jywj, i= 1,2,...,7. (2.30)

The solution of (2.25) is therefore a sum of terms w; exp (jt) and this
will be developed formally in Section 3.1. The w; are sometimes called
right characteristic vectors of A, the left vectors which are rows being
defined by :

vjiA==Api.
If the A; are all different from one another it can be shown that the w;
are linearly independent so the matrix

W= [Wi,W,; Hons»Wn]

is nonsingular. In this case we have

W? AW=W! [Aw,, AW2,..., AW]

=W! [\yW1,A2W2,-- +5 AnWy|
Maa [Nis as os Ay fk (2731)

Generally, two matrices A and B are said to be similar if there exists a

nonsingular matrix T such that T—' AT = B. Thus (2.31) shows that when
all the characteristic roots of A are distinct it is similar to the diagonal
matrix A of its characteristic roots. This is not a necessary condition: A
may have some repeated roots but will still be similar to A if it has n
linearly independent characteristic vectors. The case when A has repeated
roots will be dealt with in detail in Section 2.6. The matrix A is an example
of a canonical form, this being a standard form to which a class of matrices
can be reduced by specified operations.
The matrix .

0 1 0 0
0 0 1 . . °
Ga = ° e e. e e e (2.32)
34 Preliminary matrix theory

is the companion form matrix associated with k(A) in (2.28), since it is

easy to show that

det (AI, —C)=k(Q). (2.33)

Other companion form matrices can be written down with the k; in the
first or last column, or first row. The companion matrix is useful because
it possesses the same characteristic roots as A but has the relatively simple
form displayed in (2.32). We shall see that C plays an important role in
the study of linear control systems. Unfortunately however a practical
disadvantage of eqn (2.32) is that it is generally very difficult to calculate
the k; accurately for a given matrix A. A formula which is useful, but
mainly for theoretical rather than computational purposes, is Leverrier’s
algorithm, which states that the inverse of the characteristic matrix is
(Mn A) = [XM tN"7B, HOB, ++ ++ + By 11 /kQ)
(2.34)
where the k; and B; are determined successivelyby
Bi =A+kK,f,, B;=ABj-j tpin, i= 233,450
1 (2.35)
k, =-trA, ki=—~tr(AB;
- 1), i=2,3,cee
Another very useful result is the Cayley—Hamilton theorem, which
states that every square matrix satisfies its own characteristic equation,
ie.
A” +k, Ani +k Anz 4 ae ont Kee TA+k i ae (2.36)
The expression in (2.36) is a polynomial in the matrix A and is written
k(A). It is not to be confused with (2.28), which is a polynomial equation
in the scalar A, and is satisfied by the n values \,,..., Ap-
If B is an m x m matrix having characteristic roots and vectors Mj,¥;
respectively then by (2.5) and (2.30)

(A ®B) (w; @yj) = Aw; ®By;

= iw; B My;
= inj (w; By)
so the characteristic roots of A® Bare \juj,i=1,2...,n,7=1,2,...,
m. We can use this result to study the matrix equation

AX+XB=C (2.37)
 Preliminary matrix theory 35

where X and C are n x m. Using (2.7) and (2.9) this can be written in the
form (see Exercise 2.4)
Dx =c (2.38)
which is a set of nm equations in the familiar forms (2.18). The solution
of (2.38) is unique if and only if the mn x mn matrix
D=A®@Im +I, @BE
is nonsingular. To find the condition for this to hold, consider
(In +€A) ® Um +€BT)=Iy BI +eD +e2A OB!
which has characteristic roots (see Exercise 2.9 (c))
(1 + €dj) (1 + euj) = 1 + (A; +uy) + €? Aju,

It follows by comparing terms in € that D has characteristic roots A; + wy,

i=1,2,...,n,j=1,2,...,m. Hence by (2.29) D is nonsingular if and
only if there are no characteristic roots of A and B such that ); + uy;= 0,
and this is the condition for the solution X of the matrix equation (2.37)
to be unique.
We shall find in Chapter 5 that a particular form of eqn (2.37) enters
into the study of stability of the linear system (2.25) using Liapunov
theory. (See also Exercise 2.17.)

Exercise 2.9.
Prove the following
(a) (A")= (A) (6) 2(4*) =; ()
(c) An +A)=1+2AKA)= @_~—2GA™) = 1/2;(A),provided
A isnonsingular.
Exercise 2.10.
A square matrix U is said to be unitary if UU* =I. Use the preceding
exercise to show that IA,;(U) |= 1.
The result still holds when U is real, in which case it is called
orthogonal.

Exercise 2.11.
By considering eqn (2.30) show that if m is a positive integer the
characteristic roots and vectors of A” are Aj” and w;. Hence deduce that if

BOA)
=BoN™
+ByA™
| 46+++Bm
then the characteristic roots and vectors of B(A) are B(\;) and w;.
36 Preliminary matrix theory

Exercise 2.12.
Using the result of the preceding exercise, show that the matrix B(C),
where C is defined in (2.32), is nonsingular if and only if B(A)and k{A)
defined in (2.28) are relatively prime.
This shows that det B(C) is a resultant for the polynomials B(A)and
K(A).

Exercise 2.13.
If in the preceding two exercises m <n, show that the first row of

B(C)is
T= [Bi B= Le oesBixBox
0.0.020]e
Show also that the subsequent rows arerC, rC?,..., ross
(Hint: if e; denotes the ith row of J, first show that e; =e; -; C,
i=2,3,...,n,and then consider e;8(C)).

Exercise 2.14.
Show that if k, #0 then the inverse of C in (2.32) is also a companion
form matrix with first row

[—kn—1/Kkn»
Kn —2)has “meee» ki Kn, 1 [ky]
and deduce that the characteristic polynomial of C—!
is (N"/kn) k (1/A).
Using an argument like that in Exercise 2.13, show that if B(A) is the
polynomial in Exercise 2.11 with m <n, then the rows of B(C') are
sC"*! sC—-"*2 v2, sC~*, 8, where s = [0, .. . 50,Bo ae

Exercise 2.15.
If the characteristic roots of C in (2.32) are Ay, Az, ... , Ay, assumed
distinct, show that corresponding characteristic vectors are

Me [Ay Me ee |! Lf=12,
The matrix M= [m,,m2,..., my] is called the Vandermonde matrix.
Prove by induction on vn,or otherwise, that detM= TI (Aj —Aj).
far

Exercise 2.16.
The linear discrete-time system corresponding to (2.25) takes the form
x(k +1)= Ax (k);k=0,1,2....
 Preliminary matrix theory 37

Show that a solution of the form x(k) = wAk leads to the same character-
istic root and vector equation (2.26).

Exercise 2.17.
Show that the matrix equation

AXB-X=C,

where A isn xXn and Bis m x m has a unique solution if and only if there
are no characteristic roots dj, u; of A, B respectively such that dj uy;= 1,
Li eee i ls2, ss «5 I.
A particular version of this equation occurs in Chapter 5 involving
stability of discrete-time linear systems.

Exercise 2.18.
By a suitable choice of Wand Z in Exercise 2.3, show that for any
matrices X(m x m) and Y(m x n)

det(ul, —XY)=u" ™det(wy —YX).

This shows that, assuming m <n, the characteristic roots of XY are those
_ of YX together with n— m zero roots.

2.5. Polynomial matrices

We saw in Section 1.2 how systems described by a single linear differ-

ential or difference equation lead to scalar transfer functions. Consider
now an nth order linear system with multiple inputs and outputs
described by the vector—matrix equation (1.1), namely

% (t) =Ax(t) + Bu(t). (2.39)

where A isn x n,B isn x m, x(t) is the column n-vector of state
variables and u(t) is the column m-vector of input or control variables.
Suppose that there are r output variables, each a linear combination of the
x;, so that

y(t) = Cx(t) (2.40)

where C is a constant r x n matrix. Taking Laplace transforms of (2.39)
and assuming zero initial conditions gives
sx (s) = Ax (s) + Bu (s)
38 Preliminary matrix theory

and after rearrangement

x (s) = (sl, —A) | BuG) (2.41)

Since from (2.40) the Laplace transform of the output is

y(s) = Cx(s) (2.42)

clearly
¥(s)=C(sl —A)" B us)
= G(s) u(s) (2.43)
where the r Xxm matrix
G(s)=C(sl— A) 1B (2.44)
is called the transfer function matrix by analogy with the scalar case
(1.13), since it relates the Laplace transform of the output vector to that
of the input vector. A block diagram representation of the form shown in
Fig. 1.8 can still be drawn, with input and output now vectors, and the
operator is the matrix G(s) which of course reduces to g(s) whenr = m= 1.
Using eqn (2.34), the expression (2.44) becomes

G(s)=(s"!Gp +5"~2G, +> +* +GnaWkG)

=H(s)/k(s),. (2.45)
where k (s) is the characteristic polynomial of A and the Gx -|g i) are
constant r x m matrices. The r x m matrix H(s) is called a polynomial
matrix, since each of its elements is itself a polynomial, i.e.

hoes ge a ee Ps +
Example 2.5. Consider the electrically-heated oven described in
Example 1.2, and suppose that the values of the constants in (1.4) are such
that the state equations are

t=ie mlx+A u. (2.46

Suppose that the output is provided by a thermocouple in the jacket
measuring the jacket (excess) temperature, i.e.

y=f[l O}x.
Preliminary matrix theory 39

The expression (2.44) gives

Shooe
Or o ten a afl
oul
2 +35
using
(sf!
—A)'
=Adj
(s/—
A)/k(s).
In practice it may well happen that the differential equations describ-
ing a linear system may not all be first order, as they are in (2.39).
Suppose that there are &state variables £; and m control variables u;. After
Laplace transformation the equation corresponding to (2.41) is
T(s) £ (s) = U(s) u (s), (2.47)

again assuming zero initial conditions. The matrix—vector equation

(2.47) is obtained using the expression for the Laplace transform of a
derivative in (1.8), so each of 7(s), U(s) will be a polynomial matrix. If
the output is now a linear combination of &and its derivatives then
(2.42) is replaced by
y (s) = Vis)E(s) (2.48)
where V(s) is also a polynomial matrix. Combining (2.47) and (2.48) we
again have eqn (2.43) but now the transfer function matrix is

G(s) = V(s) T (s) UG). (2.49)

This of course reduces to (2.44) when 7(s) = sl, —A, V(s)=C, U(s)=B.
We have said that the system (2.39) has nth order since there are n first
order differential equations; more generally the order of the system
described by (2.47) is defined to be the degree of det T(s), which is the
characteristic polynomial of the system.

Example 2.6. Consider the problem of two masses joined by springs

described in Exercise 1.2. Using the notation of Fig. 1.6, Newton’s
equations of motion are
Gax;
my dt? Sky (xjx) tu

d*x
m2aapee (x;—X2)
kpx.
40 | Preliminary matrix theory

t= B) T(s)
=ie =Ky+k,mS? =i
+k, +k, |

U6)= | 1

Assume that the output is y =xz, so that V(s) = [0, 1]. Hence from
(2.49)

G(s) = V(Adj T) U/det T(s)

=k,/[myms* + (kym, +kymz +km, )s* +kyko]

and the order of the system is 4.

An extensive theory of multivariable linear control systems using
polynomial matrices has been developed by Rosenbrock (1970) and
others. The following properties of polynomial matrices are however
developed mainly for the limited objective of studying matrices with
repeated characteristic roots.
Let P(A) be an arbitrary n x n polynomial matrix having rank r, which
is the order of the largest minor of P not identically zero. The following
elementary operations on P leave its rank unaltered:
(i) interchange any two lines
(ii) multiply any line by a nonzero constant
(iii)add to any line any other line multiplied by an arbitrary poly-
nomial (‘line’ stands for either row or column).
Two matrices P; (A) and P, (A) having the same dimensions are said to
be equivalent if and only if it is possible to pass from one to the other by
a sequence of elementary operations. This turns out to be the same as
stating that

Py (A) =LOA)P2(A)MA) (2.50)

where L(A) and M(A) are both polynomial matrices having determinants
which are nonzero constants (i.e. independent of A). It can be shown that
Preliminary matrix theory 41

every square polynomial matrix} can be put into diagonal form by

appropriate elementary operations: specifically P(A) is equivalent to the
Smith canonical form

S(A) = diag[i, (A), 2Q),...,%-(), 0,..., OJ. (2.51)

where the elements in (2.51) are polynomials such that iz (A) divides
ix + 1(A), k= 1,2,...,7—1. It can also be shown that the iz (A) satisfy

ix (A) = dx (A)/de-1(r), kOe RT: (2.52)

where dx (A) is the g.c.d. of all minors of P(A) of order k (these minors
are of course polynomials) and dx is called the kth determinantal divisor
of P(A) (taking dp = 1). Notice that it easily follows from (2.52) that if
r=n then
tity ce ly =dy
= det P(A). (233)
Since the application of elementary operations to P(A) does not alter its
Smith form, the iz (A) in (2.52) are called the invariant factors of P(A).
As in (2.24), each iz (A) can be written

ip (A)=(A—0) Bk (A—
022)Bk,-- (A—04)PRS,
K=1,...,7 (2.54)
where the a; are in general complex numbers, and the notation is chosen
so that a, @2,..., Qs are the distinct roots of i, (A), which implies that
Br1, Br2, - - - Bryare all positive integers. In view of the divisibility prop-
erty of the iz, it therefore follows that all ae remaining 6xQwill be
nonnegative integers. The factors (A —aj) Bre occurring in (2.54) for all
values of j, k, 2 such that By.g#0 are tia the elementary divisors of
P(A). It can be shown that P; (A) and P, (A) are equivalent if and only if
they have the same rank and their respective sets of elementary divisors
are identical.
Example 2.7. If a matrix has Smith form

” 0 )
S=1]0 M4(A+ 1)? 0
0 0 AM(A+1)* (A+ 2)

+Extension to rectangular matrices is straightforward.

42 Preliminary matrix theory

then it is very easy, by considering minors of orders 1, 2, and 3, to obtain

the determinantal divisors

d, =03,d_ =\A7(A+1)?, dy =A? + 1)° AF 2).

The invariant factors are the diagonal elements of S, so the elementary

divisors are

NOt 1) OS Oa

Our main interest in this book is in the special case when P(A) is the
characteristic matrix MJ, —A. In this case the invariant factors are often
called the similarity invariants of A, because it can be shown that two
n X n matrices are similar if and only if their similarity invariants are
identical. This result enables us to determine when a given matrix A with
characteristic polynomial k(A) is similar to the associated companion
form matrix C defined in (2.32). The argument requires some further
definitions. Firstly, the Cayley—Hamilton theorem states in (2.36) that
k(A) = 0, but it can happen that there are other polynomials for which
this holds. Suppose that the monic polynomial m(A) is such that

m(A)= AP+m,AP~1 4+ +++ mply =0. (2.55)

If there are no polynomials which have degree less than p and satisfy
(2.55), then m(A) is called the minimum polynomial of A.

Example 2.8. If

it is trivial to verify that k(A) = \° and that A? =0, but A +m,/#0 for
any value of m,, so the minimum polynomial is m(A) = A’.
When m(A) is identical to k(A) then A is termed nonderogatory, other-
wise derogatory (much of the nomenclature in matrix theory was coined
in the Victorian era —the implication is clear that a derogatory matrix has
undesirable properties!). The next step in determining similarity of Aand
C is to note that it can be shown that m(A) is equal to the similarity invari-
Preliminary matrix theory 43

ant of A having highest degree. As we have seen in (2.53) these similarity

invariants s;(A) satisfy

$182 °° * Sy = det (Al, —A)

=k(a), (2.56)
and also s; divides sj+,. It therefore follows that if Ais nonderogatory
then sy(A) = k(A), so its similarity invariants are 1,1,..., 1, k(A). By
definition the similarity invariants of C are the invariant factors of N,, —C,
and using the form of C in (2.32) it is immediately obvious that the first
n-—1 determinantal divisors of WJ—C are equal to unity. Thus by (2.52)
and (2.53) it follows that the similarity invariants of Care 1,1,...,
1, det (AJ—C) = K(A).We have therefore established that A and C have
the same similarity invariants, and so are similar, if and only if A is non-
derogatory. This result is of course independent of whichever of the four
companion forms is used.

Example 2.9. By considering minors of 3 —A of orders 1, 2, and 3

it can be verified that when

oS) onl
b= kOenent.
¢ aOels (2.57)

‘Shine eal
the determinantal divisors are d; = 1,d, =(A— 1), d3 =(A—1)?. Thus
eqn (2.52) gives

BepiloigaN) pis=(A-1)’,
and m(A) =i3. Clearly k(A) =(A—1)° so A is derogatory, and there is no
nonsingular matrix 7 such that

It should be noted that if all the characteristic roots of A are distinct

then it has the same similarity invariants as the diagonal matrix A in
44 Preliminary matrix theory

(2.31). Because of the assumption on the ); these invariants are easily

calculated to be

1, l,--251,0A- MYO?) °**

AT%,) = deter

It therefore follows by the preceding discussion that in this case A is

nonderogatory; in other words, only a matrix with repeated roots can
be derogatory.

Exercise 2.19.
Derive the transfer function in Example 2.6 using the first order linear
state equations obtained in Exercise 1.2.

Exercise 2.20.
Suppose a harmonic input u = ugcos wt, where ug is a constant vector,
is applied to (2.39). Assume that x(t) has the form acos wt + bsin wt,
and determine the constant vectors a and b (assume A has no purely
imaginary characteristic roots). Hence show that the output (2.40) is

y= Re{G(iw)}uocoswt —Im{G(iw)}uo
sinwt
where G(s) is defined in (2.44).
This provides an interpretation for the frequency transfer function
matrix G(iw), and when G is a scalar the expression for y reduces to
precisely that given in eqn (1.20).

Exercise 2.21.
Show that the minimum polynomial of any square matrix A is a factor
of its characteristic polynomial. (Hint: assume that division of k(A) by
m(A) produces a remainder term, and obtain a contradiction).

Exercise 2.22.
Given ann x n matrix A, if a column n-vector b can be found such that

rank [b, Ab,A?b,...,A"~ 1b] =n (2.58)

deduce that A is nonderogatory. (Hint: show that in (2.55) p =n, and
apply the result of Exercise 2.21).
The condition in (2.58) will be encountered again in a control context
in Sections 3.7 and 4.1.
 Preliminary matrix theory 45

2.6. Jordan canonical form

We can now return to the problem mentioned in Section 2.4 of determin-

ing a canonical form for A when its characteristic roots are not all distinct.
A general result states that any square matrix is similar to the Jordan form.

diag Ym, (Ar), Im, Ar), - «+Amy Ar) Sn, Ar), +++Ft, Ag) I], (2.59)
using the notation of (2.3), where Jz (A) is the k x k Jordan block
r 1 0 . 0 0
0 r 1 : 0 0)
Bein) *| : : 2 : é (2.60)
0 0 : r 1
0 0 , ° 0 r
(an alternative form has the 1s below the principal diagonal). The distinct
characteristic roots of A are \y,2,..., Aq and the multiplicity of A; is
m, +m ++ + +mg, and so on. The Jordan form of a matrix is unique up
to the ordering of the blocks in (2.59), and can be written down once the
' elementary divisors of J —A are known (see Exercise 2.24). If none of
the characteristic roots of A is repeated, so that g =n, eqn (2.59) reduces
to the diagonal matrix (2.31). In this case each Jordan block is simply
J; (Ai) =);
It should also be noted that the Jordan form of A can be a diagonal
matrix even if some of the characteristic roots of A are repeated. There
is just one linearly independent characteristic vector of A associated with
each Jordan block, so the total number of such vectors is equal to the
number of blocks in the Jordan form (2.59). In other words, A is similar
to a diagonal matrix if and only if it has n linearly independent character-
istic vectors. An important special case of this is that every real symmetric
matrix (i.e. Al = A) has a diagonal Jordan form; in addition a transforming
matrix can be found which is orthogonal.

Example 2.10. The matrix in Example 2.9 is clearly in the Jordanform

(2.59), and (2.57) can be written as
A= diag[J2(1),J1,(1)]é
In the above example A is derogatory and has two Jordan blocks associa-
ted with the root 1. This illustrates a general result: A is nonderogatory
if and only if there is only one Jordan block associated with each
46 Preliminary matrix theory

Ai = 1,2,...,q). Furthermore if a; is the order of the largest Jordan

block associated with A;in (2.59) then the minimum polynomial of A is
m(A) = (A—Ar)™!(A= Az)% + 2 (AAG), (2.61)
and of course coincides with k(\) when q =n, each a; then being equal to
unity. Since J; (A) in (2.60) is triangular its rank is clearly k if X#0 and
zero if \ = 0. The matrix A is similar to (2.59) and so has the same rank,
which because of the triangular form of (2.59) is the sum of the ranks of
each Jordan block. In other words, the rank of A is equal to the number
of its nonzero characteristic roots, including repetitions.
Although A cannot be transformed into companion form when it is
derogatory we can rearrange its Jordan form (2.59) into

diag 1D, Dae)

where
D, = diag Jin, Qa),In, Q2),--- St, Ag],
Dy'= diag Um Qu), Jn, 04),-- te, CO
etc. Each D; is nonderogatory and so can be put into companion form, so
A is similar to

diag [C, C2, ...]

where the C; are companion form matrices.
It should be noted that although the Jordan form is of fundamental
theoretical importance it is of little use in practical calculations, being
generally very difficult to compute.

Exercise 2.23.
It will be shown later (Section 2.8) that if A is a real symmetric matrix
then all its characteristic roots are real. If none of these roots is negative
deduce that there exists a nonsingular matrix T such that
Tee

where p = rank A and Og denotes aq x gqmatrix of zeros.

Exercise 2.24.
Show that \/x, —Jx(Aj) has the single elementary divisor (A —dyihe
It can be shown that the elementary divisors of \J —A are

CDs Ag) gee TAL) A Ae) poet

Preliminary matrix theory 47

Exercise 2.25.
Prove by induction on r that

we(AK elie
1 2
J, QO)?
=| 0 ne Arye 6 9230:4:
Nao
0 0 nN

A similar result holds in general:

heal ies ‘ ° (x1)¥

g =ht1
0 v msA ( r a5
x (A))"
= oes
0 0) : , nN
( ‘4denotes
the
binomial
coefficient
rset
2.7. Functions of a matrix

We now extend some ideas of scalar functions to matrices. These

results will be used to solve linear system equations in Chapter 3.
To discuss convergence of infinite sequences and series involving
matrices and vectors it is first necessary to define the euclidean norm
of anm x n matrix A:

WAlle=|
> >, lay? (2.62)
where |a;| denotes modulusof aj;. For a vectorx (2.62) gives
1

n
Ixle=|_
> bel?
rT

(2.63)
48 Preliminary matrix theory

which can be thought of as the ‘length’ of the vector x. Other norms can

alsobedefined,forexampleia lajj| or» |x;|.Allnormsforboth

i=]
matrices and vectors with complex elements satisfy the conditions
|| K || >0O
unless K = 0, (2.64)

|| KK || =| ||| K || for any scalar k, (2.65)

IK+L SUK VFL I (2.66)

If K and L are matrices then &

IAL |< K WL I, (2.67)

and if 2 is a vector

| K2lle SI] K lle Il £ lle. (2.68)

A sequence {Ar}, r=0, 1;2,...of matrices is said to converge to the

limit A if each element of A; tends to the corresponding element of A as
r > c, A necessary and sufficient condition for convergence is that

Rim || A, -A || =0
T — co

for any norm. It is easy to show that if Pand Q are constant matrices
then PA,Q > PAQasr > ©, In particular a square matrix A can be
written as A= 7JT~' where J is the Jordan form in (2.59). Thus A” =
TJ’ T~* and since the elements of J’ depend upon powers of the charac-
teristic roots A; of A (see Exercise 2.25) it follows that Rim A” = 0 if and
r = yee

only if all the characteristic roots of Ahave modulus less than unity.
co
The infinite matrix series > Ax converges provided the sequence of
k=0
r
partial sums X; = Azx,r=0,1,2,..., converges to a finite limit as
k=0
co
r > ©, The series is absolutely convergent if the scalar series > || Ax |
k=0
is convergent, and absolute convergence implies convergence.
 Preliminary matrix theory 49

We consider scalar functions which can be defined by the power series

fO)=>, curt (2.69)

k=0

which is assumed to converge for | \ |< R. The associated function of an

nx n matrix A is defined by

f(A) =Coln tc1A +C2,A* +.... (2.70)

A powerful general result states that the matrix power series in (2.70) is
convergent if all the characteristic roots \j of A satisfy | Aj |<R.

Example 2.11. The series

Pe y= X—3' FIR

converges for | \ |< 1. Hence the matrix series

Qn , tA)=A-4§ A? +442 ------

converges provided A is such that all its characteristic roots have modulus
_ less than unity.
It can be shown that when all the ); are distinct, then assuming con-
vergence,

f(A) = > ZF (Ak) (2.71)

k=1
where

eames 2X2!)° AS AR 1) Cle tt). (A Nad)

(Aq—A1)AK—A2)°°* Ok = 1) OK— 41) °°OkAn)

=I(Cee Ole (2.72)

#k

The expression (2.71) with the Z, defined in (2.72) is known as Sylvester’s

formula. The reader with some knowledge of numerical analysis may
recognize the resemblance between (2.72) and the Lagrange interpolation
polynomial (see Exercise 2.28). It is important to realize that the Zx are
independent of f, and that only the values f(Ax) occur in (2.71), so any
two scalar functions (2.69) for which these values coincide will lead to
50 ) Preliminary matrix theory

the same matrix function. Note also that (2.71) still holds if f(A) is a
finite series.
An alternative way of evaluating f(A) when all the Aj are distinct is as
follows. As in (2.23) we can write

FA) = GA) KA) +7.QY, (2.73)

where k(A) is the characteristic polynomial of A in (2.28), and 6r7<n.

The expression in (2.73) is an identity, so we have

f(A) = q(A) k(A) + 7(A)

which by the Cayley—Hamilton theorem (2.36) reduces to

f(A) =r(A), (2.74)

showing that f(A) can be represented by a finite sum of powers of A of

degree not exceeding n —1. To determine

rary try 2 te trp

substitution of k(Aj) = 0 into (2.73) gives

fO) =r0\),i=1,2,...,n (2.75)

which are 7 linear equations for the n unknown coefficients 7;. Examples
using both approaches to evaluation of f(A) are given in Chapter 3.
When A has repeated characteristic roots the general expression for
f(A) is

f(A)=» [FOue)
Zier+£de) Zia+2 +fK—YD
Ag)Zkag]
k=1 (2.76)
where the a; are the indices in the expression for the minimum polynomial
in (2.61). In (2.76) f (Ax) denotes the rth derivative of f with respect
to A, (not t) evaluated at \ = Ax. The matrices Z,; are constant and again
are determined entirely by A, not the function f. Although an explicit
formula can be given for these matrices, they are best determined by
taking simple choices for f, and again this is illustrated by means of
examples in Chapter 3. When all the A; are distinct so that q =n, (2.76)
reduces to (2.71). If some of the ); are repeated, but the minimum poly-
Preliminary matrix theory 51

nomial m(A) has only simple roots (i.e. in eqn (2.61) all a; = 1) eqn
(2.76) becomes
q
f(A)= >, Zi fx) (2.71)

k=1
where the Zz are defined as in (2.72) except that the upper index in the
product is q instead of n.
We end this section with a warning: although the power series (2.69)
has been used to define a function of a matrix, it cannot be invariably
assumed that properties of the scalar function carry over to the matrix
case (see for example Exercise 2.27).

Exercise 2.26.
For what matrices A does the series for sin A converge? Use (2.71) to
determine sin A if

Similarly, write down expressions for A! and 47’.

Exercise 2.27.
If A and B are twon x n matrices prove that

sin (A +B)=sin Acos B+cosA sin B

only if Aand Bcommute with each other.

Exercise 2.28.
Consider an (n —1)th degree polynomial p(A) which takes the values
PAK) = px, k =1,2,...,n, with all the Ay distinct. By writing

PA)=> =A) =a) + AHMa1)A Mew1)A-MdPe

R=)
determine the constants P;,, and hence obtain the Lagrange interpolation
formula (compare with (2.72)).
52 Preliminary matrix theory

Exercise 2.29.
Show that in eqn (2.71) the matrices Zz satisfy

n
ee n
>) Ze 47,7 1,2,352.
k=1 k=1
2.8. Quadratic and Hermitian forms

The scalar function of the state vector x defined by

= aigre + Ay.x3 + 2419X1X2+ 2443X1X3+ Pa (2.78)

where the x; and aj are real, is called a quadratic form, since each term
in the sum is quadratic in the components of x. It is assumed that in (2.78)
A is symmetric (i.e. A? = A) and this causes no loss of generality (see
Exercise 2.31). The corresponding expression when the x; and aj are
complex is the Hermitian form

non
h(x) =x*Ax = > ‘ ayxix;, (2.79)

i=1 j=l
with A now a Hermitian matrix (A* = A). Notice that the complex con-
jugate of h is
h=x!Ax=(x*A*x)?
=h,
showing that h is real for all vectors x. We can show similarly that all the
characteristic roots Aj of A in (2.79) are real. For let w; be a characteristic
vector satisfying (2.30), so that

wj*Aw; = wi* (Aj Wj). (2.80)

However, (2.30) also gives (Aw;)* = (Aj w;)*, or wj* A = wi* Ns since
A* =A. Hence
wi*Aw;= (wi*Xi)Wi, (2.81)
and together (2.80) and (2.81) imply ri = )j, since w;*w; = || w; lla #0
by (2.64). Thus all the characteristic roots of aHermitian matrix are
 Preliminary matrix theory 53

real, and the argument still applies if Ais symmetric. Because of this
property algorithms for calculating characteristic roots of symmetric and
Hermitian matrices are simpler than for general matrices. Also, as we
remarked in Sections 2.6, every real symmetric matrix is orthogonally
similar to the diagonal matrix of its characteristic roots, and this still
holds for Hermitian matrices except that the transformation matrix is
then unitary.
In our work on Liapunov stability theory in Chapter 5 we will be
interested in Hermitian or quadratic forms v(x) which do not change sign
as x varies, so the following definitions are needed:
(i) v(x) is positive (negative) definite if v(0) = 0 and v(x) > 0 (K0)
when x # 0.
(ii) v(x) is positive (negative) semidefinite if v(O) = 0 and v(x) 2 0
(< 0) for all values of x, with at least one x #0 such that
v(x) = 0.

Example 2.12. When n = 2 the quadratic form

q = 2x? + 3x3

is clearly positive definite. However if n = 3, q is only positive semi-

' definite, since although it never takes negative values the choice x, =0,
Xz =0,x3 =c makes g = 0 for any value of c.
A form satisfying neither of (7) nor (ii) above is indefinite, and takes
both positive and negative values. The terms describing the form are also
applied to the matrix A associated with the form. The definitions on
definiteness can be extended to scalar functions which are not necessarily
quadratic (see Section 5.4).
There are a number of ways of determining the sign properties of a
form. One simple approach is to change coordinates by means of x = Uy
so that (2.79) becomes

(Uy)* A(Uy) = y* (U*AU)y (2.82)

and as stated earlier it is always possible to choose a unitary matrix U so
that (2.82) reduces merely to

n
> wl P. (2.83)
i=1
Since U is nonsingular (see Exercise 2.10) y = 0 if and only if x = 0, so
it follows that v will be positive (negative) definite if and only if all
 54 Preliminary matrix theory

Xi (A) > 0 (< 0); positive (negative) semidefinite if and only if all \; (A)
20 (<0), with at least one A; (A) = 0; and indefinite if and only if at
least one A; (A) > 0 and one ); (A) < 0. In fact it is possible to avoid
calculation of the Aj by reducing v(x) to a sum of squares by Lagrange’s
method (Mirsky 1963, p. 371), the numbers of positive, negative and zero
coefficients in the sum being the same as in (2.83).
An alternative approach involves the principal minors P; of A, these
being any 7th order minors whose principal diagonal is part of the principal
diagonal of A. In particular the Jeading principal minors of A are

G41 232 411 412 43

D; =41, Dz =| a2 22 , D3 = |a21 G22 a23| , etc.
431 432 433
The Sylvester conditions state that the Hermitian or quadratic form v is:
() positive definite if and only if D; >0,i=1,2,...,n.
(ii) negative definite if and only if (—1)'D; > 0,i=1,2,...,n;
(iii) positive semidefinite if and only if det A= 0 and P; > 0, for
all principal minors;
(iv) negative semidefinite if and only if det A= 0 and (—1)'P; >0,
for all principal minors.
If v satisfies none of the above conditions (i)—(iv) then it is indefinite.

Example 2.13. Consider the quadratic form with n = 3:

y=x? +42 + 9x3 + 4x4xq + 12x2x3 + 6%1%3.
The matrix of the form is

1 ps 3
A= |2 4 6
3 6 9
and the principalminorsare
ge: a cated
A2 4ae =0, | 4oa 6 =0, ia9 pa Lents

Hence by condition (iii) v is positive semidefinite.

A convenient numerical procedure for testing sign properties of v(x)
when n is large is provided by writing A=P TP with P a triangular matrix.
- An explicit formula is easily derived for the pj (Barnett and Storey 1970,
p. 35) and A is positive definite if and only if all pj; > 0.
Preliminary matrix theory =)

Exercise 2.30.
A skew symmetric real matrix S is defined by ST =-§.Ifq=x1Sx show
by considering (q)! that q =0 for all vectors x. Similarly show that x*Sx
=0 if S is skew Hermitian, i.e. S* =—S.

Exercise 2.31.
Show that any real matrix A can be written as A= A, + A, where A, is
symmetric and A, is skew symmetric. Hence using the result of the pre-
ceding exercise, deduce that x! Ax =x! A,x, allx.
A similar argument applies for complex matrices.

Exercise 2.32.
Let A be a Hermitian matrix, and write it as A =A, + i42, where A,
and A, are real and i=+/(— 1). Show that A, is symmetric and A is
skew symmetric. Hence deduce that if A is positive definite, so is A,.

Exercise 2.33.
Two column n-vectors a and b are said to be orthogonal if a*b = 0. Show
that if w;, w; are characteristic vectors associated with distinct charac-
teristic roots of a Hermitian matrix A then w; and wy;are orthogonal.

Exercise 2.34.
If A is an x n symmetric matrix having all aj; = 1, deduce that it has a
single nonzero characteristic root equal in value to n (Hint: use (2.29)).
3 Matrix solution of linear systems
’

The importance of linear models in physical situations was indicated in

Chapter 1, and in Section 1.2 we discussed the Laplace and z-transform
methods for solving continuous-time and discrete-time equations respect-
ively. However, except for low order systems, the algebraic manipulation
required soon becomes very heavy and it is preferable to apply the tech-
niques of matrix functions outlined in Section 2.6. We shall assume
initially that our system equations are already in vector-matrix form, and
postpone until the last section of this chapter discussion of the relation-
ship with scalar differential or difference equations of the type set out in
Section 1.2. The methods described in this chapter may seem rather
formal, but of course are applicable to any linear system model, such as
those described in the examples and exercises in Chapter 1.

3.1. Solution of uncontrolled system: spectral form

We were able to express the linear examples in Section 1.1 in the matrix
form (1.1). To begin with we shall consider systems without the presence
of controlled variables, and in this and the following two sections we
discuss methods of finding the solution for the state vector x (¢) of the
nth order system described by

x = Ax, (3.1)

subject to a given initial condition, say

x(0)=Xpo. (3:2)

We shall first assume that all the characteristic roots Ay, A2,.--, An
of A are distinct. In fact in real-life situations this is not too severe a
restriction, since if Adoes have repeated roots, very small perturbations
in a few of its elements (which will only be known to a certain degree of
accuracy) will suffice to separate these equal roots. If w; is a character-
istic vector corresponding to Aj then W,, W2,...,W» are linearly
 Matrix solution of linear systems 57

independent (see Section 2.2) so we can express the solution of (3.1) as

x(t) = S CiWi (3.3)

i=]
where the c;(t) are scalar functions of time. Differentiation of (3.3) and
substitution into (3.1) gives

Lc; =A LCjW;

= DAiciw; ,

using (2.30). Hence by the independence of the w,,

Cj = Ajj, BTN 2. ails

and these equations have the solution

c;(t) = exp (A;t)¢;(0), j= .2 See

_ giving

n
x(t)= 3 ¢; (0) exp Cyt); - (3.4)
ped
This generalizes our informal remarks in Section 1.2 on the solution of
(1.7). If W denotes the matrix whose columns are wy, W2,..-, Wy then
it is a standard result that the rows v,,2,...,, of W™ are left
characteristic vectors of A. Since we have v;w; = 1, v;w; = 0, i #/, multi-
plying (3.4) on the left by v; and setting ¢ = 0 in the resulting expression
gives v;x (0) = c;(0). Thus the solution of (3.1) satisfying (3.2) is

x()=>, @1x(0))
exp(it); (3.5)
i=]

Notice that the expression (3.5) depends only upon the initial conditions
and the characteristic roots and vectors of A, and for this reason is referred
to as the spectral form solution (the set {rj }being the spectrum of A).
58 Matrix solution of linear systems

Example 3.1. Find the general solution of

It is routine to obtain the characteristic roots and corresponding

vectors of A as
pi

1 1

Ay =—1, A, =-2,w, = > Wo= ;

= =2
vy = (21), ooedsl,
Equation (3.5) then gives the general solution as

xXy(t) 1 1
= (2x1 (0) +x (0))e* -(x, (0) +x
X(t) =I =)

(3.6)

Exercise 3.1.
Find the general solution of (3.1) subject to (3.2) in each of the follow-
ing cases:

(a) -] -] (b) 1 0 -]
A=
2 —4 Ae at 2 1

2 2 3

Exercise 3.2
If n = 2 and
nO= ve" when x(O)=| 1

It ee “2
x(t)= | e?! when x(0)=]| 1

Lieet 2,
 Matrix solution of linear systems 59

find the general solution of (3.1) subject to (3.2) using the linearity
property described in Chapter 1. Hence find also the matrix A.

3.2. Solution of uncontrolled system: exponential matrix

We now present a different approach to solving (3.1) which avoids the

need to calculate the characteristic vectors of A. It is based on the idea of
generalizing the fact that when n = 1 (so that A is a scalar) the solution
of (3.1) is simply

x(¢)=exp(Af)xo (3.7)
Wedefinetheexponential
matrixby
exp(Af)=1+tA +(t?7/2!)A?+(2/3) Ar tees. (3.8)
The topic of infinite matrix series has already been discussed in Section
2.7 and it follows from (2.69) and (2.70) that since exp (zt) converges
for all finite scalars z and ¢ then the series on the right in (3.8) converges
for all finite ¢ and all x n matrices A having finite elements. It is clear
from (3.8) that exp (0) =/ and that

d [exp (At)]/dt =A exp (Ad),

so that (3.7) does represent the solution of (3.1). However it is important

to realize that properties of the scalar exponential do not necessarily carry
over to the matrix case (see Exercises 3.3, 3.4).
Suppose that the initial condition (3.2) is replaced by the slightly more
general one
X(t) =Xo (3.9)

The solution of (3.1) subject to (3.9) is often written in the control

literature as
x(t) = B(4t)Xo (3.10)
where

@(t,f9)=exp[A(t—
t)] (3.11)
is called the state transition matrix, since it relates the state at any time rf
to the state at any other time fo. It is left as an easy exercise for the reader
60 Matrix solution of linear systems

to verify using (3.11), (3.8) and the result of Exercise 3.4 that ®(¢,to) has
the following properties:

d
Pl fo)=APCF,
fo)
&(t, A =1 (3.12)

P(t, t) =O" (t, to)

P(t, to) =P(t, t,) P(t, fo).

Evaluation of exp (Ar) when all the A; are distinct can be achieved by
Sylvester’s formula (2.71) which gives in this case

n
exp (At)= > Zq exp (Ags) (3.13)
k=1

where

n
Zp = Wy (A —D/O —y)- (3.14)
]=
J#K

Since the Z, in (3.14) are constant matrices depending only on A and

its characteristic roots, the solution in the form given by (3.7) and (3.13)
requires calculation of only the characteristic roots of A. The matrices Z;
have themselves some interesting properties (see Exercises 2.29 and 3.10).
An alternative method of determining exp (Af) was also outlined in
Section 2.7. If r(A) is a polynomial of degree at most n —1 then as in (2.74)

exp (At)=r(A) r.15)

where the n coefficients of r(A) are functions of t obtained from the

solution of the n linear equations (2.75), namely.

exp(Ajt)=r(Aj), i= 1,2,...,m. (3.16)

Matrix solution of linear systems 61

Example 3.2. Using the matrix in Example 3.1, eqn (3.14) gives
2 1
Z, =(4-(-2))/C1 - €-2)) =
>) sd
oI =
pata (1) )/iC2-—C1)) =
2 2

Hence by (3.7) and (3.13)

mL) = (6 Zyxt @ 7525) xo1s

which is easily verified to be the same as (3.6).

Alternatively, since n = 2, the polynomial r(A) can be written
r(A)=7o A +r, and (3.16) then gives
éh=r —1o
€ =r, —Ir6

so that rp =e" —e
7", r, = 2e* —e?". Hence from (3.15)
x(t)= ex,

=(7, 1+79 A)Xo

which again easily reduces to (3.6).
It is interesting to note here that taking the Laplace transform of
(3.1) subject to (3.2) gives

SXotXo=Ax;
orafterrearrangement
x(s)= (sl —
A)" Xo,
sothat
x(th=£' {(sI-A)! xo}. (3.17)

Bycomparison
with(3.7)it thenfollowsthat
£-{ (sl A)"}= exp(Ad), (3.18)
which is a generalization of the well-known result when n = 1.
62 Matrix solution of linear systems

Exercise 3.3.
Prove directly from (3.8) that exp(4 +B) # exp A exp B for two
square matrices A and B unless Aand Bcommute with each other.

Exercise 3.4.
Using the result of the previous exercise, determine the inverse of exp A
and hence deduce that exp A is nonsingular for any square matrix A.

Exercise 3.5.
If A= diag[a,,a2,...,a,] show that
expA 7 diag [exp(@,), exp(a), ste, exp(an)] :

Exercise 3.6.
Consider the equation of simple harmonic motion 2 + w*z = 0. Take as
state variables x; =z and x2 = 2Z/w,and find the transition matrix @(t,0)
using (3.11) and (3.8). (Hint: Find A”).

Exercise 3.7.
Find the general solution of (3.1) and (3.2) using (3.13) or (3.15) for the
two cases where A is given in Exercise 3.1.

Exercise 3.8.
Find the general solution of (3.1) subject to (3.2) when A is the matrix in
Exercise 3.1 (@)by determining (sJ -—A)’ and using eqn (3.17).

Exercise 3.9.
Use the exponential matrix to solve the rabbit-fox environment problem
of Exercise 1.4 subject to the same condition as in Exercise 1.7, namely,
Qa;/a3 =a2/A4.

Exercise 3.10.
Consider the expression for exp(Af) in eqn (3.13). By taking Laplace
transforms of both sides, or otherwise, show that

n
ZK= Ln
k=1
AZ = AZ, ZEA=AgZE,K=1,2,...,0
Matrix solution of linear systems 63

and
Lif; = §5/2;

where 5ij is the Kronecker delta (6, =0,7 47; 5; = 1).

Exercise 3.11.
Write Jz (A) in (2.60) in the form XN+ K, and hence show that
eee Pe wee ik 1)!
] t .

exp{Jx(A)}t=exp(Ar)
op.)

Exercise 3.12.
‘Verify that the solution of the matrix differential equation

a AW(t)+
W(1)B,
W(0)=C,
where A and B are square and constant, is W= exp(4t)C exp(Bt).

3.3. Solution of uncontrolled system: repeated roots

When some of the characteristic roots of A are repeated, evaluation of

exp(Af) is more complicated. We use the formula (2.76) with f(A) =
exp (Af) and the constant matrices Zj; are determined as in the follow-
ing example.
Example 3.3. Find exp(4f) when
4 —4 =22.
64 Matrix solution of linear systems

Since

A 4 WB)

W-A= |. -1 r 11

OF AO Na

it is easy to verify directly from the definition (see Section 2.5) that the
determinantal divisors of WW’ —A are d; = 1,d, =A—2,d3 =(A—-2)? so
by (2.52) the similarity invariants of A are i; = 1, i, =d,/d, =A-2,
i; =d3/d, =(A— 2)’, this last polynomial being the minimum poly-
nomial m(A) of A as stated in Section 2.5. Hence g = 1, Ay = 2, a, =2
in (2.76) which becomes

f(A) =211 FAQ)t+Zp f'Ar) (3.19)

and this expression holds for any function f(A). If we choose for simplic-
ity f(A) = 1 and f(A) =A —2 then this gives in turn Z,, = and Z,) =
A —21. Finally, taking f(A) = exp(Af) in (3.19) produces

exp (At)=Z,1 e7' +Z,, te*"

afer? +(A—21)te?!
LP ye —4t —22t

ed ees iets a4),

0 0 1

Exercise 3.13
Find the general solution of the system

¥1=—2x, +x,
Xa Saad —4x,

using the exponential matrix.

Matrix solution of linear systems 65

Exercise 3.14.
Evaluate exp(Ar) when
g pa ]

am Rae 0 ]

1 “A 2

3.4. Solution of controlled system

We are now in a position to give an expression for the solution of the

constant linear system with control in the state space form of Section 2.5:

X¥=Ax + Bu (3.20)

where B is ann x m matrix, and in practice m <n. After multiplication of

both sides of (3.20) on the left by exp(— Af), the equation can be written

a [exp (—At)x] = exp (-At)Bu, (3.21)

which on integration produces

x(t)=exp
(Ar)
[xo+ffexp
(—Ar)
Bu(nr| ; (3.22)
If the initial condition is x (to) =xo, integration of (3.21) from fo tot
and use of the definition of @in (3.11) gives

x(t)
=®(t,
f)xo+i (to,7)Buca)ar
|. (3.23)
Thus if u(t) is known for t > to, x(t) can be determined by finding the
state transition matrix and carrying out the integration in (3.23).

Example 3.4. Consider the equation of motion

d?z
a
— =ult(t)

of a unit mass moving in a straight line subject to an external force u(t),

66 Matrix solution of linear systems

z(t) being the displacement from some fixed point. In state-space form,
taking x, =z and x2 =Z as state variables, this becomes

d x4 (t) 0 1 xX4(t) 0)
at 5 s u(t)
xX2 (t) 0) 0 X2 (t) 1

= Ax + Bu, say.

SinceherewehaveA*=0,(3.8)reduces
toexp(At)
=/ +At,so(3.22)
gives
x, (¢) Fre x (0) i Jost ip | 6 Ora) ar.
x2(0) 0 1| |x. (0) 0:1). 4°. |Oudanaae
Solving for x, (t) leads to

z(t)=2(0)+ 120) + [" (¢-r)u(r)ar, (3.24)

where Z(0) denotes the initial velocity of the mass.

Exercise 3.15.
Simplify (3.24) when u(f) is equal to a constant for all t > 0, to obtain
the familiar formula for displacement along a straight line under constant
acceleration.

Exercise 3.16.
Determine exp(Af) for the electrically heated oven problem of Example
1.2, assuming that the state equations take the numerical form given in
eqn (2.46) in Example 2.5. Suppose that initially the oven interior and
jacket are at room temperature 7, and that at t = 0 the heating element
is switched on, providing a constant input of magnitude 2 units. Use eqn
(3.22) to find an expression for the subsequent inside temperature of the
oven.

Exercise 3.17.
Consider the system (3.20) with
Matrix solution of linear systems 67

and take u(t) to be the unit step function defined in (1.16). Evaluate
exp(A?) using the power series definition (3.8) and hence show that the
solution of (3.20) subject to x(0) = [1, 0]? is

1+3t+—t oy + eee

dL
mattuloss
ere et

Exercise 3.18.
Consider X= (A + €a)x where A and ware constant n x n matrices and
€ is a parameter. Using (3.22) show that if € is small, then a first approx-
imation for x(t) is X(t)xo where

X(t)
=exp
(Af)
E+€i exp
(—Ar)a
exp
(And
Show also that the next approximation produces an additional term of
order €?.

3.5. Time varying systems

Also of considerable importance in many applications are linear systems

in which the elements of A and B are continuous functions of time for
t > 0. In general it will not be possible to give explicit expressions for
solutions and we shall content ourselves with obtaining some general
properties. We first consider the uncontrolled case

X(t) = A(t)x(t), x(0) =X, (3.25)

and state an existence and uniqueness result:

THEOREM 3.1. If A(¢)is continuous for ¢ > 0 then (3.25) has a unique
solution for t 2 O given by x(t) = X(t)xo9, where X(f) is the unique n x n
matrix satisfying

=A(t)X(t), X(0)=/. (3.26)

68 Matrix solution of linear systems

The proof is straightforward, relying on elementary properties of matrix

norms (see Section 2.7), and can be found in Bellman (1970, p. 167).
We can no longer define a matrix exponential, but there is a result
corresponding to the fact (Exercise 3.4) that exp(A7‘) is nonsingular when
A is constant:
THEOREM 3.2. In Theorem 3.1 the matrix X (¢) is nonsingular.

Proof. Define a matrix Y(¢) as the solution of

dyY_ hy
Wt YA(t), Y(O)=/.

Such a matrix exists and is unique by an argument virtually identical to

that which is used in the proof of the previous theorem. Now

< (YX)
=vr+yXx
=— YAX + YAX

=0

so Y(t)X(t) is equal to a constant matrix, which must be the unit matrix

because of the conditions at t =0. Hence X(f) is nonsingular (and its
inverse is in fact Y(t)).
We can also generalize the idea of the state transition matrix, given in
(3.11) for time invariant systems by writing

P(t, to) = X(t)X~* (to), (3.27)

which exists for all f, tg = O by virtue of the two preceding theorems. It

is then easy to verify by direct differentiation that

x(t)= P(t, to)Xo (3.28)

is the solution of (3.25) with initial condition x (to ) = x9. The expression
(3.28) has the same form as that for the time invariant case, given in (3.10),
although of course the transition matrices involved are quite different.
However it is most interesting that although in general it is not possible
to obtain an analytic expression for the solution of (3.26), and therefore
for ®(t, fo) in (3.27), this latter matrix possesses precisely the same
properties as those for the constant case given in equations (3.12). This
Matrix solution of linear systems 69

is easily verified using (3.26) and (3.27). A further correspondence with

the time invariant case is the following generalization of (3.23):
THEOREM 3.3. The solution of

X(t) = A(t)x(t) + B(t)u(t), (3.29)

subject to the initial condition x (to) =X, is

x(t)=P(t,
fo)K+i P(to,
T)B(7)
u(r)
ar| (3.30)
where® is definedin (3.27).
Proof. Using the standard method of variation of parameters, put
x = X(t)w(t) where X(f) is defined by (3.26). Substitution into (3.29)
produces

axe
qp AAwtx
dw

=AXw + Bu.

Hence
dw _
X(t)Gp=Bu

so
dw =i
EIR — t B
Tees (t) Bu
which on integration gives

w(t) = w(t) + H X7! (t)B(n)u(n)dr.

The desired expression then follows using x9 = X(to )w(to ) and (3.27).
The development in this section shows that some of the results on
linear systems carry over even when the matrix elements are time varying.
This is clearly a useful aspect of the state space approach, since transform
methods can only be applied to equations with constant coefficients.
70 Matrix solution of linear systems

Exercise 3.19.
Show that when n = 2 in (3.26),

= {detX(t)} = trA(t) detX (0),

and hence deduce that X(t) is nonsingular, ¢ > 0.

This method can be generalized for arbitrary n.

Exercise 3.20.
Verify using (3.27) that the properties of the transition matrix listed in
(3.12) do indeed carry over to the time varying case.

Exercise 3.21.
If B(t) = f’ A (r)dr, verify that the solution of (3.25) is x(¢) = exp (B()) xo
provided B(t) and A(t) commute with each other for all > 0.

Exercise 3.22.
Show that if ® is defined by (3.27) then [®~' (t, fo)]" is the transition
matrix for the system Z = —A!(¢)z (the adjoint system).

Exercise 3.23.
A system is self-adjoint if A' (t) =—A(t). Show that (¢, fo) in (3.27) is
then an orthogonal matrix.

Exercise 3.24.
Verify that the solution of the matrix differential equation

ae A(t)W(t)h+W(HAT(D),
W(to)
=C,
is W(t) = P(t, to) CB! (t, to), using the properties of ® which is defined in
(3.27).
Compare with Exercise 3.12.

3.6. Discrete-time systems

A general introduction to discrete-time linear systems was given in

Section 1.2.2, and the method of z-transforms for solving linear difference
equations was outlined. The mathematical model in state variable form is

x(k +1)=A(KT)x(k) +B(kT)u(k) (3.31)

 Matrix solution of linear systems 71

where x(k), u(k) denote the values of the state and control vectors x(KT)
and u(kT) respectively (k =0,1,2,...). The similarity with the contin-
uous-time equation (3.20) is obvious, and the dimensions of A and B are
the same. We now develop matrix methods for solution of (3.31). Consider
first the situation when there is no control and A is a constant matrix, so
(3.31) becomes

x(k + 1)=Ax(k), x(Ko)=Xo. (3.32)

Clearly
x (Koae 1) = Ax (ko)
x (Ko + 2)=Ax(Ko + 1) =A?x (Ko), etc.,

and it is easy to see that the solution of (3.32) is simply

x(k) =A* ox. (3.33)

The matrix A* can be determined in a similar fashion to the development

in Section 3.2 for exp(A?), using either Sylvester’s formula (2.71), or eqn
(2.74) which is based on the Cayley-Hamilton theorem.
Example 3.5. Find A for the matrix A in Example 3.1.
Using (2.74) with n = 2 we have

Ak =roA try

and since the characteristic roots of Aare —1 and —2, eqn (2.75) gives

C1) =-ro tn, 2)" =-2r9 tn.

Solving for 79 and; and substituting produces

Ameer HE 2) 42,

where the Z; are identical to those found in Example 3.2. This emphasises
the fact previously mentioned that the Zz in Sylvester’s formula (2.71)
are dependent only onA, not the particular function f(A) being evaluated.
WhenA has repeated roots the formula (2.76) can again be used.
72 Matrix solution of linear systems

Example 3.6. Find 4!” when

2 4
Al
4 6

The development is similar to that in Example 3.3. It is easy to show that

A has a repeated root —2, and that its similarity invariants are 1 and
det(NJ —A) = (A + 2)*, so as discussed in Section 2.5 its minimum and
characteristic polynomials are identical. Hence g= 1, A, =—2,a, =2
in (2.76) giving

f(A) = Zu fQ1) + Z fd, (3.34)

for any function f(A). Setting f(A) in (3.34) equal to 1 gives Z,, =J, and
f(A) =X +2 gives Z,. =A + 2/. Then taking f(A) = \™ gives

A'}®= (—2p [+-100 (—2)” (4 +29)

= 398 400
= 999
— 400 402

The state transition matrix is now defined by

®(k, ky) = A* *o (3.35)

so that the solution of (3.32) can be written

x (kK)= ®(k, ko)Xo, (3.36)

corresponding to (3.10). It is easy to verify directly from (3.35) that the

following properties hold:

O(k+1,ko) =AP(K, ko)

O(k,k) =I
(3.37)
&(ko,k) =®—! (k, ko), provided A is nonsingular

D(k,ko)* (kK,ky)b(k, ’Ko),Kak, Zko

and these are the discrete analogue of (3.12).
 Matrix solution of linear systems 73

Now return to (3.31) but assume A and B are still time invariant. We
have

x(k) =A[Ax(k —2) +Bu(k —2)] +Bu(k - 1)

= A*x(k —2)+ ABu(k—2) +Bu(k—1)
=A? [Ax(k—3) +Bu(k —3)] +ABu(k—2) +Bu(k - 1)

k-1
=A4k-kox,
+> Ak-i-1Bu(i), (3.38)
i=k,
=&(k, k-1
ko)E +y ®(Ko,
i+1)Bu| (3.39)
i=k
,
_using (3.35) and (3.37). The expression for x(k) in (3.39) corresponds to
that in (3.23) for the continuous-time case.
In the case when A and B have time varying elements the development
is a little easier than for the differential equation system in Section 3.5.
We can write
X(ko + 1)=A(Ko T)X0 ,
X(ko + 2)=A[(ko + 1)T] x(Ko + 1)
=A [(ko + 1)T]A (KoT)Xo0,

and a continuation of this process gives the transition matrix as

k=1
O(k, ky)= I] A(iT). (3.40)
i=k
I=Ko

It is a straightforward exercise to verify that the properties (3.37) still

hold for (3.40). It is also easy to verify that the solution of (3.31) in the
time-varying case is
k-1
x(k) = ®(k, ky) [x0 + > D(Ko,i+ 1)B(iT)u@)| ; (3.41)

i=k,
v4 Matrix solution of linear systems

where ® is given by (3.40), and this is the expression corresponding to

(3.30).
It is clear from comparison of the preceding results with those obtained
earlier in this chapter that there is a close analogy between discrete and
continuous linear systems, as we indicated for the two transform methods
in Section 1.2. We shall encounter further aspects of this inter-relationship
in subsequent chapters.

Exercise 3.25.
Find A’, when A is the matrix in Exercise 3.1(@),by using the expres-
sion (2.71).

Exercise 3.26.
By choosing as state variables x, (k) = X(k), x2 (k) = X(k + 1), write the
scalar difference equation of Example 1.7 in the form (3.32), and find
®(k, 0).

Exercise 3.27.
Find ®(k, 0) for the system having the A matrix given in Example 3.6.

Exercise 3.28.
The Fibonacci numbers are 0, 1, 1, 2, 3,5, 8, 13, ... each number in the
sequence being the sum of the preceding two. These integers arise naturally
in a surprisingly large number of biological and other situations (see for
example Holland (1972) for an elementary account). Let X (k) denote the
(k + 1)th number, and write the difference equation for X in the form
(3.32), as in Exercise 3.26. Find ® (k, 0) and hence show that

X(k)
=E+(5)s
+(5)5°
+(7)5°
ee| jok-1,
where
(; )denotes
theusual
binomial
coefficient.
Exercise 3.29.
By using z-transforms to solve (3.32), deduce using (3.33) that

Z'{z@i-A),1}=A*.
This is the result corresponding to (3.18), and is the matrix generalization
of Example 1.6(b).
Matrix solution of linear systems WS

3.7. Relationships between state space and classical forms

We have assumed so far in this chapter that the linear system equations
are in vector-matrix form. In fact as we have seen in Chapter 1, classical
linear control theory deals with scalar differential or difference equations
of the forms (1.7) and (1.26). Consider a simplified form of (1.7):

2) [email protected] ++kyz=u(t) (3.42)

where as before the superscript denotes differentiation with respect to
t and w(t) is the single control variable. It is easy to write (3.42) in our
matrix form by taking as state variables

Wr=Z,W, =Z2), 21, w,=207), (3.43)

and this can be thought of as a basic step in developing the ‘modern’
theory of linear systems. The w; are called phase variables, and since
Wi =Wi+ 1,1=1,2,...,n—1, it is easy to see that (3.42) and (3.43)
lead to the state space form

w=Cw+du
where
1 0 0
0 1
cC=| - : : ; , : ; ES
1
ohn —kn—-1—kn-2 * ; eras.
Wa5(Wytis fyo. wy

and
Asal ONO
creer:OS |g (3.46)

The matrix C is the companion form previously encountered in (2.32),

and its characteristic polynomial is

det(\l—C) = +ky nh +e + + +k, HK), (3.47)

which has the same coefficients as those in the differential equation

(3.42). As we have seen in eqn (1.10), the polynomial &also arises when
the Laplace transform is applied to the left hand side of (3.42). The
76 Matrix solution of linear systems

more general right hand side involving derivatives of u as displayed in

eqn (1.7) can also be dealt with (see Exercise 3.35; and also Exercise
4,39). We have in fact already applied the transformation (3.43) in
some simple cases, for instance in Example 1.1.
Having seen that (3.42) can be put into matrix form, a natural question
is to ask whether the converse holds: can any linear system in state space
form with a single control variable

x = Ax + bu (3.48)
be put into the classical form (3.42), or its matrix equivalent defined by
(3.43) and (3.44)?
Example 3.7. We saw that in Example 1.2. concerning the electrically
heated oven the equations describing the system arose naturally in the
matrix form (3.48). Suppose that the values of the constants in (1.4) are
as in eqn (2.46) so that eqns (1.2) and (1.3) become

Xy =—2x, + 2ey tuk, =x, —xH

Differentiating the second equation we obtain

xo Ske Xs

= (—2x, + 2x2 tu) -@i1 x)

ope apSpeeaie
Hence
Koitise meee

which has the desired form. We would like to determine when and how
such a procedure can be carried out in general. The answer to our ques-
tions is provided by the following result:

THEOREM 3.4. A system in the form (3.48), where A is ann x n

constant matrix and b a nonzero column n-vector, can be transformed
by a nonsingular transformation w = 7x into the canonical form given by
(3.44), and (3.45), and (3.46) provided
rank:[b;Ab,A2b,...,4"7 bp =n: (3.49)

Conversely, if anonsingular T exists then (3.49) holds.

Proof. Substitution of w = Tx into (3.48) produces
W=TAT ‘w+ Tbu. (3.50)
Matrix solution of linear systems 79

A proof of the first half of the theorem is established by taking as an

explicit expression for the transformation matrix

[!tA

tA?

TS | (3.51)

Lane)
where ¢ is any row n-vector such that 7 is nonsingular, assuming for the
present that at least one suitable ¢ exists. Denote the columns of J’ by
S1,S82,...,5, and consider

tAs, tAs, C C tASy

fas, «tAs, . : 1478)
TAT *=

eas, tA"s, - . tA"s,

Comparison with the identity T7~' =/, namely

tS tS $ * Sy, ]
tAs, tAs, : ° tds, ] @)

tA sas tA"—1,, - : fA4a ts; 1

then readily establishes that the ith row of TAT™! is the (i + 1)th row of
I(@i=1,2,...,n-—1)so TAT’ has the same form as C in (3.45), with
last row given by kj =— tA"sy—;4+1,1=1,2,...,n. For (3.50) to be
identical to (3.44) we must also have 7b = d, and substitution of (3.51)
into this equation gives
ib=0,tAB=0,...,tA"~2b=0,tA" 1h =] (3.52)
or
i (bh,Abate", tb}ea (3.53)
78 Matrix solution of linear systems

which has a unique solution for ¢ in view of (3.49). It only remains to be

shown that T in (3.51) is nonsingular, and this is achieved by showing that
its rows are linearly independent (see Section 2.2). For suppose that
Q,ft+a,tdt:*-+a,t4"-!=0 (3.54)

for some scalars a;. Multiplying (3.54) on the right by b and using (3.52)
givesa, = 0. Similarly, multiplying (3.54) on the right successively by
Ab, A?b,..:,A"~"b gives a, —1 =0,@,—2 =0,..., 0, =O, establish-
ing independence.
To prove the converse of the theorem, if a nonsingular T exists then
rank [b, Ab,...,A"—!b] Stank [Tb, TAB...» eee)

=rank [7b, (TAT) Soe

(TAT awl
= rank {d, Cd,...,C) -—n(s-55)

and it is easy to verify using (3.45) and (3.46) that this last matrix has
the triangular form
ie Ove, 0) ne 6°74

Clerk Os enee 1 eeQe

ehI e > =OOOwm

nN N
QO 0 . date - "ane

L 6) 1
6, 0,
2 s< by
; On —?
On-1

and therefore has full rank.

This completes the proof of the theorem, and 7 can be constructed
using (3.53) and (3.51). However we can also give an explicit expression
for the matrix in the transformation x = T~ ‘ w. Writing

Ue [bp Abi ceo A" 1b), (3.57)

we have seen in the development of (3.55) that TU is equal to the matrix
U in (3.56). This latter matrix has elements given by

il
0:=-> K+) Opeju{OP eo oe ee =,
J=0
=
Matrix solution of linear systems 79

and it is straightforward to verify that its inverse is

kn-—te thy 2. f ky ]
ee) 1, eae ° 1 0

(Ove = c : 3 6 : . . (3.58)

ky 1 je) S)

1 0 (ee) (3
Finally the inverse of 7 is given explicitly by
Toray (3.59)

Notice that the k; in (3.58) are the coefficients in the characteristic

equation (3.47) of C. Thus since A is similar to C it is necessary to cal-
culate the characteristic equation of A in order to use (3.59).

Example 3.8. Find the matrix T and the transformed system when

1 =3 1
A= b=
4 2 1

From(3.52)with¢ = [t,, t2] wehave

ty + fy =0, 2h + 6t =]
whence f; =— 1/8, t2 = 1/8. Hence from (3.51)

=] 1
jp ane
3 5
and
es) 1
:Ric
3 1
A simple calculation gives
0 1
TAP =
—14 WwW
80 Matrix solution of linear systems

Thus the transformed system is

W, =W2, W, =— 14w, +3w, tu

or
2-324 147 =H;

It should be noted that since C=TAT™! then by virtue of the remarks

in Section 2.5, a transformation matrix T will exist only if A is nondero-
gatory. This could also be inferred from the result of Exercise 2.22. In
other words, whatever the vector b the system (3.48) cannot be trans-
formed into (3.44) if A is derogatory. In this case a canonical form for
the system equations can be obtained by choosing T in (3.50) such that
TAT! =J, the Jordan form ofA.
A result similar to Theorem 3.4 can now be obtained for systems
having zero input and a scalar output, so that the system equations are

x=Ax (3.60)
Y=Cx, (3.61)
where A is ann xXn matrix as before, c is a constant row -vector and
y(t) is the output variable.

THEOREM 3.5. Any system.described by (3.60) and (3.61) can be trans-

formed by x = Py, with P nonsingular, into the form

v=Evy, y=f, (3.62)

where
0 0) en
l 0 ~€n—]
0 1 F c :
E= (3.63)

0) 0) : 1 —e

and
f= (0; 0,04 231), (3.64)
 Matrix solution of linear systems 81

provided

cA?
rank =n. (3.65)

seat
Conversely, if a nonsingular P exists then (3.65) holds.
Proof. This follows along similar lines to the proof of Theorem 3.4, so
we do not give all the details. The transformation x = Py when applied to
(3.60) and (3.61) produces »= P"'APy, y = cPy so we first require to show
that P-'AP has the form (3.63). This is achieved by setting

RalroArs .. ae lp (3.66)
assuming that some column n-vector r exists which makes P nonsingular.
If the rows of P"' are denoted by q,,q2,...,4n, then comparison with
_the elements in the identity P-' P=J establishes that P~'! AP has the
desired form (3.63) with last column e =— gy —; + ;A"r,i=1,2,...,0n.
The second requirement, that cP = f, holds because of (3.65), and nonsingu-
larity of Pis shown by proving that its columns are linearly independent.
The proof of the converse is also similar to that in Theorem 3.4. The
matrix P is constructed using (3.66) and the condition cP =f.
Notice that F is also called a companion form matrix because its
characteristic polynomial is

detQU-E)=A2 +e,” 1 +.--- +e, (3.67)

which again is identical to the characteristic polynomial of A,by simi-
larity of Aand E.
The conditions (3.49) and (3.65) are necessary and sufficient for
systems to be transformable into the appropriate canonical forms (3.44)
or (3.62), but we shall see at the beginning of the next chapter that
these rank conditions possess further important significance.
When we turn to the linear difference equation

X(k+n)t+kX(k+n-1)+++++ky,X(k)=u(k) (3.68)
the only modification required is that (3.43) is replaced by
wy,(k) =X(k), w2(k) =X(K+1),° °° , Walk) =X(kK+n-1) (3.69)
82 Matrix solution of linear systems

so w,(k + 1) = Ww;
+ 1(K), and (3.68) is equivalent to
w(k + 1) = Cw(k) + du (k) (3.70)
where C and d are precisely as defined in (3.45) and (3.46). We have in
fact already used a simple case of (3.69) in Exercises 3.26 and 3.28.
Theorems 3.4 and 3.5 now carry over with no changes other than that
(3.44) is replaced by (3.70), and similarly in (3.48), (3.60), and (3.62)
x(k + 1) and v(k + 1) replace derivatives.

Exercise 3.30.
Find the matrix T in Theorem 3.4 and the transformed system for each
of the cases where the matrices are as given in Exercise 3.1 and the cor-
responding vectors b are

an velo
Exercise 3.31.
Two platforms P; and P, are connected to each other and to a fixed
support by springs and dampers as shown, and a control force u(f) is
applied to Py.

|
Spring constant =|
Damping
coefficient =k,

P,

Spring constant = 1
Damping
coefficient =k,

FIG. 3.1
 Matrix solution of linear systems 83

The forces exerted by the dampers are proportional to velocity, and the
springs obey Hooke’s law. Assume that the masses of the platforms and
the other components can be neglected, and that the platforms remain
horizontal. Derive the state equations in the form (3.48), taking the
displacements x; and x, from equilibrium as state variables. Find the
condition to be satisfied by k, and k, for there to exist a nonsingular
transformation of variables which puts the equations into the form
(3.44). If k, = + k, = * determine T and the transformed system.

Exercise 3.32.
Find the matrix P in Theorem 3.5 and the transformed system when

1 p 0
A={3 rail es c= [0,0, 2].

0 4 0

Exercise 3.33.
Prove the converse of Theorem 3.5.

| Exercise 3.34.
Verify that when A is a scalar multiple of the unit matrix then it is
not possible to transform it by a nonsingular transformation into either
(3.45) or (3.63).

Exercise 3.35.
Show that if

Z= [Bn—knBo, Bn-1 —Kn—1Bo. --->B1 —k1Bo] w+Bou

wherew satifies(3.44),thenz satisfies
(3.42)withtherighthandside
replaced
by
Bou+Bu) +... +By_sul)+Baw.
(Compare with eqn 1.7.)

Exercise 3.36.
If V denotes the matrix in (3.65), show that the inverse of the trans-
formation matrix in Theorem 3.5 can be written as P~' =(U)"'V,
where (U) ! is as stated in (3.58) but with the k; replaced by the e;
in(3.67),i=1,2,...,n-1.
4 Linear control systems

The material in the previous chapter on solution of linear systems is of

course applicable to any area where linear equations arise, and no use was
made of the fact that some of the variables represented control terms. We
have now done enough groundwork, however, to be able to concentrate
our attention on some important ideas which are peculiar to control
theory. This chapter can therefore be regarded as a fundamental one in
the development of state space theory, and most of the results have been
obtained within the last fifteen years. We discuss exclusively continuous-
time systems until Section 4.7, where it is pointed out that the basic
theorems carry over to discrete-time systems with only minor modific-
ation.

4.1 Controllability

As was mentioned during the discussion of the examples in Chapter 1, an

essential first step in dealing with many control problems is to determine
whether a desired objective can be achieved by manipulating the chosen
control variables. If not, then either the objective will have to be modified
or control will have to be applied in some different fashion. For example,
economists would dearly like to know whether the rate of inflation can
be controlled by altering taxes, the money supply, bank lending rate, and
so on.
Here we discuss the general property of being able to transfer a system
from any given state to any other by means of a suitable choice of control
functions. Specifically, the linear time varying system S, defined by

x =A(t)x(t) + B(t)u(t) (4.1)

y=C(t)x(O (4.2)

where A isn xXn, BisnxXmand Cisr x n, is said to be completely

controllable (c.c.) if for any to, any initial state x(to) = xo and any given
final state x¢ there exists a finite time ¢; >t) and a control u (ft),
to <t<t,, such that x(¢; )=xp. The qualifying term ‘completely’
Linear control systems 85

implies that the definition holds for all x9 and x¢, and several other
types of controllability can be defined—for example, complete output
controllability requires attainment of arbitrary final output. The control
u(t) is assumed piecewise continuous in the interval fo to f,, that is,
continuous except at a finite number of points in the interval.

Example 4.1. Suppose a wheel is rotating on an axle, the total

moment of inértia being J, and it is required to bring the system to rest
by applying a braking torque u(t). The equation of motion is

JX, =u

wherex, (¢)is the angularvelocity.Integratingweobtain

x1(G)=xi (to) +0)f°ty u(eyar
0
and sincex, (tf,)= 0 wemustchooseu(t) so that

| t u(t) dt =—Jx, (to).

to
Clearly this requirement can be satisfied for any value of x, (to) by
taking
u(t) =—Jx, (to )/(ti —to)
so the system is c.c. However, it is apparent (and intuitively obvious in
this very simple example) that there are an infinite number of suitable
choices for u(t), and to make the control force unique we would have to
add an additional requirement, such as bringing the wheel to rest as
quickly as possible or with minimum expenditure of energy.

Example 4.2. Consider the system described by

Kale a pv s +d4X2 +b,u

X2 = a3X>2.

Clearly by inspection this is not c.c., since u(t) has no influence on

X(t), which is entirely determined by the second equation and x3(fo).
From the expression (3.30) for the solution of (4.1) wehave

xqeid(ty,
te)[xo
+(x©(to,
7)
B(r)u(r)
ar| (4.3)
86 Linear control systems

or, rearranging and using the third property in (3.12),

Oxianeh)
fx.—H(to,
er}+f' &Co,
DBOuC)
ar|
Since ® (f,, fo) is nonsingular it follows that if u(t) transfers x9 to x¢,
it also transfers xy —®(fo, t,) Xpto the origin in the same time interval.
Since x and x¢ are arbitrary, it therefore follows that in the definition
the given final state can be taken to be the null vector without loss of
generality.
For time invariant systems the initial time to in the controllability
definition can be set equal to zero, and a general algebraic criterion can
be derived:
THEOREM 4.1. The constant system

x = Ax + Bu (4.4)

(or the pair [A, B]) is c.c. if and only if the n x nm controllability
matrix
U= (BAB A2B,..5A ae (4.5)

has rank n.
Proof.
Necessity. We suppose (4.4) is c.c. and wish to prove rank U =n.This
is done by assuming rank U <n, which leads to a contradiction. For then
there will exist a constant row n-vector g # 0 such that

qB=0,qAB=0,...,qgA"
1B=0. (4.6)
In the expression (3.22) for the solution of (4.4) subject to x (0) = xq set
t=f,, x(t,) = 0 to obtain, since exp(Afr,) is nonsingular,

xg=f° exp-An)Bu()
ar. (4.7
Now as in (3.15), exp(—Ar) can be expressed as some polynomial r(A )
in A having degree at most n —1, so (4.7) becomes

=e =f t ‘Gol tryA te ++ +r, 1A") Bu(r) dr. (4.8)

0
Multiplying (4.8) on the left by g and using (4.6) gives gx =0. Since
(4.4) is c.c. this must hold for any vector Xp, which implies g = 0, thus
contradicting the assumption that rank U <n.
Linear control systems 87

Sufficiency. We now assume rank U= n, and again using (3.22) we

have
exp(—At, )x(t;) =o + i ty r(A) Bu(r) dr (4.9)
0
where the coefficients of r will be functions of 7. Carrying out the inte-
gration in (4.9) will produce

exp(—Aty
)x(t;)=X +(soB+s,AB+++++s, ,A"-'B). (4.10)
Since rank U=n it follows that for any given xq it will be possible to
choose the s; (and hence by implication u(7)) so that the right hand side
of (4.10) is identically zero, giving x (¢,) = 0. This establishes that (4.4) is
c.c. since the conditions of the definition are satisfied.
COROLLARY 4.1. If rank B = p, the condition in Theorem 4.1 reduces

to
rank [B, AB,...,A"~PB] =n.

Proof.Definethematrix
Peete AR. LA BYR =0,12yacin-
‘If rank Up = rank Up, it follows that all the columns of A&*?B must
be linearly dependent on those of Up. This then implies that all the
columns of A£*?B, A**3B, . . . must also be linearly dependent on
those of Up, so that rank Ug = rank Up,, = rank Ug, =.... Hence
the rank of Ux increases by at least one when k is increased by one, until
the maximum value of rank U; is attained when k = 2. Since rank Up =
rank B = p, and rank U; <n it follows that p + &£ <n, giving <n -—pas
required.

Example 4.3. Consider again Example 1.2 describing an electrically-

heated oven, and let the constants be as in Example 2.5 so that the
equations are

ae! 2 1
= x+ Uu. (4.11)
1 eal 0
The controllability matrix (4.5) is
1 =2
=
0 1
88 Linear control systems

which has rank 2, so the system is c.c. That is, it is possible to control
(in the sense of our definition) the temperatures of both the oven interior
and the jacket by means of the heating coil in the jacket. However, this
does not tell us how the control can be applied to satisfy given objectives.
When m = 1, so that B reduces to a column vector b, the condition on
U in (4.5) becomes identical to that in (3.49). For this reason a system
described by (3.44), (3.45), and (3.46) is said to be in controllable canon-
ical form, since Theorem 3.4 can now be restated as:
THEOREM 4.2. A system in the form (3.48) can be transformed into
the canonical form (3.44) if and only if it is c.c.
Notice the important point that if the system model is taken to be in
the classical form (3.42), then by virtue of Theorem 4.2 this automatic-
ally assumes that the system is c.c. Thus the concept of controllability
could not be appreciated until linear systems in vector—matrix form
were studied, and this is one of the reasons why controllability consider-
ations have only recently been taken into account. Note also that
another remark on Theorem 3.4 can now be interpreted as stating that if
A is derogatory, (3.48) cannot be c.c. for any vector b.
Theorem 4.1 gives a criterion for determining whether a constant
linear system is c.c., but gives no help in determining a control vector
which will carry out a required alteration of states. We now give an ex-
plicit expression for such a vector for both constant and time varying
systems.
THEOREM 4.3. The system S, is c.c. if and only if the m x n symm-
etric controllability matrix

U(to,t1)= [" ©(0,7) B() BT) OT(to,7) dr. (4.12)

where ® is defined in (3.27), is nonsingular. In this case the control

u(t)=—BT(t)®T (to, t) U* (to, t1) [xo Po, tye], (4.13)

defined on to <t<fy, transfers x(t) =Xo to x(t, ) =X.

Proof.

Sufficiency. If U(to, t; ) is assumed nonsingular then the control

defined by (4.13) exists. It is then straightforward to show that S, is c.c.
by verifying that substitution of (4.13) into the solution (3.30) of (4.1) —
Linear control systems 89

does indeed givex(t, ) = x¢, as required (use the property

P(t, to) P (Co,£1)= J).
Necessity. We need to show that if S,; is c.c. then U(t, f; ) is non-
singular. First notice that if a is an arbitrary constant column n-vector
then from (4.12) since Uis symmetric we can construct the quadratic
form

a?Ua=(" 67(r,t) 0(T,to)dr

i)

=fa Wal,
2dr 0

>0, (4.14)

where 0(1, fo) = B! (1) ! (to, rT)a, so that U(to, ¢,) is positive semi-
definite. Suppose there exists some & # 0, such that @'U(to, t;) @=0.
Eqn (4.14) with @= 6 when a = a then implies

fsBl6tee
||, dr=0
which in turn implies, using the properties of norms (see Section 2.7)
that 0 (7, to) =0,t, <7<t,. However, by assumption S, is c.c. so there
exists a control v(t), say, making x(t, ) = 0 if x(to) = @.Hence from (4.3)
it follows that

@=- [- ®(to,r)B(r)v(r)
dr.
a

Therefore

Il [2=@a=- iit v?(7)B™

(7)®1(to,
tt}
r)adr
A fra pe (r) 6 (7, to) dr

=0,
which contradicts the assumption that @#0. Hence U(to, ft;) is positive
definite and is therefore nonsingular.
Since Theorem 4.3 shows that controllability of S, is independent of the
matrix C, we shall often refer to the controllability of the pair [A, B]
90 Linear control systems

instead of that of S,. The control function (4.13) which transfers the
system from (Xo, fo) to (xf, t,) requires calculation of the transition
matrix and the controllability matrix (4.12). This is not too difficult for
constant linear systems, although rather tedious, and moderates the
objection raised at the end of Example 4.3, Of course there will in general
be many other suitable control vectors which achieve the same result,
but the expression (4.13) has an additional interesting property:

THEOREM4.4 If a(t) is any other control taking (xo, fo) to (xg, f;)
then

foae le@ ie ar> t,

[7° eG) i ar
i He
where u(r) is given by (4.13), provided u #u.

Proof. Since both u andu satisfy (4.3), we obtain after subtraction

ty
O= ['* (0, A
7)B(r) [a(r) -u(r)] dr:

Multiplication of this equation on the left by

[xo —
P(to, ty xe)" [U-* (to, t1)]
and useof (4.13)gives

ie u?(7)[u(r)—aa(r)]
dr =. (4.15)
Therefore
[ ty @ay
~ @aar=
~ res [aiea
[" +a A ar
2uTa)
0

pls A
= J, Cate ele] dr,

using (4.15), so

arr, heeesf* [wie2 +e alg]dr

[Waiedr= ~ 112
7 0 ‘. 0
t; lel 2 dr
> [7°

providedu # u, as required.
Linear control systems 91

This result can be interpreted as showing that the control (4.13) is

‘optimal’, in the sense that it minimizes the integral

| eAIlu(r)112
dr=f ie (u;?+++*+Um?)
dr,
Llanes t 0

over the set of all controls which transfer (xo, fo) to (xf, f, ), and this
integral can be thought of as a measure of control ‘energy’ involved.
If the system is not c.c., it would be misleading to call it ‘uncontrol-
lable’, since the implication of our definition is that for a non-c.c. system
there are only certain final states which cannot be achieved by any
choice of control. We can however modify the argument used in Theorem
4.3 to show how such final states can be attained when §, is not c.c.

THEOREM 4.5. If, for a given xf, there exists a constant column n-
vector y such that

U(to, 1) Y=Xo—B!(fo, ty)x¢ (4.16)

then the control

u(t)=—B"(t) ®' (to,ty

transfers the system (4.1) from x (to) =Xo to x(t, ) = xz.

Proof. Substitution of the given u(t) into (3.30) with t = t, produces

x(t;)=@(t,,
to)[x0=|i: P(to,T)
B(1)B'
(1)®(to,
1 7)vdr|
=@(t,, 0) [xo— U(to,t1)¥] by (4.12)
=@(t,,fo) P(to,ti) X¢ by (4.16)
= xf, as required.

When §, is c.c., Uis nonsingular and the expression for u(t) in Theorem
4.5 reduces to (4.13). It can also be shown (Brockett 1970, p.77) that
the converse of this theorem is true, namely that only states x¢ for which
(4.16) holds can be reached.
92 Linear control systems

For time invariant systems a convenient form of Theorem 4.5 is as

follows (for a proof, see Rosenbrock 1970, p.167):
THEOREM 4.6. A given state x9 = x(Q) can be transferred into another
state x¢ provided both x9 and x¢ lie in the subspace spanned by the
columns of U in (4.5).
A further aspect of controllability is now indicated. Let P(t) be an
n X n matrix which is continuous and nonsingular for all f > f9. Then
the system S, obtained from S, by the transformation

x(t) = P(t) x (f) (4.17)

is said to be algebraically equivalent to S,.

THEOREM 4.7. If ®(f, to) is the state transition matrix for S,; then
P(t) ® (t, to) P ' (to) = P(t, fo) is the state transition matrix for S).

Proof. From Section 3.5. we recall that ®(f, to) is the unique matrix
satisfying

B(t, t.) =A B (t, to), ®(fo, to) =,

and is nonsingular. Clearly (to, to) =/, and differentiation of (4.17) and
use of (4.1) gives
dx/dt = Px + Px
= (P + PA)x + PBu
= (P+ PA)P"! x + PBu (4.18)

To establish the theorem we must show that © is the transition matrix for
(4.18), ice.

POE, oP” C0)]= KPC)+POA”! 1 POE, to)P™(o)

and this is left as an easy exercise for the reader.
One of the important properties of the transformation of state variables
in (4.17) is that it preserves controllability:

THEOREM 4.8. If S; is c.c. then so is S2.

Proof. From (4:18) we have that the system matrices for S, are

A(t) = [P() +POA P71

(0),BO =PB (4.19)
 Linear control systems 93

so the controllability matrix (4.12) for S2 is

x es a & &
U(to,t1) = iP P(to, T)B(1)B' (1) ®1(to, T)dt

=[ Po)®60,
PPB OBDP(0)
x(P71(1))T"(to,7)P"(
= P(to) U(to,t1)P' (to) (4.20)

using (4.19) and Theorem 4.7. Thus U is nonsingular since the matrices
U and P(t) in (4.20) each have rank n.
When A, B, and C are time invariant P is also time invariant, and (4.17)
is the usual definition of equivalence transformation in matrix theory.
The following important result on system decomposition then holds:
THEOREM 4.9. When §, is time invariant then if U in (4.5) has rank
n, <n there exists a system algebraically equivalent to S; having the
form

d xO) Ay Az xO) B,
aH = + u (4.21)
x2) Orley x2) 0

y=[C, C,|x

where x“!), x2) have orders n; andn—n, respectivelyand [A,, By]

1SCis
We shall postpone the proof of this until a later section (see the proof
of Theorem 4.23). However, comparing with Example 4.2, it is clear that
in (4.21) the vector x) is completely unaffected by u. Thus the state
space has been divided into two parts, one being c.c. and the other
uncontrollable,

Exercise 4.1.
Verify that the system in Example 1.1 is c.c.

Exercise 4.2.
_ Re-interpret the result of Exercise 3.31 in the light of Theorem 4.2.
94 Linear control systems

Exercise 4.3.
Two pendulums are connected by a spring as shown.

Spring constant =k

mg mg

FIG. 4.1

The masses of the rods, which have unit lengths, can be neglected. Equal
and opposite control forces u(t) are applied to the particles, which have
mass m. Write down the equations of motion for small oscillations so that
6? and higher powers can be neglected. Take x; =0, +6,xX2 =O; —02,
X3 =X ,,Xq4=X, as state variables and hence deduce that the system is
not c.c.
Notice that if the control forces were not equal the system would be

Cc

Exercise 4.4. f
Consider again the problem of two connected masses moving on a smooth
horizontal plane, described in Exercise 1.2. Suppose now that in addition
to the springs the masses are joined by dampers as shown, the forces as
usual being proportional to velocity with coefficients d, and d>.
 Linear control systems 95

Take the same state variables as before and derive the state equations in
the form (4.4). If m, =m, =1,d,; =d, =1andk, = determine under
what conditions the system is c.c.

Exercise 4.5.
A platform is supported by springs and dampers as shown, it being assumed
that the forces they produce act at the end points P and Q, and that x,
and x2 are the displacements of these points from equilibrium.

4u

ing constant = 1 Spring constant = 2

mping coefficient = | Damping coefficient = |

FIG. 4.3

This could be thought of as a simple representation of a vehicle suspension

system. The forces exerted by the dampers are proportional to velocity
and the springs obey Hooke’s law. Assume that the mass of the platform
can be neglected, so that the spring motions can be regarded as indepen-
dent. If a control force 4u(t) is applied one quarter the way along from
one end, show that the system equations are

Xy ull 0 X41 1

X4 On? X2 8

Verify that the system is c.c. If the ends of the platform are each given
an initial displacement of 10 units, find using (4.13) a control function
which returns the system to equilibrium at ¢ = 1.

Exercise 4.6.
If in the system (3.48) A has repeated characteristic roots but neverthe-
- less the Jordan form of A is a diagonal matrix, show that (3 48) is not c.c.
96 Linear control systems

Exercise 4.7.
If the pair [A, B] is c.c. show that [A + BK, B] is also c.c. for any matrix
K having appropriate dimensions. (Hint: do not use Theorem 4.1).

Exercise 4.8.
Using the matrix A in Exercise 3.1(a), find for what vectors b the system
(3.48) is not c.c. Using Laplace transforms investigate what happens to
X(s) in these cases.

Exercise 4.9.
By taking A to be the companion matrix

0 0 math

0 Sols

0 mln

assumed nonsingular, of a third degree polynomial a(A), and

b = [0,—k, 1]7, deduce using (4.5) that a(A) is divisible by \—kif
and only if the pair [A, b] is notc.c.

Exercise 4.10.
Prove that U(to, t, ) defined in (4.12) satisfies the matrix differential
equation

U(t,t1) = A(0)U(t,
1) +U(t,t))AT(1)-B(OBT
(1),Uy, t1)=.

Exercise 4.11.
In the preceding exercise let A and B be time invariant, and put
W(t, t, ) =U(t, t; )—Up where the constant matrix Up satisfies

AU + UpA! = BB".

Write down the solution of the resulting differential equation for W

using the result in Exercise 3.24, and hence show that

U(t,t,) =Up—
exp[A(t—11)]
Uoexp[AT(t-1t,)].
 Linear control systems 97

Exercise 4.12.
If

] 2 ma 0)

x=) 0 1 Oe ae Ou

1 —4 3 1

find a basis for the set of vectors which can be reached from the origin.

Exercise 4.13.
Show that the result in Theorem 4.4 still holds if the measure of control
energy is

[Put
@R@uGyar
0

where R(f) is a positive definite symmetric matrix, and (4.13) is replaced

by
u(t)=—R~*(t)B" (1)8"(to, 1)Ui' (to, t1) [xo—P(to, ts)x4]
' where

O86 i (ty, T)B(t)R~!(1)B*(r)@1 (to,

7)dr.

4.2. Observability.

Closely linked to the idea of controllability is that of observability, which

in general terms means that it is possible to determine the state of a
system by measuring only the output. For example, a political party
which wins most votes in a national election usually claims that its policies
are supported by a majority of the electorate; but can the state of opinion
of the voters on a particular point at issue be determined from the overall
election result?
Our precise definition is that the system S, described by (4.1) and
(4.2) is completely observable (c.o.) if for any tp and any initial state
X(to) =Xo there exists a finite time ft; >t) such that knowledge of u(t)
and y(t) for tp <t<f, suffices to determine xo. There is in fact no loss
in generality in assuming u(t) is identically zero throughout the interval.
98 Linear control systems

Example 4.4. Consider the system described by

X, =a,xX, +byu

X_ =d,xX2, +b2u

yHx,.

The first equation shows that x;(t) (= y(f)) is completely determined by

u(t) and x, (to). Thus it is impossible to determine x (to ) by measuring
the output, so the system is not c.o.
Notice that application of Theorem 4.1 easily shows that the system
is c.c. provided a; #a,,b, #0,b, #0.
The result corresponding to the general controllability criterion of
Theorem 4.3 is:
THEOREM 4.10. The system S, is c.o. if and only if the symmetric
observability matrix

Vitosts)=
fOr, to)CTCO)®(r,
fo)dr, (4.22
where ® is defined in (3.27), is nonsingular.

Proof.

Sufficiency, Assuming u(t) = 0, fo <t<t,, we have from (3.28)

Y(t) = C(P(E, to) xo. (4.23)

Multiply (4.23) on the left by ®1(¢, t9)C1(r) and integrate to obtain

A aT T
J 87G, t0)C Op dt = V(t0, 110,

so that if V(to, ¢; ) is nonsingular the initial state is

Xo =V—(fost) [2° 9TG, tC Oy @ar, ~~ (4.24)

SOS 1SC10,,

Necessity. We now assume S, is c.o. and prove that V(to, t; ) is non-

singular. The argument follows along very similar lines to those used in
Linear control systems 99

the proof of the second part of Theorem 4.3. First, if a is an arbitrary

column n-vector,

eo =i if[C(r)
®(r,toa]T[C(r)®(r,
0
toJor]
dr
20

so V(to, t; ) is positive semidefinite. Next, suppose there exists an @

such that @' Va = 0. It then follows as before that C(r)®(z, ty) a=0,.
to <T<t,. From (4.23) this implies that when x9 = a@the output is
identically zero throughout the time interval, so that x9 cannot be
determined in this case from a knowledge of y. This contradicts the
assumption that S,; is c.o. Hence V(fo, ¢; ) is positive definite, and there-
fore nonsingular.
Notice that when §, is c.o., (4.24) provides an explicit expression for
the initial state in terms of the output and the transition and observability
matrices.
Since the observability of S, is independent of B, we shall refer to the
_observability of the pair [A, C]. We have seen in Exercise 3.22 that
¢' (to, ¢) is the transition matrix for the system having state matrix
—A'(t). By comparing eqns (4.12) and (4.22) we can therefore see that
the controllability matrix (4.12) is identical to the observability matrix
(4.22) associated with the pair [—A!(d), B*(t)]. Conversely, the observ-
ability matrix (4.22) is identical to the matrix (4.12) associated with the
pair [—A!(t), C'(2)]. We have thus established:

THEOREM 4.11. (Duality) The system S, defined in (4.1) and (4.2)

is c.c. if and only if the dual system

X(t) =- Al (t)x(t) + C1(Q)u(t)

y(t) = BT(t)x(t) wee
4.25

is c.o.; and conversely.

If the matrices have complex elements, transpose in (4.25) is replaced
by conjugate transpose. This duality theorem is extremely useful, since
it enables us to deduce immediately from a controllability result the
corresponding one on observability (and conversely). Forexample, to
obtain the observability criterion for the time invariant case, we simply
100 Linear control systems

apply Theorem 4.1 to (4.25) to obtain (after transposition of (4.5),

which does not affect its rank):
THEOREM 4.12. When A, B, and C are time-invariant the system S,
is c.o. if and only if the observability matrix
—

v=| - (4.26)

Com
has rank n.

Example 4.5. Consider again Example 1.2 describing an electrically

heated oven, and let the state equations be as given in (4.11). Suppose
that as stated in Example 2.5 it-is only possible to measure y = x,, the
jacket (excess) temperature. Then the observability matrix in (4.26) is
1 0

2 2

which has rank 2. Thus the system is c.o., that is, it is possible to deter-
mine the temperature of the oven interior from a knowledge ofu(t) and
x,(t).
It is worth noting that in the scalar output case (i.e. = 1)ifu=0
y(t) is known in the form

n
ESviexp(Ajt),
assuming that
i=]
all the characteristic roots A; of A are distinct,
then x9
can be obtained more easily than by using (4.24). For suppose that
to = 0 and consider the solution of x = Ax in the spectral form (3.5),
namely

x(0)= >> (xO) exp Cit) wi.

i=l
 Linear control systems 101

We have

y= (vjx(0)) (ew;) exp (Aiz),

1=
i

and equating coefficients of the exponential terms gives

pint) seyy/(cw;), 2 1,22,

This represents n linear equations for the n unknown components of

x(0) in terms of the y; and left and right characteristic vectors of A.
Notice that because [A,c] is c.o., no term cw; is zero (see Exercise 4.20)
and a unique solution for x (0) therefore exists since the v; are linearly
independent.
For the scalar output case we also see that V in (4.26) is identical to
the matrix in (3.65), so Theorem 3.5 can be restated as:

THEOREM 4.13. A system in the form (3.60) and (3.61) can be trans-
formed into the canonical form (3.62) if and only if it is c.o.
For this reason a system described by (3.62), (3.63), and (3.64) is
‘often said to be in observable canonical form.
Again by duality the result corresponding to Theorem 4.9 is:
THEOREM 4.14. When S, is time invariant then if V in (4.26) has
rank n, <n there exists a system algebraically equivalent to S, having
the form
aes Aas 00 x1) B
eat = 4 u (4.27)
Re (2) A, A3 x) B,

PC R™
where x‘!),x?) have orders n; andn—n, respectively and [4;, C;]
is C.0.
As in Example 4.4, we see that in (4.27) y is completely unaffected
by x?) which is therefore said to be unobservable, and the state space
has been divided into two parts with respect to observability.
We close this section with a decomposition result which effectively
combines together Theorems 4.9 and 4.14 to show that a linear time-
invariant system can be split up into four mutually exclusive parts,
_ respectively (1) c.c. but unobservable (2) c.c. and c.o. (3) uncontrollable
102 Linear control systems

and unobservable (4) c.o. but uncontrollable (see Zadeh and Desoer,
19633 p27505);
THEOREM 4.15. When S, is time-invariant it is algebraically equivalent
to
x) All Ate Ais. gai xi B}
d x2) a 0 A? OQ Aw x2) B?
aft ,@)| 19 96 Byala eae %
x4) OiAiulOs
oof:edBebbaa
y= Cx) + C44

where the superscripts refer to the stated classifications.

Exercise 4.14.
If the output in the spring-mass system described in Exercise 1.2 is taken
to be y =x, is the system c.0.?

Exercise 4.15.
Consider again the rabbit—fox environment problem described in
Exercise 1.4. If it is possible to count only the total number of animals,
can the individual numbers of rabbits and foxes be determined?

Exercise 4.16.

In the circuit shown in Fig. 4.4 take the state variables to be the
voltage x, across the capacitor and the current x, through the inductor.
The input voltage u is the control variable and the current y can be
regarded as the output. Derive the equations describing the system, and
show that if R,;R, C=L it is not c.c. and not c.o.

FIG. 44
 Linear control systems 103

Exercise 4.17.
Using the matrix in Exercise 3.1 (@),find for what vectors c the pair
[A, c] is not c.o. By using the solution of (3.1) obtained, investigate
what happens to y(t) in these cases.

Exercise 4.18.
Bearing in mind the result of the preceding exercise, does it follow that
if some of the exponential terms present in x(¢) are missing from y(t)
then the system is not c.o.?

Exercise 4.19.
For the system xX= Ax with A matrix as in Exercise 3.1 (@’)and y = [1,2]x
find x (0) if y(t) =—20 exp (—3r)+ 21 exp (-22).

Exercise 4.20.
Suppose that the pair [A, c] is c.o., where c is a row n-vector, and that
all the characteristic roots of A are distinct. If the right characteristic
vectors of A are W,,..., W, Show that cw; #0,i=1,2,...,n.

"Exercise4.21.
If in the system described by (3.60) and (3.61) A has repeated character-
istic roots, but the Jordan form of A is a diagonal matrix, show that the
system is not c.o. (compare with Exercise 4.6).

Exercise 4.22.
Prove that V(to, t, ) defined in (4.22) satisfies the differential equation
V(t, 1) =-AT(*)V(t,t,) V(t, t1AQ) CT(DCO, Vi, t1) = 0.
Compare with Exercise 4.10.

Exercise 4.23.
Prove directly the necessity of the condition in Theorem 4.12. (Hint:
assume rank V <n and obtain a contradiction).

Exercise 4.24.

Show that if the initial state of a constant system which is not c.o.
satisfies Vx(0) = 0, where V is defined in (4.26), then y(t) = 0 for all
- t2>0. (Hint: it can be assumed that the system is in the form (4.27).)
104 Linear control systems

Exercise 4.25.
Obtain the result corresponding to Theorem 4.8 by expressing the observ-
ability matrix V(t, t,) in terms of V(to, f; ).

+4.3. Controllability and polynomials

We now discuss some interesting relationships involving controllability

and polynomials for time invariant systems. By duality, corresponding
results can also be obtained in terms of observability. Some of the ideas
have already been hinted at in Exercises 4.8 and 4.17.
Let A be in the companion form (3.63), ie.

0 0 : : Ky,
] 0 . : “Kye
A= : ; : ; 6 (4.28)

0 0 : 1 —ky

then the basic Theorem 4.1 on constant systems can be stated in the
following form:

THEOREM4.16. The pair [A, B] is c.c. if and only if the set of poly-
nomials
k(x) =det (MU
-A)=N? +k” 1 +e +e thy
and
Pj(A)=by? +byi id” 2 +92 ++b,,,15
where B = [b;;], is relatively prime. Furthermore, the columns of the
controllability matrix in (4.5) are a re-ordering of the columns of

[p1 (A), p2(A), --->Pm (A)] |

where .
pi(A) = by; An-liwee Da

Proof. For the general case this relies on a theorem on polynomial

matrices and is too lengthy to give here, so we refer the reader to Barnett
 Linear control systems 105

(1971, p.41). When m = 1, the argument is straightforward and the steps

have been indicated in Exercises 2.11 and 2.12. First, if the roots of
k (A) (which are the characteristic roots of A) are \y, A2,-.-, Ay then

detp1(A)=pi (Ai)P12) ***PiAn)

so that p, (A) is singular if and only if p;(A;)= 0 for at least one j; in
other words, if and only if p, (A) and k(A) have a common factor. To
complete the proof, let e; and7; denote the ith columns of J and p;(A)
respectively. Using (4.28) it is easy to show that

Tad [bi1, 521, - oa soni] =B.

Also, e; =Ae;-; and p,(A) and A commute with each other, so

1; = Py(A)e; = pi (4 )Ae; -1 = Api (A ej -1 =Ari-1-

Hence p;(A) = [B, AB,..., A”! B] which is the controllability matrix

(4.5).
Theorem 4.16 can be extended to give a simple method of calculating
‘the greatest common divisor of a(A) and the p; (A) if [A, B] is not c.c.
(See Barnett 1971, p.43). In particular, the degree of the g.c.d. is equal
to the rank defect of the controllability matrix (4.5). This is a special
case of another result (Rosenbrock 1970, p.85) which states that for any
n X n matrix A, the rank defect of (4.5) is equal to the degree of the nth
determinantal divisor (see Section 2.5) of the n x (n +m)polynomial
_ matrix [AJ—A, B]. When m= 1 we can relate this to the Laplace trans-
form representation of the solution of (4.4). This was obtained in eqn.
(2.41) as
X(s)=(s1—A)* Bu(s). (4.29)

If we write
qi(s)

Sy) Sty
(si —=A)| B aE@ ed ,

qn(s)
106 Linear control systems

then
qilk

[sl AJB) = [sk=A] |S U,4) 4 : (4.30)

Anlk
The n x n minors of the matrix product on the right in (4.30) are just
det(sJ —-A)=k(s), q1 (S),...5@n (S) so that these polynomials are
relatively prime if and only if the pair [A, B] is c.c. In other words, for
systems with a scalar input we have the interesting interpretation that
complete controllability is equivalent to there being no common factors
between numerators and denominator in the vector transfer function
(sl —A)" B in (4.29).

Example 4.6. In Exercise 4.8 with

the controllability matrix U in (4.5) has determinant

det[b, Ab] = (2b; —b2) (61 —5»).

Also
1 s+4 = b,
heyy2
Ce ae Leet
Tan? Pa
s? +55+6

(s +4)b, —b2
1

oeCaisbak’
| Geep
The pair [A, b] is not c.c. if bz = 2b, or by = b,, in which cases there
are common factors s + 2 and s + 3 respectively in the vector transter
function.

Exercise
4.26.UsetheCayley—Hamilton
theorem
toshowthatif
D(A)=bod"+byNt te ee+d,
 Linear control systems 107

then
DCD)1640. Tare Gl
where
Aisgiven
by(4.28)and
B=[bn—Dokn, bn1 —bokn-1,-..,51—bok,]".
Exercise 4.27.
Using the notation of Exercises 2.12 and 2.13 deduce that the pair [C, r]
is c.o. if and only if k(A) and P(A) are relatively prime.

Exercise 4.28.
If k (A) denotes the derivative of k (A) with respect to A, and A is defined
in (4.28), deduce that k’ (A) is nonsingular if and only if k (A) has no
repeated zeros.

4.4. Linear feedback

In Chapter 1 we discussed the importance of the idea of feedback,

mainly in the context of single-input single-output systems. We now
study feedback using state space representations, and our first result
provides an extremely important link with controllability. Consider the
time invariant system
= Ax + Bu. (4.31)
Suppose that we apply linear (state) feedback, that is each control
variable is a linear combination of the state variables, so that

u= Kx (4,32)
where K is a constant m x n feedback (or gain) matrix. The resulting
closed loop system obtained by substituting (4.32) into (4.31) is
%=(A+BK)x. (4.33)
Assume that as is usual in applications A and B are real, and let A, =
{6,, D5 eee On} be an arbitrary set of n complex numbers such that
any which are not purely real occur in conjugate pairs.
THEOREM 4.17. If the pair [A, B] is c.c. then there exists a real matrix
K such that the characteristic roots of A + BK are the set Ay.
In view of the development in Section 3.2 it follows that the solution
- of (4.33) depends on the characteristic roots of A+ BK, so provided
108 Linear control systems

(4.31) is c.c. the theorem tells us that using linear feedback it is possible
to exert a considerable influence on the time behaviour of the closed loop
system by suitably choosing the 6;. A further comment on this point will
be made at the end of the section. However, it should be noted that it is
not, in general, possible to choose K in order to give A + BK an arbitrary
Jordan form..
The theorem gives us a new and most interesting insight into the
meaning of linear feedback in state space terms. For example, if a single-
input single-output system is in the classical form (3.42), then application
of Theorem 4.17 to the equivalent matrix representation (3.44) shows
that in general u will be a linear combination of all the phase variables,
that is of z,z“),..., z—), not just a multiple of the output z alone.
If (4.31) is not c.c. then it follows from Theorem 4.9 that only n, of the
characteristic roots of the closed loop system can be arbitrarily chosen
as members of a set Ay, , since A3 in (4.21) is unaffected by the feed-
back.
The theorem is very recent, the first general proof being given by
Wonham in 1967. We shall present a somewhat different argument which
is nevertheless rather lengthy, so we first consider the case when there is
only a single control variable, and give a proof which includes a method
of constructing K.

Proof when m = 1. Since (4.31) is c.c. it follows from Theorems 4.2 and
3.4 that there exists a nonsingular transformation w = 7x such that (4.31)
is transformed into w = Cw + du where C and d are given by (3.45) and
(3.46) respectively. The feedback control u = kw where

KilKnikn=ieetae kal
produces the closed loop matrix C + dk which has the same companion
form as C but with last row —[Yn, Yn—1,---, 71], where

Ke hpsiypet = 1, 2) ers gite (4.34)

Since
C+dk=T(A+bxT)T"! (4.35)

it follows that the desired matrix is K= KT, the x; being given by (4.34).
In this equation the k; are the coefficients in the characteristic poly-
nomial of A, i.e.

det (MW
-A)=A" +ky tl te + +tky, (4.36)
Linear control systems 109

and the 7; are obtained by equating coefficients of in

Beat n
nctn LT0-81). (4.37)
The realness of K follows from that
l=of x and T.
Notice that (4.35) shows that in the single control variable case the
closed loop matrix A + b«T is similar to acompanion form matrix, and
so in nonderogatory (see Section 2.5). This provides an illustration of
our earlier remark that the closed loop matrix cannot be made to take
an arbitrary Jordan form. Notice also that if in the preceding argument
the canonical form involving C, d is replaced by the observable canonical
form in Theorem 3.5, we obtain:
COROLLARY 4.2. If the pair [A, c] is c.o., where c is a real row
n-vector, then there exists a real column n-vector & such that the charac-
teristic roots of A + &c are the set A,.
This result can also be deduced from Theorem 4.17 using the duality
Theorem 4.11.

Example 4.7. Consider the matrices A and b given in Example 3.8.

The characteristic equation of A is

N?- 3+ 14=0
which has roots 3/2 + i\/(47)/2. Suppose we wish the characteristic
roots of the closed loop system to be —1 and —2, so that the corres-
ponding polynomial is \* + 3d + 2. From (4.34), (4.36), and (4.37)
we have

hg Jeo

Ky,=k. -Y¥2=14-2=12.

Hence using the matrix T in Example 3.8,

K=kT
= % [12,-6] [-1
posters 1

=— [15/4, 9/4].
110 Linear control systems

It is easy to verify that

a
al
A+bK=% eee
|
El
does have the desired roots.
Before proving Theorem 4.17 for m > 1 we need a preliminary result.

LEMMA 4.1. If the pair [A, B] is c.c. and the columns of B, assumed
non-zero, are b,,b2,..., 5), then there exist real matrices K;,i= 1, 2,
..- ,@t, such that the pairs |A + BK;, b;) are c.c-

Proof. For convenience consider the case 7 = 1. Since the matrix U in

(4.5) has full rank, it is possible to select from its columns at least one
set of n vectors which are linearly independent (see Section 2.2). Define
ann X n matrix M by choosing such a set as follows:

M=[b,,4bi,...;4% ~01,0,,405,....42 2
where r; is the smallest integer such that A”! ; is linearly dependent on
all the preceding vectors, the process continuing until 7 columns of U
are taken. Define an m x n matrix N having its 7; th column equal to
€2, the second column of Jy, its (ry; +72 )th column equal to e3, its
(r; +72 +73 )th column equal to eg and so on, all its other columns
being zero. It is then not difficult to show that the desired matrix in
the statement of the Lemma is K, = NM~'. This is established by comp-
aring terms on both sides of the expression K,M = WN,from which it
follows that

b,, (4+BK,)b,,(A+BK,)°b;i,...,(4+BKiy”Oy
are linearly independent, so by Theorem 4.1 the pair [A +BK,, b, ]
ISiCuCe
It is worth remarking that the matrix M defined above can be used
as a transformation giving a canonical form for (4.31) with m > 1 (see
Chen 1970, Chapter 7 where fuller details of the proof of Lemma 4.1
can also be found).

Proof of Theorem 4.17form > 1, Let K, be the matrix in the proof of

Lemma 4.1 and define an m x n matrix K’ having as its first row some
vector k, and all its other rows zero. Then the control

u=(K, +K')x (4.38)

‘
Linear control systems Ha

leads to the closed loop system

X%=(A+BK,)x +BK'x

=(A + BK, )x + bykx,

where b, is the first column ofB. Since the pair [A + BK, ,},] isc.c.
it now follows from the proof of the theorem when m = 1 that k can
be chosen so that the characteristic roots of A +BK, + b,k are the set
A, so the desired feedback control is indeed (4.38).
If y = Cx is the output vector in (4.2) then again by duality we can
immediately deduce from Theorem 4.17:

COROLLARY 4.3. If the pair [A, C] is c.o. then there exists a real
matrix L such that the characteristic roots of A + LC are the set Ay.
It is interesting to relate the feedback theorem to the transfer function
for the scalar input and output case defined in (1.14), namely
Base PRS is Phe kB
g(s)2
st +k? tee +k,
It is easy to verify (see Exercise 4.39) that
g(s)=r(sI-—C)"'d (4.39)

where C is the companion form matrix in (3.45), d the column vector

in (3.46) and
R=2AB7, ee Biusbos 0; te 018

so g(s) is the transfer function for the state space system

x=Cx+du,y=rx.
As in the proof of Theorem 4.17 when m = 1, linear state feedback
affects only the last row of C, which contains the coefficients of the
denominator of the transfer function. In other words, linear feedback
enables the poles of the closed loop transfer function to be arbitrarily
located in the complex s-plane, but leaves the zeros unaltered. This fact
does not hold when m > 1, when in general the zeros are also affected.
We now establish a relationship between the characteristic equations
of closed loop and open loop linear systems with multivariable control.
As in (2.44) the open loop system is described by
y(s) = C(sl —A)7'Bu(s)
= G(s)u(s)
112 Linear control systems

where G(s) is the r x m open loop transfer function matrix. The closed
loop system considered is of the form shown in Fig. 4.5.

FIG. 4.5

This is the multivariable version of Fig. 1.8 and the equation correspond-
ing to (1.15) is

(I, + G(s)H(s))y(s) = G(s)u(s). (4.40)

If it is assumed that r = m, so G(s) and H(s) are square, then as in Exercise
1.5 a unity feedback loop can be used.

In Fig. 4.6 we have

(Um+ G(s))y(s) = G(s)u(s)

where by analogy with the scalar case in Section 1.2.1, J,;, + G (s) is termed
the return difference matrix. In state space terms the control is

u(t)=—y(t)
=—Cx(t)
so that the closed loop system is
x =(A-BOC)x.
An interesting relationship between the two systems is the following:
THEOREM 4.18. The ratio of the characteristic equations of the closed
Linear control systems 113

loop and open loop systems is equal to the determinant of the return
difference matrix, i.e.
eee BO) _ a
det (sI, -A +B [Im + G(s)] (4.41)

= det J, + CGI-A) *B]:

Proof.
det(s!—A+BC)=det[(s!—
A){I+(sJ]-—
A)"'BC}]
=det(sJ —A)det [J+ (sJ—A)7'BC]
=det(sJ—A)det [J+C(sJ—A)"'B],
= det (s/— A) det [J+ G(s)],
the penultimate step following from the fact that for any two matrices
X and Y having dimensions n x m and m x n respectively then
det U, +XY) = det Um + YX) (4.42)
(see Exercise 2.3).
The proof of Theorem 4.17 for m > 1 given earlier does not provide
a practical way of constructing the feedback matrix K in (4.32). However,
we can use (4.42) to derive a simple method due to MacFarlane:
THEOREM 4.19. Let all the characteristic roots \; of A be distinct and
let W=[w1, W2,..., Wy], where w; is a characteristic vector correspond-
ing to A;. With linear feedback u = —Kx, suppose that the characteristic
roots of A and of A —BK are ordered so that those of A—BK are to be
Mi, U25-++,Mp,Apt+1,---5An (PpSn). Then provided [A,B] isc.c.a
suitable matrix is
K =few (4.43)
whereWconsistsof thefirst prowsof W~!,and
&= [01/B1,&2/B2,.-.,%/Bp), (4.44)
a =T1 Ov-4)/TT Ov-¥).2 > 1, (4.45)
a ii
6 = [B1,B2,--.,8p]" = WBF, (4.46)
f being any column m-vector such that all 6; # 0.
114 Linear control systems

Proof. Since only the first p characteristic roots of the closed and open
loop systems differ, eqn (4.41) gives

det[1+K(sl—A)"'B]
=II (s—ui)/ia (s=)j). (4.47)
i=1 i=l
Furthermoresince A= WAgW~',whereAg = diag [\1,..., An], (see
(2.31))wehave

det [1+ K(sl —A)~!B]= det [I +(s!—A)"!BK]

det [I+ W(s1—Aq)! W7!BfgW]
1 +(gW) [W(sI—A)"1Wo Bf] (4.48)

where the first and last steps follow from (4.42). Using the definition of
W the right side of (4.48) becomes

1+g(s] -—Aq)”!WBf
=1+g(s!—Ag) 1B (4.49)

whereAq = diag [\y,A2,..., Ap] . Henceby (4.47) and (4.49) wehave

1+ g(6l ~Aa) 18-11 (5 ui)/(s -) (4.50)

Dp
=1+ >) a;/(s —4), (4.51)
i=
where the a; given in (4.45) are obtained by expansion of (4.50) into
the partial fractions (4.51) (note that when p = 1, a, =A, —4y). Clearly

(s—Aq) 1 = diag[1/(s= 1), ---, 1/(s - Ap

so the expression (4.44) for the elements of g follows from (4.51). Since
the system is c.c., WB has no zero rows (see Exercise 4.37) and this
ensures the existence of a suitable /.
The method can be extended to the case when A has repeated roots.
Note that the relative simplicity of the expression for K is due to the
fact that the matrix on the right in (4.43) has rank one.
Linear control systems 115

Example 4.8. Consider the matrix A given in Example 3.1, and let
b= [2,1] 1. Wehave \y =— 1, A. =—2 and

1 1 2 1
= ; whe :
“all Ry eee) alee el

Suppose that uy, =—3,u, =—4,so W=W~!. From (4.45), a, =6,

a2 =— 2, and (4.46) gives

By 2s 1 2 Shi
im i= :
Bo am| =] 1 i 3f;

Hence we can take f; = 1, which results in

&= 16/52/31

_using (4.44). Finally, from (4.43), the desired feedback matrix is

K =1.[6/5, 2/3] Ww"!

= [26/15 58/15) (4.52)

The reader should verify that the characteristic roots of A— bK are as

specified.

Example 4.9. Let b in the preceding example be replaced by

everything else remaining unaltered. From (4.46) we now obtain

ia | Sit*
B, Seely
=fr
so that f; = 1,f, =0 gives K with first row identical to (4.52) and second
 116 Linear control systems

row zero. However f; = 1, f, =— 1 gives B,; =3, 62 =—2so that g, = 2,

g, = 1, and from (4.43) we now have

K= (2. 1h
-1

3 1

-3 -1

and again the reader can verify that A —BK has the roots —3 and —4.
Methods for controlling systems by prespecifying the closed loop
system characteristic roots are often called modal control techniques,
the name mode being given to the terms exp(A;t)w; in the expression
(3.5) for the solution of a linear system. Much effort has been put into
this approach (Porter and Crossley 1972) but there are anumber of
disadvantages which arise in practical applications. For example, there
is no means of directly correlating the time behaviour of the closed
loop system with the specified values of the poles since, as we have men-
tioned, there is no control over the closed loop zeros (see MacFarlane
(1972) for further discussion).

Exercise 4.29.
Using the matrices in Exercise 3.30(a), use the method of Theorem 4.17
to find a 1 x 2 matrix K such that the closed loop system has charac-
teristic roots —-4and —S.

Exercise 4.30.
Repeat the procedure of the preceding exercise using the matrices in
Exercise 3.30(b) so as to make the closed loop system characteristic
roots —Ix=1"= 21.

Exercise 4.31.
’ Verify that the method of Theorem 4.17 when applied to Example 4.8
produces the same matrix K.
Linear control systems 117

Exercise 4.32.
If

0 1 0 1 0

A=|0 0 iad pe B= | 0 1

6 = 11 6 1 1

find using Theorem 4.19 a suitable matrix K which makes the character-
istic roots of A —BK equal to 1, 1, 3.

Exercise 4.33.
Consider yet again the rabbit—fox environment problem of Exercise 1.4.
It was seen in Exercise 1.7 that if a, —a@4 >0 in eqns (1.6) then in general
the animal populations increase without limit ast > °°. Suppose for
simplicity that the values of the constants are such that the system
equations are

xy = 2x, = Shey,X Ph a ae
It is decided to control the environment by introducing linear feedback
_in the form of a disease which is fatal to rabbits but does not affect
foxes, thereby reducing the rate of growth of the rabbit population by
an amount kx,. Find the smallest value of kKwhich prevents the “‘popula-
tion explosion’ from occurring (assuming arbitrary initial population
sizes).

Exercise 4.34.
Consider

X=Ax+bu, u=cx
where b and ¢ are respectively column and row n-vectors. Let b be a
right characteristic vector of A corresponding to a root A. Show that
b and \ + cb are corresponding characteristic vector and root of the
closed loop system.
If on the contrary b is arbitrary but c is a left characteristic vector
of A corresponding to A, show that c and ) + cb are corresponding
characteristic vector and root of the closed loop system.

Exercise 4.35.
In the preceding exercise let b and c be respectively right and left
characteristic vectors of A corresponding to the root A. Assuming that
118 Linear control systems

all the characteristic roots of A are distinct, show that the other roots
and vectors of the closed loop system are just the remaining roots and
vectors of A.

Exercise 4.36.
Let all the characteristic roots of A be distinct and let W be the matrix
defined in Theorem 4.19. By applying the transformation x = WE to
(4.31) deduce that wu;can influence £; if and only if pj; # 0, where the
n
pjj are defined by b; = » pjiw;, bj denoting the ith column ofB. Note
7-1
that as in Section 3.1 it follows that pj; = v;b; where v; are left character-
istic vectors of A.

If QO;
= bj, Abj,A7b;,..., A"—By]
it can be verified that Q; =WP;M’ where M is the Vandermonde matrix
defined in Exercise 2.15 and P; = diag [p1;, Daj, . . - ,Pu]. Hence deduce
that rank Q; is equal to the number of modes which can be controlled
independently by the input w;.

Exercise 4.37. 4,
Using the notation in Theorem 4.19, deduce that if WB has a zero row
than the associated controllability matrix (4.5) has rank less than n.

4.5. State observers

Unfortunately the usefulness of Theorem 4.17 in many real-life problems

is limited, since the feedback control (4.32) involves all state variables
and in practice it is often impossible to measure all the x;. Instead, if as
before the measured output is the column r-vector

y=, (4.53)
then linear output feedback
u=Ky = KCx (4.54)
is applied, but is no longer possible to preassign all the closed-loop
characteristic roots. However, if C has rank r, B has rank m and the
system is c.c. and c.o, then it has recently been shown (Sridhar and
Lindorff 1973) using an argument similar to that in the proof of
Theorem 4.19 that a matrix K exists such that at least s =max (m, r)
Linear control systems 119

of the characteristic roots of the closed loop system matrix A+ BKC

are arbitrarily close to any set Ag.
Instead of looking at methods for choosing K in (4.54) we consider a
different approach and show how the concept of observability can be
applied to obtain state feedback. If the system described by (4.31) and
(4.53) is c.o. then because of the definition in Section 4.2 it is possible,
at least in theory, to determine the state vector x from a knowledge of
y and u. A system whose inputs are uwand y and whose output x(t)
is an approximation to the state vector x(t) is called a state observer (or
estimator). Since the original system is assumed c.o., by Corollary 4.3
there exists a matrix L such that the characteristic roots of A +LC can
be made equal to a set A, = {0;}. In particular it follows that all the 6;
can be taken to have negative real parts, so that if we then set xg (¢) =
x(t) —x(t) and take
Xq=(At+LC)xq, (4.55)
this ensures that xq (tf) > Oast > ©, irrespective of xq (0) (see Section
5.2, Theorem 5.1). Thus X(t) > x(t) ast > ©, as required, and (4.55)
shows that X must satisfy

= Goeee)
i= ail.C)2)

SO
dx/dt=%—(A+LC)(x-x)
=Ax+Bu-(A+LC)(x-x)
=tA+tLO)x hy +Bu, (4.56)
Thus (4.56) represents a mathematical model for the desired state
observer, having inputs y and wuand output x.

Example 4.10. Consider the case of scalar control variable and output,
let the matrices A and b be as in Example 3.8, and take C = [2,3]. To
find the 2 x 1 matrix L we use a procedure similar to that given in the
proof of Theorem 4.17 for m = 1, but commencing with the observable
canonical form in Theorem 3.5. The transformation is vy=P"! x, where
from the result of Exercise 3.36 we have

ey 1 CG
pis ;
Law CA
120 Linear control systems

where

rN? +e,A te, =det(—A)=d? —3A 414.

This gives

8 -9
po =
phiae

so the matrices in (3.62) are

0), a14
E=P"'!AP= ;
1 3
f=CP=[0,1].
Suppose that the characteristic roots of the observer system matrix
A + LC are to be —3 and —4, so that its characteristic equation is
? + 7d + 12. Since

A+LC=P(E+P' LAP!

andE +P~! Lfis in companionformits last columnmustbe [-12, -7] 7,

1-€.

Hence L = [-2, —2]T the matrix required for constructing the observer
system (4.56).
An important point which is now demonstrated is that although the
feedback matrix K was obtained in Section 4.4 with respect to the actual
state x, use of linear feedback u = Kx instead of u = Kx still results in the
closed loop system having the desired set of characteristic roots. To see
this, put u = Kx in (4.31) and (4.56) and combine these two equations in
the form

x A BK G

dt ox -LC (A+LC+BK) x
Linear control systems 12

Apply a similarity transformation to the matrix in (4.57) as follows:

I Ree ca ” BK I 0| [A+BK -—BK
(4.58)
[eee LC 44 0C+BK| Li 0 A+LC
Since the right hand side of (4.58) is block diagonal, this shows that the
characteristic roots of the system in (4.57) are just those of A + BK (as
required), together with the known and also predetermined roots of the
observer system matrix A+ LC. Thus the use of the state observer enables
all the roots of the closed loop system to be preassigned even though
only the output y can be measured.
It can also be shown that the dimension of the observer system can be
reduced to n — rank C (see Chen 1970, p.295).

Exercise 4.38.
Using the matrices A and c in Exercise 3.32, find a matrix L such that
A +Lc has characteristic roots —3, —4, and —5.

4.6. Realization of constant systems

We defined in (2.44) the transfer function matrix G(s) associated with a

time-invariant system. In practice it often happens that the mathematical
description of a system in terms of differential equations is not known,
but G(s) can be determined from experimental measurements or other
considerations. It is then useful to find a system in our usual linear state
space form to which G (s) corresponds. For example, an analogue simulator
can then be constructed —this is essentially a device (usually electronic)
which duplicates the behaviour of the physical system and thus can be
conveniently used to study its properties.
In formal terms, given an r x m matrix G(s) whose elements are
rational functions of s, we wish to find constant matrices A, B, C having
dimensions n x n,n X mandr x n respectively such that

G(s) = C(sl—A)7'B,
and the system equations will then be (4.31) and (4.53). The triple
{A, BSChis termed a realization of G(s) of order n, and is not, of course,
unique. Amongst all such realizations some will include matrices A having
least dimensions —these are called minimal realizations, since the corres-
122 Linear control systems

ponding systems involve the smallest possible number of state variables.

Notice that since each element in

(s1]—AY=Adj(sI—
A)/det(sf—A)
has the degree of the numerator less than that of the denominator it
follows that C(s] -A)"'B > Oass > ©, and we shall assume that any
given G(s) also has this property, G(s) then being termed proper.

Example 4.11. Consider the scalar transfer function

As Kae i
(8)sSj= s* —5s+6°

As in (4.39), one realization of g is

Bee wa U
A= b= EN).
6 5 1

It is also easy to verify that a quite different triple is

2enn0 1
A= es c= [-11,13]. (4.59)
Cables 1
In fact both these realizations are minimal, and there is in consequence
a simple relationship between them, as will be shown later.
We begin our development with a generalization of the result of (4.39),
an example of which has just been given. This enables us to write down a
simple realization for a transfer function matrix, although it will not in
general be minimal.

THEOREM4.20. Let

g(s)=st tg,st 1 tees tgy (4.60)

be the monic least common denominator of all the elements gj; (s) of
G(s), and let

2(s)G(s)=541 Gp+897
7G, +¢ °° +Gq-1, (4.61)
Linear control systems 123

the G; being constant r x m matrices. Then a realization of G(s) is

0 ie ei 0
0 0 If,
A= wo= q
Im eee (462)
Bqlm Sq-tlm + Pcs ad 0
Im
CamereG, >,..-4 Go].

Furthermore, the pair [A, B] is c.c.

Proof. Let F denote the companion matrix of g(s) in the form (2.32).
Clearly A= F@J),, so that

(Slag —A) = [slg @lin —

Fm)
= [(slq—F)®lm)
= (slg-F)"! ®lm, (4.63)

using the result of Exercise 2.2. Also

B = eq ®lm, (4.64)

where eg denotes the last column of J/g, so (4.63) and (4.64) give

(s1—A)*B= [(lg—F)™*@In) [eg®lm]

[(lg =F)™ eg] Gm, by (2.5),

1
S
s7 Q/,/2(s), (4.65)

57-1
124 Linear control systems

using the expression given in Exercise 4.39 for the last column of
(slg -F)" = Adj (slg —F)/g(s). Finally, combining (4.65) and the
expression for C in (4.62) we have
C(sI- A) "B= [G,_,,G,_>)° a ,Go] Im /g(s)
Sl,

Rseae be

= G(s),
by virtue of (4.61), showing that{A, B, Chis a realization of G(s). f
That [A, B] is c.c. can be seen from the fact that the controllability
matrix (4.5) is
0 00 ee ee

0 0 . ° ° a¢ L€ “ome

[B, AB;+. pA TB] Sei Se ed = , a

2. ° Vine . ° . . . e
ipa? 2 é = > aan
foe Xe xe . 3 e . ° °
(X denoting nonzero terms) and so clearly has rank equal to mq, the
dimension of A.
As we have already seen in Example 4.11, when m= | the matrices
A and B in (4.62) reduce to C and d in the controllable canonical form
in Theorem 3.4. For a similar generalization of the observable canonical
form in Theorem 3.5, see Exercise 4.44. Notice also that when G (s) is
a scalar transfer function (i.e. r=m = 1) then (4.62) is minimal provided
any common factors between numerator and denominator have been
removed (see Exercise 4.41).
It is now appropriate to return to the idea of algebraic equivalence
defined in (4.17), and discuss its implications for the realization problem.
Since A, B, and C are constant P is also constant, and the transformation
x = Px in (4.17) produces a system with matrices
A=PAP™ B=PB,C=CP™. (4.66)

+ This part of the proof is due to H. K. Wimmer,

 Linearcontrolsystems 125
THEOREM 4.21.If{A,B, C}representsa c.c.(c.o.)systemthenso
does{A,B,C}.
Proof. The controllability part follows as a special case of Theorem 4.8.
Alternatively, using (4.66) we have

rank [B,AB,..., (A)”~!B] =rank (P[B,AB,..., A" 1B] =n

since P is nonsingular, so [A, B] is c.c. if [A, B] is by Theorem 4.1. The

observability condition follows similarly (see also Exercise 4.25).
Not only are controllability and observability properties preserved
under algebraic equivalence, but it is also easy to prove Theorem 4.22
(see Exercise 4.45).
THEOREM 4.22. If two systems are algebraically equivalent then their
transfer function matrices are identical, i.e.

C(sl —
A) B=C(sI—A)"B.
We can now prove the central result of this section, which links
together the three basic concepts of controllability, observability, and
realization.

THEOREM4.23. A realization{ A, B, Chofa given transfer function

matrix G(s) is minimal if and only if [A, B] is c.c. and [A, C] is c.o.
Proof.
Sufficiency. Let Uand V be the controllability and observability
matrices in (4.5) and (4.26) respectively. We wish to show that if these
both have rank n then G(s) has least order n. Suppose that there exists
a realization {A, B, C}of G(s) with Ahaving order fi. Since

C(sl-A) 1B=C(sI-A)'B

it follows(seeExercise4.45) that

Cexp (At)B=C exp (ANB

which implies, using the series (3.8), that

CAB= CA Bei=
0, 122,.13,» (4.67)
126 Linear control systems

Consider the product

G
CA
VU=|- [B, AB....,A" "sie

GA" —1
CB CABS WR CA™"B

Cia! CA"B CAL"=p

C
CA
= (B, AB,...,A”!B], using (4.67)

CA nti
= V,U,, Say. (4.68)
By assumption V and U both have rank n, so the matrix V;U, also has
rank n. However, the dimensions of V, and U, are respectively r,;n x n
and n x m,n, where r, and m, are positive integers, so that the rank of
V,U, cannot be greater than n. That is, n <A”, so there can be no real-
ization of G(s) having order less than n.

Necessity. We show that if the pair [A, B] is not c.c. then there exists
a realization of G(s) having order less than n. The corresponding part of
the proof involving observability follows by duality.
Let the rank of U in (4.5) be ny <nand let u,,u2,..., Uy, be any
set of n; linearly independent columns of U. Consider the transformation —
& = Px with the n x n matrix P defined by

P= [ayuys-.-s unas Une ise eee (4.69)

where the columns uy, + 1,..-, Um are any vectors which make the
matrix in (4.69) nonsingular. Since U has rank n, it follows that all its
columns can be expressed as a linear combination of the basis u,, u,
. Un. The matrix AU= [AB,A?B,...] containsall but the first m
Linear control systems 127

columns of U, so in particular it follows that the vectors Au;, =1,2,...,

n, can be expressed in terms of the same basis. Multiplying both sides of
(4.69) on the left by P shows that Pu; is equal to the ith column of J,.
Combining these facts together we obtain

A =PAP™', by (4.66),

=F TAU,..: - AU. yaar OFA

= A, Az

0 A3

where A, ism, xXny. Similarly, since u,,...,uU,. also forms a basis for
the columns of B we have from (4.66) and (4.69)

By

where B, isn, x m. Writing C=CP = [C,, C2], by Theorem 4.22 we

have

G(s)= C(I -A)"B

sl—A, —A, me B,
rs [Ci C2]
0 sl—-A3 0
(SE=Aq * (Sk=A)71A3(8SI—
45)" B,
= [Cy.C]
0 (sf -A 3)" 0
=C; (sI-A1)*B,

showing that {Aere Ci} is a realization of G(s) having order ny <n.

This contradicts the assumption that {A, B, CHisminimal, so [A, B]
must be 6.c.
Notice that the necessity part of the preceding proof, together
with Exercise 4.49, establishes Theorem 4.9.
128 Linear control systems

Example 4.12. Consider the electrical network in Exercise 4.16 (Fig.

4.4). Using x; and x2 as state variables and y as the scalar output,
standard techniques for circuits give

=VRC 0 1/R,C
A= p= ,c= |-l Ryan
0 Seuss 1/L

and y =cx t+u/R,.

After Laplace transformation we obtain

(s) =Se(sI-—
A)“1b+1/Ry}
Us)
andhence
g(s) M(RTC=E)s +(R; —R2) + m9:
(Ri Cs+R,) (Ls+R) R,
By the result of Exercise 4.16 it follows, using the theorem just proved,
that the above triple {A, b, c} is a minimal realization for g(s) if and only
if L #R,R,C In fact it is easy to verify that when L = R,R.C there
is a common factor between numerator and denominator of g(s).
Of course for any given G(s) there are an infinite number of minimal
realizations satisfying the conditions of Theorem 4.23. However, we now
show that the relationship between any two minimal realizations is just
that of algebraic equivalence:

THEOREM 4.24,If{A,B, C}=R isa minimalrealizationof G(s)then

{A,B,C }=R isalsominimalrealizationif andonlyif (4.66)holds.
Proof.
Sufficiency. If (4.66) holds, then by Theorem 4.22, Ris certainly a
realization and is minimal since A has the same dimensions as A.
Necessity. Let U, U, V, V be controllability and observability matrices
(4.5) and (4.26) associated with the minimal realizations Rand R. We
show that (4.66) holds by demonstrating that the transformation matrix
is
Pe(VID VT, (4.70)
Since V and V are associated with minimal realizations, by Theorem 4.23
they both have rank n, so that the matrix on the right in (4.70) certainly
Linear control systems 129

exists, and is nonsingular. Since A and A have the same dimensions we

can deduce from (4.68) that

VU=VU, (4.71)

and by a very similar argument, using (4.67), that

VAU= VAU. (4.72)

Multiplying (4.71) on the left by (V7 yy yt gives

pu=U (4.73)

i.e. P(B, AB, - ++] =[B, AB, - °°]

so that PB = B, P being defined by (4.70). From (4.73) it is clear that

P= tu ™(uuT)", (4.74)

ee (4.74) showsthat multiplicationof (4.71) on the right by

U T(UUT)" gives V= VP, whence from (4.26) C= CP. Finally, it is left
as an exercise for the reader to use (4.70), (4.72), and (4.74) to verify
that PA =AP, which completes the proof that (4.66) holds.
We do not have room in this book to discuss the general problem of
efficient construction of minimal realizations. For example, one method
is to use the c.o. realization given in Exercise 4.44 and then reduce it toa
realization which also c.c. by the procedure indicated in the second part
of the proof of Theorem 4.23. For further details we refer the reader to
Barnett (1971, p. 58), Chen (1970, p.235), and Rosenbrock (1970, p.
116) and merely give here one simple but nevertheless useful result:

THEOREM4.25, Let the denominators of the elements gj;(s) of G(s)

have only simple zeros s;,52,... »Sp. Define

ek oecim .(s—s;) G(s), 1 = 1; 2, 2. 3p (4.75)

S> S;

and let r; = rank K;. If L; and Mj are r x r; andr; x m matrices respect-

ively, each having rank 7;, such that

K; = L;M; , (4.76)
130 Linear control systems

then a minimal realization of G(s) is

Sil; 0 ee

M,
A= Sol,
ie ee he (4.77)

0) . a
Spry | Dp
C= ese At Lp].

Proof. Eqn (4.75) is equivalent to the representation of G in partial

fractions in the form
Dp
GO)= > Kills-s). (4.78)
f=1
Using the matrices in (4.77) we have

C(sl —ANyye =i diag Uy, [(s wo ; I, Ms= Se, tee Try l(s —Sp)) B

= ZL Mj |(s —8)

= £K;/(s —s;), by (4.76)

= G(s) by (4.78).

To prove that the realization (4.77) is minimal it is necessary to show

that it is c.c. and c.o. (Theorem 4.23). The controllability matrix is

M, 5,M, siM, = se —ly,

M, 52M, 53M, :

(Bee e va An Bl ae (4.79) |

Mp SpMp 2
SpMp « Sp
git Mp

where n=r, tr, + +++ +rp. Using the facts that M; has rankr;
and all the s; are distinct it follows that the matrix in (4.79) has full rank
(see Exercise 4.51). The observability condition is proved similarly.
 Linear control systems ti

Example 4.13. In the scalar transfer function in Example 4.11 the

denominator is
s* —53+ 6 =(s— 2)(s— 3).

Thus (4.75) gives

ein /(s\=2) Qs Ty)
K, i]
= 1 Sl
soz (s—2\(s—3)
Rim (s-3)(25+7) _
Ky i]
18, r=

Taking L; =K,,M, =1,L, =K2,M, =1 in (4.76) then produces a

minimal realization (4.77) which is precisely that in (4.59). However

s-|my,
| c=[—11/m,,
13/m,]
mM

can be used instead, still giving a minimal realization for arbitrary nonzero
values of m, and m).
Notice that (4.75) can be used to evaluate the order of minimal
_realizations of G(s) without actually determining a realization, since the
p
order is the dimension of A in (4.77), namely > r;. However, calculation
=|
of the 7; involves the same disadvantage as using Theorem 4.25 in general,
namely the need to calculate the s;. An alternative method of finding the
least order of G(s) which does not rely on any knowledge of the s; is
obtained by arguing as follows. The realization in (4.62) is c.c. but not
in general c.o. By Theorem 4.14, (4.62) is therefore algebraically equi-
valent to
A, O B,
5) > [C, p)0] (4.80)
Az A; By

where [A;,C,] is c.o., and by Theorem 4.22 the triple in (4.80) is a

realization of G(s). It is easy to verify that {A, ps 4 his also a reali-
zation of G(s) (compare with the second part of the proof of Theorem
4.23). From Theorem 4.21 we know that (4.80) is c.c., which implies
that [4,, B;] is also c.c. Hence {Ayi By; C; Sis a minimal realization of
G(s), since it is both c.c. and c.o. Therefore the least order of G is equal
132 Linear control systems

to the dimension of A,. Finally, from Theorem 4.14, this dimension is

equal to the rank of the observability matrix (4.26) with A and C as
given in (4.62).

Exercise 4.39.
If C, d, and k are as defined in (3.45), (3.46), and (3.47), and
G(s) =Bos” 1 +B,s?~ 2 +++ ++B, —4, verify that a realization of
B(s)/k(s) is {C d, r} where r= [8,—1;B8n—2;---,>6o]. (Hint: By con-
sidering the product (s/ —C)Adj(s/ —C) show that the last column of
Adj(sI —C)is [1,s,s?,...,8”~4]").
Show that the realization is minimal if 6(s) and k(s) are relatively
prime. (Use the result of Exercise 4.27).

Exercise 4.40.
Deduce from the result of the preceding exercise that a realization of
B(s)/k(s) is ew rie d‘} and that this is c.o. but not c.c. if B(s) and k(s)
are not relatively prime.

Exercise 4.41.
Let A be an arbitrary n x n matrix with characteristic polynomial
q(s), and let

c(sl —
A)" 'b = t(s)/q(s) =8(s)
where cis 1 x nandbisn x 1. Prove that {4,b,chis a minimal real-
ization of g(s) if and only if t(s) and q(s) are relatively prime.

Exercise 4.42.
Using the notation in Theorem 4.16, let
Px(S)

Eas:
G(s) = KS)

Pm(S)
and assume that the polynomials k(s), p;,..., Dm are relatively prime.
Generalize the result of Exercise 4.39 by verifying that a minimal real-
Linear control systems 133

ization of G(s) is {C, d, E} where C, d are defined in (3.45) and (3.46),

and
by “ i bn

bim i ; bam

Exercise 4.43.
Using Theorem 4.25, write down a minimal realization for a scalar
transfer function whose denominator has only simple zeros.

Exercise 4.44,
Let G(s) and g(s) be as defined in (4.60) and (4.61). Deduce that
{A™,B, C}, where A is defined in (4.62),

Go-1

B=| * Ce {000.7 sda ls

Gy
Go
is a realization of G(s), and that [A, C] is c.o.

Exercise 4.45.
Using (4.66) prove Theorem 4.22, and hence deduce that

C{exp(At)}B
=C.fexp(A
t)B.

Exercise4.46.
UseTheorem4.25 to obtain a minimalrealizationof
(s? +6) (s? +5 +4)

g(s)
(Qs* 7s 2) (Ss? 55—2)
where g(s) =s° + 2s? —s—2.
134 Linear control systems

Exercise 4.47.
Show that the order of a minimal realization of

(s+2) 2s +2)

s? +3s+2
i (srtih)
is three. (Notice the fallacy of assuming that the order is equal to the
degree of the common denominator).

Exercise4.48.
If{A,,B,,C,}and {A,,Bz,C>}arerealizations
of G,(s)andG,(s),
showthat
A, B,C, 0
A= , B= , C=[C, 0]
0 A, B,

is a realization of G,(s)G2(s), assuming that this product exists.

Exercise 4.49. .
Verify that in the second part of the proof of Theorem 4.23 the pair
[A,,B,] isc.c.

Exercise 4.50.
Show that the transfer function matrix of the system in Theorem 4.15
depends only on the c.c. and c.o. part. This result shows that the
uncontrollable and unobservable part of the state space representation
is lost when the transfer function matrix description is used.

Exercise 4.51.
In (4.79) let p =2,r,; =2,r2 = 2 for simplicity. By considering the
linear independence of the rows of the matrix on the right in (4.79)
show that it has full rank.

Exercise 4.52.
Verify that algebraic equivalence in (4.66) can be written as the trans-
formation
 Linear control systems 135

(this is called system similarity —see Rosenbrock (1970) for a develop-

ment of linear control theory in terms of polynomial and rational
matrices).

4.7. Discrete-time systems

So far in this chapter we have discussed only systems described by linear

differential equations. In fact the definitions of controllability and
observability given in Sections 4.1 and 4.2 carry over to systems described
by the linear difference equations (3.31) with only minor obvious
modifications.
When A and B in (3.31) are time invariant the controllability and
observability criteria are the same as those for the continuous time case
given in Theorems 4.1 and 4.2 respectively (Rubio 1971, p. 234) with one
important exception. This is the situation when the discrete-time system
A-matrix is singular. The discrete-time version of the argument in the
first part of the proof of Theorem 4.1 breaks down because the corres-
ponding transition matrix AF is also singular (Exercise 2.11), whereas
_exp(Af) is nonsingular for all matrices A (Exercise 3.4). It is necessary
to define a new concept: the linear system
x(k + 1) = Ax (k) + Bu(k) (4.81)
is completely reachable (from the origin) if given any state xf there exist
an integer V > 0 and a control sequence u(0), u(1),...,u(NV—1) such
that if x(0) = 0 then x(V) = x. When A in (4.81) is singular this is not
the same as complete controllability, which requires that given any state
Xo there exist an V> 0 and a control sequence such that if x (0) =o
then x (V) = 0. In this case the discrete analogue of Theorem 4.1 is that
(4.81) is completely reachable if and only if
rank[B,AB,...,A"~1B] =n
and the set of reachable states is a subset of the set of controllable states
(Kalman, Falb, and Arbib 1969, p.41). For the continuous time system
(4.4) the concepts of reachability and controllability are completely
equivalent.
To deal with the realization problem we apply z-transforms to (4.81)
to obtain, using (1.30) and assuming x (0) = 0.
2%(z) =Ax(z) + Bit(z). (4.82)
136 Linear control systems

Rearranging(4.82) gives
x(z) = (zl —A)"1Bu(z)
so the output y (kK)
= Cx(k) hasz-transform
p(z) = C(I —A) *Bu(z). (4.83)
Thus the transfer function matrix in (4.83) relating y(z) and u(z) has
exactly the same form as that for the continuous-time case. The theory
developed in Section 4.6 therefore carries over directly for (4.81)
virtually without modification, although in view of our remarks on the
case when A is singular, Theorem 4.23 should be interpreted as stating
that a realization is minimal if and only if the matrices (4.5) and (4.26)
have full rank. When linear feedback u(k) = Kx (k) is applied to (4.81)
the closed loop matrix is again A + BK, so the methods of Section 4.4
also apply to discrete-time systems.

Exercise 4.53.
It was seen in the buffalo population model in Exercise 1.13 that under
natural conditions the numbers would grow without limit. In fact due to
indiscriminate slaughter the number of animals was reduced from 60
million in 1830 to only 200 in 1887.
Suppose that a controlled slaughter policy had been adopted, killing
for food Sx + 1 adult females in year k + 1 so that (1.33) becomes

Fy 42=0-95Fx
41+0-12FE
- Sx 4.
If linear feedback Sx + 1 =pFx +1 is used, show that the total population
would remain constant even if 7 per cent of adult females were killed each
year. Also, find an expression for the number of adult females in the
steady state in this case.
What difference would it make if there were a year’s delay in com-
pleting the buffalo census, so that Sx + 1 =pFx?
As a footnote, a single buffalo carcase would provide enough meat
(250 kg) for at least 10 people for a whole year!

Exercise 4.54,
Show using (3.39) that when A is nonsingular, then if the control sequence
u(ko),...,U(N— 1) transfers x(Ko) =Xo to x(N) = xg, it also transfers
Xo —A”*° ~ “*x¢to the origin in the same time interval.
 Linear control systems 137

Exercise 4.55.
Verify using (3.41) that the control sequence

u(i)=—B?GT)" (0,1+ 1)W"(0,N)[xo—(0, N)xs],

i=0,1,2,...,N-—1, where® isdefinedin (3.40)and
N=
W(0,N)= x (0, i+ 1)BGT)B"(iT)OT(0,i + 1)
i=0
is assumed nonsingular, transfers the system described by (3.31) from
x(0) =Xo tox (NV)= xz.

74.8. Realization of time varying systems

We now return to systems described by eqns (4.1) and (4.2). Once again,
as in the case of the transition matrix for time varying systems discussed
in Section 3.5, we find that many of the properties established for time-
invariant systems in Section 4.6 still hold. However, as is to be expected,
' we cannot give analytical methods for calculation of realizations.
As in the constant case assume that x9 = 0. Then using (3.30) the
output is

¥())= Ci)x()

C(t)i b(t,1)B(r)u(r)
dr
K(t, ru(r) dr

where ® is the transition matrix defined in (3.27). The matrix

K(t, T) = C(t) ®(t, r)B(7) (4.84)

is called the weighting pattern matrix. For a given K (t, 7) the realization
problem is now to find a triple {A(t), B(t), C(t) } such that (4.84) is
satisfied. The notion of minimality of a realization is the same as before,
that A should have least possible dimension. The relationship with the case
when A, B, and C are constant is obtained by noting that by (3.11) the
transition matrix is then
@(t, 7) = exp [A(t—-7)], (4.85)
138 Linear control systems

and the Laplacetransformof the right sideof (4.84) is C(sJ -—A)"*B,

usingthe expressionfor £ {exp(At)} in (3.18). In this constant casesub-
stitution of (4.85) into (4.84) gives
K (t, T) = C(t) exp (At) exp (Ar) B(7)
showing that K(t, 7) can be written as a product of functions of ¢ and rT.
We now establish that this holds even when A, B, and C are time varying.

THEOREM 4.26. A realization exists for a matrix K(t, 7) if and only if

it can be expressed in the form
K(t,7T)=L()M(1) (4.86)

where J and M are matrices having finite dimensions.

Proof. From (3.27) and (4.84) if K possesses a realization then
K(t, 7) = C()X (OX *(7)B(O),

where X(t) is defined in (3.26), so (4.86) is certainly a necessary condition.

Conversely, if (4.86) holds then a realization of K(f, 7) is
{0, ECE (t)}where O, denotes ann x n zero matrix, since then
P(t, 7) =.
It is interesting that the fundamental result on controllability and
observability established in Theorem 4.23 still holds in the time varying
case.

THEOREM 4.27.ArealizationR={A(t),B(t),C(O},ofK(t, niis

minimalif andonlyif it isc.c.andc.o.
Proof.

Necessity. We assume that the pair [A (t), B(t)] is not c.c. and show
that the realization R is then not minimal. A similar argument applies
if [A, C] is assumed not c.o. Suppose that A isn x n and that the
controllability matrix U(to, tf;) in (4.12) has rank p <n. Since, as was
shown in the proof of Theorem 4.3, U(to, t; ) is positive semidefinite,
there exists a nonsingular matrix T such that

jp iitero
TUP = (4.87)
On 10
(see Exercise 2.23). Consider the algebraic equivalence transformation
with P(t) in (4.17) taken to be TX (to )X~ '(t), where X(t) is defined by
Linear control systems 139

(3.26). It is easyto verify (see Exercise4.58) that under this transforma-

tion R becomesR = {On, B(t), C(t) }whereB(t) = P(t)B(t) and C(e) =
C(t)P”'(t). Furthermore (see Exercise4.57) R is also a realization of
K(t, rT).From (4.20) the controllability matrix associatedwith R is
U =P(t) U(to, t1)P" (to)
= TUL” (4.88)
and since ®=/ for R we also have from (4.12)

Vie | a B(r)B? (s)dr. (4.89)

ty
From(4.87),(4.88),and(4.89)it followsthat
eae
=
0
where 6 has dimensions p x m. This implies that [Op. B, |], where y is
r X p,isa realization of K, and this contradicts the minimality ofR.

Sufficiency. We now assume that R is c.c. and c.o. and show that there
cannot exist a realization R, of K having order n, <n. First, by again
applying the result of Exercise 4.58 we can assume that such a realization
has the form R, ={0n, fora On (t)}. It similarly follows that under
the transformation P = X~'(t), R becomes R ={0;, , B(t), C(t)}and
remains c.c. and c.o. (see Theorem 4.8). Since

K(t, 1) =Ci (Bi (1) = C(QB(),

multiplyingthe secondand third termsin this expressionon the left
and right by C!(t) and B'(r) respectivelyand integratingwith respect
to ¢and7 gives
W,W,=VU (4.90)
where

Wes
f‘ CTY)C\(t)dt,
W,=fBi BT(oar
and Vand U are the observabilityand controllabilitymatricesfor R
definedin (4.22) and (4.12) respectively.By assumptionV andU have
rank n, so VUalsohas rank n. However,W, and W, have dimensions
140 Linear control systems

nX n, and ny, X n respectively, so that rank (W,W2)<n,. Hence

(4.90) implies that n,; 2n, showing that R is minimal.
It can also be shown by a similar argument that if rank (UV) =q <n
then the least order of realizations of K(¢, T) is equal to q.
Again following the time invariant case (Theorem 4.24) the relation-
ship between minimal realizations is that of algebraic equivalence, defined
in (4.17).
THEOREM 4.28.If R=A (t), B(t),C(t)}isa minimalrealizationof
K(t, rT)thenR ={4(0),B(t), C(t)fis alsoa minimalrealizationif and
onlyif
A=PP-! + PAP, B=PB,C=CP4 (4.91)
where P(f) is continuous and nonsingular.
The proof follows along similar lines to that of Theorem 4.27.
An interesting question is under what conditions a given weighting
matrix has a time invariant realization:

THEOREM 4.29. A given matrix K (t, r) has a realization {A, B, C}in

which A, B, C are constant matrices if and only if (4.86) holds and if in
addition
K(t, 7) =K( +s, 7+), fo S77, 8 =
Unfortunately the proof does not provide a practical method of con-
structing such a realization, so will also be omitted. A good source for
these proofs and other material on realization of time varying systems
is the review by Silverman (1971).

Exercise 4.56.
Verify that in Theorem 4.26 suitable matrices are L (t) = C(t) ®(f, fo)
and M(t) = ®(to, T)B(r), where ®(f, to) is defined in (3.27).

Exercise 4.57.
Show, using Theorem 4.7, that if two realizations are algebraically
equivalent then the corresponding weighting pattern matrices are identical

Exercise4.58. |
Showthat if P(t) in (4.17)is taken to be EX '(t), whereX(t) is defined
in (3.26)and £ is a constantnonsingularmatrix,then{A(¢),B(t), C(t)}
istransformedinto{0,P(t)B(t),C(t)P™(t)}.
Linear control systems 141

Exercise4.59.
Showthat thesystem
X=Ax + {exp(Dt)}Bu,y =C{exp(—Dt)}x
where A, B, C, and D are constant matrices and AD = DA, possesses a
constant realization.
5 Stability

In contrast with the preceding two chapters our treatment of stability

will apply to systems described by sets of linear or nonlinear equations.
As is to be expected, however, it is once again for linear systems that our
most explicit results will be obtained. Stability theory is a vast subject
and to keep this chapter within a reasonable length it has been necessary
to omit many proofs and finer points of detail. Where possible, however,
references have been given to standard textbooks which can be consulted
if further material is required.

5.1. Definitions

Consider the system described by

x =f, a (5.1)
where as before x(t) is the state vector and f is a vector having compon-
ents fj(%1,%2,...,Xn,t),i=1,2,...,m. We shall assume that the f;
are continuous and satisfy standard conditions, such as having continuous
first partial derivatives so that the solution of (5.1) exists and is unique
for given initial conditions (see, for example, Brauer and Nohel 1969).
If the f; do not depend explicitly on ¢, eqn (5.1) is called autonomous
(otherwise, nonautonomous). If f(c, t) = 0 for all t, where c is some
constant vector, then it follows at once from (5.1) that if x (to) = c then
x (t) =c, all t >to.Thus solutions starting at c remain there, and c is said
to be an equilibrium or critical point (or state), Clearly, by introducing
new variables x;’ = x; —c; we can arrange for the equilibrium point to be
transferred to the origin; we shall assume that this has been done for any
equilibrium point under consideration (there may well be several for a
given system (5.1)) so that we then have f(0, t) = 0, t >to. We shall also
assume that there is no other constant solution in the neighbourhood of
the origin, so this is an isolated equilibrium point.

Example 5.1.
The intuitive idea of stability in a dynamical setting is that for
“small” perturbations from the equilibrium state at some time fo,
Stability 143
subsequent motions x(t), f> fo, should not be too ‘large’. Suppose that
Fig. 5.1 represents a ball resting in equilibrium on a sheet of metal bent
into various shapes with cross-sections as shown. If frictional forces can
be neglected then small perturbations lead to:

(a) oscillatory motion about equilibrium;

(b) the ball moving away without returning to equilibrium;

(c) oscillatory motion about equilibrium, unless the initial perturba-

tion is so large that the ball is forced to oscillate about the new
equilibrium position on the left, or to fall off at the right.

If friction is taken into account then the oscillatory motions steadily

decrease until the equilibrium state is returned to.
Clearly there is no single concept of stability, and very many different
definitions are possible (see for example Willems 1970, p. 3). We shall
consider for the present only the following fundamental statements;

An equilibrium state x = 0 is said to be:

(i) stable if for any positive scalar € there exists a positive scalar
6 such that || x(to) lle <6 implies |]x(Z) lle <<€, t 2 fo.

(ii) asymptotically stable if it is stable and if in addition

x(t)>Oast>,

(iii) unstable if it is not stable; that is, there exists ane <0
such that for every 5 > O there exists an x(fo) with
llx(to) Il<5, |lx (£1) || = € for some f, > fo. If this holds
for every x (fo) in ||x (to) || <6the equilibrium is completely
unstable.

The definition (7) is often called ‘stability in the sense of Liapunov

(stability is. Z.) after the Russian mathematician Liapunov, whose
important work of 1892 features prominently in current control theory
literature on stability problems and will be described in Section 5.4.
In Example 5.1, case (a) in Fig. 5.1 represents stability i.s.L. if
friction is ignored, and asymptotic stability if friction is taken into
account, whereas case (b) represents instability. If the metal sheet in Fig.
5.1(a) were thought to extend indefinitely then if friction is present the
+ Throughout this chapter the Euclidean norm will be used, so the suffix e will hence-
forth be dropped. ‘
 144 Stability

ball would eventually return to equilibrium no matter how large the

disturbance. This is an illustration of asymptotic stability in the large,
which means that every motion converges to a single equilibrium point as
t > ©, and clearly does not apply to case (c) in Fig. 5.1. Asymptotic
stability in the large implies that all motions are bounded. Generally,
x = 0 is said to be bounded or Lagrange stable if there exists a constant

-=—

(a) (b) (c)

FIG. 5.1

b, which may depend on fg and x(t), such that || x(t) || <b for all
t2fo.
Regarded as a function of f in the n-dimensional state space, the
solution x(t) of (5.1) is called a trajectory or motion. In two dimensions
we can give the definitions a simple geometrical interpretation.
If the origin O is.stable, then given the outer circle C, radius e, there
exists an inner circle C,, radius 5,, such that trajectories starting within
C, never leave C. If O is asymptotically stable then there is some circle
C,, radius 62, having the same property as C, but in addition trajectories
starting inside C, tend to Oast > ~.
Some further remarks can be made at this stage:
(1) Notice that we refer to the stability of an equilibrium state of
(5.1), not the system itself, as different equilibrium points may
have different stability properties.
(2) A weakness of the definition of stability i.s.L. for practical
purposes is that only the existence of some positive 6 is required,
so 6 may be very small compared to € —in other words, only very
small disturbances from equilibrium may be allowable.
Stability 145

FIG. 5.2

(3) In engineering applications asymptotic stability is more desirable

than stability, since it ensures eventual return to equilibrium,
whereas stability allows continuing deviations ‘not too far’ from
the equilibrium state.
Some further aspects of stability are now illustrated.

Example 5.2. We again return to the rabbit—fox environment problem

described in Exercises 1.4 and 1.7. The eqns (1.6) have a single equilib-
rium point at the origin. With arbitrary initial numbers x, (0) and
x2 (0) of rabbits and foxes respectively the solution of eqn (1.6) is
found to be

x1 (t) = [a2x2(0)—a4x (0) + e%"(@,x, (0) —a2x2 (0))] /d,

where d =a, —dq, and a, /a3 = 4@/a4, with a similar expression for x(t).
If d>0,x,(t) and x2(f) tend to infinity as t > ©, so the origin is
unstable. If d<0, both x, (t) and x(t) approach constant values as
t > ©, and it is intuitively obvious that the origin is stable. This could
be verified formally by showing that the condition of the definition is
satisfied, but we shall see in Section 5.2 that linear equations like (1.6)
can be tested directly.
146 Stability

Example 5.3. Consider the system described by

Xa = oo (5:2)

It is clear that the solution exists and is unique, and in fact by integrating
we easily obtain

—1/x, =t—-1/x?

wherex, (0) =x9. Hence

D<(t) = eo
; (i/x})-t

so that if x? >0,x,(t) > © as t > 1/x$. The solution of the differ-

ential equation (5.2) is said to ‘escape’ to infinity in a finite time, or to
have a finite escape time. We shall henceworth exclude this situation and
assume that (5.1) has a finite solution for all finite t > to, for otherwise
(5.1) cannot be a mathematical model of a real-life situation.

Example 5.4. We demonstrate that the original is a stable equilibrium

point for the system described by

xy =X el = 28) (5.3)

by determining explicitly the scalar 5 in the definition. Integrating (5.3)

and using x;(to) =x? gives

x(t) =x} exp (t —t?) exp (13,—to). (5.4)

The condition |x, (t)| < ein (5.4) leads to

|.x° | <e exp (¢? —2)exp (to —#2). (5.5)

Since exp(t? —f)has a minimum value of exp(—4 ) when t =4, it

follows from (5.5) that we can take 5 = € exp[ (to —4)?]. In general
Stability 147
§ will depend upon e, but in this example it is also a function of the
initial time. If 5 is independent of fg, the stability is called uniform.
The following two examples illustrate the /ocal nature of the definit-
ion of stability i.s.L., mentioned in Remark (2) above.

Example 5.5. Consider the equation

Xan 3
ee (5.6)
0, SP

The solution of (5.6) is easily found to be

at = Detf2)iexp(t =f) voifexf>

2, (5.7)
=x? fixe
<2 :
where as before x; (to) =x?. The condition |x, (t)| <e is implied by
|x?| <2 when € > 2, for then by (5.7) |x, (#)| = |x? |<2<e. When
€ <2, |x, (0)| <€e is implied by |x?| <e, for then again |x, (t)| =
|x? |<e. Thus according to the definition the origin is a stable equilib-
rium point. However, (5.7) shows that if x? > 2 then x, (t) > ©, so for
initial perturbations x? > 2 from equilibrium motions are certainly
unstable in a practical sense.

Example 5.6. Consider

NE) x4

where

f(@)=%& 10, 0<t<10

=-l, t>10.
The solution is

x, (t) = 10°x?, 0<tr<10

= 10!°exp(10—r)x?-t > 10. 2)

Clearly x;(¢) +0 ast +, and the origin is asymptotically stable.

However, if x? changes by a very small amount, say 1075,then the
148 Stability

corresponding change in x, (¢) in (5.8) is relatively large —for example

when ft= 20, the change in x, (¢) is 10’° exp (10 —20). 107° =4.5.
Examples 5.5 and 5.6 show that an equilibrium point may be stable
according to Liapunov’s definitions and yet the system’s behaviour may
be unsatisfactory from a practical point of view. The converse situation
is also possible, and this has led to a definition of ‘practical stability’ being
coined for systems which are unstable in Liapunov’s sense but have an
acceptable performance in practice, namely that for pre-specified deviat-
ions from equilibrium the subsequent motions also lie within specified
limits (La Salle and Lefschetz 1961, p. 121).

Example 5.7. The equation of motion of a simple pendulum consist-

ing of a mass m suspended from a fixed point by a rod whose weight can
be neglected is

m2?.d?6/dt? =—mg sin 0.

For simplicity assume g/& = 1 and take as state variables x; = 0, x2 = 6

to obtain

Xa) (5.9)

mig

FIG. 5.3
The origin of these coordinates is an equilibrium point, and from (5.9)
dx, /dx ee—x,/sin xy

which on integration gives

x27 =2cosx, +a
Stability 149
where a is ‘determined by the initial conditions. Routine analysis shows
that the trajectories in the x;, x2 plane (the phase plane), for various
initial conditions, are as indicated below.
The origin is clearly a stable equilibrium point. However, if the initial
conditions are outside the hatched region (for example the point P)
then this produces a trajectory for which x, diverges to infinity.

Exercise 5.1.
Determine the equilibrium point (other than the origin) of the system
described by

Ny HS
X1> 2X1x3
Xp ==2X5 $X1X2

Apply a transformation of coordinates which moves this point to the

origin, and find the new system equations.
The equations are an example of a predator—prey population model
due to Volterra and used in biology, and are more general than the
simple linear rabbit—fox model of Exercise 1.4.

FIG. 5.4

Exercise 5.2.
Show by solving the equations explicitly and applying the definition that
the origin is a stable equilibrium point for the system

x1 oan2,9 eee."I te = Sat ke

with a <0, x; (to) = a, X2(to) = B.

150 Stability

Exercise 5.3.
If the oscillations of the simple pendulum in Example 5.7 are assumed
sufficiently small so that 67 and higher powers are neglected, and if
g/2 =k, obtain the equation of trajectories and sketch them in the phase
plane.

Exercise 5.4.
Verify that for the inverted pendulum problem in Exercise 1.3 the
origin is an equilibrium point.

5.2. Algebraic criteria for linear systems

5.2.1. Continuous time

Before studying nonlinear systems we return to the general continuous-

time linear system

x =Ax, (5.10)

where A is a constant n x n matrix, and (5.10) may represent the closed

or open loop system. The only equilibrium point of (5.10) is the origin,
so it is meaningful to refer to the stability of the system (5.10). The
two basic results on which the development of linear system stability
theory relies are now given.

THEOREM 5.1. The system (5.10) is asymptotically stable if and only

if A is a stability matrix, i.e. all the characteristic roots Ay of A have
negative real parts; (5.10) is unstable if any Re(Ax) > 0; and complet-
ely unstable if all Re(Ax) > 0.

Proof. From (3.7) the solution of (5.10) subject to x(0) =Xo is

x(t) =exp(Af)xo. (5.11)

From (2.76) with f(A) = e*! we have

q
exp (Af) = Zip t Zegtt ts * + Ziog tk 1) exp (Axt)
k=]
Stability 151
where the Zx¢ are constant matrices determined entirely by A. Using
properties of norms (see Section 2.7) we obtain

ial
llexp(At)I<> SSt2-1| exp(Age)
IlIlZeeIl
k=1Q=1
= > Dro Lexp[Rex)e]IhZeeIl (5.12)
= (Dewye S22

provided Re(Ax) < 0, all k, since (5.12) is a finite sum of terms, each of
which > Oast > ©. From (5.11)

llx(t) Il < |] exp (At) Il Ilxo Il,

so the system is asymptotically stable.

If any Re(Ax) is positive then it is clear from the expression for
exp (Af) that || x(t) || > cc ast > ©, so the origin is unstable.
If some of the Ax are purely imaginary the situation is more complicated:

THEOREM 5.2. Suppose all the characteristic roots of Ahave non-

positive real parts. Let A,,A2,..., Ag be those characteristic roots of
A having zero real parts, and let B; be the highest power of (A —Aj)
amongst the elementary divisors of J -A,i=1,2,...,&. Then the
origin of (5.10) is stable but not asymptotically stable if 8, = 6, =
+ +» = Bo = 1; unstable if there is at least one 6; > 1.
We do not give a proof of this result (see for example Lehnigk, 1966,
p. 20) but instead illustrate the important point that Theorem 5.2 does
not mean that the roots \;,..., Ag may not occur more than once if
the system is to be stable.

Example 5.8. The matrix

A, =
0) i

has characteristic root i with multiplicity two. The solution of (5.10) is

xX,(t) =c, exp (it), x2 (t) = cz exp (it)

152 Stability

(the c; are constants) so the system is clearly stable. Using the definitions
in Section 2.5 the elementary divisors of A, are found to be A-i, A-i
so 8B,=B, = 1.
If however we consider the matrix

A, =
1 i

the roots of A, are again i, i but the elementary divisors are 1, (A—i)*
so B, = 2. It is easy to verify that the solution of (5.10) in this case is

x(t) =c3 exp (it), x2(t) =cq exp (it) + c3¢ exp (it)
showing that the origin is indeed unstable.
In fact (see Exercise 2.24) the stability condition on the B; in Theorem
5.2 is equivalent to requiring that all the Jordan blocks J(\j),i=1,2,...,
in the Jordan form (2.59) of Amust have order one. This is clearly illus-
trated in the above example, where

A; =diag[J,(i),Jj(i) ’ A, =J, (i).

In particular if all the roots A,,..., Ag are distinct then the origin is
stable.

Example 5.9. In Example 5.2 the system matrix is

a sa!
A =

a3 aA

by inspection of (1.6). It is easy to show that

det(AJ-—A) =A(A-d)

using the condition a; /a3 = 4 /a4. Hence A has a single zero root, so the
system is stable provided d (=a, —aq) is negative.
The two preceding theorems apply if A is real or complex, so the
stability determination of (5.10) can be carried out by computing the
Ax using one of the powerful standard computer programs now available.
However, if 7 is small (say less than six), or if some of the elements of
A are in parametric form, or if access to a digital computer is not possible,
then the classical results given below are useful.
Because of its practical importance the linear system stability problem
Stability . 153
has attracted attention for a considerable time, an early study being by
Maxwell in 1868 in connection with the governing of steam engines. The
original formulation of the problem was not of course in matrix terms,
the system model being (3.42). This is equivalent to working with the
characteristic polynomial of A, which we shall write in this section as

det(QQU-—A)
=X?+a,;N?~
1 ++++tan _—1d+apn
=a(A),(5.13)
and this is identical to k(A) in (3.47) when the system equation is (3.42).
The first solutions giving necessary and sufficient conditions for all the
roots of a(A) in (5.13) to have negative real parts were given by Cauchy in
1831, Sturm in 1836, and Hermite in 1854, but we give here a well-known
theorem due to Hurwitz in 1895 for the case when all the a; are real (for
a good account of the history of the problem see Fuller 1974).

THEOREM 5.3. Then x n Hurwitz matrix associated with a(A) in

(5.13)
is

Qa, a3 as aed
1 az a4 G42n-—2
0 a a3 . ; G2n—-3
PaO, eT. cal inl)< 3 wototivy
ag tg? for(i)

where a, = 0,r >n. Let H; denote the ith leading principal minor of H.
Then all the roots of a(A) have negative real parts (a(A) is aHurwitz
polynomial) if and only if H; >0,i=1,2,...,7.
We shall not give a proof, which usually involves complex variable
theory (see Gantmacher 1959, 2 pp.177, 190). However, some remarks
on an alternative way of obtaining Theorem 5.3 will be given later in
Section 5.5.
Of course a disadvantage of Theorem 5.3 is the need to evaluate
determinants of increasing order, and a convenient way of avoiding this
is due to Routh whose work was published before that of Hurwitz, in 1877.
154 Stability

For simplicity consider the case = 3. It is then easy to verify that

1 0 0 ay, a3 0)
(- 1/a,) 1 0 1. @7
—a,/(a3 —a,a2) a," /(a3 a2) ] 0 ay a3

= 0 roy 122 (5.15)

wherery, =A,,r21 =H>/H,,7r3; =H3/H,.Similarlyin general

AH=R, (5.16)

where H is given by (5.14), and as in (5.15) the matrices A and R are

lower and upper triangular respectively. By considering leading principal
minors in (5.16) it can then be deduced that in general

Ty Sey Ty = fH 213 pee oy ee (5.17)

which leads to:

THEOREM 5.4. The criterion H; > 0,i=1,...,n of Theorem 5.3 is

equivalent to the condition that all the numbers 7;,,i=0,1,2,...,n
in the first column of the Routh array

Toi ‘or = Y03 : ; <

ry ae), VAs : é 3

Vr P7923 2 ‘

tis 12teem (5.18)

1 1 ee tte er ee 1
i]pe r Ele i] = * r i-2,/] r, 15 azi= 2, 5 telie ~
1 t ' 1
1i i=t 11.1 Ni-1,j 1i Ji—1,/) i 1ie -
tae ee ow wall Beemer eee om aA
Stability 155
be positive, where

{To1,
7025703;
- .# {1,42,a4,. . at
{71571257135---} {41,43rar ay,

M%-2,1 M-2,j7+1
file yet aeee eel, 20)
Fi-1,1 Ni-1,j+1

Thus each row in (5.18) is formed by taking all the 2 x 2 minors involving
the first column of the preceding two rows. This is shown by the elements
within dashed lines in (5.18). The computational effort involved is clearly
much less than for Hurwitz’s method.
As mentioned earlier, the Hurwitz and Routh theorems can be useful
for determining stability of (3.42) and (5.10) in certain cases. However,
it should be noted that a practical disadvantage of application to (5.10)
is that it is very difficult to calculate accurately the a; in (5.13). This is
‘important because small errors in the a; can lead to large errors in the
roots of a(A).

Example 5.10. Investigate the stability of the linear system whose

characteristic equation is

rt + 2A + 9A? +A 44.

The Routh array (5.18) is constructed from (5.19) and (5.20) to be

ro] l 2 :
rj 2 I
rj (18—1)/2 (8 —0)/2
= 17/2 a4
r3j [(17/2) —8] /(17/2)
=1/17
raj [(4/17) —0] /Q/17)
=4

Since allr;, are positive the system is asymptotically stable.

156 Stability

Alternatively, from (5.14) H; = 2, H, = 17,H3 = 1, H4 = 4 and it is

easy to see that (5.17) is satisfied.
Incidentally, if a(A)arises from a system in the form (3.42) having
leading coefficient a9 (>0) then the Hurwitz and Routh conditions still
hold but with the unit elements preceding the coefficient a2 in (5.14)
and (5.19) replaced by ao. Notice also that if any H; or 7;; is nonpositive
then the computational procedures of Theorems 5.3 and 5.4 need not be
continued since the system is not asymptotically stable. In fact it is then
possible to determine the numbers of roots of a (A) with positive and
negative real parts. For example, if no 7; is zero then these numbers are
respectively V and n —V, where V is the number of variations in
sign in the first column of the Routh array. For further details, including
cases when some r;; are zero, see Gantmacher (1959, 2 p. 180).
Since we have assumed that the a; are real it is easy to derive a simple
necessary condition for asymptotic stability:

THEOREM 5.5, If the a; in (5.13) are real and a(A) corresponds to an

asymptotically stable system, then

BPSOTSfee (5.21)
Proof. Any complex roots of a(A) will occur in conjugate pairs a + i,
the corresponding factor of a (A) being
(A —a —iB) (A —@
+ iB) = A? —20d + a? + B?.
By Theorem 5.1, a< 0, and similarly any real factor of a(A) can be
written (A + y) with y <0. Thus

a(d) =[TA + y)TTQ? —20d +a? + 6?) (5.22)

and since all the coefficients in (5.22) are positive the a; must also all be
positive.
Of course (5.21) is not a sufficient condition, but it provides a useful
initial check: if any a; are negative or zero then a(A) cannot be asympt-
otically stable. Furthermore, if (5.21) does hold then it turns out that
only about half the Hurwitz determinants in Theorem 5.3 need to be
calculated:

THEOREM5.6. (Liénard and Chipart). A necessary and sufficient

condition for a(A) to be a Hurwitz polynomial is that one of the following
conditions holds:
By 0, tg tae Oe te Ei Oy
aed (5.23)
 Stability £57
Ge 0 i) Hz > 0, Hy > 0, 20: (5.24)

gee 0. an-3>0,..4; Hy>0,H,>0,... (5.25)

ayes ty > 0; Gas US. es A, > 0} TaleAlUsgene (5.26)

For a proof see Gantmacher (1959, 2 p. 221). Notice that of course any
one of eqns (5.23)—(5.26) implies (5.21). Thus the Lienard-Chipart
criterion shows that if (5.21) holds then only 4 (n —1) or 7n of the Hj
in Theorem 5.3 need to be evaluated according as n is odd or even. In fact
it has been found recently that a reduction by half in the orders of the
determinants is also possible, and the development is interesting since it
involves companion matrices and is linked with the ideas of Section 4.3.
We have no room here to give details and refer the reader to Barnett
(1971, Chapter 2). We have mentioned earlier that for numerical
calculations the Routh array is preferable to the Hurwitz determinants.
However, if some of the a; depend upon parameters and the system is
asymptotically stable it is often required to determine for what values of
_ the parameters a (A) first becomes non-Hurwitzian. It can be shown
(Lehnigk 1966, p. 199) that these values are obtained by solving the
‘critical’ equations.
@n=0, Hny-1=0. (5.27)

Although encountered far less often in practice we also give a result for
the case when the a; in (5.13) are complex.

THEOREM 5.7. Let the real and imaginary parts of a; in (5.13) be

denoted by a;’ and a;” respectively. Then a (A) is aHurwitz polynomial if
and only if in the Routh array (5.18) with initial two rows

roj a=U1, a, Q@,

heels
,—~@2,43 ”,aq ies
,~45 Hh,
,~Oy,
Me wd,
(5.28)
rij =
={ar', ee a2 (ios eV ”",45 eae ,~Aeg
,—@3,@4 fire
,~ aq sree}

all the products

Ty1571121 1315711121 31 41 's1,--- (5.29)

are positive.
For a proof and other results on complex polynomials see Marden
(1966, p. 179).
158 Stability

5.2.2. Discrete-time
We now turn our attention to discrete-time linear systems in the usual

form
x(k +1)=A,x(k). (5.30)

The definitions of Section 5.1 carry over virtually without alteration.

We have seen in (3.33) that the general solution of (5.30) subject to
x (0) = Xo is
x(k) =A, "x .
In addition, we demonstrated in Section 2.7 that AS >Oask > if
and only if all the characteristic roots u; of Ahave modulus less than
unity, and Ak > cc if any |y;| > 1. It therefore follows that the result
corresponding to Theorem 5.1 is:

THEOREM5.8. The origin of (5.30) is asymptotically stable if and

only if A; is a convergent matrix, i.e. all its characteristic roots satisfy
iuj| <1. If any |y;| is greater than unity then the origin is unstable; and
completely unstable if all |u;| > 1.

Example 5.11. Consider the buffalo population model in Exercise

1.13. Following (3.69) introduce the coordinates
x1(k) = Fr, X2(k) = Fr + 1,X3(k) = Mx, x4(k) = Myan
so that (1.33) and (1.34) take the form (5.30) with
0 pred ARs ler

0.12 0.95 0 0
Ay 0 0 0 1

0.14 0 0 0.95

It is easy to obtain
det ( —A,)=A(A—0.95) (A? —0.95A —0.12)
=A(A- 0.95) (A- 1.063) (A + 0.113)
so the system is unstable since A, has a root greater than unity.
Just as in Theorem 5.2, if any of the characteristic roots of Ay have
unit modulus then the corresponding elementary divisors must be linear
for (5.30) to be stable.
Stability 159
In fact we can establish a direct relationship between Theorems 5.1
and 5.8 by taking
Mia (Ay 1) (A; +1) * (5.31)

assuming A, +J is nonsingular, i.e. that A, has no yu; =—1.Then the

roots A of A in (5.31) satisfy
oe 8
pti (5252)
and it is left as an easy exercise for the reader to show using (5.32) that
Re (A) <0 if and only if |u| <1,so A in (5.31) is the matrix in Theorem
5.1. In other words, A is a stability matrix if and only if A, is a conver-
gent matrix.
If we write the characteristic polynomial of A, in the form

ay (u)=aou" top" 1 +++++a,, a >0, (5.33)

then a; (1) is also termed convergent if all its roots have modulus less than
unity. A convenient scheme for determining whether a, (1) is convergent,
corresponding to the Routh array of Theorem S.4, is as follows:

THEOREM 5.9. A necessary and sufficient condition for a, (wu)to be

a convergent polynomial is that
ae 0, dj, < OP T= 3) Pon +'T, (5.34)
where the dj, are first column elements in the array

Qo Qy : C ¢ O—-1 ON

On On —1 o as Oy Xo

C21 C22 ‘ : s Con

dy dz2 : E ¥ dan
C31 C32 = i C3>n-1

Pee ie pont 7Mca (5.35)

(Sis Be Sut
Kili ihe HO Ci-1,n—i+3
ps otiili acer eco
paagee
aeei
y : Cj Cig +1 :
160 Stability

and
nes Cit da cled i= 2,3,...,% #9 (5.36)
“ij Gi-i,1 Ge a,p44}?

Gy=. Lin=j-it3, (5.37)

with
Cee ENS [6 OUT (NE
ge (5.38)
For a proof of the theorem see Jury (1964, p. 121). The scheme above
for constructing the array has been modified slightly from Jury’s form so
as to expose the similarity with the Routh array (5.18). The rows in
(5.35) are formed in pairs, the second row in each pair being given by
(5.37) as merely the first in reverse order. The elements in the first row
in each pair are formed by taking all the 2 x 2 minors (5.36) involving
the first column of the preceding two rows, as in (5.20).
The reader will notice the similarity between the tabular schemes of
Theorems 5.4 and 5.9, and may wonder why divisions are necessary in
(5.20) but not in (5.36). In fact, although this is very seldom pointed
out in the literature, either method may be formulated with or without
divisions (see Exercise 5.16).
Again a generalization is possible to find the number of zeros of a; (u)
outside the unit disc | | <1 (excluding the boundary). If no d;; = 0, let

Py =(—1¥'d2; day... dy +1,1,

k= 1, 2 ero)

Then the numbers of roots of a, (A) inside and outside the unit disc are
Nand n-—VN,where N is the number of negative products Py defined in
(5.39). For further details and other results, including a determinantal
criterion corresponding to that of Hurwitz in Theorem 5.3, the book by
Jury (1964, Chapter 3) should be consulted.

Example 5.12. Determine the distribution of the roots of

Qu?+ 4u? —5u +3 (5.40)

with respect to the unit disc.

 Stability 161
This array (5.35) is constructed using (5.36), (5.37) and (5.38) as

Cj 2 4 7) 3
dij 3 2) 4 2
C2;
dj =22
=> as)
23 ma)|
ree

C3; cae: ai
d3; 459 Sal
Cay —57800
= da

Thus in (5.39), P; > 0, P, <0, P3 <0 so (5.40) has two roots inside
the unit disc and one outside, and is therefore the characteristic
equation of an unstable discrete-time system by Theorem 5.8.
When the a; in (5.33) are complex the only change required is that
the second row of each pair in (5.35) is the complex conjugate of the
| preceding row in reverse order.

5.2.3. Time varying

When we turn to linear time varying equations

x (t)=A(t)x(t) (5.41)

the situation is much more complicated. In view of Theorem 5.1 it might

be thought that if the characteristic roots of A(t) all have negative real
parts for all ¢>t, then the origin of (5.41) would be asymptotically
stable. Unfortunately this conjecture is not true (see Exercise 5.15).
However, if A (ft) is almost constant, i.e.
Lim
A(t) =A co , (5.42)
[—->°o

where A oois a constant matrix, then the following result can be establish-
ed.
THEOREM5.10. If the origin as an asymptotically stable equilibrium
point for the system

£(t) = Acox, (5.43)

iael
Sd
162 Stability

then it also is for

£ (1) = (Aco B(t)) x (5.44)

provided
rim BO I=0. (5.45)

Generally, asymptotic stability of (5.43) does not ensure that of (5.44)

unless an additional condition like (5.45) holds. For a proof of the
theorem see Willems (1970, p.113), and this book together with that by
D’Angelo (1970) should be consulted for further results.

Exercise 5.5.
Analyse the problem in Exercise 5.2 using Theorems 5.1 and 5.2.

Exercise 5.6.
Determine whether the following polynomials are Hurwitzian:
(aq) 8 +1727 +2041 (b) 04 +23 +40? +4043
(c) 044643427 +A43 @ 405 +5074+2602 430K IX + 2.

Exercise 5.7.
Determine the solution & (f) of the linear system

X(t) = Ax(t) —b, x(0) =X,

where b is a constant column n-vector, by using (3.22). Hence deduce
that if A is a stability matrix then the vector Lim ¥&(F) is the solution of
t > 00

the system of algebraic equations Ax = b,

Exercise 5.8.
Determine the location of the roots of
pt +2? +p? +3ut+2

with respect to the unit disc.

Exercise 5.9.
Show that necessary and sufficient conditions for the real polynomial
a(A) to be Hurwitzian are (a) when n = 2: a; > 0,a, >0(b) when
n=3:a, >0,a3 >0,a,a, —a3 >0.
Stability 163
Exercise 5.10.
Show that necessary and sufficient conditions for the real polynomial
a;(u) to be convergent when n = 2 are a, —A <0,a,(1)>0,
a; ew:

Exercise 5.11.
Determine for what range of values of k the polynomial
(3 —kK)A?+ 2d? + (5-2k)A +2

is Hurwitzian.

Exercise 5.12.
Test the polynomial
AP + (1 +i)? +(2-3IA +7,

where i = \/(— 1), using Theorem 5.7.

Exercise 5.13.
The characteristic equation
d? + 2kwAr+ (w? —2kwMi) = 0,

where i = ./(— 1) arises in the theory of vibrating shafts, Q being the

rotational speed, w the undamped natural frequency and k a damping
coefficient. Use Theorem 5.7 to show that the system is asymptotically
stable if and only if X<w.

Exercise 5.14.
Determine for what range of values of k the real system

) ] @)

x=! 0 0 Led eran

=H =! =)

is asymptotically stable. If k =—1, and a control term

i u

is added, find a linear feedback control which makes all the characteristic
roots of the closed loop system equal to — 1.
164 Stability

Exercise 5.15.
Verify that the characteristic roots of

-4 oC.
A(t)=
—e%! 0

are both constant and negative, but that the solution of (5.41) diverges as
[ieree?ok

Exercise 5.16.
Define a Routh array (s;;) without divisions by soj = oj, S17 ="17

Si-2, 1 Si-2,j+1
Sa : i=2;53.00
Si-1,1, Si-1,j+1
where the 7j; are given by (5.19) and (5.20). Show that this corresponds
to taking 827 =/oj, S37="ualsj, Sig= Kitz, 1> 2 where
k; = kj_ki_ 271-1, 1,12 4. Hence deduce that the Hurwitz determin-
ants are given by H, =54,,H» =821,H3 =53,, Hj = 5j1/8;, i > 4 where

Ne | Sy, (i even)
i**i-1 Si-3,1Si-5,1 S11(i odd).

5.3. Nyquist criterion for linear systems

We now consider a general closed loop single-input single-output system

with unity feedback, represented in Fig. 1.10. The closed loop transfer
function in (1.15) becomes
&(S) Sages ‘ (5.46)

where g(s) is the open loop transfer function defined in (1.14) by

&(s) = B(s)/K(s). (5.47)
Substitution of (5.47) into (5.46) gives
ews) de
Bel)=5) +KS)” (5.48
Stability 165
The characteristic equation of the closed loop system is the denominator
in (5.48) (see Exercise 5.17), so stability can be tested by applying, for
example, the Routh criterion of Section 5.2 to the polynomial B(s) + k(s).
In practical applications, however, the coefficients of 6and k may not be
known precisely and in such cases the criterion due to Nyquist is useful.
It involves drawing a diagram in the complex plane of g(iw), the open
loop frequency transfer function (defined in Section 1.2), which can
usually be measured experimentally over a required range of values of w.
We shall assume to begin with that the open loop system is asymptot-
ically stable, i.e. that all the zeros of k(s) in (5.47) have negative real parts,
so g(s) has no poles in the right half of the s-plane (including the imaginary
axis) T'»:Re(s) =>0. We can then regard g(s) as a conformal mapping from
I, onto the g(s)-plane. That is, for each point z = o + iw in the complex
s-plane with o > 0 we have a corresponding point U + iV = g(z) in the
g(s)-plane.

Example 5.13. Consider the simple transfer function

60=4 (5.49)
The mapping of the imaginary axis s = iw, -° < w < is given by

Ut+iV=-
iw t+]

so
U=(1+w?)"?, V=-w(lt+w?)!

and it is easy to verify that

(U-2) as = 2.

This shows that the mapping of the imaginary axis in the s-plane is a
circle in the g(s)-plane, centre 3 + i0, radius 2 (see Fig. 5.5). As w
goes from zero to © the semicircle from A to B is traversed. Since

&(- iw) = g(iw)

it follows that as w goes from —© to zero the path is simply the complex
conjugate of BA, shown by the dashed curve in Fig. 5.5.
166 Stability

We do not have space in this book to devote to the theory of conformal

mapping, for which the reader is referred to standard texts on complex
variables (e.g. Levinson and Redheffer 1970).
Suppose that the half-plane I’, is mapped by g(s) into some region R.
If any zero py;of B(s) + k(s) lies in T, then g(u;) will be contained within
R, and the closed-loop system is not asymptotically stable (Theorem 5.1).

s-plane g(s)-plane

FIG. 5.5

From (5.47) it follows that 6(s) + k(s) = 0 is equivalent to g(s) =— 1, so

py satisfies g(u;) = —1. It therefore follows that for the closed loop
system to be asymptotically stable, the point —1 + i0 must lie outside
R. The half-plane I’, can be represented by the interior of the semi-
circular region shown in Fig. 5.6 where the radius 7 tends to infinity.

FIG. 5.6
 Stability 167
Any point on the curved part of the contour can be written in polar
form as s =r exp(i@) and substitution into (5.47) gives

g(s)=B(re!®)/k(re!®). (5.50)
If we assume, as usually happens in practice, that g(s) is proper, then
58 < 8k and in (5.50) | g(s) |> 0 asr > ©. Thus the semicircle ofinfinite
radius is mapped into the origin of the g(s)-plane. We need therefore
only consider the imaginary axis in Fig. 5.6, and the Nyquist locus
(or diagram) of g(s) is defined as the curve

U=Refg(iw)}, V=Im{g(iws)} (5.51)

for -—-°<w<, If g(s) has real coefficients then as in Example 5.13,
g(—iw) = g(iw) and the part of the mapping corresponding to the
negative imaginary axis (—°° < w <0) is the complex conjugate of the
curve for 0 <w< %. We have therefore established, albeit informally,
the Nyquist criterion:

THEOREM 5.11. If g(s) has no poles in Re(s) = O then the closed

loop system (5.46) is asymptotically stable if and only if as w passes
_ from-to % the Nyquist locus of g(s) does not encircle the point
meh) (8)
As a simple application of Theorem 5.11 we can see that there is no
encirclement of —1 + i0 in Fig. 5.5, so the corresponding closed loop
system is asymptotically stable. In fact this is very easily verified since
from (5.46) and (5.49), g-(s) = 1/(s + 2).
If g(s) has one or more poles on the imaginary axis then the contour
in Fig. 5.6 must be modified as shown in Fig. 5.7, the circles on the
w-axis having radius p > 0.

FIG. 5.7
168 Stability

The Nyquist locus is now the path traced out by g(s) as s traverses the
indented imaginary axis.
Example 5.14. Consider

1
a(s) = s(s + 1)

which has a pole at s = 0.

(a) (b)

On be, s = iw so that

g(s)= eer

giving
U=-(1+w?)!, 9 V=—-{w(1
+o)?
whichisthepathb’c’in Fig.5.8(b),Onab,s =pexp(id)withp> 0, so
Rim
cues p>0 pel®(1+pel?)
lim 1

pe pel?
Stability 169
Thus as p > 0, g(s) has modulus 1/p, argument —0 with O<@<7n/2. It
follows that a’b’ is the mapping of ab. The reflection in the U axis,
corresponding to the path eda, is shown in dashed lines. The point
—1 +10 is not encircled so the system is asymptotically stable. Again it is
easy to verify that
ee a
Eels)= ststl1-
and the characteristic polynomial is asymptotically stable (see Exercise
5.9(a)).
Clearly it is sufficient to sketch only the salient features of the Nyquist
locus, a time-consuming detailed diagram usually not being necessary.
A result more general than Theorem 5.11 for the case when the open-
loop system is not asymptotically stable can also be proved. The argument
is based on Cauchy’s residue theorem and can be found in most textbooks
on classical linear control theory.

THEOREM 5.12. The closed loop system with transfer function (5.46)
is asymptotically stable if and only if
N=-PS<0

where NV= total number of clockwise encirclements of —1 + i0 by the

Nyquist locus of g(s);

P = number of poles ofg(s) in Ty’:Re(s) > 0.

Furthermore, if V > 0, the number of zeros of 1 + g(s) (i.e. the number
of closed loop poles) in I,’ is equal to N+ P.
It should be remarked that if P# 0 then it would not be feasible to
measure g(iw) experimentally, since any open loop characteristic roots
with positive real parts would lead to the open loop output growing
without limit as t > °°.
Care must be taken in interpreting the term ‘encircle’ in Theorems
5.11 and 5.12. Precisely, a closed contour C is said to make n clockwise.
encirclements of a point D if a radial line drawn from D to a point on C
rotates in a clockwise direction through 27n radians in going completely
round C,

Example 5.15. Consider a Nyquist diagram of the form below, shown

for simplicity only for w going from zero to infinity (Fig. 5.9).
The reader should sketch the complete Nyquist locus and convince
himself that the number of encirclements of the point —1 + iO is zero.
170 Stability

FIG. 5.9

When applying Theorem 5.11 it is only necessary to know whether

—1 +0 is enclosed by the contour, and a convenient way of determining
this is to hatch the region to the right of the locus as it is traversed in the
prescribed direction (see Fig. 5.9). All points within the hatched region
are enclosed by the curve.
In practical problems it is often not sufficient to know merely that a
linear system is asymptotically stable. This certainly ensures that the
effect of perturbations from equilibrium eventually dies away, but it is
usually required in addition that this should not take ‘too long’. More
precisely, the real parts of the closed loop system characteristic roots
should be less than some given negative number o, so that the exponential
terms in the solution decay at least as fast as exp(ot). The Routh array
and other methods of Section 5.2 can be adapted to deal with such prob-
lems of relative stability, but the Nyquist diagram provides a convenient
approach,
Example 5.16. Suppose that g(s) in (5.46) is replaced by kg(s) and
that for k = 1 the Nyquist locus for a given system is as shown in Fig. 5.10.
If g(s) has no poles in I, then by Theorem 5.11 the closed loop system
is asymptotically stable for k = 1. Since the locus for any value of k is
given by (5.51) as

U=k Re{g(iey}, V=k Im{g(iw)},

then as k increases, the point where the locus meets the negative U-axis
moves from —4 + i0 when k = 1 to— 1 + i0 when k = 4. Thus the system
is asymptotically stable for k <4.
Stability 171

FIG. 5.10

If a circle with unit radius and centre the origin cuts the locus at the
point P, then the whole Nyquist locus for k = 1 can be rotated clockwise
through 70° before the point —1 + i0 is reached and the system is no
longer asymptotically stable.
The quantities 1/(4) = 4 and 70° are called the gain and phase margins,
and thus provide a measure of how far the system is from instability (in
this case when k = 1). In practice a gain margin of at least two and a
phase margin of at least 30° are desirable as ‘safety factors’.
The Bode plot (mentioned in Section 1.2.1) can also be used to det-
ermine gain and phase margins, since it presents essentially the same
information as the Nyquist diagram in a different form.
In general, if g(s) is replaced by kg(s) in (5.46) the point —1 + i0 in
the g(s)-plane in Theorems 5.11 and 5.12 is replaced by —1/k + i0.
It is worth mentioning that in Fig. 5.9 the ideas of gain and phase
margins are not very meaningful. For example, the system becomes
unstable if the gain is increased so that the point A is to the left of
—1, or if the gain is decreased so that B is to the right of —1. Such a
system is called conditionally stable.
172 Stability

Nyquist diagrams can be developed for discrete-time linear systems

(Saucedo and Schiring, 1968, p. 466) and also under certain conditions
for systems whose transfer function is not rational (Willems 1970, p. 79).
To summarize, the Nyquist criterion enables the closed loop stability
to be determined from the open loop frequency transfer function, even
if this is not known analytically and can only be determined experi-
mentally. The technique is also useful if stability is to be investigated
as a system parameter varies. For these and other reasons the Nyquist
method is a widely used practical tool for both analysis and design of
systems, and complements the algebraic criteria of the preceding section.
We shall also see in Section 5.7 that Nyquist-type loci can be applied
to certain cases of nonlinear feedback.
We close this section by mentioning briefly some recent extensions
to deal with systems having vector inputs and outputs. If

F(s) =I, + G(s)H(s) (5.52)

denotes the return-difference matrix in (4.40) with r = m, and the
Nyquist path is in Fig. 5.7, then the closed loop system is asymptotically
stable if and only if NV,=—P,,where NV,is the number of clockwise
encirclements of the origin by the Nyquist locus of det F(s) and P, is
the number of zeros of the open loop characteristic polynomial det (s/ —.A))
in T,’. The characteristic roots tx(s) of the m x m matrix T(s) = G(s)H(s),
defined in the usual way by det (AJ —7) = 0, will themselves be functions
of s. Thus by (2.29) and Exercise 2.9(c)

det F(s) = I (1 + tx(s))

and the Nyquist contours of t,(s) are called the characteristic loci, It can
be shown that the necessary and sufficient condition for closed loop
asymptotic stability becomes

m
>»nas Py
k=1
where nx is the number of clockwise encirclements of —1 + i0 by the
Nyquist locus of t;(s).
When H(s) =Jm in (4.40) the closed loop transfer function matrix is

Ge(s)= Um+ G(s)]™*G(s)

Stability t73
so that
[Go(s)T*= [G()* +Im.
A sufficient condition for closed-loop stability can be inferred from the
inverse Nyquist array, which is the name given to the set of m? Nyquist
loci corresponding to the elements of [G(s)] ~?.
These ideas have been developed into design techniques for multi-
variable systems, and the comprehensive survey by MacFarlane (1972)
should be consulted for further details, which are outside the scope of
this book.

Exercise 5.17.
Verify by substituting (5.47) into (4.41) that the characteristic equation
of the closed loop system is B(s) + k(s).
Exercise 5.18.
Determine the number of clockwise encirclements by the contour shown
below of the points (a) the origin (b) —1 + iO.

FIG. 5.11

Exercise 5.19.
Sketch the Nyquist diagram for

g(s)=
5. 1)
and hence determine the number of closed loop poles in Ty’.
174 Stability

Exercise 5.20.
Sketch the Nyquist locus for the open loop transfer function

Qs ae
& s*(s - 1) ’

and determine the stability of the closed loop system.

Exercise 5.21.
Draw the Nyquist locus for

k
8) =(4 10)
Hence calculate the gain and phase margins when k = 800/3.
Verify that your result for the gain margin is correct by applying the
Routh criterion to the closed loop characteristic polynomial.

5.4. Liapunov theory.

We shall develop the so-called ‘second’ or ‘direct’ method of the Russian

mathematician Liapunovi in relation to the autonomous system of non-
linear equations

x =f), f(O) = 0, (5.53)

subject to x(to) = Xo, but modifications needed to deal with (5.1) are
straightforward. The aim is to determine the stability nature of the equi-
librium point at the origin of (5.53) without obtaining the solution x(t).
This of course has been done algebraically for linear time invariant
systems in Section 5.2. The essential idea is to generalize the concept of
energy V for a conservative system in mechanics, where a well-known
result states that an equilibrium point is stable if the energy is aminimum.
Thus V is a positive function which has V negative in the neighbourhood
of a stable equilibrium point. More generally, we define a Liapunov func-
tion V(x) as follows:
(i) V(x) and all its partial derivatives 0V/dx;are continuous.
(ii) V(x) is positive definite, i.e. V(0) =0 and V(x) > 0 forx #0
in some neighbourhood || x || <k of the origin.
(iii) the derivative of V with respect to (5.48), namely

+Also transliterated in the literature as Lyapunov or Liapounoff.

Stability 175

Ory era? OXyae

OV oV OV
=—aicealeae
f+-— er CSO wate
eC Sr (5.54)

is negative semidefinite (i.e. V(0) = 0, and for all x in || x || <k,

V(x)<0.
Notice that in (5.54) the f; are the components of f in (5.53), so V
can be determined directly from the system equations. The definitions
of positive or negative definiteness or semidefiniteness are generalizations
of those for quadratic forms (see Section 2.8). The statements of the two
basic theorems of Liapunov are remarkably simple:
THEOREM 5.13. The origin of (5.53) is stable if there exists a
Liapunov function defined as above.

THEOREM 5.14. The origin of (5.53) is asymptotically stable if there

exists a Liapunov function whose derivative (5.54) is negative definite.

Proof of Theorem 5.13. This relies on the fact that because of the sign
property of V there exists a continuous scalar function $(7) of r which
vanishes at r = 0 and increases strictly monotonically? in0 <r<k such
that

V(x) > (Il x Il). (5.55)

Let e > 0 be given and suppose Xo can be chosen so that the inequalities
Ilxo Il<e, V@o)<¢ (©) (5.56)
are simultaneously satisfied. Since ¢(€) > 0, V(O) = 0 and V(x) is cont-
inuous, a suitable vector satisfying (5.56) can in fact always be found
by taking x9 sufficiently close to the origin. Also, since V<0,

V[x(t1)] <VEeCo)],t1 2 to
< $(€), by (5.56). (5.57)

If there exists some f, >to such that || x(t, ) || >€ then by (5.55)
Vix(t)) >oil x(t) Il
> > (€)

Fie. o(7,) > o(7,) ifr, >r,

176 Stability

which contradicts (5.57). Hence for all t; > fo we have || x(t, ) || <<e, so
the origin is stable since 6 = € in the definition in Section 5.1.
The proof of Theorem 5.14 is similar (see Hahn 1963, p. 15). It is
worth noting that if the conditions on V in Theorem 5.14 hold every-
where in state space it does not necessarily follow that the origin is
asymptotically stable in the large. For this to be the case V(x) must
have the additional property that it is radially unbounded, which means
that V(x) > © for all x such that || x || > °°. For instance, V=xj{ + x3
is radially unbounded, but V=xj /(1 + x7) + x3 is not since, for
example, V> 1 asx; >°,x 0. A similar line of reasoning shows
that if Q is the set of points outside a bounded region containing the
origin, and if throughout 2, V >0, V <0, arid Vis radially unbounded
then the origin is Lagrange stable (Willems 1970, p.34).
It may be helpful to consider a geometrical interpretation of Theorem
5.14 when n = 2.

FIG. 5.12

Since V is positive definite the curves V = constant must enclose the

origin. They can be thought of as contours of a cup-shaped surface.
Since V <0on any trajectory, each trajectory must tend towards the
origin as t > 0 (V ‘falls to the bottom of the cup’).

Example 5.17. Consider a unit mass suspended from a fixed support

by a spring as shown in Fig. 5.13,z being the displacement from equilibrium.
Stability LTT

FIG. 5.13

If first the spring is assumed to obey Hooke’s law then the equation of
motion is

Ztkz=0, (5.58)
where k is the spring constant. Taking x; =z, x2 =Z, (5.58) becomes
DeeSa xs = ho: (5.59)
Since the system is conservative the total energy

=4 kx? +4 x2 (5.60)
is a Liapunov function and it is very easy to see from (5.59) that
E = kx, (x2) +x2(—kx) =0,
so by Theorem 5.13 the origin is stable (of course this is trivial since
(5.58) represents simple harmonic motion). Suppose now that the force
exerted by the spring, instead of being linear is some function k(x, )
satisfying k(0) = 0, k(x, ) #0 if x, #0, so that the second equation in
(5.59) becomes x, = —k. The total energy is now

E=4x2+
fc k(a)do (5.61)
and
E=x,(-k) +kx, =0,
so again by Theorem 5.13 the origin is stable for any nonlinear spring
satisfying the above conditions.

Example 5.18. Consider now the system of the previous example but
with a damping force dz added, so that the equation of motion is

Z+ditkz=0 (5.62)
178 Stability

instead of (5.58). Equation (5.62) can also be used to describe an LCR

series electric circuit, motion of a gyroscope, and many other problems.
Assume first that both d and k are constant, and for simplicity let
d=1,k =2. The system equations in state space form are

a = X25 X> =-2x, mx (5.63)

and the total energy is still Fin (5.60), so that using (5.63)

E = 2x4 (x2) +X0(- 2x1 —X2)

=—x3

which is negative semidefinite, so by Theorem 5.13 the origin of (5.63)

is stable. However, now consider the function

V = Tx¥ + 2x4x9 + 3x3. (5.64)

From (5.63)

V= 14x43
(2) + 2[x2(2) +%1©24y ee
+.6X9)(S2X4)ee)
=—4x? —4x3,

Clearly V is negative definite and it is easy to verify that the quadratic

form in (5.64) is positive definite (see Section 2.8), so by Theorem 5.14
the origin of (5.63) is asymptotically stable (in fact, in the large).
This example illustrates that a suitably-chosen Liapunov function can
provide more information than the energy function. However, when V
is only negative semidefinite the following result is often useful.

THEOREM 5.15, (Barbashin). If in Theorem 5.13 there exists a Liapunov

function such that V does not vanish identically on any nontrivial trajec-
tory of (5.53), then the origin is asymptotically stable.

Example 5.19. Consider again the damped mass-spring system described

by (5.62), but now suppose that both d and k are not constant. Let k(x, )
be as defined in the second part of Example 5.17, and let d(x) have the
property d(x,) >0,x2 #0,d(0) =0. The state-space equations are

ty =exepPxge-*k =i (5.65)
Stability 79
so if F is defined as in (5.61),

E=x,(-k-dx,)+kx,
=-—x3d<0.

NowE is positive definite and E vanishes only when x2(t) = 0, which by

the second equation in (5.65) implies k(x, ) = 0, which in turn implies
x(t) = 0. Thus £ vanishes only on the trivial solution of (5.62), and
so by Theorem 5.15 the origin is asymptotically stable.

Example 5.20. The Van der Pol equation

Z + e(27—1)2 +z =0, (5.66)

where € is a negative constant, arises in a number of engineering problems.

In a control context it can be thought of as application of nonlinear
feedback

u=—z+e1—z7)z

to the system described by Z =u.

As usual take x; =Z,X2 =2Zto transform (5.66) into

RE NXo
Xe =H —(x7 —1)ra. (5.67)

Try as a potential Liapunov function V =xj +x3 which is obviously

positive definite. Then

V= par ar + 2x2X2
= 2ex3 (1 —x?).

Thus V <0ifx,? <1, and then by Theorem 5.15 the origin is asympto-
tically stable. It follows that all trajectories starting inside the region
Tl:x? +x3 <1 converge to the origin as t > ©, and I’ is therefore called
a region of asymptotic stability. The reader may be tempted to think
that the infinite strip x} < 1 is a region of asymptotic stability. This is
not in fact true, since a trajectory starting outside [ can move outside
this strip whilst continuing in the direction of decreasing V circles, and
hence lead to divergence (see Fig. 5.14).
180 Stability

FIG. 5.14

In general if a closed region R defined by V(x) < constant is bounded

and has V negative definite throughout then R is a region of asymptotic
stability.
Suppose that we now take as state variables

x1 =2Z, X3= [zat

The correspondingstate equations are X; =x, and

X, =2
=— if Zdt +e f, (z? —1)zdt (integrating(5.66))

=-x3—€(} 2° —2)
aes -e(3 x} —X)).
HenceusingV=x7+x3,
V=2x, (—x3
—3ex? +ex,) +2x3x,
=2ex}(1-3 x7)
<0 if x7<3,
so the region of asymptotic stability obtained by this different set of
state variables is x} + x3? <3, larger than before.
Stability 181
In general if the origin is an asymptotically stable equilibrium point
then the total set of initial points from which trajectories converge to
the origin as t > © is called the domain of attraction. Knowledge of this
domain is of great value in practical problems since it enables permissible
deviations from equilibrium to be determined. However, Example 5.20
illustrates the fact that since a particular Liapunov function gives only
sufficient conditions for stability, the region of asymptotic stability
obtained can be expected to be only part of the domain of attraction.
Different Liapunov functions or different sets of state variables may
well yield different stability regions. We shall discuss methods of con-
structing Liapunov functions for nonlinear systems in Section 5.6, but
the general problem of finding ‘optimum’ Liapunov functions, which
give best possible estimates for the domain of attraction, is a difficult one.
It may be a waste of effort trying to determine the stability properties
of an equilibrium point, since the point may be unstable. The following
theorem is then useful:

THEOREM 5.16. Let a function V(x) with V(0) = 0 have continuous

first partial derivatives. If there is some neighbourhood containing the
origin in which V takes negative values, and if in addition V in (5.54) is
negative semidefinite, then the origin of (5.53) is not asymptotically
stable. If V is negative definite the origin is unstable, and if both V and
V are negative definite the origin is completely unstable.
The proof follows along similar lines to those of Theorems 5.13 and
5.14. Notice that in the first two parts of Theorem 5.16 we do not need
V
itself to be negative throughout the neighbourhood, only at some
points arbitrarily close to the origin. Also, as in Theorem 5.15, if Vis
negative semidefinite but V# 0 on any non-trivial trajectory then the
origin is unstable. It is perhaps worth mentioning here that in all the
Liapunov theorems the terms ‘positive’ and ‘negative’ can be inter-
changed simply by using (—V) instead of V. It is only the relative signs
of the Liapunov function and its derivative which matter.
To close this section we point out that all the theorems in it are
applicable to discrete-time systems

x(k + 1)=f {x(k}, (5.68)

the only difference being that the derivative of V is replaced by its
difference
AV =V[x(k + 1)] —V[x(K)]
= V{fix(K))}
—Vie(K)]. (5.69)
182 Stability

Exercise 5.22.
Write the system

2+2z+27=0
in state space form. Let
V =axt+ bx? + 0x x, + dx}
and choose the constants a, b, c, d such that
7 = —x4 —x2,

Hence investigate the stability nature of the equilibrium point at the

origin.

Exercise 5.23.
Using the function

V = 5x? + 2x1X + 2x2

show that the origin of
Ky =Xy,%_ =—Xy—Xq+ (x1 + 2x2(x3- 1)
is asymptotically stable by considering the region |x. |< 1. State the
region of asymptotic stability thus determined.

Exercise 5,24,
Investigate the stability of the origin for the system

eso) oes =—2X1X2

X1=X]—X3,X_
usingthe function V= 3x,x3 —x}.

Exercise 5.25.
For the equations in Exercise 5.1, investigate stability of the origin
using V= x3 —x?.
If in Exercise 5.1 the transformed system variables are y, and y2,
use

V=yy+2y2—2kn
(1+ 41) —-Ln
(1+2y2)
to investigate the stability of the second equilibrium point.
Stability 183

Exercise 5.26.
Use the function V =x? +x} to show that the origin of the system
ero eet 2x2, X_ =-2x1 —X2

is asymptotically stable in the large. If a term —x2qis added to the right

hand side of the second equation, find conditions on q(x;, x7) such that
the stability nature of the origin is unaffected.

Exercise 5.27.
Find the domain of attraction for the origin of the system described by
the scalar equation

2=% 2(z—2).

Exercise 5.28.
The equations of motion of a gyroscope without external forces are

Aw, +(C-B)w,w; =0
Bo, +(A-C) w30, =0

Ca 3 + (B-A) w, Ww,=0
where A, B, C are the principal moments of inertia and w,, w2, Ws are
the angular rates about the principal axes. Put w, = Wo +X1, W2 =X2,
W3 =X3 (where Wo is a constant) and show that the origin of coordinates
is then stable if A < B<<C by using the function

Vie Ajxs + C(C—A)x? + [Bx2 +Cx3 +A (x7 +2x, W)]?

(For further study of stability of gyroscopes see Leipholz (1970, p. 155);
see also Exercise 5.54.)

Exercise 5.29.
Show that if the origin is an asymptotically stable equilibrium point for
(5.53) with Liapunov function V (x), then it also is for

X=f(x) + [S)-Q(&)] 6 &)

where (x) =V V, Q(x) is any positive semidefinite n x n matrix and
S(x) is any skew symmetric n x n matrix, assuming that conditions for
existence and uniqueness of solution still hold.
184 Stability

Apply this result to the system in Exercise 5.23 by choosing S=0 and
a suitable matrix Q(x) to show that the second equation can be replaced
by
F2=—xX, —X2+ (&, + 2x2)[h@1,%3) +25

where h(x,, x2) is any continuous function satisfying h(x;,x2) <0.

5.5. Application of Liapunov theory to linear systems

We now return to the real linear time invariant system

X=Ax. (5.70)

In Section 5.2 we gave criteria for determining asymptotic stability via

the characteristic equation of A. We now show how Liapunov theory can
be used to deal directly with (5.70) by taking as a potential Liapunov
function the quadratic form

ViextPx (S.71)
where P is a real symmetric matrix. The time derivative of Vwith respect
to (5.70) is

where
A'P+PA=-Q, (5.72)
and it is easy to see that Q is also symmetric. If P and Q are both positive
definite then by Theorem 5.14 the (origin of) system (5.70) is asymptot-
ically stable. If Q is positive definite and P is negative definite or indefinite
then in both cases V can take negative values in the neighbourhood of the
origin so by Theorem 5.16, (5.70) is unstable. We have therefore proved:

THEOREM 5.17. The real matrix A is a stability matrix if and only if

for any given real symmetric positive definite (r.s.p.d.) matrix Q the
solution P of the continuous Liapunov matrix equation (5.72) is also
positive definite.
Notice that it would be no use choosing P to be positive definite and
Stability 185
calculating Q from (5.72). For unless Q turned out to be definite or
semidefinite (which is unlikely) nothing could be inferred about asympto-
tic stability from the Liapunov theorems. If A has complex elements then
Theorem 5.17 still holds but with P and Q in (5.72) Hermitian, AT
replaced by A*. A direct algebraic proof of Theorem 5.17, without
relying on Liapunov theory, has also been given including an extension
to the numbers of characteristic roots of A with positive and negative
real parts. Furthermore, by a suitable choice of P and Q in (5.72) it is
possible to deduce the Hurwitz stability criterion of Theorem 5.3
(Lancaster 1969, p. 272), so that acompletely algebraic proof of this
result can be achieved. In fact the sufficiency part of Theorem 5.17 is
very easily shown. Let A be complex, and let \ and w be any correspond-
ing characteristic root and vector. Then from (2.30)

Aw =)hw, w*d* =) w*, (5.73)

and using(5.73) and the complex form of (5.72) we obtain

w*Qw =—w*(A*P+ PA)w
=—(A+ d)w*Pw.

Henceif P and Q are both positive definite,

Re(d) = + (AX
+2) =—3 w*Ow/w*Pw<0,
showingA is a stability matrix.
Eqn (5.72) is a special case of (2.37) with B= A’. Since the character-
istic roots of A? are the same as those of A (see Exercise 2.9), by the
result obtained in Section 2.4 it follows that the solution of (5.72) for
P will be unique (given arbitrary Q) if and only if the characteristic roots
dy of A are such that Aj + Aj #0, alli, 7, When A is complex, B in (2.37)
is set equal to A* and by again appealing to Exercise 2.9 the condition
for uniqueness is seen to be A; + Aj #0, all g,7.
Assuming the appropriate uniqueness condition is satisfied it is con-
venient to set Q equal to /,,. Although (5.72) can be expressed in the
form of n? linear equations for the unknown elements of P as in (2.68),
it follows since P and Q are symmetric that only 4 n(n + 1) of the
equations and unknowns are independent. However, when n is larger
than about ten, direct solution of (5.72) in this form becomes unwieldy
and a solution method involving an infinite matrix series is to be preferred
(Barnett and Storey 1970, p. 100; see also Exercise 5.35). Equations
similar in form to (5.72) also arise in other areas of control theory (see
186 Stability

for example Section 6.1). However, it must be admitted that since a

digital computer will be required to solve (5.72) except for small values
of n, so far as stability determination of (5.70) is concerned it will be
preferable instead to find the characteristic roots of A. The true value
and importance of Theorem 5.17 lies in its use as a theoretical tool, and
we shall see some applications of this later. Also Liapunov theory, unlike
the classical methods of Section 5.2, is applicable to both linear and
non linear systems and therefore provides a unified approach.
We now return to the discrete-time system described by (5.30). Again
take V to be the quadratic form in (5.71) and from (5.69)

AV =x"(k + 1)Px(k + 1)—x(kK)Px(k)

=x1(kK)A}TPA
x(k) —x(k)Px(k)
=—x"(k) Q,x(k)
wherenow
ATPA, -P=-Q,. (5.74)
Otherwise the argument used to establish Theorem 5.17 is unchanged, so
its discrete analogue is:

THEOREM 5.18. The real matrix A, is convergent if and only if for

any given r.s.p.d. matrix Q, the solution P of the discrete Liapunov matrix
equation (5.74) is also positive definite.
The condition for uniqueness of the solution of (5.74) is that the
roots ¢ of A; are such that uju; # 1. When A, is complex, transpose in
(5.74) is replaced by conjugate transpose, and the uniqueness condition
is Mju; # 1 (see Exercise 2.17).
It is interesting to realize that the two Liapunov matrix equations are
closely related. If we apply the transformation (5.31), which sends a
stability matrix into a convergent matrix, to eqn (5.72) then we obtain

(Af +I '(AT -DP +P(A; - (A; +1)" =-@

whichcanbe simplifiedto
ATPA,-P=-3(A] +NO(A, +2). (5.75)
Provided no y; =—1, the matrix on the right in (5.75) is positive
definite, so this equation is equivalent to (5.74). The matrices P in (5.72)
and (5.75) are identical, so solution of either of the two types of Liapunov
matrix equation can be achieved in terms of the other.
Stability 187
The usefulness of linear theory can be extended by use of the idea of
linearization. Suppose the components of f in (5.53) are such that we
can apply Taylor’s theorem to obtain

f(x) =A’x + 2(x), (5.76)

using f(0) = 0. In (5.76) A’ denotes the n x n constant matrix having

elements (0f;/0x;)x=0, g(0)=
0 and the components of g have power
series expansions inx,,...,X, beginning with terms of at least second
degree. The system
X=A'x 6517)

is called the first approximation to (5.53). We then have:

THEOREM 5.19. (Liapunov’s linearization theorem). If (5.77) is
asymptotically stable so is (5.76); if (5.77) is unstable so is (5.76).
Proof,Considerthe functionV= x! PxwhereP satisfies
Gis PAs O,
Q being an arbitrary r.s.p.d. matrix. If (5.77) is asymptotically stable
then by Theorem 5.17 P is positive definite. The derivative of Vwith
respect to (5.76) is

V=—xTQx
+27 Px. (5.78)
Because of the nature of g, the second term in (5.78) has degree three at
least, and so for x sufficiently close to the origin, V<<0. Hence by Theorem
5.14 the origin of (5.76) is asymptotically stable.
Conversely, if (5.77) is unstable, V remains negative definite but P is
indefinite so V can take negative values and therefore satisfies the condi-
tions of Theorem 5.16 for instability.
In fact the condition on g can be relaxed to || g(x) || <A || x || for
sufficiently small \ > 0 (Lehnigk 1966, p. 55). Notice that if (5.77) is
stable but not asymptotically stable, Theorem 5.19 provides no informa-
tion about the stability of the origin of (5.76), and other methods must
be used.
Furthermore, it is clear that linearization cannot provide any informa-
tion about regions of asymptotic stability for the nonlinear system (5.53),
since if the first approximation (5.77) is asymptotically stable then it is
so in the large. Thus the extent of asymptotic stability for (5.53) is
determined by the nonlinear terms in (5.76).
188 Stability

The following example illustrates the simplicity of Theorem 5.19 when

it is applicable.
Example 5.21. Consider the scalar equation
Zt+az+bz+g(z, z)=0

or
Ngee
XoswXe, =—bx ypsexes Oi, xo (5.79)
The linear part of (5.79) is asymptotically stable if and only ifa >0,
b > 0 (see Exercise 5.9), so if g is any function of x, and x2 satisfying
the conditions of Theorem 5.19, the origin of (5.79) is also asymptotically
stable.

Exercise 5.30.
Using the matrix A in Exercise 3.1(a) and taking
Pi P2

P2 P3

solve the equation (5.72) with Q =/. Hence determine the stability
nature of A.

Exercise 5.31.
Integrate both sides of the matrix differential equation in Exercise 3.12
with respect to t from t = 0 to t = %°, Hence deduce that if A is a stability
matrix the solution of (5.72) can be written as

P=(e exp
(A'L)Q
exp
(Af)dt.

Exercise 5.32.
Convert the second order equation
Za+a,Z + azZ =0

into state space form. Using V in (5.71) with V =—x3obtain the necessary
and sufficient conditions a; >0,a, > 0 for asymptotic stability (see .
Exercise 5.9).
 Stability 189
Exercise 5.33.
By using the quadratic Liapunov function V(x) which has derivative
—2(x} + x3), determine the stability of the system

xy =—kx, Sehr Xo =kx, = Ob;

when k = 1. Using the same function V(x) obtain sufficient conditions on
k for the system to be asymptotically stable. What is the necessary and
sufficient condition?

Exercise 5.34.
Write (5.72) in the form
(PA +3Q)+(PA+4Q)' =0

and hence deduce that P = (S —+ Q)A', assuming A is nonsingular,

where S is the real skew symmetric matrix satisfying
A'S +SA = 3(A'Q-QA).

This represents a reduction to $n(n —1) of the number of unknowns and

, equations when (2.38) is used.

Exercise 5.35.
Verify that the matrix equation (5.74) has solution

P=Q, +AlQ;A, + (A7)?Q,A?2+(4/)° 0,43 +...

Assume that A, is similar to a diagonal matrix, and hence show that the
series for Pconverges provided A, is a convergent matrix.

Exercise 5.36.
Replace A by A + o/ in Theorem 5.17, where o is a real number. Hence
deduce the relative stability condition that the real parts of the roots of
A are less than — o if and only if the solution P of

A™P+PA+20P=-Q
is positive definite.

Exercise 5.37.
Use Theorem 5.17 to show that the matrix A = Po(So —Qo) is a stability
matrix, where Pp and Qo are arbitrary r.s.p.d. matrices, and So is an
arbitrary real skew symmetric matrix.
190 Stability

Exercise 5.38.
If for a given real stability matrix A the solution of (5.72) is P, show that
A +(S; —Q,)P is also a stability matrix, S, and Q; being arbitrary real
skew and positive semidefinite n x n matrices respectively. (Compare with
Exercise’ 5.29):

Exercise 5.39.
If n = 2, show that eqn (5.30) is asymptotically stable if and only if
|det A, |1<1 and 1 + det A, >| trA,|. (Hint: use the result in Exercise
5.10).
What is the corresponding result for eqn. (5.70)?

Exercise 5.40.
The equations describing a time varying LCR series electric circuit are
X, =X2/L(t), X2. =-x1/C(H)-—x2RW/LO,
where x, is the charge on the capacitor, x2 is the flux in the inductor
and L(t), C(t), R(t) >0,¢t 20.
Use the Liapunov function V = x" P(t)x, where

R+2L/RC 1
P(t) =
- 1 2/R
to show that V = —x™Q(t)x, where
AIC. 0 ,
Q(t) = ar
0 2/L
and hence deduce that sufficient conditions for the origin to be asympto-
tically stable are

1+R(L/R? -$C)+CL/RC-L/R>O,
1+RL/R?>0
(it can be assumed that Theorem 5.14 still holds even though the system
is non-autonomous).
Notice that it would be difficult to obtain ‘sharp’ (i.e. almost necessary)
conditions without considering the nature of the functions L(t), C(t),
R(t) in detail.
Stability 191

Exercise 5.41.
Investigate the stability nature of the equilibrium points at the origin for
the systems
(a) %, =7x,+2sinx, —x}
a= exp (x4) —3X2—1 +:5x7
(b) X1 =(3/4)sinx, —7x2 (1-x2) 5 +x3
a= 2/3) x1 —3x2 Cos'xg 11x37:

Exercise 5.42.
In the inverted pendulum problem of Exercise 1.3 assume that motions
about equilibrium are so small that second and higher degree terms in
the Taylor series expansion of f(x, xX)can be neglected, and hence obtain
the linearized form of the state equations. By finding the characteristic
equation, use Theorem 5.19 to determine the stability nature of the
origin (see Example 1.4 and Exercise 5.4).

Exercise 5.43.
Solve the scalar equation
z=z-etz3

by using the substitution z = e'w. Hence verify that z(t) > 0 as t > ©,
although the linear part is unstable.
This illustrates that Theorem 5.19 does not carry over to linear time
varying systems.

5.6. Construction of Liapunov functions.

We have seen in the previous section how a quadratic form can always be
used as a potential Liapunov function for a constant linear system. Before
seeking Liapunov functions for the nonlinear system (5.53) we must be
certain that we are not wasting our time, and this is ensured by a number
of results which are the converse of the stability theorems in Section 5.4.
We quote one of these as an example, and refer the reader to Hahn (1963,
Chapter 4) for further details.

THEOREM5.20. If the origin of (5.53) is asymptotically stable, and

all the components of f and their first partial derivatives with respect to
the x; are continuous in some region, then there exists in this region a
Liapunov function for (5.53) having V negative definite.
192 Stability

This theorem is of only theoretical interest since it does not help in

construction of Liapunov functions. We now briefly describe two out of
the many methods which are available for this purpose (for further
details see Schultz 1965).

5.6.1. Variable gradient method.

In both the asymptotic stability and instability theorems of Section 5.4

we require V to be negative definite, and as in (5.54)

oV OV
PestsEiglace
td oy aV
ax,1”
=(VV)'f (5.80
where

ei: be 0X5 ce Ore

OVoar
av |oval (5.8
is the gradient of V. The idea of the method is to let the gradient have
the form
| oyix1 +Oy2x2 +20 + O1nXn|

Qa ky eee
Se Bae

VV = . (5.82)

peegna
xn: z wn Onna
the a; being functions of the xs and chosen so as to satisfy the conditions:
(1) Vin (5.80) is negative definite
(2) VV in (5.82) is indeed the gradient of a scalar function.
From vector theory this requires that the n-dimensional curl of VV be
identically zero, i.e.

aG;_ 9G;
wal
pelea de, Delletel
nty 5.83
(5.83)
G#/),
where G; is the ith component in (5.82).
After the conditions (5.83) have been satisfied, acheck is then needed
Stability 193
to ensure that Vis still negative definite —if not, the procedure will have
to be repeated. Finally, V is obtained from the line integral

| (Cone aed) (G,dx, ++** +Gydxy)

(05OFe.510)
for which a convenient expression is

[o'GiGe1,0,
0,.oe,O)dx,
1 yeG2(X1,X2,0,.
o-, O)dx,
feoe t+ fin Gn(%15,X2;
besXn dxn: (5.84)

Of course the n? functions aj; are not determined uniquely by the above
conditions so to that extent the method is one of ‘trial and error’. Note
also that failure to find a suitable function using the variable gradient
method does not imply anything about the stability nature of the equi-
librium point.

Example 5.22. Consider the system described by

4y°=— 3x, —x3
Ry = IN HF

The first approximation (5.77) has matrix

0 =—3

0 a=
which has characteristic roots 0 and —2, and so is only stable (Theorem
5.2). Hence the linearization Theorem 5.19 does not apply.
From (5.80) and (5.82)

V=-(3x. +x} (aux, + &yX2)—(2X2 —xX}

)M(amx1
+ ax)
=—x?(044X4—OX) +X1X2(—3a 012 X4—2021+ 2X7)
—x} (3042+ 2099). (5.85)
A simple way of obtaining V< 0, although of course not the only one,
is to make the x,;x, term in (5.85) vanish by taking

Q12=0, G1 =0,
a Oy M
= § aneXt.
4
194 Stability

This gives
V=—01x§ —2ay,x?

=—OQ (Fx}° + 2x3)

which is negative definite provided a, > 0. In this case

5 On X} G,
Vics =
Q22X2 G,

so the curl condition (5.83) is just

a3 099X7) ig 0(Q22X2)
0x2 Ox,
which will be satisfied if aw. is a constant. There is no need to check V,
so from (5.84)

V=\,Xi3QpQx7
4 5 dx,+ifXo
92X72
dxX2
= Q(x} + 9x3)/18

which is positive definite, so the origin is asymptotically stable in the

large by Theorem 5.14.

5.6.2, Zubov’s method

The aim is again to try and find a Liapunov function by starting with a
negative definite function ¢(x) as its derivative, so we now wish to solve
for V(x,,X2,...,Xn) the partial differential equation

OV
et het da ot ee ee (5.86)
Oxy 0x2 OXp
subject to the boundary condition V(0) = 0. Equation (5.86) can be
written
dV/dt =
which on integration with respect to ¢ gives

VEe(T)]
~Vero)
=f° ole(Oldr, (5.87
Stability 195
where Xo =X(to ). If the origin of (5.53) is asymptotically stable, and
Xo lies within its domain of attraction then letting T > © in (5.87)
produces

Vito}
=—ikedt
which is positive, so we can expect the solution of (5.86) to be positive
definite. As Xo approaches the boundary of the domain of attraction then
in (5.87) V(xq) tends to infinity since Vix(T)} > occas T > °°, To avoid
this difficulty the procedure can be altered by defining a function

W(x) = 1—exp [—V(@x)]. (5.88)

Thedifferential
equation
(5.86)
thenbecomes
Lae
axa” axe*7 een
Onee
a =e Wye
Ke 5:99
CE
The boundary of the domain of attraction is now given by W(x) = 1.

Example 5.23. Consider the system described by

xy = 2x te es Ses

With@=—24(x7 +x3) equation (5.86) becomes

OV OV
ee 2) 572) 24s 133)

and this has solution

V= 6x? + 12x23+ 4x1x3 +8,
= 2x} + 12x} + (2x, +4)?
which is positive definite, showing the origin is asymptotically stable in
the large.

Exercise 5.44.
Consider the equilibrium point at the origin for the system

Xa SX, oct2x a teen = ND.

Construct a Liapunov function using the variable gradient method by
taking a4, = 1, @22= 2, Q42 = 2; =0 in (5.82), and determine the
corresponding region of asymptotic stability.
196 Stability

Exercise 5.45.
In Exercise 5.22 deduce the stated expression for V using the variable
gradient method.

Exercise 5.46.
The equations describing a homogeneous atomic reactor with constant
power extraction can be written

X, =—ax,/7,
ty ={exp(r1)- Ile
where P(t) = exp x, (t) is the instantaneous reactor power, x, (f) is the
temperature, a > 0 is the temperature coefficient, e> 0 is the heat
capacity and 7 > O the average life of a neutron. Use the variable gradient
method with a2 = 1, 2 = 21 = 0 in (5.82) to determine the stability
nature of the equilibrium point at the origin. (Hint: set V = 0).

Exercise 5.47.
Consider the first order system

x1 mre mds
Use Zubov’s method with ¢ =— 2x? to determine the stability nature of
the origin.

5.7. Stability and control

Apart from Section 5.3, where we developed the Nyquist criterion for
closed loop linear systems, our discussions in this chapter have so far
not directly involved the control terms in the system equations. We now
consider some stability problems associated explicitly with the control
variables.

5.7.1. Input—output stability

Our definitions in Section 5.1 referred to stability with respect to

perturbations from an equilibrium state. When a system is subject to
inputs it is useful to define a new type of stability. The nonlinear system

x =f(x,u,t), f(0,0,t)=0
with output y = g(x, u, f) is said to be bounded input —bounded output
Stability 197

(b.i.b.0.) stable if any bounded input produces a bounded output. That

is, given
| u(t) |< 2,,t2 fo (5.90)

where , is any positive constant, then there exists anumber 2, > 0 such
that || y(t) || <2, for t > to, regardless of initial state x(t). The problem
of studying b.i.b.o. stability for nonlinear systems is a difficult one, but
we can give some results for the usual linear system

x =Ax + Bu (5.91)

y=Cx. (5.92)

Using (3.22) and the properties of norms (see Section 2.7) we have
Iy@ ISIC Il tI

<I Cll [exp (Adxo 1] +1 Cl {i ll exp [A(t—7)] I | Bu Il dr.(5.93)

If A is a stability matrix then

|| exp (At) || <K exp (-at)<K,t20 (5.94)
for some positive constants K and a (see (5.12)). Thus (5.90), (5.93),
and (5.94) imply

SICH IKllxo +2K BI (1-expCat)}/a]

<I]Cll[Kllxo+ 2KWB
Ul/al,¢>0,
showing that the input is bounded, since || C || and || B || are positive
numbers. We have therefore established:

THEOREM5.21. If x =Ax is asymptotically stable, then the system

described by (5.91) and (5.92) is b.i.b.o. stable.
An interesting link with the ideas of Chapter 4 is provided by the
fact that the converse of Theorem 5.21 holds only if {A, B, c} isa
minimal realization. In other words, by Theorem 4.23:

THEOREM 5.22. If the system (5.91) and (5.92) is c.c. and c.o. and
b.i.b.o. stable, then x = Ax is asymptotically stable.
For a proof see the book by Willems (1970, p. 53). Similar results for
discrete-time systems also hold (see Exercise 5.51). However, for linear
time varying systems Theorem 5.21 is not true (see Exercise 5.52) unless
for all t the norms of B(t) and C(t) are bounded and the norm of the
198 Stability

transition matrix ® (t, to) in (3.27) is bounded and tends to zero as

t > © independently of to (Willems 1970, p. 105).
In the definition of complete controllability in Section 4.1 no
restrictions were applied to u(f), but in practical situations there will
clearly always be finite bounds on the magnitudes of the control variables
and on the duration of their application. It is then intuitively obvious
that this will imply that not all states are attainable. As a trivial example,
if a finite thrust is applied to a rocket for a finite time then there will
be a limit to the final velocity which can be achieved. We give here one
formal result for linear systems due to Mohler (1973).
THEOREM5.23. The linear system (5.91), with A a stability matrix
and u bounded by (5.90), is not c.c.
Proof. Let V be a quadratic form Liapunov function (5.71) for the
unforced system. Then with respect to (5.91)
V=~xTQx+u™BT(VV), (5.95)
where P and Q satisfy (5.72) and VV is defined in (5.81). The second
term on the right hand side of eqn (5.95) is linear in x and since u is
bounded it follows that for || x || sufficiently large, V = X' (VV) is negative.
This shows that x points into the interior of the region V(x) = M for
some M sufficiently large. Hence points outside this region cannot be
reached, so by definition the system is not c.c.
It is most interesting that bilinear systems, which contain product
terms u;x;, can still be c.c. even when the inputs are bounded. An example
given by Mohler is that the linear system

x4 =X2,X =—2x, —X4 +u

with |u |<1 is not c.c. by Theorem 5.23, but if the second equation is

Sei = Dee Xo he Fux + Dae

then the resulting bilinear system is c.c. Bilinear mathematical models
of control systems are being increasingly studied, and a number of
practical applications are described in Mohler’s book (see also Example
1.5 and Exercise 6.13).

5.7.2. Linear feedback.

Consider again the linear system (5.91). If the open loop system is
unstable (i.e. by Theorem 5.1 one or more of the characteristic roots of
Stability 199
A has a positive real part) then an essential practical objective would be
to apply control so as to stabilize the system —that is, make the closed
loop system asymptotically stable. If (5.91) is c.c. then we saw in Section
4.4 (Theorem 4.17) that stabilization can always be achieved by linear
feedback u = Kx, since there are an infinity of matrices K which will
make A + BK a stability matrix. If the pair [A, B] is not c.c. then we can
define the weaker property that [A, B] is stabilizable if and only if there
exists a constant matrix K such that A + BK is asymptotically stable. We
can see at once from the canonical form of (5.91) displayed in Theorem
4.9 that the system is stabilizable if and only if A3 in (4.21) is a stability
matrix. In this case the feedback is u = K,x), where K, can be chosen
to make A, + 8B,K, asymptotically stable since the pair [A,, 8, ] isc.c.
By duality (see Theorem 4.11) we define the pair [A, C], where C is the
output matrix in (5.92), to be detectable if and only if [47, C™] is
stabilizable.
Methods for constructing feedback matrices K were discussed in
Section 4.4, where the disadvantages of the method of prespecifying
closed loop poles were indicated. Two simple illustrations of the applica-
tion of stabilizing linear feedback have been provided by the rabbit—fox
problem (Exercise 4.33), in which a disease lethal to rabbits keeps the
animal population finite; and by the buffalo model (Exercise 4.53) in
which slaughtering animals for food fulfils the same function. Another
method of constructing matrices K will be developed in Chapter 6 using
the ideas of optimal control. It is interesting to consider here a simple
application of the Liapunov metheds of Section 5.5. First notice that if
(5.70) is asymptotically stable with Liapunov function V = x" Px, where
P satisfies (5.72), then
V/V=—x"!Qx/x'Px
<-o (5.96)
wheregis the minimumvalueof the ratio x Qx/x! Px(in fact this is
equal to the smallest characteristic root of QP~'). Integrating (5.96)
with respect to ¢ gives
V[x(t)] <exp ~ ot)V[x(0)]. (5.97)
Since V[x(t)] > 0 as t >, (5.97) can be regarded as a measure of the
way in which trajectories approach the origin, so the larger o the ‘faster’
does x(t) > 0. Suppose now we apply the control

u=(S—Q,)B™Px (5.98)
200 Stability

to (5.91), where P is the solution of (5.72) and S and Q, are arbitrary

skew-symmetric and positive definite symmetric matrices respectively.
The closed loop system is thus

X= [A+B(S—Q,)B'P]x (5.99)
and it is easy to verify that if V=x! Px then the derivative with respect
to (5.99) is
V=—x™Qx—2xTPBO,B™Px
<-x'Qx
since PBQ ,B' P = (PB)Q ,(PB)' is positive definite. Hence by the argu-
ment just developed it follows that (5.99) is ‘more stable’ than the open
loop system (5.70), in the sense that trajectories will approach the
origin more quickly. Of course (5.98) is of rather limited practical value
because it requires asymptotic stability of the open loop system, but
nevertheless the power of Liapunov theory is apparent by the ease with
which the asymptotic stability of (5.99) can be established. This would
be impossible using the classical methods of Section 5.2, requiring
calculation of the characteristic equation of (5.99). Furthermore, the
Liapunov approach often enables extensions to nonlinear problems to
be made (see for example Exercise 5.53).

+5. 7.3. Nonlinear feedback.

We now consider the system with a single control variable

x = Ax —bu (5.100)
subject to nonlinear feedback
u=f(y) (5.101)
where
y=cx. (5.102)
In (5.101) f(v) is a continuous function of the scalar output y, and A, b,
and c in (5.100) and (5.102) are such that the triple {A, , c} is a minimal
realization of a scalar transfer function g(s). It is assumed that f(0) = 0
so the origin x = 0 is an equilibrium point for (5.100) and (5.101).
The stability theorems to be quoted in this section involve the notion
of a positive real function r(s), which we define to be a rational function
 Stability 201

p(s)/q(s) with real coefficients such that p(s) and q(s) are relatively prime
and
(a) r(s) has no poles in Re(s) > 0
(b) any purely imaginary poles of r(s) are simple and have real
positive residues
(c) Re{r(iw)}>
2 0 forallreal w>
Wecannowstate
arecent
result
athegives
asifficient
stability
condition.
THEOREM 5.24. (Popov). The origin of the system described by
(5.100), (5.101) and (5.102) is asymptotically stable in the large if

(i)
i
0< fy)
. <ky,ally£0, (5.103)
5.103

where k, is a positive constant, and

(ii) there exists a real number a@such that

(1 +as)g(s) + 1/k, (5.104)

is positive real.
Proofs of the theorem are too complicated to give in this book. One
(Siljak 1969, p. 330) relies on using a Liapunov function of the form

xTPx+0 Ik f(o)do
0
where P is the solution of the Liapunov matrix equation (5.72) (note
that positive-realness of (5.104) ensures that A has no characteristic
roots with positive real parts). Another approach to the proof uses
functional analysis (see Willems 1970, p. 152).
Notice that (5.103) means that provided condition (i) of Theorem
5.24 is satisfied, the origin is asymptotically stable in the large for any
continuous function f(y) lying within the sector bounded by the line
z = k,y and the y axis, as shown in Fig. 5.15. In this case the system is
said to be absolutely stable in the sector [0, k,].
An interesting and useful aspect of the Popov criterion is that it has a
simple graphical interpretation. Positive realness of (5.104) requires that
g(s) must have no poles in Re(s) > 0, and that any imaginary poles of
g(s) are simple and the corresponding residues of (5.104) are real and
- positive. In particular, if A is a stability matrix (i.e. the open loop system
is asymptotically stable) these conditions will certainly be satisfied.

allen
—
202 Stability

FIG. 5.15

Condition (c) of the positive realness definition applied to (5.104)

gives
Re {(1 + iaw)g (iw) + 1/k,}>0 (5.105)
for w > 0. We define a curve in the complex plane by means of
U, =Refg(iw)}, V; = wlm{g(iw)}, w > 0, (5.106)

which differs from the Nyquist locus (5.51) only in that the imaginary
part has an extra factor w. Substitution of (5.106) into (5.105) produces
av, Ol < 1/k,.

It follows that for (5.105) to be satisfied it must be possible to draw a

straight line through the point —1/k, + i0 such that the locus has no
points to the left of the line, as shown in Fig. 5 16.

FIG. 5.16
Stability 203

The value of a is given by the slope of the straight line a V; —U, = 1/k;
(the Popov line). The largest value of k; would be when —1/k, + i0 is
at the point A in Fig. 5.16. If k, is increased further so that the point B is
reached (5.105) cannot be satisfied (although this does not mean the
origin is unstable, as Theorem 5.24 is only sufficient).
When f is a linear function then we simply have linear feedback, so
u=Ky and the Nyquist criterion of Theorem 5.11 (in the modified form
of p. 171) applies. The necessary and sufficient condition for absolute
stability in the sector [0, k,] (ie.0<x <k;,) is that the Nyquist locus
of g(s) should not intersect the real axis to the left of —1/k, + iO (includ-
ing the point itself), for if it did there would be some point —1/kp + i0
with kg < k, enclosed by the locus. If the system with u given by (5.101)
is to be absolutely stable, it clearly must be stable for the particular case
when fis linear, so the condition on the Nyquist locus is a necessary one
for absolute stability of the nonlinear system in the sector [0, k, ].

Example 5.24. Consider the transfer function

1
g(s)=
S(S7obeS
ti)
By Exercise 5.9, 5? +s + 2 is a Hurwitz polynomial, and for the simple
pole at the origin the residue is ¥, 80 it remains to satisfy (5.105). From
(5.106)
1 w* -2
U.eeee cy + (os* = 2) V. Fe?
a VoL ae
+(wo?—2)?
and the locus is shown below.

FIG. 5.17
204 Stability

A tangent can be drawn from —1/k, +i0 to the curve provided B is to

the left of A, ie. kK,<2. Thus (5.104) is positive real, so by Theorem
5.24 the system is absolutely stable in the sector [0, 2].
We now consider (5.100) with feedback of the form

u=h(x)cx (5.107)

where /(x) is a scalar function of x. A sufficient condition for stability

is then the following:

THEOREM 5.25. The origin of the system described by (5.100) and

(5.107) is asymptotically stable in the large if

k3 <h(x)<kp, all x, (5.108)

and the function
1 +k2g(s)
iaetea
thats) (5.109)

is positive real.
We also omit this proof, which can be developed either by Liapunov
theory (Willems 1970, p. 161) or using functional analysis. Instead we
shall again develop a graphical interpretation using the frequency transfer
function.
The positive realness condition (c) on p. 201 applied to (5.109) requires
that

ce[tt]
pete)
Neenae |
e119
for w> 0, and this time the standard Nyquist locus (5.51) is used.
Substituting g(iw) = U + iV into (5.110) reduces it to

k,k3(U? + V7)+ (ky +k3)U+120. (5.111)

Whenequality occursin (5.111) the expressionrepresentsa circlein the

g(s) —plane,
(ko +k3) .. » eee
tre— ————
centre 2k, ka + 1i0, radius
di kok;

This circle @ has the points —1/k3 + i0,— 1/k, +i0 at opposite ends
of a diameter, as shown in Fig. 5.18.
 Stability 205

FIG. 5.18

The condition (5.111) is equivalent to requiring that the Nyquist locus

of g(s) does not enter @ if k, and k3 have the same sign, and must not
have any point outside @ if k.k3 <0. For (5.109) to be positive real
_ the conditions (a) and (b) of the definition on p. 201 must also be satisfied.
It can be shown that these can be replaced by the requirement that
p(s) + q(s) be a Hurwitz polynomial. Since k, and k3 are constants,
positive realness of (5.109) is equivalent to that of

k3 1 + k2g(s) ry k(s)/ky + B(s)

ky 1+k3g(s) = k(s)/k3 + B(s)
when k,k3 > 0, using the expression for g(s) in (5.47). Hence the
polynomial

28(s)
A +) k(s) (5.112)
must be of Hurwitz type. The zeros of (5.112) are the poles of
B(s) é g(s)

and the condition that they should all be in Re(s) < 0 is (by Theorem
5.12) that the complete Nyquist locus of g(s) for —°° <w < should
encircle the point —4(1/k, + 1/k3) + i0 in an anticlockwise direction
P times, where P is the number of poles of g(s) in Re(s) > 0. This point
is just the centre of the circle @ , so when kz and k3 have the same sign
206 Stability

Theorem 5.25 is equivalent to requiring that the complete Nyquist locus

of g(s) does not enter @ , and encircles it P times in an anticlockwise
direction. When k,k3 <0 the condition is simply that the entire Nyquist
locus must lie inside @ .
An instability result when k,k3 > 0 is that at least one solution of
(5.100) and (5.107) is unbounded as t > »° if the Nyquist locus ofg(s)
does not intersect @ and encircles it fewer than P times in an anticlock-
wise direction.
Because of the graphical interpretation Theorem 5.25, and variants of
it in which h can also be a function of f, are called circle criteria. The
theorem is applied to find values of k, and k3 by drawing circles, if
possible, in the g(s) —plane satisfying the above requirements with respect
to the Nyquist locus. When k, > k3 the circle criterion tends to the
Nyquist stability criterion ( @ shrinks to the point —1/k + iO).
An extension of circle theorems to multivariable systems has recently
been given by Rosenbrock and Cook (1973).

Exercise 5.48.
Consider the mass-spring system in Example 5.17, which is stable but
not asymptotically stable. If a vertical control force is applied to the
mass, determine whether the system is b.i.b.o. stable.

Exercise 5.49,
Consider again the inverted pendulum of Example 1.4 and Exercise 1.3,
and use the linearized model developed in Exercise 5.42 in the following:
It is required to keep the rod as nearly vertical as possible by means of
the horizontal control force u(t) applied to the platform. Determine the
control terms in the linearized system equations (see Fig. 1.5). Show that
stabilization cannot be achieved by linear feedback u = k6, but that if
k >(m + M)g then the rod can be given an oscillatory motion about the
vertical position.
What would be the implications of Theorem 4.17 for this problem?

Exercise 5.50.
Show that the system described by

0 A! 0 0
yal 0 OS Xehe sO ae pie
 Stability 207

is stable i.s.L. and b.i.b.o. stable, but not asymptotically stable. (It is
easy to verify that the system is not c.c., so Theorem 5.22 does not apply).

Exercise 5.51.
Prove the analogue of Theorem 5.21 for the discrete-time system

x(k + 1) = Ax(k) + Bu(k), y(k) = Cx(k).

(Hint: use (3.38)).

Exercise5.52.
Considerthe systemdescribedby the scalarequation
¥1()=— x1 (H(t +3) +uf), x1(0) =Xo.
Show that if u(t) = 0, ¢ > O then the origin is asymptotically stable, but
that if w(t) is the unit step function (defined in (1.16)) then im x(t) =~.
foo

"Exercise 5.53.
If the origin of the system described by x =f(x) is stable, with Liapunov
function V(x), show that the system x = f(x) + u is made asymptotically
stable by taking

u = [S(x) —O(x)] o@),

where $(x) = VV, S(x) is an arbitrary skew symmetric matrix and Q(x)
is an arbitrary positive definite matrix.

Exercise 5.54.
Consider the equilibrium point at the origin for the gyroscope system of
Exercise 5.28 with external forces, so that the equations of motion are

Aw, +(C—B)w2w3 =Au,

Buy +(A —C)w3Ww,
= Bu,
Cad3a (B—A)wWw= Cu3.
Verify that the total rotational energy V = 4 (Aw? + Bud + Cu3) isa
Liapunov function for the unforced system, and use the result of Exercise
5.53 with S=0 and Q diagonal to obtain a stabilizing control vector.
208 Stability

Exercise 5.55.
If in Theorem 5.24 g(s) = 1/(s* + as* + bs) where a > 0, b> 0, show that
the sector of absolute stability is [0, ab].

Exercise 5.56.
By considering the Nyquist locus of r(s) = p(s)/q(s) deduce from Theorem
5.11 that if7(s) is positive real and has no purely imaginary poles then
p(s) + q(s) is aHurwitz polynomial.

Exercise 5.57.
Sketch the Nyquist locus for g(s) = 1/(s? + 2s). By finding the largest
circle centred on —1 + iO in the g(s) plane which satisfies the conditions
of Theorem 5.25, show that the resulting bounds (5.108) are
2(2 —/3) < h(x) < 22 + V3).
6 Optimal control

This chapter deals with the problem of compelling a system to behave in

some ‘best possible’ way. Of course, the precise control strategy will
depend upon the criterion used to decide what is meant by ‘best’, and
we first discuss some choices for measures of system performance. This
is followed by a description of some mathematical techniques for
determining optimal control policies, including the special case of linear
systems with quadratic performance index when a complete analytical
solution is possible. Optimal control theory also has an extremely large
literature so our treatment is of necessity very concise.

6.1. Performance indices

6.1.1, Measures of performance.

We consider systems described by a general set of n nonlinear differential

equations

HT sta fia Xnlig achestint) (6.1)

subjectto
X(to)=Xo, (6.2)
where the components of f satisfy the same conditions as for (5.1). A
performance index is a scalar which provides a measure by which the
performance of the system can be judged.

Minimurm-time problems, Here u(t) is to be chosen so as to transfer the

system from an initial state xo to a specified state in the shortest possible
time. This is equivalent to minimizing the performance index
J=t, —to

= Mdt (6.3)
where f; is the first instant of time at which the desired state is reached.
210 Optimal control

Example 6.1. An aircraft pursues a ballistic missile and wishes to

intercept it as quickly as possible. For simplicity neglect gravitational
and aerodynamic forces and suppose that the trajectories are horizontal.
At t = 0 the aircraft is a distance a from the missile, whose motion is
known to be described by x (f) =a + bt”, where b is a positive constant. The
motion of the aircraft is given by 4 =u, where the thrust u(t) is subject
to |u| <1, with suitably chosen units. Clearly the optimal strategy for
the aircraft is to accelerate with maximum thrust u(t) = + 1. After a time
t the aircraft has then travelled a distance ct + 4? , where X,(0) =, so
interception will occur at time T where

cT+4T* =atbT?.

This equation may not have any real positive solution; in other words,
this minimum-time problem may have no solution for certain initial
conditions.

Terminal control, In this case the final state xf = x(t, ) is to be brought

as near as possible to some desired state r(t, ). Asuitable performance
measure to be minimized is

eN(t,)
Me(t1), (6.4)
where e(t) = x(t) —r(t) and M is a real symmetric positive definite
(1.s.p.d.) n x n matrix. A special case is when M is the unit matrix and
(6.4) reduces to || x¢—r(t1) Il 2. More generally, if M= diag [m,,..., my]
then the m; are chosen so as to weight the relative importance of the
deviations [x; (t; )—7; (t1)] *. If some of the 7; (t, ) are not specified then
the corresponding elements of M will be zero and M will be only positive
semidefinite (r.s.p.s.d.).
Minimum effort. The desired final state is now to be attained with
minimum total expenditure of control effort. Suitable performance indices
to be minimized are

if > Biluilae, (6.5

or

f‘ ulRudt (6.6
 Optimal control 211

where R is r.s.p.d. and the 8; and 7; are weighting factors. We have already
encountered expressions of the type (6.6) in Chapter 4 (Theorem 4.4 and
Exercise 4.13).
Tracking problems. The aim here is to follow or ‘track’ as closely as
possible some desired state r(t) throughout the interval tg <t<tf,.
Following (6.4) and (6.6) a suitable index is

iPe'Qdet (6.7)
where Q is r.s.p.s.d.
ly
We introduced the term servomechanism for such
systems in Section 1.2, a regulator being the special case when r(f) is
constant or zero. If the u;(t) are unbounded then minimization of (6.7)
can lead to a control vector having infinite components. This is unaccept-
able for real life problems, so to restrict the total control effort a
combination of (6.6) and (6.7) can be used, giving

te (e' Qe+ u' Ru) dt (6.8)

| Expressionsof the form(6.6),(6.7),and (6.8) aretermedquadratic

performance indices.
Example 6.2. A landing vehicle separates from a spacecraft at time
t = 0 at an altitude h from the surface of a planet, with initial downward
velocity v. For simplicity assume that gravitational forces can be neglected
and that the mass of the vehicle is constant. Consider vertical motion only,
with upwards regarded as the positive direction. Let x, denote altitude,
xX,velocity and u(t) the thrust exerted by the rocket motor, subject to
| u(t) | <1 with suitable scaling. The equations of motion are

Ky =Xpgsca, SU, (6.9)

and the initial conditions are
x,(0)=h, x2(0)=-v». (6.10)
For a ‘soft landing’ at some time T we require
x,;(T)=0, x2(7)=0. (6.11)
A suitable performance index might be

fe(iul>iar, (6.12)
212 Optimal control

which is a combination of (6.3) and (6.5). The expression (6.12) represents

a sum of total fuel consumption and time to landing, k being a factor
which weights the relative importance of these two quantities.
The expression for the optimal control which minimizes (6.12) subject
to (6.9), (6.10), and (6.11) will be developed in Section 6.3. Of course
the simple equations (6.9) arise in a variety of situations (e.g. take angular
displacement and velocity as state variables in Example 4.1).
The performance indices given above are termed functionals, since
they assign a unique real number to a set of functions x;(¢), u;(f). In the
classical optimization literature more general functionals are used, for
instance the problem of Bolza is to choose u(t) so as to minimize

J(u) = $[x(t1), 1] + F(x, u,t) dt (6.13)

subject to (6.1), the scalar functions ¢ and F being continuous and

having continuous first partial derivatives.

6.1.2. Evaluation of quadratic indices.

Before dealing with problems of determining optimal controls we return

to the constant linear system
X(t) = Ax(t), x(0)=xo (6.14)
and show how to evaluate associated quadratic indices

J, = {. tx" Oxde, r= 0,1 nna (6.15)

0
where Q is r.s.p.d. If (6.14) represents a regulator, with x(t) being the
deviation from some desired constant state, then minimizing J, with
respect to system parameters is equivalent to making the system approach
its desired state in an ‘optimal’ way. Increasing the value of r in (6.15)
corresponds to penalizing large values of ¢ in this process.
To evaluate Jo we use the techniques of linear Liapunov theory in
Section 5.5. It was shown that

<(x™Px)
LSEthe (6.1
whereP andQ satisfy the Liapunovmatrix equation (5.72), namely
A'P+PA=-Q. (6.17)
Optimal control 213

Integratingboth sidesof (6.16) with respect to f gives

Jo == [xh (Px) 75 9

=x6 Pxg; (6.18)

provided A is a stability matrix, since in this case x(t) > 0 as tf >
(Theorem 5.1). Furthermore, by virtue of Theorem 5.17, P is positive
definite soJg > 0 for all x9 #0. A repetition of the argument leads to a
similar expression for J,, r > 0. For example,

& (tx?Px)= x?Px—tx?Qx,

andintegrating
wehave

J, ={, "Oxdt
_

0
= { x™Pxdt—[tx7Px] : 0

=xeP, XotaOs

whereby comparisonwith (6.17)

A'P, +P,A =-P.

It is left as an easyexercisefor the readerto verifythat

J. =rixd Px, (6.19)

where
MP) +Pp AEP) 20) 1928.) (6.20)

withPo =P. Thusevaluationof (6.15) involvesmerelysuccessivesolution

of the linearmatrix equations(6.20); there is no need to calculatethe
solution x(t) of (6.14). Some remarks on solving equations of the form
(6.20) were made in Section 5.5.
214 Optimal control

Example 6.3. A general second order linear system can be written as

Z + 2wtt + w?z=0

where w is the natural frequency of the undamped system and ¢ is a

damping coefficient (compare with ean (5.62)). With the usual choice of
state variables x; =z,x. =2Z,and taking Q = diag [1,q] in (6.17), it is
easy to obtain the corresponding solution P with elements

Piur=$/w+(1 +qw?)/4fw,Pi2=1/2w?,Por=(1 +qw*)/4fu?.

In particular if x(0) = [1,0] 7 then (6.18) gives Jo = p11. Regarding ¢ as
a parameter, optimal damping could be defined as that which minimizes
Jo. By setting d/Jy/d¢ = 0 this gives

6? =(1+qw?)/4.

For example, if g = 1/w* then the ‘optimal’ value of ¢ is 1/\/2. In fact

by determining x(t) it can be deduced that this value does indeed give
desirable system transient behaviour. However, there is no @priori way
of deciding on a suitable value for the factor q, which weights the relative
importance of reducing z(f) and Z(f) to zero.
This illustrates a disadvantage of the performance index approach,
although in some applications it is possible to use physical arguments to
choose values for weighting factors.
Equivalent results for the discrete-time system

Xp] =AprxK,KEV, 12,252 (6.21)

can be derived using the associated matrix equation (5.74). The perfor-
mance index corresponding to (6.15) is

Kp = > Wx¥O.xn, = 0, 1,2) (6.22)

k=0
where Q, is r.s.p.d. To find the expression for Kgcorresponding to
(6.18), from eqn. (5.74) we have

xp QixK=-x} (APPA,—Pixg .
= x}Px,—xp41Px, poe (6.23)
Optimal control 215

the second step following by virtue of (6.21). Summing both sides of

(6.23) from k = 0 to k == produces

Ko ye
= X09
Pxo,
provided A, is a convergent matrix so that x, > 0 as k > (Theorem
5.8). Similarly, multiplying both sides of (6.23) by k and summing leads
to

K,=> kf Ox,
k=0
=> xfPxy xdPro
0
a xo (P; —P)xo

where by comparison with (5.74)

A{P,A, —P, =—P.

Generally it can be shown that for r > 0

r+]

K, =x¢ a byiPi-1| Xo; (6.24)

i=l
where

meres 14; —P34 1 =—P;, i=0,1,2,...

with Py =P, and general expressions for the constants b,; can be derived.

Exercise 6.1.
In Example 6.1 show that if b > + and the missile is initially a distance
greater than c?/2(2b —1) from the aircraft then it cannot be caught.

Exercise 6.2.
If x(t) is the solution of (6.14), show by considering d[x(t) @x(t)] /dt
that if A is a stability matrix then

(es(x®x)dt=—B™'x9
@Xo,
where B=A @/J+/]®A.
216 Optimal control

Similarly differentiate tx ® x to show that

jo @x dt=B 7x5 @Xo.

0

Exercise 6.3.
Determine K, in the form (6.24).

Exercise 6.4.
Show that if xx is the solution of (6.21) and A, is a convergent matrix
then
co

i XE@xz~=B°! (xo @Xo), ¥ kx~ @xz =B°? (x, @x;),

k=0 1
where B=J—A, @A,.

6.2. Calculus of variations.

This subject dates back to Newton, and we have room for only a brief
treatment. In particular we shall not mention the well known Euler equa-
tion approach which can be found in standard texts (e.g. Kirk 1970).
We consider the problem of minimizing J(u) in (6.13) subject to the
differential equations (6.1) and initial conditions (6.2). We assume that
there are no constraints on the control functions u;(t), and that J(u) is
differentiable, i.e., if uand u + 6u are two controls for which J is defined
then
AJ = J(u + §u)—J(u)

= 6 J(u, 5u) + j(u, du) || du || (6.25)

where 6J is linear in 5u and j(u, 5u) > 0 as || 5u || > O (using any suitable
norm). In (6.25) 5J is termed the variation in J corresponding to a
variation 6u in u. The control u* is an extremal, and J has a relative
minimum, if there exists an €> 0 such that for all functions u satisfying
|| uw
—u* || Cethe difference J(u) —J(u*) is nonnegative. Afundamental
result is the following.
THEOREM 6.1. A necessary condition for u* to be an extremal is that
5J(u*, 5u) = 0 for all 5u.
 Optimal control 217

For a proof see the book by Kirk (1970, p. 121).

We now apply Theorem 6.1 to (6.13). Introduce a vector of Lagrange
multipliers p = [p,,...,Pn] T so as to form an augmented functional
incorporating the constraints:

J, = [x(t1), ti] + if t [F(x, u, t)+p'(f—-x)] dt (6.26)

Integrating the last term on the right hand side of (6.26) by parts gives

Ja=Oberst] + J +p T¢+
Tx] dr px f

=o x(t), 1 —pTx]i + ffs [H+(p)"x]dr (6.27)

where the Hamiltonian function is defined by

H(x, u, t)= F(x, u, t+ p's. (6.28)

Assume that u is continuous and differentiable on to <t<t, and that

to and f, are fixed. The variation in J, corresponding to a variation du
in u is

= 0g a)
[CPDagtS
6J,=|{——p'}6 -E
Lae
OFoH
tr et AT
|
MTXae, 6.29)
+ —6x+—but5x|dt,(6.29
where 5x is the variation in x in the differential equations (6.1) due to
du, using the notation

a =|me 0H
ax ae|
and similarly for 0¢/0x and 0H/du. Notice that since x(to ) is specified,
(5x)r=1, = 0. It is convenient to remove the term in (6.29) involving
5x by suitably choosing p, i.e. by taking

i adiees
mealtptle Rese, (6.30)
and

Pi(t1)=
od vt | (6.31)
218 Optimal control

Eqn (6.29) then reduces to

5J,= |j, t,*(+ou

{0Hsu) dt j
ne)

soby Theorem6.1a necessary

conditionforu*to beanextremalisthat
~*) eo
en aes 0, fo SUS 1 eee (6.33)
We have therefore established:

THEOREM 6.2. Necessary conditions for u* to be an extremal for

(6.13) subject to (6.1) and (6.2) are that (6.30), (6.31), and (6.33) hold.
The state equations (6.1) and the adjoint equations (6.30) give a total
of 2n nonlinear differential equations with mixed boundary conditions
X(to) and p(t, ). In general analytical solution is not possible and numerical
techniques have to be used.

Example 6.4. Choose u(t) so as to minimize

f »Tip
Gtayaanen
tut) de (6.34)

subject to
X, =—ax,; tu, x,(0)=Xo (6.35)
where a and 7 are positive constants.
From (6.28)
H=xi +u? +p,(-ax, tu)
and (6.30) and (6.33) give respectively
pt =—2xt + apt (6.36)

and
2u* +/p;* =0, (6.37)
where xj and pj denote the state and adjoint variables for an optimal
solution. Substituting (6.37) into (6.35) produces

Xf =—axt—-tp}, (6.38)
and since ¢ = 0 the boundary condition (6.31) is just p; (7) = 0. The
equations (6.36) and (6.38) are linear and so can be solved using the
Optimal control 219

methods of Chapter 3. It is easy to verify that xf and p{ take the form

c exp(At) + d exp(—Ar) where X = /(1 + a”), and the constants c and d
are found using the conditions at ¢ = 0 and t = 7. From (6.37) the optimal
control is u7(t) =— , pi(t) (although of course we have only found
necessary conditions for optimality; further discussion of this point is
outside the scope of this book, and will not be referred to in subsequent
examples).
Notice that (6.34) is a simple case of the quadratic index (6.8), and
that together with the linear system equation (6.35) this leads to the
adjoint equation (6.30) being linear, so that an analytical solution to the
problem is possible. We shall generalize this in Section 6.4.
If the functions f and F do not explicitly depend upon ¢ then from
(6.28)

Pa ae aad ae ed ol
for tos et) s0r
i 1s r+(p)TS
ots FO")s
using (6.1). Since on an optimal trajectory (6.30) and (6.33) hold it
follows that dH/dt = 0 when u = u*, so that
(A)y=y* = constant, fo <t<4y. (6.39)
We have so far assumed that f, is fixed and x(t, ) is free. If this is
not necessarily the case, then by considering (6.27) the terms in 6J, in
(6.29) outside the integral are obtained to be

(Ge-27)os+(4+$s | wey (6.40)

ey

The expression in (6.40) must be zero by virtue of Theorem 6.1, since

(6.30) and (6.33) still hold, making the integral in (6.29) zero. The
implications of this for some important special cases are now listed. The
initial conditions (6.2) hold throughout.
220 Optimal control

Final time t, specified.

(i) x(t) free

In (6.40) we have 5¢, = 0 but 6x(t, ) is arbitrary, so the conditions
(6.31) must hold (with (6.39) when appropriate), as before.
(ii) x(t, ) specified
In this case 6¢, = 0, 5x(¢, ) = 0 so (6.40) is automatically zero.
The conditions are thus

x*(t1) = xf (6.41)

and (6.41) replaces (6.31).

Final time t, free

(iii) x(t, ) free
Both 5t, and 5x(t, ) are now arbitrary so for the expression in
(6.40) to vanish, (6.31) must hold together with

(| og
=Fat eet (6.4
it,
In particular if ¢, F’, and f do not explicitly depend upon ¢ then (6.39)
and (6.42) imply

(A) an® = 0, ToSt<f,. (6.43)

(iv) x(t; ) specified

Only 6f, is now arbitrary in (6.40) so the conditions are (6.41)
and (6.42) (or (6.43)).
If the preceding conditions on x(t, ) apply to only some of its compon-
ents, then since the 5x;(t, ) in (6.40) are independent it follows that the
appropriate conditions hold only for these components.

Example 6.5. A ship moves through a region of strong currents. For

simplicity, and by a suitable choice of coordinates, assume that the current
is parallel to the x, axis and has velocity c = —Vx,/a, where a is a positive
constant, and V is the magnitude (assumed constant) of the ship’s velocity
relative to the water. The problem is to steer the ship so as to minimize
the time of travel from some given point A to the origin. .
Optimal control 221

Resultant
velocity V

Current

FIG. 6.1

We see in Fig. 6.1 that the control variable is the angle u. The equations
of motion are

X, =Vcosut+c
= V(cos u—x2/a) (6.44)
x, =Vsinu, (6.45)
where (x, (f), x2 (t)) denotes the position of the ship at time t. The
performance index is (6.3) with tf) = 0, so from (6.28)
H=1+p,V(cosu-—x,/a)+p2V sinu. (6.46)
The condition (6.33) gives
—p*V sinu* + p*V cosu* =0

so that
tan u* =pF/pt . (6.47)
The adjoint equations (6.3u) are
pi =0,BF=prVia, (6.48)
222 Optimal control

which imply that pf =c,, a constant. Since f, is not specified we have

case (iv) on p.220, so that (6.43) holds. From (6.46) this gives
0=1+pfV(cosu* —xf/a)+ pz V
sin u*

=1+0¢,V(cos u* —x}/a)+c,V sin*u* /cos u*, (6.49)

using the expression for pj in (6.47). Substituting x}(t, ) = 0 into (6.49)
leads to
Cy;=—cosu,/V, (6.50)
where u, =u*(t, ). Eqn (6.50) reduces (6.49), after some rearrangement,
to
x} /a = sec u* —sec uy. (6.51)

Differentiating (6.51) with respect to ¢ gives

(du*/dt) sec u* tan u* = xX7/a
= V sin u*/a, by (6.45).

Hence
(V/a)(dt/du*) = sec? u*

which on integration produces

tan u* —tanu, =(V/a\(t-t;). (6.52)
Use of (6.44), (6.51), and (6.52) and some straightforward manipulation
leads to an expression for xj in terms of u* and u,, which enables the
optimal path to be computed. Further details can be found in the book
by Bryson and Ho (1969, p. 77), and a typical minimum-time path is
shown in Fig. 6.2.
If the state at final time f, (assumed fixed) is to lie on a surface
defined by some function m [x(t)] = 0, where m may in general be a
k-vector, then it can be shown (Kirk 1970, p. 189) that in addition to
the k conditions

m[x*(t,)] =0 (6.53)
there are a further n conditions which can be written as

A
Pay oN ea ( ——
a deh?\ ——
ox heels Hedge—
\ ox 6.54
eae

both sides being evaluated at f=t,,u =u*,x =x*, p =p*. The d; in

(6.54) are constants to be determined. Together with the 2” constants of
 Optimal control 3

x2

EG

aa OUTKeNt
as

FIG. 6.2

integration there are thus a total of 2” + k unknowns and 2n + k condi-

tions (6.53), (6.54), and (6.2). If ¢, is free then in addition (6.42) holds.
The conditions which hold at t= ?¢, for the various cases we have
covered are summarized in Table 6.1.

TABLE 6.1

t, fixed

x(t, ) free (6.31) (6.31) and (6.42)

x(t, ) fixed (6.41) (6.41) and (6.42)
x(t, ) lies on m[x(t,)] =0 (6.53) and (6.54) (6.42), (6.53), and (6.54)

Example 6.6. If a second order system is to be transferred from the

origin to a circle of unit radius, centre the origin, at some time T then we
must have

XEGP)) pttL =1 (6.55)

Since
m(x) =.x3 +x32-1
(6.54) gives
~ [p¥(7), PF(T)] = 4, [2xF), x7],
224 Optimal control

assuming ¢ = 0, and hence

Di(T)/p2(T) = xi(T)/x7(7). (6.56)

Egns (6.55) and (6.56) are the conditions to be satisfied at t = T.

Example 6.7. A system described by

Xp =Xs, 67 ae (6.57)

is to be transferred from x(0) = 0 to the line

ax; + bx, =c

at time T so as to minimize

\e u? dt,

which is of the form (6.6). The values of a, b, c, and T are given.

From (6.28)
H=u? + pyX. —p2X2 +pru,

and (6.33) gives

u*=—4pk. (6.58)

pi=0, py =—pi +p?

so that
Pr=C1, Preece +c, (6.59)

where c, and c, are constants. Substituting (6.58) and (6.59) into (6.57)
leads to
t t
xf=es6'—de,e'—deittcq, xf =—cse'— de,6° He)}
and the conditions
xT(0) = 0, xF(0) = 0, ax#(7) + bxF(T) =, (6.60)
must hold.
It is easy to verify that (6.54) produces

Pi(T)/p3(T) = a/b, (6.61)

Optimal control 225

and (6.60) and (6.61) give four equations for the four unknown constants
cj. The optimal control u*(¢) is then obtained from (6.58) and (6.59).
In some problems the restriction on the total amount of control effort
which can be expended to carry out a required task may be expressed in
the form

f t, &(%,,u,4) dt =c, (6.62)

where c is a given constant, such a constraint being termed isoperimetric.

A convenient way of dealing with (6.62) is to define a new variable

xn+1 (= | t eeu tat

so that
Xn +1 (t)=2(%, u, f). (6.63)

The differential equation (6.63) is simply added to the original set (6.1)
together with the conditions

Xn +1 (to) =0, Xn +1 (ti) =e

and the procedure then continues as before, ignoring (6.62).

Exercise 6.5.
A system is described by

x4 =—2x, aed

and the control u(t) is to be chosen so as to minimize

| 1 u? dt.
0

Show that the optimal control which transfers the system from x, (0) = 1
to x,(1) =O is u* =—4e?"/(e* —1).

Exercise 6.6.
The equations describing a production scheduling problem are
226 Optimal control

where /(t) is the level of inventory (or stock), S(t) is the rate of sales and
a is a positive constant. The production rate P(t) can be controlled and
is assumed unbounded. It is also assumed that the rate of production
costs is proportional to P*. It is required to choose the production rate
which will change /(0) = Jo, S(O) = So into (7) =J,, S(T) = S, ina fixed
time T whilst minimizing the total production cost. Show that the optimal
production rate has the form P* =k, + k2¢tand indicate how the constants -
k, and k, can be determined.

Exercise 6.7.
A particle of mass m moves on a smooth horizontal plane with rectangular
cartesian coordinates x and y. Initially the particle is at rest at the origin,
and a force of constant magnitude ma is applied to it so as to ensure that
after a fixed time 7 the particle is moving along a given fixed line parallel
to the x-axis with maximum speed. The angle u(t) made by the force with
the positive x direction is the control variable, and is unbounded. Show
that the optimal control is given by

tan u* = tan Uo + ct
where c is a constant and ug = u*(0). Hence deduce that
y*(t) = (a/c)(sec u* —sec ug)
and obtain a similar expression for X*(¢) (hint: change the independent
variable from f¢to wu).

Exercise 6.8.
For the system in Example 6.7 described by eqn (6.57), determine the
control which transfers it from x(0) = 0 to the line x, + 5x2 = 15 and
minimizes

L fx,(2)—5] 2+ 4 [xi (2)—-2]?2+ a |’ u?dt.

6.3. Pontryagin’s principle

In real life problems the control variables are usually subject to constraints
on their magnitudes, typically of the form | u;(t) | <k;. This implies that
the set of final states which can be achieved is restricted (see Section
5.7.1). Our aim here is to derive the necessary conditions for optimality
Optimal control 997

corresponding to Theorem 6.2 for the unbounded case. An admissible

control is one which satisfies the constraints, and we consider variations
such that u* + 5u is admissible and || 5u || is sufficiently small so that
the sign of

AJ = J(u* + du) —J(u*),

where J is defined in (6.13), is determined by 6/J in (6.25). Because of

the restriction on 5u, Theorem 6.1 no longer applies, and instead a
necessary condition for u* to minimize J is

5J(u*, Su) > 0. (6.64)

The development then proceeds as in Section 6.2; Lagrange multipliers

are introduced to define J, in (6.26) and are chosen so as to satisfy (6.30)
and (6.31). The only difference is that the expression for 6J, in (6.32)
is replaced by

5J,(u,du)= [" [H(x,

u+8u,p,t)-H(x,up,D)dt. (6.65)
It therefore follows by (6.64) that a necessary condition for u = u* to be
a minimizing control is that 5/,(u*, 5u) in (6.65) be nonnegative for all
admissible 5u. This in turn implies that

H(x*,u*+du,p*, t) > H(x*,u*,p*,t), (6.66)

for all admissible 6u and all tfin tpg <t<1, ; for if (6.66) did not hold in
some interval t, <t<¢3, say, with t3 —?¢,arbitrarily small, then by
choosing 5u = 0 for ¢ outside this interval 6/,(u*, 5u) would be made
negative. Eqn (6.66) states that u* minimizes H, so we have established:

THEOREM6.3. (Pontryagin’s minimum principle) Necessary conditions

for u* to minimize (6.13) are (6.30), (6.31), and (6.66).
With a slightly different definition of H the principle becomes one of
maximizing H, and is then referred to in the literature as the maximum
principle. Note that u* is now allowed to be piecewise continuous.
Our derivation assumed that ¢, was fixed and x(t, ) free; the boundary
conditions for other situations are precisely the same as those given in the
preceding section and summarized in Table 6.1. It can also be shown that
when H does not explicitly depend upon ¢ then (6.39) and (6.43) still
hold for the respective cases when the final time ¢, is fixed or free.
228 Optimal control

Example 6.8. (Takahashi et al. 1970, p. 637). Consider again the ‘soft
landing’ problem described in Example 6.2, where (6.12) is to be minimi-
zed subject to the system equations (6.9). The Hamiltonian (6.28) is

H=|ul+k+pix. +pou. (6.67)

Since the admissible range of control is— 1 <u(t) < 1, it follows that H
will be minimized by the following:

u*(t) =—lifpx(t)>1
=(0if1>pk(t)>-1 (6.68)
=+1lifps(t)<-1.
Such a control is referred to in the literature by the graphic term bang—
zero—bang, since only maximum thrust is applied in a forward or reverse
direction; no intermediate nonzero values are used. If there is no period
in which u* is zero the control is called bang—bang. For example, a
racing-car driver approximates to bang—bang operation, since he tends
to use either full throttle or maximum braking when attempting to circuit
a track as quickly as possible.
In (6.68) u* switches in value according to the magnitude ofp¥(¢),
which is therefore termed the switching function, We must however use
physical considerations to determine an actual optimal control. Since
the landing vehicle begins with a downwards velocity at an altitude h,
logical sequences of control would seem to either

u* = 0, followed by u* = + 1

(upwards is regarded as positive), or

u* =— 1, thenu* =0, thenu* =+ 1. (6.69)

Consider the first possibility and suppose that u* switches from zero to
one at time f,. By virtue of (6.68) this sequence of control is possible if
pz decreases with time. It is easy to verify that the solution of (6.9)
subject to the initial conditions (6.10) is
xf=h-vt,xf=-v,0<t<t,
(6.70)
xt =h=vt+d(t-1,)?, xf =-v +(tt), t, <tST,
Substitutingthe softlandingrequirements(6.11)into (6.70)gives
T=hiv + dv, ty=h/v—4y. (6.71)
Optimal control 229

Because the final time is not specified and because of the form of A in
(6.67), eqn (6.43) holds, so in particular (H),,=,,* = 0 at t = 0, i.e. with
t = 0 in (6.67)
k + p¥(0)x7(0) = 0

or
pi(0) =k].
The adjoint equations (6.30) are
pt=0, Bf =—pt.
Hence
Di(t) =k/v,t>0
and
p3(t) =— kt/v-—1+kt,/v. (6.72)

using the assumption that p}(t, ) =— 1. Thus the assumed optimal control
will be valid if t; >0 and p3(0) < 1 (the latter conditions being necessary
since u*(0) = 0), and using (6.71) and (6.72) these conditions imply that
h>tv?,k<2v?/(h- $y?). (6.73)
If these inequalities do not hold then some different control strategy,
such as (6.69), becomes optimal. For example, if k is increased so that
the second inequality in (6.73) is violated then this means that more
emphasis is placed on the time to landing in the performance index
(6.12). It is therefore reasonable to expect this time would be reduced
by first accelerating downwards with u* = —1before coasting with
u* = 0, as in (6.69). It is interesting to note that provided (6.73) holds
then the total time 7 to landing in (6.71) is independent of k.
Example 6.9. Suppose that in the preceding example it is now required
to determine a control which achieves a soft landing in the least possible
time, starting with an arbitrary given initial state x(0) =x 9.The perfor-
mance index is just (6.3) with tp = 0, t; = 7. The Hamiltonian (6.28) is
now
H=\1 + P1X2 + p2u

and Theorem 6.3 gives the optimal control:

MAS
7h1,p} <0;u*=-1,p, ao,
230 Optimal control

or more succintly,
u*(t) = sgn p2) (6.74)

where the signum function is defined by

sen(z)=+1,z>0
=e li)
The optimal control thus has bang—bang form, and we must determine
the switching function p3(t). Using (6.30) we again obtain

Nie)

where c, and c, are constants. Since pj is a linear function of ¢ it can

change sign at most once in 0 <t<7, so the optimal control (6.74)
must take one of the following forms:

Hi OSs 7
a sc < tea
u*(t) = (6.75)
+105 fst trea
=1,0<1<t,;41,<7T<7

Integrating
thestateequations
(6.9)withu =+ 1gives
X, St Fl? +03f+04,x%2 =tt+c9. (6.76)
Eliminating
¢in(6.76)produces
x1(t)=3.x3(t)+s, whenu*=+ 1, (6.77)
x4(t) =—3.x3(t)+cg, whenu*=—
1. (6.78)
The trajectories (6.77) and (6.78) represent two families of parabolas,
shown in Fig. 6.3(a) and Fig. 6.3(b) respectively. The direction of the
arrows represents ¢increasing.
We can now investigate the various cases in (6.75):

(@) u*=+1,0<t<T. The initial state x9 must lie on the lower

part of the curve PO corresponding to cs = 0 in Fig. 6.3(a).
(ii) Similarly, if u* =— 1,0<t<T, thenxo must lie on the upper
part of the curve QO corresponding to cg = 0 in Fig. 6.3(b).
Optimal control 231

X2 Xy

xX, xy

Cra OMCal)
P C—O
a0 ee (0

(b)
FIG. 6.3

(iii) | With the third case in (6.75), since u* =—1 fort; <t<T it
follows that x*(t, ) must lie on the curve QO. The point x*(f, )
is reached using u* = + 1 forO <t<f,, so the initial part of
the optimal trajectory must belong to the curves in Fig. 6.3(a).
The optimal trajectory will thus be as shown in Fig. 6.4. The
point R corresponds to ¢ =f, , and is where u* switches from
+ 1 to— 1; QO is therefore termed the switching curve. By
considering Fig. 6.3 it is clear that the situation just described
holds for any initial state lying to the left of both PO and QO.

FIG. 6.4
232 Optimal control

(iv) A similar argument shows that the last case in (6.75) applies
for any initial state lying to the right of PO and QO, a typical
optimal trajectory being shown in Fig. 6.5. The switching now
takes place on PO, so the complete switching curve is QOP,
shown in Fig. 6.6.

FIG. 6.5

FIG. 6.6
 Optimal control 233

To summarize, if xo lies on the switching curve then u* = + 1 according

as X; (0) is positive or negative. If x» does not lie on the switching curve
then u* must initially be chosen so as to move x*(t) towards the switch-
ing curve.
We can now discuss a general linear regulator problem in our usual
form

x =Ax + Bu, (6.79)

where x(t) is the deviation from some desired constant state. The aim is
to transfer the system from some initial state to the origin in minimum
time, subject to | u; (t) | <k;. The Hamiltonian (6.28) is

H=1+p'(Ax +Bu)
=1+p'Ax+(p'h,,... sp)DmJe
m

=1+pTAx
+s (p"bu, (6.80)
i=1
_ where the b; are the columns of B. Application of Theorem 6.3 to (6.80)
gives the necessary condition for optimality that

uj*(t) =—k;sgn[s;(O],i=1,...,m, (6.81)

where
si (1)= [p*(O)1
"5; (6.82)

is the switching function for the ith variable. The adjoint equations
(6.30) are

pP=-
52(p*)"4x]
or
p* =—A'p*. (6.83)

The solution of (6.83) can be written as in (3.7) in the form

p*(t) = exp (—A‘t)p(0)

so the switching function in (6.82) becomes

si(t) = p' (0) exp (—At))j.

234 Optimal control

If s;(t) =0 in some time interval then u;‘(f) is indeterminate in this inter-

val. We now therefore investigate whether the expression in (6.82) can
vanish. Firstly, we can assume that b; # 0. Next, since the final time is
free the condition (6.43) holds, which for (6.80) gives

1+ [p*(t)]? (Ax* + Bu*) =0, all ¢,

so clearly p*(t) cannot be zero for any value of t. Finally, if the product
(p*)' b; in (6.82) is zero, then s; = 0 implies together with (6.83) that
3; (2) = —[p*()]* Ab; = 0,
and similarly for higher derivatives of s;. This leads to
[p*(t)]7 [b;,Ab;, A2b;,..., A” ~ 1 bj] =O. (6.84)

If the system (6.79) is completely controllable by the ith input acting

alone (i.e. u; = 0,7 #7)then by Theorem 4.1 the matrix in (6.84) is
nonsingular, and eqn (6.84) then has only the trivial solution p*(t) = 0.
However, we have already ruled out this possibility, so s; cannot be zero.
Thus provided the controllability condition holds there is no time interval
in which uj* is indeterminate. The optimal control for the ith variable
then has the bang—bang form uj* = + k; in (6.81). The following more
general result has also been proved (Boltyanskii 1971, Chapter 3):
THEOREM 6.4. If A is a stability matrix then there exists a unique
optimal control u* which transfers the system (6.79) from an arbitrary
initial state to the origin in minimum time. Furthermore if all the
characteristic roots of A are real then each uj‘(t) can switch at most
n— | times.
The second part of Theorem 6.4 states that each uj‘(t) consists of
at most n periods with constant values + k;. In general the switchings will
not occur simultaneously for each variable, so u*(t) will switch more
than n —1 times.

Exercise 6.9.

A system is described by
d°z
dp ( )
me ee]AU

where z(t) denotes displacement. Starting from some given initial position
with given velocity and acceleration it is required to choose u(t), which
is constrained by | u(t) | <k, so as to make displacement, velocity, and
 Optimal control 235

acceleration equal to zero in the least possible time. Show using Theorem
6.3 that the optimal control consists of u* = + k with zero, one, or two
switchings.

Exercise 6.10.
A linear system is described by

Z(t)+a2(t)+bz(t)=u(t)
where a > 0 and a* < 4b. The control variable is subject to | u(t) |<k
and is to be chosen so that the system reaches the state z(7) = 0, 2(7) =0
in minimum possible time. Show that the optimal control is u* = k sgnp(t),
where p(t) is a periodic function of f.

Exercise 6.11.
A rocket is ascending vertically above the earth, which is assumed flat. It
is also assumed that aerodynamic forces can be neglected, that the gravita-
tional attraction is constant, and that the thrust from the motor acts
_ vertically downwards. The equations of motion are

Ch
Eee af cu
oye dm_
gtin RU)

where A(f) is the vertical height, (¢) is the rocket mass, and c and g are
positive constants. The propellant mass flow can be controlled subject to
0 <u(t) <k. The mass, height, and velocity at time ¢ = 0 are all known
and it is required to maximize the height subsequently reached. Show
that the optimal control has the form
u*(t)=k,s>0; u*(t)=0, s<0
where the switching function s(t) satisfies the equation ds/dt =—c/m.
If switching occurs at time ¢, , show that

s(t)=+&nmea phat

=c(t; —f)/m(t,), th S<t<T.

Exercise 6.12.
A reservoir is assumed to have a constant cross-section, and the depth of
the water at time f is x, (t). The net inflow rate of water u(t) can be
236 Optimal control

controlled subject to 0 <u(t)<k, but the reservoir leaks, the differential

equation describing the system being

xX =-0.1x, tu.

Find the control policy which maximizes the total amount of water in
the reservoir over 100 units of time, i.e.

{ 100 x, (t)dt.
0

If during the same period the total inflow of water is limited by

iF 00u(t) dt = 60k

determine the optimal control in this case.

Exercise 6.13.
The following model can be thought of as representing a situation where
acup of hot coffee is to be cooled as quickly as possible using a finite
amount of cold milk, but could refer to a more general problem where
cold liquid is being added to a container of hot liquid. Suppose that the
coffee cup is originally full and at 100° C. The flow of milk, assumed to
be at a constant temperature, is represented by u(t) and is subject to
0 <u(t) <1. The total amount of milk is limited by

[pwde=1.
The thermodynamic equation describing the liquid in the cup is bilinear,
namely

e keNaas LOUa 1 UL
Nae

where the first term on the right hand side corresponds to heat loss to
the surroundings, the second term is due to the inflow of cold milk and
the third term is for the overflow. It is required to reduce the temperature
x, of the coffee to 0° C in minimum time 7. Show that the optimal
strategy isu* =0,0<t< 0.69; u* = 1, 0.69 <t < 1.69 (thus, somewhat
unexpectedly, there is a delay before the milk is added).
 Optimal control 237

Exercise 6.14.
In Example 6.9 let x, (0) = a, x2 (0) = 6 be an arbitrary initial point to
the right of the switching curve in Fig. 6.5, with a > 0. Show that the
minimum time to the origin is 7* = B + (4a + 267)?.

Exercise 6.15.
For the problem in Example 6.8 suppose that the control is now to be
chosen so as to minimize the total fuel consumption, measured by

(e |u(t)|de
0
where 7; is fixed. Let the initial state be as in Exercise 6.14. If the
optimal control has the form

Th OCRALISfy
u*(t) = iis b<
eto: Sat Sel «

| showthat
t,t, = [Ti +BF KE SID —(40+267)}2];
and deduce that 7, = T* where 7* is the minimum time in Exercise 6.14.

Exercise 6.16.
Consider again the system described by the equations (6.9) subject to
| u(t) | <1.It is required to transfer the system to some point lying on
the perimeter of the square in the (x; , x2) plane having vertices (+ 1, + 1)
in minimum time, starting from an arbitrary point outside the square.
Determine the switching curves.

6.4. Linear regulator.

We have noted that a general closed form solution of the optimal control
problem is possible for a linear regulator with quadratic performance
index. Specifically, consider the time varying system (3.29), i.e.

X(t) =A(t) x(t) +B(t) u(t) (6.85)

238 Optimal control

witha criterionobtainedby combiningtogether(6.4) and(6.8):

1 T(t.) Mx(t,)+ 4 le xTO(D tuTR(tu]dt (6.86)

with M and R(f) r.s.p.d. and Q(f) r.s.p.s.d. for t > 0 (the factors 4 enter
only for convenience). The quadratic term in u(t) in (6.86) ensures that
the total amount of control effort is restricted, so that the control variables
can be assumed unbounded.
The Hamiltonian (6.28) is

H=4x'Qx+4ulRutp! (Ax+Bu),

and the necessarycondition(6.33) for optimalitygives

“)
a, [Eu* Rut + (p*)"Bu*] =Ru* +BTp*= 0,

so that
u* =—R™!BTp*, (6.87)

R(t) being nonsingular since it is positive definite. The adjoint equations

(6.30) are

p* =—Qx*-A'p*. (6.88)

Substituting(6.87) into (6.85) gives

£* =Ax*—
BR™'B™p*

and combining this equation with (6.88) produces the system of 2n linear
equations

qf
dt | = A(t) — BR (QBY(t)][x*@
| . (6.89)
p*(t) SODcee Al) p*(t)
Since x(t, ) is not specified we have case (i) on p. 220, so the boundary
condition is (6.31) with ¢ = +x™Mx,ice.

p*(t,) = Mx*(t,). (6.90)

Optimal control 239

It is convenient to express the solution of (6.89) using eqn (3.28), but

in terms of the conditions at time ¢,, i.e.

aid @(t,ee))
t,) (01)

p*(t) 31 92]
| ie[x*(ty1 (6.91)

where ® is the transition matrix for (6.89). Hence

x*(t) = oix*(t,) + d2p*(t1)
= ($1 + o2.M)x*(t,) (6.92)
using (6.90). Equations (6.90) and (6.91) also give
D*(t) = ($3 + baM)x*(t; )

= (3 + d4M)(o1 + $.M)~* x*(t)

= P(t) x*(t), say, (6.93)
using (6.92) (it can be shown that ¢, + $M is nonsingular for all t > 0).
It now follows from (6.87) and (6.93) that the optimal control is of
linear feedback form
u*(t)=—R7~'(t) B'(t) P(t) x*(0). (6.94)
To determine the matrix P(t), differentiating(6.93) gives
Px*+ PR* —p* =0,

and substituting for X*, 5* from (6.89) and p* from (6.93) produces
(P+ PA —PBR7'B™P+Q+A"P)x*(t) =0.

Since this must hold throughout 0<t <¢;, it follows that P(t) satisfies
P=PBR~'B'’P—A™P-PA-Q (6.95)

with boundary condition given by (6.90) and (6.93) as

P(t,)=M. (6.96)
Equation (6.95) is often referred to as a matrix Riccati differential
equation. This is however rather a misnomer, as Riccati studied only
some special cases of the scalar version of (6.95) in the early eighteenth
240 Optimal control

century, before the introduction of matrices. Since M in (6.96) is

symmetric it follows that P(t) is symmetric for all t, so (6.95) represents
+n(n + 1) first order quadratic differential equations, which can be
integrated numerically.
Example 6.10. A wire is being wound onto a reel which is driven by an
electric motor. Let J(t) be the inertia of the reel and wire, and w(t) the
angular velocity at time ¢. The equation of motion is
2(t) =—k,z(H/J(t) + kyv(t) (6.97)

where z(t) = J(t) w(2), v(£) is the input voltage, k, is a friction coefficient
and ky a torque coefficient. The speed of winding the wire on the reel
is s(t) = r(t) w(t), where 7(t) is the total radius at time ¢. It is required
to regulate the system so that s(t) is kept at a constant value so. Since
2(t)/s(t) = J(o)/r(t)
the corresponding value of z(t) is
Zo(t) = SoJ(t)/r(¢),
and from (6.97)
() 1 ( fe ty k1Zo
v0 cs aes
ie Be
0 J .

Clearly /(t) and 7(t) increase with time, and w(t) must be decreased in
order to keep s(t) constant.
Define as state variables
x1 (t) = 2(t)—Zo(t), x2(t)=s(t)—So
and as control variable

u(t) = v(t) —vo(Z).

It is easy to verify that (6.97) becomes
X1 =—kyx1/J(t) + kyu,
which is in the form (6.85), and that
Xq =r(t)x1 /J(t).
A suitable index to be minimized could be

t +ku?)
Se(x3 dt,
 Optimal control 241

where k is a weighting factor, and the Riccati equation (6.95) is a scalar

equation:
B(t) = p?(t)k3/k + 2k, p(t)/I(t) —r?(HJ? (0),

subject to (6.96), namely p(t; ) = 0. The feedback control (6.94) is

u*(t) = —kyp(t)xi(O/k.

For a numerical case see the book by Kwakernaak and Sivan (1972,
p. 234).
It should be noted that even when the matrices A, B, Q, and R are
all time invariant the solution P(t) of (6.95), and hence the feedback
matrix in (6.94), will in general still be time varying. However of particu-
lar interest is the case when in addition the final time ft, in (6.86) tends
to infinity. There is then no need to include the terminal expression in
(6.86) since the aim is to make x(t, ) > 0 as t; >, so we set M= 0. Let
Q, be a matrix having the same rank as Q and such that Q = OF O07; plt
can then be shown (Kwakernaak and Sivan 1972, p. 237) that the solution
P(t) of (6.95) does become a constant matrix P, and we have:

THEOREM 6.5. If the constant linear system (6.79) is c.c. and [A, Q;]
is c.o. then the control which minimizes

iP (x?Ox+ u?Ru)dt (6.98)

is givenby
u* =—R™'B'Px, (6.99)
where P is the unique r.s.p.d. matrix which satisfies the so-called algebraic
Riccati equation

PBR7'B'P-A'P-PA-Q=0. (6.100)
Equation (6.100) represents tn(n + 1) quadratic equations for the
unknown elements of P, so the solution will not in general be unique.
However, it is easy to show that if a positive definite solution of (6.100)
exists (which is ensured by the conditions of Theorem 6.5) then there is
only one such solution (see Exercise 6.20).
The matrix Q,; can be interpreted by defining an output vector
y = Q;x and replacing the quadratic term involving the state in (6.98)
by y'y(= x7 QF Q;x).
242 Optimal control

The closed loop system obtained by substituting (6.99) into (6.79) is

X=Ax (6.101)

where / = A—BR~'B"P. It is easy to verify that

S™P+ Pg = ATP+PA-—2PBR=!B'P
(6.102)
=- PBR '*B'P—O,

using the fact that P is the solution of (6.100). Since R™is positive
definite and Q is positive semidefinite the matrix on the right in (6.102)
is negative definite, so by Theorem 5.17 & is a stability matrix. It can
also be shown (Kucera 1972) that if the triple [A, B, Q,] is not c.c. and
c.o., but is stabilizable and detectable (see Section 5.7.2) then eqn. (6.100)
has a unique positive semidefinite solution, and the closed loop system
(6.101) is asymptotically stable.
Thus solution of (6.100) leads to a stabilizing linear feedback control
(6.99) irrespective of whether or not the open loop system is stable. This
provides an alternative to the methods of Section 4.4. For example, in
the linearized model of the inverted pendulum problem presented in
Exercise 5.49, the system can be stabilized by linear feedback in all the
state variables, and this could be generated via (6.99) with a suitable
choice of elements of Q and R in (6.98).
If x*(£) is the solution of (6.101), then as in (6.16) eqn. (6.102)
implies

. [@*)'Px* =") (PBRYBT? + Oye

=—(u*)' Ru*—(x*)' Ox* (6.103)

using(6.99).Since ®W
isa stabilitymatrix,wecanintegrateboth sides
of (6.103) with respect to t from zero to infinity to obtain the minimum
value of (6.98):

[& *)Ox*+(u*)Rudt =x3Pxo, (6.104)

using the same argument as for (6.18). Of course when B = 0, (6.100) and
(6.104) reduce simply to (6.17) and (6.18) respectively.
A number of methods for solving (6.100) have been developed. One
Optimal control 243

relies on calculating the right characteristic vectors of the matrix in

(6.89), namely
A ERs LBs
A= (6.105)
Q = At

in which A, B, Q, and R are now time invariant. Assuming that L has a

diagonal Jordan form, if a,,..., a, are characteristic vectors of L
corresponding to roots having negative real parts and a} = [a7 ; 7} ie
where ;, yj are column n-vectors, then the r.s.p.d. solution of (6.100) is
Be yes es ynl [675 Cee Bn bot
(for a proof see Barnett 1971, p. 122).
The frequency domain interpretation of Theorem 6.5 is interesting.
First, suppose m = 1 so B is a column vector b, and without loss of
generality we can set R = 1 in (6.98). It can then be shown (Anderson
and Moore 1971, p. 69—73) that a necessary and sufficient condition
for optimality is

[1+ t(iw)|>1 (6.106)

where
t(s) =k(sI -—A)~*'b (6.107)

and k is the feedback matrix b’P obtained from (6.99). It is easy to see
that (6.106) implies that the Nyquist locus of ¢(s) must not enter the
disc of unit radius centred on —1 + i0. It can be shown that the
gain margin (see Section 5.3) of the closed loop system is infinite and
that the phase margin is at least 60°. These are larger than necessary, and
this constitutes a practical disadvantage of using Theorem 6.5 for genera-
ting linear feedback.
For the multivariable case m > 1, eqns. (6.106) and (6.107) have been
generalized by MacFarlane (1970). It turns out that
A +t, Gico)| > 1, 7 = 1s i,
wherethe functions 1 + ¢,(s)are the characteristicroots of the optimal
return differencematrix
F(s) =Im +K(sl-—A) 1B
with K = R~!B"P, so the characteristic loci (see p. 172) must not enter
the unit disc centred on —1 + i0.
244 Optimal control

For linear tracking problems the performance index (6.8) is used,

together with the terminal expression (6.4) if ¢, is finite. Because of the
quadratic term in u the optimal control is again linear state feedback as
in (6.99) but contains an added term which depends upon r(f) (see
Exercise 6.22).

Exercise 6.17.
Use Theorem 6.5 to find the feedback control which minimizes

be(x2+0.1u?)
dt
subject to
x1 SN ait: tb X2 Chis

Exercise 6.18.
Use the Riccati equation formulation to determine the feedback control
for the system

Ka SaX4 tees

whichminimizes
S {1(3x?+u?)dt.
0
(Hint: in the Riccati equation for the problem put P(t) =~ w(t)/w(2)).
If the system is to be transferred to the origin from an arbitrary
initial state with the same performance index, use the calculus of
variations to determine the optimal control.

Exercise 6.19.
If the matrices X(t), Y(t) satisfy

blll
where L is defined in (6.105), show that P(t) = Y(t)X~1() is a solution
of eqn (6.95) with A, B, Q, R time invariant.
Optimal control 245

Exercise 6.20.
If P, and P, are two positive definite solutions of (6.100) show that

(P; —P,)(A-BR™'B'™P,)+(A! —P,BR71B'\(P; —P,)=0.

Hence deduce, using the uniqueness condition derived for eqn. (2.37),
that P, = °

Exercise 6.21.
Let y = Cx be an output-vector for the system (6.79), where C is a
constant matrix. If u is chosen so as to minimize

£ Pe(vyQy+u™
Ru) de,
where Q and R are both r.s.p.d., write down the appropriate Riccati
equation corresponding to (6.100) and denote its positive definite
solution by P,; (the system is assumed c.c. and c.0.).
Consider the dual system (4.25) with A, B, C time invariant, and
suppose that for this system wuis to be chosen so as to minimize

Bes(vIR7!y
+u™Q>1u)
dt.
Show that the positive definite solution of the associated Riccati equation
nPy.

Exercise 6.22.
Consider the optimal linear tracking problem in which it is required to
minimize (one half) the sum of (6.4) and (6.8) subject to (6.85). It can
be shown (Kirk 1970, p. 220) that eqn. (6.93) is replaced by

p*(t) = P(t)x*(t) + s(2).

Show that the optimal control has the form u*(t) + v(t), where u*(f) is
given by (6.94) with P(t) satisfying (6.95) and (6.96), and
v(t) =—R7! BY s(t) where

§(t) =[PBR~'BT—AT] s(t) +Or(t),s(t) =-Mr(ty).

246 Optimal control

6.5. Dynamic programming

The basic idea of dynamic programming can be understood through a

simple example.
Example 6.11. Suppose a road map is available on which distances
between all junctions are marked. By trial the shortest route between two
particular junctions A and B could be found, as indicated by arrows in
Fig. 6.7. If C is some intermediate point on this shortest route, then it
follows that CDB must be the shortest path between C and B; for if some
other path, such as CEB were shorter, then this would provide a shorter
route from A to B.

ES |
FIG. 6.7

At each road junction a decision has to be made as to which direction

to take, and the problem is an example of what is called a multistage
decision process. The argument we applied to obtain the property of an
optimal route is expressed more formally by the Principle of Optimality
(P.O.) (Bellman 1957, p. 83):
An optimal policy has the property that whatever the initial state and
initial decision are, the remaining decisions must constitute an optimal
policy with regard to the state resulting from the first decision.
Example 6.12. Find the set of numbers x,,... , X» satisfying
xX,++ *°+ +X, =c, where c is a positive constant, such that the product
X4X2...Xy is amaximum.
To apply dynamic programming, define the function f,,(c) as the
Optimal control 247

maximum attainable value of the product for given values of c and n.

Clearly f,(c) =c, and

fr(c) = ' max fey (cax,)} (6.108)

<x, <c

since x, =c—x,. The expression to be optimized in (6.108) is a function

of the single variable x, , and it is easy to verify by elementary calculus that

fr(c) =c?/4 (6.109)

when x, =4c¢=x>. Suppose that we have solved the problem for

n=2,3,...,m,and consider the case nm =m+ 1. The maximum attain-
able product for c—x, divided into m parts is by definition f,,(c —x,).
Regarding x, as the initial decision, we can conclude from the P.O. that

Fmt 1c) = ; Pee {1 tm(c —x1)}} (6.110)

and this agrees with (6.108) when m = 1. When m = 2, (6.110) gives

f3(c)= 0pags: £1 (c —x1)7/4}

<x, <

using the expression for f, in (6.109). Again using calculus of one variable
it is easy to show that
fa(c) = 3/27 (6.111)

with x, =c/3, and since c —x, = 2c/3 it follows by our result for n = 2
that x. =x3 =c/3. In view of (6.109) and (6.111) it is reasonable to
conjecture that
fn(c) = (e/ny" (6.112)
with
xX,=X_ =° °° =X, =Ce/n, (6.113)

and we now prove by induction on n that this is indeed correct. From

(6.110) and the induction hypothesis we have

LOTees tseo 1
248 Optimal control

and again by straightforward calculus the optimal value of x, is found to

bec/(m+1), sothat
mt+1
G (c eee= m ie P=
In ero

sete Ut(pated)aoc
establishing (6.112). Also c -x, =cm/(m + 1) so by the induction
assumption x, =x3 =* * * =Xm =c/(m + 1), and this verifies (6.113)
In general it is not possible to obtain analytical results, as in the
preceding example. However, dynamic programming is essentially an
ingenious way of reducing a multistage decision process to a sequence of
single stage processes, and as such is a useful tool for constructing
algorithms for solving optimization problems. We now sketch an appli-
cation to a simple optimal control problem.

Example 6.13. Consider a two-stage process described by the scalar

difference equation
x(k + 1) = ax(k) + bu(k), k =0, 1, (6.114)
where a and b are constants. It is required to choose u(0) and u(1) to as
to minimize the quadratic performance index

qix? (2) + q2u?(0) + q3u7(1),

where the q; are nonnegative weighting factors, subject to the constraints
that x(k) and u(k) take certain discrete values within prescribed finite
ranges.
We first find the optimal policy for the last stage of the process: put
k = 1, and for each allowable value of x(1) calculate the cost

Jia = qix? (2) +q3u?(1)

of going from x(1) to x(2), the latter being evaluated from (6.114). Of
course J; is a function of x(1) and u(1), and we can write

J [x(1)]=minJ12.
The penultimate stage k = 0 is then considered. By the P.O. the minimum
cost over the last two stages is

min{Jor[x(0),u(0)] +J73[x(1)]}, (6.115)

Optimal control 249

where Jo, = q2u*(0) is the cost in the interval k = 0 to k = 1. The procedure

represented by (6.115) can be converted into an algorithm suitable for
computer application, and can obviously be extended for processes invol-
ving more than two stages. For specific details see the book by Kirk (1970,
p. 60). In this same text (p. 78) a development is given for the general
discrete linear system (3.31) subject to a quadratic performance index,
providing a discrete analogue of the continuous-time regulator problem of
Section 6.4.
We now derive a continuous form of dynamic programming by con-
sidering again the problem of Section 6.2 of minimizing (6.13) subject to
the system equations (6.1). Define

TO), tw] = Obes), 1] + fl FEL), u(r), 7] ar,

with t <7 <1f,, and write

J*[x(0), t] = = J (x(t), t, u(7)]

"
ic Far+ ae Far+o[x(¢1),t1]}
= =)
(6.116)
ein eng Fdr+J*[x(t
+Ad),
t+Ar]}
the last step following by the P.O. Assuming that J* can be expanded in
a Taylor series we have

A ttat oJ*
J* [x(t),
(x(t), t]t] = mi
— {{, F ar + 42J*[x(7),
tf] + pains
yi At

rN
ha
fia. [x(t+At)—x(t)]
+higher
order
terms
i (6.117)
Since J* is independent of u, the terms J*[x(f), t] cancel out in (6.117),
and if Af is small the resulting expression can be written as
oJ*
O=min(Farpo npr *earike terms
)'
u ot Ox
using (6.1). Dividing by Ar and letting At > 0 produces
*

ie = chant
GN: (6.118)
ice
250) Optimalcontrol

where by analogy with (6.28) the Hamiltonian is

a* \ at
a(x,u,Zr) =r+ fi+ au*
aefie wG:1
Eqn (6.118) is a partial differential equation to be solved for J*, subject
to the boundary condition obtained by putting ¢ = ¢, in (6.116), namely
J*[x(t1), 01] = Oi), 41]. (6.120)

Example 6.14. Find the control which minimizes

xi(T)+ |"‘e u? at
0
subject to the scalar equation
xy ae urile
The expression (6.119) gives

H=u? fe * (x, +u), (6.121)

Ox,
and assuming that u is unbounded a necessary condition for optimality is
(6.33) in Theorem 6.2, namely

OH ee
y 0) ee GE
Substituting u* =— +dJ*/dx, into (6.121) gives

minH=4Ux,)?+Jz,(1—
2, )
u
=—403,)? txiJz,,
andeqn.(6.118)thenbecomes
(LM a 4a(2) .
ot Ox,
subjectto (6.120)whichisjust
J*[x(1), T] =xi(7).
In general such a partial differential equation must be solved numerically,
but in fact in this example an analytical solution is possible (see Exercise
6.24).
 Optimal control 201

We close by noting that the Pontryagin principle (Theorem 6.3) can

be derived from the dynamic programming equation (6.118) (see Kirk
1970, p. 417).

Exercise 6.23.
Use dynamic programming to find the numbers x;, x2, x3 which mini-
mize the sum x? + 2x3 + 3x3 subject tox, +x, +x3 =c, wherecisa
constant.

Exercise 6.24.
In Example 6.14 assume that J* takes the form k(t)xj, and hence obtain
the optimal control. Deduce that as T > © this approaches constant
linear feedback.
References

ANDERSON, B. D. O. and MOORE, J. B. (1971). Linear optimal control.

Prentice—Hall, Englewood Cliffs, New Jersey.
BARNETT, S. (1971). Matrices in control theory. Van Nostrand Reinhold,
London.
and STOREY, C. (1970). Matrix methods in stability theory.
Nelson, London.
BELLMAN, R. (1957). Dynamic programming. Princeton University Press,
Princeton, New Jersey.
(1970). Introduction:to matrix analysis, 2nd edn. McGraw—Hill,
New York.
BOLTYANSKII, V. G. (1971). Mathematical methods of optimal control.
Holt, Rinehart and Winston, New York.
BRAUER, F. and NOHEL, J. A. (1969). The qualitative theory of ordinary
differential equations. Benjamin, New York.
BROCKETT, R. W. (1970). Finite dimensional linear systems, Wiley, New
York.
BRYSON, A. E. Jr. and Ho, Y. —C. (1969). Applied optimal control.
Blaisdell, Waltham, Massachusetts.
CADzow, J. A. (1973). Discrete-time systems. Prentice—Hall, Englewood
Cliffs, New Jersey.
CHEN, C. —T. (1970). Introduction to linear system theory. Holt, Rinehart
and Winston, New York.
Cook, P. A. (1973). In Recent mathematical developments in control
(ed. D. J. Bell), p. 367. Academic Press, London.
CULLEN,C. G. (1972). Matrices and linear transformations, 2nd edn.
Addison—Wesley, Reading, Massachusetts.
D’ANGELO, H. (1970). Linear time-varying systems. Allyn. and Bacon,
Rockleigh, New Jersey.
DESOER, C. A. (1970). Notes for a second course on linear systems. Van
Nostrand Reinhold, New York.
DIRECTOR, S. W. and ROHRER, R. A. (1972). Introduction to systems
theory. McGraw—Hill, New York.
ELGERD, O. I. (1967). Control systems theory. McGraw—Hill, New York.
EVELEIGH,
V.W.
McGraw— ( 1972).
Introduction
Hill,NewYork. t ocontrol
s ystems
d esign.
FULLER, A. T. (Ed. ) (1974). Stability of motion. Taylor and Francis,
London.
GANTMACHER, F. R. (1959). The theory of matrices, 1 and 2. Chelsea
Publishing Company, New York.
HAHN, W. (1963). Theory and application of Liapunov’s direct method.
Prentice—Hall, Englewood Cliffs, New Jersey.
References 253

HOLLAND, J. M. (1972). Studies in structure. Macmillan, London.

INTRILIGATOR, M. D. (1971). Mathematical optimization and: economic
theory. Prentice—Hall, Englewood Cliffs, New Jersey.
JurRY, E. I. (1964). Theory and application of the z-transform method.
Wiley, New York.
KALMAN, R. E., FALB, P. L., and ARBIB, M. A. (1969). Topics in
mathematical system theory. McGraw—Hill, New York.
KIRK, D. E. (1970). Optimal control theory. Prentice—Hall, Englewood
Cliffs, New Jersey.
KUCERA, V. (1972). LE.E.E. Trans, autom. Control, 17, 344.
KWAKERNAAK, H. and SIVAN, R. (1972). Linear optimal control
systems, Wiley—Interscience, New York.
LANCASTER, P. (1969). The theory of matrices. Academic Press, New
York.
LA SALLE, J. P. and LEFSCHETZ, S. (1961). Stability by Liapunov’s
direct method with applications, Academic Press, New York.
LEHNIGK, S. H. (1966). Stability theorems for linear motions with an
introduction to Liapunov’s direct method. Prentice—Hall, Englewood
Cliffs, New Jersey.
LEIPHOLZ, H. (1970). Stability theory. Academic Press, New York.
LEVINSON, N. and REDHEFFER, R. M. (1970). Complex variables.
Holden—Day, San Francisco.
MACFARLANE, A. G. J. (1970). Proc, Instn, elect, Engrs, 117, 2037.
(1972). Automatica, 8, 455.
MARDEN, M. (1966). Geometry of polynomials. American Mathematical
Society, Providence, Rhode Island.
MAYR, O. (1970). The origins of feedback control. M.I.T. Press,
Cambridge, Massachusetts.
MIRSKY, L. (1963). An introduction to linear algebra, Clarendon Press,
Oxford.
MOHLER, R. R. (1973). Bilinear control processes. Academic Press,
New York.
PORTER, B. and CROSSLEY, R. (1972). Modal control. Taylor and
Francis, London.
ROSENBROCK, H. H. (1970). State-space and multivariable theory.
Nelson, London.
(1973). In Recent mathematical developments in control (ed.
D. J. Bell) p. 345. Academic Press, London.
RuBio, J. E. (1971). The theory of linear systems. Academic Press,
New York.
SAUCEDO, R. and SCHIRING, E. E. (1968). Introduction to continuous
and digital control systems. Macmillan, New York.
SCHULTZ, D. G. (1965). In Advances in control systems.2 (ed. C. T.
Leondes), p. 1. Academic Press, New York.
SILJAK, D. D. (1969): Nonlinear systems, Wiley, New Yrok.
SILVERMAN, L. M. (1971). .E.E.E. Trans, autom. Control, 16, 554.
SRIDHAR, B. and LINDORFF, D. P. (1973). Int. J. Control, 18, 993.
 \

254 References

TAKAHASHI, Y., RABINS, M. J. and AUSLANDER, D. M. (1970).

Control and dynamic systems. Addison—Wesley, Reading,
Massachusetts.
TRUXAL, J. G. (1972) Introductory systems engineering. McGraw—Hill,
New York.
WILLEMS, J. L. (1970). Stability theory of dynamical systems. Nelson,
London.
ZADEH, L. A. and DESOER, C. A. (1963). Linear system theory.
McGraw—Hill, New York.
Answers to exercises

Chapter 1
12s 0 0) j 0) 0)
0 0 0 1 0
A= ; = ’
—k,/m, k,/m, 0 0 I/m,
Ky/m, —(ki +kz)/m, 0 0 0
123. Xy =X2,X2 = (3g/2) sin x, —3x3 sinx, cos x, —3X, sin 7x, +
(3M/m2)X4 cos x1,X3 =X4,
X4 =—(m/M) (Kq + WM.cos x, —Lx}?sin x,).
1.6. g182/(1 +182h —8183/).
1.7. x4(t) = [a2x2(0)—agx1(0) + €7(aix1 (0) —42x20) 4, d= ay ~ a4;
x,(t) =x, (0), x2(¢) = x2(0), all t > 0, irrespective of d.
1.9. gi(1 —g183h)/(1+g182h —g183h)’.
1.10. (@)Tz/(z-1)? (b) z/(z-e”*).
1.13. Fr =(0-096F>+ 0-85F,\(1-:063)*+ (0:906F'—0-85F;)(—0-1faye
My,=a, (1:063)*+a, (—0-113)*+45 (0-95),
a; constants.
Chapter2
sin 5+sin 1 sin 5 —sin 1
2.26. Allfinite A;sin A= + :
Sinvoasitin Sinvor-tasin:

en, 2 (Supa de ewg

st AM =}
se 3 Sri lile So tls

Chapter3

Sealx,(t)
(@)|=(2x 1|+(x3(0)
,(0)-x2(0))e** 1
-s,0)e|
l 2
256 Answers to exercises

x(t) oak b
(b)|x2(t)|=@20)-Fx3(0)e?
| 1|+ 10) +x2(0))e*"]
-1
x3(t) 0 Ww
=a

+(x1(0)+x2(0)+ $x3(0)e**| 1
2

Wo) x(t)= (ef +262")et +e2) | x(0);A= -3 -1 :

Ae -e2') (et aaa 2 0

BiGr cos wt sin wt

@= :

sinwt coswt
wes (1+de%! tert!
x(t) = x(0).
—te3! (1-fe734

3.14. e'(1 +2) —te?! reat

=e +2 +?) ef —te2! | !

=e ee ef —e2t e2t

3.15. 2(t)—z(0)= t2(0) + Fut?.

3.16)Lpapiereat’ anaes
3 } Ty+ (2/9\(-1 +3t + &9*).
Laer MEPore?
3.20. ie = 510 (—210043100)
(218= 9.31%) 71 4.9.31) I

3.26 iw 2)*—2-3)* (-2)* -(-3)*

~6(-2)* +6(-3)* =—2(-2)* +3(-3) FN
Answers to exercises 25

3.20% —2.4k —4k

aor
-2-4k
4k
3.30. =3 1
@) T=3 | A22) +5720) 4 6z =u.
Seal

wot] =P 5= 23).
2) —67) +112) - 62 =u.

=3) = 6

3.31, eee
ae k Kia
Bie: be
ki

3:32 ' 1 1 a
ee C213 Oe Se

0 0 6

Chapter 4
4.2. c.c. if and only ifk,; #k.

4.4. 0 0 1 0
0 0 0 1
A=
~K/m, k,/m, —d,/m, d,/m,
ky /m, —(ky +k2)/my d,/m, —(d, +d)/m,

0
%* 0
| Amy
ky #4.
258 Answers to exercises

4.5. —20(9e?+ Se?+ Set 5)e’+ 120(2e?+e + 1)e”!

e> +e+ —Se? + 8e? —e—-1 ;
4.8. b, = +b, , 5, =b,. Cancellationsoccur.
4.12: [1,.0,0)"; [0, 0,1) 225 4cl4e ves: 4.15. Yes
4.16. X,=(-x, +u)/RiG x, =(—x2R,2 tu)/L,

y=(—x1 +Rix2 tu)/Ri.

4.17. Cy =—C2,¢, =—2c,. Modes vanish.
4.18. No. 4.19. [3,-1]. 4.29. [11,—-5].
4.30. 4[37,85,-64].
4.32. 3 a a

K= pees 371:

3 -4 1
4.38. —[33/2,14,6]!.

4.43, A=diag[s,,52,...,5p],B=[1,1,...,1]1,C=[ki,.-.,
kp).

4.46, Tegan!
[=r=| , 4.|
A=diag[1,—-1,-2],
B= ,C= 10/36 2 112

4.53. O-11F> + 0°89F;, ; no change in percentage killed, but steady

state, 0-059 + 0-95F,, reached more quickly.

Chapter 5
51. (2,4).91=—
2y21+2),32=Vi1
V2+2).
53; kx? +x2 =c. 5.6. (a) Yes (b) No (c) No (d) Yes.
527. [exp(At)](x9-A™'b)+A‘ Dd.
535 One inside, three outside. RAY, KA 2.

5.12. Unstable. 5.14. 0<k <2; u=$(— 2x —3x2—2x9).

5488 J(a)=-3()-—1,
Answers to exercises 259

52198 two.

FIG. Al

5.20 U unstable.

5tes. 48° 5.22. a=b=%,c=d=1; Asymptotically

stable in large.

5.23. 25x? + 10x,x. + 10x? -9 <0. 5.24. unstable.

5.25. Unstable;stable. 5.26. q(0,0)=0;q2-1,x #0.

Sree — 2.

5.30. 1 13 ma!
P= 30 ; asymptotically stable.
=1 4
5.33. Asymptotically stable; +<<k<4;k>0.

5.39. detA>0,trd <0. 5.41. (@)unstable (b) asymptotically stable.

260 Answers to exercises

5.42. OF Sle O rea)

fy WD OC _3g(M +m) 8) fii
= 0 0 0 1 Xia, “14M +m)? 7? \ 4M epee

a, 0 0 0
5.44. $x? +x3;x? + 2x3 <2. 5.46. stable. 5.47. asymptotically
stable.
5.48. No. 5.49. [0,—3/2,0, 4]! u/(4M +m); stabilizable by u = Kx.
5.54. uz = qjwj, qj (w) <0.

Chapter 6
6.3. xd(P—3P,+2P)xXo.
6.7. 4c (oasec u*+tan
+tan u*
ug)

6.8. 26 +0-04e". 6.12.u*=k,0<t<100;u*=k,0<t<60;

u*=0,60<t<100.
6.16. x, = 4x3 +4, to(1,—1);x, =—4x3 —4, to, 1).
(HIG 1 7/Ilbie,stole’ ye,
6.18. u*=3(ky —k2)ei/(ky + 3k2); u* =(Bk, + koe /(ki —ka);
kK,=exp[2(t—1)], k2 =exp[201—-4)].

6.23. ‘x, =6c/11,x,. =3e/11,x3 =2ce/11.

6.24, ut=—2e7(T
— Oy,11+eT —),
Index

adjoint equations, 218 function

autonomous equations, 142 positive definite, 174
positive real, 200
Barbashin theorem, 178 radially unbounded, 176
block diagram, 3 rational, 31
Bode plot, 14 signum, 230
Bolza problem, 212 unit step, 13
buffalo problem, 21, 136, 158, 199
gain, 14
calculus of variations, 216
gain margin, 171
canonical form, 33
controllable, 88 Hamiltonian, 217
Jordan, 45 Hermitian form, 52
observable, 101 Hooke’s law, 10
Smith, 41 Hurwitz stability theorem, 153
Cayley—Hamilton theorem, 34
characteristic loci, 172, 243 instability, 143
circle criteria, 206 complete, 144
control, 1 inverted pendulum problem, 9, 10,
admissible, 227 150, 191, 206, 242
bang—bang, 228 isoperimetric constraint, 225
optimal, 209
piecewise continuous, 85 Kronecker delta, 63
variables, 2 Kronecker product, 24
controllability, 84
controllability and polynomials, 104 Laplace transform, 11
Leverrier algorithm, 34
Liapunov, 144
detectability, 199 continous matrix equation, 184
duality, 99 direct method, 174
dynamic programming, 246 discrete matrix equation, 186
instability theorem, 181
linearization theorem, 187
encirclement, 169
stability theorems, 175, 191
equilibrium (critical) point, 142
Liapunov function, 174
extremal, 216
construction of, 191
variable gradient method, 192
feedback, 3 , Zubovy’smethod, 194
linear, 107, 198, 239, 242 Lienard—Chipart stability criterion, 156
nonlinear, 200 linear difference equation, 17
output, 118 linear differential equation, 11
unity, 19, 112 linear system, 6
finite escape time, 146 adjoint, 70
first approximation, 187 algebraic equivalence, 92
functional, 212 closed loop, 107, 242
262 Index

linear systems (contd.) order, 23

continuous-time, 11, 56, 84 orthogonal, 35
controllability, 84 partitioning, 24
detectability, 199 polynomial, see polynomial matrix
discrete-time, 16, 56, 84 principal minor, 54
dual, 99 rank, 29
feedback, 107, 198, 242 rank defect, 29
observability, 97 return difference, 112
order, 39 sequence, 48
reachability, 135 series, 48
realization, 121, 135, 137 similar, 33
self-adjoint, 70 similarity invariants, 42
similarity, 135 singular, 25
spectral form solution, 57 skew Hermitian, 55
stability criteria, 150, 184 skew symmetric, 55
stabilizability, 199 spectrum, 57
time invariant (constant), 6 stability, 150
time varying, 67, 137, 161 state transition, 59, 68, 72
unforced, 12 symmetric, 52
with quadratic performance traces, o2
index, 237 transfer function, 38, 121
linearization, 187 transpose, 23
triangular, 25
matrix, 23 unit, 23
adjoint, 25 unitary, 35
almost constant, 161 Vandermonde, 36, 118
block diagonal, 24 weighting pattern, 137
characteristic roots (eigenvalues), 32 minimum effort, 91, 210
characteristic vectors minimum-time problems, 209
(eigenvectors), 33 modal control, 116
canonical form, 33 mode, 116
companion form, 34, 75, 81, 108 multistage decision process, 246
conjugate transpose, 23
controllability, 86, 88 Nyquist
convergent, 158 criterion, 167
derogatory, 42, 80, 88 inverse array, 173
determinant, 25 locus (diagram), 167
diagonal, 24
element, 23 observability, 97
euclidean norm, 47 output variables, 2
exponential, 59 oven problem, 5, 38, 66, 76, 87, 100
feedback (gain), 107
function, 49 performance index, 209
Hermitian, 52 quadratic, 211
Hurwitz, 153 evaluation of, 212, 242
inverse, 25 phase margin, 171
Jordan form, 45, 80 phase plane, 149
Kronecker product, 24 phase variables, 75
minimum polynomial, 42 polynomial, 30
minor, 26 and controllability, 104
nonderogatory, 42, 44, 80, 109 characteristic, 12, 32
nonsingular, 25 convergent, 159
observability, 98, 100 degree, 30
Index 263

greatest common divisor (g.c.d.), 31, definitions, 143

105 domain of attraction, 181
Hurwitz, 153 in sense of Liapunov (i.s.L.), 145
leading coefficient, 30 Lagrange, 144
minimum, 42 matrix, 150
monic, 30 of continuous linear systems, 150,
resultant, 31 184
toot (zero), 31 of discrete linear systems, 158,
polynomial matrix, 38 186
determinantal divisors, 41, 105 of optimal linear regulator, 242
elementary divisors, 41 of time-varying linear systems,
elementary operations, 40 161
invariant factors, 41 practical, 148
tank, 40 region of asymptotic, 179
Smith form, 41 relative, 170
Pontryagin principle, 227, 251 uniform, 147
Popov line, 203 stabilization, 199
Popov theorem, 201 state observer, 119
Principle of Optimality (P.O.), 246 state space, 2
state variables, 2
quadratic form, 52 switching curve, 231
positive (negative) definite, 53 switching function, 228
Sylvester formula, 49, 60
rabbit—fox problem, 10, 19, 62, 102, Sylvester conditions, 54
117, 145, 152, 199 system, 1
bilinear, 198
reachability, 135
closed loop, 3
realization, 121, 135
discrete-time, 16
minimal, 121
dynamical, 3
order, 121, 131
P f linear, 6
time varying, 137 Se
regulator, 11, 211, 233, 237, 249 multivariable, 13
open loop, 3
return difference, 16
matrix, 112
terminal control, 210
Riccati equation,
tracking problem, 211, 244, 245
algebraic, 241
trajectory, 144
differential, 239
transfer function, 12
root locus, 15
closed loop, 13
Routh array, 154
frequency, 14
matrix, 38, 121
Schur formula, 26 poles and zeros, 12, 111
servomechanism, 11, 211 proper, 122
signum function, 230 sensitivity, 15
soft-landing problem, 211, 228, 229,
236 unit step function, 13
stability
absolute, 201 Van der Pol equation, 179
and control, 196 vector
asymptotic, 143 basis, 28
bounded, 144 characteristic, 32
bounded input—bounded output euclidean norm, 47
(b.i.b.0.), 196 linear combination, 27
conditional, 171 linear dependence, 27
264

vector (contd.) subspace, 28

orthogonal, 55 unit, 29
space, 27
spanning
set, 28 z-transform,17 |

+
?

| Sk Fhe
pee tem nobus
ey
a+—' ©,
a ial tard he
< rr
me
This is a concise, readable account of some basic
mathematical aspects of control. The approach
concentrates on so-called state-space methods, and
emphasizes points of mathematical interest. However,
undue abstraction has been avoided and the contents
should be accessible to those with a basic background in
calculus and matrix theory. The book will be useful to final-
year honours mathematics students, postgraduate control
engineers, and qualified engineers seeking an introduction
to contemporary control theory literature. Problems are
provided at the end of each section.

Stephen Barnett is Professor of Applied Mathematics at the

University of Bradford.

‘An excellent introduction to modern control theory, with

an extensive treatment of matrix theory and its application
to linear systems and linear control problems. The
treatment is direct and sufficiently complete for self-study.
Strongly recommended.’ Bulletin of the./nstitute of
Mathematics and its Applications

‘Could hardly be bettered for an introduction to the

mathematical theory concentrating on state-space
methods.’ The Times Higher Educational Supplement

OXFORD UNIVERSITY PRESS ISBN0 19859619

7 BE