Ee 701 Lecture Notes
Uploaded by
arafatasgharEe 701 Lecture Notes
Uploaded by
arafatasgharName: _________________________________
EE/ME 701 Advanced Linear Systems
Lecture Notes
Prof. Brian Armstrong
September 11, 2012
EE/ME 701: Advanced Linear Systems
Linear Algebra
State Space Modeling, Basics of Linear Algebra
EE/ME 701: Advanced Linear Systems
1.11 Eigenvalues and eigenvectors
Linear Algebra
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.11.1 Some properties of Eigenvectors and Eigenvalues
Contents
. . . . . . . . . . . . . 24
1.11.2 Additional notes on Eigenvectors and Eigenvalues . . . . . . . . . . . . . 28
1.11.3 One final fact about the eigensystem: V diagonalizes A . . . . . . . . . . . 32
1 Elements of Linear Algebra
1.1
Scalars, Matrices and Vectors
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2
Transpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3
Basic arithmetic: +, - and * are well defined . . . . . . . . . . . . . . . . . . . . .
1.3.1
Multiplication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Commutative, Associative, Distributive and Identity Properties . . . . . . . . . . .
1.4.1
Commutative property . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.2
Associative property . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.3
Distributive property . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.4
Identity Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.4.5
Doing algebra with vectors and matrices
. . . . . . . . . . . . . . . . . .
1.5
Linear Independence of Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.6
Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
1.4
1.6.1
Computing the determinant . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.6.2
Some relations involving determinants
1.6.3
The determinant of triangular and diagonal matrices . . . . . . . . . . . . 12
1.7
Rank
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.8
The norm of a vector . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.9
Basic properties of the singular value decomposition (svd) . . . . . . . . . . . . . 16
1.8.1
1.9.1
Example norms
2 Two equations of interest
2.1
2.2
35
Algebraic Equations, y = A b . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.1.1
Case 1: Where we have n = p independent equations, the exactly
constrained case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.1.2
Case 2: Where n > p, we have more equations than unknowns, the over
constrained case (Left pseudo-inverse case): . . . . . . . . . . . . . . . . 39
2.1.3
Case 3: Where n < p we have fewer equations than unknowns, the underconstrained case (Right pseudo-inverse case): . . . . . . . . . . . . . . . 41
Differential Equations, . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3 Summary
45
. . . . . . . . . . . . . . . . . . . 12
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
SVD, condition number and rank . . . . . . . . . . . . . . . . . . . . . . 17
1.10 The condition number of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.10.1 Condition number and error . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.10.2 Example of a badly conditioned matrix . . . . . . . . . . . . . . . . . . . 20
1.10.3 How condition number is determined . . . . . . . . . . . . . . . . . . . . 21
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 1
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
Section 1.2.0
1 Elements of Linear Algebra
EE/ME 701: Advanced Linear Systems
Section 1.3.0
1.3 Basic arithmetic: +, - and * are well defined
Define:
1.1 Scalars, Matrices and Vectors
A=
Matrices are rectangular sets of numbers or functions, examples:
Matrices have zero or more rows and columns:
v (t)
1 0 7
31
,
x
(t)
R
x (t) =
A
=
w (t)
4 5 0
z (t)
2 3 6
A R33
1 2
3 4
B=
x=
2
3
1 1
w=
C = 4 1
5 2
3 4
Operations +, - and * are well defined. The dimensions of the operands
must be compatible.
Vectors are special cases of Matrices, with only one row or column,
x is a column vector,
h
i
w = 3 4 R12 is a row vector
For addition and subtraction, the operation is element-wise, and the
operands must be the same size:
Scalar values (numbers or functions with one output variable) can also be
treated as matrices or vectors:
3 = [3] R11
A+B =
0 3
5 7
AB =
2 1
1 1
If not the same size, there is no defined result (the operation is
impossible)
A + C = undefined
Array is a synonym for Matrix
1.2 Transpose
Transposing an array rearranges each column to a row:
#
"
3 1
3 4 5
T
C
C=
=
4 1
1 1 2
5 2
C R32 , CT R23
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 3
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 1.3.1
EE/ME 701: Advanced Linear Systems
Section 1.4.2
1.4 Commutative, Associative, Distributive and Identity
1.3.1 Multiplication
For multiplication, the operation is by row on the left and column on the
right.
To produce one element of the result, go across each row on the left and
multiply with elements of the column on the right,
A B = AB =
"
(1 (1) + 2 2) (1 1 + 2 3)
(3 (1) + 4 2) (3 1 + 4 3)
"
3 7
5 15
A and B dont have to be the same size to multiply
Ax =
"
1 2
3 4
#"
2
3
"
(1 2 + 2 3)
(3 2 + 4 3)
"
18
The size of the result matrix is determined by the number of rows in
A and columns in B
AC =
1 2
3 4
Unlike scalar algebra, multiplication does not generally commute:
A B 6= B A
3 7
5 15
BA =
11 16
Generally: There are many properties of linear algebra which may or
may not be true for some special cases.
Does not generally commute means that perhaps special matrices can
be found that do commute, but that not all matrices commute.
Like scalar algebra, +, - and * have the associative property:
3 1
4 1 = undefined
5 2
(n = 3 , m = 2 , j = 2 , k = 2)
"
#
6 10
3 1
1 2
=
CA =
1 4
4 1
3 4
11 18
5 2
Lecture: Basics of Linear Algebra
Like scalar algebra, subtraction commutes with a - sign:
A B = (B A)
1.4.2 Associative property
To multiply A and B, we must have m = j , then C Rnk
Like scalar algebra, addition commutes: A + B = B + A
AB =
The number of columns in A must match the number of rows in B
that is, m = j
"
1.4.1 Commutative property
Example:
For A B = C , given A Rnm and B R jk
Examples:
Properties
(Revised: Sep 06, 2012)
(A + B) + C = A + (B + C)
(A + B) C = A + (B C)
(A B) C = A (B C)
Page 5
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 1.4.4
1.4.3 Distributive property
EE/ME 701: Advanced Linear Systems
Section 1.4.5
1.4.5 Doing algebra with vectors and matrices
Like scalar algebra, * distributes over + and - :
Starting with an equation,
(A + B) C = A C + B C
We can add, subtract or multiply on the left or right by any allowed term
and get a new equation.
C (A B) = C A C B
Examples, given:
Note: left-to-right order of multiplications in second example.
A+B = C
1.4.4 Identity Matrix
Then
Like scalar algebra, linear algebra has a multiplicative identity:
(A + B) + D = C + D
IL C = C IR = C
E (A + B) = E A + E B = E C
Examples:
1 0 0
3
0 1 0 4
0 0 1
5
3 1
4 1
0
5 2
3 1
=
1 4 1
2
5 2
3 1
=
4 1
1
5 2
(A + B) F = C F
Where A, B, C, D, E and F are compatible sizes.
Matrices that are the appropriate size for an operation are called
commensurate .
If A is square, then IL = IR , and we just call it I, the identity matrix:
1 0
0 1
1 2
3 4
Lecture: Basics of Linear Algebra
1 2
3 4
1 0
0 1
1 2
3 4
(Revised: Sep 06, 2012)
Page 7
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 1.5.0
1.5 Linear Independence of Vectors
EE/ME 701: Advanced Linear Systems
Section 1.6.0
1.6 Determinant
The determinant is a scalar measure of the size of a square matrix.
Linear Dependence: A set of p n-dimensional vectors,
v1 , v2 , v p , vi Rn
det (A) = |A| R1
is linearly dependent if there exists a set of scalars {ai } , i = 1...p , not all
of which are zero, such that:
p
..
a1 v1 + a2 v2 + + a p v p = ai vi = 0 0 = .
(1)
i=1
0
Linear Independence: The set of vectors is said to be linearly independent if
there is no set of values {ai } which satisfies the conditions for Linear
Dependence. In other words,
p
ai vi = 0
(2)
(5)
The determinant is not defined for a non-square matrix.
The determinant of a matrix will be non-zero if and only if the rows (or,
equivalently, columns) of a matrix are linearly independent. Examples:
(a)
1 2 3
4 5 6 =0
10 14 18
(b)
1 2 3
4 5 6
7 8 9
= 54
(6)
In case (a), the third column is given by 2xCol2 - 1xCol1.
In case (b), the three columns are independent.
i=1
implies that the {ai } are all zero.
Notice that in case (a), the third row is given by 2xRow1 + 2xRow2.
Written another way, vectors {vi } are linearly independent if and only if
(iff)
p
ai vi = 0
ai = 0 , i
Always for a square matrix, if the columns are dependent, the rows
will be dependent.
(3)
i=1
or written in the form of Eqn (1):
| |
v1 v2
| |
a1
| |
a
2
=0
vp
...
| |
ap
Lecture: Basics of Linear Algebra
iff
a
1
a2
.. = 0
.
ap
(Revised: Sep 06, 2012)
(4)
Page 9
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 1.6.1
1.6.1 Computing the determinant
ai j i j
for any i = 1, 2, ..., n
(7)
j=1
where Mi j is called a minor. Mi j is the same matrix as A, with the ith row and jth
column removed.
For example, with i = 1,
2. det (A B) = det (A) det (B)
3. Invertibility of a matrix: Given M Rnn, M is invertible iff det (M) 6= 0
j=2
j=3
a b c
d f
e f
d e
+ c det
b det
det d e f = a det
g i
h i
g h
g h i
Closed-form expressions for 1x1, 2x2 and 3x3 matrices are sometimes
handy. They are:
det [a] = a ,
(b) Similarity relation:
det M A M 1 = det (M) det (A) 1/ det (M) = det (A)
det
a b
c d
= ad bc
(Revised: Sep 06, 2012)
1. If a matrix has the upper-triangular form of Au or the lower-triangular form
of Al
0
d1
T
d1
d2
d2
, or Al =
Au =
...
...
T
dn
0
dn
Then det (Au ) = nk=1 dk , or det (Al ) = nk=1 dk .
Example:
>> A = [ 1 2 3 ; 0 4 5 ; 0 0 6]
A = 1
2
3
0
4
5
0
0
6
2. A diagonal matrix is special case of 1).
a b c
det d e f = a e i a h f b d i + b g f + c d h c g e
g h i
Lecture: Basics of Linear Algebra
(a) det M 1 = 1/ det (M)
1.6.3 The determinant of triangular and diagonal matrices
j=1
1. det (I) = 1, where I is the identity matrix
4. Given M, an invertible matrix,
where ai j is the element from the ith row and jth column of A; and i j is called the
cofactor, given by:
(8)
i j = (1)i+ j det Mi j
Section 1.6.3
1.6.2 Some relations involving determinants
The determinant of a square matrix (the only kind !) is defined by Laplaces
expansion (following Franklin et al.):
det A =
EE/ME 701: Advanced Linear Systems
Page 11
Lecture: Basics of Linear Algebra
>> det(A)
ans = 24
(Revised: Sep 06, 2012)
Page 12
EE/ME 701: Advanced Linear Systems
Section 1.7.0
1.7 Rank
EE/ME 701: Advanced Linear Systems
Section 1.8.0
1.8 The norm of a vector
|||| :
The rank of a matrix is the number of independent rows (or columns)
r = rank (A) ,
A Rnm
The number of independent rows is always equal to the
independent columns.
Example:
1 2 3 4
rT1
r =
A = rT2 = 5 6 7 8
1
6 8 10 12
rT
3
(9)
number of
r3 = 1.0 r1 + 1.0 r2, so the set of 3 vectors is linearly dependent. Because
there are 2 independent vectors,
(10)
The norm of a vector, written ||x|| is a measure of the size of the vector. A norm is
any function of a vector x with these properties, for any vector x and scalar a, then
1. Positivity, the norm of any vector x is a non-negative real number
Rn 7 R
||x|| 0
2. Triangle inequality
||x + v|| ||x|| + ||v||
3. Positive homogeneity or positive scalability
||a x|| = |a| ||x||
rank (A) = 2
where |a| is the absolute value of a .
The third row is the sum of the first two.
Notice that the 3rd column is 2 x Col2 - Col1
And the 4th column is 3 x Col2 - 2 x Col1
Actually, property 1 follows from properties 2 and 3, so 2 and 3 are sufficient for
the definition.
The rank of a matrix can not be greater than the number of rows or columns
rank (A) min (n , m)
A matrix is said to be full rank if rank (A) takes its maximum possible value,
that is
rank (A) = min (n , m)
Otherwise the matrix is said to be rank deficient.
A square matrix is invertible if it is full rank.
The determinant of a square matrix is zero if the matrix is rank deficient.
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 13
Additionally, the norms we will use have the property
4. Positive definiteness
||x|| = 0
if and only if x = 0
That is, the norm is zero only if the vector is a vector of all zeros
(called the null vector). Technically, a norm with property 4 is
called a seminorm, but this distinction will not be important
for us.
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 1.8.1
EE/ME 701: Advanced Linear Systems
Section 1.9.0
1.9 Basic properties of the singular value decomposition (svd)
1.8.1 Example norms
The 2-norm, (a.k.a. the Euclidean norm)
||x||2 =
example:
h
i
xT = 2 3 4 ,
||x||2 =
A matrix A Rnm has p singular values, where p = min (n, m)
n
x2i
(11)
r
22 + (3)2 + (4)2 = 5.39 (12)
Other common norms are the 1-norm and the -norm, these are all special
cases of the p-norm
We can write the 2-norm as:
x2i
||x||2 =
The p-norm is given as:
|xi|
||x|| p =
||x||1 =
|xi|
i
!1
||x|| = lim
Examples:
h
i
2 3 4 = 9
1
Lecture: Basics of Linear Algebra
|xi|
i
1
4
7
2
2
5
6
1
3
6
5
1
>> S = svd(A)
S = 14.1542
2.5515
0.3857
i , the singular values, are a measure of how a matrix scales a vector.
For example, for matrix A,
(13)
!1
(14)
another vector v3 so that A v3 is scaled by 0.3857.
With example matrix A, choosing v1 and v3 to illustrate the largest and
smallest singular values (more later on choosing v1 and v3) gives:
v1 = -0.5763,
-0.5735
-0.5822
= |xi |
v3 = 0.3724
-0.8184
0.4375
(15)
>> y1 = A*v1
y1 = -3.4700
-8.6660
10.3862
-2.3083
>> y3 = A*v3
y3 = 0.0482
0.0227
-0.1159
0.3640
Now ||y1|| = 14.1542 ||v1 || and ||y3|| = 0.3857 ||v3 ||
The -norm is the maximum absolute value:
!1
A =
there is a vector v1 so that A v1 is scaled by 14.1542, and
!1
The 1-norm is the sum of the absolute values:
(also called the Manhattan metric)
1
For example:
i=1
= max |xi |
(16)
h
i
2 3 4 = 4
Since 1 = 14.1542 is the largest singular value, there is no vector v that
gives a y = A v larger than y1 .
Since 3 = 0.3857 is the smallest singular value, there is no vector v that
gives a y = A v smaller than y3
(Revised: Sep 06, 2012)
Page 15
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 16
EE/ME 701: Advanced Linear Systems
Section 1.10.0
1.9.1 SVD, condition number and rank
The SVD is used to determine the rank and condition number of a matrix.
rank (A) = # singular values > 0 ,
rank (A) = 3
(Read # singular values significantly different from zero)
cond (A) = ratio largest / smallest singular value,
For square matrices, |det (A)| = ni=1 i ,
EE/ME 701: Advanced Linear Systems
Section 1.10.1
But suppose the ideal values y are unknown, and instead we have
measurements y given by:
y = y + e
y
(20)
Where
y is a measurement of y,
y is the ideal value of y (generally unknown), and
cond (A) = 36.70
det (A) = undefined
We will be looking at the SVD in detail in several weeks (cf. Bay, Chap. 4).
e
y represents measurement noise,
For example e
y = N (0, y ) are samples with a normal (Gaussian) random
distribution, with zero mean and y standard deviation.
Then what we can compute is
1.10 The condition number of a matrix
x = A1
1
The condition number of a matrix indicates how errors may be scaled when
the matrix inverse is used.
For example if we have these two equations in two unknowns,
7
= 3 x1 + 4 x2
y = Ax
becomes
Where x is an estimate of x . Notation:
x:
a vector in R p (to be estimated)
x :
The true value of x
x :
The estimated values of x
x = A1
1 y=
Lecture: Basics of Linear Algebra
0.2 0.4
1 2
x1
x2
One answer is
(18)
In the worst case, the error is amplified by the condition number of A1
max
01.
0.3
e
x = x x
An important question is, how much error does measurement noise e
y
introduce into the calculation of x ?
We can solve for unknowns x1 , x2 with
e
x: Misadjustment of estimates
1.10.1 Condition number and error
Writing in the matrix-vector form
7
(21)
(17)
9 = x1 + 2 x2
7
9
5
2
(Revised: Sep 06, 2012)
||e
x||
||e
y||
= cond (A1 )
||x||
||y ||
or
(19)
Page 17
||e
x|| ||x|| cond (A1 )
Lecture: Basics of Linear Algebra
(22)
||e
y||
||y||
(Revised: Sep 06, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 1.10.1
Example:
EE/ME 701: Advanced Linear Systems
Section 1.10.3
1.10.2 Example of a badly conditioned matrix
>> A1 = [3 4 ;-1 2]
>> ystar = [7; -9]
A1 =
ybar =
3
-1
Rather than Eqn (17) suppose we have the data
4
2
2 = 2 x1 + 4 x2
198 = 200 x1 + 401 x2
7
-9
Then x is estimated by:
>> xhat0 = A1 \ ystar
xhat0 =
5
-2
>> cond(A1)
ans =
2.6180
x 2 = A1
2 y2
%% Notice left-division operator
%% Given by calculation with no noise
>> A2 = [2 4 ;200 401],
%% This is the condition number.
2
200
4,
401
ystar2 =
2
198
The Condition number of A2 is large
>> cond(A2),
If ||e
y|| = 0.01 , then
max (||e
x|| / ||x ||) = 2.6180 (0.01/11.4) = 0.0023 , or
(23)
||e
x|| 2.6180 (0.01/11.4) 5.39 = 0.012
ans =
>> ytilde1 = 0.01 * rand(2,1)
ytilde1 = 0.0087
-0.0093
>> xhat1 = A1 \ (ystar1 + ytilde1)
xhat1 =
5.0055
%% Errors of about 0.0023 * ||x||
-2.0019
%%
= 0.0023 * 5.39 = 0.012
Estimating x 2 using A2
>> ytilde2 = 0.01 * rand(2,1)
ytilde2 =
0.0121
-0.0140
>> ybar2 = ystar2 + ytilde2
ybar2 =
2
+
0.0121
198
-0.0140
>> xhat2 = A2 \ ybar2
xhat2 =
7.4592
-3.2266
(Revised: Sep 06, 2012)
1.0041e+05 = 100410.
>> xstar2 = A2 \ ystar2
xstar2 = 5
%% Calculating with ideal data still gives
-2
%% ideal results
Example numerical values:
Lecture: Basics of Linear Algebra
A2 =
Page 19
Lecture: Basics of Linear Algebra
%% An example sample of noise
%% An example measurement of Y
2.0121
197.9860
%% This is way off !
(Revised: Sep 06, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 1.10.3
EE/ME 701: Advanced Linear Systems
Section 1.11.0
1.11 Eigenvalues and eigenvectors
1.10.3 How condition number is determined
Condition number is given as the ratio of the largest to smallest singular
value
A
(24)
cond (A) =
A
Where A is the largest singular value of A , and
A is the smallest singular value of A .
A square matrix has eigenvectors and eigenvalues, making up the
eigensystem of the matrix.
The main properties of eigenvectors and eigenvalues are introduced here.
Consider: A vector has both a direction and a magnitude. For example:
We will learn about the singular values later in this course
v1 =
>> SingularValues = svd(A1)
SingularValues =
5.117
1.954
%% cond(A1) = 2.618
>> SingularValues = svd(A2)
SingularValues =
448.1306
0.0045
%% cond(A2) = 1.0041 x 105
1
2
, v2 =
1.5
3.0
, v3 =
1.58
1.58
v1 has the same direction as v2 , but has a different magnitude.
v3 has the same magnitude as v1 , but has a different direction.
x2
v1
In general, multiplying a vector v by a matrix A
v3
x1
y4 = A v4
What matrices have condition numbers:
Singular values are defined for any matrix, and so condition number can
be computed for any matrix.
introduces both a change of magnitude and a change of
direction. For example:
y4 = A v4 =
A rank-deficient matrix B has B = 0 , and so cond (B) = .
(The error multiplier goes to infinity !)
2.0 0.5
1.0
0.5
1
1
2.5
0.5
v2
x2
y4
x1
v4
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 21
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 22
EE/ME 701: Advanced Linear Systems
Section 1.11.0
For any square matrix A, certain vectors have the property that the matrix
changes magnitude only, not direction. That is, writing
EE/ME 701: Advanced Linear Systems
Section 1.11.1
In general, we can write:
i vi = A vi
where the i
C1
(25)
are the eigenvalues and the vi are the eigenvectors.
y = Av
then if v is an example special vector, y and v have the same direction, but
possibly different magnitude. If the directions are the same,
y = v or
Notice that in general a x 6= A x, because multiplication by matrix A will
rotate a general vector x
(Choosing vectors x at random, what is the probability of selecting an
eigenvector ?)
v = Av
1.11.1 Some properties of Eigenvectors and Eigenvalues
These special vectors are called the eigenvectors.
1. Only square matrices have an eigensystem
For example matrix A, the special vectors are:
1.5
1.5
2.0 0.5
1.0
0.5
1
1
2. The eigenvalues are the solutions to the equation
2.0 0.5
1.0
0.5
1
2
A vi i vi = 0
An eigenvector is scaled by the matrix
1.5
1.5
1
2
2.0 0.5
1.0
0.5
2.0 0.5
1.0
0.5
Lecture: Basics of Linear Algebra
1
1
1
2
det (A i I) = 0
= 1.5
= 1.0
1
1
1
2
, so v1 =
, so v2 =
, 1 = 1.5
(26)
Proof: From Eqn (25)
so
(A i I) vi = 0
But, given that vi 6= 0, the second expression is only possible if (A i I) is
rank deficient.
QED
, 2 = 1.0
(Revised: Sep 06, 2012)
Page 23
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 24
Student
Exercise
EE/ME 701: Advanced Linear Systems
Section 1.11.1
Example using det (A i I) = 0: Starting with
det
a b
c d
EE/ME 701: Advanced Linear Systems
5. A complete set of eigenvectors exists when an n n matrix has n linearly
independent eigenvectors (so V, the matrix of eigenvectors, is invertible).
= ad bc
For the example matrix
A=
and plugging in
det
2.0 0.5
1.0
0.5
1 0
0 1
= det
Section 1.11.1
2.0
0.5
1.0
0.5
= (2.0 ) (0.5 ) 1.0 (0.5) = 2 2.5 + 1.5 = 0
1.0
0.5
the matrix of eigenvectors is
V=
The expression leads to a polynomial equation in :
2.0 0.5
v1 v2
1 1
1.5 0
1 2
and the matrix of eigenvalues is
3. The polynomial equation given by Eqn (26) is called the characteristic
equation of the matrix A
(a) A 3x3 matrix gives an equation in 3, a 4x4 gives an equation in 4 , etc.
(b) Abels theorem states: there is no closed-form solution for the roots of
a polynomial of 5th order and above, therefore there is no closed-form
solution for the eigenvalues a 5x5 matrix or larger.
U=
1 0
0 2
6. A matrix that lacks a complete set of eigenvectors is said to be defective,
but this name is meaningful only from a mathematical perspective. For
many control systems, some combinations of parameter values will give a
defective system matrix.
4. Special case, when A is upper-triangular, lower-triangular or diagonal
n
det (A I) = (dk )
(27)
k=1
A defective matrix can only arise when there are repeated eigenvalues.
Where dk is the kth diagonal element.
Eqn (27) shows that for upper-triangular, lower-triangular and diagonal
matrices, the eigenvalues are the diagonal elements.
Lecture: Basics of Linear Algebra
This case corresponds to the case of repeated roots in the study of
ordinary differential equations, where solutions of the form y (t) = t et
arise.
(Revised: Sep 06, 2012)
Page 25
In this case, a special tool called the Jordan Form is required to solve
the equation x (t) = A x (t) + B u (t).
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 26
EE/ME 701: Advanced Linear Systems
Section 1.11.1
7. The eigensystem is computed in Matlab with the eig() command,
1. As mentioned, some matrices do not have a complete set of eigenvectors
(Student Exercise: What is the name for this ?)
V is the matrix of eigenvectors, U is a matrix with the corresponding
eigenvalues on the main diagonal.
1
2
-2
4
5
6
>> [V,U] = eig(A)
V =
0.5276
0.2873
-0.7994
U =
Section 1.11.2
1.11.2 Additional notes on Eigenvectors and Eigenvalues
>> [V,U] = eig(A)
>> A =
EE/ME 701: Advanced Linear Systems
For a symmetric real matrix (or Hermitian complex matrix), it is
guaranteed that all eigenvalues are real.
3
1
3 ]
Furthermore, it is guaranteed that a complete set of eigenvectors exists !
0.5267 - 0.0250i
-0.1294 + 0.1030i
0.8335
-2.4563
0
0
0
5.2282 + 0.5919i
0
0.5267 + 0.0250i
-0.1294 - 0.1030i
0.8335
0
0
5.2282 - 0.5919i
8. Notice that eigenvalues can be real or complex.
>> Poly = poly(diag(u))
Poly =
1.0000
-8.0000
2.0000
68.0000
3 8 2 + 2 + 68 = 0
(28)
As with many things, wikipedia has a very nice article on eigenvectors and
eigenvalues:
Everything you wanted to know about the eigensystem of A, in 3 pages.
(Revised: Sep 06, 2012)
System matrices (A in x = Ax) are rarely symmetric, but certain other
important matrices are.
3. The determinant of a matrix is equal to the product of the eigenvalues of the
matrix
n
det (A) = i
i=1
Example:
>> aa = [ 1 2 3 ; 4 -1 -3; 5 2 1]
aa =
1
2
3
4 -1 -3
5
2
1
>> [Evecs, Evals] = eig(aa)
The characteristic equation corresponding to 3 3 matrix A is
Lecture: Basics of Linear Algebra
2. Symmetric matrices are a special case.
Page 27
Evecs = 0.6128
0.0136
0.7901
0.4667
-0.8751
-0.1276
0.2146
-0.8542
0.4734
>> prod(diag(Evals))
ans = 6.0000
>> det(aa)
ans = 6.0000
Lecture: Basics of Linear Algebra
%% Find the eigensystem
Evals = 4.9126
0.0000
0.0000
0.0000
-3.5706
0.0000
0.0000
0.0000
-0.3421
%% Product of the eigenvalues
%% Equals the determinant
(Revised: Sep 06, 2012)
Page 28
EE/ME 701: Advanced Linear Systems
Section 1.11.2
EE/ME 701: Advanced Linear Systems
Section 1.11.2
5. Matlab does not create the characteristic equation using det (A I) = 0,
and then solving for the polynomial roots.
4. The determinant equation
det (A I) = 0
(29)
Matlab actually goes the other way, and solves for the roots of a polynomial
by forming a matrix and finding the eigenvalues of that ! The polynomial
gives a polynomial in
1 3 8 2 + 2 + 68 = 0
+ an1
n
n1
+ + a1 + a0 = 0
(30)
is represented in Matlab as the vector
Eqn (30) is the characteristic equation of matrix A. The eigenvalues of
A are the roots of Eqn (30).
Poly = [ 1, -8, 2, 68]
and the command roots() finds the roots:
Eqn (29) is useful for theoretical results, but generally not a practical method
for calculating the eigenvalues, for n > 2:
>> roots(Poly)
Once you get Eqn (30), how are you going to find the roots of the
polynomial ? (Answer on the next page)
Working with the determinant does not lead to a numerically stable
algorithm.
ans = [ 5.2282 + 0.5919i
5.2282 - 0.5919i
-2.4563
Actually under the hood Matlab forms a matrix (called the companion
matrix) and applies the eigenvalue routine to that:
>> compan(Poly)
ans =
The determinant does not give the eigenvectors
8.0000
1.0000
0
-2.0000
0
1.0000
-68.0000
0
0
Going from Eqn (29) to Eqn (30) by solving the determinant involves
symbolic manipulation of matrix A. It is a lot of work.
>> eig(compan(Poly))
ans = [ 5.2282 + 0.5919i
5.2282 - 0.5919i
-2.4563 ]
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 29
Lecture: Basics of Linear Algebra
3 + a2 2 + a1 + a0 = 0
a2 a1 a0
C= 1
0
0
0
1
0
(Revised: Sep 06, 2012)
Page 30
EE/ME 701: Advanced Linear Systems
Section 1.11.2
Relationship
EE/ME 701: Advanced Linear Systems
Section 1.11.3
1.11.3 One final fact about the eigensystem: V diagonalizes A
Roots of Characteristic Equation Eigenvalues of the Companion Matrix
For a state-variable model, the poles of the system are the eigenvalues of the
A matrix
When a complete set of eigenvectors exists, the matrix of the eigenvectors
can be used to transform A into a diagonal matrix, by similarity transform.
The general form for the similarity transform is:
If the A matrix has poles in the right half plane, the system is unstable.
Note: I havent told you an algorithm for eig(A).
b = T A T 1
A
T must be invertible
The algorithm is: Use Matlab !
(31)
THEOREM: The similarity transform preserves eigenvalues, that is
Matlabs eig() command uses one from a library of algorithms,
depending the details of the matrix. The study of efficient algorithms
to find eigenvectors and eigenvalues has been an active area of research
for at least 200 years.
b
eig (A) = eig(A)
Proof: starting with Eqn (26),
det (A I) = 0
Left multiplying by T and right multiplying by T 1
det T (A I ) T 1 = det (T ) det (A I)
1
=0
det (T )
(32)
Where the right hand equality arises because det (A I) = 0 and det (T ) is finite.
From (32) it follows that
b I = 0
det T (A I ) T 1 = det T A T 1 T I T 1 = det A
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 31
b
Therefore any value that is an eigenvalue of A is an eigenvalue of A.
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
QED
Page 32
EE/ME 701: Advanced Linear Systems
Section 1.11.3
THEOREM: When a square matrix A has a complete set of independent
eigenvectors
i
h
V = v1 , v2 , , vn ,
EE/ME 701: Advanced Linear Systems
V diagonalizes A (continued)
A numerical example:
the array V of eigenvectors provides a similarity transform
where U is a diagonal matrix with the eigenvalues on the main diagonal.
Proof: Since the vectors vi are eigenvectors,
V=
A vi = i vi
AV = A
v1 , v2 , , vn
1 v1 , 2 v2 , , n vn
1 v1 , 2 v2 , , n vn
v1 , v2 , , vn
1 0 0
0 2 0
.. .. . . . ..
.
. .
0
0 n
1 0
[V, U] = eig (A)
0.2320 0.7858
16.1168
1.1168
0.0000
= VU
Left multiplying both sides by V1 gives:
b = U.
Equation (33) is a similarity transform with T = V1 and A
(Revised: Sep 06, 2012)
= 0.5253 0.0868 0.8165
0.8187 0.6123 0.4082
V1 A V =
16.1168
1.1168
0.0000
1 2 3
V U V1 = 4 5 6
7 8 9
VU = AV
V1 V U = U = V1 A V
0.4082
We have:
So
Lecture: Basics of Linear Algebra
v1 v2 v3
U = 0 2 0 =
0 0 3
The matrix on the right can be expressed:
And so
h
1 2 3
A= 4 5 6
7 8 9
U = V1 A V
Section 1.11.3
(33)
(and, of course,
=U
=A
i vi = A vi )
QED
Page 33
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 34
EE/ME 701: Advanced Linear Systems
Section 2.0.0
2 Two equations of interest
y, A : known;
Section 2.1.0
2.1 Algebraic Equations, y = A b
1. Solving for unknown constants (an algebraic equation)
y = Ab
EE/ME 701: Advanced Linear Systems
solve for: b
(34)
Think of n equations in p unknowns, for example an experiment with a
process:
y (k) = b1 v (k) + b2 v2 (k)
(35)
where v (k) is an independent variable and y (k) is the measurement.
The objective is to determine the elements of the b vector from Eqn (35)
with v (k) known and y (k) measurements.
2. Solving a differential equation
x (t) = A x (t) + B u (t)
A, B, x (t = 0) : known;
solve for x (t)
Introducing some notation, we can write:
y (k) = (k) b =
T
v (k) v2 (k)
b1
b2
(36)
where (in this example)
(k) =
For example, if b =
2 3
iT
y (1) = (1) b =
T
y (2) = T (2) b =
..
.
y (n) = (n) b =
T
. See Strang for a nice treatment of the two problems of linear algebra.
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 35
Lecture: Basics of Linear Algebra
h
h
h
v (k)
v2 (k)
(37)
and v (1) = 2, v (2) = 3, ... v (n) = 1 then
2 4
3 9
1 1
= 2 2 + 4 3 = 16
= 3 2 + 9 3 = 33
2
3
= 1 2 + 1 3 = 1
(Revised: Sep 06, 2012)
Page 36
EE/ME 701: Advanced Linear Systems
Section 2.1.0
With several measurements (equations) we get a matrix vector equation (the
- emphasize that T (k) is a row vector:
y (1)
T (1)
..
y = ... =
b = Ab
.
T
y (n)
(n)
Vocabulary
EE/ME 701: Advanced Linear Systems
2.1.1 Case 1: Where we have n = p independent equations, the exactly
constrained case
In this case A Rnn is a square matrix.
(38)
When A is full rank an inverse matrix exists so that
A1 A = A A1 = I
Example
b R p is the parameter vector.
b=
y Rn is the data vector
b1 b2
iT
v (k) R1 is an independent variable that determines the regressor
vector
iT
h
(k) = v (k) v2 (k)
(k) R p is the regressor vector
A Rnp is the regressor matrix
The columns of A, which correspond to the elements of () are called
the basis vectors or basis functions of model A
The solution to Eqn (34) is given as:
y
b
y
e
y
: Measured values
b
b
e
b
: Estimated values
: Misadjustment
Generally, the true values are unknown
b
y = Ab
b
(left multiply by A
Example,
y = A1 A b
(39)
(40)
T (k) =
based on Eqn (36), i.e.,
v (k) v2 (k)
i
Suppose we have data for k = 1, 2, with v (1) = 1.5, v (2) = 3 and
y (1) = 12.75, y (2) = 43.5
Page 37
b
b = A1 y
12.75
43.5
y (1)
y (2)
b
b=
(Revised: Sep 06, 2012)
which gives the solution
and so
Lecture: Basics of Linear Algebra
A1 y = I b = b
e
y=b
yy
y = Ab
(original equation)
Notation
y b : True values
Section 2.1.1
T (1)
T (2)
1.5 2.25
3.0 9.0
Lecture: Basics of Linear Algebra
b
b = Ab
b=
y,
b
b=
2.5
4.0
1.5 2.25
3.0 9.0
b
b
(Revised: Sep 06, 2012)
Page 38
EE/ME 701: Advanced Linear Systems
Section 2.1.2
2.1.2 Case 2: Where n > p, we have more equations than unknowns, the
over constrained case (Left pseudo-inverse case):
This is a common case. For example, a model has 3 parameters, and we run
an experiment and collect 20 data. We have 20 equations in 3 unknowns.
b.
In this case there is generally no value of b which exactly satisfies y = A b
Define e
y = y Ab
b,
minimizing ||e
y||2
EE/ME 701: Advanced Linear Systems
Section 2.1.2
Consider the dimensions of A and A# :
h
:
AT
AT
Student exercise: what are the dimensions of A# ?
the left pseudo-inverse gives the solution
To derive the left pseudo-inverse, start with:
y = Ab
b
: [] =
[]
1. Left multiply each side by AT :
AT y = AT A b
b
Note that AT A is an p p matrix, for example a 3 3 matrix when
there are 3 unknowns.
1
2. When AT A R pp is full rank, left multiply each side by AT A
gives:
1
1
b
(41)
AT A b
b = Ib
AT A
AT y = AT A
3. Which gives
1
b
AT y = A# y
b = AT A
(42)
1 T
A is called the left pseudo-inverse of A .
where A# = AT A
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 39
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 40
EE/ME 701: Advanced Linear Systems
Section 2.1.3
2.1.3 Case 3: Where n < p we have fewer equations than unknowns, the
under-constrained case (Right pseudo-inverse case):
In the under-constrained case, A is a row matrix, so our equation has the
shape:
h
i
y = Ab
: [] =
b which exactly satisfy
In this case there are generally many values of b
b
b
y = A b , the right pseudo-inverse gives the solution with minimum b
2
EE/ME 701: Advanced Linear Systems
Section 2.1.3
Remarks
1. For each of cases 1, 2, and 3 we had the requirement that A , AT A or
A AT must be full rank. When these matrices are not full rank, a more
general method based on the Singular Value Decomposition (SVD) is
needed.
2. With the SVD and four fundamental spaces of matrix A we will be able to
find the set of all vectors b
b which solve
y = Ab
The right pseudo-inverse is given by:
1
T
A+
A AT
R = A
AT
When A AT Rnn is full rank then:
AT
We will see the SVD and four fundamental spaces of a matrix in Bay,
Chapter 4.
1
T
A AT
=I
A A+
R = AA
In this case, one solution for b
b is given by:
(43)
Plugging Eqn (43) into y = A b
b gives:
(44)
b
b = A+
Ry
1
= AT A AT
y
1
y = A AT A AT
y = Iy
Demonstrating that b
b = A+
R y is a solution.
Student exercise: what are the dimensions of A+
R ?
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 41
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 42
EE/ME 701: Advanced Linear Systems
Section 2.2.0
2.2 Differential Equations,
EE/ME 701: Advanced Linear Systems
Section 2.2.0
The state-variable model is the topic of the second half of the course.
The linear state variable model is a very general form for modeling dynamic
systems in control theory, economics and operations research, biology and
other fields.
The state-variable model is a differential equation with the form:
x (t) = A x (t) + B u (t)
Here we only note that solving the differential equation
x (t) = A x (t) + B u (t)
is fundamentally different from solving the algebraic equation
(45)
where
y = Ab
and involves the eigensystem of the system matrix A and using a similarity
transform and transformation to modal coordinates to determine the
solution.
x (t) Rn is the state vector,
A Rnn is the system matrix,
The differences between the solution of algebraic and differential equations
is summarized in table 1.
B Rnm is the input matrix, and
u (t) Rm is the input vector.
Equation Type
Just as solving an algebraic equation,
Algebraic
Differential
Nature of Solution
A value
A function of time
Main Tools:
Gaussian Elimination
Matrix Inverse
SVD
Eigenvalues
Eigenvectors
Singular Matrix
Problems,
Solution requires
SVD
O.K.
Complete set of Eigenvectors
Not important
Important
Rectangular matrix
OK
Impossible
3x = 7
is very different from solving a scalar differential equation,
3 x (t) = 2 x (t) + 7 u (t)
Solving a matrix differential equation requires tools very different from
those for solving a matrix algebraic equation.
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 43
Table 1: Differences in the approach to algebraic and differential equations. A
singular matrix is rank deficient.
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 44
EE/ME 701: Advanced Linear Systems
Section 3.0.0
3 Summary
Basics of linear algebra have been reviewed.
The two problems of linear algebra have been discussed.
Algebraic equations
y = Ab
(46)
take us into vector subspaces, projection and the Singular Value
Decomposition (SVD)
Differential equations
x (t) = A x (t) + B u (t)
(47)
take us into the eigensystem and coordinates transformations.
Lecture: Basics of Linear Algebra
(Revised: Sep 06, 2012)
Page 45
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Vectors and Vector Spaces
Sn : the standard basis . . . . . . . . . . . . . . . . . .
28
4.1.3
Representations on vector spaces other than Rn . . . . . .
30
4.2
A Span of a Vector Space
. . . . . . . . . . . . . . . . . . . . .
31
4.3
Change of basis . . . . . . . . . . . . . . . . . . . . . . . . . . .
32
Contents
1 Introduction
4.1.2
4.3.1
Notation for a change of basis . . . . . . . . . . . . . . .
33
1.1
Definitions: three ways to multiply vectors . . . . . . . . . . . . .
4.3.2
Chaining changes of basis . . . . . . . . . . . . . . . . .
34
1.2
Definitions: Properties of Vectors
. . . . . . . . . . . . . . . . .
4.3.3
Change of basis example . . . . . . . . . . . . . . . . . .
34
1.2.1
Parallel . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2.2
Perpendicular (or Orthogonal) . . . . . . . . . . . . . . .
Change of basis viewed geometrically (this section is connected
to Bay, section 2.2.4) . . . . . . . . . . . . . . . . . . . . . . . .
36
1.2.3
Norm (magnitude) . . . . . . . . . . . . . . . . . . . . .
4.4.1
Example, change of basis viewed geometrically
. . . . .
37
1.3
Direction Cosine . . . . . . . . . . . . . . . . . . . . . . . . . .
10
4.4.2
1.4
Parallel, Perpendicular and the Zero Vector . . . . . . . . . . . .
11
Numerical example based on representing from basis
vectors on the to basis vectors . . . . . . . . . . . . . .
38
Summary . . . . . . . . . . . . . . . . . . . . . . . . . .
40
4.4
4.4.3
2 Vector Spaces
12
2.1
Properties the scalars in Eqn (12) must have . . . . . . . . . . . .
14
2.2
Properties a Vector Space must have . . . . . . . . . . . . . . . .
15
3 Properties of ensembles of vectors
3.1
3.2
17
5.1
5.2
41
Example proper vector subspace . . . . . . . . . . . . . . . . . .
43
5.1.1
Observations on subspaces of Euclidean spaces . . . . . .
45
What about other dimensions of A ? . . . . . . . . . . . . . . . .
46
Defining a vector space . . . . . . . . . . . . . . . . . . . . . . .
19
5.2.1
The set-theoretic meaning of almost all . . . . . . . . .
46
3.1.1
A vector space defined by a spanning set of vectors . . .
20
5.2.2
A vector y orthogonal to a proper subspace B . . . . . . .
47
3.1.2
A vector space defined by a set of basis vectors . . . . . .
20
Dimension of a Vector Space . . . . . . . . . . . . . . . . . . . .
21
4 Basis of a Vector Space
4.1
5 Vector Subspace (following Bay 2.4)
23
Representation of vector x on basis V . . . . . . . . . . . . . . .
4.1.1
(Revised: Sep 10, 2012)
Page 1
48
6.1
Projection Theorem . . . . . . . . . . . . . . . . . . . . . . . . .
50
6.2
Projection of a vector onto a proper subspace . . . . . . . . . . .
51
6.2.1
First projection example, projection onto a 1-D subspace .
51
Normalization of the basis vectors . . . . . . . . . . . . . . . . .
54
24
Introducing notation to help keep track of vectors and bases 26
Part 2: Vectors and Vector Spaces
6 Projection Theorem
6.3
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
6.4
EE/ME 701: Advanced Linear Systems
Projection Matrices . . . . . . . . . . . . . . . . . . . . . . . . .
56
6.4.1
Bay example 2.10, projecting f onto a 2-D subspace . . .
57
9.1
Important properties of Inner Products . . . . . . . . . . . . . . .
87
6.4.2
Projection matrix for the orthogonal complement . . . . .
60
9.2
Important properties of Norms . . . . . . . . . . . . . . . . . . .
88
6.4.3
Projection with normalized basis vectors . . . . . . . . .
61
9.3
Study tip: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
89
7 Gram-Schmidt ortho-normalization
7.1
63
7.1.1
Example Gram-Schmidt Ortho-normalization . . . . . . .
63
7.1.2
Tolerance value for Gram-Schmidt algorithm . . . . . . .
67
7.2
Projection matrix with GS Ortho-normalization . . . . . . . . . .
68
7.3
Projection Coefficients . . . . . . . . . . . . . . . . . . . . . . .
69
7.4
Projection onto the orthogonal complement. . . . . . . . . . . . .
70
7.5
Projection and fitting parameters to experimental data . . . . . . .
71
8 Additional Topics
8.2
72
The Four Fundamental Spaces of a Matrix . . . . . . . . . . . . .
72
8.1.1
Numerical Examples of the four fundamental spaces, . . .
74
8.1.2
Computing bases for the four fundamental spaces . . . . .
78
8.1.3
Bases for the Four Fundamental Spaces, Numerical
Example . . . . . . . . . . . . . . . . . . . . . . . . . .
80
8.1.4
The Four Fundamental Spaces of a Matrix, revisited . . .
81
8.1.5
Questions that can be answered with the four fundamental
spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
Two ways to determine the four fundamental spaces . . .
83
Rank and degeneracy . . . . . . . . . . . . . . . . . . . . . . . .
84
8.1.6
Part 2: Vectors and Vector Spaces
86
62
Process of Gram-Schmidt Ortho-normalization . . . . . . . . . .
8.1
9 Summary and Review
(Revised: Sep 10, 2012)
Page 3
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 1.0.0
1 Introduction
EE/ME 701: Advanced Linear Systems
Section 1.1.0
1.1 Definitions: three ways to multiply vectors
Three types of vector products are
A vector: an n-tuple of numbers or functions
A 3-vector: x = 1
An n-vector: z =
z1
z2
..
.
zn
sin (2k/4)
cos (2k/4)
A 4-vector: x (k) =
sin (4k/4)
cos (4k/4)
A 2-vector: u (t) =
2.0 sin (t)
2.0
Euclidean Vector: A Euclidean n-space corresponds to our intuitive notion of
2-D or 3-D space:
Inner product (also scalar or dot product):
gives a scalar value
Outer product (also tensor or matrix product):
gives a matrix, and
Vector product (also cross product):
gives a vector result.
Inner (scalar) product (also dot product):
A scalar measure of the interaction of two vectors of the same dimension
hv, wi = vT w = wT v =
vi wi
where v , w , Rn
(1)
i=1
If v is complex, then vT is the complex-conjugate transpose, (note: in
Matlab the transpose operation forms the complex conjugate).
The inner product of vectors gives a scalar,
hv, wi = vT w = (a scalar)
Outer (matrix) Product:
Axes are orthogonal
The outer product gives a matrix,
Each vector corresponds to a point
v >< w = v wT = (a matrix)
Example:
Intuitive because we live in Euclidean 3-space, R3
Length (L2 -norm) corresponds to our every-day notion of length.
v= 2
3
w= 3
1
hv , wi = 11
2 3 1
v >< w = 4 6 2
6 9 3
Example of using the Outer Product: each contribution to the covariance
of parameter estimates is given by:
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 5
Part 2: Vectors and Vector Spaces
2eb = e
be
bT
(Revised: Sep 10, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 1.1.0
EE/ME 701: Advanced Linear Systems
Section 1.2.2
1.2 Definitions: Properties of Vectors
Cross (vector) product:
1.2.1 Parallel
vw = z
(2)
Two vectors v and w are parallel if they can be written
Produces a vector perpendicular to each of v and w
w = av
The magnitude of the vector is give by:
(4)
where a is a scalar.
||z|| = ||v|| ||w|| sin ()
(3)
Example:
v = 2 , w = 6 ,
3
9
where is the angle between v and w.
The cross-product is only defined for vectors in 3-space. It is used, for
example, electro-magnetics, 3D geometry and image metrology.
w=3 2 ,
3
so
vkw
Co-linear is another term for parallel vectors, as in
Cross product and Outer product are not much used in EE/ME 701
Vectors v and w are co-linear.
Inner (or dot) product will be used All the time.
1.2.2 Perpendicular (or Orthogonal)
If
hv, wi = 0
(5)
then v and w are said to be perpendicular (or, equivalently, orthogonal).
Example:
v = 2 , w = 5 ,
3
3
vT w =
1 2 3
so
5 = 1 10 + 9 = 0
vw
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 7
Part 2: Vectors and Vector Spaces
(6)
(Revised: Sep 10, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 1.2.3
EE/ME 701: Advanced Linear Systems
Section 1.3.0
1.3 Direction Cosine
1.2.3 Norm (magnitude)
The norm of a vector x is a non-negative real number, ||x|| 0
The inner prod. indicates how closely two vectors are related.
A norm is a measure of size (of a vector or matrix)\
If two vectors are parallel
(This is the correct test for ||)
The L p norms are most familiar.
|hv, wi| = ||v|| ||w||
L: Henri Lebesgue (1875-1941), set calculus (integration and
differentiation) on a rigorous foundation based on set theory.
The L2 norm, ||v||L2 (or ||v||2 , or simply ||v||) is the familiar notion of
distance:
r
||v||2 = v2i
(7)
i
hv, wi = 0
||v|| p =
|vi| = (|v1| + |v2| + ... + |vn| )
p
or
hv, wi
=0
||v|| ||w||
For vectors forming angles in between || and ,
Define :
1
p
1.0
(This is the correct test for )
If two vectors are perpendicular
The general L p norm ||v||L p (or simply ||v|| p):
r
hv, wi
= 1.0 or
||v|| ||w||
or
(8)
hv, wi
= cos ()
||v|| ||w||
(10)
The L2 norm is the default, throughout the literature, ||v|| refers to ||v||2,
unless expressly defined to be a different norm (or any norm).
The L2 norm is related to the dot product of a vector with itself:
1
2
||v||2 = hv, vi =
vT v
v2i
A handy corollary is:
hv, wi
= cos1
(11)
||v|| ||w||
w
(9)
Figure 1: Direction cosine.
In a general framework, vectors may be made with elements which are
not numbers, but so long as the dot product is defined, the 2-norm is still
given by Eqn (9).
The L2 -norm is said to be induced by the dot product. It is called the
induced norm.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 9
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 1.4.0
Section 2.0.0
2 Vector Spaces
1.4 Parallel, Perpendicular and the Zero Vector
Looking at Eqn (4)
w = av
EE/ME 701: Advanced Linear Systems
(4, repeated)
When we talk about vector spaces we talk about sets of vectors. Closure is
the first important property of sets.
Rn is the most basic vector space. It is the set of all vectors with n elements.
If w = 0 (the zero vector) then
Closure: A set is said to be closed under an operation, if the operation is
guaranteed to return an element of the set.
0
0
= w = 0v
..
.
0
Examples:
The integers are closed under addition, subtraction and multiplication.
The integers are not closed under division.
with a = 0 .
The positive integers are closed under addition and multiplication.
The positive integers are not closed under subtraction or division.
Vector spaces are collections of vectors which are closed under scaled
addition (like superposition).
And likewise, if w = 0, then clearly
hv, wi = 0
(5, repeated)
So in a formal sense, we can say that every vector is both parallel and
perpendicular to the zero vector.
For a vector space S
S : {a set of vectors}
if
then
x1 , x2 S
x3 = a1 x1 + a2 x2 S
(12)
The central topic of Bay, chapter 2, is describing the ai , xi and S in the most
general possible way.
Lets start with a specific example, S = R3, the space in which we live.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 11
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 12
EE/ME 701: Advanced Linear Systems
Section 2.0.0
EE/ME 701: Advanced Linear Systems
Section 2.1.0
2.1 Properties the scalars in Eqn (12) must have
Defining a vector x:
x
1
x = x2
x3
Eqn (12) comprises scalars and vectors
,
x1 , x2 , x3 R
(13)
The scalars must be elements of a field. For a set with operations to form a
field, it must have:
1. An additive identity 0:
Then S = {x} = R3 , the vector space S is the set of all vectors x
comprised of 3 real numbers.
a+0 = a
(15)
a1 = a
(16)
2. A multiplicative identify 1:
Examples:
2 ,
1 ,
5
0
0
3. Every element must have a negative:
if a F, then a F
Verifying that a1 x1 + a2 x2 S with an example (choosing the ai to be
real numbers):
1
2
0
2.0 2 + 1.0 1 = 5
1
5
3
(14)
a + a = 0
(17)
4. Operations of addition, multiplication and division must be defined, and
the set must be closed under these operations.
Examples:
The integers do not form a field (not closed under division)
The rational numbers do form a field (+, , , all work)
The field we will use most often is the real numbers, a R .
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 13
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 2.2.0
2.2 Properties a Vector Space must have
EE/ME 701: Advanced Linear Systems
Section 2.2.0
4. Closure under scalar multiplication: For all x S and for all a F, the
vector y = a x must be an element of S.
These are the fundamental properties of vector spaces and operations on
vectors. Other important properties, quantities and calculations are built out
of these basics.
A vector space is a set of vectors. It must have at least these properties:
x S,
a F , y = ax S
(23)
: for all
5. Associativity of scalar multiplication:
1. S must have at least 2 elements.
a, b F and x S ,
2. The linear vector space S must contain a zero element, 0 such that:
x+0 = x,
xS
(18)
(24)
6. Distributivity of scalar multiplication over vector addition:
3. Vector space S must contain the additive inverse of each element,
if x S , then y S s.t. x + y = 0
a (b x) = b (a x)
(19)
(a + b) x = a x + b x
(25)
a (x + y) = a x + a y
(26)
: there exists
The vector space (a set of vectors) and its operations must also have the properties:
1. The vector space S must be closed under addition:
if x + y = v, then v S
(20)
The fundamental properties of fields are such standard components of
standard algebra that we forget to think about them.
Any Vector Space (that is, set of vectors which satisfies the fundamental
properties above) will have the derived properties of vector spaces which
we will develop, such as
Orthogonality
2. Commutative property for addition:
Projection
x+y = y+x
(21)
length
etc.
3. Associative property for addition:
For 99% of what we will do in EE/ME 701, we will work with vectors in Rn
over the field of the real numbers.
(x + y) + z = x + (y + z)
(22)
Which is to say the familiar vectors and scalars, such as in Eqn (14).
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 15
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 16
EE/ME 701: Advanced Linear Systems
Section 3.0.0
3 Properties of ensembles of vectors
In many cases it is interesting to consider a finite set of vectors, such as:
v1 , v2 , ..., v p S
(27)
Example:
V=
1
2 ,
0
0 ,
0
1 ,
1
Section 3.0.0
The expression for linear dependence can also be written:
b1
| | |
|
b2
v p = v1 v2 v p1 .
=V b
..
| | |
|
b p1
(29)
And recall that a set of vectors is linearly independent if there is no set of
values {bi } which satisfies the conditions for Linear Dependence. In other
words,
1
1
1
EE/ME 701: Advanced Linear Systems
R3
bi vi = 0
i=1
implies that the {bi } are all zero.
We shall shortly see that a finite set of vectors can define a Vector Space.
Notation for a finite set of vectors: let V connote a set with p vectors:
V = v1 , v2 , ..., v p ,
or equivently ,
V = v1 v2 v p (28)
| | | |
New vectors can be formed as linear combinations of vectors v1 v p :
w = a1 v1 + a2 v2 + + a p v p
Recall that a set of vectors is linearly dependent if one or more of the
vectors can be expressed as a linear combination of the others, such as:
v p = b1 v1 + b2 v2 + + b p1 v p1
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 17
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 3.1.0
3.1 Defining a vector space
EE/ME 701: Advanced Linear Systems
Section 3.1.2
3.1.1 A vector space defined by a spanning set of vectors
In many cases, vector spaces contain an infinite number of vectors, so it
is not possible to define a vector space by writing down all its elements.
However, we can define a vector space in terms of a finite set of vectors.
Most often, we define a vector space as the set of all vectors which can be
created by linear combinations of specified vectors. For example:
2
1
S1 = x : x = a1 1 + a2 0 , a1, a2 R
1
2
(30)
S = x : x = a1v1 + a2v2 + ... + a pv p
(31)
where S is a vector space;
S is the set of all vectors formed as linear combinations of the spanning
vectors v1 , v2 , v p .
We can say that S is the space spanned by v1 , v2 , v p, and that vectors
v1 , v2 , v p span vector space S.
Vector space S1 is the set of all vectors x such that
Given an arbitrary set of p n-dimensional vectors V = v1 , v2 , v p , vi Rn ,
the set of vectors defines a vector space
Any set of vectors, at least one of which is non-zero, spans a vector space.
3.1.2 A vector space defined by a set of basis vectors
x = a1 1 + a2 0
1
2
A span is a very general concept, perhaps a bit too general. The concept of a basis
for a vector space is more restrictive. It works like a span, we write that the set of
basis vectors {vi } defines a vector space according to Eqn (31) , but for a set {vi }
to be a set of basis vectors they must additionally be linearly independent. When
the vectors vi are independent and we write
for any values of a1 and a2 , which are real numbers.
Read Eqn (30) as saying:
x = a1v1 + a2v2 + ... + ar vr
(32)
S1 =
Vector space S1 is
{}
the set
of vectors x
A set of basis vectors is the minimum set of vectors that spans a space
such that
For a given vector space S, the choice of basis vectors is not unique.
x =
x satisfies the conditions given
Any basis set for S is also a span for S
Part 2: Vectors and Vector Spaces
there is a unique solution {a1, a2... ar } for every vector x S. Because the basis
vectors vi are linearly independent, there is exactly one solution for Eqn (32).
(Revised: Sep 10, 2012)
Page 19
Part 2: Vectors and Vector Spaces
(is any span S also a basis S?)
(Revised: Sep 10, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 3.2.0
3.2 Dimension of a Vector Space
EE/ME 701: Advanced Linear Systems
Definition of vector space S1
Dimension: The dimension of a linear vector space is the largest number of
linearly independent vectors that can be taken from that space.
Examples:
The dimension of vector spaces Rn is n
The dimension of the vector space given by Eqn (33) is 2
A vector subspace is itself a vector space (if we call S1 a subspace, we
are emphasizing that it is embedded in a larger vector space).
Example:
Universe is R3
S1 is 2-D
(Eqn (33) repeated)
2
1
3
S1 = x : x = a1 1 + a2 0 , a1, a2 R , xi R
1
2
Set S1 is a vector space. Thus, if v and w are elements of S1 , then
2
1
3
S1 = x : x = a1 1 + a2 0 , a1 , a2 R , xi R
(33)
1
2
Section 3.2.0
z = b1 v + b2 w ,
z S1 ,
b1 , b2 R
Consider the properties of a vector space one by one, you will see that
they all hold.
The dimension of S1 is the number of linearly independent vectors that can
be selected from S1 .
Note that even though v, w, z R3 , it is still true that dim (S1 ) = 2.
1
0
1
2
3
3
2
1
0
2
Y
2
3
Figure 2: S1 , a 2-dimensional vector subspace embedded in R3 .
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 21
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 22
Student
Exercise
EE/ME 701: Advanced Linear Systems
Section 4.0.0
4 Basis of a Vector Space
EE/ME 701: Advanced Linear Systems
Section 4.1.0
4.1 Representation of vector x on basis V
Working with a particular vector space V of vectors v, x, y Rn , if we select
a set of vectors BV = {v1, v2, ..., vr } from V such that
1. We have r vectors, where r is the dimension of vector space V, and
VECTOR REPRESENTATION THEOREM: Given a set of vectors BV =
{vi } = {v1 , v2 , ..., vr } that are basis vectors for r-dimensional vector space V
embedded in Rn , then any vector sx V can be represented as a unique linear
combination of basis vectors
2. The vectors are linearly independent
s
x = a1 v1 + a2v2 + ... + ar vr
Then we have created a basis for vector space V.
Basis: A set of linearly independent vectors, BV , in vector space V is a basis for
V iff every vector in V can be written as a unique linear combination of
vectors from this set.
x = a1 v1 + a2 v2 + + ar vr
BV = {v1 , v2 , ..., vr } is a basis for V iff
(35)
Since V is r-dimensional and the set of vectors {x, v1, v2, ..., vr } contains
r + 1 vectors, the set must be linearly dependent. Since the set is linearly
dependent there exists a set of scalars ai (called the basis coefficients) such
that x + a1 v1 + a2 v2 + + ar vr = 0, which gives
Or written mathematically,
Example basis:
x, v1 ..vr Rn
Proof:
Existence:
A more formal definition is given:
exactly one a = {a1, ..., ar }
x V,
Uniqueness:
s.t. x = a1v1 + + ar vr
1
2
BV = 1 , 0
1
2
(36)
Suppose the ai giving vector x were not unique. This means there is a second
set of basis coefficients {bi } distinct from the values {ai } such that
x = b1 v1 + b2 v2 + + br vr .
(37)
(34)
Subtracting Eqns (36) and (37) gives
BV is a basis for a Vector Space V, which has dimension = 2.
0 = (a1 b1) v1 + (a2 b2) v2 + + (ar br ) vr
(38)
Basis BV is illustrated by the two vectors in figure 2.
A basis for V is not unique, any set of r independent vectors in V is a basis
for V
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 23
But since the vi are linearly independent, Eqn (38) is possible only if each
of the (ai bi) = 0, which is to say that {ai } = {bi }, and so the basis
coefficients representing vector x on {vi } are unique.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 24
EE/ME 701: Advanced Linear Systems
Section 4.1.0
Eqn (36) gives us a way to represent a vector x on the basis vectors.
x = 2
8
Inspection shows that
x = 2 v1 + 3 v2 =
Definition:
The vector a =
2
3
Sometimes we are interested in multiple bases. In some cases a vector x
will have a representation on each basis. It helps to have some notation to
keep track of things.
1
2
V = {v1 , v2 } = 1 , 0
1
2
v1 v2
2
3
Given a vector space S1 of vectors sx Rn with a basis V = [v1 , v2] and also
with a different basis W = [w1 , w2 ], then a vector sx S1 can be represented
3 ways:
s
x , Vx , Wx
(39)
where the left superscript indicates the basis on which the vector is
represented.
The vector sx is represented on the standard or Euclidean basis. This is
the ordinary representation of the vector (more on this later).
is the representation of the vector x on basis V .
Example: given the bases V and W for vector subspace S1 , and given the
vector sx below, find the representation for sx on each of bases V and W .
x = 2 ,
Define V to be the vector subspace spanned by V .
Notice that
x , v1 , v2 R3
while
2
aR
Vectors x , v1 , v2 have the dimension of the universe in which vector
subspace V is embedded.
The representation of a vector on a subspace always has the dimension
of the subspace (which may be less than the dimension of the universe).
(Revised: Sep 10, 2012)
Page 25
V = [v1 , v2 ] = 1
0 ,
x = a1 v1 + a2 v2 = [v1 , v2]
Part 2: Vectors and Vector Spaces
a1
a2
=V
W = [w1 , w2 ] = 1 2
3 4
Solution: since the columns of V are one basis for S1 , we know there exist
basis coefficients a1 and a2 to represent sx on V . The relationships can be
written:
The dimension of vector subspace V is 2
Part 2: Vectors and Vector Spaces
Section 4.1.1
4.1.1 Introducing notation to help keep track of vectors and bases
Example:
EE/ME 701: Advanced Linear Systems
x,
so V x =
a1
a2
(Revised: Sep 10, 2012)
Page 26
EE/ME 701: Advanced Linear Systems
Section 4.1.1
When r < n , we can solve for the basis coefficients with the left-pseudo
inverse. Recalling that it is given that sx S1 , then
Since sx = V V x ,
Example:
1
V
x = V TV
V T sx
>> x
x =
-1
0
2
x = b1w1 + b2 w2 = [w1 , w2 ]
(40)
Solving, we find
Confirm that x is represented by Vx =
>> V * Vx
ans =
1
-2
8
V x R2
2
3
Wx =
b2
=W
1
x = WTW
WT x
2.4
0.2
b1
, which gives.
2 = 2.4 1 0.2 2
8
7
4
4.1.2 Sn : the standard basis
%% should give sx
A basis that we have been using all along is the standard basis in Rn. For R3
the basis vectors are:
is the representation of vector x R3 on basis V .
The calculation of Eqn (40) always gives a well defined result.
1
The only calculation that might not give a valid result is V T V
. We
are assured the solution exists by the fact that the columns of V (the basis
vectors) are linearly independent.
Notice that x R3 while V x R2
The vector has the dimension of the universe, while the representation of
the vector has the dimension of the subspace
Part 2: Vectors and Vector Spaces
>> Vx = inv(V*V) * V * x
Vx =
2
3
1
-2
8
Section 4.1.2
We can also represent vector x on basis W :
so
>> V
V =
2
-1
1
EE/ME 701: Advanced Linear Systems
(Revised: Sep 10, 2012)
Page 27
1 0 0
S3 = 0 1 0
0 0 1
h
i
= s1 s2 s3
(41)
So we might write x = S3 sx. In words,
Every vector x is represented on the standard basis by representation sx,
with sx = x and Sn = I (the n n identity matrix) is the set of standard basis
vectors.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 28
EE/ME 701: Advanced Linear Systems
Section 4.1.2
EE/ME 701: Advanced Linear Systems
Section 4.1.3
4.1.3 Representations on vector spaces other than Rn
Thus we have found three ways to represent vector x
We are chiefly concerned with vectors of numbers, but there are other types
of vectors and vector spaces, and our notions of basis and representation
carry over.
1. On the standard basis
x = S3 sx
2. On the V basis
x =V
x = a1 v1 + a2 v2
x=W
x = b1w1 + b2w2
Consider, for example, the set of polynomials of s of degree up to 2, such as
p (s) = 3 + 7 s + s2. It turns out that this set is a vector space. We can define
the vector space P by
3. And on the W basis
n
o
P = p (s) : p (s) = a1 + a2s + a3s2
If we choose the set of basis functions
Basis Facts:
A representation is always on a specific basis.
representation of vector x on basis V .
such as
Vx
is the
You cant talk about the representation of a vector without specifying the
basis.
The dimension of a representation:
Notice that
vi , wi
R3 ,
whereas
Each vector in a universe has the dimension of the universe, but
A representation of a vector has the dimension of the basis on which it is
represented.
(Revised: Sep 10, 2012)
with
a1
x = a2
a3
Even when the vectors of the vector space are not vectors of numbers, a
representation of a vector will be (because the representation is always an
r-dimensional vector of basis coefficients).
V x, W x R2 .
This is because, for this example, the universe has dimension n = 3, while
the vector subspace has dimension r = 2.
Part 2: Vectors and Vector Spaces
then the representation of a polynomial p (s)is simply
p (s) = P px
The representation of a given vector on a given basis is unique.
sx ,
o
n
P = p1 (s) = 1 , p2 (s) = s , p3 (s) = s2
Page 29
There is a one-to-one correspondence between a vector p P and its
representation px Rr on basis P. The representation is the vector of
basis coefficients.
Equivalently, the vector and its representation are isomorphic.
Useful tools, such as dot product and norm, can be applied to px .
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 30
EE/ME 701: Advanced Linear Systems
Section 4.2.0
4.2 A Span of a Vector Space
EE/ME 701: Advanced Linear Systems
4.3 Change of basis
Just as a set of basis vectors defines a vector space, a set of spanning vectors
v1, v2, , v p defines a vector space:
Vector Space V = x : x = a1v1 + a2v2 + + a pv p
(42)
The vector space spanned by vectors v1 , v2 , , v p is the set of all vectors
given as linear combinations of the spanning vectors
A vector space V is spanned by a set of vectors {vi } if every x V can be written
as a linear combination of the vi s .
It may be interesting to transform a vector from its representation on one
basis to its representation on another basis. A change of basis is important,
for example, for exploring some properties of state variable models.
Example:
3
2 1
V Vx = 1 0 = sx = 4
14
1 2
Find
We can say that the vi s span Vector Space V ,
Write V = span {vi }
Section 4.3.0
Wx
1 7
W = 1 2
3 4
with
, the representation of sx on basis W .
We actually know how to solve this problem, since we know how to find
from sx using W (see Eqn (40) in section 4.1.1).
( V is the vector space spanned by {vi } )
Wx
However, it will be handy to introduce a more general notation that express
from what basis and to what basis a vector is being transformed.
All bases are a spans, but not all spans are bases, the difference:
A span is any set of vectors, for example:
1
0
3
1
,
,
V = span ,
2
5
1
1
The solution is:
W
(43)
Since
defines vector space V .
Compare Eqn (42) with Eqn (43), both define vector space V
The example shows that a span may have redundant vectors
Eqn (44) gives
1
1
x = WTW
W T sx = W T W
WTV
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 31
(44)
1
0.60
0.40
= W
WTV =
WTW
VT
0.20 0.20
Equivalently, the vectors of the spanning set may be linearly dependent.
Shortly, we will learn how to form a basis from any span
x=W
VT x=
Part 2: Vectors and Vector Spaces
0.60
0.40
0.20 0.20
Vx =
4.4
0.2
(Revised: Sep 10, 2012)
Page 32
EE/ME 701: Advanced Linear Systems
Section 4.3.1
EE/ME 701: Advanced Linear Systems
Note: when V is a square matrix
4.3.1 Notation for a change of basis
To change from one representation to another requires a transformation
matrix
W
V
x= W
VT x
where
Wx
Section 4.3.3
V
sT
= V 1
(46)
is the representation on basis W , V x is the representation on V .
The symbol
4.3.2 Chaining changes of basis
W
VT
We can chain together changes of basis. For example, given
indicates that transformation matrix T converts a vector from representation
on basis V (left subscript) to basis W (left superscript). Using the left suband super-scripts leaves the right positions open, for example
W
V T1 ,
the transformation from basis V to the standard basis can be written
s
V
VT x
s
VT
with
1
x = V TV
V T sx ,
so
V
sT
=V
1
= V TV
VT
Part 2: Vectors and Vector Spaces
(47)
G
sT
(48)
it follows that
F
=G
FT s T
Find transformation matrix to convert vector sx to its representation on basis
F , and find the transformation matrix to convert this to basis G
(45)
which gives:
V
F s
x= G
FT s T x
4.3.3 Change of basis example
For the transformation from s to V , weve seen that the basis coefficients
are given by:
V
F
sT
Solution, since
Transformation from basis V to the standard basis: Since, by definition (see
Eqn (39))
s
x = V Vx
x=
find G
sT
W
V T2
might be the transformations from V to W at t1 and t2
G
FT
10
s
x= 4 ,
11
2 2
F = 1 2 ,
1 3
2 4
G = 4 1
1 4
x = Vs T sx
(Revised: Sep 10, 2012)
Page 33
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 34
EE/ME 701: Advanced Linear Systems
Section 4.3.3
EE/ME 701: Advanced Linear Systems
Section 4.4.0
4.4 Change of basis viewed geometrically (this section is
Solution: find the two transformation matrices
connected to Bay, section 2.2.4)
F
sT
and then evaluate
Fx = FT sx
s
and
and
G
FT
In section 4.3 we saw vectors represented on the standard basis and two
particular bases,
s
x , Fx , and Gx
Gx = GT Fx.
F
with
%% Find the two transformation matrices
>> FsT = inv(F*F)*F
FsT =
0.3117
-0.3506
0.0260
0.0260
0.2208
0.1688
>> GFT = inv(G*G)*G*F
GFT =
0.3333
-0.3333
0.3333
0.6667
F = 1 2 ,
1 3
G = 4 1
1 4
(49)
And we found the coordinate transformation matrix to transform between
bases
(see Eqn (44))
G
FT
1
GT G
GT F
(50)
which was derived by solving for the basis coefficients.
Now, evaluating Fx and Gx
>> sx =[10; 4; 11]
sx =
10
4
11
>> Fx = FsT * sx
Fx =
2
3
>> Gx = GFT * Fx
Gx =
-0.3333
2.6667
Bay presents a second way to derive the coordinate transformation in section
2.2.4, based on representing the basis vectors of the first basis in the second
basis.
Since F and G are sets of basis vectors for the same vector space, we can
represent each basis vector in the other basis.
Double check that both Fx and Gx give the original sx
>> F*Fx
ans =
10
4
11
Part 2: Vectors and Vector Spaces
>> G*Gx
ans = 10.0000
4.0000
11.0000
(Revised: Sep 10, 2012)
Page 35
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 36
EE/ME 701: Advanced Linear Systems
Section 4.4.1
Writing the basis vectors, explicitly showing them to be represented on the
standard basis (these are just the basis vectors shown in Eqn 49,
h
iT
)
for example s f1 = 2 1 1
F=
sf
sf
G=
sg
sg
(51)
We can form G
F T by representing the F basis vectors on the G basis:
h
i
F = G G f1 G f2
EE/ME 701: Advanced Linear Systems
Section 4.4.2
4.4.2 Numerical example based on representing from basis vectors on the
to basis vectors
>> F = [ 2 2
-1 2
1 3 ]
>> G = [ 2 4
-4 1
-1 4 ]
Now
h
x = F Fx = G
Gf
Gf
i
x=G
h
Gf
Gf
x = G Gx (52)
Using the uniqueness of the representation of the vector sx,
i
h
G
F
F
Gf
Gf
x=
x =G
1
2
FT x
So
G
FT
Gf
Gf
(53)
(54)
The transformation from one coordinate frame to another is given as the
representation of the from basis vectors on the to basis vectors.
>> sf1 = F(:,1)
>> sf2 = F(:,2)
sf1 = 2
sf2 =
2
-1
2
1
3
%% Find the representation of the F basis vectors on G
>> Gf1 = inv(G*G) * G * sf1
Gf1 =
0.3333
0.3333
>> Gf2 = inv(G*G) * G * sf2
Gf2 = -0.3333
0.6667
4.4.1 Example, change of basis viewed geometrically
In robotics, rotations are used to change the expression of a vector from one
coordinate frame to another (that is, a change of basis). The rotation matrix
from B to A is given by the axes of B expressed in A
|
|
|
A
A
AY
AZ
(55)
BR = X B
B
B
|
|
|
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 37
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 38
EE/ME 701: Advanced Linear Systems
Section 4.4.3
EE/ME 701: Advanced Linear Systems
Section 4.4.3
4.4.3 Summary
%% Transformation from F to G
>> GFT = [Gf1 Gf2]
GFT =
0.3333
-0.3333
0.3333
0.6667
Looking at the way G
F T is given,
G
FT
%% Representation of x on F
>> Fx = [ 2
3 ]
Fx =
2
3
Gf
Gf
A coordinate transformation is given by representing the from basis
vectors on the to basis.
Note that the equations are, of course, the same. In section 4.3 we wrote
G
FT
%% Representation of x on G
>> Gx = GFT * Fx
Gx =
-0.3333
2.6667
1
= GT G
GT F
In section 4.4 we wrote
G
FT
=
=
%% Finding sx from each of the representation of F and on G
>> sx = F * Fx
sx = 10
4
11
Gf
Gf
1 T s
G f1 ,
GT G
1
h
GT s f1
GT G
sf
1 T s i
GT G
G f2
1
GT F
= GT G
By representing the basis vectors of F in G , we can write the transformation
from F to G .
>> sx = G * Gx
sx = 10.0000
4.0000
11.0000
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 39
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 40
EE/ME 701: Advanced Linear Systems
Section 5.0.0
EE/ME 701: Advanced Linear Systems
5 Vector Subspace (following Bay 2.4)
Section 5.0.0
Recall the terminology from set theory: subset and proper subset.
From the set of colors (the universe) {blue, green, red},
Vector subspace: A vector space of dimension r , comprising vectors of ndimensional elements, where r n.
{blue, green, red} is a subset but not a proper subset.
A vector subspace is a type of vector space, it must demonstrate all of
the properties of a vector space.
{blue,green,red} Colors
{blue, red}, is both a subset and a proper subset.
Proper vector subspace: A vector space of dimension r , comprising vectors of
n elements, where r < n.
{blue,red} Colors
A vector subspace is defined by a basis or a span, for example
1
2
S1 = x : x = a1 1 + a2 0 ,
2
1
Then B =
1 ,
1 ,
B = 1 ,
B =
a1 , a2 R
A vector space is a subset of the universe in which it is embedded. For
example
(56)
S2
is a basis for S1 .
S2 R3
1 is also a basis for S1 .
1
1
0 , 1 is one possible span for S1 .
2
3
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
0
1
0
x : x = a1 0 + a2 1 + a3 0 ,
=
0
1
0
Page 41
Exercise:
A proper vector subspace is a proper subset of the universe in which it is
embedded. For example
Make up
Another
span
and basis
for S1 .
a1 , a2 , a3 R
S1
2
1
x : x = a1 1 + a2 0 ,
=
1
2
S1 R3
Part 2: Vectors and Vector Spaces
a1 , a2 R
(Revised: Sep 10, 2012)
Page 42
EE/ME 701: Advanced Linear Systems
Section 5.1.0
5.1 Example proper vector subspace
EE/ME 701: Advanced Linear Systems
Section 5.1.0
Terminology
An example 2-D subspace embedded in R3 :
2
1
S1 = x : x = a1 1 + a2 0 ,
1
2
The set given by Eqn (57) is illustrated in figure 3.
a1 , a2 R
Given S and B are vector spaces, if B S (if B is a proper subset of S),
then we can say that vector space B is embedded in vector space S .
Such as: Vector space S1 is embedded in R3.
(57)
Proper vector subspace is used to emphasize that
r = dim (B) < dim (S) .
Often we omit the words proper and vector, such as saying:
Subspace B is embedded in S, which is isomorphic to R3 .
Sub can also be omitted:
Vector space B is embedded in space S.
If a vector space is 2 dimensional, then dim (B) = 2 and we say that B is
2-D, such as:
1
0
B is a 2-D subspace in R7 ,
1
2
If dim (B) = r:
3
3
B is a r-D subspace in Rn .
1
0
2
Y
2
3
Figure 3: 2D Vector subspace embedded in R3 .
Recall that vector space S1
must include the zero element.
must be closed under scalar multiplication.
must be closed under vector addition.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 43
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 44
EE/ME 701: Advanced Linear Systems
Section 5.1.1
EE/ME 701: Advanced Linear Systems
Section 5.2.1
5.2 What about other dimensions of A ?
5.1.1 Observations on subspaces of Euclidean spaces
Proper subspaces of R3 have dimension 1 or 2.
When B A, and thus r = dim (B) < dim (A), there must be vectors in A
not lying in B.
B, dim (B) = 0 contains only one vector: the zero vector.
Does the zero vector, 0 Rn, by itself satisfy the conditions to be a
vector space ?
B, dim (B) = 1: the vectors in B lie on a line passing through the origin.
B, dim (B) = 2: the vectors in B lie in a plane which includes the origin.
Proper subspaces of Rn have dimension 1 ... (n 1).
A 2-D vector subspace can be referred to as a plane.
Student
Exercise
Actually, when dim (B) < dim (A), almost all vectors in A are
outside B.
Consider a class room with the origin in the center of the top of the
desk in front. Almost all vectors in R3 (the class room) do not lie in the
surface of the desktop (a 2-D subspace). The surface of the desktop
has zero volume, and so 0% of the volume of the room.
5.2.1 The set-theoretic meaning of almost all
An r-D vector subspace, with r < n, can be referred to as a hyperplane or
hypersurface.
Almost all y A have property w has a precise meaning in set theory:
The elements of set A which do not have property w have measure
0.
The set of states of the system forms a 4-D hyperplane in R12
A vector spaces of dim (S) > 3 can be referred to as hyperspace, as in
The points lie on the surface of a 6-dimensional hyper-cube lying in
an 8-dimensional hyper-space.
Measure theory is whole subject unto itself (remember Lebesgue). Think of
it this way:
if you are choosing y randomly from A, the chance of getting one with
property w is 100% (almost all y have property w).
There are elements of A which do not have property w, but the chance of
choosing one is infinitesimal. In set theory we say
The set of y in A without property w has zero measure.
or, equivalently
Almost all y in A have property w.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 45
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 46
EE/ME 701: Advanced Linear Systems
Section 5.2.2
EE/ME 701: Advanced Linear Systems
6 Projection Theorem
5.2.2 A vector y orthogonal to a proper subspace B
When proper subspace B is embedded in A, vector y A, but outside
subspace B can have two relationships to B:
Before introducing the projection theorem, we need to introduce a
generalized notion of orthogonality.
1. Vector y can partially overlap subspace B , formally:
x B
s.t.
hx, yi =
6 0
Generalized Orthogonality: three flavors of orthogonality
For this discussion (as elsewhere) b, u, v, w, x, y and z are vectors in Rn ;
bold capitals, such as B, U, S and S1 , refer to subspaces.
(58)
Case 1: A vector is orthogonal to a vector
2. Vector y can be orthogonal to subspace B, which is to say that y does
not overlap B at all. Formally:
if
hx, yi = 0
x B then
Section 6.0.0
y B
This is our familiar case, based on the the inner product:
(59)
vw
Or, equivalently,
hv, wi = 0
(63)
Case 2: A vector is orthogonal to a subspace
y B x B , hx , yi = 0
Student
exercise
Student exercise, show that if
y bi ,
This implies that the vector is orthogonal to each vector in the
subspace:
vB
hv, bi = 0 b B
(64)
(60)
Case 3: A subspace is orthogonal to a subspace
This implies that each vector in the first subspace is orthogonal to each
vector in the second subspace:
{bi } a basis for B
(61)
BU
hb, ui = 0
b B u U
(65)
(Revised: Sep 10, 2012)
Page 48
then
yB
Part 2: Vectors and Vector Spaces
(62)
(Revised: Sep 10, 2012)
Page 47
Part 2: Vectors and Vector Spaces
EE/ME 701: Advanced Linear Systems
Section 6.0.0
EE/ME 701: Advanced Linear Systems
Section 6.1.0
6.1 Projection Theorem
Now lets define a projection:
Projection: A projection of a vector onto a subspace is a vector. It is the
component of the original vector lying in the subspace. The remainder
of the original vector must be orthogonal to the subspace.
Looking at figure 4, u is the component of x lying in subspace U. This
means that the other part of x (that part not in subspace U, given by
w = x u) is orthogonal to U, or w U.
Observation: the subspace can have dimension 1, that is, we can project
a vector onto a vector.
u U
PROJECTION THEOREM: Given vector spaces U and S with U a
proper subspace of S, then for all vectors x S there exists a unique
vector u U such that w = x u and w U.
Alternatively, in formal notation, given U S ,
x S u U s.t. w = x u , and w U
(66)
The projection theorem is illustrated in figure 4.
The projection theorem tells us that given any vector and subspace (not
necessarily proper !) the vector can be broken down into 2 parts:
1. A part lying in the subspace, and
w = xu
w U
2. A part that is orthogonal to the subspace.
Notation: Introduce the notation PU x for the projection of x onto U. When u is
the projection of x onto U, we may write:
u = PU x
Consider the part which does not lie in subspace U.
Define w = x u , we may say that
Figure 4: Orthogonal projection of vector x onto subspace U (from Bay).
vector w is orthogonal to subspace U.
The set of all possible ws forms a vector subspace !
Orthogonal Complement: if U is a subspace of S, the set {w : w S , w U}
is the orthogonal complement of U in S , written: U .
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 49
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 50
EE/ME 701: Advanced Linear Systems
Section 6.2.1
6.2 Projection of a vector onto a proper subspace
Starting with
6.2.1 First projection example, projection onto a 1-D subspace
with a 1-D subspace U = span {u1 }
, u1 =
f=
hw, u1 i = 0
(70)
hf, u1 i
hf, u1 i
=
hu1 , u1 i ||u1 ||2
(71)
1 =
(67)
The inner product gives us the projection of one vector onto another.
Projection onto a vector is illustrated in figure 5.
Suppose we want to find the projection of f onto U. Whatever is left over,
must be orthogonal to U, so:
PU f = 1 u1
(68)
w = f PU f
hf, u1 i 1 hu1 , u1 i = 0
we can solve 1 :
Because the inner product operation is linear, Eqn (69) can be broken into two
parts. Keeping in mind that w = (f PU f) and for the 1-D case PU f = 1 u1
hw, u1 i = h(f PU f) , u1 i = hf, u1 i hPU f, u1 i = hf, u1 i 1 hu1 , u1 i = 0
This is the problem of projecting a vector onto a subspace.
Consider vectors in
Section 6.2.1
Example: Projection onto a 1-D subspace (continued)
In section 6.1 the projection theorem is laid out. The question arises: given
S, U and x, how can u and w be determined.
R4,
EE/ME 701: Advanced Linear Systems
(69)
Notice that the magnitude of the projected vector (PU f) is independent of
the length of the basis vector (u1 ).
u1
Pf
Pf
w
u1
(a) Projection of f onto u1
(b) Projection of f onto a similar u1
Figure 5: Projection of f onto u1 and onto u1 .
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 51
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 52
EE/ME 701: Advanced Linear Systems
Section 6.2.1
Putting numbers to the example:
EE/ME 701: Advanced Linear Systems
Eqn (71) is:
Using f and u1 as given:
i
h
4 0 2 1
PU f =
2
PU f =
0.1
0.2
4 0 2 1
0.2
0.1
PU f =
2
0.1
0.2
0.2
0.1
0.7
2 1.4
2
= 7 =
10
2 1.4
2
1
0.7
1
||u1 ||2
u1
(72)
1. The direction, given by u1 .
The projection of f must lie in the subspace spanned by u1 .
2. The magnitude, given by the scaled inner product of f and u1 . Call this
magnitude the projection coefficient.
6.3 Normalization of the basis vectors
If the length of u1 is one, then Eqn (72) reduces to:
PU f = hf, u 1 i u 1
hf, u1 i
which has 2 parts:
Using f and u1 , a shorter version of u1 :
Section 6.3.0
0.1
0.7
0.2 1.4
0.2
0.7
=
0.1
0.2 1.4
0.2
0.1
0.1
0.7
0.1
(73)
and the projection coefficient is simply the inner product. This property is
sufficiently handy that we need to call it something:
Normal basis vector: a basis vector with length 1 is said to be normal,
and is sometimes written with a hat : u 1 .
Normalization: the process of making a vector normal.
Matlab example:
>> u1 = [ 1 2 2 1 ];
>> u1hat = u1 / norm(u1);
Of course, the projection of f onto u1 is the same as onto u1 .
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 53
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 54
EE/ME 701: Advanced Linear Systems
Section 6.3.0
EE/ME 701: Advanced Linear Systems
6.4 Projection Matrices
Consider example 1 one more time, with u 1 normalized:
Normalized :
Now
0.316
Section 6.4.0
One more interesting fact about projecting f onto U: since hf, u 1 i is a scalar,
Eqn (73) can be written:
0.632
u1
=
u 1 =
||u1 ||
0.632
0.316
(74)
PU f = hf, u 1 i u 1 = u 1 hu 1 , fi = u 1 u T1 f = u 1 u T1 f
(76)
M = u 1 u T1 is thus a term that multiplies a vector to give the projection of
the vector onto a subspace:
||u 1 || = 1
And so
PU f =
hf, u 1 i
||u 1 ||2
u 1 = hf, u 1 i u 1
(75)
In the example, this gives
0.316
0.316
0.316
0.70
0.632 0.632
0.632 1.40
= (2.214)
=
PU f =
4
0
2
1
0.632 0.632
0.632 1.40
0.316
0.316
0.316
0.70
Also, defining
0.316
0.1 0.2 0.2 0.1
i
0.632 h
0.2 0.4 0.4 0.2
0.316 0.632 0.632 0.316 =
M=
0.632
0.2 0.4 0.4 0.2
0.316
0.1 0.2 0.2 0.1
Now
0.1 0.2 0.2 0.1
0.70
0.2 0.4 0.4 0.2 0 1.40
Mf =
0.2 0.4 0.4 0.2 2 1.40
0.1 0.2 0.2 0.1
1
0.70
W = U
notice that
Now for any vector f, the projection is given as: PU f = M f .
3.3
1.4
W
w = f PU f =
0.6
1.7
M is pretty handy also, and so lets call it a projection matrix or
projection operator.
Student exercise, what is the dimension of W ?
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 55
Student
exercise
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 56
EE/ME 701: Advanced Linear Systems
Section 6.4.1
EE/ME 701: Advanced Linear Systems
Notice that if we write
6.4.1 Bay example 2.10, projecting f onto a 2-D subspace
then:
0
2
0
, u1 = , u2 =
f=
2
2
1
1
1
1
(77)
and also:
Solution:
and so:
Since PU f U , PU f can be written:
PU f = 1 u1 +2 u2
1 , 2 R
u1 , u2 R4 ,
U = span {u1 , u2 }
(78)
Writing f = 1 u1 + 1 u1 + w, and using Eqn (78) and the linearity of
inner product
hf , u1 i = h1 u1 + 2 u2 + w , u1 i = 1 hu1 , u1 i + 2 hu2 , u1 i + 0
1
2
Part 2: Vectors and Vector Spaces
hu1 , u1 i hu2 , u1 i
hu1 , u2 i hu2 , u2 i
hf , u1 i
hf , u2 i
(Revised: Sep 10, 2012)
hu1 , u1 i hu2 , u1 i
hu1 , u2 i hu2 , u2 i
hf , u1 i
hf , u2 i
hu1 , u1 i hu2 , u1 i
hu1 , u2 i hu2 , u2 i
= UT U
hu1 , fi
hu2 , fi
(82)
= UT f
hf , u1 i
hf , u2 i
(83)
(84)
1
= UT U
U T f (85)
The projection coefficients are given by Eqn (85).
Equation (85) for projecting f onto basis vectors U is related to what
equation that we have seen before ?
Finally, the projection is give by:
(80)
Which solves to give:
(79)
hf , u2 i = h1 u1 + 2 u2 + w , u2 i = 1 hu1 , u2 i + 2 hu2 , u2 i + 0
Eqn (79) may be written:
hu1 , u1 i hu2 , u1 i
hf , u1 i
1
hf , u2 i
hu1 , u2 i hu2 , u2 i
2
U = u1 u2
| |
Consider again these vectors in R4, and project f onto the
subspace U = span {u1 , u2 }
Section 6.4.1
PU f = 1 u1 + 2 u2 = U
1
2
1
= U UT U
UT f
(86)
And in projection matrix form:
(81)
PU f = M f with
Page 57
Part 2: Vectors and Vector Spaces
1
M = U UT U
UT
(Revised: Sep 10, 2012)
(87)
Page 58
Student
Exercise
EE/ME 701: Advanced Linear Systems
Section 6.4.1
Running the numbers:
1 0
1 0
Given a vector subspace U with an array of basis vectors U
1
2
2
1
2
0
10
3
=
UT U =
0 0 1 1 2 1
3 2
1 1
hf , u1 i
1
2
2
1
0
7
=
= UT f =
hf , u2 i
0 0 1 1 2
1
or
Using the projection PU f we can find the orthogonal complement, U
1
2
10 3
3 2
7
1
1
1
w = f Pf = (I M) f
where M is the projection matrix. We can say that
w = M f
where M = I M is the projection matrix for projecting a vector onto the
orthogonal complement, and w U .
Using the data of the example,
(88)
0.091 0.091
0.182 0.182
,
0.545 0.455
0.455 0.545
Part 2: Vectors and Vector Spaces
Since
1
2
PU f = 1.0 u1 1.0 u2 =
1
0
1
M = U UT U
UT
0.182 0.364
0.364 0.727
M =
0.091 0.182
0.091 0.182
w = f PU f =
Section 6.4.2
6.4.2 Projection matrix for the orthogonal complement
2 0
U =
2 1
1 1
EE/ME 701: Advanced Linear Systems
1.0
2.0
PU f = M f =
1.0
0.0
(Revised: Sep 10, 2012)
Page 59
0.818
0.364 0.091
0.091
0.364
0.273
0.182
0.182
M = I M =
0.091 0.182 0.455 0.455
0.091 0.182 0.455 0.455
and w = M f =
3 2 1 1
Part 2: Vectors and Vector Spaces
iT
(Revised: Sep 10, 2012)
Page 60
EE/ME 701: Advanced Linear Systems
Section 6.4.3
It would be nice to get rid of the matrix inversion step in projecting a vector
onto a subspace
b=
bT U
U
0.671
0.671
For many reasons, not least because the matrix inverse can be badly
conditioned.
Starting with
hf , u 1 i
0.316 0.632 0.632 0.316 0
2.214
T
=
b f =
=U
0
0
0.707 0.707 2
0.707
hf , u 2 i
1
1
0.671
3.162
2.214
=
=
2
0.671
1
0.707
1.414
1.0
2.0
PU f = 3.16 u 1 1.414 u 2 =
1.0
0.0
Of course, the projection matrix does not change by rescaling
vectors:
0.182 0.364 0.091 0.091
1
0.364 0.727 0.182 0.182
b
bT =
b U
bT U
U
M =U
0.091 0.182 0.545 0.455
0.091 0.182 0.455 0.545
Part 2: Vectors and Vector Spaces
Section 7.0.0
7 Gram-Schmidt ortho-normalization
6.4.3 Projection with normalized basis vectors
With normalized vectors:
0.316
0
0.632
0
b =
U
0.632 0.707
0.316 0.707
EE/ME 701: Advanced Linear Systems
(Revised: Sep 10, 2012)
the basis
(90)
UT U = I
so that
Since
1
M = U UT U
UT = U I 1 UT = U UT
hu1 , u1 i hu2 , u1 i
UT U = hu1 , u2 i hu2 , u2 i
..
..
...
.
.
(91)
(92)
Computing a projection without a matrix inverse requires that
hu1 , u1 i = hu2 , u2 i = = hur , ur i = 1
hu1 , u2 i = hu2 , u1 i = = ui , u j , i 6= j = 0
If
1
UT
M = U UT U
In other words, the basis vectors must be normal and orthogonal.
(89)
Page 61
Orthonormal: Basis vectors which are both orthogonal and normal are
called orthonormal.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 62
EE/ME 701: Advanced Linear Systems
Section 7.1.1
7.1 Process of Gram-Schmidt Ortho-normalization
EE/ME 701: Advanced Linear Systems
Section 7.1.1
1. Start with zero basis vectors, V = {}.
Gram-Schmidt Ortho-normalization is a process that starts with any span of
a vector subspace, and produces an orthonormal basis, V .
2. Take any vector from the subspace:
u1 =
2
1
In broad terms, the process to make orthogonal basis vectors works this way:
1. Start with zero basis vectors
2. Choose any vector u in the subspace
Choosing, for example, a vector from any spanning set.
V=
3. Subtract off the projection onto the existing basis vectors.
3. Subtract from u the projection of u onto the existing basis vectors.
4. If a sufficient vector remains after the subtraction, normalize the
remaining vector and add it to the set of basis vectors.
7.1.1 Example Gram-Schmidt Ortho-normalization
Starting with the spanning set, let subspace S
[u1 , u2 , u3 ]
1
0
2 0
S = span
, ,
1
1
Find a set of ortho-normal basis vectors for S .
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
0.316
0.632
u1
v1 = =
u
0.632
1
0.316
2
3
4. If a sufficient vector remains after the subtraction (if u1 > tol), normalize
the remaining bit and add it to the set of basis vectors.
be spanned by vectors A =
2 0 2
u1 =
=
2 0 2
1
0
1
5. Repeat steps 2-4 until the basis set is complete
The basis set is complete when r = n, or when all of the spanning
vectors have been used.
V = {v1} =
0.632
0.632
0.316
0.316
5. Repeat
Page 63
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 64
EE/ME 701: Advanced Linear Systems
Section 7.1.1
2. Take any vector from the subspace:
0
0
u2 =
,
1
1
V=
0.632
0.632
0.316
0.1 0.2 0.2 0.1
0.3
u3
0.286
0.572
u2
,
v2 = =
u
0.381
2
0.667
V = {v1 , v2} =
(93)
0.286
0.632 0.572
,
0.632 0.381
0.316
0.667
0.316
0.182
0.364
0.091 0.091
0.364 0.727 0.182 0.182
u3
= u3 V V u3 = u3
0.091 0.182 0.545 0.455
0.091 0.182 0.455 0.545
In general, vector u2 is not parallel to u2 , that is u2 u2 .
If u2 > tol, u2 can not perpendicular to u2 either.
4. If a sufficient vector remains after the subtraction (if u2 > tol), normalize
the remaining bit and add it to the set of basis vectors.
Note that u2 is the projection of u2 onto the orthogonal complement of V.
0.286
0.632 0.572
,
0.632 0.381
0.316
0.667
0.316
3. Subtract off the projection onto the existing basis vectors.
V=
1
2
u3 =
,
3
2
0.316
0.6
0.2 0.4 0.4 0.2
u2 =
u2 = u2 V V u2 = u2
0.4
0.2 0.4 0.4 0.2
0.7
0.1 0.2 0.2 0.1
Section 7.1.1
2. Take any vector from the subspace:
3. Subtract off the projection onto the existing basis vectors.
EE/ME 701: Advanced Linear Systems
= 10
0.444
0.888
0.444
0.444
15
4. If a sufficient vector remains, normalize the remaining bit and add it to the set
of basis vectors.
u3 < tol, u3 lies in V = span (v1, v2)
No vector added to V
5. All out of vectors in the original spanning set. Done.
5. Repeat
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 65
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 66
EE/ME 701: Advanced Linear Systems
Section 7.1.2
In step 4 of the Gram-Schmidt algorithm, a vector ui is accepted to be a basis
vector if sufficient magnitude remains after projection onto the orthogonal
complement of V :
,
include ui / ui
Using
S = span
(94)
is given by:
where
A=
u1 u2 u p
0.316
0.286
The projection matrix onto the subspace given by
One possible answer is a value that scales with the number and size of
vectors and the machine precision,
0.632 0.572
0.632 0.381
0.667
0.316
V=
This begs the question, what should be the value of tol ?
tol = max (n, p) max ||ui ||
Section 7.2.0
7.2 Projection matrix with GS Ortho-normalization
7.1.2 Tolerance value for Gram-Schmidt algorithm
if ui > tol
EE/ME 701: Advanced Linear Systems
Rnp is the initial set of p spanning vectors,
maxi ||ui || is the maximum of the norms of the columns ui Rn ,
is the machine precision
tol given by Eqn (94) reflects the fact that round-off errors in step 4 depend
on the dimensions of A and the magnitude of the vectors that make up the
calculation.
0 2
1 3
2
1
2
2
1
0.182
0.364
0.091 0.091
0.364
0.727
0.182
0.182
M = V VT =
0.091 0.182 0.545 0.455
0.091 0.182 0.455 0.545
(95)
And (compare with Eqn (88))
1.0
0
2.0
= Mf =
PU f = P
2
1.0
1
0.0
(96)
Importantly, nowhere in the whole process is a matrix inversion required !
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 67
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 68
EE/ME 701: Advanced Linear Systems
Section 7.3.0
EE/ME 701: Advanced Linear Systems
7.3 Projection Coefficients
7.4 Projection onto the orthogonal complement.
Note that when the set of basis vectors V is orthonormal, then:
PU f = 1 v1 + 2 v2 = V
1
2
= V VT f = V VT f
At each iteration of the GS Ortho-normalization, in step 3, we are
subtracting the projection onto the existing basis vectors from the candidate
vector.
(97)
This is the same as taking the projection onto the orthogonal complement of
the existing basis vectors. For example:
and so the projection coefficients to project f onto V are given simply by:
1
2
= VT f
x2 = x2 V V T x2 = I V V T x2
(98)
is the projection of x2 onto V .
Given a basis set V, the projection matrix to project onto the orthogonal
complement is given by:
Compare with Eqn (81) (repeated here)
1
2
Section 7.4.0
hu1 , u1 i hu2 , u1 i
hu1 , u2 i hu2 , u2 i
hf , u1 i
hf , u2 i
hu1 , u1 i hu2 , u1 i
hu1 , u2 i hu2 , u2 i
Eqn (98) requires no matrix inverse operation.
inv(X*X)
can be computed in
We are accustomed to thinking
Eqn (81), and this is true for reasonably conditioned matrices that arent
too big.
UT f
M = I V V T
x2 = M x2
(99)
Eqn (99) is not too surprising, it is saying:
The portion of x2 not lying in V is the total, minus the bit that is lying
in V.
Eqn (99) can be quite handy.
But for singular, poorly conditioned or even just large matrices (50x50
or larger)
inv(X*X)
may not exist, or the computation may lead
to large errors.
Gram-Schmidt is one of the most scalable and robust algorithms in linear
algebra.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 69
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 70
EE/ME 701: Advanced Linear Systems
Section 7.5.0
7.5 Projection and fitting parameters to experimental data
EE/ME 701: Advanced Linear Systems
Section 8.1.0
8 Additional Topics
Think back to y = A b and modeling experimental data
8.1 The Four Fundamental Spaces of a Matrix
When we write a model:
Consider matrix A and the operation y = A b :
y (k) = T (k) b
and then have
y = A b
b
with
b Rp
y Rn , b
Where p is the number of parameters, and n is the number of data points.
Where y is measured data and y is estimated from the model.
Then
1
b
AT y ,
b = AT A
b ,
y = A b
= y y
The model set (also called the reachable set) is set of outputs, y , given
by the model for any possible tuning of parameters b
b.
For a linear model, the model set forms a linear vector space:
The columns of A are the spanning set for the r-dimensional model
set in the n-dimensional output space.
b are the projection coefficients of the data y onto the
The parameters b
basis vectors of the model set (namely the columns of A).
The G-S algorithm applied to the columns of A gives a basis for the
model set.
The projection theorem tells us that with b
b given by Eqn (100):
(Revised: Sep 10, 2012)
0
1
1
y = Ab
(101)
2. Output space: y Rn
It turns out that each of these spaces is further divided in 2, so there are a total of
four fundamental spaces of a matrix. For an n p matrix:
Input Space:
1. Null Space: Set of vectors b R p such that A b = 0
2. Row Space:
Orthogonal complement of the Null Space.
The row space is spanned by the columns of AT
A basis for the row space is given by Vrow = GS AT
Output Space:
3. Column space:
b Rp
A basis for the column space is given by Vcol = GS (A)
4. Left null space:
Said another way, there is no signal power remaining in which can
possibly be modeled by y (k) = T (k) b.
Part 2: Vectors and Vector Spaces
2
A=
The set of y given by the equation y = A b
Then lies in the orthogonal complement of the model set
(write A ).
Two vector spaces associated with multiplying a vector by matrix A are:
1. Input space: b R p
(100)
Page 71
The orthogonal complement of the column space
The set of all y such that y A b , b R p
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 72
EE/ME 701: Advanced Linear Systems
Section 8.1.0
Space of b vectors
Space of y vectors
Ro
w
row Sp
(A) ace
A b=y
br
Ab
ft N
lnu ull S
ll(A pac
)
e
b
0=A n
bn
Input Space, Rp
0
2
= A bn =
0
2
0
1
Le
Nu
l
Nu l Spa
ll(A ce
)
b=br+bn
8.1.1 Numerical Examples of the four fundamental spaces,
Null Space: The set of input vectors that give no output
Y=
Section 8.1.1
Input Space = Row Space + Null Space
Co
lu
co mn
l(A Sp
)
ac
Y=A br
EE/ME 701: Advanced Linear Systems
0
0
1
1
1
2
1
3
1
2
1
1 null (A)
(102)
Row Space: Vectors with a component from the row space give a nonzero output
Output Space, Rn
Figure 6: Pictorial representation of the four fundamental spaces of a matrix.
Row space and Null space lie in the input space, while Column space
and Left-null space lie in the output space. (Adapted from Strang).
The equation illustrated in figure 6 is:
y = A b = A br + A bn = y + 0
3
1
6
=A
1
9
2
6
1
1 contains a component from row (A)
2
(103)
Consider for the Row space:
Inputs come from
All vectors which give a non-zero output contain a component from
the row space.
A part from the Row space and a part from the Null space.
Outputs lie in the Column space
Any component from the null space adds to length ||b||2 , but
contributes nothing to the output.
The Left-Null space is unreachable by
So the minimum b vector giving y lies entirely in the row space.
y = Ab
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 73
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 74
EE/ME 701: Advanced Linear Systems
Section 8.1.1
Row and Null Space example
Section 8.1.1
Column Space example
3
1
2
=A
2 =A 1
9
2
1
6
b1 = 2 = br +bn = 1 + 1 ,
1
1
2
Vector b1 has a component from the row space and from the null space.
The contribution
determines the output
EE/ME 701: Advanced Linear Systems
1
6
=A
1
9
2
6
6
col (A)
9
6
(104)
The dimension of the column space is equal to the rank of A
b1r = 1
2
The contribution
The Column Space is the set of output vectors possible from A
dim col A = rank A
The columns of A span the column space
col A = {y : y = A b }
b1n = 1
Left-Null Space
The Left-Null Space is the set of vectors in the output space that can
not be reached by
y = Ab
makes no contribution to the output.
If
for any value of b
A b = y 6= 0 ,
and
It is the orthogonal complement of col A .
b has the minimum ||b||2 such that A b = y
then b row (A) .
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 75
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 76
EE/ME 701: Advanced Linear Systems
Section 8.1.1
Left-Null Space example:
Row Space:
1
lnull A
yln =
h
iT
Since 1 1 1 1 lies in the Left Null space of A , then the choice
for b which minimizes
is
1
1
1
1
Section 8.1.2
8.1.2 Computing bases for the four fundamental spaces
The Left-Null Space is the
orthogonal complement of the
column space. For example,
e
y =
EE/ME 701: Advanced Linear Systems
(105)
b= 0
0
where
(107)
R is a set of basis vectors spanning the Row Space of A, and
GS AT indicates applying the Gram-Schmidt algorithm to AT
y=
(106)
, gives b Vr , y Rn
with the suitable choice of y .
0
than b = 0
0
(Revised: Sep 10, 2012)
Since the Row Space is spanned by the rows of A, every vector in the
row space is given by:
b = AT y
What Eqns (106) and (105) are saying is that there is no choice for b
which gets any closer to
Part 2: Vectors and Vector Spaces
R = GS AT
Note: Recall that the GS algorithm takes any set of vectors, and returns an
ortho-normal basis on the space spanned by the vectors.
A b
The Row Space is spanned by the rows of A, therefore an ortho-normal
basis on the row space is given by:
Column Space:
The Column Space is spanned by the columns of A
C = GS (A)
(108)
where C is a set of basis vectors spanning the column space of A.
Since the Column Space is spanned by the columns of A, every vector y
in the column space is given by:
y =Ab
with the suitable choice of b .
Page 77
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 78
EE/ME 701: Advanced Linear Systems
Section 8.1.2
Null Space:
EE/ME 701: Advanced Linear Systems
Section 8.1.3
8.1.3 Bases for the Four Fundamental Spaces, Numerical Example
Note that
Using the GS algorithm, we can determine bases for the four fund. spaces
Mr = R RT
(109)
is a projection matrix, projecting any vector b onto the Row Space.
Since the null space is the orthogonal complement of the row space, the
projection matrix onto the null space is given by:
Mn = I Mr = I R RT
(110)
Since the columns of any projection matrix span the space onto which
the matrix projects, a basis set for the null space is given by:
null (A) = GS (Mn) = GS I R RT
(111)
>> A = [ 1 0 1; 2 0 2 ; 2 1 3; 1 1 2]
A =
1
0
1
2
0
2
2
1
3
1
1
2
n = 4
p = 3
r = 2
>> Col = GramSchmidt(A)
Col =
0.3162
-0.2860
0.6325
-0.5721
0.6325
0.3814
0.3162
0.6674
dim Col =
dim Row =
rank A = 2
Left-Null Space
The Left-Null Space is the orthogonal complement of the Column Space.
The projection matrix onto the column space is given as
Mc = CCT
(112)
and so the projection matrix onto the left-null is given as:
Mln = I Mc
(113)
And finally, a set of basis vectors for the left-null space is given by:
lnull (A) = GS (Mln) = GS I CCT
Part 2: Vectors and Vector Spaces
>> Row = GramSchmidt(A)
Row =
0.7071
-0.4082
0
0.8165
0.7071
0.4082
(Revised: Sep 10, 2012)
(114)
Page 79
>> Null = GramSchmidt( eye(3) - Row*Row)
Null =
0.5774
0.5774
-0.5774
dim Null =
p - dim Row = 1
>> LNull = GramSchmidt( eye(4) - Col*Col)
LNull =
0.9045
-0.0000
-0.4020
0.3333
-0.1005
-0.6667
0.1005
0.6667
dim LNull =
n - dim Col = 2
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 80
EE/ME 701: Advanced Linear Systems
Section 8.1.4
EE/ME 701: Advanced Linear Systems
Section 8.1.4
8.1.4 The Four Fundamental Spaces of a Matrix, revisited
Space of b vectors
Space of y vectors
Ro
w
row Sp
(A) ace
A b=y
Co
lu
co mn
l(A Sp
)
ac
Y=A br
br
Ab
b=br+bn
0=A
bn
Input Space, Rp
bn
Le
Nu
l
Nu l Spa
ll(A ce
)
ft N
lnu ull S
ll(A pac
)
e
Y=
Output Space, Rn
Figure 7: Pictorial representation of the four fundamental spaces of a matrix
(repeated).
Looking back to figure 6,
The row space and null space are orthogonal complements
The input space, R p for an n p matrix, is the union of the row and
null spaces:
p = dim row (A) + dim null (A)
(115)
The column space and left-null space are orthogonal complements
The output space, Rn for an n p matrix, is the union of the column
and left-null spaces:
n = dim col (A) + dim lnull (A)
(116)
Additionally, the dimensions of the row and column spaces must be equal
and are equal to the Rank of A
dim col (A) = dim row (A) = rank (A)
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
(117)
Page 81
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 82
EE/ME 701: Advanced Linear Systems
Section 8.1.6
8.1.5 Questions that can be answered with the four fundamental spaces
Given y = A b
EE/ME 701: Advanced Linear Systems
Section 8.2.0
8.2 Rank and degeneracy
Rank: In matrix theory the rank of a matrix is defined as the size of the largest
sub-array that gives a non-zero determinant.
What is the set of all possible b
b which give a specific b
y?
This is satisfactory as a formal, mathematical definition
When y is specified, is there an exact solution for b ?
But determinant is an unsatisfactory numerical calculation, because it is
numerically very sensitive, and cant handle non-square matrices.
If there is no exact solution, so = y A b 6= 0 ,
What is the set of all possible ,
Using the Gram-Schmidt algorithm, we can find a set of basis vectors for
the column space (or row space) of A, to determine the rank of A.
What is the dimension and a basis for this set
Is there any non-zero value e
b such that A e
b=0
In Matlab, Rank is obtained by the singular value decomposition, and
counting the number of singular values larger than a tolerance value. The
help message of rank() is instructive
What is the set of all possible e
b,
What is the dimension and a basis for this set
Given y, what is the smallest y A b
b
8.1.6 Two ways to determine the four fundamental spaces
1. With Gram-Schmidt ortho-normalization
Ortho-normalize the columns of A to get the column space, the left-null
space is the orthogonal complement.
>> help rank
RANK
Matrix rank.
RANK(A) provides an estimate of the number of linearly
independent rows or columns of a matrix A.
RANK(A,tol) is the number of singular values of A
that are larger than tol.
RANK(A) uses the default tol = max(size(A)) * norm(A) * eps.
(emphasis added)
Ortho normalize the rows of A to get the row space, the null space is the
orthogonal complement.
Range: The range of any function is the set of all possible outputs of that
function.
2. With the singular value decomposition (SVD)
The range space of matrix A is another name for the column space of A.
It is the range of function y = A b. It is the vector space spanned by the
columns of A.
Gives additional insight.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 83
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 84
EE/ME 701: Advanced Linear Systems
Section 8.2.0
EE/ME 701: Advanced Linear Systems
The nullity of a matrix is the dimension of its null space, denoted by q (A)
9 Summary and Review
with A Rnp ,
q (A) = p r (A)
Part 2 offers many definitions and concepts. However, as is often the case
with mathematical domains, there are only a few essential ideas:
(118)
where r (A) is the rank of A.
A vector space is a set of vectors, and in general will not include all
vectors of the universe.
Degeneracy
Simple operations, such as y = A b , lead naturally to vector spaces, and
our understanding of the solution can be in terms of vector spaces.
If rank (A) = min (n, p) we say the matrix is full rank. (It has the greatest
possible rank).
A vector space is defined by a span or basis.
If rank (A) < min (n, p) we say
The inner product is a measure of the degree of overlap (parallelism) of
two vectors.
The matrix is rank deficient
The norm is a measure of the length of a vector.
The matrix has lost rank (if something happened that made it rank
deficient, such as a robot has reached a singular pose)
Vectors and a vector space can be parallel, orthogonal, or somewhere in
between (include a component of each).
The matrix is degenerate.
The projection operation determines the components of a vector lying in
or orthogonal to a vector space.
THEOREM: The rank of a matrix product.
Given A Rnm and B Rmp, and form C = A B Rnp , then the following
properties hold:
rank (C) + q (C) = p
Gram-Schmidt ortho-normalization produces a basis that is handy for
determining projections.
(119)
rank (C) min (rank (A) , rank (B))
q (C) q (A) + q (B)
Naturally, there are a variety of details fleshing out each of these essential
ideas.
(120)
(121)
The rank and dimension of the Null Space of C are determined by how the
column space of B falls on the row space of A .
rank (C) = dim intersection (col (B) , row (A))
Part 2: Vectors and Vector Spaces
Section 9.0.0
(Revised: Sep 10, 2012)
Page 85
Student
Thought
Problem
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 86
EE/ME 701: Advanced Linear Systems
Section 9.1.0
9.1 Important properties of Inner Products
EE/ME 701: Advanced Linear Systems
Section 9.2.0
9.2 Important properties of Norms
An inner product is an operation on two vectors producing a scalar result.
A norm is a measure (in a general sense) of the size of something.
The following properties hold for inner products:
A norm is a function of a single vector that produces a real scalar result
1. Commutativity: hx, yi = hy, xi (overbar indicates complex conjugate).
||x|| R
2. Distributivity (linearity): hx, y1 + y2i = hx, y1 i + hx, y2i
3. Can be an induced norm: hx, xi 0 x , and hx, xi = 0 iff x = 0 .
A norm must have the following properties:
1. Positive definiteness: ||x|| 0 , ||x|| = 0 if and only if x = 0 .
It follows that:
2. Scalar Multiplication: || x|| = || ||x||
1. Scalar multiplication, right term: hx, yi = hx, yi
3. Triangle Inequality:
2. Scalar multiplication, left term: h x, yi = hx, yi
||x + y|| ||x|| + ||y||
The length of the sum of two vectors can not be greater than the
individual lengths of the vectors.
Additionally: hx, yi may be zero, positive or negative.
4. Cauchy-Schwarz Inequality: |hx, yi| ||x||2 ||y||2
Two vectors can not be more parallel than fully parallel.
Technically, a vector space can be a vector space without having any norm
defined.
To be a vector space requires only a set of elements and the 8 properties
described in section 2.2.
But for the familiar vector spaces of Rn we have seen several norms,
||x||1 , ||x||2 , ||x|| , etc.
Specifically called a normed vector space.
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 87
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 88
EE/ME 701: Advanced Linear Systems
Section 9.3.0
9.3 Study tip:
EE/ME 701: Advanced Linear Systems
Section 9.3.0
Embedded vector subspace
Proper vector subspace
Learn the definitions !
Hyperplane, Hypersurface
Terms appear in italic bold where they are introduced and defined.
All most elements of A have property p
Working together and talking about the subject mater will help toward this
goal.
Flash cards and drill may also be useful.
Orthogonality of subspaces
Representation of a vector
Transformation from one representation to another
Vocabulary :
Projection
Vector
Projection operator, Projection matrix, Projection coefficients
Euclidean vector
Non-orthogonal projection
Inner product (dot product)
Normalization of a vector, Normalized vector
Outer product (matrix product)
Ortho-normal vectors
Cross product (vector product)
Gram-Schmidt orthogonalization
Orthogonality
Model set, reachable set
Norm, Induced norm
Four fundamental spaces
Parallel, Co-linear, Orthogonal, Direction cosine
Row, Column, Null, Left-null Spaces
Definition Vector space
Range, Range space
Scalar, Vector, Closure, Linear combination of vectors
Rank, degeneracy
Additive identity, Multiplicative identity
Full rank, Rank deficient, degenerate
Span, Spanning set
Basis vectors
Dimension of a vector space
Standard basis
Vector universe
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 89
Part 2: Vectors and Vector Spaces
(Revised: Sep 10, 2012)
Page 90
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Linear Operators on Vector Spaces
4.3.2
4.4
Contents
Operator from data, example
. . . . . . . . . . . . . . .
46
Conclusions change of basis as a tool for analysis . . . . . . . . .
47
5 Operators as Spaces (Bay section 3.2)
1 Linear Operator
2 Rotation and Reflection Matrices
5.1
Operator Norms
. . . . . . . . . . . . . . . . . . . . . . . . . .
49
Operator norm properties . . . . . . . . . . . . . . . . . .
49
Determining the value of Operator norms . . . . . . . . . . . . .
50
5.1.1
5.2
48
2.1
Example Rotation and Reflection Matrices . . . . . . . . . . . . .
5.2.1
The L1 norm of an operator . . . . . . . . . . . . . . . . .
50
2.2
Three theorems and a corollary defining rotation matrices . . . . .
5.2.2
The L2 -norm of an operator . . . . . . . . . . . . . . . .
52
2.3
Summary of mathematical properties of rotation matrices . . . . .
14
5.2.3
The L -norm of an operator . . . . . . . . . . . . . . . .
53
2.4
Multi-axis rotations comprise rotations about each axis.
. . . . .
15
5.2.4
The Frobenius norm . . . . . . . . . . . . . . . . . . . .
54
Rotation matrix in terms of the from-frame axes
expressed in to-frame coordinates . . . . . . . . . . . .
18
Example: Photogrammetry, measurement from images . .
20
2.4.1
2.4.2
3 Linear Operators in Different Bases, or
A Change of Basis (Bay section 3.1.3)
3.1
21
Linear Operators and a change of Bases . . . . . . . . . . . . . .
4 Change of Basis as a Tool for Analysis
22
25
4.1
Example: non-orthogonal projection onto a plane . . . . . . . . .
28
4.2
Looking at the Fourier Transform as a change of basis
. . . . . .
37
4.2.1
The Fourier transform as a change of basis . . . . . . . .
42
4.2.2
Using the Fourier transform . . . . . . . . . . . . . . . .
43
4.3
Additional examples using change of basis
4.3.1
. . . . . . . . . . . .
44
Matching input and output data to discover an operator . .
44
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 1
5.3
Boundedness of an operator
. . . . . . . . . . . . . . . . . . . .
55
5.4
Operator Norms, conclusions
. . . . . . . . . . . . . . . . . . .
55
5.5
Adjoint Operators . . . . . . . . . . . . . . . . . . . . . . . . . .
56
6 Bay Section 3.3
57
7 Forming the intersection of two vector spaces
58
7.1
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
8 Conclusion
Part 3: Linear Operators
59
60
(Revised: Sep 10, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
Section 1.0.0
1 Linear Operator
EE/ME 701: Advanced Linear Systems
Section 2.0.0
2 Rotation and Reflection Matrices
An operator is a generalization of the notion of a function. Operators are
functions of numerical arguments, and also functions of functions.
See: http://mathworld.wolfram.com/Operator.html
Rotation Matrix: A rotation matrix R is an n n matrix with the properties:
Well be focusing on functions of numerical arguments, so an operator is
essentially a synonym for a function.
Linear Operator: An operator A from vector space X to vector space Y, denoted
A : X Y, is linear if it verifies superposition:
A (1 x1 + 2 x2 ) = 1 A x1 + 2 A x2
Rotation and Reflection matrices are good examples of linear operators.
They are used extensively and will play an important role in the Single Value
Decomposition.
1. The length of vectors is preserved:
||R x|| = ||x||
x Rn , R Rnn and a rotation matrix
(2)
2. Angles between vectors are preserved:
hR x1 , R x2 i
hx1, x2i
=
||x1|| ||x2 || ||R x1 || ||R x2 ||
(1)
x1 x2 Rn
(3)
(recall the direction cosine).
For vectors x Rm
and y Rn , then Eqn (1) corresponds to ordinary matrix-
Since the denominators are the same, Eqn (3) implies
vector multiplication with A Rnm .
hx1 , x2i = hR x1, R x2 i
Bay discusses linear operators using examples 3.1 - 3.13.
(4)
3. Handedness: Rotation matrices preserve handedness, equivalently for
rotation matrix R
det (R) = +1
Example: The projection operator is a linear operator
(mapping a vector from Rn to a subspace of dimension n).
Reflection Matrix:
1. Reflection matrices preserve length
2. Reflection matrices preserve angles
3. Handedness: Reflection matrices reverse handedness, for reflection
matrix Q
det (Q) = 1
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 3
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 2.1.0
EE/ME 701: Advanced Linear Systems
Setup rotation matrix R2 with = +40o, det (R2) = +1
2.1 Example Rotation and Reflection Matrices
2-D, Rotation Matrix:
cos () sin ()
R=
sin () cos ()
(5)
(Determinant will equal +1)
2-D, Rotation Matrix with Reflection:
Section 2.1.0
>>
>>
>>
>>
%% Setup a +40 deg rotation matrix
theta2 = 40
Ct2 = cosd(theta2)
St2 = sind(theta2)
Rot2 = [ Ct2, -St2; St2, Ct2 ]
Rot2 =
0.7660
-0.6428
0.6428
0.7660
>> det(Rot2)
ans =
1.0000
1 0 cos () sin () cos () sin ()
Q=
sin () cos ()
sin () cos ()
0 1
(6)
Add a reflection to R2 , det (Q) = 1
(Determinant will equal -1)
Setup rotation matrix R1 with = 40o , det (R1) = +1
%% Setup a -40 deg rotation matrix
>> theta=-40
>> Ct = cosd(theta); St = sind(theta);
>> Rot1 = [ Ct, -St; St, Ct ]
Rot1 =
0.7660
0.6428
-0.6428
0.7660
>> det(Rot1);
ans = 1.0000
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 5
%% A reflection operator can be an identity matrix
%% with an odd number -1 elements.
>> Rot2PlusReflection = [-1
0 ] * Rot2;
[ 0
1 ]
Rot2PlusReflection =
-0.7660
0.6428
>> det(Rot2PlusReflection)
ans =
-1.0000
Part 3: Linear Operators
0.6428
0.7660
(Revised: Sep 10, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 2.1.0
EE/ME 701: Advanced Linear Systems
2.2 Three theorems and a corollary defining rotation matrices
Process points with rotation and reflection matrices.
The rotation and reflection matrices are linear operators, they map points
in R2 onto points in R2 .
>> P1 = [ 0.5000
0.5000
0.7000
0.7500
0.2500
0.2500]
>> P2
= Rot *
P1
P2 =
0.8651
0.5437
0.6969
0.2531
-0.1299
-0.2584
>> P3
= Rot2 * P1
P3 =
-0.0991
0.2223
0.3755
0.8959
0.5129
0.6415
>> P3b = Rot2PlusReflection * P1
P3b =
0.0991
-0.2223
-0.3755
0.8959
0.5129
0.6415
THEOREM 3.1: A rotation matrix must have the property that RT = R1.
Proof: Because lengths must be preserved, the angles condition can be rewritten:
hx1, x2i = xT1 x2 = xT1 RT R x2 = hR x1, R x2i
(7)
For Eqn (7) to be true x1 , x2 Rn, RT R must be the identity matrix, ergo
RT = R1.
QED
Example: Rotation matrices are linear operators
For a rotation of [degrees] in R2, the rotation matrix is:
C
S
R =
S C
Rotation and Rotation with Reflection
1
Rotated by 40 deg
and Reflected over the Y axis
(X values inverted)
Section 2.2.0
Original
(8)
where C = cos () and S = sin ().
0.5
As an example, consider = 20o, then
0
0.94
0.34
R=
0.34 0.94
Rotated by 40 deg
0.5
1
0.8
0.6
0.4
0.2
0.2
0.4
0.6
0.8
Verify that R given by Eqn (8) is a ortho-normal matrix for any value of .
Figure 1: Example of rotation and reflection.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 7
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 8
Student
Exercise
EE/ME 701: Advanced Linear Systems
Section 2.2.0
EE/ME 701: Advanced Linear Systems
If the vi are normal, then
THEOREM 3.2: For any square ortho-normal matrix A:
i)
The transpose of the square ortho-normal matrix is the matrix inverse,
AT = A1.
ii)
Square ortho-normal matrices preserve length: ||A x|| = ||x|| x Rn .
ii)
Square ortho-normal matrices preserve angles:
1
0
0
T
A A= 0 1 0
0 0 1
ii)
iv) The determinant of A is either det (A) = 1 or det (A) = 1.
Proof:
i)
To show that
A=
then
v1 v2 v3
By property i) RT R = I. By property ii) ||R xi || = ||xi ||, so
hv1, v1i hv1 , v2 i hv1 , v3i
AT A = hv2, v1i hv2 , v2 i hv2 , v3i
hv3, v1i hv3 , v2 i hv3 , v3i
hR x1 , R x2 i
hx1, x2i
=
||x1|| ||x2 || ||R x1 || ||R x2 ||
(9)
Part 3: Linear Operators
0
0
hv1, v1i
AT A =
0
0
hv2, v2i
0
0
hv3 , v3 i
(Revised: Sep 10, 2012)
iv) Two properties of the determinant are:
1. For square matrices B, Q and R with B = Q R, det (B) = det (Q) det (R),
2. For any square matrix B, det (B) = det BT
If the vi are orthogonal, then
q
q
p
Let x2 = A x1, then ||x2 || = hx2, x2i = xT1 AT A x1 = xT1 x1 = ||x1||,
where the AT A is eliminated in the 3 step because AT A = I .
iii) hR x1 , R x2 i = xT1 RT R x2
, consider the matrix
(11)
and AT = A1. The argument of this example extends directly to
A Rnn .
x1 , x2 = R x1 , R x2
AT A = I
Section 2.2.0
(10)
Since AT A = I, it follows that det AT det (A) = det (I) = 1. But since
det AT = det (A), it follows that det (A) = 1, thus, given that A is a square
ortho-normal matrix, det (A) = +1 or 1.
QED
~
Page 9
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 2.2.0
THEOREM 3.3: Any square ortho-normal matrix A is a rotation matrix if
det (A) = +1 and a reflection matrix if det (A) = 1, and any rotation or
reflection matrix is a square ortho-normal matrix.
EE/ME 701: Advanced Linear Systems
x1 =
0 1 0
Section 2.2.0
with the 1 in the kth element. Now
hx1, x2i = xT1 x2 6= xT1 x3 = xT1 RT R x2 = hR x1 , R x2i
Proof:
(13)
This contradicts the hypothesis that R is a rotation or reflection matrix.
i)
Any square ortho-normal matrix is either a rotation or reflection matrix
Theorems 3.1 and 3.2 establish that a square ortho-normal matrix preserves
lengths and angles, therefore it is either a rotation or reflection matrix.
ii)
Any rotation or reflection matrix R is a square ortho-normal matrix, proof by
contradiction.
Assume there is a rotation or reflection matrix R which is not a square
ortho-normal matrix, that would imply that either the columns of R are not
orthogonal or that the columns of R are not normalized. Show that each leads
to a contradiction.
ii.a) If the columns are not orthogonal, show that R can not preserve angles.
First, looking back to Eqn (9), if the columns are not orthogonal, there must
be at least one pair vi , v j , i 6= j such that vi , v j = ai j 6= 0. Therefore
RT R 6= I.
Next we need to show that since RT R 6= I ,
x1 , x2 s.t.
hx1 , x2i = xT1 x2 6= xT1 RT R x2 = hR x1 , R x2i
(12)
Note: In a proof, it is not enough to simply assert Eqn (12). Even though
RT R 6= I, how do we know there are allowed choices for x1 and x2 such
that xT1 x2 6= xT1 RT R x2 . The last step of the proof gives a prescription to
construct such a x1 and x2.
ii.b) If one or more columns are not normalized, show that R can not preserve
angles.
In this case there is no vi , v j = ai j 6= 0, i 6= j ; but there is hvi , vi i = aii 6= 1.
Select x2 = vi , then RT R x2 = aii x2 , and
hx1, x2i = xT1 x2 6= xT1 aii x2 = xT1 RT R x2 = hR x1 , R x2i
This contradicts the hypothesis that R is a rotation or reflection matrix.
Thus, it is shown that given R is a rotation or reflection matrix, the assumption that
R is not ortho-normal leads to a contradiction.
QED
Thus all rotation or reflection matrices are ortho-normal matrices.
Because RT R 6= I , there exists at least one x2 such that RT R x2 = x3 6= x2.
Because x3 6= x2, there must be at least one element of x3 which does not
equal the corresponding element of x2, call this the kth element, and choose
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 11
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 12
EE/ME 701: Advanced Linear Systems
Section 2.2.0
THEOREM 3.4: Any two ortho-normal coordinate frames (sets of basis vectors)
A and B in Rn are related by a rotation matrix R and at most one reflection.
Proof: We can transform vectors represented in either coordinate frame to the
standard frame by
s
s
x = A ax ,
x = B bx
EE/ME 701: Advanced Linear Systems
Section 2.3.0
2.3 Summary of mathematical properties of rotation matrices
Rotations and reflections are linear operators, they map from Rn to Rn by a
matrix multiplication.
Matrices that preserve lengths and angles are either rotation or reflection
matrices.
and so
a
bT
= A1 B
and so
det ( abT ) =
All rotation or reflection matrices are square ortho-normal matrices, giving
RT R = I
1
det (B)
det (A)
Thus RT = R1.
Note: with A, Q, R Rnn :
If R is a rotation matrix, RT is a rotation matrix.
det A1 = 1/ det (A) ,
If R1 and R2 are rotation matrices, R3 = R1 R2 is also a rotation matrix.
All square ortho-normal matrices are either rotation or reflection matrices,
if
det (Q R) = det (Q) det (R)
Since A and B are ortho-normal bases, det (A) = 1 , det (B) = 1, which
shows that det ab T = 1, and so abT incorporates at most one reflection.
Even in Rn !
det (R) = +1 the matrix is a rotation matrix,
det (R) = 1 the matrix is a reflection matrix.
~
COROLLARIES 3.4: The action of any two reflections in Rn is to restore the
original handedness.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 13
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 2.4.0
EE/ME 701: Advanced Linear Systems
Section 2.4.0
2.4 Multi-axis rotations comprise rotations about each axis.
Original Coordinates
Bay gives the example of a rotation operator.
Pitch 30.00 [deg]
Za
In Robotics and elsewhere, the rotation operator is called simply a
rotation matrix.
Za
1
0.5
0.5
0.5
0.5
Ya
Pitch, roll and yaw rotations are seen in figure 2. These are the rotations
about the three axes, and can also be referred to as:
Yb
0
0
Rotation about the X-axis, Rx (pitch)
0.5 0.5
0.5
0.5
0.5
1
Y
0.5
0
0
1
1.5 1.5
Ya
Xa
0.5
0.5
1
Y
1
1.5 1.5
Rotation about the Z-axis, Rz (roll)
Rotation about the Y -axis, Ry (yaw)
The terms pitch, roll and yaw are assigned to different axes by different
authors.
Roll 30.00 [deg]
Yaw 30.00 [deg]
Z
b
Z
Rx , Ry , Rz have the advantage of being unambiguous.
Zb Za
0
0.5
0.5
An example multi-axis rotation is seen in figure 3.
Xa
0.5
0.5
Z
Multi-axis rotations preserve length and angles (so a Euclidean basis set
such as the X, Y, Z axes, remain orthogonal).
Xb
0
0.5
0.5
1
Y
1
1.5 1.5
0.5 0.5
0.5
0.5
0
0
0.5
0.5
1
Y
1
1.5 1.5
Figure 2: 3D rotations, illustrating individual rotations Rx (pitch), Rz (roll), and
Ry (yaw). Note right-hand rule for rotation direction.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 15
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 16
EE/ME 701: Advanced Linear Systems
Section 2.4.0
EE/ME 701: Advanced Linear Systems
Section 2.4.1
2.4.1 Rotation matrix in terms of the from-frame axes expressed in toframe coordinates
Above view
Z
Side View
Pitch: 30.00, Roll: 20.00, Yaw: 25.00
1
0.5
For example, the rotation from B to A in figure 4 is given as
0.866 0.500
C S
A
=
BR =
0.500 0.866
S C
XX
b
0.5
Xa
Ya
0.5
0.5
0
0.5
0.5
0.5
1
Y1.5
1.5
0.5
0.5
0 0.5
1 1.5 1.5 1
Y
A rotation matrix provides a transformation from one coordinate frame to
another, we can call these the from frame and the to frame.
0.5 0
0.5
Point Pa is given as
Pa =
0.7
0.5
Pa = AB R BPa =
0.356
0.783
Pa
a
YB
xis
Y axis
0.5
xis
Top View
Pitch: 30.00, Roll: 20.00, Yaw: 25.00
a
XB
0
X axis
0.5
1
1
0.5
0
0.5
1.5
Y
1
Za
0.5
0.5
Z
0
0.5 0.5
0.5
Figure 4: Rotation from the A frame to the B frame, BA R.
1.5
0.5
Figure 3: General 3D rotation, combining rotations about all three axes.
The rotation matrix from B to A given by the axes of B expressed in A
Look at figure 4,
|
|
A
A
AY
BR = X B
B
|
|
Look at the B axes in figure 4, expressed in the A coordinate frame.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 17
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 2.4.1
3-D example
EE/ME 701: Advanced Linear Systems
Section 2.4.2
2.4.2 Example: Photogrammetry, measurement from images
A typical application comes from photogrammetry, where it is often
necessary to shift vectors from target to camera coordinates.
The rotation from the B frame to the A frame in figure 3 is given by:
0.85
0.49
0.17
A
B R = 0.31 0.74 0.60
0.42 0.45 0.78
To shift coordinates from camera to target coordinates:
By convention, the rotation from the camera frame to the target frame is
given by rotations about the three axes, corresponding to three angles pitch,
roll and yaw: Rx (pitch, ) Rz (roll, ) Ry (yaw, ).
0.85
A
X B = 0.31
0.42
c
tR =
In general, the rotation from a B coordinate frame to an A coordinate
frame is given by:
A
A
BR = X B
AY
AZ
= tc R cPa + tPc.
Where tPc R3 is an offset vector.
The X-axis of the B frame, expressed in A coordinates is:
tP
a
(14)
c
tR
1 0
1 0
0
0
0 Rz Ry Rx
0 1
1
C S 0
0 S
= 0 1 0 S C 0 0 1 0 0 C S
0 0 1
0
0 1
S 0 C
0 S C
CC +C S CS S S S CC S
= SC
(15)
S SS +CC C SS CS
S
C S
CC
Compare Eqn (15) with Bay, Eqn (3.16). Bay uses a different ordering for
the rotations.
Because matrix multiplication does not commute, Bays the 3-axis
rotation matrix - while similar - is not exactly the same as tcR.
There are at least 48 ways to put together a 3-Axis rotation matrix, and
they are all found somewhere in the literature.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 19
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 3.0.0
EE/ME 701: Advanced Linear Systems
3 Linear Operators in Different Bases, or
3.1 Linear Operators and a change of Bases
A Change of Basis (Bay section 3.1.3)
A linear operator is just a map from one vector onto another
When we make a change of basis, we change the axes on which a vector is
represented. For example, given two bases on R2 ,
U=
u1 u2
The vector f =
1 0
=
,
0 0.5
0.5 1
T
V=
v1 v2
y1 = A x1
If we change the representation of the vectors, we have to make a suitable
change in the linear operator.
0.8 1
=
0.8 0
Change the representation of the vectors
x2 = Bx x1
f=U f=
u1 u2
y2 = By y1
(17)
0.8
1
or f = 1.25
+0.5
0.8
0
where Bx and By are basis transformations in the input and output spaces,
respectively, and must be square and invertible.
Rewriting Eqn (16) with the change of basis
y1 = A x1
y2 = A x2
Which is equivalent to writing
(16)
can be represented
0.5
1
0
f=
= 0.5 +2.0
1.0
0
0.5
Section 3.1.0
0.5
,
2.0
or f = V f =
v1 v2
1.25
0.5
With a change of basis from U to V , the representation of a vector changes,
but the vector itself (vector f) remains the same.
(Eqn 16, repeated)
(18)
Relating Eqn (16) to Eqn (18) gives
By y1 = A Bx x1
or
y1 = B1
y A Bx x1
(19)
Using the uniqueness of Eqns (16) and (18), Eqn (19) implies that
A = B1
y A Bx
or, equivalently
A = By A B1
x
We started with the equation
y1 = A x1
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 21
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 22
EE/ME 701: Advanced Linear Systems
Section 3.1.0
EE/ME 701: Advanced Linear Systems
Section 3.1.0
Notice that when we write
and ended up with the equation
y2 = A x2
y2 = By y1
where the input and output bases of linear operator A have changed.
Which brings up this point: implicit in any linear operator are the bases
in which the input and output are expressed.
We normally assume these to be the standard Euclidean bases for Rm
and Rn .
x2 = Bx x1
the transformation matrices By and Bx must be square matrices, and full rank
to be invertible. By Rnn , Bx Rmm
A special case of basis transformation arises when A is a square matrix, or
A : Rn Rn . In this case the input and output transformations can be the
same,
y2 = B y1 , x2 = B x1
Combining with Eqns (16)-(19) above, we can write
y1 = A x1 = B1 A B x1
Which gives:
A = B1 A B
1
BAB
(20)
= A
(21)
When A and B are square and B is invertible, Eqn (20) has a special name.
It is called a similarity transformation.
Similarity transformations preserve the eigenvalues, in other words
eig (A) = eig A , where eig (A) is the vector of eigenvalues of A
Similarity transformations are going to give us the freedom to re-write
system models from one basis to another, to explore model properties such
as modal response, controllability and observability.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 23
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 24
EE/ME 701: Advanced Linear Systems
Section 4.0.0
4 Change of Basis as a Tool for Analysis
EE/ME 701: Advanced Linear Systems
Section 4.0.0
The basic architecture of an application of change of basis is this,
Given an application:
We have seen the basic mechanics of
1. The application domain has natural basis
Representation of a vector on a specific basis, and
Example:
How to change the basis of vectors, and finally
How to adapt an operator, when the vectors it operates on change basis
These are powerful tools for several types of problems.
Time-domain, for waveform compression
Call this basis s, for the standard basis
2. The application is difficult in the original basis
Bay homework problems 3.4, 3.5, 3.6, 3.7, 3.9, 3.10 and 3.11 are all
addressed by a change of basis.
Example:
A change of basis is the mathematical foundation for many common
methods
If we just throw out 9 out of 10 samples in the time domain, the
reconstruction will be very bad
3. There is an alternative basis on which the application is easy
Fourier transform
Wavelet transform
Example:
Expansion onto Lebesgue and Legendre polynomials
If the data vector (a voice signal) is represented on a set of
wavelet vectors (discrete wavelet transform), the application is
easy:
And applications
Just throw out the coefficients for basis vectors that make
little difference for human perception
Reconstructing MR images
jpeg image compression
Speech compression for cell-phone transmission
And for control systems, a change of basis is necessary to solve
x (t) = A x (t) + B u (t)
at all.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 25
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 26
EE/ME 701: Advanced Linear Systems
Section 4.0.0
EE/ME 701: Advanced Linear Systems
4.1 Example: non-orthogonal projection onto a plane
4. We solve the problem or achieve the application in 3 steps:
Step 1. Transform the data from the s basis into a basis where the
application is easy, call this this F basis (it can have a different
name for each application).
Bay problem 3.9 raises an interesting challenge from computer graphics.
Problem 3.9 (adjusted values):
Let P be the plane in R3 defined by 1x 1y + 2z = 0, and let l be the vector
Step 2. Solve the application with data in the new F basis
2
l= 1 .
1
Step 3. Transform the results back to the s basis, for utilization
s
A = sF T
Problem expressed
in s basis
(Problem is generally
unsolvable on the s basis)
F
sT
(22)
Solution expressed
in s basis
Alternative, F, basis
Solve
Problem
Section 4.1.0
Denote by A : R3 R3 the projection operator that projects vectors in R3
onto the plane P, not orthogonally, but along vector l.
Projected points can be pictured as shadows of the original vectors onto
P, where the light source is at an infinite distance in the direction l.
(Problem is solvable
on the F basis)
1. Find the matrix
of operator
F A which projects a vector represented in
basis F = f1, f2 f3 onto plane P, where f1 and f2 are any non-
Figure 5: Problem solving steps using a change of basis.
collinear vectors in the plane P, and f3 = l.
Problem: Canons blowing
up, Heat distribution
known in s basis
(s: just x, y, z)
No solution to
Heat-conduction
Equation for general
h0(x,y,z)
Alternative, F, basis is
sine and cosine functions
Solve
Problem
Heat conduction can
be solved for sin(x)
initial distrubtion.
Solution expressed
in s basis.
Hot points determined,
cannon redesigned.
By superposition, the solution
for many sine functions, is the
sum of the solutions for each
individual sine fucntion.
(Revised: Sep 10, 2012)
Non-orthogonal projection is a standard operation in computer graphics.
Rendering systems do shading and compute reflections by representing
complex surfaces as many flat triangular tiles, and computing the
intersection point of many rays with these tiles.
Figure 6: Problem solving steps using a change of basis.
Part 3: Linear Operators
2. Find the matrix of operator s A which projects a vector represented in
standard R3 basis onto plane P.
Page 27
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 28
EE/ME 701: Advanced Linear Systems
Section 4.1.0
The origin For the plane to be a 2-D vector subspace, the origin must lie in the
plane. In practical systems with many tiles (planes), the origin is offset to
a point in each plane as it is processed, so that a projection operator can be
used.
y1
ne
Section 4.1.0
Definitions and background details:
x1
Pla
EE/ME 701: Advanced Linear Systems
Surface Normal The plane is specified by its surface normal, a vector that is
orthogonal to the plane. This is a standard way to specify an n 1
dimensional hyper-surface in Rn .
x2
~n: Surface normal
y2
It is necessary to find basis for vectors for plane P . We can use the fact
that all vectors in plane P are orthogonal to n.
~l: Shadow ray
The projection The calculation
y = Ax
Figure 7: Illustration of projection along ray ~l. In computer graphic, complex
surfaces can be represented as a mesh of triangular tiles.
is a projection of a point (x) onto the surface (to point y) . Since x and y are
both 3-vectors, A is a 3x3 matrix.
The problem can be stated: Given a ray originating at point
x
x= y ,
z
following the line
Note: up to now we have considered only orthogonal projections, that is, if
g = P f is the projection of f onto a subspace, and w = f g, then w g.
2
l= 1
1
But in this case, as with many projections in computer graphics, the
projection is not orthogonal. That is:
Where does the ray strike a plane defined by the surface normal n = 1 ?
2
Being able to cast the relation as a linear operator, of course,
greatly simplifies and accelerates the calculation.
A Play Station 3 can perform this calculation several billion times per
second.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 29
The line l is not orthogonal to the plane or, equivalently,
l is not parallel to n, l n .
Bases: This problem is approached in problem 3.9 (a) on a basis F. In this
note, points in the standard space (x , y , z) are labeled sx and sy. Vectors
expressed on basis F are labeled Fx and Fy .
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 30
EE/ME 701: Advanced Linear Systems
Section 4.1.0
Suggested Approach: With many engineering challenges, it is good to ask the
question: when I have an answer, how can I verify that it is the correct
answer. Turn problem 3.9 around and ask:
given a point sxi in R3 and its shadow in plane P (call this syi ) how
can I independently verify that syi is correct ?
For this problem, the reverse problem, verification, may be easier than the
analysis to determine sy, and thinking through how to verify a correct
answer may help find how to determine sy.
EE/ME 701: Advanced Linear Systems
(b): to verify that syi is the shadow of sxi consider what it means to be the shadow:
yi sxi = l
Example Data:
1. Given a point sxi and a point syi in the plane, derive the calculation to verify:
(b) that syi is the shadow of sxi
(a): to verify that point sy1 lies in plane P we need basis vectors for P. These will
be any two independent vectors lying in P. That is, any two independent
vectors, each orthogonal to n.
1
sx
2
2.67
s
1 = 3 which projects onto y1 = 0.67
4
1.67
Verifying that syi lies in plane P: One way to check that sy1 lies in P is to form
the projection of sy1 onto P, and verify that it equals sy1 .
(a) that syi is in plane P, and
(24)
If syi sxi = l , then syi is the shadow of sxi .
Consider the point
Considering verification of the correctness of a solution:
Section 4.1.0
Considering that n = 1 ,
one choice is: P = 0 1
1 0
(23)
The projection of sy1 onto P is given by:
>> P = [ 2 1 ; 0
P =
2
1
0
1
-1
0
1 ; -1 0 ]
sy1 = -2.67
0.67
1.67
>> %% Verify sy1 lies in P
>> sy1hat = P * inv(P*P) * P * sy1
sy1hat = -2.67
0.67
1.67
>> sy1hat-sy1
ans =
1.0e-15 *
-0.8882
0
-0.1665
1
PT syi
y = P PT P
If y = syi , then sy1 lies in P.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 31
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 32
EE/ME 701: Advanced Linear Systems
Section 4.1.0
Verifying that syi is the shadow of sxi : To show that sy1 is the shadow of sx1,
show that ( sy1 sx1 ) is parallel to l . For the example data we find:
%% Difference Vector
>> ll = sy1-sx1
ll = -4.6667
-2.3333
-2.3333
%% Term-by-term ratio
>> ll ./ l
ans = -2.3333, -2.3333, -2.3333
%% Projection ray l
l =
2
1
1
EE/ME 701: Advanced Linear Systems
Section 4.1.0
2. Given a vector Fx1 represented on basis F, find the operator FA that
determines Fy1, that is, determines the shadow point represented on basis F.
Discussion: Using F as a set of basis vectors, than any point sxi is represented as
2
1
2
s
xi = a1 0 + a2 1 + a3 1 = F F xi
1
0
1
sy1-sx1 = -2.3333 l
with
Now considering text problem 3.9.
(Problem 3.9 part a)
Find yi on the F basis. Given
1. Construct the set of basis vectors
F=
f1 f2 l
where f1 and f2 are basis vectors in the plane, and l is the projection ray.
Given P above, the set of basis vectors for F is:
F=
f1 f2 l
b1
yi = b2
b3
so that
2 1 2
= 0 1 1
1 0 1
(Revised: Sep 10, 2012)
Part 3: Linear Operators
a1
xi = a2
a3
This gives:
b1 1 0 0
F
yi = b2 = 0 1 0
0 0 0
0
Page 33
2
1
2
yi = b1 0 + b2 1 + b3 1 = F F yi
1
0
1
To find syi in plane P , set b3 to 0 !
Many F matrices are possible, with f1 and f2 as basis vectors for P.
Part 3: Linear Operators
(25)
F
xi
(Revised: Sep 10, 2012)
Page 34
EE/ME 701: Advanced Linear Systems
Section 4.1.0
For yi to be the projection of xi along l, in the F basis vectors, we can
only modify the 3rd coefficient, giving
b2 = a2
b1 1 0 0
b2 = 0 1 0
0 0 0
0
Or
yi = FA Fxi
FA
with
(Problem 3.9 part b)
Discussion: We have these three relationships:
Properties (a) yi lies in P, and (b) yi is the projection along l of xi
determine the projection in basis F :
Section 4.1.0
1. Starting with FA, find the operator sA that determines sy1 corresponding to
a point sx represented on the standard basis.
b1 = a1
EE/ME 701: Advanced Linear Systems
a1
a2
a3
1
0
0
F
A= 0 1 0
0 0 0
(26)
Standard map from F coordinates to standard coordinates:
s
yi = F Fyi ,
s
FT
so
=F
(28)
The standard map from s coordinates to F coordinates (just the inverse
of Eqn (28))
xi = F 1 sxi ,
note
F
sT
= F 1
(29)
And we have the operator in F coordinates, FA .
(27)
Answer: put the three pieces together
s
y =
s
FT
F
sT
xi
= sA sx
(30)
is the projection operator on the F basis.
>> sA = F*[1 0 0; 0 1 0; 0 0 0] * inv(F)
sA =
0.3333
0.6667
-1.3333
-0.3333
1.3333
-0.6667
-0.3333
0.3333
0.3333
Note: there is only one operator sA that projects along line l to plane P in standard
coordinates. The numerical values come out to those given, whatever basis
is chosen for P.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 35
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 36
EE/ME 701: Advanced Linear Systems
Section 4.2.0
4.2 Looking at the Fourier Transform as a change of basis
EE/ME 701: Advanced Linear Systems
Consider the inverse DFT
The Fourier transform is written as a convolution integral. Given f (t)
Fourier Transform:
Inverse Fourier Transform:
F () =
f (t) =
2 j t
f (t) e
dt
F () e+2 j x d
f ( j) =
(31)
(32)
F (k) =
1
f ( j) =
N
Inverse DFT:
f ( j) e
j=1
N
2i
N ( j1)(k1)
F (k) e
2i
N ( j1)(k1)
(34)
k=1
The DFT gives a signal in frequency domain, k = (k 1)/N [cycles/sample].
f(j)
2
0
20
40
60
80
j [sample number]
100
120
140
real F(k)
100
50
0
50
0
10
20
30
40
50
60
70
imag F(k)
50
50
0
10
20
30
40
k [wave number]
50
60
70
>> w1 = 1/16; w2 = 1/32;
>> jjs = 1:128;
>> for j1 = jjs,
f(j1) = cos(2*pi*w1*j1) ...
+ cos(2*pi*w2*j1);
end
>> F = fft(f);
>>
>> kk = (0:127)/2;
>> figure(1),
>>
subplot(3,1,1)
>>
plot(jjs, f)
>>
subplot(3,1,2)
>>
plot(kk, real(F))
>>
subplot(3,1,3)
>>
plot(kk, imag(F))
F (k) e
2i
N ( j1)(k1)
(Eqn 34 , repeated)
k=1
2i
N (11)(k1)
2i (21)(k1)
eN
vk =
..
2i
e N (N1)(k1)
(33)
where f ( j) is in time-domain signal and F (k) is in frequency domain, and
DFT is the Discrete Fourier Transform.
1
N
Defining basis vectors
For the Discrete Fourier Transform (DFT), the DFT and Inverse DFT are
given by summations in the place of integrals:
DFT:
Section 4.2.0
ei k (11)
i (21)
e k
=
..
.
ei k (N1)
(35)
where
2 (k 1)
N
is the frequency corresponding to component F (k).
k =
(36)
Using the vk , the inverse DFT of Eqn (34) takes the form:
f=
1
(F (1) v1 + F (2) v2 + + F (N) vN )
N
(37)
T
where f = f (1) f (2) f (N)
is the time-domain signal, and
T
is the frequency-domain signal.
F = F (1) F (2) F (N)
Eqn (37) has the form of expanding f on a set of basis vectors.
The 1/N term is a normalization.
Figure 8: Signal and Discrete Fourier Transform.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 37
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 38
EE/ME 701: Advanced Linear Systems
Section 4.2.0
EE/ME 701: Advanced Linear Systems
Putting the basis vectors together in an array
V=
Putting together Eqns (39) and (41) gives
v1 v2 vN
(38)
f=
We find
1
VF
N
(where F is the Fourier transform of f).
f=
1
V VTf
N
(42)
where V V T is a projection operation, when the basis vectors are orthogonal.
(39)
Lets ample, with N = 6. Eqn (35) gives the code:
An important property of the Fourier basis functions is that they are
orthogonal . That is, for Eqns (35) and (38):
V VT = N I
Section 4.2.0
or, V T = N V 1
(40)
(The Fourier basis functions require term N to be normalized)
Looking at Eqn (33), and considering the vT is the complex conjugate
transpose of v, we find that the DFT is given by:
F =VTf
(41)
Summary:
Eqn (35) defines an orthogonal basis of N- element vectors (or functions,
for the continuous time FT)
>> w0 = 2*pi/N
>> for kk = 1:N,
>>
for jj = 1:N,
>>
V(jj,kk) =
>>
end
>> end
V =
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.50
-0.50
-1.00
-0.50
+
+
+
-
0.87i
0.87i
0.00i
0.87i
0.50 - 0.87i
%% Fundamental frequency
%% Build the N basis vectors
%% The N elements of each vec
exp(j*w0*(jj-1)(kk 1));
1.00
-0.50
-0.50
1.00
-0.50
+
+
0.87i
0.87i
0.00i
0.87i
-0.50 - 0.87i
1.00
-1.00
1.00
-1.00
1.00
+
+
-
0.00i
0.00i
0.00i
0.00i
-1.00 + 0.00i
1.00
-0.50
-0.50
1.00
-0.50
+
-
0.87i
0.87i
0.00i
0.87i
-0.50 + 0.87i
1.00
0.50
-0.50
-1.00
-0.50
+
+
0.87i
0.87i
0.00i
0.87i
0.50 + 0.87i
Because the Fourier basis functions are orthogonal, no matrix inverse
is required to compute the change of basis (Fourier would have had a
difficult time doing a large matrix inverse in 1805!)
The DFT, Eqn (41), is a change of basis, from the standard basis of f ( j)
to the Fourier basis F (k).
The IDFT is the change of basis back.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 39
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 40
EE/ME 701: Advanced Linear Systems
Now consider the FFT
>> f = [ 1 2 3 4 5 6]
f =
1
2
3
4
5
6
Section 4.2.0
Finally, V is a set of orthogonal basis vectors (and 1/N is required to
normalize)
6
0
0
0
0
0
Part 3: Linear Operators
0
6
0
0
0
0
Section 4.2.1
4.2.1 The Fourier transform as a change of basis
%% Using the standard FFT function
>> F1 = fft(f)
F1 = 21.0000
-3.0000 + 5.1962i
-3.0000 + 1.7321i
-3.0000
-3.0000 - 1.7321i
-3.0000 - 5.1962i
%% Using a matrix multiply, for a change of basis
>> F2 = V*f
%% The inverse Fourier transform
F2 = 21.0000
>> f3 = (1/N)* V * F2
-3.0000 + 5.1962i
-3.0000 + 1.7321i
f3 = 1
-3.0000 - 0.0000i
2
-3.0000 - 1.7321i
3
-3.0000 - 5.1962i
4
5
6
>> V*V
ans =
EE/ME 701: Advanced Linear Systems
0
0
6
0
0
0
0
0
0
6
0
0
0
0
0
0
6
0
1. The matrix multiply form only works for sampled signals (discrete
signals, or the discrete Fourier transform). Continuous signals require
integrals (and can be defined in terms of inner products).
2. Matrix multiplication is conceptually simple, but computationally
inefficient. For a 1024 element FFT, a 1024x1024 matrix V would be
required.
Fourier actually focused on the columns of V . Fouriers insight contained 2
parts:
1. He could solve the problem of heat conduction for an initial heat
distribution given by a basis vector vk
2. The basis vectors of the Fourier transform are orthogonal !
Ordinarily given basis vectors V and data f, to find the basis
coefficients we need to solve
1 T
F = V 1 f ,
or V T V
V f
But the Fourier basis vectors are orthogonal, with a very simple
normalizing factor,
F =VTf
0
0
0
0
0
6
(Revised: Sep 10, 2012)
If the Fourier Transform is just a matrix multiplication for a change of basis,
why are text book derivations based on summations and integrals:
so no matrix inversion is required !
Page 41
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 42
EE/ME 701: Advanced Linear Systems
Section 4.2.2
Section 4.3.1
4.3 Additional examples using change of basis
4.2.2 Using the Fourier transform
Suppose we had an operator FA that operates on frequency data, not on
time-domain data, and given f time-domain data.
1
y= V
N
A VT f
(43)
Where the logic is:
Convert f to frequency domain
(action of V T )
Apply the operator in frequency domain
(action of FA)
Convert the answer back to time domain
(action of V )
Now, using Fouriers technique, we can just make a time-domain operator
1
A= V
N
EE/ME 701: Advanced Linear Systems
A V
(44)
And so
y = sA f
(45)
4.3.1 Matching input and output data to discover an operator
Given the data
1
4
1
{x1 , x2 , x3} =
2 , 5 , 2
3
0
6
0
4
1
{y1 , y2 , y3} =
2 , 1 , 0
2
6
1
Find the operator A to solve
y = Ax
The key to finding a good basis for solving the problem is to realize that a
basis F is found so that the x data are
1 0 0
F
x1 , Fx2 , Fx3 = 0 , 1 , 0 , with
0
1
0
F
y1, Fy2 , Fy3
Then the operator would simply be:
F
A = Fy1 , Fy2 , Fy3
1
0
so that
F
y1 = FA Fx1 = FA 0 , Fy2 FA Fx2 = FA 1
0
0
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 43
Part 3: Linear Operators
(Revised: Sep 10, 2012)
etc.
Page 44
EE/ME 701: Advanced Linear Systems
Section 4.3.1
So the trick is to find a basis F on which
1
F
x1 = 0 ,
0
EE/ME 701: Advanced Linear Systems
Section 4.3.2
4.3.2 Operator from data, example
0
F
x2 = 1 , etc.
0
Setting the X and Y data
>> X = [1
2
3
The solution is to choose as basis vectors the xi vectors themselves. Then
F = [x1 , x2 , x3]
(46)
4 -1
5 2
6 0]
>> Y = [ 0
2
-2
Now setting up the change of bases
4 -1
1 0
6 1]
Double checking:
>> sFT = X;
which gives
With Eqn (46)
x1 = F Fx1 = [x1 , x2 , x3] 0 ,
0
s
FT
F
sT
=F,
>> FsT = inv(sFT)
FsT =
-0.8000
0.4000
-0.2000
etc.
= F 1
(47)
On the F basis:
F
y =
A x
with
A = Fy1 , Fy2 , Fy3 = FsT [ sy1, sy2 , sy3 ]
Now, to find the operator in standard coordinates, it is the usual equation:
s
A = sF T
F
sT
(48)
And the operator gives y s from x s in the standard coordinates:
-0.4000
0.2000
0.4000
0.8667
-0.2667
-0.2000
The operator on the F basis
>> FA = FsT * Y
FA =
-2.5333
0.9333
1.2000
1.6000
0.2000
-1.6000
1.6667
-0.6667
0
Now convert the operator to the standard basis
>> sA = sFT * FA * FsT
sA =
1.8000
0.4000
-1.2000
-0.6000
3.8000
2.4000
-0.8667
1.4667
-3.5333
>> sA * X(:,1)
ans =
0.0000
2.0000
-2.0000
>> sA * X(:,2)
ans =
4.0000
1.0000
6.0000
>> sA * X(:,3)
ans =
-1.0000
-0.0000
1.0000
y = sA sx
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 45
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 46
EE/ME 701: Advanced Linear Systems
Section 4.4.0
EE/ME 701: Advanced Linear Systems
Section 5.0.0
5 Operators as Spaces (Bay section 3.2)
4.4 Conclusions change of basis as a tool for analysis
Some problems are much more easily solved in a special coordinate frame
that is different from the natural (standard) coordinates of the problem.
In these cases the best way, and possibly the only way, to solve the problem
is to transform the data into the special coordinates, solve the problem, and
transform the answer back out again.
With linear transformations and operators, we can then build the operator
directly in standard coordinates with Eqn (22).
Since the time of Fourier, coordinate transformation is a standard tool, but
we might call it by other names, or never explicitly compute the matrices F
and F 1
Coordinate transformation is the only general way to solve the equation
The set of linear operators from one vector space into another (or into itself)
also forms a vector space.
Working with the space of all operators A : Rm Rn , and with
y1 = A1 x , y2 = A2 x
then
A1 x + A2x = (A1 + A2) x
and all the required properties of a vector space are satisfied
1. The 0 operator is included in the set
2. For every operator there is the additive inverse operator
x (t) = A x (t) + B u (t)
3. Closure under addition
4. Commutativity of addition
5. Associativity of addition
6. Closure under scalar multiplication
7. Associativity of scalar multiplication
8. Distributivity of scalar multiplication over operator addition
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 47
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 48
EE/ME 701: Advanced Linear Systems
Section 5.1.1
5.1 Operator Norms
EE/ME 701: Advanced Linear Systems
Section 5.2.1
5.2 Determining the value of Operator norms
Operators also have norms. Intuitively, the size of an operator relates to
how much it changes the size of a vector. Given:
Just as there are several vector norms, each induces an operator norm.
5.2.1 The L1 norm of an operator
y = Ax
with suitable norms ||y|| and ||x||. Then the operator norm ||A||op is defined
by:
||y||
(49)
= sup ||y||
||A||op = sup
x6=0 ||x||
||x||=1
where sup, supremum, indicates the least upper bound.
The operator norm is induced by the vector norms ||y|| and ||x||.
An operator matrix does not have to be square or full rank, it can be any
matrix, A Rnm
5.1.1 Operator norm properties
2. ||A1 + A2 ||op ||A1 ||op + ||A2 ||op
3. ||A1 A2 ||op ||A1 ||op ||A2 ||op
4. || A||op = || ||A||op
Or said another way, what is the maximum (L1 ) gain of operator A?
Example
1 4
y = Ax = 2 5 x,
3 6
what is the max. possible ||y||1 / ||x||1
For the moment lets consider x with ||x||1 = 1 . (Through linearity, ||y||1
just scales with ||x||1 ) .
Operator norms have these properties
1. ||A x|| ||A||op ||x||
The L1 norm of an operator answers this question: Given a vector x with
||x||1 , and y = A x , what is the largest possible value of ||y||1 ?
(from the definition of the operator norm)
(triangle inequality)
(Cauchy-Schwarz inequality)
It turns out that the choice for x that gives the largest possible 1-norm
for the output, is to put all of ||x||1 on the element corresponding to the
largest column vector A . (Largest in the L1 sense.)
Choose:
(scalar multiplication)
so
0
x=
1
then
||x||1 = 1 , ||y||1 = 15
||A||1op = 15
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 49
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 50
EE/ME 701: Advanced Linear Systems
Section 5.2.1
Example:
A=
EE/ME 701: Advanced Linear Systems
Section 5.2.2
5.2.2 The L2 -norm of an operator
1
4
,
a2 =
2
5
3 6
a1
y = Ax ,
The L2 -norm of an operator gives the maximum L2 length of an output
vector for a unit input vector.
so
0.5
for x =
,
0.5
0
for x = ,
1
||a2||1 = 15
||A||2 = sup
x6=0
1
y= 2 ,
3
1
for x = ,
0
||a1||1 = 6 ,
||y||1 = 6
||A||2 =
max
||x||2 =1
xT A A x
o1/2
yT y
(52)
Eqn (52) is just a re-statement of the definition of an operator norm, with
the ||y||2 expanded inside the brackets.
The L2 norm of a matrix is the largest singular value of the matrix
||A||2 = (A)
||A||1 = max a j 1
j
(Revised: Sep 10, 2012)
(53)
It is determined using the singular value decomposition (a topic of Bay
chapter 4). For Example:
>>[U,S,V] = svd(A)
U = -0.429
0.806
-0.566
0.112
-0.704
-0.581
V =
S =
0.408
-0.816
0.408
9.508
0
0
0
0.773
0
-0.386
-0.922
(50)
The L1 norm of matrix A is the largest L1 norm of any column vector of A.
Part 3: Linear Operators
Where the overbar indicates complex conjugate.
2.5
y = 3.5 , ||y||1 = 10.5
4.5
4
y = 5 , ||y||1 = 15
6
then
(51)
Bay defines the L2 norm of a matrix operator as:
Given
A = [a1 , a2 ... , am ]
||y||2
||A x||2
= sup
= sup ||A x||2
||x||2 x6=0 ||x||2
||x||2 =1
Page 51
>> x2 = [.386; .922]
;
y2 = A*x2 ,
y2 = [ 4.0740,
5.3820
,
6.6900]
>>norm(y2)/ norm(x2),
ans =
9.5080
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 52
-0.922
0.386
EE/ME 701: Advanced Linear Systems
Section 5.2.3
EE/ME 701: Advanced Linear Systems
5.2.3 The L -norm of an operator
5.2.4 The Frobenius norm
An alternative to the operator norms induced by the vector norms (the 1-, 2and - norms, above), one can define an norm directly on the elements of
the matrix. The entry-wise norms are given by:
The L -norm of an operator gives the maximum L length of an output for
a unit input .
||A|| = sup
x6=0
Section 5.2.4
||y||
||A x||
= sup
= sup ||A x||
||x|| x6=0 ||x||
||x|| =1
(54)
n
||A||opp =
Bay defines the L norm of a matrix operator as:
m
||A|| = max
i
j=1
ai j
i=1 j=1
p
ai j
!1
(56)
Note, care must be taken with notation, because the normal notion of the
operator norm is induced, as in section 5.2.2, not given by Eqn (56)
(55)
A special case of these so-call entry-wise norms is the Frobenius norm
n
The L -norm is given by the row vector of A with the greatest L1 -norm.
||A||F =
For the example matrix:
i=1 j=1
!1
2
2
ai j
(57)
The Frobenius norm has the properties a norm (described in section 5.1)
r
1 4
1
A = r2 = 2 5 ,
3 6
r3
r1 =
r2 =
r3 =
h
h
h
1 4
2 5
3 6
i
i
i
||r1 ||1 = 5
,
The Frobenius norm has the additional advantage of being easy to
compute
||r2 ||1 = 7
(Recall that finding ||A||2 requires solving the singular value
decomposition).
||r3 ||1 = 9
So ||A|| = 9
Additional relationships for the Frobenius norm are given by:
To see why ||A|| is given by the 1-norms of the rows
-norms of anything !) consider that:
(rather than the
||A||F =
0
1
1
1
1
= =
=
= = 1
1
1
1
1
0
One of the above vectors, multiplying A, gives the maximum -norm
output. And that max is the greatest 1-norm of a row.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
||A||F
Page 53
i1/2
h
T
tr A A
v
umin(n,m)
u
= t 2i (A)
(58)
(59)
i=1
where tr () denotes the trace of a matrix, which is the sum of the elements
on the diagonal; and the i in Eqn (59) are the singular values.
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 54
EE/ME 701: Advanced Linear Systems
Section 5.4.0
EE/ME 701: Advanced Linear Systems
Section 5.5.0
5.5 Adjoint Operators
See, for example
J-C Lo and M-L Lin, Robust H Control for Fuzzy Systems
with Frobenius Norm-Bounded Uncertainties, IEEE Transactions on
Fuzzy Systems 14(1):1-15.
5.3 Boundedness of an operator
Adjoint Operator: The adjoint of a linear operator A is denoted A and must
satisfy the relationship
hAx, yi = hx, A yi
for all x and y
An operator is said to be bounded if there exists a finite such that
||A x|| ||x||
The transpose is a special kind of operator, called an adjoint operator
Adjoint Operator is a general concept that applies to all types of vector
spaces (such as vectors of polynomials)
For finite vectors on an ortho-normal basis (e.g., x Rn),
which is equivalent to saying ||A|| .
A linear operator is just a matrix, A
5.4 Operator Norms, conclusions
All 4 matrix norm definitions satisfy properties 1-4 of operator norms
(section 5.1.1).
The adjoint of a real-valued operator is just the matrix transpose
A = AT
The adjoint
operator is just the complex-conjugate
of a complex-valued
T
transpose A = A
Example:
A=
Matlab:
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 55
4 j
2
5
3+ j 6
1
A =
2 3 j
4+ j 5 6
1
is A
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 56
EE/ME 701: Advanced Linear Systems
Section 6.0.0
When A = A
e.g., A =
3+ j 7
A is a symmetric matrix for the real case,
2
A is a Hermitian matrix for the complex case (Hermitian = complex
conjugate transpose, A in Matlab)
EE/ME 701: Advanced Linear Systems
Section 7.0.0
7 Forming the intersection of two vector spaces
Given vector space U, with basis vectors {u1 , u2 , ..., unu } and vector space
V, with basis vectors {v1 , v2 , ..., vnv } , where nu is the dimension of U and
nv is the dimension of V.
Sometimes it is interesting to find the intersection of the two vector spaces,
which is a vector space W given by
We can also say that A is a self-adjoint operator
Hermitian matrices have two important properties that we will use to
develop the singular value decomposition
W = UV
(60)
When U = [u1 , u2 , ..., unu ] and V = [v1 , v2 , ..., vnv ], for vectors lying in W,
it must be the case that there are representations on both U and V , that is
They always have a complete set of eigenvectors
w W,
The eigenvalues are always real (even if the matrix is complex)
w = a1 u1 + a2 u2 + + anu unu = b1 v1 + b2 v2 + + bnv vnv
(61)
Eqn (61) can be rewritten as Eqn (62).
6 Bay Section 3.3
A system of simultaneous linear equations can be written in the form
y = Ab
Which is to say, vectors made of the U
and V basis coefficients of vectors in the
intersection, must lie in the null space of
the matrix [U, V ].
Bay address the various possibilities of the equation y = A b .
We will come back to this topic after covering the singular value
decomposition
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 57
Part 3: Linear Operators
a
1
a2
.
..
an
u
[U, V ]
=0
b1
b
2
.
..
bnv
(Revised: Sep 10, 2012)
Page 58
(62)
EE/ME 701: Advanced Linear Systems
Section 7.1.0
The dimension of the intersection is the number of linearly independent
solutions to Eqn (62), which is the dimension of the null space of [U, V ]
Vectors
a
1
wi = U ...
anu
(63)
8 Conclusion
For finite vectors, linear operators are matrices
Rotation + Reflection matrices Ortho-normal matrix
Linear Operators have a number of characteristics
Are basis vectors for W.
They change when we make a change of basis
The form a vector space
7.1 Example
>> U = [
Section 8.0.0
The rotation matrix is an example operator
b
1
wi = V ...
bnv
or equivalently
EE/ME 701: Advanced Linear Systems
1
2
3
4
5
6 ]
>> Null = null([U, -V])
Null = 0.5000
0.5000
0.5000
0.5000
>> V = [ 2
3
5
The norm is well defined
3
4
4 ]
For any operator, there is an adjoint operator
>> w1 = U*Null(1:2)
w1 = 2.5000
3.5000
4.5000
Verify that w1 lies in U and V by checking the projection onto each.
(Notice the projection calculation for non-orthogonal basis set.)
>> U*inv(U*U)*U*w1
ans = 2.5000
3.5000
4.5000
Part 3: Linear Operators
>> V*inv(V*V)*V*w1
ans = 2.5000
3.5000
4.5000
(Revised: Sep 10, 2012)
Page 59
Part 3: Linear Operators
(Revised: Sep 10, 2012)
Page 60
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Singular Value Decomposition
6 Determining the rank of A and the four fundamental spaces with the
SVD
28
6.1
Contents
1 Introduction
Example: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
31
7 Exact, Homogeneous, Particular and General Solutions to a Linear
Equation
34
1.1
Rotation+ Matrices . . . . . . . . . . . . . . . . . . . . . . . . .
7.1
The four types of the solution for y = A x . . . . . . . . . . . . .
34
1.2
Scaling Matrices . . . . . . . . . . . . . . . . . . . . . . . . . .
7.2
Numerical example showing the generalized inverse
36
2 The Singular Value Decomposition
2.1
Numerical Example: . . . . . . . . . . . . . . . . . . . . . . . .
2.2
Forms for the SVD . . . . . . . . . . . . . . . . . . . . . . . . .
Demonstrating that A = rk=1 uk k vTk . . . . . . . . . . .
2.2.1
3 Proof of the SVD theorem (following Will)
Solving for the j and u j , given the v j . . . . . . . . . . . . . . .
12
3.2
Example, good choices for v j . . . . . . . . . . . . . . . . . . . .
13
3.3
Example, bad choices for v j . . . . . . . . . . . . . . . . . . . .
15
3.4
Choosing the v j correctly
. . . . . . . . . . . . . . . . . . . . .
16
3.5
Proof by construction of the SVD . . . . . . . . . . . . . . . . .
19
4 The generalized inverse of A, A#
40
9 History of the Singular Value Decomposition
42
10 Conclusions
43
20
Using generalized inverse of A . . . . . . . . . . . . . . . . . . .
23
4.1.1
24
A numerical example using the generalized inverse . . . .
5 SVD Conclusion
Part 4a: Singular Value Decomposition
8 Additional Properties of the singular values
12
3.1
4.1
. . . . . . .
27
(Revised: Sep 10, 2012)
Page 1
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
Section 1.0.0
1 Introduction
EE/ME 701: Advanced Linear Systems
Section 1.1.0
Gilbert Strang calls the SVD:
"Absolutely a high point of linear algebra."
An n m matrix maps a vector from Rm to Rn :
y = Ax ,
y Rn ,
x Rm ,
A Rnm
(1)
For a nice explanation of the SVD by Todd Will, see:
http://www.uwlax.edu/faculty/will/svd
Ax=y
y
1.1 Rotation+ Matrices
As we have seen, rotation+ matrices preserve lengths and angles.
Output Space, Rn
When lengths and angles are preserved, shapes are preserved.
Figure 1: Action of a matrix, A maps x onto y.
As we shall see, in all cases, the linear transformation from Input Space
to Output Space is made up of 3 parts:
1. An ortho-normal transformation from input coordinates to singular
coordinates
2. Scaling (in singular coordinates)
3. An ortho-normal transformation from singular to output coordinates
As we saw in the previous section, an orthonormal matrix gives either a
rotation (det R = +1) or reflection (det R = 1). Some authors use the term
rotation matrix for actions 1 and 3, because it captures the intuition of
what is going on, but rotation alone is not sufficiently complete.
The relation y = A x is the general linear operator from an Input Vector
Space to an Output Vector Space.
Input Space, Rm
1
1
1
X
1
1
1
X
Figure 2: Example: a 45o rotation.
All full-rank orthonormal matrices are rotation+ matrices.
We will use rotation+ to refer the orthonormal matrices.
If we can understand y = A x in terms of rotations+ and scaling, many
interesting results and practical methods will follow directly.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 3
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 1.2.0
1.2 Scaling Matrices
EE/ME 701: Advanced Linear Systems
A scaling value is also called a singular value, and can be zero:
Scaling matrices scale the Euclidean axes of a space. An example Scaling
matrix is:
1 0
0.5 0
=
(2)
=
0 2
0 3
15
10
10
Scaled by 3.0
5
5
5
5
5
X
10
15
1 0
0 2
15
15
10
10
(3)
Scaled by 0.0
Scaled by 0.5
0.5 0
15
Section 1.2.0
0
5
5
5
X
10
15
Scaled by 0.5
5
X
10
5
5
15
5
X
10
15
Figure 4: Action of a scaling matrix with a singular value of zero for the Y axis.
Figure 3: Action of a scaling matrix.
A Scaling matrix does not have to be square. The matrices
Figure 4 was generated with:
FF = [ A collection of 2-vectors (specifying points) ]
Scale = [ 0.5, 0; 0 3.0];
FF1 = Scale * FF;
0.5 0 0
0
3 0
0.5 0
= 0
Scaling Matrices:
1. Are diagonal
2. Have zero or positive scaling values
A negative value flips the configuration, well do this with rotations+.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 5
1 0 0
1 0
0 2 0
3 = 0 2
0 0
0
(4)
(5)
are also scaling matrices, in operators which map from R3 R2 or
R2 R3 , respectively
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 2.0.0
2 The Singular Value Decomposition
EE/ME 701: Advanced Linear Systems
Section 2.1.0
2.1 Numerical Example:
The SVD THEOREM:
Every matrix A can be decomposed into:
1. An rotation+ from input coordinates onto Singular Coordinates (the
coordinates in which the scaling occurs).
%% First SVD example
>> A = [ 2 3 ;
3 4
A =
2
3
3
4
4
-2
-2 ]
>> [U, S, V] = svd(A)
2. Scaling in singular coordinates
3. An rotation+ from singular coordinates onto output coordinates.
U =
-0.5661
-0.7926
-0.2265
-0.1622
-0.1622
0.9733
S =
6.2450
0
0
0
4.3589
0
Said another way:
A Rnm ,
A = U VT
-0.8082
0.5878
-0.0367
(6)
and so
y = Ax =U V x
T
(7)
V =
-0.7071
-0.7071
0.7071
-0.7071
where:
1. V T is an rotation+ from input coordinates onto singular coordinates.
2. is a scaling matrix.
3. U is an rotation+ from singular coordinates onto output coordinates.
%% A can be remade from U, S, V
>> A1 = U*S*V
A1 = 2.0000
3.0000
3.0000
4.0000
4.0000
-2.0000
Uniqueness:
Remarks:
The singular values are unique
The columns of U and V corresponding to distinct singular values are
unique (up to scaling by 1)
1. The columns of U form an ortho-normal base for Rn (the output space). U
is a rotation+ matrix.
The remaining columns of U and V must lie in specific vector subspaces,
as described below.
2. The columns of V form ortho-normal base for Rm (the input space). V is a
rotation+ matrix.
3. The scaling matrix is zero everywhere except the main diagonal.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 7
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 2.2.1
2.2 Forms for the SVD
EE/ME 701: Advanced Linear Systems
Section 2.2.1
With the SVD
The singular value decomposition can be written
A = U V
or
(8)
A=
uk k vTk
(9)
| | | |
0 0
T
A = U V = u1 u2 un 0 2 0
0 0 ...
| | | |
k=1
where uk are the columns of U, vk are the columns of V , and r is the rank of
A.
Eqn (8) is a decomposition into matrices, while Eqn (9) is a decomposition
into individual component vectors.
vT1
vT2
(12)
..
T
vm
Recall
p = min (n, m)
r = rank (A)
Because of the zeros in the scaling matrix, Eqn (12) reduces to one of
Eqn (13) or (14), below.
2.2.1 Demonstrating that A = rk=1 uk k vTk
The development of the proof of the SVD and generalized inverse rest on
the decomposition
When m n , A is given by Eqn (13), where the columns of U for
j = m + 1..n multiply zero rows in , and are omitted.
A=
uk k vTk
(10)
k=1
Which is the relationship that tells us the connection between a specific
vector vk in the input space and a specific vector uk in the output space.
A general proof of the singular value decomposition theorem follows in
section 3.
Here we show that,
Given that a decomposition exists with the form
A =U V
(11)
where U and V are ortho-normal matrices and is a scaling matrix,
Then a decomposition according to Eqn (9) follows from Eqn (8).
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 9
And when n < m, then A is given by Eqn (14), where the rows of V for
i = n + 1..m multiply zero columns in , and are omitted.
A =
A =
1 vT1
| |
|
T
2 v2
u1 u2 um
..
| |
|
m vTm
|
|
|
1 u1 2 u2 n un
|
|
|
Part 4a: Singular Value Decomposition
when p = m < n
,
keep only p columns of U (13)
vT1
vT2
when p = n < m
,
keep only p rows of V T (14)
..
vTn
(Revised: Sep 10, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 2.2.1
Eqns (13) and (14) both reduce to Eqn (15)
A = 1 u1 2 u2
|
|
Section 3.1.0
3 Proof of the SVD theorem (following Will)
vT1
vT
2
(15)
p up
..
|
vTp
|
EE/ME 701: Advanced Linear Systems
Going across the rows on the right, and down the columns on the left, the
elements of 1 u1 form a vector outer product with the elements of vT1 ,
2 u2 with vT2, etc.
The mechanics of multiplying the terms of Eqn (15) give the result
Following the demonstration given by Will (see: http://www.uwlax.edu/faculty/will/svd)
we first show that given the v j , the decomposition
r
A=
uk k vTk = U V T
(18)
k=1
can be derived.
Then we show how to determine the v j satisfying a necessary condition.
3.1 Solving for the j and u j , given the v j
A=
k uk vTk
(16)
k=1
If r < p then the k , k = {r + 1 , ... p} are zero, and those terms can be
dropped from Eqn (16), giving the sought after form:
r
A=
k uk vTk
(17)
k=1
Given Eqn (18) we find that
r
Avj =
uk k vTk
k=1
v j = u j j vTj v j = u j j
(19)
where all the terms k 6= j drop out because vTj v j = 0, k 6= j, and 1, k = j.
Thus, the u j and v j vectors and j must satisfy
Avj = uj j
(20)
Eqn (20) immediately solves for all of the j and vectors u j corresponding
to j > 0:
j = A v j
uj =
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 11
Part 4a: Singular Value Decomposition
(21)
1
Avj
j
(22)
(Revised: Sep 10, 2012)
Page 12
EE/ME 701: Advanced Linear Systems
Section 3.2.0
V =
0.39 0.92
0.92
0.39
Doing calculations of Eqns (21) and (22)
4.08
A v1 = 5.38
6.69
u1 =
1
1
9.51
0.43
1.00 4.00
0.39 0.92
=
2.00 5.00
0.77
0.92 0.39
0
3.00 6.00
(23)
For U to be an ortho-normal matrix, u1 and u2 must be orthogonal and
normal.
They are automatically normalized by the calculation of Eqn (22).
Check that they are orthogonal:
>> u1 = U(:,1)
u1 = -0.4287
-0.5663
-0.7039
>> u2=U(:,2)
u2 = 0.8060
0.1124
-0.5812
>> u1*u2
ans =
1.1102e-16
5.38 = 0.57
6.69
0.70
Repeating steps (21) and (22) for v2 gives:
9.51 0
= 0 0.77 ,
0
0
0.43 0.81 |
U = 0.57 0.11 ?
0.70 0.58 |
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
0.81
Remarks:
4.08
1 = 5.38 = 9.51
6.69
4.08
0.43
A = 0.57 0.11 ? 0
0.70 0.58 |
0
1 4
A= 2 5
3 6
Choose:
Section 3.2.0
Then A = U V T gives:
3.2 Example, good choices for v j
Consider
EE/ME 701: Advanced Linear Systems
Our algorithm does not tell us how to choose u j if j is zero or non-existent.
But either way, it doesnt matter for Eqn (23).
Since U is an ortho-normal matrix, the missing columns must be an
orthonormal basis for the orthogonal complement of the u j , j = 1..r .
Page 13
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 3.3.0
3.3 Example, bad choices for v j
EE/ME 701: Advanced Linear Systems
For U to be an ortho-normal matrix, its columns must be orthonormal. The
columns are normal automatically, by step (22), but orthogonal ?
What happens if we dont choose the correct ortho-normal matrix V ?
0.49 +
* 0.36
0.55 , 0.57 = 0.99 6= 0
0.66
0.75
For example, choose the V corresponding to a rotation of 20o :
V =
0.94 0.34
0.34
0.94
No ! if V is not correctly chosen, U does not turn out to be an ortho-normal
matrix !
(yet to be proven).
Doing calculations of Eqns (21) and (22)
2.31
2.31
1 = 3.59 = 6.48
4.87
A v1 = 3.59
4.87
2.31
0.36
3.4 Choosing the v j correctly
For finding the j and u j according to Eqns (21) and (22), the challenge is
to find v j which are orthonormal, such that u j are orthogonal.
This requires:
u1 =
3.59 = 0.55
1
4.87
0.75
For j 6= i ,
6.48 0
= 0 7.00 ,
0
0
Part 4a: Singular Value Decomposition
(24)
When j is zero or non-existent we are free to select u j in the orthogonal
complement of {ui } (the left-null space of A), so we are interested in the
case j , i > 0 .
In this case 1/ j (1/i ) 6= 0 , so Eqn (24) is verified if and only if
0.36 0.49 |
U = 0.55 0.57 ?
0.75 0.66 |
(Revised: Sep 10, 2012)
uTj ui = 0, and so
1 1
uTj ui = vTj AT
(A vi ) = 0
j i
Repeating steps (21) and (22) for v2 gives:
Section 3.4.0
vTj AT (A vi ) = vTj AT A vi = 0
Page 15
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
(25)
Page 16
EE/ME 701: Advanced Linear Systems
Section 3.4.0
Lemma 1: When the vectors vi are ortho-normal and span Rm , then
vTj AT A vi = 0 j 6= i if and only if
AT A vi = i vi
(26)
EE/ME 701: Advanced Linear Systems
Section 3.4.0
In the general case of an m m matrix, eigenvalues and eigenvectors can be
a bit messy:
They can be complex.
The eigenvectors will not, in general, be orthogonal, and
in which case vTj AT A vi = i vTj vi = 0
There may not even be a complete set of eigenvectors (in which case we
say the matrix is defective)
Proof:
Necessary: Because the vi are ortho-normal, vectors v j , j 6= i span the
entire m 1 dimensional space orthogonal to vi , therefore if AT A vi must
lie in the 1-D space spanned by vi .
Sufficient: Since
v j vi .
AT
A vi = i vi , then
vTj AT
A vi =
vTj i vi
= 0, because
QED
The solutions to Eqn (26) are the eigenvalues and eigenvectors of
Q = AT A.
Square matrices have special vectors which give a scaled version of
themselves back under multiplication. For example:
1
1
2
2
Lemma 2: When Q is Hermitian (that is, when Q = Q , Q is the complex
conjugate), then
1. There is a complete set of eigenvectors, and furthermore, they are
orthogonal (ortho-normal, when we normalize them), and
2. The eigenvectors and eigenvalues are real (even if Q is complex).
Quick review of eigenvalues and eigenvectors:
However when Q is a Hermitian matrix (as Q = AT A must be) we are in
luck.
= 2
1
1
(27)
Remarks:
1. A symmetric matrix, such as Q = AT A , is a real Hermitian matrix.
2. Further discussion is deferred until our in-depth discussion the eigen-system
of a matrix.
These special vectors are called eigenvectors and the scale factors are
called the eigenvalues.
For an m m matrix, there can be at most m eigenvectors.
Eigenvectors can be scaled (considering multiplying left and right of Eqn
(26) by a scale factor). We usually normalize eigenvectors.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 17
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 3.5.0
EE/ME 701: Advanced Linear Systems
4 The generalized inverse of A, A#
3.5 Proof by construction of the SVD
With the above lemmas taken together, for any matrix A
Given the equation
y = Ax
AT A is Hermitian (symmetric if A is real)
Lemma 2 shows that AT A has a complete set of real, orthogonal
eigenvectors.
Lemma 1 shows that these eigenvectors satisfy the condition to be the
input basis vectors the SVD, the v j in
r
A=
Section 4.0.0
With A and y given. We are looking for a particular solution that solves
y = A x p
where
y is the projection of y onto the column space of A
uk k vTk
= U V
(28)
x p lies in the row space of A
k=1
Once the v j are known, determine the j u j corresponding to j 6= 0
according to
j = A v j
uj
1
=
Avj
j
(29)
(30)
The remaining u j are determined as basis vectors for the orthogonal
complement of the u j corresponding to j 6= 0.
Following these steps, the singular value decomposition of A is constructed.
QED
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 19
Whenever we have an ortho-normal basis for the column space of A
i
h
Uc = u1 u2 ur
(31)
We may find y by projection:
y = Uc UcT y =
uk uTk y
(32)
k=1
Writing Eqn (32) out graphically shows
uT
| |
|
| |
|
uT
2
y=
y = u1 u2 ur
u1 u2 ur
..
| |
|
| |
|
uTr
a
1
| |
|
a2
= u1 u2 ur
. = a1 u1 + a2 u2 + + ar ur
..
| |
|
ar
Part 4a: Singular Value Decomposition
uT1 y
uT2 y
...
uTr y
(Revised: Sep 10, 2012)
(33)
Page 20
EE/ME 701: Advanced Linear Systems
Section 4.0.0
Eqn (33) shows that the projection of y onto the column space of A can be
written
y = a1 u1 + a2 u2 + + ar ur
(34)
EE/ME 701: Advanced Linear Systems
Section 4.0.0
When we replace x in Eqn (38) with the expanded x of (39), we get
ak = k vTk x = k vTk (b1 v1 + b2 v2 + + br vr )
(40)
with
ak = uTk y
(35)
So far there is nothing surprising about Eqns (33-35). They say that we
can write the output of
y = A x p
(36)
as a linear combination of basis vectors on the column space of A.
But the basis vectors vk are ortho-normal, so
ak = k vTk (b1 v1 + b2 v2 + + br vr ) = k vTk vk bk = k bk
Putting the pieces together
We can write Eqns (33-35) whenever we have Uc , an ortho-normal basis
for the column space of A .
Using Eqns (35) and (41) gives bk , the basis coefficients of x p
Without the SVD, the analysis would stop here, there is no way to
discover contributions of x to give the basis coefficients ak .
With the SVD there is a way to discover the contributions of x to each term !
bk = (1/k ) ak = (1/k ) uk T y
uk k vTk
y = Ax =
k=1
xp =
x
(42)
Plugging the bk into (39) to find x gives
We know each term in the output is given by:
!
(41)
k=1
k=1
vk bk = vk (1/k ) uk
y=
vk (1/k ) uk
k1
y = A# y
(37)
= a1 u1 + a2 u2 + + ar ur
(43)
Where the term in parentheses gives the generalized inverse.
with
ak = k vTk x
(38)
A# =
vk (1/k) uk T
(44)
k=1
Any vector x in the row space of A can be written
Or in Matlab:
x = b1 v1 + b2 v2 + + br vr
(39)
>> Asharp = pinv(A)
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 21
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 22
EE/ME 701: Advanced Linear Systems
Section 4.1.0
4.1 Using generalized inverse of A
EE/ME 701: Advanced Linear Systems
Section 4.1.1
4.1.1 A numerical example using the generalized inverse
Consider the data
Given the relationship
y = Ax
(45)
With data vector y, find
A# =
vk (1/k) uk T
(46)
A = [ 1
2
3
4
5
6
7
8
6
8
10
12 ]
ybar =
6
5
5
3
k=1
With y = A x, find x p
And the solution
x p = A#
(47)
lies in the row space of A, and gives
y = A x p
(48)
Before proceeding, lets find the correct y , the projection of y onto col (A)
>> W = GramSchmidt(A)
W = 0.1826
0.8165
0.3651
0.4082
0.5477
0.0000
0.7303
-0.4082
>> yhat = W*W*ybar
yhat =
6.1000
5.2000
4.3000
3.4000
With y the projection of y onto the column space of A .
Trying the left pseudo-inverse:
In Eqn (46) there is no matrix inverse step.
>> xp1 = inv(A*A) * A * ybar
Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 8.606380e-18.
xp1 = -2.8750
0.6875
-1.0000
Eqn (46) always works, for every shape or rank of matrix.
Eqn (46) gives a new degree of freedom: we can choose r !
(More on this at the end of the notes on SVD)
Double checking that Ax p1 gives y
A mile off from y !
>> A*xp1,
ans = -5.4375
-9.6250
-13.8125
-18.0000
(The warning is to be taken seriously !)
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 23
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 24
EE/ME 701: Advanced Linear Systems
Section 4.1.1
Now with the generalized inverse (also called the pseudo inverse)
>> [U,S,V] = svd(A)
U = -0.3341
0.7671
-0.4359
0.3316
-0.5378
-0.1039
-0.6396
-0.5393
S =
V =
-0.4001
0.2546
0.6910
-0.5455
23.3718
0
0
0
0
1.3257
0
0
0
0
0.0000
0
-0.2301
-0.5634
-0.7935
-0.7834
0.5910
-0.1924
-0.5774
-0.5774
0.5774
Section 4.1.1
or
>> xp = Asharp * ybar
xp =
-2.3500
2.0500
-0.3000
-0.3741
0.7970
-0.4717
0.0488
>> xp=pinv(A)*ybar
xp = -2.3500
2.0500
-0.3000
Double checking that A x p1 gives y
Previously we saw three cases for the equation
y = Ax ,
A Rnm
Case
Size of A
Exactly constrained
n=m
Matrix Inverse
Tool
Over constrained
n>m
Left Pseudo-Inverse
Under constrained
n<m
Right Pseudo-Inverse
(49)
Must Exist
Table 1: Three cases for the solution of Y = A x
A1
1
ATA
1
A AT
Each of these cases requires that a matrix be invertible, or other methods
are required. With the generalized inverse
There are 2 non-zero s , using Eqn (46)
>> Asharp = V(:,1)*(1/S(1,1))*U(:,1) + V(:,2)*(1/S(2,2))*U(:,2)
Asharp = -0.4500
-0.1917
0.0667
0.3250
0.3500
0.1583
-0.0333
-0.2250
-0.1000
-0.0333
0.0333
0.1000
Eqn (46) is implemented by the pinv() command in Matlab
>> Asharp = pinv(A)
Asharp = -0.4500
0.3500
-0.1000
EE/ME 701: Advanced Linear Systems
xp =
k=1
k=1
vk k = vk (1/k) uTk y = A#x p
with
A# =
(50)
vk (1/k ) uTk
k=1
Equation (50), is valid in all cases where A 6= 0.
-0.1917
0.1583
-0.0333
0.0667
-0.0333
0.0333
0.3250
-0.2250
0.1000
There is no matrix inversion.
At most the basis vectors corresponding to k > tol are retained (we have
the freedom to choose r < rank (A)).
If A 6= 0, at least one k is guaranteed to be greater than zero.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 25
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 26
EE/ME 701: Advanced Linear Systems
Section 5.0.0
5 SVD Conclusion
:
U:
Section 6.0.0
6 Determining the rank of A and the four funda-
The singular value decomposition gives us an expansion for any real matrix
A of the form
A = U V T
V T:
EE/ME 701: Advanced Linear Systems
rotation+
from input
i coordinates to singular coordinates, where
h
V = v1 vm Rmm
mental spaces with the SVD
The rank of a matrix is the number of singular values which are greater than
a tolerance.
Recall, singular values are always positive or zero.
Scaling in singular coordinates, where Rnm (same as A)
In any numerical calculations we have round-off error, j may be greater
than zero due to round-off error.
rotation+ from
singular icoordinates to output coordinates,
h
where U = u1 un Rnn
So we set a tolerance for the minimum output of Y = A b which will be
considered non-zero.
Because the columns of U and V are orthonormal, we have a second
expansion for A:
r
A=
uk k vTk
Matlabs default value is given by:
tol = max(size(A)) * norm(A) * eps
>> eps = 2.2204e-16 (for 64 bit floating point calculation)
k=1
We have seen that if the vi are the eigenvectors of AT A then the k and uk
are straight-forward to compute.
Since Q = AT A is a Hermitian matrix, the eigenvectors exist and are
orthogonal.
Note: if n < m it may be numerically more convenient to compute the
eigenvectors of Q = A AT , which is a smaller matrix.
Student exercise: how would the SVD be determined from the eigenvec
tors of Q ?
The expansion of A is given as:
..
.
..
.
..
.
..
.
..
.
..
.
U V T = u1 ur ur+1 un
..
..
..
...
...
...
.
.
.
1
..
.
r
vT1
..
.
0
vT
vT
0
r+1
..
..
.
.
vTm
0
where 1 ... r are the singular values greater than the tolerance, and the
rest of the diagonal elements of are effectively zero.
As we are about to see, bases for the four fundamental spaces of a matrix
are given directly from the singular value decomposition.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 27
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 28
EE/ME 701: Advanced Linear Systems
Section 6.0.0
U V T partitions according to the non-zero singular values:
.. .. ..
. . .
u1 ur
.. .. ..
. . .
.. ..
..
. .
.
ur+1 un
..
.. ..
.
. .
0
...
r
vT1
..
vr
vTr+1
..
T
vm
Section 6.0.0
A program to give the four fundamental spaces
EE/ME 701: Advanced Linear Systems
0
...
function [Row, Col, Null, Lnull] = spaces(M)
%
function [Row, Col, Null, Lnull] = spaces(M)
%
%
Return the four fundamental spaces
%
[n,m] = size(M);
[u,s,v] = svd(M);
r = rank(M);
%% select spaces
Row=v(:,1:r);
Null = v(:,(r+1):m);
Col=u(:,1:r);
Lnull = u(:,(r+1):n);
where the columns of Uand V provide bases for the Column, Left-Null,
Row and Null spaces:
..
.
..
.
..
.
Uc = u1 ur ,
.. .. ..
. . .
..
.
..
.
..
.
Uln = ur+1 un ,
..
.. ..
.
. .
Part 4a: Singular Value Decomposition
..
.
..
.
..
.
..
.
Vr = v1 vr
.. .. ..
. . .
..
.
..
.
Vn = vr+1 vm
..
..
..
.
.
.
(Revised: Sep 10, 2012)
Page 29
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 30
EE/ME 701: Advanced Linear Systems
Section 6.1.0
6.1 Example:
EE/ME 701: Advanced Linear Systems
Section 6.1.0
We need to take a look at the third singular value to see if it is approximately
zero, or just quite small.
Consider
2
A=
0
0
1
1
>> ss = diag(S); ss(3)
ans =
4.2999e-16
(51)
3 is effectively zero (zero to within the round off error), so A is rank 2.
Now form the 4 fundamental spaces.
The four fundamental spaces of matrix A can be found through the SVD:
>> r = sum(ss > norm(A)*max(n,m)*eps)
r = 2
>> [U, S, V] = svd(A)
U =
-0.2543
0.3423
-0.5086
0.6846
-0.6947
-0.2506
-0.4405
-0.5928
S =
0.8987
-0.3615
-0.1757
0.1757
5.3718
0
0
0
0
1.0694
0
0
0
0
0.0000
0
V = -0.5774
-0.2113
-0.7887
0.5774
-0.7887
-0.2113
0.5774
0.5774
-0.5774
-0.1030
0.3769
-0.6509
0.6509
>> [n, m] = size(A)
n =
4 ,
m =
3
Since the rank is 2, the a basis for the row space is given by the first 2
columns of V
>> Vr
Vr =
The null space is spanned by the remaining columns of V
>> Vn
Vn =
Part 4a: Singular Value Decomposition
= V(:, 1:r)
-0.5774
0.5774
-0.2113
-0.7887
-0.7887
-0.2113
(Revised: Sep 10, 2012)
Page 31
= V(:, (r+1):NRows)
0.5774
0.5774
-0.5774
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 32
EE/ME 701: Advanced Linear Systems
Section 6.1.0
Since the rank is 2, a basis for the column space is given by the first 2
columns of U
>> Uc
= U(:, 1:r)
Uc = -0.2543
0.3423
-0.5086
0.6846
-0.6947
-0.2506
-0.4405
-0.5928
EE/ME 701: Advanced Linear Systems
Section 7.1.0
7 Exact, Homogeneous, Particular and General
Solutions to a Linear Equation
7.1 The four types of the solution for y = A x
There are four elements in the solution for
y = Ax
The left null space is spanned by the remaining columns of U
(52)
that we need to consider. The terminology is similar to that for differential
equations:
>> Un = U(:, (r+1):NCols)
Un =
0.8987
-0.1030
-0.3615
0.3769
-0.1757
-0.6509
0.1757
0.6509
Exact Solution: An exact solution is one that exactly satisfies Eqn (52). Recall
that if x is over-constrained (more ys than xs), then we can get a solution
which is not an exact solution.
Homogeneous Solution: A homogeneous solution is a solution to:
A xh = 0
There is the trivial homogeneous solution, x = 0. To have more interesting
homogeneous solutions, matrix A must nhave a non-trivial onull space.
Suppose the null space is given by a basis n1 , n2 , , nn :
xh = n1 n1 + n2 n2 + + nn nn
A xh = 0
(53)
If there is any non-trivial homogeneous solution, there will be infinitely
many of them.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 33
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 34
EE/ME 701: Advanced Linear Systems
Section 7.1.0
Particular Solution: A particular solution is one that solves Eqn (52), either
exactly or in a least squared error sense. x p is the particular solution, and
y = A x p. The residual,
e
(54)
y = y y
may be zero (an exact solution) or may be minimal.
Each of the terms relating to the particular solution comes from a
fundamental space:
y lies in the column space.
x p lies in the row space.
Recall the several different norms, minimizing the residual for each
different norm can give a different meaning to x p. One can assume that
is minimized in the least-squares sense, unless a different norm is given.
When working with the 2-norm (the usual case),
EE/ME 701: Advanced Linear Systems
Section 7.2.0
7.2 Numerical example showing the generalized inverse
Continuing with the example in section 6.1,
>> Uc = U(:, 1:r)
Uc = [ -0.2543
-0.5086
-0.6947
-0.4405
>> Vr = V(:, 1:r)
Vr = [ -0.5774
-0.2113
-0.7887
'
Keep in mind:
0.3423
0.6846
-0.2506
-0.5928 ]
A = Uc
Even though A R43 , r is
a 2x2 matrix, and Uc and Vr
each have 2 columns.
0.5774
-0.7887
-0.2113 ]
>> SigmaR = Sigma(1:r, 1:r)
SigmaR = [
5.3718
0
0
1.0694
r VrT
2 is the rank of matrix A .
&
y is given as the projection of y onto the column space.
Choose an example y , project y onto the column space.
e
y lies in the left-null space.
>> Y = [ 1 2 3 4 ]
x p is given as
xp = A y
(55)
General Solution: The general solution is a parametrized solution, it is the set of
all solutions which give y :
xg = x p + xh
y = A xg
(56)
>> Yhat = Uc * Uc * Y
Yhat = [ 0.8182
1.6364
3.9091
3.0909 ]
If the null space and any particular solution is known, then all of the general
solutions are given by Eqn (56).
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 35
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 36
EE/ME 701: Advanced Linear Systems
Section 7.2.0
EE/ME 701: Advanced Linear Systems
Section 7.2.0
Lets take a look at the residual e
y = y y
Now compute x p
>> Xp = Vr* inv(SigmaR) * Uc * Y
Xp = [ -0.2121
1.2424
1.0303 ]
ytilde =
0.1818
0.3636
-0.9091
0.9091
Double check that A x p gives y
>> A * Xp
ans = [ 0.8182
1.6364
3.9091
3.0909
e
y should lie in the left-null space. The basis for the left null space is Un
>> Un = U(:, (r+1):NCols)
Un = [ 0.8987
-0.1030
-0.3615
0.3769
-0.1757
-0.6509
0.1757
0.6509
The example y = A x looks like an over-constrained problem (4 data, 3
unknowns), try matrix pseudo-inverse:
>> XX = inv(A*A) * A * Y
Warning: Matrix is close to singular or badly scaled.
Results may be inaccurate. RCOND = 5.139921e-18.
XX = [ -6.2500
1.5000
4.2500 ]
Checking that e
y lies in Un :
>> ytilde2 =
0.1818
0.3636
-0.9091
0.9091
Un * Un * ytilde
Check result (the warning from inv() is to be taken seriously!)
A * XX
---------2.0000
-4.0000
1.7500
3.7500
A * Xp
-------0.8182
1.6364
3.9091
3.0909
Part 4a: Singular Value Decomposition
Yp
-----0.8182
1.6364
3.9091
3.0909
Y
----1
2
3
4
(Revised: Sep 10, 2012)
Page 37
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 38
EE/ME 701: Advanced Linear Systems
Section 7.2.0
EE/ME 701: Advanced Linear Systems
Section 8.0.0
8 Additional Properties of the singular values
The general solution to y = A x is given as:
Define is the largest singular value of A and is the smallest singular
value of A. Also, order the singular values so that 1 = , ... , p = .
xg = x p + xh
In some cases the null space will be empty, so xg = x p . But in this case
dim null (A) = 1
.577
xh = a1 vm = a1 .577
.577
||A||2 =
By writing y = rk=1 uk k vTk x , it is clear that the largest ||y||2 is given
when x corresponds to the largest singular value of A. Choosing x = v1
gives y = 1 u1 and ||y||2 = 1 .
The condition number of A is given as
So xg is given as:
cond (A) =
0.2121
.577
xg = 1.2424 + a1 .577
1.0303
.577
Consider that when we form A# we use 1/k , if there is a k which is very
small, there will be terms in A# which are very large. With
x = A# y
0.2121
.577
xg = 1.2424 + 0.42 .577
1.0303
.577
>> Xg = Xp - 0.42*Xh
Xg = [ -0.455
1.000
1.273 ]
(57)
The condition number is a measure of how much round off errors can be
expanded by multiplying a matrix by A.
Coefficient a1 can be used to satisfy one constraint, such as make
x (2) = 1.0 . Which gives a1 = .2424/.577 = 0.42
if there are noise components in y, when they hit the large terms in A#
they will give large errors.
Example: A problem with cond (A) = 10 is said to be well conditioned,
if cond (A) = 10000 the problem is said to be poorly conditioned.
>> A * Xg = Yhat
= 0.8182
1.6364
3.9091
3.0909
If = 0, the condition number is unbounded.
A method to handle poorly conditioned problems:
q
use A# = k=1vk (1/k ) uTk with some q < r
That is, throw out the smallest singular values.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 39
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 40
EE/ME 701: Advanced Linear Systems
Section 8.0.0
The absolute value of the determinant of any matrix is the product of its
singular values
p
|det (A)| = k
(58)
k=1
Corollary: if any k = 0, then det (A) = 0.
Recall that when A Rnn
n
det (A) = k
k=1
So for square matrices,
n
p
k = k
k=1 k=1
EE/ME 701: Advanced Linear Systems
Section 10.0.0
9 History of the Singular Value Decomposition
The singular value decomposition was originally developed by differential
geometers, who wished to determine whether a real bilinear form could be made
equal to another by independent orthogonal transformations of the two spaces
it acts on. Eugenio Beltrami and Camille Jordan discovered independently,
in 1873 and 1874 respectively, that the singular values of the bilinear forms,
represented as a matrix, form a complete set of invariants for bilinear forms
under orthogonal substitutions. James Joseph Sylvester also arrived at the singular
value decomposition for real square matrices in 1889, apparently independent
of both Beltrami and Jordan. Sylvester called the singular values the canonical
multipliers of the matrix A. The fourth mathematician to discover the singular
value decomposition independently is Autonne in 1915, who arrived at it via
the polar decomposition. The first proof of the singular value decomposition
for rectangular and complex matrices seems to be by Eckart and Young in 1936;
they saw it as a generalization of the principal axis transformation for Hermitian
matrices.
In 1907, Erhard Schmidt defined an analog of singular values for integral operators
(which are compact, under some weak technical assumptions); it seems he was
unaware of the parallel work on singular values of finite matrices. This theory was
k
further developed by mile Picard in 1910, who is the first to call the numbers sv
singular values (or rather, valeurs singulires).
Practical methods for computing the SVD date back to Kogbetliantz in 1954, 1955
and Hestenes in 1958 resembling closely the Jacobi eigenvalue algorithm, which
uses plane rotations or Givens rotations. However, these were replaced by the
method of Gene Golub and William Kahan published in 1965 (Golub & Kahan
1965), which uses Householder transformations or reflections. In 1970, Golub
and Christian Reinsch published a variant of the Golub/Kahan algorithm that is
still the one most-used today.
[Wikipedia]
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 41
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 42
EE/ME 701: Advanced Linear Systems
Section 10.0.0
10 Conclusions
The Singular Value Decomposition gives us a geometric picture of matrixvector multiplication, comprised of
A rotation+
Scaling
A rotation+
Using the SVD we can find basis vectors for the four fundamental spaces.
And the basis sets are ortho-normal
And the basis vectors of the row and column spaces are linked by the
singular values
Numerically, computation of the SVD is robust because computation of the
eigenvectors of a symmetric matrix is robust.
The SVD can be used to compute:
rank (A)
||A||2
cond (A)
Ortho-normal basis vectors for the four fundamental spaces
The solution to y = A x for the general case, without matrix inversion.
Part 4a: Singular Value Decomposition
(Revised: Sep 10, 2012)
Page 43
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Eigenvalues, Eigenvectors and the Jordan
Form
3.2.4
3.3
The Jordan form, a second example . . . . . . . . . . . .
27
One more twist, freedom to choose the regular eigenvector . . . .
33
3.3.1
Contents
3.4
1 Introduction
1.1
Review of basic facts about eigenvectors and eigenvalues . . . . .
1.1.1
Repeated eigenvalues . . . . . . . . . . . . . . . . . . . .
Analysis of the structure of the eigensystem of a matrix . . . . . .
2 Properties of the Eigensystem
Bay section 4.1, A-Invariant Subspaces . . . . . . . . . . . . . . .
2.2
Finding eigenvalues and eigenvectors . . . . . . . . . . . . . . .
2.3
Interpreting complex eigenvalues / eigenvectors . . . . . . . . . .
10
2.3.1
Example: 3D Rotation . . . . . . . . . . . . . . . . . . .
11
The eigen-system of symmetric (Hermitian) matrices . . . . . . .
13
3 The Jordan-form
3.1
3.2
41
Why Matlab does not have a numeric Jordan command . .
42
4 Conclusions
43
5 Review questions and skills
45
16
How the Jordan form relates to the real world . . . . . . . . . . .
16
3.1.1
An example calling for the Jordan form . . . . . . . . . .
17
Constructing the Jordan Form . . . . . . . . . . . . . . . . . . .
20
3.2.1
Regular and Generalized Eigenvectors
. . . . . . . . . .
20
3.2.2
First Jordan Form Example . . . . . . . . . . . . . . . . .
22
3.2.3
More on Jordan blocks . . . . . . . . . . . . . . . . . . .
26
Part 4b: Eigenvalues and Eigenvectors
3.4.1
. . . . . . . . . . . . . . . . . . .
2.1
2.4
Summary of the Jordan Form
35
Looking at eigenvalues and eigenvectors in relation to the
null space of ( I A) . . . . . . . . . . . . . . . . . . .
1.1.2
1.2
Example where regular E-vecs do not lie in the column
space of (A k I) . . . . . . . . . . . . . . . . . . . . .
(Revised: Sep 10, 2012)
Page 1
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
Section 1.1.0
1 Introduction
EE/ME 701: Advanced Linear Systems
Section 1.1.2
1.1.1 Looking at eigenvalues and eigenvectors in relation to the null space
of ( I A)
We have seen the basic case of eigenvalues and eigenvectors
Starting from
A vk = k vk .
(k I A) vk = 0
(1)
In this chapter we will elaborate the relevant concepts to handle every case.
(4)
The eigen-values are values of k such that (k I A) has a non-trivial null
space.
The eigenvectors are the basis vectors of the null space !
1.1 Review of basic facts about eigenvectors and eigenvalues
Since
Only square matrices have eigensystems
det (k I A) = 0
Every n n matrix has n eigenvalues, 1 ... n
We know the null space is at least 1 dimensional.
The eigenvector satisfies the relationship A vk = k v , which leads to the
eigenvector being the solution to
(A k I) vk = 0
(2)
Theorem: for each distinct eigenvalue, there is at least one independent
eigenvector.
Proof: The proof follows directly from Eqns (4) and (5).
or, said another way, the eigenvector is a vector in the null space of the
matrix (k I A) .
1.1.2 Repeated eigenvalues
Notes:
Example:
1. Any vector in the null space of (k I A) is an eigenvector. If the null
space is 2 dimensional, then any vec. in this 2D subspace is an E-vec.
2. Since the determinant of any matrix with a non-empty null space is
zero, we have:
(3)
det (k I A) = 0 , k = 1..n
which gives the characteristic equation of matrix A.
Part 4b: Eigenvalues and Eigenvectors
(5)
(Revised: Sep 10, 2012)
Page 3
Matrices can have repeated eigenvalues.
>> A = [ 2 1;
0 2]
>> [V,U] = eig(A)
V =
1.0000
-1.0000
0
0.0000
Part 4b: Eigenvalues and Eigenvectors
A =
2
0
U = 2
0
1
2
0
2
(Revised: Sep 10, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 1.1.2
EE/ME 701: Advanced Linear Systems
Section 1.2.0
1.2 Analysis of the structure of the eigensystem of a matrix
When there are repeated eigenvalues:
1. We are assured to have at least 1 independent eigenvector.
Analysis of the eigensystem of a matrix proceeds by completing table 1.
2. There may fewer independent eigenvectors than eigenvalues
1. Group the into k sets of repeated eigenvalues (one set for each unique
)
Definitions:
The algebraic multiplicity of an eigenvalue is the number of times the
eigenvalue is repeated.
The number of k in the kth set is called the algebraic multiplicity,
and is given by mk
The geometric multiplicity is the number of independent eigenvectors
corresponding to the eigenvalue. (dim null (A I))
Since (k I A) always has a non-trivial null space, every k set of
eigenvalues has at least one eigenvector vk ,
Consider the example above:
>> A = [ 2 1;
0 2]
A =
2
0
2. Determine the number of independent eigenvectors corresponding to k
by evaluating
q (k I A) = dim null (k I A) .
1
2
This is called the geometric multiplicity, and is given by gk .
>> [V,U] = eig(A)
V =
1.0000
-1.0000
0
0.0000
U = 2.0000
0
If mk 2, it is possible that there are fewer independent eigenvectors
than eigenvalues.
0
2.0000
1 q (k I A) mk
Eigenvalue: 2.0
Algebraic multiplicity: 2
(6)
3. If mk > gk for any k, the Jordan form and generalized eigenvectors are
required.
Geometric Multiplicity: 1
Number of missing eigenvectors: 1
k k mk gk
Recall the eigen-decomposition of a matrix:
1
2
..
.
A = V U V 1
The eigen-decomposition only exists if V is invertible. That is if there is a
complete set of independent eigenvectors.
Part 4b: Eigenvalues and Eigenvectors
# Needed Generalized Evecs
mk gk
(Revised: Sep 10, 2012)
Page 5
Table 1: Analysis of the structure of the eigensystem of A.
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 1.2.0
EE/ME 701: Advanced Linear Systems
Section 2.1.0
2 Properties of the Eigensystem
Example,
>> A = [ 2 3 4 ; 0 2 1 ; 0 0 2 ]
A =
2
3
4
0
2
1
0
0
2
First well cover properties of the eigenvalues and regular eigenvectors.
2.1 Bay section 4.1, A-Invariant Subspaces
Recall: for triangular and diagonal matrices, the eigenvalues are the
diagonal elements
>> [V, U ] = eig(A)
V =
1.0000
-1.0000
0
0.0000
0
0
U =
2
0
0
0
2
0
1.0000
-0.0000
0.0000
>> RoundByRatCommand(V)
ans = 1
-1
1
0
0
0
0
0
0
0
0
2
(7)
D = E
Table 2: Analysis of the structure of the eigensystem of A.
Part 4b: Eigenvalues and Eigenvectors
(8)
In an isotropic dielectric medium, the electric field follows the relation
# Needed Generalized Evecs
mk gk
( I A) v = 0
Example (Bay Example 4.1): Electric Fields
The Jordan form will be required.
Av = v
Eqn (8) shows that the Eigenvectors lie in the null space of ( I A)
dim (1 I A) = 1, there is one eigenvector.
The eigenvectors of A are basis vectors for A-invariant subspaces.
v Av = 0
We see that there is one eigenvalue that is triply repeated.
When the operator A is understood from context, X1 is sometimes said
to be simply invariant.
Eqn (7) gives:
The analysis of the structure of the eigensystem of matrix A is seen in
table 2.
k k mk gk
Let X1 be a subspace of linear vector space X. This subspace is A-invariant
if for every vector z X1 , y = A z X1 .
(Revised: Sep 10, 2012)
where E is the electric field vector, D is the electric flux density (also called the
displacement vector) and is the dielectric constant.
Page 7
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 2.2.0
D
1 11 12 13
D2 = 21 22 23
D3
31 32 33
Section 2.3.0
2.3 Interpreting complex eigenvalues / eigenvectors
Some materials, however, are anisotropic, governed by
EE/ME 701: Advanced Linear Systems
Real eigenvalues correspond to scaling the eigenvector
E
1
E2
E3
Complex eigenvalues lead to complex eigenvectors, and correspond to
rotations
Complex eigenvalues come in complex conjugate pairs
Find the directions, if any, in which the E-field and flux density are collinear.
Examples: Consider the basic rotation matrix:
Solution:
For the E-field and flux density to be collinear they must satisfy D = E,
R=
Which is to say, the anisotropic directions are the eigenvectors of the
dielectric tensor.
C S
S
Eigenvalues:
2.2 Finding eigenvalues and eigenvectors
For ( I A) v = 0 to have a solution, ( I A) must have a non-trivial null
space. This is equivalent to saying
det ( I A) = 0
(9)
( I R) =
which gives the characteristic equation
det (I R) = 2 2C +C2 + S2 = 2 2C + 1 = 0
which solves to give
We have seen that Eqn (9) gives an nth order polynomial in
This is more important for understanding than as a solution method
Actual eigenvalue / eigenvector algorithms are beyond the scope of
EE/ME 701.
Use:
q
4C2 4
2
2C
q
2
4 S2
= C j S
The eigenvalues of R are a complex conjugate pair for any value of
6= 0o , 180o !
i = 1.0
>> [V, U] = eig(A)
Part 4b: Eigenvalues and Eigenvectors
2C
(Revised: Sep 10, 2012)
Page 9
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 2.3.1
EE/ME 701: Advanced Linear Systems
Section 2.3.1
The matrix R of Eqn (11) also describes a rotation of 600 about the axis v
2.3.1 Example: 3D Rotation
Weve seen the rotation matrix
c
tR
CC +C S CS S S S CC S
= SC
S SS +CC C SS CS
S
CS
CC
0.577
v3 = 0.577
0.577
(10)
w
y
Rw
x
z
For example, the rotation matrix
Figure 1: Illustration of the action of rotation matrix R on a vector w to give
vector R w.
2/3 1/3 2/3
R = 2/3 2/3 1/3
1/3 2/3 2/3
(11)
The mathematical manifestation of the absence of any other R-invariant
subspace is that the other two eigenvalues are complex.
Corresponds to
= 135o ,
= 135o ,
= 19.47o
When we form solutions of the form
This matrix (like all 3D rotation matrices with general angles) has one real
eigenvalue and a complex conjugate pair
>> [V, U] = eig(R)
V = -0.5774
0.2887 + 0.5000i
0.2887 - 0.5000i
U =
Every vector w v will be rotated by R. So the only R-invariant subspace
lies along the axis of rotation.
0.5000 + 0.8660i
0
0
Part 4b: Eigenvalues and Eigenvectors
-0.5774
0.2887 - 0.5000i
0.2887 + 0.5000i
0.5774
0.5774
0.5774
0
0.5000 - 0.8660i
0
0
0
1.0000
(Revised: Sep 10, 2012)
x (t) = V eU t V 1 x (0)
(12)
the complex eigenvalue and eigenvector pair combines to give solutions that
can be written
x (t) = a1et cos (t + ) w1 + a2et sin (t + ) w2
(13)
where the complex is written = j and the complex eigenvectors
form the real eigenvectors:
1
(real part)
(14)
w1 = (v1 + v2)
2
1
(v1 v2)
2
Part 4b: Eigenvalues and Eigenvectors
w2 = j
Page 11
(imaginary part)
(Revised: Sep 10, 2012)
(15)
Page 12
EE/ME 701: Advanced Linear Systems
Section 2.4.0
2.4 The eigen-system of symmetric (Hermitian) matrices
The eigensystem of a Hermitian matrix Q , (symmetric matrix, if real) have
special, and very helpful properties.
EE/ME 701: Advanced Linear Systems
Property 3: The eigenvectors of a Hermitian matrix, if they correspond to distinct
eigenvalues, must be orthogonal.
Proof: Starting with the given information, eigenvalue values 1 6= 2, and
corresponding eigenvectors v1 and v2
Notation: use A to be the complex-conjugate transpose (equivalent to A in
Matlab).
Property 1: If A = A, then for all complex vectors x, x A x is real.
Proof: Define y = (x A x). Applying the transpose to the product,
Section 2.4.0
Q v1 = 1 v1
(16)
Q v2 = 2 v2
(17)
Forming the complex-conjugate transpose of Eqn (16)
y = (x A x) = x A x = x A x
v1 Q = 1 v1 = 1 v1
Since A = A , y = y . For a number to equal its complex conjugate, it
must be real.
(18)
where we can drop the complex conjugate, because we know 1 is real.
Now multiplying on the right by v2 gives
Property 2: The eigenvalues of a Hermitian matrix must be real.
Proof: Suppose the is an eigenvalue of Q , with v a corresponding
eigenvector, then
Qv = v
v1 Q v2 = 1 v1 v2 = 1 v1 v2
(19)
But the multiplication also gives:
Now multiply on the left and right by v,
v1 (Q v2) = v1 2 v2 = 2 v1 v2
(20)
1 v1v2 = 2 v1 v2
(21)
v Q v = v v = v v = ||v||
So we find
By property 1,
v Q v
must be real, and ||v|| must be real, there for
=
If 1 6= 2, Eqn (21) is only possible if v1v2 = 0, which is to say that v1 and
v2 are orthogonal.
v Q v
||v||2
must be real.
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 13
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 2.4.0
Property 4: A Hermitian matrix has a complete set of orthogonal eigenvectors.
EE/ME 701: Advanced Linear Systems
Section 3.1.0
3 The Jordan-form
Proof: Any square matrix Q has a Schur decomposition
3.1 How the Jordan form relates to the real world
T = V 1 QV
(22)
The solution to the equation
where V is an orthonormal matrix and T is an upper-triangular matrix.
Since T is upper-triangular, T will be lower-triangular. However, V is
ortho-normal and Q is Hermitian, so
V = V 1 ,
So
Q = Q , and
V 1 = (V ) = V
For T to be both upper-triangular and lower-triangular, it must be
diagonal. Let U = T be a diagonal matrix.
Multiplying Eqn (22) on the left by V and on the right by V 1gives
=VUV
(23)
Which is precisely the form of the eigen-decomposition of Q, where
Diagonal matrix U holds the eigenvalues of Q, and
Remarks:
Since T = T , the diagonal elements (the eigenvalues) must be real
(see property 2).
When Q is real (symmetric), V will be real.
(Revised: Sep 10, 2012)
(25)
U =
0
2
...
and
(26)
eU t =
e1 t
0
e2 t
But what if the set of vk is incomplete, so there is no V 1 !
...
(27)
Such matrices can arise with repeated eigenvalues and do not have a
complete set of eigenvectors. Such matrices are called defective.
If we generated A matrices randomly, defective matrices would be quite
rare.
But repeated poles (eigenvalues) are relatively common in the analysis
of dynamic systems, and sometimes even a design goal.
Orthogonal matrix V holds the eigenvectors of Q.
Part 4b: Eigenvalues and Eigenvectors
x (t) = eAt x (0)
has terms of the form
eAt = V eU t V 1
Since T = T , T must be both upper-triangular and lower-triangular.
Q=V TV
(24)
When there is a complete set of eigenvectors,
T = V Q V 1 = V 1 QV = T
x (t) = A x (t) + B u (t)
Page 15
When A has repeated eigenvalues and missing eigenvectors (gk < mk ),
analysis of eAt requires converting matrix A to the Jordan form.
When we have a complete set of independent eigenvectors, eAt is given by
Eqn (26).
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 16
EE/ME 701: Advanced Linear Systems
Section 3.1.1
EE/ME 701: Advanced Linear Systems
Section 3.1.1
Consider eJ t for a matrix of the form
3.1.1 An example calling for the Jordan form
J=
With scalar differential equations, we know that equations with repeated
roots give solutions of the form
1
0
(31)
The expression for eJ t is
y (t) = c1 e1 t + c2 t e1 t .
For example,
y (t) + 6 y (t) + 9y (t) = 0
(28)
has the characteristic equation s2 + 6 s + 9 = 0 which as the roots
s = {3, 3} . The solution to Eqn (28) is:
y (t) = c1 e3t + c2 t e3t .
(29)
J2 t 2
Jk t k
eJ t = I + J t +
+ +
+
(32)
2!
k!
2
3
1 0
1
2 2
3 32
+ t
+ t
+
+ t
=
1! 0
2 ! 0 2
3 ! 0 3
0 1
t k k k k1
+
(33)
+
k! 0
k
The (1,1) and (2,2) elements give the series
But Eqn (27) for eAt has no terms of the form t e3t . And yet Eqn (28) is
simply represented in state space with:
x (t) =
y (t)
y (t)
d y (t) 6 9 y (t)
x (t) =
=
dt y (t)
1 0
y (t)
(30)
And the solution to Eqn (30) is (as always for x (t) = A x (t))
1 k k
t = et
k
!
k=1
1+
The (2,1) element of eJ t is 0. Now consider the (1,2) element, which is
given as
0+
1
t
+
k k1 t k =
1 ! k=2 k !
x (t) = eAt x (0) .
t 1+
k=2
So how can eAt have a term of the form t e3t ?
(Revised: Sep 10, 2012)
1
k1t k1
(k 1) !
= t 1+
k=1
1 kk
t
k!
= t et
So if J has the form of Eqn (31), then
Jt
Part 4b: Eigenvalues and Eigenvectors
(34)
Page 17
e = exp
1
0
Part 4b: Eigenvalues and Eigenvectors
t =
et t et
0
et
(Revised: Sep 10, 2012)
(35)
Page 18
EE/ME 701: Advanced Linear Systems
Section 3.1.1
et t et
1 0
exp 0 1 t = 0 et
0
0
0 0
1 2 t
2t e
t et
et
(36)
1 0 0
(37)
3.2.1 Regular and Generalized Eigenvectors
The regular eigenvectors are what we have considered all along, they satisfy
the relationship
For our specific example, A decomposes according to
A = M J M 1
A=
M=
And
J is a block-diagonal matrix, it is composed of mk mk blocks along the
main diagonal. Eqns (31), (36) and (37) give examples of 2x2, 3x3 and
4x4 blocks.
Each block on the diagonal of J is called a Jordan block.
gives terms of the form t 3 et , etc.
6 9
(39)
The columns of M are regular eigenvectors and generalized eigenvectors
0 1 0
t
exp
0 0 1
0 0 0
Matrix A has been transformed to the Jordan form (sometimes called the
Jordan canonical form) when
J = M 1 A M
which gives terms of the form t 2 et ,
with
Section 3.2.1
3.2 Constructing the Jordan Form
By the argument of Eqns (31)-(35) above,
EE/ME 701: Advanced Linear Systems
0.949
0.316
eAt = M eJ t M 1 = M
(38)
0.0316
0.0949
, J=
e3t t e3t
0
e3t
3
0
1
3
A v = k v
or
(A k I) v = 0
(40)
From Eqn (40), a set of independent regular eigenvectors is given as the null
space of (A k I).
M 1
So the solution x (t) = eAt x (0) will have terms of the form e3t and t e3t ,
as need !
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 19
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 3.2.1
The generalized eigenvectors form chains starting with a regular
eigenvector. The generalized eigenvectors satisfy the relationship
l+1
l
A Vk,l+1
j = k Vk, j +Vk, j
(41)
l
(A k I) Vk,l+1
j = Vk, j
(42)
Or, rearranging
is the the next generalized eigenvector in a chain
(see Bay Eqn (4.14)).
In this notation,
(43)
is the first element of a chain; it is a regular eigenvector, and it is the jth
regular eigenvector of the kth distinct eigenvalue.
= 1 designates that Vk,1 j
The l
be a regular eigenvector.
Section 3.2.2
3.2.2 First Jordan Form Example
Consider eAt , with A given as:
>> A = [ 3 3 3 ; -3 3 -3 ; 3 0 6 ]
A =
3
3
3
-3
3
-3
3
0
6
First look at the eigenvalues
Where Vk,l+1
j
Vk,1 j
EE/ME 701: Advanced Linear Systems
is the first eigenvector in a chain, so it must
Eqn (42) is an example of a recursive relationship, it is an equation that is
applied repeatedly to get all elements of the chain.
The method presented here to determine the Jordan form is the bottom-up
method presented in Bay, section 4.4.3.
>> U = eig(A)
U = 3.0000
6.0000
3.0000
%% This command rounds-off values to nearby rational numbers
%% which may be integers
>> U = RoundByRatCommand(U)
U =
3
6
3
A has a repeated eigenvalue, we can make a table analyzing the structure of
the eigensystem of A
k k mk
1
2
3
6
2
1
gk
# Needed Gen. Evecs
mk gk
1 or 2
1
0 or 1
0
Table 3: Analysis of the structure of the eigensystem of A.
Table 3 shows that A has two distinct eigenvalues, and we dont yet know if
1 has 1 or 2 independent eigenvectors.
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 21
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 22
EE/ME 701: Advanced Linear Systems
Section 3.2.2
Evaluate the geometric multiplicity of 1
EE/ME 701: Advanced Linear Systems
In the example, we need 1 generalized eigenvector for the k = 1 eigenvalue.
>> lambda1=3; lambda2=6; I = eye(3);
>> v1 = null(A-lambda1*I); v1 = v1/v1(1)
v1 = 1
%% Eigenvector, scaled so the
1
%% first element is an integer
-1
In this case for k = 1 we have only one regular eigenvector, so it must serve
as the first element, or anchor, of the chain of generalized eigenvectors
regular eigenvector:
generalized eigenvector:
The geometric multiplicity is the dimension of the null space in which the
eigenvectors lie. For 1, g1 = 1. Putting this information into the table
k k mk gk
1
2
3
6
2
1
# Needed Generalized Evecs
mk gk
1
1
1
0
Table 4: Analysis of the structure of the eigensystem of A.
The total number of eigenvectors (regular+generalized) needed for k is mk .
The number of regular eigenvectors is gk . The regular eigenvectors get the
notation:
1
1
1
Vk,1
, Vk,2
, ..., Vk,g
k
where j = 1 ... gk .
The number of needed generalized eigenvectors, corresponding to k , is
mk gk .
Part 4b: Eigenvalues and Eigenvectors
Section 3.2.2
(Revised: Sep 10, 2012)
Page 23
1
V1,1
solves
2
V1,1
solves
1
(A 1 I)V1,1
= 0
(44)
2
1
(A 1 I)V1,1
= V1,1
In Matlab
>> lambda1=1; lambda2=2; I = eye(3);
%% Find the first regular eigenvector,
>> V111 = null(A-lambda1*I); V111=V111/V111(1)
V111 = 1.0000
1.0000
-1.0000
%% Find the generalized eigenvector by solving Eqn (44)
>> V112 = pinv(A-lambda1*I)*V111
V112 = -1.0000
1.0000
0.0000
%% Find the regular eigenvector for lambda2
>> V211 = null(A-lambda2*I); V211=V211/V211(2)
V211 =
0
1.0000
-1.0000
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 24
EE/ME 701: Advanced Linear Systems
Section 3.2.2
Put the eigenvectors (regular and generalized) together in the M matrix.
The regular and generalized eigenvectors of a chain must be put in order.
vectors Vk,l j
For each k and j, put the
going to the end of the chain.
into M, starting with l = 1, and
For the example,
>> M = [V111, V112, V211]
M =
1.00
-1.00
0
1.00
1.00
1.00
-1.00
0.00
-1.00
..
.
..
.
..
.
..
.
3.2.3 More on Jordan blocks
A matrix in Jordan canonical form has a block diagonal structure, with
Eigenvalues on the main diagonal
>> J = inv(M) * A * M
>> J = RoundByRatCommand(J)
J =
3
1
0
0
3
0
0
0
6
0
,
A 2x2 block has 1 one, a 3x3 block has 2 ones, etc.
One Jordon block corresponds to each regular eigenvector
If the regular eigenvector has no generalized eigenvectors, then it creates a
1x1 block.
i
h
1 V2 V1
M = V1,1
1,1
2,1
3 1
0 3
J=
0 0
If the regular eigenvector anchors a chain with one generalized eigenvector,
then it creates a 2x2 block, etc.
Each Jordan block corresponds to:
1x1 block:
a regular eigenvector
n n block, n 2: a chain anchored by a regular eigenvector, with n 1
generalized eigenvectors
eAt = M eJ t M 1
(45)
For a system governed by x (t) = A x (t), and considering the J matrix, the
output of the system will have solutions of the form
y (t) = c1 e3t + c2 t e3t + c3 e6t
(46)
Using the Vk,l j notation, if we look at the structure of the M matrix, we can
determine the layout of Jordan blocks. For example,
h
i
1 V1 V2 V1 V2
M = V1,1
1,2
1,2
2,1
2,1
Then the blocks of J are arranged:
1x1
J=
where the first two terms correspond to the first Jordan block, and the last
term to the second Jordan block.
Part 4b: Eigenvalues and Eigenvectors
Section 3.2.3
Ones on the upper diagonal within each block.
Put in the chain corresponding to for each regular eigenvector ( j) for
each distinct eigenvalue (k). Chains may have a length of 1.
J has 2 Jordon blocks
EE/ME 701: Advanced Linear Systems
(Revised: Sep 10, 2012)
Page 25
2x2
2x2
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 26
EE/ME 701: Advanced Linear Systems
Section 3.2.4
3
1
0
0
0
0
-1
1
0
0
0
0
1
-1
2
0
0
0
1
-1
0
2
0
0
0
0
1
-1
1
1
0
0
1
-1
1
1
>> U = eig(A)
U = 2.0000
2.0000
2.0000
2.0000
2.0000
0
>> lambda1=0; lambda2=2; I = eye(6);
>> [Row, Col, Null, LNull] = spaces(A-lambda2*I);
>> g2=rank(Null); g2 = 2
(A 2 I) has a 2-dimensional null space, so there are 2 independent regular
eigenvectors.
=
0
0.7071
0
0.7071
-0.7071
0.0000
0.7071
0.0000
0.0000
0
0.0000
0
For convenience, scale the eigenvectors to get integer values
>> V211 = Null(:,1)/Null(3,1)
V211 =
0
0
1
-1
0
0
Part 4b: Eigenvalues and Eigenvectors
>> %% Check that the eigenvectors are indeed eigenvectors
>> %% These norms come out to zero, very small would be sufficient
>> NDiff1 = norm( A*V211 - lambda2*V211 )
NDiff1 =
0
>> NDiff2 = norm( A*V221 - lambda2*V221 )
NDiff2 =
0
Note: All vectors from null (A 2I) are eigenvectors. For example,
1 = 0 , 2 = 2 is repeated 5 times.
Null
Section 3.2.4
Check that the eigenvectors are actually eigenvectors for 2 = 2
3.2.4 The Jordan form, a second example
A =
EE/ME 701: Advanced Linear Systems
>> V221 = Null(:,2)/Null(1,2)
V221 = 1
1
0
0
0
0
(Revised: Sep 10, 2012) Page 27
>> x = 0.3*V211 + 0.4*V221
x = 0.4
>> NDiffx = norm( A*x - lambda2*x )
0.4
NDiffx = 0
0.3
-0.3
0.0
0.0
Make a table of the structure of the eigensystem.
k k mk gk
1
2
0
2
1
5
1
2
# Needed Gen. Evecs
mk gk
0
3
Table 5: Structure of the eigensystem of A.
We need 3 generalized eigenvectors to have a complete set.
These three will be in chains anchored by one or both of the regular
eigenvectors of 2 .
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 28
EE/ME 701: Advanced Linear Systems
Section 3.2.4
EE/ME 701: Advanced Linear Systems
Section 3.2.4
2
Find V2,1
The equation giving generalized eigenvectors is
l
(A k I) Vk,l+1
j = Vk, j
(47)
This is simply the relation
Bx = y
(48)
with B = (A k I) , y = Vk,l j and x = Vk,l+1
j .
We know some things about the solutions of Eqn (47)
1. For an exact solution to exist Vk,l j must lie in the column space of
(A k I)
2. If we find a solution Vk,l+1
j , it is not unique. We can add any component
from the null space of (A k I) , and it will still be a solution.
>> V212 = pinv(A-lambda2*I)*V211
V212 =
0
0
0.00
-0.00
0.50
0.50
%% Check that V212 is a generalized eigenvector
>> NDiffV212 = norm( (A-lambda2*I)*V212 - V211 )
NDiffV212 = 2.7581e-16
2 is a generalized eigenvector.
Yes, V2,1
Considering again the example problem,
Check that V211 and V221 lie in the column space of (A 2 I) by
checking that the projection of each onto the column space is equal to
the original vector
2 is in the column space of (A I)
Test to see if V2,1
2
>> norm( Col*Col*V212-V212 )
ans = 0.7071
>> [Row, Col, Null, LNull] = spaces(A-lambda2*I);
3
No, so there is no V2,1
>> NIsInColumnSpaceV211 = norm( Col*Col*V211-V211 )
NIsInColumnSpaceV211 = 1.1102e-16
%% V211 is in the column space of (A-lambda2*I)
>> NIsInColumnSpaceV2 = norm( Col*Col*V221-V221 )
NIsInColumnSpaceV2 = 1.1430e-15
%% V221 is in the column space of (A-lambda2*I)
Both vectors lie in the column space of (A 2 I) , so each will have at
least one generalized eigenvector.
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 29
2
Now evaluate V2,2
>> V222 = pinv(A-lambda2*I)*V221
V222 = 0.50
>>
%% Check that V222 is a gen. eigenvector
-0.50
>>
NDiffV222
= norm( (A-lambda2*I)*V222 - V221 )
-0.00
NDiffV222
=
6.2804e-16
-0.00
0.00
2 is a generalized eigenvector
Yes, V2,2
0.00
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 30
EE/ME 701: Advanced Linear Systems
Section 3.2.4
2 is in the column space of (A I)
Now check to see that V2,2
2
3 . This will be the third generalized eigenvector
Yes, so there is a V2,2
V223 =
0.00
-0.00
0.25
0.25
-0.00
-0.00
0
0
0
1
2
0
0
0
0
0
1
2
J has 3 Jordan blocks
J =
First we need the regular eigenvalue corresponding to 1
>> V111 = null(A-lambda1*I);
>> V111 = V111/ V111(5)
V111 =
0
0
0
0
1
-1
Put in the chains of E-vecs, starting each chain with its regular E-vec
Part 4b: Eigenvalues and Eigenvectors
>> J = inv(M)*A*M;
>> J = RoundByRatCommand(J)
J =
0
0
0
0
0
2
1
0
0
0
2
0
0
0
0
2
0
0
0
0
0
0
0
0
Interpreting the result
Now build the M matrix
>> M = [ [V111] [V211 V212]
>> M = RoundByRatCommand(M)
M =
0
0
0
0
0
0
0
1.00
0
0 -1.00
0
1.00
0
0.50
-1.00
0
0.50
Section 3.2.4
Now find J
>> NIsInColumnSpaceV222 = norm( Col*Col*V222-V222 )
NIsInColumnSpaceV222 = 4.2999e-16
>> V223 = pinv(A-lambda2*I)*V222,
EE/ME 701: Advanced Linear Systems
[V221 V222 V223] ]
1.00
1.00
0
0
0
0
0.50
-0.50
0
0
0
0
(Revised: Sep 10, 2012)
Correspondingly, M has 3 chains of eigenvectors
M
0
0
0.25
0.25
0
0
0
| 0
0 |
0
0
0
----------------0
| 2
1 |
0
0
0
0
| 0
2 |
0
0
0
----------------------------0
0
0 |
2
1
0
0
0
0 |
0
2
1
0
0
0 |
0
0
2
0
0
0
0
1.00
-1.00
|
0
|
0
| 1.00
| -1.00
|
0
|
0
0
0
0
0
0.50
0.50
|
|
|
|
|
|
1.00
1.00
0
0
0
0
0.50
-0.50
0
0
0
0
0
0
0.25
0.25
0
0
The first eigenvector in each chain is a regular eigenvector.
Page 31
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 32
EE/ME 701: Advanced Linear Systems
Section 3.3.0
3.3 One more twist, freedom to choose the regular eigenvector
Fact: If a matrix A has repeated eigenvalues, with gk > 1 independent eigenvector,
the gk eigenvectors form a vector subspace, any vector from which is an
eigenvector.
When mk 3, it is possible that gk 2, and we need to find a generalized
eigenvector.
EE/ME 701: Advanced Linear Systems
Section 3.3.0
When gk 2, we have the freedom to choose the anchor for the chain of
generalized eigenvectors
1 , V 1 , ...
Not just from a list, Vk,1
k,2
But as any vector from the null space of (A k I) .
1 , V 1 , neither one of
It may be that we have valid eigenvectors , Vk,1
k,2
which lies in the column space of (A k I) !
By forming the intersection of the null and column spaces of (A k I), we
can find the needed regular eigenvector.
In this case,
dim null (A k I) = gk 2
and any vector from the 2-dimensional (or larger) null space of (A k I) is
an eigenvector.
W = col (A k I) null(A k I) ,
Vk,1 j W
(50)
Consider the generating equation for the generalized eigenvector
(A k I) Vk,2 j = Vk,1 j
(49)
The anchor Vk,1 j must also lie in the column space of (A k I)
A regular eigenvector that anchors a chain of generalized eigenvectors must
lie in 2 spaces at once:
The null space of (A k I) . . . . . . . . . . . . . . . To be a regular e-vec of A.
The column space of (A k I) . . . To generate a generalized e-vec of A.
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 33
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 34
EE/ME 701: Advanced Linear Systems
Section 3.3.1
3.3.1 Example where regular E-vecs do not lie in the column space of
(A k I)
Consider the matrix
2 1
0 4
A=
0 1
0 1
>> NIsInColumnSpaceV1 =
NIsInColumnSpaceV1 =
>> NIsInColumnSpaceV2 =
NIsInColumnSpaceV2 =
>> RoundByRatCommand ( eig(A) )
ans = 2
2
2
2
-0.7118
-0.2468
-0.4650
-0.4650
norm( Col*Col*v1-v1 )
0.6325
norm( Col*Col*v2-v2 )
0.6325
But what about the possibility that there exists another eigenvector which
lies in the null space and column space of (A k I):
Null =
0
0.8165
-0.4082
-0.4082
1
0
0
0
So the structure of the eigensystem is given in table 6
k k mk gk
Null(3,1) )
No, neither eigenvector lies in the column space of (A k I)
>> I = eye(4) lambda1=2,
>> [Row, Col, Null, Lnull] = spaces(A - lambda1*I)
0.3055
0.7342
-0.4287
-0.4287
A first choice for eigenvectors are the two basis vectors of the null space of
(A k I)
Determine if the eigenvectors like in the column space of (A k I)
Analyzing the structure of its eigensystem
Col =
Section 3.3.1
>> v1 = RoundByRatCommand( Null(:,1) /
>> v2 = RoundByRatCommand( Null(:,2) )
v1 =
0
v2 = 1
-2
0
1
0
1
0
2 2
0 0
EE/ME 701: Advanced Linear Systems
x1 = a1 v1 + a2 v2
First, consideration of the possibilities
The universe is R4 , or 4D, with dim col (A k I) = 2, and
dim null (A k I) = 2. So there are 3 possibilities:
1. Two 2D spaces can fit in a 4D universe and not intersect, so it is
possible
col (A k I) null (A k I) = 0
# Needed Gen. Evecs
mk gk
2
2. It is possible that the intersection is 1 D
Table 6: Analysis of the structure of the eigensystem of A.
3. It is possible that the intersection is 2 D
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 35
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 36
EE/ME 701: Advanced Linear Systems
Section 3.3.1
Previously, we have seen how to form the intersection of two subspaces,
Given sets of basis vectors U = [u1 , u2 , ..., unu ] and V = [v1 , v2 , ..., vnv ],
vectors in the intersection
W = UV
That is
Now find w1 , a vector in both the column and null spaces of (A k I)
>> w1 = Col*CoeffVec(1:2)
>> w1 = RoundByRatCommand( w1/w1(1)),
w1 =
1
2
-1
-1
Verify that w1 is an eigenvector
wi = a1 u1 + a2 u2 + + anu unu = b1 v1 + b2 v2 + + bnv vnv
Section 3.3.1
(51)
Are solutions to
for some non-zero values
EE/ME 701: Advanced Linear Systems
a1 anu , b1
(52)
i
a1 anu , b1 anv .
i
anv must solve
a
1
[U, V ] ... = 0
bnv
(A k I) has only one regular eigenvector that can anchor a chain. So the
chain must have length 3 (2 generalized E-vecs).
2 as solution to
Compute a candidate for the first generalized eigenvector, V1,1
2 = V1
(A k I) V1,1
1,1
(53)
h
i
The coefficient vector must lie in the null space of Col, Null , where
[Col] and [Null] are sets of basis vectors on the column and null spaces.
>> CoeffVec = null([Col, -Null])
CoeffVec =
0.7033
-0.0736
0.6547
0.2673
h
i
Since the null space of Col, Null is one dimensional, the intersection
of the column and null spaces is 1D
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
>> NIsEigenvectorW1 = norm( A*w1 - lambda1 * w1)
NIsEigenvectorW1 =
0
Page 37
>> V111 = w1,
>> v3 = pinv(A - lambda1*I) * V111,
v3 =
0
0.3333
0.3333
0.3333
3 , V 2 must be in the
To find the remaining generalized eigenvector, V1,1,
1,1
column space of (A k I)
>> NIsInColumnSpaceV112 = norm( Col*Col*v3 - v3)
NIsInColumnSpaceV112 = 0.4216
It is not !
2 ,
v3 is a candidate generalized eigenvector, but we can not use it for V1,1
3 .
because it does not lead to V1,1
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 38
EE/ME 701: Advanced Linear Systems
Section 3.3.1
2 must solve
Going back to the generating Eqn, V1,1
EE/ME 701: Advanced Linear Systems
Section 3.3.1
2. Determine the candidate vector
>> v3b = v3 + Null * CoeffVec2(3:4),
2
1
(A k I) V1,1
= V1,1
(54)
v3 is a particular solution to Eqn (54), but it is not the only solution. Any
vector
2
V1,1
= v3 + b1 n1 + b2 n2
(55)
is a solution to (54), where n1 and n2 are basis vectors for the null space of
(A k I).
2 that is in the column space of (A I), we need a
To find a value for V1,1
k
solution to
2
= v3 + b1 n1 + b2 n2 = a1 c1 + a2 c2
(56)
V1,1
Or
c1 c2 n1
1. Find the coefficient vector
a
1
i
a2
= v3
n2
b1
b2
(57)
3. Check to be sure the new candidate is in the column space of (A k I)
>> NIsInColumnSpaceV112b = norm( Col*Col*v3b - v3b)
NIsInColumnSpaceV112b =
6.0044e-16
Yes !
2 and compute V 3
Set V1,1
1,1
>> V112 = v3b
V112 = 0.5714
0.1429
0.4286
0.4286
>> V113 = pinv(A - lambda1*I) * V112
V113 =
0
0.1905
0.6905
-0.3095
1 is any independent regular eigenvector. Compute J.
Build the M matrix, V1,2
>> M = [ V111, V112, V113, V121 ]
M = 1.0000
0.5714
0
2.0000
0.1429
0.1905
-1.0000
0.4286
0.6905
-1.0000
0.4286
-0.3095
>> CoeffVec2 = pinv( [Col, -Null]) * v3
CoeffVec2 = -0.0880
-0.8406
-0.2333
0.5714
v3b = 0.5714
0.1429
0.4286
0.4286
1.0000
0
0
0
>> J = RoundByRatCommand( inv(M) * A * M )
J = 2
1
0
0
0
2
1
0
0
0
2
0
0
0
0
2
J has a 3x3 block, and a 1x1 block.
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 39
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 40
EE/ME 701: Advanced Linear Systems
Section 3.4.0
3.4 Summary of the Jordan Form
EE/ME 701: Advanced Linear Systems
Section 3.4.1
3.4.1 Why Matlab does not have a numeric Jordan command
Square matrices always have n eigenvalues, i
Strikingly, Matlab has no numerical routine to find the generalized
eigenvectors or Jordan form (standard Matlab no jordan() routine !)
The regular eigenvectors are given as the null space of (A i I)
For a repeated eigenvalue k
The algebraic multiplicity, mk , is the number of times k is repeated
This is because the Jordan form calculation is numerically very sensitive,
a small perturbation in A produces a large change in the chains of
eigenvectors
This sensitivity is true of the differential equations themselves,
The geometric multiplicity, gk , is the dimension of null(A k I)
When eigenvalues are repeated, we may not have enough independent
regular eigenvectors (gk < mk ), in which case the Jordan form is required.
The Jordan form corresponds to scalar differential equations with
repeated roots and solutions of the form
y (t) + 6 y (t) + 9.00001y (t) = 0
has two distinct roots !
Consider the stages where a decision must be made
When there are two eigenvalues with a b, are they repeated or
distinct ?
y (t) = a1 e1 t + a2 t e1 t + a3 t 2 e1 t ...
What is the dimension of null (A I) ?
For repeated eigenvalues, regular eigenvectors give rise to chains of
generalized eigenvectors. The generalized eigenvectors are solutions to
l+1
l
(A i I) vk,
j = vk, j
Does vlk, j lie in the column space of (A I), or does it not ?
l+1
Is vk,
j independent of the existing eigenvectors ?
(58)
There is no known numerical routine to find the Jordan form that is
sufficiently numerically robust to be included in Matlab.
Examples have shown several characteristics of eigensystems with repeated
roots.
The Matlab symbolic algebra package does have a jordan() routine. It
runs symbolically on rational numbers to operate without round-off error,
for example:
where Eqn (58) is repeated as needed to obtain mk eigenvectors.
21/107 52/12
A=
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 41
119/120
1/1
11/12
Part 4b: Eigenvalues and Eigenvectors
3/2
8/5
13/14
(Revised: Sep 10, 2012)
Page 42
EE/ME 701: Advanced Linear Systems
Section 4.0.0
4 Conclusions
EE/ME 701: Advanced Linear Systems
Section 5.0.0
The solution in Jordan for comprises Jordan blocks
To solve
Each Jordan block corresponds to a chain of generalized eigenvectors.
x (t) = A x (t) + B u (t)
The generalized eigenvectors solve the equation
we are interested to make a change of basis from physical (or other)
coordinates to modal coordinates.
This involves the eigenvalues and eigenvectors. The eigenvectors solve the
equation
(A i I) vi = 0
l+1
l
(A i I) vk,
j = vk, j
For the l th generalized eigenvector, of the jth Jordan block, of the kth distinct
eigenvalue
v1k, j is a regular eigenvector, corresponding to k
So the eigenvectors lie in null (A i I)
l+1
l
For vk,
j to exist, vk, j must lie in the column space of (A k I)
Eigenvalues may be complex
The Jordan form leads to solutions of the differential equation in the form
t et , t 2 et , etc. For example:
Complex eigenvalues correspond to solutions with terms
x (t) = a2et cos (t + ) w1 + a2et sin (t + ) w2
The complex terms correspond to the action of a rotation in state space,
in the sub space spanned by the complex eigenvectors.
The eigenvectors corresponding to an eigenvalue (or a complex conjugate
pair of eigenvalues) define and A-invariant subspace. Vectors in this
subspace stay in this subspace.
The modal matrix is the transformation from from modal to physical
p
coordinates, M = m T .
If we lack a complete set of regular eigenvectors, M includes generalized
eigenvectors, and
J = M 1 A M
gives the system matrix in Jordan form.
(Revised: Sep 10, 2012)
Page 43
..
.
..
.
..
.
.. 2 t
. e
t e2 t
..
. 0 e2 t
..
. 0
0
..
1 1 .
1 ..
...
J =
..
. 2 1
..
.
2 1
..
.
2
With a complete set of eigenvectors, M = V .
Part 4b: Eigenvalues and Eigenvectors
(59)
eJt
1 t t e1 t
e
0 e1 t
Part 4b: Eigenvalues and Eigenvectors
1 2 2 t
2t e
t e2 t
e2 t
(Revised: Sep 10, 2012)
Page 44
EE/ME 701: Advanced Linear Systems
Section 5.0.0
5 Review questions and skills
1. In what fundamental space do the regular eigenvectors lie.
2. Given the eigenvalues of a matrix, analyze the structure of the eigensystem
(a) Determine the number of required generalized eigenvectors
3. Indicate the generating equation for the generalized eigenvectors.
4. Indicate in what fundamental space the vectors of the generating equations
must lie.
5. When gk 2, and no regular eigenvector lies in the column space of
(A k I), what steps can be taken ?
6. When additional generalized eigenvectors are needed, and v, a candidate
generalized eigenvector does not lie in the column space of (A k I), what
steps can be taken ?
Part 4b: Eigenvalues and Eigenvectors
(Revised: Sep 10, 2012)
Page 45
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Review-By-Example of some of the Basic
Concepts and Methods of Control System
Analysis and Design
4.1.1
4.2
Contents
1 Differential Eqns, Transfer Functions & Modeling
4.3
Real Pole location and response characteristics . . . . . .
Basis of pole-zero maps, 2nd order, p1,2 = j
27
. . . . . . .
28
4.2.1
Notation for a complex pole pair . . . . . . . . . . . . . .
29
4.2.2
Complex pole location and response characteristics
. . .
30
4.2.3
The damping factor: . . . . . . . . . . . . . . . . . . .
33
Determining approximate performance measures from a dominant
second-order mode . . . . . . . . . . . . . . . . . . . . . . . . .
35
4.3.1
Rise time from pole locations, 10%-90% . . . . . . . . .
36
4.3.2
Peak time from pole locations . . . . . . . . . . . . . . .
36
4.3.3
Settling time from pole locations . . . . . . . . . . . . . .
36
1.1
Example 1, Golden Nugget Airlines . . . . . . . . . . . . . . . .
1.2
Block diagram
. . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3
Laplace Transforms and Transfer Functions . . . . . . . . . . . .
1.3.1
Laplace transform . . . . . . . . . . . . . . . . . . . . .
1.3.2
Some basic Laplace transform Pairs . . . . . . . . . . . .
5 Design
1.3.3
Transfer Functions are Rational Polynomials . . . . . . .
5.1
Design methods
. . . . . . . . . . . . . . . . . . . . . . . . . .
38
1.3.4
A transfer function has poles and zeros . . . . . . . . . .
10
5.2
Root Locus Design . . . . . . . . . . . . . . . . . . . . . . . . .
38
1.3.5
Properties of transfer functions . . . . . . . . . . . . . . .
12
1.3.6
The Impulse Response of a System . . . . . . . . . . . .
15
2 Closing the loop, feedback control
16
2.1
Analyzing a closed-loop system . . . . . . . . . . . . . . . . . .
20
2.2
Common controller structures: . . . . . . . . . . . . . . . . . . .
21
2.3
Analyzing other loops
22
. . . . . . . . . . . . . . . . . . . . . . .
3 Analysis
4 Working
4.1
37
6 Summary
39
7 Glossary of Acronyms
40
25
with
the
pole-zero
constellation
26
Basics of pole-zero maps, 1st order, p1 = . . . . . . . . . . .
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
26
Page 1
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
Section 1.1.0
1 Differential Eqns, Transfer Functions & Modeling
1.1 Example 1, Golden Nugget Airlines
EE/ME 701: Advanced Linear Systems
Section 1.2.0
1.2 Block diagram
A block diagram is a graphical representation of modeling equations and
their interconnection.
Dynamic systems are governed by differential equations
(or difference equations, if they are discrete-time)
Eqns (1)-(3) can be laid out graphically, as in figure 2.
Example (adapted from Franklin et al. 4thed., problem 5.41, figure 5.79)
Mp(t)
Elevator
servo
v(t)
GNA
7
s + 10
Aircraft
Dynamics
+
Me(t)
+
Mt(t)
Integrator
(t)
s+3
s2 + 4 s + 5
(t)
1
s
(t), (t)
Figure 2: Block diagram of the aircraft dynamics and signals.
Mp(t) Me(t)
Figure 1: Golden Nugget Airlines Aircraft. Me (t) is the elevator-moment (the
control input), M p (t) is the moment due to passenger movements (a
disturbance input), and (t) is the aircraft pitch angle, (t) = (t) .
For example, the aircraft dynamics give:
1
d Mt (t)
d 2 (t)
d (t)
+ 5 (t) = 1
+ 3 Mt (t)
+4
dt 2
dt
dt
Mt (t) = Me (t) + M p (t) ;
1
(t) =
(1)
d (t)
dt
(2)
d Me (t)
+ 10 Me (t) = 7 v(t)
dt
(3)
Where Mt (t) is the total applied moment.
Eqn (1) has to do with the velocity of the aircraft response to Mt (t).
Eqn (2) expresses that the pitch-rate is the derivative of the pitch angle.
And Eqn (3) describes the response of the elevator to an input command
from the auto-pilot.
Main Fact: The Differential Equations come from the physics of the system.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 3
Signal Symbol
Signal Name / Description
Units
v (t)
Voltage applied to the elevator drive
[volts]
Me (t)
Moment resulting from the elevator surface
[N m]
M p (t)
Moment resulting from movement of the passengers
(t)
[N m]
Pitch Rate (angular velocity)
[rad/sec]
Aircraft pitch angle
[radians]
(t)
Table 1: List of signals for the aircraft block diagram.
When analyzing a problem from basic principals, we would also have a list
of parameters.
Parameter Symbol
Signal Name / Description
Value
Units
b0
Motion parameter of the elevator
[N-m/volt]
...
...
...
Table 2: List of parameters for the aircraft block diagram.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 1.3.1
1.3 Laplace Transforms and Transfer Functions
EE/ME 701: Advanced Linear Systems
Section 1.3.2
1.3.2 Some basic Laplace transform Pairs
To introduce the Transfer Function (TF), we need to review the Laplace
transform.
Time Domain Signal
L{f(t)}
Scaled Impulse:
F(s)
f(t)
L-1{F(s)}
Unit step:
F (s) = b
1
s
f (t) = t, t 0
F (s) =
1
s2
n!
Decaying Exponential:
"Time Domain"
Space of all
functions of s
"Frequency Domain"
f (t) = t n , t 0
F (s) =
(t) = b et
F (s) =
f (t) = sin (wt)
Sinusoid:
f (t) = Bc cos (t) + Bs sin (t)
Sinusoidal oscillation:
f (t) = et (Bc cos (t) + Bs sin (t))
Oscillatory Exp. Decay:
s2 +2
c s+Bs
F (s) = sB2 +0
s+2
Bc s+(Bc +Bs )
F (s) = s2 +2 s+2
n
2 + 2
Table 3: Basic Laplace-transform pairs.
Answers:
U(s)
u(t)
Solve
Algebraic
Eqn.
Solve
Diff Eq.
Y(s)
y(t)
2. The LT makes possible the
transfer function.
1/
Area = 1.0
1+(t)
t=0
t=0
t=0
(t) , impulse function
1+ (t), unit step
Unit ramp
e-t
Space of all functions
of time
Space of all
functions of s
t=0
4. Frequency domain: find Guy (s)
for all U (s).
Figure 3: Solve differential equation in time domain or solve an algebraic
equation in s domain.
(Revised: Sep 08, 2012)
t=0
Decaying expon.
Page 5
t=0
A Sinusoid
Part 1A: Controls Review-By-Example
sn+1
b
s+
F (s) =
n =
Why pay attention to Laplace
transforms ?
Part 1A: Controls Review-By-Example
f (t) = b (t)
F (s) =
Higher Powers of t:
Space of all functions
of time
3. Time domain: find y (t) for one
u (t).
F (s) = 1
f (t) = 1+ (t) = 1.0, t 0
Unit ramp:
The Inverse Laplace transform
maps F (s) to f (t).
1. Differential equations in time
domain correspond to algebraic
equations in s domain.
f (t) = (t)
Unit Impulse:
1.3.1 Laplace transform
The Laplace transform (LT) maps
a signal (a function of time) to a
function of the Laplace variable s .
Laplace transform
Oscil. decay exp.
(Revised: Sep 08, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 1.3.2
Two theorems of the Laplace transform permit us to build transfer functions:
1. The Laplace transform obeys superposition and scaling
Given: z (t) = x (t) + y (t)
then
Z (s) = X (s) +Y (s)
z (t) = 2.0 x (t)
then
Z (s) = 2.0 X (s)
Given:
Given: X (s) = L {x (t)} ,
then
d x (t)
dt
Section 1.3.2
Eqns (5)-(7) tell us something about the ratio of the LT of the output to the
LT of the input:
a2 s2 + a1 s + a0 Y (s) = (b1 s + b0) U (s)
(7, repeated)
(b1 s + b0)
Output LT
Y (s)
=
=
= G p (s)
U (s)
(a2 s2 + a1 s + a0)
Input LT
2. The Laplace transform of the derivative of a signal x (t) is s times the LT
of x (t):
EE/ME 701: Advanced Linear Systems
(8)
A Transfer Function (TF) is a ratio of the input and output LTs
Given
G p (s) =
= s X (s)
(b1 s + b0)
,
(a2 s2 + a1 s + a0)
Y (s) = G p (s) U (s)
(9)
Where G p (s) is the TF of the plant.
Putting these rules together lets us find the
transfer function of a system from its governing
differential equation.
u(t)
System
y(t)
Gp(s)
The transfer function is like the gain of the system, it is the ratio of the
output LT to the input LT.
Consider a system that takes in the signal u (t) and gives y (t), governed by
the Diff Eq:
a2
Important fact:
d 2 y (t)
d y (t)
d u (t)
+ a1
+ a0 y (t) = b1
+ b0 u (t)
dt
dt
dt 2
However, the TF depends only on the parameters of the system
(coefficients a2..a0 and b0..b1 in the example), and not on the actual
values of U (s) or Y (s) (or u (t) and y (t)).
(4)
(Notice the standard form: output (unknown) on the left, input on the right.)
Whatever signals y (t) and u (t) are, they have Laplace transforms. Eqn (4)
gives:
Basic and Intermediate Control System Theory are present transfer
function-based design.
By engineering the characteristics of the TF, we engineer the system to
achieve performance goals.
r(t)
d 2 y (t)
d y (t)
+ a1
+ a0 y (t)
L a2
dt 2
dt
d u (t)
= L b1
+ b0 u (t)
dt
a2 s2 Y (s) + a1 sY (s) + a0 Y (s) = b1 sU (s) + b0 U (s)
a2 s2 + a1 s + a0 Y (s) = (b1 s + b0) U (s)
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
(5)
(6)
(7)
Page 7
e(t)
Controller
u(t)
ys(t)
Plant
y(t)
Gp(s)
KcGc(s)
Sensor Dynamics
Hy(s)
Figure 4: Block diagram of typical closed-loop control.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 1.3.3
1.3.3 Transfer Functions are Rational Polynomials
EE/ME 701: Advanced Linear Systems
Section 1.3.4
1.3.4 A transfer function has poles and zeros
A TF has a numerator and denominator polynomial, for example
GNA
G p (s) =
(t), (t)
Mp(t) Me(t)
Figure 5: Golden Nugget Airlines Aircraft. (t) [radians/second] is the
pitch-rate of the aircraft, and Mt (t) is the moment (torque) applied by
the elevator surface.
Consider the differential equation of the Aircraft pitch angle:
1
d 2 (t)
d (t)
d Mt (t)
+4
+ 5 (t) = 1
+ 3 Mt (t)
dt 2
dt
dt
N (s)
2 s2 + 8 s + 6
= 3
D (s) s + s s2 + 4 s + 0
The roots of the numerator are called the zeros of the TF, and the roots of
the denominator are called the poles of the TF. For example:
>> num = [2 8 6]
>> den = [1 2 4 0 ]
num =
2
8
6
den =
1
2
4
>> zeros = roots(num)
zeros =
-3
-1
>> poles = roots(den)
poles =
0
-1.0000 + 1.7321i
-1.0000 - 1.7321i
(10)
From (10) we get the TF, take LT of both sides, and rearrange:
s2 + 4 s + 5 (s) = (s + 3) Mt (s)
(s)
s+3
= 2
Mt (s)
s +4s +5
(11)
We can also use Matlabs system tool to find the poles and zeros
>> Gps = tf(num, den)
Transfer function:
Note:
We can write down the TF directly from the coefficients of the differential
equation.
We can write down the differential equation directly from the coefficients
of the TF.
Transfer functions, such as Eqn (11), are ratios of two polynomials:
s+3
numerator polymonial
(s)
=
:
Mt (s) s2 + 4 s + 5
denominator polynomial
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
>> zero(Gps),
>> pole(Gps),
We call a TF such as (11) a rational polynomial.
Page 9
(12)
%% Build the system object
2 s^2 + 8 s + 6
----------------s^3 + 2 s^2 + 4 s
ans = -3
-1
ans =
0
-1.0000 + 1.7321i
-1.0000 - 1.7321i
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 1.3.4
Interpreting the poles (p1, p2, ..., pn) and zeros (z1, ..., zm)
EE/ME 701: Advanced Linear Systems
Section 1.3.5
1.3.5 Properties of transfer functions
We can use the poles and zeros to write the TF in factored form:
Just as a differential equation can be scaled by multiplying the left and right sides
by a constant, a TF can be scaled by multiplying the numerator and denominator
by a constant.
2 s2 + 8 s + 6
b2 (s z1) (s z2)
=
s3 + 2 s2 + 4 s (s p1) (s p2) (s p3)
2 (s + 3) (s + 1)
=
(s 0) (s + 1 1.732 j) (s + 1 + 1.732 j)
G (s) =
Monic: A TF is said to be monic if an =1. We can always scale a TF to be monic.
If G1 (s) is scaled to be monic, then
b0
(13)
G1 (s) =
s + a1
With a complex pole pair we can do two things,
1. Use a shorthand
G (s) =
2 (s + 3) (s + 1)
s (s + 1 1.732 j)
with b0 = b0/an and a1 = a1 /an .
(Because poles always come in complex conjugate pairs.)
(a) Write the complex pole pair as a quadratic, rather than
G (s) =
1st
order terms
Rational Polynomial Form: A TF is in rational polynomial form when the
numerator and denominator are each polynomials. For example
2 (s + 3) (s + 1)
s (s2 + 2 s + 4)
G p (s) =
The zeros are values of s at which the transfer function goes to zero
(14)
An example of a TF not in rational polynomial form is:
The poles are values of s at which the transfer function goes to infinity
We can plot a pole-zero map:
2 s2 + 8 s + 6
s3 + s s2 + 4 s + 0
G3 (s) =
PZ Map
2 (s + 3)/ (s)
(s2 + 2 s + 4)/ (s + 1)
(15)
imaginary
>> Gps=tf([2 8 6], [1 2 4 0])
>> pzmap(Gps)
By clearing the fractions within the fraction, T3 (s) can be expressed in rational
polynomial form
1
0
G3 (s) =
1
2
4
2 (s + 3)/ (s)
(s2 + 2 s + 4)/ (s + 1)
(s) (s + 1)
2 s2 + 8 s + 6
2 (s + 3)(s + 1)
= 3
= 2
(s) (s + 1) (s + 2 s + 4)(s) s + 2 s2 + 4 s
Note the middle form above, which can be called factored form.
2
real
Figure 6: Pole-Zero constellation of aircraft transfer function.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 11
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 12
EE/ME 701: Advanced Linear Systems
Section 1.3.5
General Form for a rational polynomial transfer function
for a rational polynomial transfer function is
G (s) =
sm + b
The general form
sm1 + b
bm
m1
1 s + b0
ansn + an1 sn1 + + a1 s + a0
1.3.5 Properties of Transfer function (continued)
(16)
So, for example, the aircraft transfer function from elevator input to pitch
rate gives a type 0 system:
(s)
s+3
=
Me (s) s2 + 4 s + 5
m = number of zeros, n = number poles
A TF with m n is said to be proper.
poles : s = 2 1 j ,
type : 0
But the TF from the elevator to the pitch angle gives a type I system:
When m < n the TF is said to be strictly proper
s+3
(s)
=
Me (s) s (s2 + 4 s + 5)
Example of a TF that is not proper:
2s2 + 5 s + 4
s+1
Section 1.3.5
System Type: A property of transfer functions that comes up often is the system
type. The type of a system is the number of poles at the origin.
Proper and Strictly Proper Transfer Functions:
G4 (s) =
EE/ME 701: Advanced Linear Systems
note: m = 2,
n=1
poles : s = 0, 2 1 j ,
type : I
If we put a PID controller in the loop, which adds a pole at the origin, the
system will be type II.
such a TF can always be factored by long division:
G4 (s) =
2s2 + 5 s + 4 2 s (s + 1) 3 s + 4
3s +4
1
=
+
= 2 s+
= (2s + 3)+
s+1
(s + 1)
s+1
s+1
s+1
A non-proper TF such as G4 (s) has a problem: As s = j j , the gain
goes to infinity !
Since physical systems never have infinite gain at infinite frequency,
physical systems must have proper transfer functions.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 13
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 2.0.0
EE/ME 701: Advanced Linear Systems
Section 2.0.0
2 Closing the loop, feedback control
1.3.6 The Impulse Response of a System
When we have a system such as in figure 7, the Laplace transform of the
output (a signal) is given by
u(t)
Gp(s)
r(t) + u(t)
y(t)
y(t)
Gp(s)
sensor
Y (s) = G p (s) U (s)
(17)
A unit impulse input is a very short pulse with an area under the curve of
the pulse of 1.0.
Since the Laplace transform of a unit-impulse is 1, then the Laplace
transform of the impulse response is the transfer function
Open Loop
Closed Loop
Figure 8: A plant with TF G p (s) in open and closed loop. Closed loop requires a
sensor.
Feedback is the basic magic of controls. A feedback controller can
Make an unstable system stable . . . . . . . . . . . . . . . . . . Helicopter autopilot
Make a stable system unstable . . . . . . . . . . . . . . . . Early fly-ball governors
Yimp (s) = G p (s) Uimp (s) = G p (s) 1 = G p (s)
Make a slow system fast . . . . . . . . . . . Motor drive, industrial automation
Make a fast system slow . . . . . . . . F16 controls, approach / landing mode
u(t)
Make an inaccurate system accurate . . . . . . . . . . . . . . . . . . . . machine tools
y(t)
t
u(t): impulse
U(s)=1
The magic comes because closing the loop changes the TF
t
System
Gp(s)
Open loop:
y(t): impulse response
Y(s)=Gp(s)
Closed loop:
Figure 7: The transfer function is the Laplace transform of the impulse response.
The connection between the impulse response and TF can be used to
determine the TF
Apply an impulse and measure the output, y (t). Take the LT and use
G p (s) = Y (s).
The connection between the impulse response and TF helps to understand
the mathematical connection between an LT and a TF
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Y (s)
= G p (s)
U (s)
Page 15
Y (s) = G p (s) (R (s) Y (s)) = G p (s) R (s) G p (s)Y (s)
Y (s) (1 + G p (s)) = G p (s) R (s)
G p (s)
Y (s)
=
R (s)
1 + G p (s)
(18)
Y (s)
Forward Path Gain
=
= Try (s)
R (s)
1 + Loop Gain
(19)
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 16
Mt(t)
+
s+3
s +4s+5
(t)
Figure 9: Simple feedback of aircraft pitch angle.
s+3
G p (s)
(s)
s (s2 +4 s+5)
=
=
R (s) 1 + G p (s) 1 + 2 s+3
s (s +4 s+5)
(20)
Eqn (20) is not in rational polynomial form, so
s+3
(s)
s (s2 +4 s+5)
=
R (s) 1 + 2 s+3
s (s +4 s+5)
Analyzing the response
Gps
= tf([1 3], [1 4 5 0])
Try = tf([1 3], [1 4 6 3])
figure(1), clf
[Y_open, Top]
= step(Gps, 6);
[Y_closed, Tcl] = step(Try, 6);
plot(Top, Y_open, Tcl, Y_closed)
xlabel(t [seconds]);
ylabel(\Omega, pitch-rate)
title(Open- and closed-loop)
text(3, 1.6, Open-loop, rotation, 45)
text(4, 0.8, Closed-loop)
SetLabels(14)
print(-deps2c, OpenClosedResponse1)
s s2 + 4 s + 5
s+3
=
(21)
s (s2 + 4 s + 5) s (s2 + 4 s + 5) + (s + 3)
The closed-loop TF is still not quite in Rat Poly form, here is the final step:
(s)
s+3
=
R (s) s3 + 4 s2 + 6 s + 3
(22)
Open and closedloop response, aircraft
3.5
The open-loop
system is type I
The closed-loop
system is type 0
r(t)
(s)
2.5
1.5
1
Closedloop
(t)
0.5
R(s)
0
0
Figure 10: Block with r (t) as input and (t) as output.
Part 1A: Controls Review-By-Example
The response
completely
changes !
, pitchrate
r(t)
(Revised: Sep 08, 2012)
Section 2.0.0
oo
p
Example, wrapping a simple feedback loop around the aircraft dynamics
EE/ME 701: Advanced Linear Systems
n
l
Section 2.0.0
O
pe
EE/ME 701: Advanced Linear Systems
3
t [seconds]
Figure 11: Open and Closed loop response of the aircraft, the two responses have
very different characteristics.
Page 17
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 2.0.0
e(t)
Kc
Mt(t)
Section 2.1.0
2.1 Analyzing a closed-loop system
Introduce a proportional controller gain
r(t)
EE/ME 701: Advanced Linear Systems
s+3
s2 + 4 s + 5
A typical, basic loop (such as velocity PI control of a motor drive) has 3
components:
(t)
1. Plant (thing being controlled)
Figure 12: Feedback for aircraft pitch control, with P-type gain Kc .
Look at Kc = 1.0, 3.0, 10, 0
Kc = 1; Try1 = tf(Kc*[1 3], [1 4 5
Kc = 3; Try2 = tf(Kc*[1 3], [1 4 5
Kc = 10; Try3 = tf(Kc*[1 3], [1 4 5
figure(1), clf
...
plot(Top, Y_open, Tcl, Y_closed1,
...
2. Controller or compensator (usually a computer, a often PLC for motor
drives)
0]+Kc*[0 0 1 3])
0]+Kc*[0 0 1 3])
0]+Kc*[0 0 1 3])
3. A sensor
r(t)
e(t)
Controller
KcGc(s) =
Nc(s)
Kc D (s)
c
u(t)
Plant
Gp(s) =
y(t)
Np(s)
Dp(s)
Tcl, Y_closed2, Tcl, Y_closed3)
Sensor Dynamics
Ny(s)
Hy(s) = D (s)
y
ys(t)
Open and closedloop response, aircraft
3.5
Figure 14: Basic loop with a plant, compensator and sensor.
lo
op
The TF is given as
pe
2
O
System gets less
stable as Kc
increases
2.5
, pitchrate
System gets much
faster as Kc
increases
Try (s) =
Kc =10
1.5
Kc =3
Kc Gc G p
Y (s)
Forward Path Gain
=
=
R (s)
1 + Loop Gain
1 + Kc Gc G p Hy
1
Closedloop, Kc =1
Often, for the controls engineer the plant, G p (s), is set (e.g., the designer of
a cruise-control does not get to change the engine size).
0.5
0
0
3
t [seconds]
Figure 13: Open and Closed loop response of the aircraft, with Kc = 1.0,
Kc = 3.0, and Kc =10.0 .
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 19
As a controls engineer, we get to pick Gc (s) and maybe influence Hy (s)
(e.g., by convincing the project manager to spring $$$ for a better sensor).
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 2.2.0
2.2 Common controller structures:
PI
PID
Proportional-Derivative
Prop.-Integral
Prop.-Int.-Deriv.
Gc (s) =
k p s +ki
s
Section 2.3.0
2.3 Analyzing other loops
PD
Gc (s) = kd s + k p
EE/ME 701: Advanced Linear Systems
Gc (s) =
kd s2 +k p s +ki
s
d(t)
Lead-Lag
Disturbance Filter
Nd
Gd(s) = D
d
(s+z ) (s+z )
Gc = Kc (s+p11 ) (s+p22 )
r(t)
Input Shaping
Hr(s)
Common Applications
PID: Many many places, Astrom has estimated that 80% of controllers are
PID (good, speed accuracy, stability)
Lead-Lag: Used where a pole at the origin is unacceptable, can be as good
as PID (notice, 5 parameters rather than 3)
e(t)
Controller
N
KcGc(s) = Kc Dc
c
uc(t)
Plant
up(t)
Gp(s) =
y(t)
Np
Dp
Sensor Dynamics
Ny
Hy(s) = D
y
ys(t)
PI: Velocity control of motor drives, temperature control (good speed and
accuracy, acceptable stability)
PD: Position control where high accuracy is not required (good speed and
stability, so-so accuracy)
Figure 15: Basic loop with a disturbance input, d (t) , and sensor noise, Vs (t)
added.
In some cases we may want to consider additional inputs and outputs.
Many systems have a disturbance signal that acts on the plant, think of
wind gusts and a helicopter autopilot.
All systems have sensor noise.
Any signal in a system can be considered an output. For example, if we
want to consider the controller effort, uc (t), arising due to the reference
input
Tru (s) =
Hr (s) Kc Gc (s)
Uc (s) Forward Path Gain
=
=
R (s)
1 + Loop Gain
1 + Kc Gc (s) G p (s) Hy (s)
(23)
If we wanted to consider the error arising with a disturbance, we would have
Tde (s) =
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 21
Gd (s) G p (s) Hy (s) (1)
E (s)
=
D (s) 1 + Kc Gc (s) G p (s) Hy (s)
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Vs(t)
(24)
Page 22
EE/ME 701: Advanced Linear Systems
Section 2.3.0
2.3 Analyzing other loops (continued)
As a final example, lets consider the output arising with sensor noise
Tvy (s) =
Y (s) Hy (s) (1) Kc Gc (s) G p (s)
=
Vs (s)
1 + Kc Gc (s) G p (s) Hy (s)
(25)
The example transfer functions, Eqns (23), (24) and (25) show some
interesting properties. The TFs are repeated here (omitting the many (s)s)
Try (s) =
Hr Kc Gc G p
1 + Kc Gc G p Hy
Tru (s) =
Hr Kc Gc
1 + Kc Gc G p Hy
Ted (s) =
Gd G p Hy (1)
1 + Kc Gc G p Hy
Tvy (s) =
Hy (1) Kc Gc G p
1 + Kc Gc G p Hy
EE/ME 701: Advanced Linear Systems
Section 2.3.0
If we consider what happens as Kc , we can see what happens for very
high gain. For this, assume that Hr (s) = 1.0 and Gd (s) = 1.0, since these
two terms merely pre-filter inputs.
When Kc , 1 + Kc Gc G p Hy Kc Gc G p Hy , so
Try (s)
Kc Gc G p
1
=
Kc Gc G p Hy Hy
Tde (s)
Gd G p Hy (1) Gd v
=
Kc Gc G p Hy
Kc Gc
Tru (s)
,
Kc Gc
1
=
Kc Gc G p Hy G p Hy
Tvy (s)
Hy (1) Kc Gc G p
= 1
Kc Gc G p Hy
Try (s) 1/Hy (s) shows that the TF of the plant can be compensated, it
disappears from the closed-loop TF as Kc .
Try (s) 1/Hy (s) also shows that the TF of the sensor can not be
compensated. If the characteristics of Hy (s) are bad (e.g., a cheap sensor)
there is nothing feedback control can do about it !
Tde (s) 1/Kc Gc shows that disturbances can be compensated, as Kc ,
errors due to disturbances go to zero ;)
The denominators are all the same
The poles are the same for any input/output signal pair
The stability and damping (both determined by the poles) are the same
for any signal pair
The numerators are different
The zeros are in general different for each input/output signal pair
Since the numerator help determine if the signal is small or large, signals
may have very different amplitudes and phase angles
Tru (s) 1/G p Hy shows that U1 (s) does not go up with Kc , and also, if the
plant has a small gain (G p (s) is small for some s = j ) then a large control
effort will be required for a given input.
Tvy (s) 1 shows that there is no compensation for sensor noise. If there
is sensor noise, it is going to show up in the output !
Summary:
Feedback control can solve problems arising with characteristics of the
plant, G p (s), and disturbances, d (t).
Feedback control can not solve problems with the sensor, Hy (s), or
sensor noise, vs (t) .
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 23
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 24
EE/ME 701: Advanced Linear Systems
Section 3.0.0
EE/ME 701: Advanced Linear Systems
3 Analysis
Section 4.1.0
4 Working with the pole-zero constellation
4.1 Basics of pole-zero maps, 1st order, p1 =
Analysis is determining system performance based on a model.
You have seen that:
The performance criteria any controls engineer should be aware of are seen
in table 4.
Each real pole is associated with a mode of the system response.
The criteria fall into 3 groups
A real pole gives the terms of Eqn (27), as seen in figure 16.
In EE/ME 574 we will be using all 18 of the criteria listed
Y (s) =
Many performance measures are introduced in first semester controls (we
will review them here). Those with (*) are introduced in 2nd semester
controls.
As seen in table 4, some performance measures are evaluated in the time
domain, and others in the frequency domain.
Amplitude
1
0
(26)
Impulse response
PoleZero Map
()
y (t) = C1 et
2
Imaginary Axis [sec1]
Speed
Degree of Stability Accuracy
Time
Rise Time
Stable / Unstable
Steady-State Error
Domain
Settling Time
Overshoot
ISE ()
IAE ()
Peak Time
Damping Factor
Frequency Pole Locations
Pole Locations
Disturbance Rejection
Noise Rejection ()
Domain
Bandwidth ()
Phase Margin ()
()
()
(Or S-plane) Cross-over Freq.
Gain Margin
Tracking Error ()
Table 4: A range of controller specifications used in industry.
Note: () marks items developed in 2nd semester controls.
3
C1
=
s+2 s+
Faster
h(t) =
t
t/
2 t
e
=e =e
C1 / e
Splane
2
3
2
1
0
1
Real Axis [sec ]
0
0
1
2
Time (secs)
Figure 16: First order pole and impulse response.
Depending on the performance measure, one of four methods of evaluation
is used:
1. Evaluated directly from the transfer function (e.g., steady state error)
2. Evaluated by looking at the system response to a test signal (e.g., trise )
3. Evaluated from the pole-zero constellation (e.g., stability, settling time)
4. Evaluated from the bode plot (e.g., phase margin)
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 25
A real pole has these characteristics:
y (t) et/ where [sec] = 1/ is the time constant.
Further to the left indicates a faster response (smaller ).
The pole-zero constellation does not show either the KDC or the Krlg of
the TF or the amplitude of the impulse response.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 26
EE/ME 701: Advanced Linear Systems
Section 4.1.1
Example 1: Shifting poles to the left accelerates transient decay
And we have seen for second order:
Two example first-order responses are shown.
X Case:
Case:
= 16 [sec1]
Each complex pole pair gives a mode of the system response.
= 1/4 [sec]
A complex pole pair gives the terms of Eqn (27), as seen in figure 19.
= 1/16 [sec]
Using the Laplace transform pair
F (s) =
Bc s + (Bc + Bs )
f (t) = et (Bc cos (t) + Bs sin (t))
s2 + s s + 2n
one finds
Y (s)
Faster Decay
5
15
10
5
0
Real Axis [sec1]
y (t)
Figure 17: A change in pole location changes the decay rate and damping.
2 s + 14
b1 s + b0
=
s2 + 2 s + (2 + 2) s2 + 3 s + 18.25
PoleZero Map
0.5
0.5
2
t
0.5
Time (secs)
0
0
0.5
Time (secs)
6 4 2
0
Real Axis [sec1]
t/
=e
1.5t
=e
0
1
5
8
0
0
Impulse response
3
Splane
System step response
1.5
= 0.06 [sec]
Amplitude
Amplitude
Imaginary Axis [sec ]
X System step response
1.5
= 0.25 [sec]
(27)
A et cos (t + ) = 3.4 e1.5t cos (4t 53.97o) (28)
Amplitude
Imaginary Axis [sec1]
PZ map showing two systems
5
20
Section 4.2.0
4.2 Basis of pole-zero maps, 2nd order, p1,2 = j
4.1.1 Real Pole location and response characteristics
4 [sec1]
EE/ME 701: Advanced Linear Systems
2
0
3.40 cos(4t 54.0o)
2
4
Time (secs)
Figure 19: Second order pole and impulse response.
Figure 18: A change in changes the time constant.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 27
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 28
EE/ME 701: Advanced Linear Systems
Section 4.2.1
4.2.1 Notation for a complex pole pair
*
p1, p1
=1.5 j 4
p1
X Case: 1
Case: 4
[sec1]
PO
4 [rad/sec] 4.12 [rad/sec] 0.25
[Dim less]
44%
4 [rad/sec] 5.66 [rad/sec] 0.71
[Dim less]
4%
PZ map with sgrid (indicates and n)
PZ map showing two systems
1
0
Real Axis [sec1]
Description
Given by
Units
Decay Rate
Damped Nat. Freq.
Natural Freq.
Pole Angle
Damping Factor
p1 = + j
p1 = + j
2n = 2 + 2
= atan2 (, )
= /n
[sec1]
[rad/sec]
[rad/sec]
[deg]
Dimless, []
4
2
Faster Decay
0
2
4
6
8
4 2
0
2
1
Real Axis [sec ]
0.7
2
0
Amplitude
Polar
2
4
0.9
0.7 0.5 0.3
6 4 2
0
2
1
Real Axis [sec ]
6
8
System step response
X System step response
1.5
p1 = + j
p1 = n (90 + )
= n
= sin ()
p
= 1 2 n
= / n
p
n 1 2
H (s) =
H
(s)
=
s2 + 2 n s + 2n
(s + )2 + 2
Table 6: The terms of table 5 relate to rectangular or polar form for the poles.
0.5 0.3
4 0.9
Figure 21: A change in changes the decay rate and damping.
Table 5: Factors derived from the location of a complex pole.
(Note: Franklin et al. often use , d and .)
Rectangular
6
Imaginary Axis [sec1]
Splane
p1
= 0.24
46 % Overshoot
0.5
0
0
1.5
Amplitude
Figure 20: Complex pole pair with n and defined.
or
or d
n
or
[sec1]
6
3
Term
Example 1: Shifting poles to the left accelerates transient decay
Imaginary Axis [sec1]
Imaginary Axis [sec ]
6
4
Section 4.2.2
4.2.2 Complex pole location and response characteristics
A complex pole pair can be expressed in polar or rectangular coordinates:
PZ Map,
EE/ME 701: Advanced Linear Systems
2
4
Time (secs)
= 0.71
4 % Overshoot
0.5
0
0
2
4
Time (secs)
Figure 22: Step response: a change in , is unchanged.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 29
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 30
Section 4.2.2
Example 2: Shifting poles out vertically increases oscillation frequency
X Case: 4 [sec1]
4 [rad/sec]
4%
Case: 4 [sec1] 16 [rad/sec] 16.49 [rad/sec] 0.24 [Dim less] 44%
Faster
Oscillation
0
10
20
0.5 0.3
0.9
20
12
10
0.9
20
20 10
0
10
Real Axis [sec1]
30
0.5
0
0
1
Time (secs)
4%
16 [rad/sec] 22.63 [rad/sec] 0.71
[Dim less]
4%
5.66 [rad/sec]
10
0
10
20
0.7 0.50.3
20
0.9
10
0
28 20 12
= 0.24
46 % Overshoot
0.5
0.9
20
0.7 0.50.3
30 20 10
0
10
1
Real Axis [sec ]
System step response
1.5
= 0.71
4 % Overshoot
0.5
0
0
1
Time (secs)
= 0.71
4 % Overshoot
0.5
= 0.25 [sec]
1
Time (secs)
10
X System step response
0
0
0.71
4 [rad/sec]
20
1.5
Amplitude
1.5
= 0.71
4 % Overshoot
PO
Figure 25: A radial change in pole location changes the decay rate and oscillation
frequency, but not the damping.
System step response
Amplitude
Amplitude
1.5
[Dim less]
30 20 10
0
10
1
Real Axis [sec ]
0.7 0.5 0.3
20 10
0
10
Real Axis [sec1]
Figure 23: A change in changes the oscillation frequency and damping.
X System step response
[sec1]
PZ map showing two systems PZ map with sgrid (indicates and n)
10
28
[sec1]
Case: 16
Imaginary Axis [sec ]
0.7
20
Imaginary Axis [sec1]
Imaginary Axis [sec1]
20
30
X Case:
PZ map with sgrid (indicates and n)
PZ map showing two systems
10
PO
0.71 [Dim less]
5.66 [rad/sec]
Example 3: Shifting poles out radially rescales time
Section 4.2.2
Imaginary Axis [sec ]
EE/ME 701: Advanced Linear Systems
Amplitude
EE/ME 701: Advanced Linear Systems
= 0.06 [sec]
2
0
0
0.2
0.4
Time (secs)
Figure 26: Maintaining , time is rescaled.
Figure 24: Step response: a change in .
Note: The S-plane has units of [sec1 ].
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 31
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 32
EE/ME 701: Advanced Linear Systems
Section 4.2.3
4.2.3 The damping factor:
b0
b0
=
s2 + s a1 + a0 s2 + 2 n s + 2n
(29)
value
System characteristic
<0
=0
Unstable
Marginally Stable (poles on imaginary axis)
0<<1
giving:
Section 4.2.3
A full list of what we can learn from is seen in table 7.
Plugging = /n back into the 2nd order form gives:
H (s) =
EE/ME 701: Advanced Linear Systems
a1
a1
=
=
2 n 2 a0
(30)
is defined by Eqn (30) for either real poles: ( 1.0), or
a complex pole pair: (0.0 < < 1.0).
Stable Complex Pole Pair
0.5 < 1.0 Typical value for a well designed controller
=1
>1
Critical Damping (repeated real poles)
Over Damped (two separate real poles)
Table 7: Ranges of damping factor.
As illustrated in figure 24, above, on the range 0.0 < < 1.0, the damping
factor relates to percent overshoot. For a system with two poles and no
zeros, the percent overshoot is given by Eqn (31) and plotted in figure 27
q
P.O. = 100 e / 1 2
(31)
S-plane
Stable Region
( >0 )
Unstable Region
( <0 )
Marginally Stable Region ( =0 )
100
Figure 28: Damping factor and stability.
90
80
Percent Overshoot
70
60
50
40
30
20
10
0
0
0.1
0.2
0.3
0.4
0.5
0.6
Damping factor [.]
0.7
0.8
0.9
Figure 27: Percent overshoot versus damping fact. Exact for a system with two
complex poles and no zeros, and approximate for other systems.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 33
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 34
EE/ME 701: Advanced Linear Systems
Section 4.3.0
EE/ME 701: Advanced Linear Systems
4.3 Determining approximate performance measures from a
Section 4.3.3
4.3.1 Rise time from pole locations, 10%-90%
dominant second-order mode
1st order:
Returning to performance measures, in section 3 weve seen that these are
defined from the step response.
Rise time
Settling time
Peak time
Overshoot
Step response of complex pole pair
1.5
Step response of complex pole pair
2.5
Output y(t)
Output y(t)
ss
0.5
| |
T : 1090%
r
Rise Time
yp
r (1)
ss
2%
4 |
6
8
Ts: 98% Settling Time
tr =
1.8
n
y (1)
ss
1.5
Time [sec]
(32)
4.3.3 Settling time from pole locations
Settling time is sometimes defined as the time to approach within 4% or 2% or
even 1% of the steady-state value. These give slightly different definitions of Ts .
The one will will use (corresponding approximately to 2%) is:
|
tp
90
y (0)
0
ss
tp =
Mp
0.5
rss(0) y10
1.8
4.3.2 Peak time from pole locations
Overshoot
1
tr =
2nd order:
T : 1090% Rise Time
ts =
0 |
t
10
| 1
t
90
2
Time [sec]
(33)
Figure 29: Quantity definitions in a CL step response.
While there are not equations that give these measures exactly for any
system other than 2 poles and no zeros. But for this special case
T (s) =
b0
s2 + 2 n s + 2n
We can derive the relations below.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 35
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 36
EE/ME 701: Advanced Linear Systems
Section 5.0.0
5 Design
EE/ME 701: Advanced Linear Systems
Section 5.2.0
5.1 Design methods
The major design methods are:
In some sense, Design is the opposite of Analysis
Root locus
Analysis
Speed, Stability: determine by determining pole locations
Accuracy: increase the system type, check the SSE
Completed
Controller
Design
Performance
Specifications
Frequency response
()
Speed: Bandwidth and Cross-over frequency directly from bode plot
Stability: Phase margin, Gain margin directly from bode plot
Accuracy: tracking accy., disturbance rejection directly from bode plot
Design
State Space methods
Figure 30: Design = Analysis1
In Analysis, we use mathematical methods to determine performance
specifications from a completed controller design (all structure and
parameters specified).
Design using state-space design methods, check speed, stability and
accuracy from the step response
5.2 Root Locus Design
Devised by Walter R. Evans, 1948 (1920 1999)
In Design, we use whatever method works !
Mathematical
W.R. Evans, Control system synthesis by
root locus method, Trans. AIEE, vol. 69,
pp. 6669, 1950.
Gut feeling
Trial and error
(Amer. Institute of Elec. Engs became
IEEE in 1963)
Calling a colleague with experience
to determine a controller structure and parameters to meet performance
goals.
Part 1A: Controls Review-By-Example
()
(Revised: Sep 08, 2012)
Page 37
Evans was teaching a course in controls,
and a student (now anonymous) asked a
question about an approximation.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 38
EE/ME 701: Advanced Linear Systems
Section 6.0.0
6 Summary
EE/ME 701: Advanced Linear Systems
Section 7.0.0
7 Glossary of Acronyms
Dynamic systems are governed by differential equations
LT: Laplace Transform
Every input or output signal of the system has a unique Laplace transform
TF: Transfer Function
For linear systems,
FPG: Forward path gain
The ratio of Laplace transforms, however, does not depend on the signals.
LG: Loop Gain, (also Gl (s))
The ratio depends only on properties of the system. We call it the transfer
function. For example:
RHP: Right-half of the S plane (unstable region)
LHP: Left-half of the S plane (stable region)
b1 s + b0
Guy (s) = 2
s + a1 s + a0
(34)
PO: Percent Overshoot
Transfer function gives us many results
The pole locations tells us the stability and damping ratio
We can get approximate values for rise time, settling time and other
performance measures
Control system analysis is the process of determining performance from a
system model and controller design
Control system
* Steady-state error
* Step response
LF: Low frequency (as in LF gain).
frequency.
Frequencies below the cross-over
HF: High frequency (as in HF gain).
frequency.
Frequencies above the cross-over
CL: Closed loop
OL: Open loop
We have various tools for evaluating performance, including
* Pole locations
SSE: Steady-state error
* Bode plot
P, PD, PI, PID: Proportional-Integral-Derivative control, basic and very common
controller structures.
Design =Analysis1
it is the process of determining a controller design given a system model
and performance goals.
The root locus method is one method for controller design.
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 39
Part 1A: Controls Review-By-Example
(Revised: Sep 08, 2012)
Page 40
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Building Linear System Models
Contents
1 System modeling,
classical vs. state-variable modeling
1.1
1.2
1.3
1.4
State variable modeling: . . . . . . . . . . . . . . . . . . . . . .
1.2.1
Signals that are not states or inputs . . . . . . . . . . . . .
1.2.2
Writing the n first-order differential equations in standard
form . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Standard form for the linear, time-invariant state-variable
model: . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
Why consider linear state-variable models ?
(when algebraic form is just fine) . . . . . . . . . . . . . . . . .
11
A quick history of state variable modeling . . . . . . . . . . . . .
12
2 Formal properties of systems
21
3.6
Lumped Parameter system . . . . . . . . . . . . . . . . . . . . .
22
3.7
Continuous-time versus sampled (discrete-time) signals and systems 23
3.8
Discrete-time (sampled) signals and systems . . . . . . . . . . . .
24
3.9
Continuous signals and systems, continuity in the mathematical
sense . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
25
3.10 Quantized signal . . . . . . . . . . . . . . . . . . . . . . . . . .
26
3.11 Deterministic Signals and systems . . . . . . . . . . . . . . . . .
27
3.12 A Note on units . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
4 State, revisited
14
4.1.2
4.1.3
For direct application of Superposition and Scaling, state
must be zero . . . . . . . . . . . . . . . . . . . . . . . .
30
Definition of a Linear System considering non-zero state
(e.g., DeCarlo definition 1.8) . . . . . . . . . . . . . . . .
31
Time-invariant system, considering non-zero state
32
. . . .
6 Steps of building a state-variable model (for a linear system)
35
18
7 Examples with state-variable models
37
2.2
Simple example system
3.1
Admissible signal . . . . . . . . . . . . . . . . . . . . . . . . . .
18
3.2
Linear System (without internal variables) . . . . . . . . . . . . .
19
3.3
Time-invariant system (without internal variables) . . . . . . . . .
20
3.4
Causal (non-anticipative) system . . . . . . . . . . . . . . . . . .
21
(Revised: Sep 10, 2012)
4.1.1
30
16
15
3 Classification of Systems and Signals
Modified definition of linearity, considering state . . . . . . . . .
33
A system Maps inputs to outputs . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
4.1
28
5 Standard Notation for State Model
2.1
Part 5: Models of Linear Systems
Realizable system . . . . . . . . . . . . . . . . . . . . . . . . . .
Modeling with a single, higher-order differential equation
(sometimes called classical modeling) . . . . . . . . . . . . . . .
1.2.3
3.5
Page 1
7.1
Example 1, an electrical circuit
. . . . . . . . . . . . . . . . . .
37
Building and exercising the circuit model . . . . . . . . .
43
Example 2, a mechanical system . . . . . . . . . . . . . . . . . .
45
7.2.1
53
7.1.1
7.2
Building and exercising the suspension model
Part 5: Models of Linear Systems
. . . . . .
(Revised: Sep 10, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
7.2.2
7.3
7.4
EE/ME 701: Advanced Linear Systems
Conclusions, Quarter Suspension Example . . . . . . . .
59
Example 3, state-variable model from a differential equation . . .
60
7.3.1
Steps of deriving the state-variable model . . . . . . . . .
61
7.3.2
Building and exercising a differential equation model
. .
64
State-variable model and simulation diagram . . . . . . . . . . .
66
8 Some basic operations with state-variable models
8.1
73
Deriving the transfer function from the state-variable model . . . .
73
8.1.1
Interpreting the transfer function
. . . . . . . . . . . . .
74
8.1.2
DC gain of a state-variable model . . . . . . . . . . . . .
77
1 System modeling,
classical vs. state-variable modeling
To introduce state modeling, lets first look at an example. Consider the
RLC circuit of figure 1. The parameters and signals of the circuit are:
Parameters
8.2
Interpreting D . . . . . . . . . . . . . . . . . . . . . . . .
77
Coordinate transformation of a State Variable Model . . . . . . .
78
8.2.1
81
Example coordinate transformation . . . . . . . . . . . .
9 State-variable Feedback control
Determination of a new model with state feedback
. . . . . . . .
85
9.2
State-variable feedback example: Inverted pendulum . . . . . . .
87
9.2.1
Designing a pole placement controller
. . . . . . . . . .
90
9.2.2
LQR Design . . . . . . . . . . . . . . . . . . . . . . . .
92
volts
amp
volts
amp/sec
amps
volt/sec
Units
Signals
Units
Vs (t)
[volts]
VL (t)
[volts]
iS (t)
[amps]
iL (t)
[amps]
VR (t)
[volts]
VC (t)
[volts]
iR (t)
[amps]
iC (t)
[amps]
iR
+
is
Vs(t)
10 Conclusions
Signals
Table 1: List of parameters and signals.
84
9.1
Units
L
8.1.3
Section 1.0.0
iL
+
L
iC
+
C
VC(t)
95
Figure 1: RLC Circuit with voltages and currents marked.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 3
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Vs(t)
iL
+
EE/ME 701: Advanced Linear Systems
Section 1.1.0
1.1 Modeling with a single, higher-order differential equation
iR
is
Section 1.0.0
iC
(sometimes called classical modeling)
VC(t)
Develop the nth order differential equation, write:
iR (t) iL (t) iC (t) = 0
1
(Vs (t) VC (t)) iL (t) CVC (t) = 0
R
1
1
1
VC (t) + iL (t) =
Vs (t)
VC (t) +
RC
C
RC
The constituent relations are:
VR (t) = R iR (t)
d iL (t)
VL (t) = L
dt
d VC (t)
iC (t) = C
dt
(1)
Where known quantities are on the right and unknowns on the left.
(2)
In Equation (5) we have unknown signals VC (t) and iL (t) on the left, we
must eliminate one of them, to have one equation in one unknown.
Take the derivative of Eqn (5) to produce iL (t), and use inductor
constituent relation and VC (t) = VL (t) to give:
The continuity constraints are (Kirchhoffs laws for electrical systems):
VR (t) +VL (t) Vs (t) = 0
(3)
VC (t) VL (t) = 0
is (t) iR (t) = 0
iR (t) iL (t) iC (t) = 0
(5)
(4)
1
1
1
VC (t) +
VC (t) + iL (t) =
Vs (t)
RC
C
RC
1
1
1
VC (t) +
VC (t) =
Vs (t)
VC (t) +
RC
LC
RC
(6)
Limitations of classical modeling:
Complexity goes up quickly in the number of variables, 3rd order 4-5X
as much algebra as 2nd order , 4th order 10-20 X more algebra.
Two approaches to modeling this type of dynamic system
1. Classical modeling, with a single high-order differential equation
The significance of the initial conditions is unclear: VC (t0 ) , VC (t0 ) ...,
how to get VC (t0 ) ?
2. State-variable modeling
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 5
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 1.2.0
1.2 State variable modeling:
EE/ME 701: Advanced Linear Systems
Section 1.2.1
1.2.1 Signals that are not states or inputs
Develop n first order differential equations in terms of states and inputs:
Determine the state variables of the system, these are variables that
make up the initial condition for the system (formal definition and more
examples coming later).
For an nth order system, there are n states
For the RLC circuit, there are two states, we can choose them to be
VC (t) and iL (t), giving the state vector:
x (t) =
VC (t)
iL (t)
Systems generally have many signals. Table 2 shows 8 signals for this
simple circuit. Only 2 are states, and 1 is an input, so 5 are just signals
within the system.
A Deep Property of state variable modeling: all of the signals in the
system have a unique expression in terms of the states and inputs.
If no expression exists, the model does not have enough states, and
If there are multiple possible expressions, the model has redundant
states that should be removed from the state vector.
Examples of using Eqns (1) - (4) to write other signals in terms of the
states, VC (t) and iL (t) and the input Vs (t)
We are working toward the state equation:
x (t) = A x (t) + B u (t)
(7)
VL (t) = VC (t)
VR (t) = Vs (t) VL (t) = Vs (t) VC (t)
Starting with differential equations that come directly from the model:
1
VC (t) =
iC (t)
C
(8)
1
iL (t) = VL (t)
L
(9)
Note that the derivative is written on the left, and all signals that
determine the derivative are on the right.
n
o
All signals on the right must be states, VC (t) , iL (t) , or inputs, Vs (t)
iR (t) = VR (t) /R = 1/R (Vs (t) VC (t))
Use the basic modeling equations to re-write the differential equations with
only states and inputs on the right hand side:
1
1
1
iC (t) = (iR (t) iL (t)) =
VC (t) =
C
C
C
1
(Vs (t) VC (t)) iL (t)
R
(10)
1
1
iL (t) = VL (t) = VC (t)
L
L
(11)
Notice that in Eqns (8) and (9), the right hand terms including iC (t) and
VL (t), neither of which is a state or an input.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 7
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 1.2.2
1.2.2 Writing the n first-order differential equations in standard form
EE/ME 701: Advanced Linear Systems
1.2.3 Standard form for the linear, time-invariant state-variable model:
The model equations are written with:
Input
u(t)
Each state derivative on the left
A, B, C, D
(12)
(13)
Standard form for state-variable model equations:
(14)
State Equation: x (t) = A x (t) + B u (t)
(16)
Output Equation: y (t) = C x (t) + D u (t)
(17)
y (t) = C x (t) + D u (t)
Putting Eqns (12) and (13) in the form of (14)
VC (t)
iL (t)
1
RC
1
L
C1
VC
x(t)
Figure 2: Block diagram showing the elements of a state-variable model,
including inputs, outputs and states, and model matrices A, B, C, D.
Put the model equations in matrix / vector form:
x (t) = A x (t) + B u (t)
Output
y(t)
System
The right hand side written with states and inputs as the only signals
1
1
1
VC (t) =
VC (t) iL (t) +
VS (t)
RC
C
RC
1
iL (t) = VC (t)
L
Section 1.2.3
(t) +
1
RC
iL
0
0
h
i VC
(t) + [0] Vs (t)
y (t) =
1 0
iL
Signals
Vs (t)
x (t)
Name
State
Vector
u (t)
(15)
Input
Units
"
[volts]
[amps]
Parameters
#
A=
"
C1
0
"
1
L
B=
[volts]
C=
Vector
y (t)
Output Vector
1
RC
D = [0]
x (t) =
VC (t)
iL (t)
[volts]
[amps]
Name
1
RC
[volts]
where the state vector is:
x (t0 )
Initial Condition :
System Matrix
Input Matrix
Output Matrix
Feed-forward
Units
i
h
volts/sec
sec1
i amp
h
amps/sec
1
sec
volt
" 1 #
h sec i
amps/sec
volt
[]
volts
amp
[]
i i
Matrix
Table 2: List of parameters and signals of the State-Variable Model. Each of the
seven elements has a name, and each has physical units, which depend
on the details of the system.
States are signals (i.e., functions of time)
States have units (like all physical signals)
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 9
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 1.3.0
1.3 Why consider linear state-variable models ?
EE/ME 701: Advanced Linear Systems
Section 1.4.0
1.4 A quick history of state variable modeling
(when algebraic form is just fine)
Prior to the development of state-variable approaches, analytic work focused
on algebraic equations for modeling.
Many types of systems are modeled with sets of linear equations
Plants and controllers for control
Controls, design, economics, statistics, etc.
Multi-variable statistical analysis (e.g. data analysis)
Algebraic approaches tended to give specific results without general
theory
Business and economic models
Example, it was not known until publication of Masons rule in 1953
when control system equations could be simplified.
Signal processing
And more
In fact, the only models which can be worked out in a general way are linear.
Many properties of the system made only clear by state-variable
approach:
Many truly nonlinear results are known, but they are applicable only in
specific cases.
Controllability and Observability
Nonlinear systems are often analyzed by linearizing them about an
operating point ...
Coordinate transformations and modal coordinates
Interest in state variable modeling for analysis and design of systems
accelerated with the early work of Kalman, Bellman and others.
To apply the methods of linear analysis.
Any time there are two or more variables, vectors and matrices become
powerful tools for analysis
Earlier algebraic approaches do not extend well to multi-variable systems
(such as multi-input, multi-output (MIMO) systems).
Powerful mathematical results for linear systems are applicable to problems
from all domains. !!!! Geometric interpretation !!!!
Vector spaces bring us notions of size, orthogonal, independence,
sufficiency, degrees of freedom (DOF), which apply equally well across
all domains.
Part 5: Models of Linear Systems
Optimal control (and Kalman filtering)
(Revised: Sep 10, 2012)
Page 11
Rudolf E. Kalman, On the General Theory of Control Systems, Proc.
1st Inter. Conf. on Automatic Control, IFAC: Moscow , pp 481-93, 1960.
Richard E. Bellman, Dynamic Programming, Princeton University Press,
1957.
Rudolf E. Kalman, Mathematical Description of Linear Systems,
SIAM J. Control, vol. 1, pp 152-192, 1963.
Vasile M. Popov, On a New Problem of Stability for Control Systems,
Automatic Remote Control, 24(1):1-23, 1963.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 12
EE/ME 701: Advanced Linear Systems
Section 1.4.0
The computational tools to work with vectors and matrices were being
introduced at about the same time
EE/ME 701: Advanced Linear Systems
Section 2.0.0
2 Formal properties of systems
Definition: A system is something which maps inputs to outputs.
Wilkinson, James Hardy, Rounding errors in algebraic processes,
Englewood Cliffs:Prentice-Hall, 1963.
George E. Forsythe and Cleve B. Moler, Computer solution of linear
algebraic systems, Englewood Cliffs: Prentice-Hall, 1967.
Cleve Moler went on to co-found Mathworks (Matlab); and is
Mathworks chief scientist today.
u(t)
y(t)
Aircraft
Outputs:
- Pitch
- Roll
- Yaw
- Velocity
Inputs:
- Aileron
- Rudder
- Elevator
- Throttle
Figure 3: A System maps inputs to outputs.
Analysis: What can we say about how an aircraft responds ?
Design: How do we engineer the aircraft so that it responds well ?
Control: How do we pick u(t) ?
Starting Point
What is a system ?
What are the properties of a system ?
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 13
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 2.1.0
2.1 A system Maps inputs to outputs
EE/ME 701: Advanced Linear Systems
Section 2.2.0
2.2 Simple example system
The System
System
N{}
u(t)
.y(t)
+
+
u(t)
y(t)
Vc(t)
Space of All
Possible Input Signals
Figure 5: Simple example system with a single state. This system is an RC
circuit.
Space of All
Possible Output Signals
1+(t)
Figure 4: Spaces of input and output signals.
y(t) = N [u(t)]
(18)
0 t<0
u(t) = 1+ (t) =
1 t0
Examples (describe the components and signals):
System
Car Cruise Control
Components
Signals
Car, Engine, Throttle
v (t) , (t) , RPM (t)
Rocket, engine,
v(t) , (t)
Rocket Guidance
steering hydraulics
(t) , (t)
(19)
Look at two cases:
Solving the Diff. Eqn. gives:
Case 1: VC (t = t0 ) = 0.0 [volts]
Case 2: VC (t = t0 ) = 3.0 [volts]
Temperature in Oil Refining
t=0
Input Signal, unit step function:
y1 (t) = 1 e RC t
y2
1
(t) = 1 + 2 e RC t
t >0
t >0
Economy
Biosphere
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 15
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 16
EE/ME 701: Advanced Linear Systems
Section 2.2.0
EE/ME 701: Advanced Linear Systems
Section 3.1.0
3 Classification of Systems and Signals
2.5
Signals and systems are characterized by several properties
2
y (t)
These have formal mathematical definitions (See Bay section 1.1)
y(t)
1.5
Characteristic
* Admissible
* Linear
* Time Invariant
* Causal (Non-anticipative)
* Realizable
* Lumped
* Continuous
* Discrete
* Quantized
* Deterministic
y (t)
0.5
0
0
0.5
1.5
Time [sec]
Figure 6: Responses y1 (t) and y2 (t) of the RC circuit.
So the system response, y(t) , depends on two things:
The input u(t)
Something inside the system at t = t0 , in this case VC (t = t0)
Definition: The state of the system is information about conditions inside the
system at t = t0 that is sufficient and necessary to uniquely determine the
system output.
Signal System
Table 3: Characteristics of Signals and Systems
3.1 Admissible signal
Write:
y(t) = N [u(t), x(t0 )]
(20)
Some facts about the system state:
A signal is admissible if it has mathematical properties such that it has a Laplace
transform (all physical signals are admissible). Signal u (t) is admissible if:
i)
it is piecewise continuous (discontinuous at most at a finite number of
locations in any finite interval)
ii)
t0
It is equal to the number of independent storage elements in the system
(such as independent capacitors, inductors, masses and springs)
iii)
u (t) is exponentially bounded (there exists an exponential signal which goes
to infinity faster than u (t))
The selection of state variables is not unique (more on this and state-variable
transformations later)
iv)
u (t) is a vector of the correct dimension (e.g., for aircraft inputs [aileron,
rudder, elevator, throttle], u (t) R4).
The number of state variables is equal to the order of differential equation
required to describe the system.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 17
s.t.
u (t) 0 t t0
Part 5: Models of Linear Systems
( :
there exists;
for all)
(Revised: Sep 10, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 3.2.0
EE/ME 701: Advanced Linear Systems
Section 3.3.0
3.2 Linear System (without internal variables)
3.3 Time-invariant system (without internal variables)
System: a system is linear if and only if (iff) it obeys the superposition rule (which
incorporates scaling and homogeneity).
A system is time invariant if a time shifted input gives the time-shifted output.
Given:
Given:
y = N [u]
y1 = N [u1]
y2 = N [u2 ]
N [u (t T )] = y(t T )
and
Examples:
u3 = c1 u1 + c2 u2
Time Invariant: a boiler
fuel in heat out (independent of clock time t0 )
Then:
Not Time Invariant (time varying)
y3 = c1 y1 + c2 y2
Example: the economy
Define: u(t) = Loans for buying corn seed
Or
u(t) = 1 in April is not the same as u(t) = 1 in August
N [c1 u1 + c2 u2 ] = c1 N [u1] + c2 N [u2]
Example: flight dynamics
Related:
x (t) = A (t) x (t) + B (t) u (t)
Scaling:
N [c1 u1 ] = c1 y1
Homogeneity:
N [0] = 0
y (t) = C (t) x (t) + D (t) u (t)
A (t) , B (t) , C (t) and D (t) depend on altitude and Mach number.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 19
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 3.5.0
3.4 Causal (non-anticipative) system
EE/ME 701: Advanced Linear Systems
Section 3.6.0
3.6 Lumped Parameter system
For a causal system, the output at time t0 , y(t0 ), is completely determined
by the inputs for t t0 .
That is: u(t1 ) , t1 > t0 does not influence y(t0 ) .
A lumped parameter system is characterized by ordinary differential
equations (the coefficients of the differential equation are the lumped
parameters)
Distributed systems are characterized by partial differential equations
Causal Example:
A good example of an Lumped Parameter system is an RLC circuit
iR
y(t) = 2 u(t 1)
is
Vs(t)
Non-causal (anticipative) Example:
iL
+
iC
+
VC(t)
y(t) = 2 u(t + 1)
Figure 7: An RLC circuit.
3.5 Realizable system
A realizable system is a physically possible system, one which in principle
could be built. To be realizable, a system must be:
1. Causal
When the frequency is not too high, the circuit of figure 7 is characterized
by:
1
1
1
vc (t) +
(21)
vc (t) +
vc (t) =
vs (t)
RC
LC
RC
When the frequency is high enough wave phenomena become important
At some high frequency, lumped parameter models break down
2. y(t) is real (not complex) if u(t) is real
Partial differential equations and wave propagation are required
A distributed system model is required.
We will deal exclusively with lumped parameter systems
All continuous-time systems have a cross-over frequency where
distributed phenomena (often wave phenomena) become important
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 21
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 22
EE/ME 701: Advanced Linear Systems
Section 3.7.0
3.7 Continuous-time versus sampled (discrete-time) signals
EE/ME 701: Advanced Linear Systems
Section 3.8.0
Continuous (time) system: governed by differential equations, such as:
and systems
x(t)
= 2 x(t) + 3 u(t)
(22)
y(t) = x(t)
0.9
0.8
0.7
3.8 Discrete-time (sampled) signals and systems
u(t)
0.6
0.5
Signal: A discrete (sampled) signal u(k) is defined (sampled) only at specific
sample instants, t = tk (see figure 8).
0.4
0.3
System: A discrete (sampled) system is governed by a difference equation, such
as:
0.2
0.1
0
0
0.5
1.5
Time [sec]
2.5
xk = 2 xk1 1 xk2 + 3 uk
Figure 8: A continuous and sampled signal.
Signals:
A continuous signal is defined
for all values of time, e.g.,
u (t) = 0.5 + 0.2 cos t/3 +
6
+0.2 cos (2t/3)
+0.2 cos (2t + )
Part 5: Models of Linear Systems
yk = xk
A sampled (discrete) signal is defined only
particular instants, e.g.,
t (k) =
0.3
0.6
0.9
1.2
1.5
1.8
2.1
2.4
(23)
u (k) =
0.805
0.297
0.235
0.400
0.128
0.0937
0.5247
0.528
(Revised: Sep 10, 2012)
Page 23
Usage: generally we say sampled signal and discrete system, though all
combinations are sometimes seen.
Any computer-based data acquisition results in sampled signals; any
computer-based signal processor is a discrete system
Some mathematical results are more straight-forward or intuitive (or
possible) for one time or the other.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 24
EE/ME 701: Advanced Linear Systems
Section 3.9.0
3.9 Continuous signals and systems, continuity in the mathematical sense
Signal: A signal is continuous if the limit of a sequence of values is the value of
the limit:
lim u (t) = u (t1 )
(24)
tt1
Said another way
EE/ME 701: Advanced Linear Systems
Section 3.10.0
3.10 Quantized signal
A signal can be quantized, the signal takes only certain possible values, for
example
u(t) {0, 1/4, 1/2, 3/4, 1, 5/4, ...}
(27)
Discrete signals are often quantized, but continuous signals can also be.
Example signals with the possible combinations of the characteristics
continuous and discrete and/or quantized are seen in figure 9.
if t t1 ,
then
u (t) u (t1 )
System: A system is continuous if, when a sequence of input signals ui converges
to u, then the corresponding sequence of outputs converges to the output
signal of the limiting input. That is, if
lim ui = u
(25)
N [u ] = lim N [ui ]
(26)
then
i
Figure 9: Example signals: a) Continuous-time signal (with a discontinuity), b)
Continuous-time, quantized signal, c) Discrete signal, d) Discrete,
quantized signal.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 25
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 26
EE/ME 701: Advanced Linear Systems
Section 3.12.0
EE/ME 701: Advanced Linear Systems
Section 4.0.0
4 State, revisited
3.11 Deterministic Signals and systems
Signal: Deterministic signals have no random component, examples:
We saw above that internal variables partially determine the output of a
system. Look at two cases:
Deterministic signal:
u(t) = 1+ (t)
the ideal voltage on an RC circuit, with u(t), VC (t0 ) given.
y1 (t) = 1 e RC t
Case 1: vc (t = t0 ) = 0 [volts]
Non-deterministic signal:
Case 2: vc (t = t0) = 3 [volts]
y2
1
(t) = 1 + 2 e RC t
t >0
t >0
wind gusts acting on a helicopter.
3
System: A deterministic system does not introduce random components into the
output signals
System
2.5
R
+
2
u(t)
Vc(t)
RLC circuit
y (t)
y(t)
y(t)
Deterministic:
1.5
y (t)
0.5
Non-deterministic:
0
0
0.5
economy,
1
Time [sec]
Figure 10: Simple example system.
biological systems,
These internal variables are called states.
quantum mechanical systems
Definition: The state of a system at a time t0 is the minimum set of internal
variables which is sufficient to uniquely specify the system outputs given
the input signal over [t0 , ).
3.12 A Note on units
All physical signals have units
Systems with physical inputs and outputs have units.
The units of the system are [Output] / [Input].
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 27
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 28
1.5
EE/ME 701: Advanced Linear Systems
Section 4.0.0
EE/ME 701: Advanced Linear Systems
Section 4.1.1
4.1 Modified definition of linearity, considering state
Examples of states
Elementary dynamics:
position and velocity of particles
Circuits:
voltages across a capacitor, currents through
inductors
4.1.1 For direct application of Superposition and Scaling, state must be zero
Fluid system:
Rate of flow, levels in tanks
Economy:
balances in accounts, levels of material in
inventories, position of material in transit
For superposition and scaling to apply to a system in the simple way, the
internal states must be zero, consider:
y (t) = N [u (t) , x (t0 )] = u (t) + x (t0 )
(28)
so with u3 = u1 + u2
Some facts about states:
The number of states is equal to the order of the differential (or
difference) equation of the model
States are often associated with energy-storage elements,
y3 (t) = u1 (t) + u2 (t) + x (t0)
(29)
but simple application of superposition would require:
More generally, states are associated with storage of something, where
the stored amount changes with time, giving:
y3 (t) = (u1 (t) + x (t0 )) + (u2 (t) + x (t0))
(30)
= u1 (t) + u2 (t) + 2 x (t0 )
d Amount Stored
= f (x, u,t)
dt
The definition of the states of a system is not unique, consider analyzing
a circuit for a voltage, or for a current.
Keep in mind that the state variables must be independent. Consider the
circuit of figure 11
So x (t0 ) 0 is required for Eqns (29) and (30) to be consistent.
(0 is the Null or zero vector, it is a vector of zeros)
Definition: zero state response:
R1
iL1(t)
y (t) = N [u (t) , 0] = u (t) + 0
(31)
L1
Va (t)
iL2(t)
L2
Figure 11: Circuit with inductors in series. This system has only one state, iL1 (t) .
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 29
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 30
EE/ME 701: Advanced Linear Systems
Section 4.1.2
EE/ME 701: Advanced Linear Systems
4.1.2 Definition of a Linear System considering non-zero state (e.g., DeCarlo
definition 1.8)
4.1.3 Time-invariant system, considering non-zero state
The definition of a time-invariant system is a bit more complex when state
is considered, because we must account for the time-shifted state.
Let N [u, x (t0 )] be the response of system N [] to the input signal u ()
defined over [t0, ) , with initial state x (t0 ) .
Definition:
Then system N [] is linear if and only if, for any two admissible inputs
signals u1 and u2 , and for any scalar k,
k (N [u1, x (t0 )] N [u2 , x (t0 )]) = N [k (u1 u2) , 0 (t0)]
Section 4.1.3
A system is time-invariant if, t t0 x1 Rn such that
NT [N [u, x (t0)]] = N [NT [u] , x1 (t0 + T )]
x (t0 ) Rn
(32)
(33)
the time-shifted output = output from the time-shifted input
where 0 () is the zero vector.
For linear systems, the response can be factored into the response due to the
initial state and the response due to the input
where NT [] is the time delay system:
Student
Exercise
NT (u (t)) = u (t T ) .
(34)
Interpretation:
Eqn(33) says that there exists a possibly different initial condition x1 (t + T )
such that the delayed output of the original system with IC x (t0 ) is identical
to the output of the system with a delayed input and the new IC x1 (t + T ) .
Study question:
For a linear time-invariant system, what is the relationship between x (t0 )
and x1 (t0 + T )
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 31
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 32
EE/ME 701: Advanced Linear Systems
Section 5.0.0
5 Standard Notation for State Model
EE/ME 701: Advanced Linear Systems
Section 5.0.0
For the linear, time-varying, continuous-time system we add (t) as an
argument to the model matrices
The notation for a state model depends on its properties, whether it is linear
or nonlinear, time invariant or time varying, continuous or discrete, etc.
Most general continuous case, nonlinear and time varying
State equation:
x (t) = f (x (t) , u (t) , t)
(35)
Output equation:
y (t) = g (x (t) , u (t) , t)
(36)
x (t) = A (t) x (t) + B (t) u (t)
(41)
y (t) = C (t) x (t) + D (t) u (t) ;
(42)
The discrete-time system does not have derivatives, the system equation
gives x (k) at the next sample instant:
x (k + 1) = A x (k) + B u (k)
(43)
y (k) = C x (k) + D u (k)
(44)
all models have x (t0 ) as the IC. Also, see DeCarlo example, Eqn (1.24).
For the nonlinear, time invariant system, time is no longer an argument of
f () and g ()
And if the discrete-time system is time-varying, A, B, C, D become
functions of sample:
State equation:
x (t) = f (x (t) , u (t))
(37)
x (k + 1) = A (k) x (k) + B (k) u (k)
(45)
Output equation:
y (t) = g (x (t) , u (t))
(38)
y (k) = C (k) x (k) + D (k) u (k)
(46)
y (t) = C x (t) + D u (t) ;
(40)
nx1
u(t)
+
B
nxm
n: number of states
m: number of inputs
p: number of outputs
x(t)
px1
y(t)
=
C
pxn
u(t)
+
D
pxm
mx1
Output equation:
(39)
A
nxn
nx1
State equation:
x (t) = A x (t) + B u (t)
x(t)
mx1
.
x(t)
nx1
The linear, time-invariant, continuous-time system we have seen (and will
be the one we most commonly use)
Figure 12: Configuration of the signals (vectors) and parameter (matrices) of a
state variable model.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 33
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 34
EE/ME 701: Advanced Linear Systems
Section 6.0.0
6 Steps of building a state-variable model (for a
linear system)
EE/ME 701: Advanced Linear Systems
Section 6.0.0
3. Write the differential equations in state-variable form
(a) Higher order differential equations are written as a chain of first-order
equations
1. Write the relevant relations for the system
(b) Put derivative in the left-hand side, these must be the state derivatives
(a) Define symbols for the signals and parameters
(c) All signals on the right-hand side must be expressed in terms of the states
and inputs
(b) Write the equations
(d) Put the model in state-variable form
i. Constituent relations (for elements)
ii. Continuity constraints (for how elements are linked into a system)
4. Write the equation of the output signal (or signals) using the states and
inputs
(c) Record the units, verify that units balance in the equations
5. Check units throughout, to verify correctness.
The equations express laws of physics, the units must balance
Essential things to keep in mind:
2. Identify the differential equations
1. Always distinguish between signals and parameters
(a) Determine the system order
The system order will almost always be sum of the orders of the
contributing differential equations
Rarely, differential equations may be inter-dependent in a way that
reduces the order
(b) Select the state variables
Signals are functions of time and change when the input (signals) change.
Parameters are generally constant (or slowly varying) and are
properties of the system.
Both have physical units
2. Pay special attention to the states
n state variables for an nth order system
Systems have many signals, only n signals are states
State variables must be independent
States correspond to the initial condition needed to determine the output
of the system
The choice is not unique
Often the storage variables are a good choice (often called physical
coordinates)
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 35
With experience, it is usually pretty straight-forward to determine the
states.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 36
EE/ME 701: Advanced Linear Systems
Section 7.1.0
7 Examples with state-variable models
EE/ME 701: Advanced Linear Systems
Section 7.1.0
(b) Write the equations
i. Constituent relations
7.1 Example 1, an electrical circuit
v1(t)
Is(t)
d
Vc (t)
dt 1
d
VL1 (t) = L1 IL1 (t)
dt
VR1 (t) = R1 IR1 (t) ,
IR1(t) VR1(t)
+
R1
Ic1 (t) = C1
VR2 (t) = R2 IR2 (t) ,
IL1(t) VL1(t)
v2(t)
+
Ic1(t)
L1
+
+
VIs(t)
Vc1(t) C1
ii. Continuity Constraints
IR2(t)
+
VR2(t)
Kirchhoffs voltage law
R2
Kirchhoffs current law
VIs (t) +VL1 (t) +VC1 (t) = 0
IR1 (t) + IL1 (t) Ic1 (t) IR2 (t) = 0
Vc1 (t) = VR2 (t)
Is (t) IR1 (t) IL1 (t) = 0
VL1 (t) = VR1 (t)
Is (t) Ic1 (t) IR2 (t) = 0
Figure 13: Electrical circuit.
Sum of the voltages around a loop, + if you enter at the + terminal
Sum of the currents entering a node
1. Write the relevant relations for the system
(c) Record the units, verify that units balance in the equations
(a) Define symbols for the signals and parameters
Units recorded in table 4. Check
Signals
Is (t)
IR1 (t)
Sup. current
[Amps]
Vs (t)
Sup. voltage
Parameters
[volts]
R1 voltage
[volts]
R1 , R2
Resistance
VR2 (t)
R2 voltage
[volts]
C1
Capacitance
Vc1 (t)
C1 voltage
[volts]
L1
Inductance
VL1 (t)
L1 voltage
[volts]
R1 current
[Amps]
VR1 (t)
IR2 (t)
R2 current
[Amps]
Ic1 (t)
C1 current
[Amps]
IL1 (t)
L1 current
[Amps]
i
volts
amp
h
i
amp-sec
h volt i
volt-sec
amp
Table 4: Signals and parameters of the electrical circuit.
Ic1 (t) = C1
d
Vc (t)
dt 1
[amps] =
VL1 (t) = L1
d
IL (t)
dt 1
[volts] =
h amp-sec i volt
volt
second
volt-sec h amp i
amp
second
2. Identify the differential equations
d
Vc (t)
dt 1
d
VL1 (t) = L1 IL1 (t)
dt
Ic1 (t) = C1
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 37
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 38
EE/ME 701: Advanced Linear Systems
Section 7.1.0
(a) Determine the system order
EE/ME 701: Advanced Linear Systems
3. Write the differential equations in state-model form
1 Cap + 1 Inductor, 2nd Order
(a) Higher order differential eqns are written as a chain of first-order eqns.
(b) Select the state variables
(b) Put derivative in the left-hand side, these must be the state derivatives
For this RLC circuit there is a clear choice:
Vc1 (t)
x (t) =
IL1 (t)
x2 (t) =
VR1 (t)
IR2 (t)
x3 (t) =
1
d
Ic (t)
Vc (t) =
dt 1
C1 1
1
d
VL (t)
IL (t) =
dt
L1 1
(47)
VIs (t)
VR2 (t)
(48)
(49)
This step shows why Vc1 (t) and IL1 (t) are natural choices for the
states.
Other possible choices
Section 7.1.0
(c) All signals on the right-hand side must be expressed in terms of the states
and inputs
Later, we will see how to convert the state model into a state model
with any of these state vectors.
If we wanted a model based on x3 (t), it is probably easiest to derive
the model based on the physical coordinates, Eqn (47), and then
make a change of basis to transform the model to x3 (t).
Illegal choices
In Eqn (48) we need to express Ic1 (t) in terms of {Vc1 (t) , IL1 (t) , Is (t)}
This involves using the constituent and continuity equations that
describe the system.
From
Is (t) Ic1 (t) IR2 (t) = 0
From Vc1 (t) = VR2 (t) , IR2 = VR2 /R2
=
=
Ic1 (t) = Is (t) IR2 (t)
Ic1 (t) = Is (t) Vc1 (t) /R2
(50)
These selections for the states are not independent
x4 (t) =
VR1 (t)
IR1 (t)
, x5 (t) =
Vc1 (t)
VR2 (t)
This is the needed form. For VL1 (t)
VL1 (t) = VR1 (t) = R1 IR1 (t) = R1 (Is (t) IL1 (t))
Using Eqns (50) and (51) in Eqns (48) and (49)
And this one is not allowed because Is (t) is an input
Part 5: Models of Linear Systems
x6 (t) =
Vc1 (t)
Is (t)
(Revised: Sep 10, 2012)
(51)
Page 39
d
1
1
1
(Is Vc1 (t) /R2) =
Vc (t) + 0 IL1 (t) + Is (t)
Vc (t) =
dt 1
C1
C1 R2 1
C1
R1
R1
1
d
R1 (Is (t) IL1 (t)) = 0Vc1 (t) IL1 (t) + Is (t)
IL (t) =
dt
L1
L1
L1
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012) Page 40
EE/ME 701: Advanced Linear Systems
Section 7.1.0
EE/ME 701: Advanced Linear Systems
(d) Put the model in state-variable form
x (t) = A x (t) + B u (t)
d Vc1 (t) C11R2 0 Vc1 (t)
=
+
x (t) =
dt IL (t)
0
RL11
IL1 (t)
1
1
C1
R1
L1
5. Check units throughout, to verify correctness.
[volts/sec]
[volts]
, x (t) :
x (t) :
[amps/sec ]
[amps ]
(52)
Is (t)
sec1
A:
1 ,
sec
This is the state equation, with
x (t) =
Vc1 (t)
IL1 (t)
u (t) = [Is (t)] ,
A=
C11R2
0
0
RL11
4. Write the equation of the output signal (or signals) using the states and
inputs
Suppose the output were VR1 (t). In Eqn (51) we have already derived
that VR1 (t) = R1 (Is (t) IL1 (t)), so
y (t) =
0 R1
Vc1 (t)
IL1 (t)
y (t) = C x (t) + D u (t)
This is the output equation, with
y (t) = [VR1 (t)] , C =
0 R1
Units check in the state equation.
Units in the output equation are given as:
h h
i i [volts]
volts
+ volts [amps]
[volts] = amp
amp
[amps ]
Units check in the output equation.
We need to find a way to express Vs (t) in terms of the states and inputs,
{Vc1 (t) , IL1 (t) , Is (t)}. Going back to the original equations,
, D = [R1 ]
(Revised: Sep 10, 2012)
So units in the state equation are given as:
h
i
volt
sec1
[volts/sec]
[volts]
=
+ amp-sec
1
1 [amps]
sec
[amps/sec ]
[amps ]
sec
y (t) = [Vs (t)]
Notice, the output is made up of a signal, and the output and feed-forward
matrices are made up of parameters.
Part 5: Models of Linear Systems
1
C1
R1
L1
i
volt
amp-sec
B:
1
sec
h
Alternative output, suppose we wanted to determine Vs (t), for example to
calculate the supply power Ps (t) = Vs (t) Is (t). The output will be Vs (t):
+ [R1 ] Is (t)
B=
Notice, the state and input are made up of signals, and the system and
input matrices are made up of parameters.
Section 7.1.0
Page 41
Vs (t) = VR1 (t) +Vc1 (t) = R1 (Is (t) IL1 (t)) +Vc1 (t)
(53)
This gives Vs (t) in terms of the states and inputs !
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 42
EE/ME 701: Advanced Linear Systems
Section 7.1.1
EE/ME 701: Advanced Linear Systems
For y (t) = [Vs (t)]
Section 7.1.1
Build the Matlab state-variable model object
y (t) = C x (t) + D u (t)
>> SSmodel = ss(A, B, C, D)
(54)
a =
with
C =
1 R1
, D = [R1 ]
x1
x2
x1
-500
0
y1
x1
0
x2
0
-1000
b =
x1
x2
(55)
c =
A deep property of state models: just as with Eqns (54) and (55) we can find
a row for the C and D matrices to give any signal in the system.
Now we can look at poles and zeros
x2
-100
>> Poles = pole(SSmodel)
Poles = -1000
-500
>> Zeros = zero(SSmodel)
Zeros = -500
0
d =
u1
y1 100
Continuous-time model.
7.1.1 Building and exercising the circuit model
To build the model, build the A , B , C and D matrices:
>>
>>
>>
>>
%%
R1
R2
L1
C1
Looking at the step and frequency response
Set up the parameter values
= 100
%% Ohms
= 200
%% Ohms
= 0.1
%% Henries
= 10e-6
%% Farads
>> figure(1), clf; step(SSmodel); print(-deps2c, CircuitAStepResponse)
>> figure(2), clf; bode(SSmodel); print(-deps2c, CircuitAFreqResponse)
Bode Diagram
Step Response
40
100
30
Magnitude (dB)
90
80
%% Build the state equation
>> A = [ -1/(C1*R2), 0; 0, -(R1/L1) ]
A = -500
0
0
-1000
Amplitude
70
Part 5: Models of Linear Systems
10
0
50
10
90
Phase (deg)
40
30
%% Build the output Eqn
>> C = [ 0, -R1]
C =
0 -100
>> D = [ R1 ]
D = 100
(Revised: Sep 10, 2012)
20
60
20
>> B = [ 1/C1; R1/L1]
B = 1.0e+05 *
1.0000
0.0100
u1
1e+05
1000
Page 43
45
10
0
3
Time (sec)
6
3
x 10
0
1
10
10
10
Frequency (rad/sec)
10
10
Figure 14: Circuit step and frequency response
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 44
EE/ME 701: Advanced Linear Systems
Section 7.2.0
7.2 Example 2, a mechanical system
The parameters are listed in table 5. Signals are listed in table 6.
See tables 5 and 6.
(b) Write the equations
m2
i. Constituent relations (for elements)
zv(t)
vcar
ks
bs
m1
zw(t)
kw
Road Surface
r(t)
Figure 15: Quarter vehicle suspension.
Parameters
m1
kw
wheel mass
[kg]
ks
i
Newtons
h meter i
Newtons
Suspension damping meter/sec
Suspension stiffness
[kg]
bs
i
Newtons
tire stiffness
meter
Table 5: Parameters of the mechanical quarter suspension.
h
F (t) = m x (t)
(56)
Hooks law:
F (t) = k (x1 (t) x2 (t))
(57)
Damper Eqn:
F (t) = b (x1 (t) x2 (t))
(58)
ii. Continuity constraints (for how elements are linked into a system)
Inerial Reference
m2 1/4 vehicle mass
The constituent relations for mechanical systems are:
Newtons 2nd law:
It is desired to model the system
and determine the response zv (t)
to the road profile.
Section 7.2.0
(a) Define symbols for the signals and parameters
A Quarter Suspension is shown in figure 15.
The vehicle is moving across a
road, with the surface profile given
by r (t).
EE/ME 701: Advanced Linear Systems
Sum of the forces acting on any free body is zero
Sum of the velocities around any loop is zero
(c) Record the units, verify that units balance in the equations
See tables 5 and 6.
2. Identify the differential equations
The differential Eqns come from Eqns (56) and (58)
Signals
zv (t)
Height of vehicle
[m]
zw (t) Height for tire center [m]
r (t) Height of road surface [m]
Table 6: Signals of the mechanical quarter suspension.
1. Write the relevant relations for the system
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 45
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 46
EE/ME 701: Advanced Linear Systems
Section 7.2.0
EE/ME 701: Advanced Linear Systems
Model equations (repeated)
(a) Determine the system order
We get two 2nd order differential equations:
m1 zw (t) =
m2 zv (t) =
Section 7.2.0
m1 zw (t) = kw (r (t) zw (t)) ks (zw (t) zv (t)) bs (zw (lt) zv (t))
m2 zv (t) = +ks (zw (t) zv (t)) + bs (zw (t) zv (t))
Forces on wheel
Forces on 1/4 vehicle
The system will be 4th order. From the free-body diagrams
Lets look at one of the 2nd order terms first
m1 zw (t) = kw (r (t) zw (t)) ks (zw (t) zv (t)) bs (zw (t) zv (t))
(59)
m2 zv (t) = +ks (zw (t) zv (t)) + bs (zw (t) zv (t))
(60)
zw (t) =
(b) Select the state variables
The natural physical coordinates are
mass
zw (t)
zw (t)
x (t) =
zv (t)
zv (t)
Part 5: Models of Linear Systems
position and velocity of each
[m]
[m/sec]
[m]
[m/sec]
(Revised: Sep 10, 2012)
zw (t)
Examining Eqn (62)
h
Likewise, from Eqn (63)
h
+ks /m2 +bs /m2 ks /m2
Part 5: Models of Linear Systems
zw (t) = (kw + ks ) /m1 bs /m1 +ks /m1
zv (t) =
i
zw (t)
+ [__] r (t)
__ __ __ __
zv (t)
zv (t)
(61)
Page 47
zw (t)
zw (t)
d x (t)
=
x (t) =
dt
zv (t)
zv (t)
- The system order will almost always be sum of the orders of the
contributing differential equations -
We need 4 state variables, we have two 2nd order differential
equations.
(63)
3. Write the differential equations in state-variable form
(62)
zw (t)
i
zw (t)
+[kw ] r (t)
+bs /m1
zv (t)
zv (t)
(64)
z (t)
w
i
zw (t)
+ [0] r (t)
bs /m2
zv (t)
zv (t)
(65)
(Revised: Sep 10, 2012)
Page 48
EE/ME 701: Advanced Linear Systems
Section 7.2.0
Eqns (64) and (65) gives two of the rows of the A matrix.
Looking back at x (t),
zw (t) =
zw (t)
d x (t)
=
x (t) =
dt
zv (t)
zv (t)
For the first row we must write:
zw (t)
zw (t)
i
zw (t)
+ [__] r (t)
__ __ __ __
zv (t)
zv (t)
zw (t) is an element of the state vector, we just hook it up !
zw (t) =
Likewise
z (t)
i
zw (t)
+ [0] r (t)
0 1 0 0
zv (t)
zv (t)
zw (t)
h
i
zw (t)
+ [0] r (t)
zv (t) = 0 0 0 1
zv (t)
zv (t)
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 49
Section 7.2.0
Putting the pieces together
x (t) = A x (t) + Bu (t)
0
1
0
0
(kw +ks )
bs
ks
m1 + m1 + mbs1
m1
x (t) =
0
0
0
1
ks
bs
ks
+ m2 m2 mbs2
+ m2
(66)
0
z (t)
zw (t) mkw
+ 1 r (t)
zv (t) 0
zv (t)
0
4. Write the equation of the output signal (or signals) using the states and
inputs
If the output of interest is the vehicle response
y1 (t) = zv (t) =
EE/ME 701: Advanced Linear Systems
zw (t)
i
zw (t)
+ [0] r (t)
0 0 1 0
zv (t)
zv (t)
(67)
Suppose the desired output is the force in the suspension spring. Since
Fs (t) = ks (zw (t) zv (t))
The output equation is given as:
y2 (t) = Fs (t) =
Part 5: Models of Linear Systems
ks 0 ks
zw (t)
i
zw (t)
+ [0] r (t)
zv (t)
zv (t)
(Revised: Sep 10, 2012)
(68)
Page 50
EE/ME 701: Advanced Linear Systems
Section 7.2.0
Suppose the desired output is the road force on the tire. Since
EE/ME 701: Advanced Linear Systems
Section 7.2.0
5. Check units throughout, to verify correctness.
Noting that 1.0 [Newton]=1.0 kg m/sec2
Fw (t) = kw (r (t) zw (t))
Terms such as kw /m1 or bs /m2 have units of
The output equation is given as:
y3 (t) = Fw (t) =
kw
zw (t)
kw
:
m1
i
zw (t)
+ [kw ] r (t)
0 0 0
zv (t)
zv (t)
(69)
y (t) = C x (t) + D u (t)
z (t)
v
y (t) = Fs (t)
Fw (t)
zw (t)
0 0 1 0
zw (t)
= ks 0 ks 0
zv (t)
kw 0 0 0
zv (t)
(70)
kg-m 1
sec2 m
kg-m 1
sec2 m/s
1
kg
1
kg
=
=
i
h
sec2
i
h
sec1
Thus the state equation has units of:
Now suppose the desired output is all three
bs
:
m2
0
+ 0
r (t)
kw
[]
[1]
[]
[]
[m/sec]
1 2 1
sec2
m/sec2
sec
sec
sec
=
x (t) :
[]
[m/sec]
[]
[]
[1]
1 2 1
2
2
sec
sec
sec
m/sec
sec
[]
sec2
[m]
+
[]
[]
(Revised: Sep 10, 2012)
Page 51
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
[m/sec]
[m]
[m/sec]
Units check !
Part 5: Models of Linear Systems
[m]
Page 52
EE/ME 701: Advanced Linear Systems
Section 7.2.1
EE/ME 701: Advanced Linear Systems
Examine the poles and zeros and response
7.2.1 Building and exercising the suspension model
Build the model
= 10000
= 2500
= 10000
= 25
= 250
%%
%%
%%
%%
%%
N/m
N/m
N/m/s
kg
kg
>> A = [ 0
1
0
0
-(kw+ks)/m1 -bs/m1
ks/m1
bs/m1
0
0
0
1
ks/m2
bs/m2 -ks/m2 -bs/m2 ]
>> B = [ 0; kw/m1; 0 ; 0]
>> C1 = [ 0, 0, 1, 0];
D1 = [0]
>> SSmodel2a = ss(A, B, C1, D1)
%% Fast mode
%% Lightly damped mode
%% Slow mode
Bode Diagram
Step Response
50
1.8
Magnitude (dB)
kw
ks
bs
m1
m2
>> Poles = pole(SSmodel2a)
Poles =
-438.92
-0.41 + 6.00i
-0.41 - 6.00i
-0.25
>> Zeros = zero(SSmodel2a)
Zeros = -0.2500
1.6
Amplitude
1.4
50
100
1.2
150
200
0
0.8
Phase (deg)
>>
>>
>>
>>
>>
Section 7.2.1
0.6
0.4
90
180
0.2
0
a =
b =
x1
x2
x3
x4
x1
0
-500
0
10
x2
1
-400
0
40
x3
0
100
0
-10
c =
x4
0
400
1
-40
x1
x2
x3
x4
u1
0
400
0
0
d =
y1
x1
0
x2
0
x3
1
x4
0
y1
u1
0
6
Time (sec)
10
12
14
270
1
10
10
10
10
Frequency (rad/sec)
10
10
Figure 16: Suspension step and frequency response.
The shock absorber is not doing its job !
Examine the damping of the modes
>> [Wn, rho] = damp(SSmodel2a)
Wn = 0.2516
6.0186
6.0186
438.9211
rho =
1.0000
0.0687
0.0687
1.0000
Mode with damping of 0.07 is too lightly damped.
Continuous-time model.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 53
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 54
EE/ME 701: Advanced Linear Systems
Section 7.2.1
EE/ME 701: Advanced Linear Systems
Modify the parameters
>>
>>
>>
>>
>>
>>
%%
kw
ks
bs
m1
m2
Examine the poles, zeros and damping
2nd parameter
= 10000
%%
= 1000
%%
= 1000
%%
= 25
%%
= 250
%%
set
N/m
N/m
N/m/s
kg
kg
>> Poles = pole(SSmodel2b)
Poles = -31.4478
-5.4731 + 1.3136i
-5.4731 - 1.3136i
-1.6059
>> Zeros = zero(SSmodel2b)
Zeros = -1.0000
%% Softer spring
%% Softer shock absorber
x1
x2
x3
x4
x1
0
-440
0
4
x2
1
-40
0
4
x3
0
40
0
-4
x4
0
40
1
-4
%% Fast Mode
%% Oscillatory mode,
%%
now alpha>omega
%% Slow mode
The slow mode is faster, the fast mode is much slower, and the oscillatory
mode is better damped
>> A = [ 0
1
0
0
-(kw+ks)/m1 -bs/m1
ks/m1
bs/m1
0
0
0
1
ks/m2
bs/m2 -ks/m2 -bs/m2 ]
>> B = [ 0; kw/m1; 0 ; 0]
>> C1 = [ 0, 0, 1, 0]; D1 = [0]
>> SSmodel2b = ss(A, B, C1, D1)
a =
Section 7.2.1
Compute the damping factors
>> [Wn, Z] = DAMP(SSmodel2b)
Wn = 1.6059
5.6286
5.6286
31.4478
b =
x1
x2
x3
x4
Z =
1.0000
0.9724
0.9724
1.0000
Damping factor of 0.97 is much better
u1
0
400
0
0
Examine the step and frequency response
Bode Diagram
Step Response
50
1.4
x3
1
x4
0
d =
y1
u1
0
50
100
Continuous-time model.
0.8
150
0
0.6
Phase (deg)
x2
0
Amplitude
y1
x1
0
Magnitude (dB)
1.2
c =
0.4
0.2
0.5
1.5
Time (sec)
2.5
3.5
90
180
270
1
10
10
10
Frequency (rad/sec)
10
10
Figure 17: Suspension step and frequency response.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 55
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 56
EE/ME 701: Advanced Linear Systems
Section 7.2.1
Lets consider the Transfer Function
EE/ME 701: Advanced Linear Systems
The model now has 3 outputs
>> TFmodel2b = tf(SSmodel2b)
r(t)
A, B, C, D
Section 7.2.1
zv(t)
Fs(t)
y(t)
Transfer function:
1600 s + 1600
-------------------------------------s^4 + 44 s^3 + 444 s^2 + 1600 s + 1600
x(t)
Fw(t)
n: states
m: inputs
p: outputs
mxp: TFs
Figure 18: Block diagram of quarter suspension system reflecting 3 outputs.
Building the state-variable model and computing the transfer function
may be the easiest way to get the TF of a complex system.
Lets consider multiple outputs, from Eqn (70).
The C and D matrices change.
Output Eqn: y (t) = C x (t)+D u (t)
>> C2c = [ 0 0
1 0
ks 0 -ks 0
-kw 0
0 0 ];
>> D2c = [ 0; 0; kw]
>> SSmodel2c = ss(A, B, C2c, D2c)
a =
x1
x2
x3
x4
c =
y1
y2
y3
x1
0
-440
0
4
x1
0
1000
-1e+04
x2
1
-40
0
4
x3
0
40
0
-4
x2
0
0
0
x3
1
-1000
0
x4
0
40
1
-4
b =
x1
x2
x3
x4
x4
0
0
0
u1
0
400
0
0
d =
y1
y2
y3
u1
0
0
1e+04
Continuous-time model.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 57
Examine the transfer function and step response
>> TFmodel2c = tf(SSmodel2c)
Transfer function from input to output...
1600 s + 1600
#1: -------------------------------------s^4 + 44 s^3 + 444 s^2 + 1600 s + 1600
#2:
4e05 s^2 - 3.207e-10 s + 2.754e-09
-------------------------------------s^4 + 44 s^3 + 444 s^2 + 1600 s + 1600
#3:
10000 s^4 + 4.4e05 s^3 + 4.4e05 s^2 + 6.487e-10 s + 1.913e-10
------------------------------------------------------------s^4 + 44 s^3 + 444 s^2 + 1600 s + 1600
Each input/output pair (in this case 1x3) gives a TF.
The zeros and gain are different between the TFs
The poles are the same
Note: the coefficients of 109 1010 in TFs 2 and 3 are due to round-off
error. These TFs have a double zero at the origin.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 58
EE/ME 701: Advanced Linear Systems
Section 7.2.2
equation
Given a differential equation, such as
Step Response
a3 y(t) + a2 y (t) + a1 y (t) + a0 y (t) = b2 u (t) + b1 u (t) + b0 u (t)
1.5
We can immediately write down the TF model
0.5
0
To: Out(2)
(71)
T (s) =
1000
Amplitude
Section 7.3.0
7.3 Example 3, state-variable model from a differential
>> figure(1), clf;
>> step(SSmodel2c);
>> print(-deps2c, Mechanism2StepResponsec)
To: Out(1)
EE/ME 701: Advanced Linear Systems
b2 s2 + b1 s + b0
3
a3 s + a2 s2 + a1 s + a0
(72)
500
How do we construct a state-variable model ?
A. In Matlab
500
>> num = [b2, b1, b0]; den = [a3, a2, a1, a0]
>> TFmodel = tf(num, den);
>> SSmodel = ss(TFmodel)
To: Out(3)
10000
5000
0
B. Put together the state-variable model in one of the canonical forms.
5000
0.5
1.5
2
Time (sec)
2.5
3.5
Figure 19: Step response shown with 3 outputs.
There are 4 canonical forms. The canonical forms have a strong
relationship to properties called controllability and observability, and
we will see all four (cf. Bay chapter 8).
Here we consider Controllable Canonical Form
(cf. Bay section 1.1.3)
7.2.2 Conclusions, Quarter Suspension Example
First, the Diff Eq must be monic. That means that the an coefficient is 1.0
Follow the steps to put together the model
Divide through Eqn (71) (or 72) by a3
It is straight forward to
test various parameter configurations
1.0y(t) + a2 y (t) + a1 y (t) + a0 y (t) = b2 u (t) + b1 u (t) + b0 u (t) (73)
obtain a transfer function
State-variable model naturally represents systems with multiple outputs
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 59
With a2 = a2 /a3 , a1 = a1 /a3 , ... , b2 = b2 /a3 , ... , b0 = b0 /a3
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 60
EE/ME 701: Advanced Linear Systems
Section 7.3.1
EE/ME 701: Advanced Linear Systems
Section 7.3.1
1.0z(t) + a2 z (t) + a1 z (t) + a0 z (t) = 1.0 u (t)
7.3.1 Steps of deriving the state-variable model
(74, repeated)
1. Write the relevant relations for the system
2. Identify the differential equations
(a) Define symbols for the signals and parameters.
(a) Determine the system order:
Parameters
h
i
1
y
Signals
a2 Coefficient sec
b2 Coefficient
u sec
h
i
y (t) Output y
y
a1 Coefficient sec2 b1 Coefficient
2
u (t) Input u
h u sec i
y
a0 Coefficient sec3 b0 Coefficient
u sec3
Table 7: Signals and Parameters for the Differential Equation.
Units result from
dividing through by a3 , which has units of sec3 .
3rd order
(b) Select the state variables
Considering Eqn 74, select the state variables to be the output variable
and its derivatives up to the n 1 derivative
(b) Write the equations
To model a differential equation in Controllable Canonical form,
break the TF into two components, with the denominator first.
u(t)
y(t)
B(s)
A(s)
(a) Original TF
u(t)
1
A(s)
z(t)
(76)
The state vector is said to be in phase-variable form.
The derivative of the last phase variable is given by Eqn (74), re-written
as Eqn (77) (cf. Bay Eqn (1.25))
The breakdown of figure 20 gives two equations:
1.0z(t) = a2 z (t) a1 z (t) a0 z (t) + 1.0 u (t)
(74)
(75)
x1 (t)
x (t) = x2 (t)
x3 (t)
(c) Record the units, verify that units balance in the equations
See table 7.
(Revised: Sep 10, 2012)
The derivative of each of the phase variables is simply its successor
Figure 20: Transfer function as two components.
Part 5: Models of Linear Systems
x (t) = z (t)
z(t)
3. Write the differential equations in state-variable form
(b) TF in two components
y (t) = b2 z (t) + b1 z (t) + b0 z (t)
z (t)
The state-derivative vector includes the highest derivative of z (t) .
y(t)
B(s)
1.0z(t) + a2 z (t) + a1 z (t) + a0 z (t) = 1.0 u (t)
z (t)
x (t) = z (t)
,
z (t)
Page 61
z (t)
= z (t)
z(t)
Part 5: Models of Linear Systems
= 0
0
1
a0 a1 a2
z (t)
z (t)
z (t)
(77)
+ 0 u (t)
1
(78)
(Revised: Sep 10, 2012)
Page 62
EE/ME 701: Advanced Linear Systems
Section 7.3.1
y (t) = b2 z (t) + b1 z (t) + b0 z (t)
(75, repeated)
4. Write the equation of the output signal (or signals) using the states and
inputs
z (t)
h
i
y (t) = b0 b1 b2 z (t) + [0] u (t)
(79)
z (t)
EE/ME 701: Advanced Linear Systems
5. Check units throughout, to verify correctness (continued)
Putting the units together:
u-sec1
x (t) : u-sec2
u-sec3
(Note: Bays development includes a b3 term. Eqn (1.25) show how the b3
term is incorporated).
5. Check units throughout, to verify correctness.
From Eqn (74), z (t) has units of [u], so the phase-variable state vector
has units
z (t)
x (t) = z (t)
z (t)
[u]
x (t) : u-sec1
u-sec2
[]
1
[]
A : []
[]
1
2 1
3
sec
sec
sec
C:
h
y
u
i h
y-sec
u
Part 5: Models of Linear Systems
i
y-sec2
u
[]
B : []
sec3
y
u
y
u
i h
[]
[]
[u]
= []
u-sec1
[]
1
2 1
u-sec2
sec
sec
sec3
[]
+ [] [u]
sec3
y-sec
u
i
[u]
y-sec2
u-sec1
u
u-sec2
(Revised: Sep 10, 2012)
Page 63
>> a2 = 5; a1 = 7; a0=8;
>> b2= -2; b1 = 3; b0 = 4;
>> A = [ 0
1
0 ;
0
0
1;
-a0, -a1, -a2];
>> B = [ 0;0; 1];
>> C = [ b0, b1, b2];
D = 0;
Part 5: Models of Linear Systems
y
+ [u]
Build the model
D:
y (t) : [y] =
h
7.3.2 Building and exercising a differential equation model
Looking at table 7, the model matrices have units of
Section 7.3.2
(Revised: Sep 10, 2012)
Page 64
EE/ME 701: Advanced Linear Systems
Section 7.3.2
EE/ME 701: Advanced Linear Systems
Look at the step and frequency response
>> SSmodel3a = ss(A, B, C, D)
x1
x2
x3
x1
0
0
-8
x2
1
0
-7
x3
0
1
-5
b =
>> figure(2), clf; step(SSmodel3a);
>> figure(3), clf; bode(SSmodel3a);
u1
0
0
1
x1
x2
x3
Bode Diagram
Step Response
10
1.2
0
Magnitude (dB)
a =
y1
x1
4
x2
3
x3
-2
d =
u1
0
y1
0.8
10
20
30
Amplitude
c =
Section 7.4.0
0.6
40
360
Phase (deg)
0.4
0.2
>> Poles = pole(SSmodel3a) >> Zeros = zero(SSmodel3a)
Poles = -0.6547 + 1.3187iZeros = 2.3508
-0.8508
-0.6547 - 1.3187i
-3.6906
Looking at the pole-zero map (constellation)
0.2
4
5
Time (sec)
270
180
90
1
10
10
10
10
Frequency (rad/sec)
Figure 22: Circuit step and frequency response.
7.4 State-variable model and simulation diagram
>> figure(1), pzmap(SSmodel3a);
Analog computers include integrators, gain blocks and summing junctions
PoleZero Map
1.5
b3
y(t)
Imaginary Axis
0.5
b1
b2
b0
u(t)
0.5
.
x3
.
x2
x3
x2
.
x1
x1
1.5
4
-a2
Real Axis
-a1
-a0
a3=1
Figure 21: PZ map of the state-variable model.
(Controllable Canonical Form, Notation follows Bay)
Figure 23: Simulation diagram for system in controllable canonical form.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 65
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 66
EE/ME 701: Advanced Linear Systems
Section 7.4.0
Integrators, gain blocks and summing junctions are built with Op-Amps
C1
Integrators:
Vin(t)
+
iR1(t)
R1
V-(t)
V+(t)
Gain Blocks:
Rf
Vin(t)
Vo(t)
Ra
V+(t)
(105..107),
iRf(t)
+
Vol(t)
V (t)
iRa(t)
Figure 24: An Op-Amp integrator.
Op Amp gain is very high
negative feedback, so
Section 7.4.0
iC1(t)
EE/ME 701: Advanced Linear Systems
Figure 25: An Op-Amp gain block.
and the circuit is configured with
An Op-Amp gain block is shown in figure 26.
V (t) V + (t)
Using the principles of virtual ground and low input current
And in the circuit of figure 24, V + (t) is wired to ground. So node V
becomes a virtual ground
V (t) 0
Op Amp input currents are very small (109.. 1012 amps), so
IR1 (t) + Ic1 (t) = 0
V0 (t) =
Rf
Vin (t)
Ra
(83)
Summing junctions:
Rf
(80)
Eqn (80) gives
1
d
Vin (t) V (t) +C1
Vo (t) V (t) = 0
R1
dt
IRa (t) + IR f (t) = 0
(81)
Va(t) i
Ra
Ra
+
Vb(t) i
Rb
V-(t)
Rb
V+(t)
iRf
+
Vol(t)
With the properties of the virtual ground, Eqn (82) becomes
Figure 26: An Op-Amp summing junction.
d
1
d
1
Vin (t) +C1 Vo (t) = 0 , Vo (t) =
Vin (t)
R1
dt
dt
R1 C1
The Op-Amp virtual ground configuration sums the currents at the V
node.
Giving
Vo (t) =
Part 5: Models of Linear Systems
Z t
t0
1
Vin (t) dt
R1 C1
(Revised: Sep 10, 2012)
(82)
Page 67
V0 (t) =
Part 5: Models of Linear Systems
Rf
Rf
Va (t) + Vb (t) + ...
Ra
Rb
(Revised: Sep 10, 2012)
(84)
Page 68
EE/ME 701: Advanced Linear Systems
Section 7.4.0
A note on the inversions. Recall
Vo (t) =
Z t
t0
1
Vin (t) dt
R1 C1
EE/ME 701: Advanced Linear Systems
Section 7.4.0
Returning to the simulation diagram, we can write down the state model
directly from the simulation diagram (and vice-versa)
(82, repeated)
b3
In each of Eqns (82), (83) and (84) the output voltage is inverted relative
to the input voltage.
This is an inherent property of the virtual ground configuration.
b1
b2
u(t)
.
x3
.
x2
x3
x2
y(t)
b0
.
x1
x1
In analog computers, either
-a2
1. Introduce - signs as needed, and invert signals as needed with a gain
blocks, g = 1, or
2. Include a second Op-Amp in each element (Integrator, Gain block,
Summing junction), so that the block is non-inverting.
-a1
-a0
a3=1
(Controllable Canonical Form, Notation follows Bay)
Figure 27: Simulation diagram for system in controllable canonical form.
The output of each integrator is a state
1. Write the relevant relations for the system
(a) Define symbols for the signals and parameters
Signals
u(t)
Input
y (t)
Output
x1 (t) , x2 (t) , x3 (t) States
Parameters
b0..b3 Numerator Coefficients
a0..a2 Denominator Coefficients
Table 8: Signals and parameters of the simulation diagram.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 69
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 70
EE/ME 701: Advanced Linear Systems
Section 7.4.0
b3
b1
b2
u(t)
.
x3
.
x2
x3
-a2
x2
EE/ME 701: Advanced Linear Systems
y(t)
.
x1
-a1
y3(t)
b3
b0
Section 7.4.0
b1
b2
x1
u(t)
.
x3
.
x2
x3
-a0
-a2
a3=1
b0
.
x1
x2
y(t)
x1
-a1
-a0
a3=1
(Controllable Canonical Form, Notation follows Bay)
(Controllable Canonical Form, Notation follows Bay)
Figure 28: Simulation diagram for system in controllable canonical form.
Figure 29: Simulation diagram for system in controllable canonical form.
3. Write the differential equations in state-variable form
(b) Write the equations
Directly transcribe from (or to !) the block block diagram
Examining the block diagram we may write:
x1 (t) = x2 (t)
(85)
x2 (t) = x3 (t)
(86)
x3 (t) = a2 x3 (t) a1 x2 (t) a0x1 (t) + u (t)
y (t) = b3 (a2 x3 (t) a1 x2 (t) a0x1 (t) + u (t))
x (t)
1
x (t) = x2 (t)
x3 (t)
(87)
+b2 x3 (t) + b1 x2 (t) + b0 x1 (t)
(88)
0
1
0
= 0
0
1
a0 a1 a2
x (t)
1
x2 (t)
x3 (t)
0
+ 0 u (t) (89)
1
4. Write the equation of the output signal (or signals) using the states and
inputs (cf. Bay Eqn (1.25))
2. Identify the differential equations
(a) Determine the system order:
y3 (t) = b3 (u (t) a0 x1 (t) a1 x2 (t) a2 x2 (t))
3rd order
(b) Select the state variables
The physical coordinates of the system are
the integrator outputs
These are voltages we can observe on an
oscilloscope.
Part 5: Models of Linear Systems
x1 (t)
x (t) = x2 (t)
x3 (t)
(Revised: Sep 10, 2012)
y (t) =
x (t)
i 1
(b0 b3a0) (b1 b3a1) (b2 b3a2) x2 (t)
x3 (t)
(90)
+ [b3] u (t)
(91)
5. Check units throughout, to verify correctness.
See example 3.
Page 71
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 72
EE/ME 701: Advanced Linear Systems
Section 8.1.0
8 Some basic operations with state-variable models
8.1 Deriving the transfer function from the state-variable
model
It is straight forward to derive the transfer function from a state variable
model.
Starting with the state equation
x (t) = A x (t) + B u (t)
y (t) = C x (t) + D u (t)
EE/ME 701: Advanced Linear Systems
Section 8.1.1
8.1.1 InterpretingYthe
(s)transfer function
= C (s I A)1 B+D
U (s)
(96, repeated)
Eqn (96) can be solved symbolically by Cramers rule, to give the symbolic
transfer function.
Recall from basic linear algebra that Cramers rule gives the matrix
inverse as:
1
U 1 =
adj U
(97)
det U
Where U is an n n matrix, and adj U is the adjugate of matrix U.
Defining
Considering the Laplace transform of the state equation:
Vi, j = (1)i+ j Mi, j
(98)
adj U = V T
(99)
Then
Eqn (92) leads to:
s X (s) = A X (s) + B U (s)
(92)
Y (s) = C X (s) + D U (s)
(93)
(s I A) X (s) = B U (s)
Examples:
or
X (s) = (s I A)1 B U (s)
(94)
With (94), Eqn (93) leads to:
Y (s) = C (s I A)1 B U (s) + D U (s)
(95)
If m = 1 (one input) and p = 1 (one output), then Eqn (95) gives the transfer
function:
Y (s)
= C (s I A)1 B + D
(96)
U (s)
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Mi, j is the i, jth minor of matrix U and is the determinant of the matrix by
removing the ith row and jth column from U.
Page 73
U2 =
a b
c d
a b c
U3 = d e f
g h i
Part 5: Models of Linear Systems
adj U2 =
e f
+
h i
adj U3 =
..
.
b c
h i
...
(Revised: Sep 10, 2012)
Page 74
EE/ME 701: Advanced Linear Systems
Section 8.1.1
Using Cramers rule, we can symbolically solve Eqn (96). Since
(s I A)
It follows that
EE/ME 701: Advanced Linear Systems
Section 8.1.1
Multiplying out
adj (s I A)
=
det (s I A)
h
1
Y (s)
= 2
U (s) s + 7 s + 10
(100)
Y (s)
1
=
C adj (s I A) B + D
U (s) det (s I A)
(101)
2 s + 19
3s +6
= 28 s + 131
s2 + 7 s + 10
(103)
Today, we wouldnt want to apply Cramers rule by hand for any case larger
than 3x3.
Under the hood, this is how Matlab finds the TF from a state-variable model.
Example
A=
C=
B=
2
3
D = [0]
>> A = [ -2 3 ; 0 -5] B = [ 2 ; 3 ] C = [ 5
>> SSmodel = ss(A, B, C, D)
>> tf(SSmodel)
Transfer function:
Then
det (sI A) = det
(s + 2)
(s + 5)
adj (sI A) = adj
= (s + 2)(s + 5) 0 = s2 + 7 s + 10
(s + 2)
(s + 5)
So the TF is given by:
h
Y (s)
1
= 2
U (s) s + 7 s + 10
Part 5: Models of Linear Systems
(s + 5)
(s + 2)
i (s + 5)
3
2
+ [0] (102)
6
0
(s + 2)
3
(Revised: Sep 10, 2012)
Page 75
6 ] D = 0
28 s + 131
-------------s^2 + 7 s + 10
Note:
The denominator is given by det (s I A), showing the the poles are the
eigenvalues of the A matrix.
The B, C, and D matrices play no role in determining the poles, and thus
the stability
If we had additional inputs (additional columns in B) or outputs (additional
rows in C) the term
C adj (s I A) B
would give an array of numerator polynomials, one for each input/output
pair.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 76
EE/ME 701: Advanced Linear Systems
Section 8.1.3
From Eqn (96) we can determine the DC Gain of a state-variable model,
s j 0
Section 8.2.0
8.2 Coordinate transformation of a State Variable Model
8.1.2 DC gain of a state-variable model
KDC = lim
EE/ME 701: Advanced Linear Systems
Given a state-variable model with state vector x (t)
Y (s)
= lim C (s I A)1 B + D = C (A)1 B + (D104)
U (s) s j 0
C A1 B + D (105)
x (t) = A x (t) + B u (t)
(106)
y (t) = C x (t) + D u (t)
(107)
And an invertible transformation matrix T giving a new state vector z (t)
8.1.3 Interpreting D
z (t) = T x (t)
A direct transmission term corresponds to a transfer function that is not
strictly proper.
(108)
We can derive a new state model based on state vector z (t).
If D =0 , then
We can say that we have transformed the system from the coordinate
system of x (t) to the coordinate system of z (t).
the number of zeros equals the number of poles.
If D = 0 , we call the system strictly proper
Derivation of the transformation is straight forward. From Eqn (108) we
can solve for x (t)
If D 6= 0 , we call the system is proper, but not strictly proper
x (t) = T 1 z (t)
x (t) = T
z (t)
(109)
(110)
Plugging (108) and (109) into Eqns (106) and (107) gives
T 1 z (t) = A T 1 z (t) + B u (t)
(111)
From (111) we can write
z (t)
T A T 1 z (t) + T B u (t)
y (t) = C T
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 77
Part 5: Models of Linear Systems
(112)
z (t) + D u (t)
(Revised: Sep 10, 2012)
Page 78
EE/ME 701: Advanced Linear Systems
Section 8.2.0
EE/ME 701: Advanced Linear Systems
Eqn (112) gives the transformed state model
Equation (115) is a similarity transform.
b z (t) + Bb u (t)
z (t) = A
b = T A T 1
A
(113)
y (t) = Cb z (t) + D u (t)
with
Section 8.2.0
(114)
A similarity transform preserves eigenvalues,
b = T A T 1
A
(115)
Cb = C T 1
(117)
Bb = T B
b
eig (A) = eig A
The poles of system II are the same as the poles of the original system.
(116)
D : unchanged
Coordinate transformation is very powerful
(118)
We can convert a given model with x (t) to an equivalent model with z (t)
by choice of any invertible matrix T
The transformation is illustrated by figure 30.
This directly gives the set of all possible equivalent models !
The input and output are unchanged
Note that the D matrix, which directly couples u (t) to y (t), is unchanged
Only the internal representation of the system is changed
u(t)
A, B, C, D
y(t)
u(t)
A, B, C, D
x(t)
y(t)
z(t)
Figure 30: Block diagrams of original linear state-variable system and
transformed system.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 79
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 80
EE/ME 701: Advanced Linear Systems
Section 8.2.1
EE/ME 701: Advanced Linear Systems
Suppose we were interested in the suspension deflection
8.2.1 Example coordinate transformation
Consider the example of section 7.2.
m2
zs (t) = zw (t) zv (t)
zv(t)
If our interest was such that we wanted a state model with zs (t) and zs (t)
directly as states, we could introduce the transformation
vcar
ks
bs
m1
zs (t)
kw
r(t)
Inerial Reference
zw (t)
,
x (t) =
zv (t)
zv (t)
1 0 1
x (t) =
y1 (t) = zv (t) =
(kw +ks )
m1
mbs1 + mks1
+ mks2
+ mbs2
mks2
zw (t)
kw
m
+ mbs1
x (t) + 1 r (t)
0
1
0
mbs2
1
0
zw (t)
1 zw (t)
= T x (t)
0 zv (t)
zv (t)
1
(120)
(Revised: Sep 10, 2012)
(119)
a =
x1
x2
x3
x4
c =
y1
Page 81
1 0 1
0
1
0
Original State Variable Model (section 7.2.1)
i
zw (t)
+ [0] r (t)
0 0 1 0
zv (t)
zv (t)
Part 5: Models of Linear Systems
0 1
T =
0 0
0 0
We derived the model
zw (t)
So transformation T is given as:
Figure 31: Quarter vehicle suspension.
zs (t) 0 1
=
z (t) =
zv (t) 0 0
zv (t)
0 0
zw(t)
Road Surface
Section 8.2.1
x1
0
-440
0
4
x1 x2
0
0
x2
x3
1
0
-40
40
0
0
4
-4
x3 x4
1
0
Part 5: Models of Linear Systems
x4
0
40
1
-4
b =
x1
x2
x3
x4
d =
y1
u1
0
400
0
0
u1
0
(Revised: Sep 10, 2012)
Page 82
EE/ME 701: Advanced Linear Systems
Section 8.2.1
EE/ME 701: Advanced Linear Systems
9 State-variable Feedback control
Introduce the transformation
>> T = [ 1 0 -1 0 ; 0 1 0 -1 ; 0 0 1 0 ; 0 0 0 1]
T =
1
0
-1
0
0
1
0
-1
0
0
1
0
0
0
0
1
Put in State Feedback control, control signal u (t) is given as:
u (t) = K x (t) + N f r (t)
x1
x2
x3
x4
c =
y1
x1
0
-444
0
4
x1 x2
0
0
x2
x3
1
0
-44 -400
0
0
4
0
x3 x4
1
0
x4
0
0
1
0
Putting in feedback control is illustrated in figures 33, 34 and 35.
Input
u(t)
b =
u1
0
400
0
0
u1
0
x1
x2
x3
x4
d =
y1
Ap, Bp, Cp, Dp
System
Output
y(t)
x(t)
Figure 33: State-variable model of the open-loop system. This is the plant before
feedback control is applied, u (t) as input, and y (t) as output.
r(t)
r(t)
Nf
u(t)
y(t)
Ap, Bp, Cp, Dp
x(t)
x(t)
-K
Bode Diagram
Step Response
(121)
The control signal depends on the state vector and a reference input.
>> Ahat = T * A * inv(T);
Bhat = T * B
>> Chat = C1 * inv(T);
Dhat = D1
>> SSmodelHat = ss(Ahat, Bhat, Chat, Dhat)
a =
Section 9.0.0
50
1.4
New System
0
Magnitude (dB)
1.2
Figure 34: State-variable model of the closed-loop system with feed-forward gain
in input. The closed-loop system has r (t) as input and y (t) as output.
50
Amplitude
100
0.8
150
0
Phase (deg)
0.6
0.4
0.2
r(t)
90
Acl, Bcl, Ccl, Dcl
New System
0.5
1.5
Time (sec)
2.5
3.5
270
1
10
10
10
Frequency (rad/sec)
10
Of course, the step and frequency response are unchanged. The model
transformation changes only the internal representation of the system.
(Revised: Sep 10, 2012)
x(t)
10
Figure 32: Suspension step and frequency response of transformed model.
Part 5: Models of Linear Systems
y(t)
180
Page 83
Figure 35: State-space model of the closed-loop system, with r (t)as input and
y (t) as output.
Feedback control fundamentally transforms the system. Changing the statevariable model from figure 33 to 35.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 84
EE/ME 701: Advanced Linear Systems
Section 9.1.0
9.1 Determination of a new model with state feedback
EE/ME 701: Advanced Linear Systems
Section 9.1.0
Plugging the control law into the output equation, we find
To determine the state-variable model of the system with feedback, start
with the open-loop model (figure 33)
y (t) = C p x (t) + D p u (t) = C p x (t) D p K x (t) + D p N f r (t)
= (C p D p K) x (t) + D p N f r (t)
x (t) = A p x (t) + B p u (t)
(16, repeated)
y (t) = C p x (t) + D p u (t)
(17, repeated)
So we can write
y (t) = Ccl x (t) + Dcl r (t)
(125)
Ccl = C p D p K
(126)
Dcl = D p N f
(127)
with
Plugging the control law
u (t) = K x (t) + N f r (t)
(121, repeated)
Eqns (122)-(127) describe how we determine the state-variable model of the
system with feedback control.
into the state equation, we find
x (t) = A p x (t) + B p u (t) = A p x (t) B p K x (t) + B p N f r (t)
State feedback control is fundamentally different from single-loop,
compensated feedback:
= (A p B p K) x (t) + B p N f r (t)
Feedback is based on the state vector
So we can write
There is no compensator transfer function Gc (s)
x (t) = Acl x (t) + Bcl r (t)
(122)
Acl = A p B p K
(123)
Bcl = B p N f
(124)
The control gains form a vector, K Rmn , where m is the number of
inputs.
with
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 85
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 86
EE/ME 701: Advanced Linear Systems
Section 9.2.0
9.2 State-variable feedback example: Inverted pendulum
An inverted pendulum is a mechanism comprising a cart and pendulum that
holds the pendulum upright by feedback control. As illustrated in figure 36.
The system is open-loop unstable
EE/ME 701: Advanced Linear Systems
Section 9.2.0
Linearizing the equations about the operating point
0 = 0 , 0 = 0
For simplification, defining
p = I (M + m) + M m l 2
The system models an aspect of the challenge of rocket launch.
Choosing x (t) and forming the state-space model:
(t)
Ff=B z(t)
z (t)
0
Ap =
z (t)
,
x (t) =
(t)
(t)
z(t)
F(t)
Figure 36: Mechanical schematic of an inverted pendulum.
From Lagranges equations, the equations of motion for the inverted
pendulum:
(M + m) z (t) + b z (t) + m l (t) cos ( (t)) m l 2 (t) sin ( (t)) = F (t)(128)
I + m l 2 (t) + m l g sin ( (t)) + m l z (t) cos ( (t)) = 0 (129)
(I+m l 2 )
p
Bp =
ml
p
Cp =
(I+m l 2 ) b
p
( m2 g l 2 )
mpl b
m g l (M+m)
p
1 0 0 0
0 0 1 0
Dp =
0
0
Parameters
Mass of cart
[kg]
z (t)
Cart position
[m]
Mass of pendulum
[kg]
(t)
Pendulum Angle
[deg]
Length of Pendulum
F (t)
Applied force
[N]
Friction coef.
Inertial of pendulum
g = 9.8
Accel. of gravity
Part 5: Models of Linear Systems
[m]
kg m
2
[m/s2 ]
N
m/s
(Revised: Sep 10, 2012)
Page 87
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 88
EE/ME 701: Advanced Linear Systems
Section 9.2.0
EE/ME 701: Advanced Linear Systems
Example Data:
9.2.1 Designing a pole placement controller
M = 0.5;
i = 0.006;
m = 0.2;
g = 9.8;
>>
p = i*(M+m)+M*m*l^2,
>>
Ap = [
0
0
0
0
Ap =
0
0
0
0
b = 0.1;
l = 0.3;
p =
1
-(i+m*l^2)*b/p
0
-(m*l*b)/p
1.0000
-0.1818
0
-0.4545
>> Bp = [0;
(i+m*l^2)/p;
0;
m*l/p]
Bp =
Section 9.2.1
0
1.8182
0
4.5455
0.0132
0
(m^2*g*l^2)/p
0
m*g*l*(M+m)/p
0
2.6727
0
31.1818
>> Sdesired = [ -3 -4 -5 -6]
0;
0;
1;
0]
0
0
1.0000
0
1
0
7.0310
>> Acl = Ap - Bp * K
Acl =
0
1.0000
0
14.6939
13.9592 -63.2776
0
0
0
36.7347
34.8980 -133.6939
0
-12.7837
1.0000
-31.9592
>> Nf = 1/(Cp*inv(-Acl)*Bp)
Nf =
-8.0816
0
>> Cp = [ 1 0 0 0 ;
0 0 1 0 ]
Cp =
>> K = place(Ap, Bp, Sdesired)
K =
-8.0816
-7.7776
36.2727
0
0
0
1
0
0
>> SScl = ss(Acl, Nf(1)*Bp, Cp, Dp)
a =
x1
x2
x3
x4
x1
0
1
0
0
x2
14.69
13.96 -63.28 -12.78
x3
0
0
0
1
x4
36.73
34.9 -133.7 -31.96
b =
>> Dp = [ 0 ; 0 ]
Dp =
0
0
x1
x2
x3
x4
u1
0
-14.69
0
-36.73
Continuous-time model.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 89
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 90
EE/ME 701: Advanced Linear Systems
Section 9.2.1
With linear time-invariant systems there is a very nice optimal control result.
Consider the cost function
Step Response
1
0.5
J=
x (t)T Q x (t) + u (t)T R u (t) dt
(130)
t=0
0
Amplitude
Section 9.2.2
9.2.2 LQR Design
>> step(SScl)
To: Out(1)
EE/ME 701: Advanced Linear Systems
0.5
This cost function is minimized by a suitable choice of controller K
0.4
To: Out(2)
0.2
First LQR controller example
0.2
0.4
>> Q = diag( [1 1 1 1])
0
0.5
1.5
Time (sec)
2.5
3.5
Q =
Figure 37: Step response of Inverted Pendulum with control.
1
0
0
0
0
1
0
0
0
0
1
0
0
0
0
1
>> R = 1
R = 1
>> K = lqr(Ap, Bp, Q, R)
K =
-1.0000
-2.0408
>> Acl = Ap - Bp * K
Acl =
0
1.0000
1.8182
3.5287
0
0
4.5455
8.8217
20.3672
0
-34.3585
0
-61.3961
3.9302
0
-7.1458
1.0000
-17.8646
>> Nf = 1/(Cp*inv(-Acl)*Bp)
Nf =
-1.0000
0
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 91
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 92
EE/ME 701: Advanced Linear Systems
Section 9.2.2
Step Response
1.5
To: Out(1)
0.5
Amplitude
0.5
0.1
0.05
Step Response
1.5
0.05
1
0
4
Time (seconds)
To: Out(1)
0.1
Section 9.2.2
Example use of Brysons rules, to accelerate the response, place greater
cost on position error
>> Q = diag( [100 1 100 1])
Q =
100
0
0
0
0
1
0
0
0
0
100
0
0
0
0
1
>> K = lqr(Ap, Bp, Q, R)
K = -10.0000
-8.2172
38.6503
7.2975
>> Acl = Ap - Bp * K
>> Nf = 1/(Cp*inv(-Acl)*Bp)
Nf = -10.0000
0
>> Poles2 = eig(Acl)
Poles2 =
-6.9654 + 2.7222i
-2.2406 + 1.7159i
-6.9654 - 2.7222i
-2.2406 - 1.7159i
>> Poles1 = eig(Acl)
Poles1 =
-8.3843
-3.7476
-1.1020 + 0.4509i
-1.1020 - 0.4509i
To: Out(2)
EE/ME 701: Advanced Linear Systems
Figure 38: Step response of first LQR controller.
0.5
Amplitude
0.5
0.4
Second LQR controller example
0.2
To: Out(2)
Brysons rules: choose elements of Q and R to be 1/ x2i , where xi is the
allowed excursion of the ith state.
0.2
0.4
4
Time (seconds)
Figure 39: Step response of second LQR controller.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 93
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 94
EE/ME 701: Advanced Linear Systems
Section 10.0.0
10 Conclusions
Weve seen
Some of the basic properties of system models, and classification of signals.
How to build a state-variable model in 5 steps:
1. Write the relevant relations for the system
2. Identify the differential equations
3. Write the differential equations in state-variable form
4. Write the equation of the output signal (or signals) using the states and
inputs
5. Check units throughout, to verify correctness.
Advantages of state-variable modeling
List at least three reasons why state-variable models are advantageous
relative to differential equation modeling
Construction of several example state-variable models
Basic operations on a state variable model, including
Determining the transfer function
Coordinate transformation
State feedback control.
Part 5: Models of Linear Systems
(Revised: Sep 10, 2012)
Page 95
Student
exercise
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Solutions to the State Equation and
Modes and Modal Coordinates
Section 1.1.0
1 Modal coordinates
1.1 Derivation of modal coordinates
Contents
Considering the response of a linear system:
1 Modal coordinates
1.1
Derivation of modal coordinates . . . . . . . . . . . . . . . . . .
1.1.1
x (t) = A p x (t) + B p u (t)
(1)
y (t) = C p x (t) + D p u (t)
(2)
Choose the basis vectors to be the columns of the modal
matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1.2
Example transformation into modal coordinates . . . . . .
1.1.3
Interpretation of the transformations
. . . . . . . . . . .
10
. . . . . . . . . . . . . . . . . . .
12
1.2
Example, a double pendulum
1.3
Transformation of the state model to and from modal
coordinates (Similarity transform from A p to Am or back) . . . . .
1.3.1
1.3.2
18
1 (t)
(t) = ...
n (t)
19
2.1
Example, double pendulum revisited . . . . . . . . . . . . . . . .
2.2
General form for combining complex conjugate parts of a 2nd
order mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23
2.3
Deriving real basis vectors for a complex mode . . . . . . . . . .
25
2.4
Example with 2nd order modes, converting basis vectors to real . .
27
3 x (t) = A p x (t) defines A p -invariant spaces
(3)
where the i (t) are the basis coefficients and
Example, second-order system, 2 first-order modes.
19
is the representation of x (t) on basis vectors
Writing M =
e1 , e2 en
o
,
x (t) = M (t)
e1 , e2 en
o
.
so (t) = M 1 x (t)
(4)
30
4 Conclusions
Part 6: Response in Modal Coordinates
x (t) = i (t) ei
16
17
2 Complex eigenvalue pairs
Given a state vector x (t) Rn and a set of basis vectors {ei } for Rn , we
know that we can represent x (t) on basis {ei } by
i=1
Case of a full set of real, distinct eigenvalues
(diagonalizing A p ) . . . . . . . . . . . . . . . . . . . . .
. . .
where A p, B p , C p, D p form the model in physical coordinates.
31
(Revised: Sep 10, 2012)
Page 1
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
Section 1.1.0
Likewise, we can represent the input signal, B p u (t) on the same basis:
Section 1.1.1
1.1.1 Choose the basis vectors to be the columns of the modal matrix
i (t) ei = B p u (t)
EE/ME 701: Advanced Linear Systems
(5)
i=1
However, when the ei are the columns of the modal matrix of A p (assuming
for now a complete set of independent eigenvectors), then the terms
where the i (t) are the basis coefficients representing B p u (t) on {ei }.
Writing
i (t) A p ei
i (t) i ei
become
i (t) ei = M (t)
(6)
And Eqn (7) becomes
i=1
then
B p u (t) = M (t)
so (t) = M 1 B p u (t)
i=1
If we expand x (t) in the state equation using the representation on {ei }, we
find:
from:
x (t) = A p x (t)
+ B p u (t)
n
i (t) ei
i=1
i=1
i=1
i (t) A p ei + (t) ei
(7)
x (t) A p x (t) B p u (t) = 0
n
i (t) ei i (t) A ei (t) ei
i=1
i=1
= 0
(8)
i=1
Even though the i (t) and i (t) terms in Eqn (8) are scalar, because of the
middle term, with matrix A p, Eqn (8) leads in general to the vector equation
n
i=1
i (t) ei i (t) A ei i (t) ei = 0
i=1
i=1
i=1
(10)
Since the ei are independent, Eqn (10) is verified only if each term in
parenthesis is zero, which gives a set of simultaneous scalar equations (see
Bay Eqn (6.25))
i (t) i (t) i i (t) = 0
Rearranging the state equation gives (see Bay, Eqn (6.24))
from:
n
i (t) ei i (t) i ei i (t) ei = i (t) i (t) i i (t) ei = 0
(8, repeated)
i=1
Eqn (11) we know how to solve:
i (t) = i (t0 ) e (tt0 ) +
t0
e (t) i () d
(12)
And the full solution is given from Eqn (3)
i=1
(Revised: Sep 10, 2012)
Z t
(9)
x (t) = M (t)
which is not especially helpful.
Part 6: Response in Modal Coordinates
(11)
Representing the state and input on the modal matrix of A p , an nth order
coupled differential equation becomes a set of n first order uncoupled
differential equations !
i=1
!
n
n
i (t) I
i (t) A i (t) I ei = 0
i (t) = i i (t) + i (t)
or
Page 3
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 1.1.1
The basis vectors ei that decouple the system are the columns of the modal
matrix
i
h
1
(13)
e1 e2 en = M , recall A p = M J M
Eqn (13) is the general form.
eigenvectors exists, then
M =V ,
J =U
EE/ME 701: Advanced Linear Systems
1.1.2 Example transformation into modal coordinates
Consider the system governed by
When a complete set of independent
and
A p = V U V 1
(14)
Section 1.1.2
x (t) =
0.164
0.059
0.059
0.164
with initial condition
where V is the matrix of eigenvectors and U has the eigenvalues on the main
diagonal.
The eigensystem is:
x (t) +
x (t0 ) =
1
0
1.085
0.031
u (t)
(15)
>> [V, U] = eig(Ap)
V =
-0.7071
-0.7071
0.7071
-0.7071
U =
-0.2231
0
0
-0.1054
There is a complete set of independent eigenvectors. The modal matrix is
given by:
1 1
1
(16)
V , M=
M=
0.707
1 1
Eqn (16) illustrates that the basis vectors of the modal matrix can be
scaled by a parameter (actually each vector can be scaled independently).
By scaling M to 1.0, the some of the coefficients below get simpler.
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 5
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 1.1.2
The i are given from Eqn (5), which can be rewritten as
1 (t)
.. = M (t) = B u (t)
.
p
n (t)
so
(t) = M
B p u (t) =
0.527
0.558
The initial conditions are given from
u (t)
EE/ME 701: Advanced Linear Systems
Section 1.1.2
For a step input, which gives constant i (t), the form for the solution is:
1 t
e 1
i (t) = i (0) et + i
(17)
(20)
which gives
1 0.223t
1
e
0.223
1 0.105t
0.527
1
e
0.105
1 (t) = 0.5 e0.223t 0.558
(18)
2 (t) = 0.5 e0.105t
(21)
The transformation back to physical coordinates is given by:
(t0 ) = M 1 x (t0 ) =
0.5
0.5
(19)
And the two uncoupled solutions are (with t0 = 0)
1 (t) = 0.5 e0.223t 0.558
Z t
e0.223 (t) u () d
2 (t) = 0.5 e0.105t 0.527
Z t
e0.105 (t) u () d
x (t) = M (t) = e1 1 (t) + e2 2 (t) =
1
1
1 (t) +
1
1
2 (t) (22)
Eqn (22) can be used in two ways
1. Eqn (22) shows that the output x (t) is the superposition of n
contributions. Each contributing vector is a basis vector. They are the
columns of the modal matrix. And each has a basis coefficient i (t).
2. We can add up the contributions for an individual xi (t), such as:
x1 (t) = .5 e0.223t + 2.5 1 e0.223t + .5 e0.105t + 5 1 e0.105t
(23)
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 7
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 1.1.2
Putting the pieces together:
EE/ME 701: Advanced Linear Systems
Section 1.1.3
1.1.3 Interpretation of the transformations
From
1. By transformation into modal coordinates, the states are uncoupled.
rT1
..
(t) = M 1 x (t) =
.
rTn
2. The uncoupled equations can be solved
3. The solution in physical coordinates is found by transformation from modal
coordinates back into physical coordinates.
The order of an equation in the uncoupled equations can be:
x (t)
(24)
the jth row of M 1 gives the coupling of the physical states into the jth
mode. We can write that
1 (t) = rT1 x(t) = hr1, x(t)i
..
.
First, for a real pole
Second, for a complex pole pair
(25)
n(t) = rTn x(t) = hrn , x(t)i
Eqn (11) works directly for complex pole pairs, but it may be more
convenient to organize each pair of complex poles into a 2nd order realvalued mode (see section 2).
nth order for an n n Jordan block.
When there is not a complete set of eigenvectors, the Jordan form is used,
and gives l coupled states for each chain of l eigenvectors (one regular
and l 1 generalized).
From
x (t) = M (t)
(26)
each mode contributes to the physical states according to a column of M,
and each physical state is determined by a combination of modes according
to a row of M.
The response of a linear system can be thought of as a collection of firstand second-order mode responses with forcing functions.
Through the M matrix, the response of the physical states of the system
(voltages and velocities, say), generally involves a superposition of all of
the modes of the system.
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 9
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 1.1.3
EE/ME 701: Advanced Linear Systems
1.2 Example, a double pendulum
From (assuming B p is constant)
= M 1 B p
(27)
the input is coupled to each mode according to the elements of . Following
the example of Eqn (25)
i = ri , B p
(28)
Consider 2 masses connected by a spring, this system will have two oscillatory
modes:
a lower frequency mode in which the masses swing in phase, and
a higher frequency mode in which the mass swing in opposite phase.
Notice in particular that if there is an element in that is zero, there is no
forcing function for that mode in Eqn (12).
i (t) = i (t0 ) e (tt0 ) +
Z t
t0
e (t) i () d
Section 1.2.0
2
k
(Eqn (12), repeated)
u
Since the modes are uncoupled, if a i is zero, the input u (t) is not
connected to that mode.
m1
m2
Figure 1: Two mass system at rest.
We call the system expressed in terms of (t) and using Am , Bm , Cm , D p
as the system in modal coordinates, because each state variable is uniquely
associated with a mode.
In Phase
Opposite Phase
Figure 2: Modes of the two mass system.
The linearized equations of motion are:
m1 l 2 1 + d 1 + (k + m g l) 1 = k 2 + u
m2 l 2 2 + d 2 + (k + m g l) 2
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 11
Part 6: Response in Modal Coordinates
= k 2
(Revised: Sep 10, 2012)
(29)
(30)
Page 12
Section 1.2.0
With m = 2, l = 1, d = 3, k = 20, and g = 9.8 , we find:
1 (t)
1.50 19.80 0.00 10.00
1 (t)
1.00
0.00
0.00
0.00
, Ap =
x (t) =
2 (t)
0.00 10.00 1.50 19.80
2 (t)
0.00
0.00
1.00
0.00
EE/ME 701: Advanced Linear Systems
Section 1.2.0
0.5
Theta 1
EE/ME 701: Advanced Linear Systems
(31)
0.5
0
10
5
6
t [seconds]
10
Theta 2
0.5
0.5
0
Figure 4: Two masses swinging in phase (mode 2, mode 1 not excited).
2
Theta 1 [degrees]
A slower, in-phase mode, excited from the initial condition (figure 4)
h
iT
xT (0) = 0 1 0 1
The system has two 2nd order modes:
A faster, out-of-phase mode, excited from the initial condition (figure 3)
h
iT
xT (0) = 0 1 0 1
We can excite the modes individually. Setting up initial conditions that lie
in the column space formed by the basis vectors of the mode.
General initial conditions give motion that is a superposition of all modes.
1
0
10
5
6
t [seconds]
10
1
Theta 2 [degrees]
Theta 1 [degrees]
1
0.5
0
0
0.5
0.5
1
0
1
0
1
10
Figure 5: Two masses swinging, both modes excited. (Figure 5 is the
superposition of figures 4 and 3.)
1
Theta 2 [degrees]
0.5
0.5
0
0.5
Notes:
1
0
5
6
t [seconds]
10
Figure 3: Two masses swinging out of phase (mode 1, mode 2 not excited).
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 13
The unforced response (to initial conditions) is considered in figures 3-5.
The forced response (to u(t)) can also be understood in terms of modes.
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 1.2.0
For an n dimensional system there may be n1 first order modes and n2
second order modes, where n = n1 + 2 n2
In modal coordinates, we can write the system dynamics as:
(t) = Am (t) + (t)
1.3 Transformation of the state model to and from modal
coordinates (Similarity transform from A p to Am or back)
(32)
Choosing the modal matrix M as a special transformation matrix, and
introducing (t), the state vector in modal coordinates,
x(t) = M (t)
Am = M 1 A p M
(33)
A p = M Am M 1
(34)
If the system has second order modes, Am will have 2x2 blocks, one
corresponding to each 2nd order mode.
Then:
(t) =
Or:
(t) = = Am (t) + Bm u (t)
y (t) = Cm (t) + D p u (t)
Where:
Am = M 1 A p M
Bm = M 1 B p
Cm = C p M
Modes give distinct contributions to the total output, such as
second order:
M 1 A p M x(t) + M 1 B p u(t)
y (t) = C p M (t) + D p u (t)
If the system requires the Jordan form, Am will have a block
corresponding to each Jordan block.
The important physical property of modes is that they are uncoupled. In
figure 4 mode 1 evolves without mode 2, and likewise in figure 3. In figure
5 both modes evolve, independently.
(35)
(t) = M 1 x(t)
If the system has only first order modes, Am is diagonal (all blocks are
1x1).
first order:
Section 1.3.0
Weve seen that we can transform a state model to a new set of basis vectors
with a transformation matrix.
where Am is a block diagonal matrix,
and
EE/ME 701: Advanced Linear Systems
In the general case, Am is block diagonal:
y(t) = a1e1t
1x1 block for every real eigenvalue
y(t) = a2e2t cos(2t + 2)
2x2 block for every pair of complex eigenvalues
Modal coordinates are best (and perhaps only) understood in state space.
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 15
l l block for every l l block in the Jordan form (if needed)
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 16
EE/ME 701: Advanced Linear Systems
Section 1.3.1
1.3.1 Case of a full set of real, distinct eigenvalues
(diagonalizing A p)
U =
1
...
n
..
.
..
.
Section 1.3.2
1.3.2 Example, second-order system, 2 first-order modes.
When we have real and distinct eigenvalues we will have a complete set of
independent eigenvectors, and
EE/ME 701: Advanced Linear Systems
Given the system of section 1.1.2. Recall that the solution for x (t) is of the
form
1
1
1 (t) +
2 (t)
x (t) = M (t) = e1 1 (t) + e2 2 (t) =
1
1
Figure 6 is a plot of the phase portrait, showing the state trajectory from
initial condition
1
x (0) =
0
V = v1 vn
..
..
.
.
A phase portrait is a plot of the state as a function of time.
It can be in any coordinate frame.
It will be an n-dimensional plot, for an nth order system.
Then
M =V
and Am = M
Ap M
(36)
1
The modal matrix is the eigenvector matrix.
x2
The system matrix in model coordinates, Am, is the diagonal matrix of
eigenvalues.
e2
x2
0.5
x(t)
x1
0.5
e1
1
0.5
0.5
x1
1.5
Figure 6: Response of a system with two first-order modes. The x show the
state at 1.0 second intervals.
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 17
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 2.1.0
2 Complex eigenvalue pairs
EE/ME 701: Advanced Linear Systems
When each pole is distinct, the state-response of a system is the
superposition of the individual modal responses. Putting together Eqns (25)
and (24) above, the response due to initial condition and each mode is given
by:
Complex pole pairs correspond to 2nd order, oscillatory modes.
2.1 Example, double pendulum revisited
Considering the double pendulum example of section 1.2, the A p matrix
gives the eigenvectors and eigenvalues:
0.70 + j 0.00, 0.70 j 0.00, +0.68 + j 0.00, +0.68 j 0.00
+0.02 + j 0.13, +0.02 j 0.13, 0.05 j 0.21, 0.05 + j 0.21
V =
+0.70 + j 0.00, +0.70 j 0.00, +0.68 + j 0.00, +0.68 j 0.00
0.02 j0.13, 0.02 + j0.13, 0.05 j 0.21, 0.05 + j 0.21
Am =
0.75 + j 5.41
0.00
0.00
0.00
0.00
0.75 j 5.41
0.00
0.00
0.00
0.00
0.75 + j 3.04
0.00
0.00
0.00
0.00
Section 2.1.0
0.75 j 3.04
x (t) = e1 1 (t) + e2 2 (t) + e3 3 (t) + e4 4 (t)
(37)
i (t) ei = eit ei rTi x(0)
(38)
with
When the poles are complex, Eqns (37), (38) are none-the-less valid. For
the example above,
1 (t) e1
2 (t) e2
e(0.75+ j5.41)t
e(0.75 j5.41)t
0.70 + j 0.00
+0.02 + j 0.13
+0.70 + j 0.00
0.02 j0.13
0.70 j 0.00
+0.02 j 0.13
+0.70 j 0.00
0.02 + j0.13
h
i
0.36 j 0.05, +0.00 j 1.98, +0.36 + j 0.05, 0.00 + j 1.98 x (0)
h
i
0.36 + j 0.05, +0.00 + j 1.98, +0.36 j 0.05, 0.00 j 1.98 x (0)
Making the example very specific, with
Corresponding to 2 complex modes (figures 3 and 4).
x (0) =
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 19
Part 6: Response in Modal Coordinates
and
t = 0.6
(Revised: Sep 10, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 2.1.0
Matlab code to evaluate Eqn (37):
EE/ME 701: Advanced Linear Systems
Section 2.1.0
The example above illustrates
>> [V, U] = eig(Ap);
>> Vinv = inv(V);
>> v1 = V(:,1)
v1 =
-0.6955 + 0.0000i
0.0175 + 0.1262i
0.6955
-0.0175 - 0.1262i
>> r1 = Vinv(1,:)
r1 =
-0.36 - 0.05i
0.00 - 1.98i
>> v2 = V(:,2) %% v1*
>> r2 = Vinv(2,:) %% r1*
>> x0 = [ 1;2;-1;-2]
>> t = 0.6
>> xi1 = exp(U(1,1)*t) * v1 * r1 * x0
xi1 =
0.0477 - 3.5723i
-0.6494 + 0.0813i
-0.0477 + 3.5723i
0.6494 - 0.0813i
>> xi2 = exp(U(2,2)*t) * v2 * r2 * x0
xi2 =
0.0477 + 3.5723i
-0.6494 - 0.0813i
-0.0477 - 3.5723i
0.6494 + 0.0813i
If i is complex, ei and ri will in general also be.
In this case Eqn (37) can make a complex contribution xi (t), but complex
terms always come in complex conjugate pairs, and two equations like
(37) make up the mode.
We can combine the two 1st order complex terms into a single 2nd order
real term.
0.36 + 0.05i
-0.00 + 1.98i
>> RoundByRatCommand(xi1+xi2) %% First and second terms
ans =
0.0954
%% create a real-valued 2nd order mode
-1.2988
-0.0954
1.2988
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 21
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 22
EE/ME 701: Advanced Linear Systems
Section 2.2.0
EE/ME 701: Advanced Linear Systems
2.2 General form for combining complex conjugate parts of a
2
nd
Section 2.2.0
All the imaginary terms cancel (as they must !) and the result reduces to:
order mode
x(t) = 2 a er rr T b er ri T a ei ri T b ei rr T x (0)
Considering two terms from Eqn (37), a 2nd order mode makes a
contribution to x (t) according to:
x(t) = [ (er + jei ) (er jei ) ]
...
e(+ j )t
e( j )t
..
.
(rr + jri )T
(rr jri )T
..
...
.
(39)
Define a + j b as
a + j b = e(+ j )t = et (cos( t) + j sin( t))
(40)
Or, equivalently,
a = et cos ( t) ,
Which is given by:
x(0)
x(t) = [ er ei ]
b a
rTr
rTi
x(0)
(42)
Considering the numerical example above
>> alpha = real( U(1,1) )
alpha = -0.7500
>> omega = imag( U(1,1) )
omega =
5.4072
>> a =
a =
>> b =
b =
b = et sin ( t)
(41)
exp(alpha*t) * cos(omega*t),
-0.6343
exp(alpha*t) * sin(omega*t),
-0.0654
Multiplying out Eqn (39) gives:
x(t) = [a er rr T + j a er ri T + j b er rr T b er ri T
+ j a ei rr T a ei ri T b ei rr T j b ei ri T
a er rr T j a er ri T j b er rr T b er ri T
j a ei rr T a ei ri T b ei rr T + j b ei ri T ] x(0)
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 23
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 24
EE/ME 701: Advanced Linear Systems
Section 2.3.0
2.3 Deriving real basis vectors for a complex mode
EE/ME 701: Advanced Linear Systems
Section 2.3.0
When M has a complex conjugate pair of eigenvectors
Going back to the definition of modal coordinates, starting with
M = ei ei
x (t) = A p x (t) + B p u (t)
Convert to modal coordinates
ei = er + j e j
with
(44)
The 2 2 element in N that selects the real and imaginary parts of ei is:
(t) = = Am (t) + Bm u (t)
11 j
N2 =
2 1 j
with
Am = M 1 A p M
Bm = M 1 B p
Then
Am
(Revised: Sep 10, 2012)
(45)
...
|
(43)
Part 6: Response in Modal Coordinates
j j
Now considering Am . The complex pair of eigenvectors will have complex
conjugate eigenvalues
and
M = er e j
| |
Then
Am = N 1 M 1 A p M N = N 1 Am N
N2
...
N =
We can introduce a modification to the modal matrix by:
M 1 = N 1 M 1
N21 =
which gives
Choose N to be
M = MN
Page 25
1
= N Am N =
...
1
j j
Part 6: Response in Modal Coordinates
...
...
i
i
...
1
2
(Revised: Sep 10, 2012)
...
(46)
1 j
...
1
Page 26
EE/ME 701: Advanced Linear Systems
Section 2.4.0
...
=
2
...
Note, if it is convenient, we can scale the columns of M as desired. For
example, scaling the largest element of each column to 1.0 gives
(47)
(48)
2.4 Example with 2nd order modes, converting basis vectors to
1.00
1 j
N=
2
0
0
then
j 0 0
0 1 1
0 j j
Finding the system matrix in Modal coordinates,
real
The double pendulum has two second order modes. Using the example from
section 2.1, choose
0.03 1 0.08 1
M =
1.00 0
1
0
0.03 1 0.08 1
where
= + j
Section 2.4.0
...
+
j ( )
=
j ( ) +
EE/ME 701: Advanced Linear Systems
0.75 5.41
5.41
1 1
1
Am = N M A p M N = N Am N =
0.75
0
0
0.75 3.04
3.04 0.75
0
The first mode oscillates in a vector subspace of state space, given by
S1 = {x : x = 1 e1 + 2 e2 }
0.70
0.00
0.68
0.02 0.13 0.05 0.21
M =NM =
0.70 0.00 0.68
0
0.02 0.13 0.05 0.21
Part 6: Response in Modal Coordinates
where e1 and e2 are the first two columns of M .
(Revised: Sep 10, 2012)
The second mode oscillates in a vector subspace of state space, given by
S2 = {x : x = 3 e3 + 4 e4 }
where e3 and e3 are the third and fourth columns of M .
Page 27
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 28
EE/ME 701: Advanced Linear Systems
Section 2.4.0
For example, the initial condition of figure 3 in physical and modal
coordinates is:
1
,
x (0) =
(0) = M p
Section 3.0.0
3 x (t) = A p x (t) defines A p-invariant spaces
Considering the unforced response of
1.0
x (0) =
EE/ME 701: Advanced Linear Systems
x (t) = A p x (t) + B p u (t)
(49)
and the initial condition of Eqn (49) excites only the first (faster) mode.
(that is, u (t) = 0), a real i and ei define a 1-D A p -invariant subspace. That
is, if
x (t = 0) {x : x = ai ei }
then
x (t) {x : x = ai ei }
x (0) =
,
0
1
(0) = M p
x (0) =
1.0
If
(50)
(Revised: Sep 10, 2012)
x (0) {x : x = a1 er + a2 ei }
then
x (t) {x : x = a1 er + a2 ei } t > 0
and the initial condition of Eqn (50) excites only the second (slower) mode.
Part 6: Response in Modal Coordinates
(51)
In the same way, complex eigenvalues define a 2-D A p-invariant subspace,
with
and
ei = imag (vi )
(52)
er = real (vi )
The initial condition of figure 4 is
t > 0
Page 29
(53)
Eqn (53) implies that rr , ri {x : x = a1 er + a2 ei } , which is verified
numerically for the example above.
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 30
EE/ME 701: Advanced Linear Systems
Section 4.0.0
4 Conclusions
The response of linear dynamic systems can be decomposed into the
responses of individual modes.
The modes are decoupled: if the input or initial condition excites only
one mode, only that mode responds.
Modes are either
First order, or
Second order, or
Correspond to a Jordan block
The basis vectors of the modes (vectors ei in Eqn (3) et seq.) are the bases
of A p -invariant subspaces in state-space, defined by the modes.
The modal matrix M is used to transform between modal and physical
coordinates.
The forcing function, u (t), can also be decomposed into forcing functions
for each of the modes.
Part 6: Response in Modal Coordinates
(Revised: Sep 10, 2012)
Page 31
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Phase Portraits and Lyapunov Stability
Section 1.0.0
1 Introduction to Stability Theory
We will consider the stability of
Contents
Linear time-invariant systems
1 Introduction to Stability Theory
Linear time-varying systems
2 The phase plane (or phase portrait)
Nonlinear systems
2.1
Phase space in higher dimensions . . . . . . . . . . . . . . . . . .
2.2
Local stability, definitions . . . . . . . . . . . . . . . . . . . . . .
2.3
Local stability, additional terminology . . . . . . . . . . . . . . .
13
2.4
Determining local stability . . . . . . . . . . . . . . . . . . . . .
14
3 Limit Cycles
15
4 Lyapunov stability
17
4.1
Generalization of Lyapunovs energy function: . . . . . . . . . . .
20
4.2
Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . .
21
4.3
Lyapunov stability of a linear system
22
. . . . . . . . . . . . . . .
easy, given by the eigenvalues
harder, some unexpected results
in general quite hard
Stability of linear time-invariant systems is relatively simple:
Examine poles (eigenvalues of the system matrix)
LTI systems can show exponential decay, marginal stability, or
exponential growth
Linear time-varying systems:
We cant look at the succession of instantaneous systems
Even if the eigenvalues are always in the left half-plane, the system can
be unstable
In discrete time, a sequence of stable systems can be unstable.
More powerful tools needed.
5 Summary
23
For nonlinear systems the picture is even more complex:
Local stability,
Properties for small inputs may not extend to large input
Stability may be input dependent
Stability may be initial-condition dependent
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 1
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
Section 1.0.0
EE/ME 701: Advanced Linear Systems
Section 2.0.0
2 The phase plane (or phase portrait)
A tool is needed to understand stability properties in general cases
For time-varying and nonlinear systems, various flavors of stability:
For general systems given by:
Stable (or delta-epsilon stable)
x (t) = f (x, u, t)
If the system starts within of the equilibrium point, it never goes
outside of distance from the equilibrium point.
(1)
y (t) = g (x, u, t)
Uniform stability
we need a tool for looking at the behavior of the system.
Stability is not dependent on the time (for time-varying systems).
One general and powerful tool is the phase plane.
Asymptotic stability
Example, consider the undamped system :
As t , the state draws ever closer to the equilibrium point.
d 2y
+ 2 y = 0
dt 2
Exponential stability
As t , the state draws ever closer to the equilibrium point at
least as fast as some exponential curve.
x (t) =
x (t) =
0 2
1
Undamped oscillator response
A time-plot shows:
1
0.8
0.6
State signals
0.4
0.2
0
0.2
0.4
0.6
0.8
1
0
10
15
20
25
time [sec]
An nth order system will have n curves in the time plot.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 3
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 2.0.0
Phase plane (continued)
EE/ME 701: Advanced Linear Systems
Section 2.0.0
Phase plane (continued)
To draw the phase portrait, take away time, and make the axes of the plot
the states (2-D plot for 2nd order system, 3-D plot for 3rd order system, etc.)
A path is called a phase trajectory or simply a trajectory .
We can consider any number of trajectories, to create a phase portrait.
Undamped oscillator response
Undamped oscillator response
1
1.5
Velocity [m/s]
0.5
Velocity [m/s]
0.5
0.5
0.5
0.5
0
Position [m]
0.5
Figure 1: Trajectory on phase plane.
1.5
1.5
0.5
Consider the damped oscillator.
0
Position [m]
0.5
1.5
Figure 3: Phase portrait, several trajectories.
Undamped oscillator response
1
Undamped oscillator response
Each point on a phase plane corresponds to one specific value of state, x .
0.5
Velocity [m/s]
0.8
State signals
0.6
0.4
An important property of phase portraits comes from the very definition of
state:
0.2
0.5
0
What is state, it is all the information needed to determine the future
trajectory of the system (given inputs).
0.2
1
0.4
0.6
0
4
time [sec]
0.5
0
Position [m]
0.5
Corollary: the trajectory departing from a point xa is a function only of
xa , with no dependence on how the trajectory arrived to state xa .
Figure 2: Damped oscillator.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 5
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 2.0.0
Phase plane (continued)
Section 2.2.0
2.1 Phase space in higher dimensions
At each point in phase-space (at each possible state) there is a direction of
departure.
The concept of a trajectory through phase space extends naturally to higherorder systems.
We can plot the phase arrows along trajectories, or at any point, such as grid
points.
There are two challenges:
Undamped oscillator response, vector field
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0
0.2
An equilibrium point is a steady-state operating point of a system, it is a
point xe where:
x = f (xe , u, t) = 0
(2)
0.4
0.6
0.6
0.8
0.8
2.2 Local stability, definitions
0.2
0.4
For homogeneous linear systems the origin is always an equilibrium
point
x = A 0 = 0
1
1
0.5
0
Position [m]
0.5
1. We cant easily plot 3rd and higher order phase portraits.
2. The theorem that phase trajectories can not cross is less useful.
Undamped oscillator response, vector field
Velocity [m/s]
Velocity [m/s]
EE/ME 701: Advanced Linear Systems
(a) Along trajectories
0.5
0
Position [m]
0.5
If A has a null space, there will be infinitely many equilibrium points
(a null space in A corresponds to a pole at the origin, consider they
dynamics of a car, there are infinitely many places your car can stop).
(b) At grid points
Figure 4: Phase portrait, plotting x vectors at x points.
To plot 4(b), we need only be able to compute x (x), it is not necessary to
solve the differential equation.
If there is an input, the equilibrium will be shifted:
x (t) = A x (t) + B u ,
xe : A xe + B u = 0
A handy feature for nonlinear systems.
In 2-D we have powerful theorem: phase trajectories can not cross.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 7
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 2.2.0
EE/ME 701: Advanced Linear Systems
Section 2.2.0
Equilibrium points come in four flavors
1. Stable node
3. Saddle point
2. Unstable node
4. Center
Stable node, the phase trajectories travel into xe ,
Unstable node, the phase trajectories travel away from xe ,
Figure 6: Stable and unstable equilibrium points, 2nd order modes.
Figure 5: Stable and unstable equilibrium points, 1st order modes.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 9
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 2.2.0
Saddle Point: Trajectories move in on some paths and out on others.
EE/ME 701: Advanced Linear Systems
Section 2.2.0
Center, marginal stability (the first example).
Figure 8: Phase portrait for marginal stability.
Figure 7: Saddle points, system has both stable and unstable modes.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 11
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 12
EE/ME 701: Advanced Linear Systems
Section 2.3.0
2.3 Local stability, additional terminology
EE/ME 701: Advanced Linear Systems
Section 2.4.0
2.4 Determining local stability
As you would expect, a variety of terminology is applied to the
characteristics of differential equations.
Consider again the most general model,
x = f (x, u, t)
In the controls literature, one will sometimes see an equilibrium point
referred to as:
(3)
With x = xe + x , x (x) is given by the Taylor series expansion by:
A fixed point
x (x) = x (xe ) +
An attractor (if the system is stable).
If the fixed point is called an attractor , we can call the region (of
state space) from which trajectories converge to the attractor the basin of
attraction.
Figure 7.2 of Bay illustrates multiple attractors and basins of attraction of a
pendulum. Notice stable and unstable equilibria.
f (x, u, t)
x + O x2
x
x=xe
(4)
0+J x
The derivative of ha vector is a matrix of thei individual scalar derivatives.
T
, then:
With f (x, u, t) = f1 () f2 () fn ()
f1
x1
f2
x1
f1
x2
f2
x2
..
.
..
.
f1
xn
f2
xn
f (x, u, t)
J=
=
x
x=xe
.. fn
fn fn
. xn
x1 x2
x=xe
Near to xe , the first-order term will dominate and the stability properties
(stable vs. unstable vs. saddle point vs. center) is determined by the
f (x, u,t)
, which is sometimes called the Jacobian matrix.
eigenvalues of
x
Local stability is usually computable, because we can usually form the
Jacobian matrix, even for non-linear systems.
Figure 9: Multiple attractors and basins of attraction (Bay figure 7.2).
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 13
Local stability is a limited result. For example, even for a stable fixed point,
it may be difficult to determine the size of the basin of attraction.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 3.0.0
3 Limit Cycles
EE/ME 701: Advanced Linear Systems
Section 3.0.0
Nonlinear systems have trajectories which are closed curves.
Contrasting marginal stability (figure 8), a phase portrait for marginal
stability has closed curves, but these are isolated, and are not the limit of
a trajectory.
Consider different stability cases
Establishing the existence of a limit cycle has no general solution
Limited solution for the 2D case:
Poincares Index: for the region enclosed by a closed curve, write
n = N S
where N is the number of centers, foci and nodes,
and S is the number of enclosed saddle points.
For the closed curve to be a limit cycle, it is necessary but not
sufficient that n = 1 .
Summary on Stability
The local stability of a (sufficiently smooth) nonlinear system can be
determined by examining the stability of the equilibrium points.
Nonlinear systems can have a new kind of behavior: a limit cycle.
Needed: a global stability result.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 15
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 16
EE/ME 701: Advanced Linear Systems
Section 4.0.0
4 Lyapunov stability
EE/ME 701: Advanced Linear Systems
Section 4.0.0
With nonlinear stiffness and damping
Goal: address stability without solving the differential equation.
k (y)
Lyapunov method:
b (y)
Introduce an energy function (or, in general, a Lyapunov function)
Show that energy decays everywhere, except at the equilibrium point,
where it is zero.
Write the state space model:
x=
First published in Russian in 1892,
Translated into French in 1907
x =
Fist use in controls in 1944
y
0
x1
x2
k (y) b (y)
Consider the second order system
Assuming unit mass, the sum of potential and kinetic energy is given by:
E (x1 , x2 ) =
x22
+
2
k (x1 ) dx1
Looking at the rate of change of energy in the system:
dE
= x2x2 + k (x1 ) x1
dt
= x2 [x2 + k (x1 )]
But from the dynamics, x2 = k (x1 ) b (x2), and so
dE
dt
becomes:
dE
= x2 [k (x1 ) b (x2) + k (x1)]
dt
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 17
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 4.0.0
EE/ME 701: Advanced Linear Systems
Section 4.1.0
4.1 Generalization of Lyapunovs energy function:
Introduce a scalar function of the system state:
V (x)
which is positive-definite
V (x) > 0 ,
x 6= 0
(5)
Evaluate the Lyapunov derivative:
dV
V d x
=
dt
dx dt
Conclusions:
(6)
and show:
If b () = 0 and the system has only a spring,
in the system remains constant.
dE
dt
= 0 and the total energy
dV
< 0 , x 6= 0
(7)
dt
Which establishes the Bounded-input bounded-output (BIBO) stability:
If b > 0 x2 the system is dissipative, and
V (t) V (t0)
As long as
then :
t > t0
(8)
E>0
dE
dt
Challenge: how do we find V (x) ?
<0
Energy function
Clever choice
Observations:
For any initial condition giving energy E0 , the system never goes to a
state with E (x) > E0
When b > 0 energy steadily decays, and the system converges to
E = 0.
Typical choice:
V (x) = xT P x ,
P : positive definite matrix
(9)
Of course, failure to find a suitable V (x) does not prove instability.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 19
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 4.2.0
EE/ME 701: Advanced Linear Systems
4.2 Quadratic Forms
Section 4.3.0
4.3 Lyapunov stability of a linear system
Physical energy is often quadratic in the state variables
Given
x (t) = A x (t)
Consider Vc = 12 CV 2
(11)
Introduce the Lyapunov function
Because we want a positive definite function V , a quadratic form is often
chosen for a Lyapunov function.
V (x) = xT P x
(12)
General quadratic form:
V (x) = xT P x
(10)
where P is a symmetric positive definite matrix.
Where
Then V (x) is given by
P is a matrix
V (x) = x T P x + xT P x
It can be shown (see chapter) P may be restricted to symmetric matrices
without loss of generality
(What is the advantage of a symmetric matrix, in terms of the
eigensystem of P?)
= xT AT P x + xT P A x = xT Q x
Student
exercise
where
Q = AT P + P A
If P is positive-definite
(13)
(14)
Q positive definite proves the stability of system (11).
xT P x > 0 x 6= 0
Eqn (14) is called the matrix Lyapunov equation.
Requirement for a PD matrix:
For a stable linear system, Eqn (14) has a solution for every positive
definite P
All eigenvalues > 0.
P may be written P = RT R
(Student exercise, prove that any P written P = RT R is symmetric and
positive (semi)definite, for arbitrary choice of R.)
Equations (11)-(14) show the concept applied to a linear system
And provide a starting point for studying nonlinear systems.
P is positive-semidefinite if all eigenvalues 0, and some eigenvalues
may be 0.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 21
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 22
EE/ME 701: Advanced Linear Systems
Section 5.0.0
5 Summary
Weve seen that stability is more complex for nonlinear systems
Property of system, and also of initial conditions and inputs
(Think of the stability of a plane in flight).
A system at equilibrium and perturbed can:
Return to equilibrium
Diverge
Go to a different equilibrium point
Go into a limit cycle
The Lyapunov stabiltiy method gives one approach for non-linear systems.
Part 7: Phase Portraits and Stability
(Revised: Sep 10, 2012)
Page 23
EE/ME 701: Advanced Linear Systems
EE/ME 701: Advanced Linear Systems
Controllability and Observability of
Linear Systems
3.2
3.3
Contents
1 Introduction
1.1
Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2
Main facts about uncontrollable or unobservable modes: . . . . .
2 Tests for Controllability and Observability
Controllability, discrete time system . . . . . . . . . . . . . . . .
2.2
Controllability, continuous time system . . . . . . . . . . . . . .
Polynomial expressions on matrices, and the CayleyHamilton Thrm . . . . . . . . . . . . . . . . . . . . . . .
Controllability for continuous-time systems . . . . . . . .
10
2.3
Controllability example . . . . . . . . . . . . . . . . . . . . . . .
12
2.4
Observability, discrete time system . . . . . . . . . . . . . . . . .
15
2.5
Observability, continuous time system . . . . . . . . . . . . . . .
18
2.6
Observability example . . . . . . . . . . . . . . . . . . . . . . .
18
2.7
Popov-Belevitch-Hautus Tests . . . . . . . . . . . . . . . . . . .
19
2.7.1
PBH Test of Controllability . . . . . . . . . . . . . . . .
19
2.7.2
PBH Test of Observability . . . . . . . . . . . . . . . . .
20
2.2.2
2.8
Controllability and Observability in modal coordinates.
. . . . .
3 Controllability and observability example
3.1
3.2.1
A badly placed actuator . . . . . . . . . . . . . . . . . .
34
Summary Controllability and Observability . . . . . . . . . . . .
37
3.3.1
38
Uncontrollable and unobservable modes and rocketry . . .
4 Additional topics in Bay
4.1
39
Alternative definitions of Controllability and Observability . . . .
39
21
26
Testing Controllability and Observability . . . . . . . . . . . . . .
Part 8: Controllability and Observability
31
2.1
2.2.1
A badly placed sensor . . . . . . . . . . . . . . . . . . . . . . . .
(Revised: Sep 10, 2012)
30
Page 1
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 2
EE/ME 701: Advanced Linear Systems
Section 1.1.0
1 Introduction
EE/ME 701: Advanced Linear Systems
Section 1.1.0
Figure 1(B) shows an unobservable system.
No discussion of State-Variable models would be complete without a
discussion of observability and controllability (R. E. Kalman [1, 2]).
y(t)
s+3
u(t)
(s+3) (s+4)
Modes correspond to energy storage elements
When a pole and a zero cancel, the mode - nonetheless - remains in the
system:
(A) System with pole-zero cancellation.
Example: a pole-zero cancellation (fig 1(A)).
u(t)
y(t)
s+3
(s+3) (s+4)
The mode disappears from the transfer function.
The mode - and energy storage element - remains in the system:
(B) Unobservable system (state hidden from output)
Either the signal of the mode does not appear in the output
(Unobservable, fig 1(B));
Or the input signal cant reach the mode (Uncontrollable, fig 1(C)).
1.1 Definitions
u(t)
s+3
y(t)
(s+3) (s+4)
(C) Uncontrollable system (state untouched by input)
Controllable System: The input signal reaches each of the modes, so that the
system can be driven from any state, x0 , to the origin by suitable choice of
input signal.
Figure 1: A pole-zero cancellation can physically occur in either the input or
output path.
Figure 1(C) shows an uncontrollable system.
Observable System: The response of each mode reaches the output. An initial
condition, x0, can be determined by observing the output and input of the
system.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 3
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 4
EE/ME 701: Advanced Linear Systems
Section 1.2.0
1.2 Main facts about uncontrollable or unobservable modes:
Even if a pole and zero cancel, the system still has that mode in its response.
EE/ME 701: Advanced Linear Systems
Section 2.1.0
2 Tests for Controllability and Observability
2.1 Controllability, discrete time system
Consider example 3.10 in [Bay]. For the LTI discrete-time system:
Challenge: if a pole and zero cancel, it means that feedback control can do
nothing about the mode:
x (k + 1) = A x (k) + B u (k)
If the mode is unstable, feedback control cannot stabilize the system.
(1)
y (k) = C x (k) + D u (k)
If the mode is slow or lightly damped, it will remain slow or lightly
damped.
If the mode is fast and well damped: being uncontrollable or
unobservable is often not a problem.
Exception: rocket, the stress introduced by oscillation of an
uncontrollable or unobservable mode can over-stress the structure.
How do uncontrollable or unobservable modes get excited:
Controllability, example 3.10: Given and arbitrary initial condition x (0),
under what conditions will it be possible find an input sequence,
u (0) , u (1) , ..., u (l) which will drive the state vector to zero, x (l) = 0.
Solution:
Consider the output for a few samples of the input:
At time k = 0 :
Disturbances (dont go through zeros of control input path),
Initial conditions,
x (1) = A x (0) + B u (0)
(2)
Non-linearities, etc.
At time k = 1 :
x (2) = A x (1) + B u (1)
(3)
= A [A x (0) + B u (0)] + B u (1)
= A2 x (0) + A B u (0) + B u (1)
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 5
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 6
EE/ME 701: Advanced Linear Systems
Section 2.1.0
At time k = 2 :
EE/ME 701: Advanced Linear Systems
Section 2.1.0
Define the controllability matrix to be:
x (3) = A x (2) + B u (2)
i
h
= A A2 x (0) + A B u (0) + B u (1) + B u (2)
= A3 x (0) + A2 B u (0) + A B u (1) + B u (2)
P , B | A B | ... | Al2 B | Al1 B
and also
The ascending powers of A arise because x (k + 1) depends on x (k)
(recursion).
For 0 = x (l), rearranging Eqn (4) gives:
then
x (l) Al x (0) = Al x (0) = B u (l 1) + A B u (l 2) + ... + Al1 B u (0)
(5)
The right hand side of (5) can be put in matrix-vector form, to give:
u
(l
1)
u (l 2)
h
i
..
Al x (0) = B | A B | ... | Al2 B | Al1 B
.
u (1)
u (0)
Part 8: Controllability and Observability
u
(l
1)
u (l 2)
..
(7)
u=
.
u (1)
u (0)
(4)
(Revised: Sep 10, 2012)
Al x (0) = P u
(8)
From Eqn (8), it is clear that if Al x (0) lies in the column space of P , then
a solution u exists which is the control sequence to drives the initial state to
the origin. Thus:
If the rank of P = n , then a control sequence u is guaranteed to exist;
and the system is controllable.
(6)
Page 7
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 8
EE/ME 701: Advanced Linear Systems
Section 2.2.1
2.2 Controllability, continuous time system
EE/ME 701: Advanced Linear Systems
2.2.2 Controllability for continuous-time systems
Controllability for a continuous time system follows from a similar analysis,
except that integrals and the Cayley-Hamilton theorem are required.
THEOREM (Controllability): A n-dimensional continuous-time LTI
system is controllable if and only if the matrix
A polynomial expression p () has the form (following Bay, chap 5):
p () = + n1
n1
+ + 1 + 0
p (A) = A + n1 A
n1
(9)
PROOF: Recall that the solution of the LTI equations is:
x (t) = eA(tt0 ) x (t0) +
+ + 1 A + 0I
Z t
eA(t) B u () d
(13)
t0
(10)
We want to establish whether a control signal u (t)exists such that
x (t1 ) = 0, so consider:
Cayley-Hamilton Theorem:
A matrix satisfies its own characteristic equation
Z t1
THEOREM (Cayley-Hamilton theorem): If the characteristic
polynomial of an arbitrary n n matrix A is denoted (), computed
as () = det (A I), then matrix A satisfies its own characteristic
equation, denoted by (A) = 0.
t0
(A) = An + n1 An1 + + 1A + 0I
n
n1
= n1 A
1 A 0I
(11)
where the i are the coefficients of the characteristic equation.
In any polynomial expression in A, we can always replace An and higher
powers with the right hand side of Eqn (11).
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
eA(t1 ) B u () d = eA(t1 t0 ) x (t0 ) , 1 Rn
(14)
We dont need to solve for the 1 for the proof, only to know it
exists.
As a result, all matrix polynomial expressions are equivalent to an
expression of order n or less, because
so
(12)
has rank n.
A matrix polynomial expression has a similar form:
n
P , B | A B | ... | Al2 B | Al1 B
2.2.1 Polynomial expressions on matrices, and the Cayley-Hamilton Thrm
Section 2.2.2
Page 9
Term eA(t1 t0 ) has a polynomial expansion, so by the CayleyHamilton theorem, it can be expressed in terms of a low-order
polynomial:
eA(t1 t0 ) =
i () Ani
(15)
i=1
We dont need to solve for the i () for the proof, only to know
they exist.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 10
EE/ME 701: Advanced Linear Systems
Section 2.2.2
EE/ME 701: Advanced Linear Systems
2.3 Controllability example
From Eqns (14) and (15):
1 =
=
Z t1
t0
Z t1
t0
"
i () A
ni
i=1
"
Ani B i ()
i=1
Z t1 h
n1
t0
= An1 B
t0
Consider the electrical circuit given in figure 3:
B u () d
#
(16)
R
u () d
i
B 1 () u () + + A B n1 () u () + B n () u () d
Z t1
Section 2.3.0
1 () u () d + + A B
Define:
i ,
Z t1
t0
Z t1
t0
n1 () u () d + B
Z t1
t0
+
vs(t)
+
vx(t)
-
(17)
Eqn (17) with Eqn (16) gives:
n1
.
..
(18)
(Revised: Sep 10, 2012)
The state equations are given as:
2
1
1
RC
d vC (t)
C vC (t) RC
vs (t)
=
+
1
dt iL (t)
L1 0
iL (t)
L
h
i vC (t)
+ [1] vs (t)
vx (t) =
1 0
iL (t)
P = [B | A B] =
Page 11
(19)
The Controllability matrix is given as:
From Eqn (14), i is a vector depending on x (t0); from Eqns (17) and (18), if
P is full rank, then there exists a control input u (t) , t [t0 , t1 ] to give some
i such that x (t1 ) = 0 . Thus, the system is guaranteed to be controllable.
Part 8: Controllability and Observability
n () u () dx
As before, we dont need to be able to compute the i , only to know they
exist.
n1
h
i
=P
1 = B | A B | ... | Al2 B | Al1 B
+ vc(t)
Figure 2: RLC circuit example.
i () u () d
iL(t)
1
RC
1
L
1
R22C2 + LC
1
R LC
(20)
When P becomes rank-deficient, the determinant will be zero. So we can
compute the determinant to test for values of R which make the system
uncontrollable.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 12
EE/ME 701: Advanced Linear Systems
Section 2.3.0
Note: The controllability matrix doesnt have to be square, (well see this
example with an extra input below). A determinant calculation can only
be used when P is square.
|P | =
So rank (P ) < 2 if
1
RC
1
L
1
R22C2 + LC
= 1 1
R2 LC2 L2 C
1
R LC
R=
It means that the transfer function has a pole-zero cancellation, and
that the zero is effectively in the input path:
1
s s + RC
Vx (s)
=
2
1
Vs (s) s2 + RC
s + LC
(23)
p
L/C then L = R2 C the denominator factors into:
s
Vx (s)
=
Vs (s) s2 + 2
iL(t)
+ vc(t) I (t)
b
+
vx(t)
-
(22)
What does this strange situation mean physically:
1
s + RC
1
1 2
RC s + RC
+
vs(t)
L
C
Section 2.3.0
Consider adding another control input (one way to deal with an
uncontrollable system)
(21)
What it means for the system to be uncontrollable: for arbitrary to , t1 and
x (t0 ) , we can not find a control input vs (t) , t [t0, t1 ] to give x (t1 ) = 0; in
other words, to drive the state to 0.
If R =
EE/ME 701: Advanced Linear Systems
1
s s + RC
1
1
s + RC
s + RC
Figure 3: RLC circuit example with second input.
The state model becomes:
2
1
1
1
RC
vC (t)
vs (t)
d vC (t)
C
RC
C
(24)
=
1
dt iL (t)
L1 0
iL (t)
Ib (t)
0
L
h
i vC (t)
+ [1] vs (t)
vx (t) =
1 0
iL (t)
The controllability matrix is:
s
1
s + RC
The second mode is not gone from the system, it is just unreachable from
the input.
P = [B | A B] =
1
RC
1
L
1
C
0 |
1
R22C2 + LC
1
R LC
R2
1
LC
(25)
Notice that P changes shape. It still has n rows, but now has 2n columns.
A 2 4 matrix has to be quite special to have rank less than 2.
In the present case, P is rank 2 as long as the two rows are independent.
The zero in the (2, 2) element assures the rows will be independent.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 13
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 14
EE/ME 701: Advanced Linear Systems
Section 2.4.0
2.4 Observability, discrete time system
EE/ME 701: Advanced Linear Systems
Section 2.4.0
At time k = 0 :
Consider example 3.11 in [Bay]. For the LIT discrete-time system:
y (0) = C x (0) + D u (0)
x (k + 1) = A x (k) + B u (k)
(27)
(26)
At time k = 1 :
y (k) = C x (k) + D u (k)
y (1) = C x (1) + D u (1)
Observability, example 3.11: Given and arbitrary initial condition x (0),
when l samples of the inputs and outputs, u (k) and y (k), k {0..l},
are known, under what conditions will it be possible to determine
(observe) the initial condition x (0).
(28)
= C [A x (0) + B u (0)] + D u (1)
= C A x (0) +C B u (0) + D u (1)
At time k = 2 :
Duality: Duality situations are those where, with a systematic set of
changes or exchanges, one system turns out to have properties equivalent
to another. For example, circuit duality:
y (2) = C x (2) + D u (2)
i
h
= C A2 x (0) + A B u (0) + B u (1) + D u (2)
= C A2 x (0) +C A B u (0) +C B u (1) + D u (2)
Voltage sources Current Sources
Capacitors
Inductors
Resistances
Conductances
(29)
Generalizing, Eqn (29) becomes:
and the new dual circuit will behave identically to the original.
In controls there are several dualities, an important one is the
Observability / Controllability duality. As we will see, with an
appropriate transpose and C substituted for B, the properties carry over.
Solution:
C
y (1) C A
2
y (2)
= CA
..
..
.
.
C Al1
y (l 1)
y (0)
x (0) +
Big Term
(based on u (k))
(30)
Consider the output for a few samples of the output:
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 15
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 16
EE/ME 701: Advanced Linear Systems
Section 2.4.0
Observability (continued)
y (0)
y (1)
y (2)
..
.
y (l 1)
Section 2.6.0
2.5 Observability, continuous time system
It doesnt matter what the big term is, as long as it is known. Write
EE/ME 701: Advanced Linear Systems
Proof Q must be full rank for a continuous time system to be observable is
omitted. It can be constructed by considering y (t) and its derivatives.
Big Term
(based on u (k))
= Q x (0)
2.6 Observability example
(31)
Using the circuit example,
C
1
0
Q =
=
2
1
CA
RC C
where the observability matrix is given as:
(33)
which is independent of L and full rank for any values of R and C.
CA
2
Q ,
CA
..
C Al1
(32)
From Eqn (31), it is clear that if x (0) lies in the row space of Q , then by
knowing the left hand side it is possible to determine x (0). It is guaranteed
that x (0) lies in the row space of Q if the rank of Q is n.
If the rank of Q = n , then x (0) can be determined from known samples
of the input and output; and the system is observable.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 17
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 18
EE/ME 701: Advanced Linear Systems
Section 2.7.2
2.7 Popov-Belevitch-Hautus Tests
EE/ME 701: Advanced Linear Systems
Section 2.7.2
2.7.2 PBH Test of Observability
These tests for controllability and observability do not involve testing any
matrix for rank, but rather examine the left eigenvectors (eigenvectors) for
orthogonality with B (C) .
LEMMA (PBH Test of Controllability): An LTI system is not
controllable if and only if there exists a eigenvector v of A such that
Cv = 0
Left Eigenvectors: Left eigenvectors are row vectors which have the property
that
vA = v
(34)
PROOF: We will see that it is a direct consequence of examining
observability in modal coordinates.
For example,
so v =
1 1
1 1
3 1
= 3 1 1
0 2
(35)
is a left eigenvector. Left eigenvectors are row vectors,
and they are the (right) eigenvectors of A .
2.7.1 PBH Test of Controllability
LEMMA (PBH Test of Controllability): An LTI system is not
controllable if and only if there exists a left eigenvector of A such
that
B = 0T
PROOF: The text establishes the PBH test by direct proof. We will see
that it is a direct consequence of examining controllability in modal
coordinates.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 19
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 20
EE/ME 701: Advanced Linear Systems
Section 2.8.0
2.8 Controllability and Observability in modal coordinates.
EE/ME 701: Advanced Linear Systems
The rotation to modal coordinates is given by
Recall that we can use the eigenvectors of A to transform the state variable
model onto a new basis (modal coordinates) in which the modes are
decoupled.
V = Evecs A ; T = V 1
b = T A T 1
A
Bb = T B
Controllability: For any decoupled mode (all modes in modal coordinates)
we must have an input into the mode, in order for the mode to be
controllable.
The system is controllable if and only if all elements of the
input matrix B are non-zero in modal coordinates.
Observability: For any decoupled mode (all modes in modal coordinates)
we must have an output from the mode, in order for the mode to be
observable.
The system is observable if and only if all elements of the
output matrix C are non-zero in modal coordinates.
Section 2.8.0
Cb = C T 1
The eigenvectors of A are given as:
>> A = [-2/(R*C), (1/C)
-(1/L)
0]
>> B = [1/(R*C) ; (1/L)]
>> C = [ -1 0]; D = 1;
>> [V, U] = eig(A)
Controllability and Observability in modal coordinates Example:
Consider the original circuit (figure 3) with L = 1 [H], C = 1 [F] and R = 2.
p
R 6= L/C , so the system is controllable. The state model is given by:
1
1
A=
1 0
0.5
B=
1.0
C=
D = [1]
1 0
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 21
V =
0.3536 - 0.6124i
0.7071
0.3536 + 0.6124i
0.7071
Notice that the eigenvalues and eigenvectors are complex. This is fine. The
complex terms will come in complex-conjugate pairs, with the imaginary
components canceling in the output.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 22
EE/ME 701: Advanced Linear Systems
Section 2.8.0
Controllability and Observability in modal coordinates Example (continued)
>> T = inv(V)
T =
0.0000 + 0.8165i
-0.0000 - 0.8165i
Controllability and Observability in modal coordinates Example (continued)
>> VV = [V(:,1), V2]
VV =
0.7071
-0.3536
0.7071
0.3536
0 + 0.0000i
-0.5000 - 0.8660i
>> Bhat = T * B
Bhat = 0.7071 - 0.0000i
0.7071 + 0.0000i
>> Chat = C * inv(T)
Chat = -0.3536 + 0.6124i
Section 2.8.0
Following the steps in section 4.4 to construct the T matrix (called the modal
matrix M by Bay), the transformation is given by:
0.7071 - 0.4082i
0.7071 + 0.4082i
>> Ahat = T * A * inv(T)
Ahat = -0.5000 + 0.8660i
-0.0000 - 0.0000i
EE/ME 701: Advanced Linear Systems
>> T = inv(VV)
T = 0.7071
-1.4142
0.7071
1.4142
-0.3536 - 0.6124i
The system is controllable and observable, B has no zero elements, neither
does C .
>> Ahat = T * A * inv(T)
Ahat = -1.0000
1.0000
0
-1.0000
When R = 1 [] .
The is more difficult because of the repeated eigenvalue (the repeated
pole at s = 1/ (RC)).
It is a coincidence that the pole-zero cancellation occurs at the same
value of R as the double pole, and that this A matrix has only 1
eigenvector.
>> Bhat = T * B
Bhat = 1.4142
0.0000
>> Chat = C * inv(T)
Chat = -0.7071
0.3536
It is necessary to use the Jordan form.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 23
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 24
EE/ME 701: Advanced Linear Systems
Section 2.8.0
Section 3.0.0
3 Controllability and observability example
With
b x (t) + Bu
b (t)
x (t) = A
Notice:
EE/ME 701: Advanced Linear Systems
(36)
y (t) = Cb x (t) + D u (t)
A double pendulum will be used to illustrate how uncontrollable or
unobservable modes can arise in a physical system.
The example shows that modes arise with choices of where or how
actuators or sensors are placed.
b
1. The repeated poles at 1 in A.
2. Bb (2) = 0 (there is no input into x (2) of the system: uncontrollable).
3. Cb has all non-zero elements (the system is observable).
A typical remedy for uncontrollable or unobservable modes is to reengineer a system to change an actuator or sensor.
The double pendulum, seen in figures 4 and 5:
Has 4 energy storage elements (kinetic energy of each mass, potential
energy of each mass);
Is a 4th order system;
Has 2 oscillatory modes.
l1
M1
l2
M2
Figure 4: Double pendulum: example system for unobservable and
uncontrollable modes.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 25
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 26
EE/ME 701: Advanced Linear Systems
Section 3.0.0
Controllability and observability example (continued)
Mode 1: Motion together, 2 = 1
(Potential energy, no spring energy)
Section 3.0.0
Controllability and Observability Example (continued)
Mode 2: Motion together, 2 = 1
(Potential energy and spring energy, faster oscillation)
Ap = 0
Figure 5: Double pendulum: two modes of the double pendulum.
-19.8000 0
10.0000
1.0000 0
0
0
0
10.0000 0
-19.8000
0
0
1.0000 0
>> OLPoles = eig(Ap)
OLPoles = 0.0000 +
0.0000 -0.0000 +
-0.0000 -
The modeling equations are:
M1 l12 1 (t) = 1 (t) = M1 g l1 1 k (1 2 ) b 1
M2 l22 2 (t) = 2 (t) = M2 g l2 2 k (2 1 ) b 2
h
iT
With state vector x (t) = 1 1 2 2 we get the state variable model:
>>
>>
>>
>>
>>
>>
>>
>>
EE/ME 701: Advanced Linear Systems
>> %% Double Pendulum, y/observability example
%% Setup parameters
M1 = 2; M2 = M1;
% Ball mass
l1 = 1; l2 = l1;
% Link length
b = 0.1;
% Damping factor
k = 20
% Spring stiffness
M1l2 = M1*l1^2
% Link M1 inertia
M2l2 = M2*l2^2
% Link M2 inertia
g = 9.8
% Gravity constant
5.4589i
5.4589i
3.1305i
3.1305i
%% Fast mode
%% Slow mode
And if we apply an input force on M1and measure 2 as illustrated in
figure 6, we get the input and output matrices:
>> Bp = [ l1/M1l2 0 0 0]
%% Input is force on M1
>> Cp = [ 0 0 0 1 ]
>> Dp = [ 0]
%% Output is 2
>> Ap = [ -b/M1l2 -k/M1l2-g/l1 0
+k/M1l2 ;
1
0
0
0 ;
0
+k/M2l2
-b/M2l2 -k/M2l2-g/l2 ;
0
0
1
0 ]
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 27
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 28
EE/ME 701: Advanced Linear Systems
Section 3.0.0
This model gives the step response of figure 7, where both modes are
present.
EE/ME 701: Advanced Linear Systems
Section 3.1.0
3.1 Testing Controllability and Observability
1. Controllability: construct the controllability matrix
2(t)
(Sensor)
>>
%% Controllability matrix
>> Ccontrol = [Bp, Ap*Bp, Ap*Ap*Bp, Ap*Ap*Ap*Bp]
Ccontrol = 0.5000
0 9.9000 0
0
0.5000 0
-9.9000
0
0
5.0000 0
0
0
0
5.0000
u(t)
(Applied Force)
M1
M2
>> rank(Ccontrol)
ans = 4
Figure 6: Double pendulum with sensor and actuator.
The system is controllable.
Step Response
2. Observability: construct the observability matrix
0.05
0.04
>>>> %% Observability matrix
>> Oobserve = [Cp; Cp*Ap; Cp*Ap*Ap; Cp*Ap*Ap*Ap]
Oobserve =
0
0
0
1.0000
0
0
1.0000
0
0
10.0000
0
-19.8000
10.0000
0
-19.8000
0
Amplitude
0.03
0.02
0.01
0.01
0.02
50
100
150
200
250
Time (sec)
>> rank(Oobserve)
ans =
4
Figure 7: Response of the first double pendulum model.
The system is observable.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 29
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 30
EE/ME 701: Advanced Linear Systems
Section 3.2.0
3.2 A badly placed sensor
u(t)
>> Oobserve1 = [Cp1; Cp1*Ap; Cp1*Ap*Ap; Cp1*Ap*Ap*Ap]
Oobserve1 = 0
0.5000 0
0.5000
0.5000 0
0.5000 0
0
-4.9000 0
-4.9000
-4.9000 0
-4.9000 0
>> rank(Oobserve1)
ans = 2
(Sensor)
k
M1
M2
Figure 8: Double pendulum with sensor moved to a new location.
The output signal is given by:
y (t) =
C p1 =
The controllability matrix:
l1 1 + l2 2
2
0 l1 /2 0 l2 /2
P = B p A p B p A2p B p A3p B p ;
The observability matrix:
>> Cp1 = [ 0 l1/2 0 l2/2 ]
Cp1 = [0
0.5000 0
0.5000 ]
y(t)
or
Section 3.2.0
A badly placed sensor (continued)
Now lets model a system with a position sensor placed in the middle of the
spring (rather than sensing position 2 ):
(Applied Force)
EE/ME 701: Advanced Linear Systems
Rank(Q ) < n , the system is unobservable.
i
unchanged
(37)
Moving the sensor does not change the controllability of a system.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 31
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 32
EE/ME 701: Advanced Linear Systems
Section 3.2.0
A badly placed sensor (continued)
EE/ME 701: Advanced Linear Systems
Section 3.2.1
3.2.1 A badly placed actuator
Lets take a look at the poles and zeros
Lets go back to the original system, and rather than applying a force to M1,
lets put in a linear motor that acts between M1 and M2
Original system:
>> Sys0 = ss(Ap, Bp, Cp, Dp);
>> Poles0 = pole(Sys0)
Poles0 = 0.0000 + 5.4589i
0.0000 - 5.4589i
-0.0000 + 3.1305i
-0.0000 - 3.1305i
>> Zeros0 = zero(Sys0)
Zeros0 = Empty matrix: 0-by-1
%% Fast mode
Linear Motor
f(t) = ks va(t)
u(t)
%% Slow mode
2(t)
(Sensor)
u(t)
(Applied Force)
(Applied Force)
k
M1
%% No zeros
M2
Figure 9: Double pendulum with actuator moved to a new location.
System with the sensor in the middle of the spring:
>> Sys1 = ss(Ap, Bp, Cp1, Dp);
>> Poles1 = pole(Sys1);
%% Poles1:
>> Zeros1 = zero(Sys1)
Zeros1 =
0 + 5.4589i
0 - 5.4589i
Unchanged.
0
B p2 =
l2 /M2 l22
The input signal is given by:
%% Zeros overlap Fast mode
The new sensor position introduces zeros which collide with the fast mode
poles.
This system is controllable but unobservable.
The observability matrix:
l1 /M1 l12
Cp
Cp A p
Q =
C p A2p
C p A3p
unchanged (38)
Moving the actuator does not change the observability of a system.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 33
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 34
EE/ME 701: Advanced Linear Systems
Section 3.2.1
A badly placed actuator (continued)
EE/ME 701: Advanced Linear Systems
Section 3.2.1
A badly placed actuator (continued)
The controllability matrix:
This system is observable by uncontrollable, take a look at the system in
modal coordinates:
>> Bp2 = [l1/M1l2; 0; -l2/M2l2; 0 ]
Bp2 = 0.5000
0
-0.5000
0
%% Rotate to modal coordinates
[V, P] = eig(Ap);
mpT = V;
mpT1 = inv(mpT)
Am = mpT1 * Ap * mpT
= diag([0+j5.5, 0-j5.5, 0+j3.1, 0-j3.1])
>> Ccontrol2 = [Bp2, Ap*Bp2, Ap*Ap*Bp2, Ap*Ap*Ap*Bp2]
Ccontrol2 = 0.5000 0
-14.9000
0
0
0.5000
0
-14.9000
-0.5000 0
14.9000
0
0
-0.5000
0
14.9000
>> rank(Ccontrol2)
ans = 2
Input matrix of system with badly placed actuator:
Bm = mpT1 * Bp2
= [3.6; 3.6; 0; 0]
%% slow mode uncontrollable.
Output matrix of system with badly placed sensor:
Cm = Cp2 * mpT
= [0, 0, 0+j0.22, 0-j0.22]
Rank(P ) < n , the system is uncontrollable.
%% fast mode unobservable
Lets take a look at the poles and zeros
System with the actuator between the links.
>> Sys2 = ss(Ap, Bp2, Cp, Dp);
>> Zeros2 = zero(Sys2)
Zeros2 =
0 + 3.1305i
%% Zeros overlap Slow mode
0 - 3.1305i
The new actuator design introduces zeros which collide with the slow mode
poles.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 35
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 36
EE/ME 701: Advanced Linear Systems
Section 3.3.0
3.3 Summary Controllability and Observability
EE/ME 701: Advanced Linear Systems
Section 3.3.1
3.3.1 Uncontrollable and unobservable modes and rocketry
The order of the system is based on the number of storage elements (or 1st
order differential equations).
If a pole and zero cancel, it does not remove an energy storage element
from the system: the mode is still in there.
Space launch vehicles are one type of system where controllability and
observability problems arise:
Flexible structures
Restricted freedom to place sensors or actuators
Controllability and observability are properties of the system, not the
controller.
The poles and zeros move around vigorously during launch, as the
aerodynamic condition changes and mass changes.
There is no way to fix a controllability or observability problem by
choosing a different controller.
Uncontrollable or unobservable modes must be very carefully considered
for rocketry
Most often controllability or observability problems are fixed by changing
(or adding) an actuator; or changing (or adding) a sensor.
Open-loop unstable mode (without control, the rocket will fall over)
Lightly damped structural modes:
Unobservable or uncontrollable modes may not be a problem if they are
sufficiently fast and well damped.
Can be excited by vibrations during launch
Moderate oscillations can (and did!) exceed structural limits.
Transfer functions provide little insight into controllability and observability. State variable models were needed to see what was going on (R.E.
Kalman [1])
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 37
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 38
EE/ME 701: Advanced Linear Systems
Section 4.1.0
4 Additional topics in Bay
EE/ME 701: Advanced Linear Systems
Section 4.1.0
Definitions related to Controllability
Bay also addresses time varying systems and alternative definitions of
Controllability and Observability
4.1 Alternative definitions of Controllability and Observability
We have several alternative definitions for the concepts of Controllability
and Observability
In most cases (except as noted), these are equivalent to basic Controllability
and Observability for Linear Time-Invariant Systems.
The differences become more important (and subtle) for nonlinear or timevarying systems.
Controllability:
A system is controllable if there exists a u (t) to drive from an arbitrary
state x0 to the origin.
Reachability:
Can find u (t)to drive from an arbitrary state x0 to any second
arbitrary state x1 .
LTI systems: Controllability are reachability are equivalent for continuous
systems. For discrete systems, there are (defective) discrete
systems which are controllable in a trivial way, consider:
xk+1 =
This is partly why it takes a Ph.D. to design control for an air or space
craft.
0 0
0 0
xk +
0
0
uk
Reachability is actually the more interesting property, requiring
Invertibility of the controllability matrix.
Stabilizability:
A system is stabilizable if its uncontrollable modes, if any, are openloop stable. Its controllable modes may be stable or unstable.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 39
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 40
EE/ME 701: Advanced Linear Systems
Section 4.1.0
Definitions related to Observability
Observability:
Can the initial state be determined.
Reconstructability:
Can the final state be determined.
Detectability:
A system is detectable if its unobservable modes, if any, are open-loop
stable. Its observable modes may be stable or unstable.
Stabilizability and Detectability
Stabilizability and Detectability relate to whether a closed-loop system can
be stabilized.
For simple stability (as opposed to good performance), we dont need to
observe or control open-loop stable modes.
If any unobservable or uncontrollable modes are fast enough and well
damped, and not excessively excited by disturbances, we may also be
able to achieve good control.
References
[1] R.E. Kalman. On the general theory of control systems. In Proc. 1st Inter.
Congress on Automatic Control, pages 481493. Moscow: IFAC, 1960.
[2] R.E. Kalman, Y.C. Ho, and K.S. Narendra. Controllability of Linear
Dynamical Systems, Vol 1. New York: John Wiley and Sons, 1962.
Part 8: Controllability and Observability
(Revised: Sep 10, 2012)
Page 41
You might also like
- (Nicholas J. Higham) Functions of Matrices Theory PDFNo ratings yet(Nicholas J. Higham) Functions of Matrices Theory PDF446 pages
- Feature Selection and Feature Extraction in Pattern Analysis: A Literature ReviewNo ratings yetFeature Selection and Feature Extraction in Pattern Analysis: A Literature Review14 pages
- Quantum Mechanics Theoretical Minimum - Notes Part 2No ratings yetQuantum Mechanics Theoretical Minimum - Notes Part 262 pages
- Equilibrium Problems and Circuit AnalysisNo ratings yetEquilibrium Problems and Circuit Analysis23 pages
- Matrices and Linear Algebra in Control ApplicationsNo ratings yetMatrices and Linear Algebra in Control Applications38 pages
- Lecture Notes On Linear Algebra: Aklal S PatiNo ratings yetLecture Notes On Linear Algebra: Aklal S Pati176 pages
- Linear Algebra Review and Reference: Zico Kolter (Updated by Chuong Do and Tengyu Ma) June 20, 2020No ratings yetLinear Algebra Review and Reference: Zico Kolter (Updated by Chuong Do and Tengyu Ma) June 20, 202029 pages
- Linear Algebra Review and Reference: Zico Kolter (Updated by Chuong Do and Tengyu Ma) April 3, 2019No ratings yetLinear Algebra Review and Reference: Zico Kolter (Updated by Chuong Do and Tengyu Ma) April 3, 201928 pages
- Sydsaether Students Manual Further Smfmea2100% (1)Sydsaether Students Manual Further Smfmea2115 pages
- Matrix Algebra: Determinants, Inverses, EigenvaluesNo ratings yetMatrix Algebra: Determinants, Inverses, Eigenvalues16 pages
- Solutions, Further Mathematics For Economic Analysis, 2008,115p - Sydsaeter & HammondNo ratings yetSolutions, Further Mathematics For Economic Analysis, 2008,115p - Sydsaeter & Hammond115 pages
- CT5123 - LectureNotesThe Finite Element Method An Introduction100% (2)CT5123 - LectureNotesThe Finite Element Method An Introduction125 pages
- Introduction To Ecological Multivariate AnalysisNo ratings yetIntroduction To Ecological Multivariate Analysis79 pages
- 06(a) Linear Algebra (Row operation) SC (2)No ratings yet06(a) Linear Algebra (Row operation) SC (2)47 pages
- Trifocal Tensor: Exploring Depth, Motion, and Structure in Computer VisionFrom EverandTrifocal Tensor: Exploring Depth, Motion, and Structure in Computer VisionNo ratings yet
- Matrices with MATLAB (Taken from "MATLAB for Beginners: A Gentle Approach")From EverandMatrices with MATLAB (Taken from "MATLAB for Beginners: A Gentle Approach")3/5 (4)
- Student Solutions Manual for Mathematics for Economics, fourth editionFrom EverandStudent Solutions Manual for Mathematics for Economics, fourth editionNo ratings yet
- Digital SAT Math Prep For Dummies, 3rd Edition: Book + 4 Practice Tests Online, Updated for the NEW Digital FormatFrom EverandDigital SAT Math Prep For Dummies, 3rd Edition: Book + 4 Practice Tests Online, Updated for the NEW Digital FormatNo ratings yet
- Laboratory Alert - 20e32 Heating Test - TransformersNo ratings yetLaboratory Alert - 20e32 Heating Test - Transformers2 pages
- Pension Papers Complete For Fresh RetirementNo ratings yetPension Papers Complete For Fresh Retirement19 pages
- Rad e Rawafiz (Refutation of Ithna Ashari Creed)No ratings yetRad e Rawafiz (Refutation of Ithna Ashari Creed)63 pages
- Download full Wavelets Theory and Its Applications A First Course Mani Mehra ebook all chapters100% (3)Download full Wavelets Theory and Its Applications A First Course Mani Mehra ebook all chapters65 pages
- Hilbert Space Methods in Signal Processing 1st Edition Rodney A. Kennedy All Chapters Instant Download100% (1)Hilbert Space Methods in Signal Processing 1st Edition Rodney A. Kennedy All Chapters Instant Download61 pages
- Complete Download Procrustes Problems Oxford Statistical Science Series John C. Gower PDF All Chapters100% (12)Complete Download Procrustes Problems Oxford Statistical Science Series John C. Gower PDF All Chapters60 pages
- Dot Product and Vector Projections (Sect. 12.3) : We Choose The First Way, The Textbook Chooses The Second WayNo ratings yetDot Product and Vector Projections (Sect. 12.3) : We Choose The First Way, The Textbook Chooses The Second Way25 pages
- Algebra For Ai Miscellaneous Code SnippetsNo ratings yetAlgebra For Ai Miscellaneous Code Snippets7 pages
- Full download Linear and Generalized Linear Mixed Models and Their Applications 2nd Edition Jiming Jiang pdf docx100% (3)Full download Linear and Generalized Linear Mixed Models and Their Applications 2nd Edition Jiming Jiang pdf docx65 pages
- Week 9 NULL SPACE KERNAL AND RANGE OF A LINEAR TRANSFORMATION.pdf (2)No ratings yetWeek 9 NULL SPACE KERNAL AND RANGE OF A LINEAR TRANSFORMATION.pdf (2)3 pages
- 04 - Deep-Learning-Based Surrogate Model For Reservoir Simulation With Time-Varying Well ControlsNo ratings yet04 - Deep-Learning-Based Surrogate Model For Reservoir Simulation With Time-Varying Well Controls20 pages
- Econometrics I: Professor William Greene Stern School of Business Department of EconomicsNo ratings yetEconometrics I: Professor William Greene Stern School of Business Department of Economics29 pages
- An Overview of Space-Time Adaptive Processing For RadarNo ratings yetAn Overview of Space-Time Adaptive Processing For Radar7 pages
- Algebraic Quantum Field Theory: Wojciech DybalskiNo ratings yetAlgebraic Quantum Field Theory: Wojciech Dybalski51 pages
- 2008 The Modern Algebra of Information RetrievalNo ratings yet2008 The Modern Algebra of Information Retrieval332 pages
- Carrell J. - Basic Concepts of Linear AlgebraNo ratings yetCarrell J. - Basic Concepts of Linear Algebra344 pages
- 6.3.2 Implicit Dynamic Analysis Using Direct Integration Products: Abaqus/Standard Abaqus/CAENo ratings yet6.3.2 Implicit Dynamic Analysis Using Direct Integration Products: Abaqus/Standard Abaqus/CAE9 pages
- Dominated Splittings For Flows With Singularities: NonlinearityNo ratings yetDominated Splittings For Flows With Singularities: Nonlinearity18 pages
- CNN-Based Projected Gradient Descent For Consistent CT Image ReconstructionNo ratings yetCNN-Based Projected Gradient Descent For Consistent CT Image Reconstruction14 pages
