APPENDIX Math Review

Do not worry about your difficulties in Mathematics. I can

assure you mine are still greater.
—Albert Einstein

The last word, when all is heard: Fear God and keep his com-
mandments, for this is man’s all; because God will bring to
judgment every work, with all its hidden qualities, whether
good or bad.
—Ecclesiastes 12:13–14

The appendix reviews mathematical techniques that are used in the book. The appendix also
serves as a glossary of symbols, notation, and concepts employed in the text. For more infor-
mation on the topics covered here, the reader should consult references supplied at the end of
the appendix.
In an attempt to keep the presentation as clear as possible, we employ the theorem-proof
style whenever appropriate. Major results of independent interest are encapsulated as theorems.
A lemma is used to denote an auxiliary result that is used as a stepping stone toward a theorem.
We use the term proposition as a way to announce a minor theorem. A corollary is a direct
consequence of a theorem or proposition. A hypothesis is a statement that is taken for further
reasoning in a proof. For more information on the subject of writing for mathematical or technical
sciences, we recommend a very clear and useful book by Higham [122].

A.1 Notation and Methods of Proof

We often use the term set. The great German mathematician G. Cantor (1845–1918), the creator
of the theory of sets, defines a set as any collection of definite, distinguishable objects of our
thought, to be conceived as a whole [193, p. 1]. In the definition the objects are the elements
or members of the set. Usually, we use capital letters to denote sets and use lowercase letters
to denote elements of the set. Thus, if X is a set, then we write x ∈ X to mean that x is an
element of X . When an object x is not an element of a set X , we write x ∈ / X . Often, we use
the curly bracket notation for sets. For example, {x : x < 7} denotes the set of all x less than 7.
624
 A.1 NOTATION AND METHODS OF PROOF 625

The colon, :, following x is read “such that.” The following sets are used in the book:
N = {1, 2, 3, . . .}, the set of positive integers,
Z = {0, 1, −1, 2, −2, . . .}, the set of integers,
R, the set of real numbers,
C, the set of complex numbers.
Sometimes, we take a fixed set and then carry out manipulations with reference to the fixed set.
We call this fixed set the universe of discourse, or just the universe, for short.
Throughout the text, we work with statements that are combinations of individual statements.
It is necessary to distinguish between sentences and the statements. A sentence is a part of a
language. Sentences are used to make different statements. In classical logic, statements are
either true or false. Consider the following two statements:
A : x > 7;
B : x 2 > 49.
We can combine the above two statements into one statement, called a conditional, that has the
form “IF A, THEN B.” Thus, a conditional is a compound statement obtained by placing the
word “IF” before the first statement and inserting the word “THEN” before the second statement.
We use the symbol “⇒” to represent the conditional “IF A, THEN B” as A ⇒ B. The statement
A ⇒ B also reads as “A implies B,” or “A only if B,” or “A is sufficient for B,” or “B is
necessary for A.” Statements A and B may be either true or false. The relationship between the
truth or falsity of A and B and the conditional A ⇒ B can be illustrated by means of a diagram
called a truth table shown in Table A.1. In the table, T stands for “true” and F stands for “false.”
It is intuitively clear that if A is true, then B must also be true for the statement A ⇒ B to be
true. If A is not true, then the sentence A ⇒ B does not have an obvious meaning in everyday
language. We can interpret A ⇒ B to mean that we cannot have A true and B not true. One can
check that the truth values of the statements A ⇒ B and “not (A and not B)” are the same. We
can also check that the truth values of A ⇒ B and “not B ⇒ not A” are the same. The above
is the basis for three methods, inference processes, of proving statements of the form A ⇒ B.
Inference is a process by which one statement is arrived at, using reasoning, on the basis of one
or more other statements accepted as a starting point. Reasoning is a special kind of thinking
in which inference takes place using the laws of thought. By a proof we mean the process of
establishing the truth of a statement by arriving at it from other statements using principles of
reasoning. Most of the time we are concerned with the statements of the form A ⇒ B and use

Table A.1 Truth Table for

Conditional

A B A⇒B

F F T
F T T
T F F
T T T
626 APPENDIX • MATH REVIEW

the following types of proofs of such statements:

1. The direct method
2. The proof by contraposition
3. The proof by contradiction
In the first case, the direct method, one starts with A and then tries to deduce B. In the proof by
contraposition, one starts with “not B” and then deduces a chain of consequences to end with
“not A.” This method of proving is based on the fact that the truth values of A ⇒ B and “not B ⇒
not A” are the same. The proof by contradiction is based on the fact that the truth values of
A ⇒ B and “not (A and not B)” are the same. In this method, one starts with “ A and not B”
and tries to arrive at a contradiction, which is a statement that is always false.
It is possible to combine two conditionals to form a biconditional, which is a statement of
the form “ A if and only if B.” Occasionally, we use the symbol “⇔” to write A ⇔ B to mean
“A if and only if B.” Another way of expressing A ⇔ B is “A is equivalent to B”, or “A is
necessary and sufficient for B.”

A.2 Vectors
We define an n-dimensional column vector to be an n-tuple of numbers:
⎡ ⎤
x1
⎢ x2 ⎥
⎢ . ⎥, (A.1)
⎣.⎦
.
xn
where n is a positive integer. Thus a vector can be viewed as an ordered set of scalars. We
use lowercase bold letters—for example, x—to denote such n-tuples. We call the numbers
x1 , x2 , . . . , xn the coordinates or components of the vector x. The number of components of
a vector is its dimension. When referring to a particular component of a vector, we use its
component—for example, xi —or the subscripted name of the vector—for example (x)i —to
denote xi . The set of all n-tuples, that is, n-vectors, with real coefficients is denoted Rn and called
the real n-space. The transpose of the vector x, denoted x T , is an n-dimensional row vector
⎡ ⎤T
x1
⎢ x2 ⎥
xT = ⎢ ⎥
⎣ .. ⎦ = [x1 x2 · · · xn ]. (A.2)
.
xn
The sum of two n-vectors x and y is an n-vector, denoted x + y, whose ith component is
xi + yi . The following rules are satisfied:
1. (x + y) + z = x + ( y + z), that is, vector addition is associative.
2. x + y = y + x, which means that vector addition is commutative.
3. Let 0 = [0 0 · · · 0]T denote the zero vector. Then,
0 + x = x + 0 = x.
4. Let x = [x1 x2 · · · xn ]T and let −x = [−x1 −x2 · · · −xn ]T . Then,
x + (−x) = 0.
 A.2 VECTORS 627

The product of an n-vector x by a scalar c is an nth vector, denoted cx, whose ith component
is cxi .
For two vectors x and y from Rn , we define their scalar product, also called the inner product,
to be
x, y = x T y = x1 y1 + x2 y2 + · · · + xn yn . (A.3)
The scalar product of vectors from Rn is commutative, that is,
x, y =  y, x,
and distributivity over vector addition holds:
x, y + z = x, y + x, z.
Two vectors x and y are said to be orthogonal if x, y = 0.
A vector norm, denoted  · , satisfies the following properties:
VN 1. x > 0 for any nonzero vector.
VN 2. For any scalar c, we have cx = |c|x.
VN 3. For any two vectors x and y, the triangle inequality holds, that is, x+ y ≤ x+ y.
The Euclidean norm, or the 2-norm, of a vector x ∈ Rn , denoted x2 , is defined as

x2 = x, x = x12 + x22 + · · · + xn2 . (A.4)
Note that x22 = x, x = x T x. The Euclidean norm is a special class of norms called the
Hölder or p-norms defined by
x p = (|x1 | p + · · · + |xn | p )1/ p , p ≥ 1. (A.5)
Other often-used norms that are special cases of the above class of norms are the 1-norm and
the infinity norm, defined respectively as
x1 = |x1 | + · · · + |xn | (A.6)
and
x∞ = max |xi |. (A.7)
i
Most of the time, we use the Euclidean norm for vectors. So for the sake of brevity, we write
x to mean x2 .

Theorem A.1 (Pythagoras) If x and y are orthogonal, then

x + y 2 = x2 +  y 2 .
Proof We have
x + y 2 = ( x + y ) T ( x + y )
= x2 + 2x, y  +  y 2
= x2 +  y 2 ,
because by assumption x, y  = 0. The proof of the theorem is complete.

We now introduce the notion of projection. Let x, y be two vectors, where y = 0. Let
z = c y, c ∈ R, be the vector such that the vector x − z is orthogonal to y—see Figure A.1 for
628 APPENDIX • MATH REVIEW

x x⫺z

z ⫽ cy
0 Figure A.1 Computing the projection of x along y.

an illustration of the above construction. Thus,

(x − z)T y = 0. (A.8)
Substituting z = c y into (A.8) yields
(x − z)T y = (x − c y)T y = 0. (A.9)
Hence,
x, y
c= . (A.10)
 y2
The number c given by (A.10) is called the component of x along y. The projection of x along
y is the vector
x, y
z = cy = y. (A.11)
 y2

Theorem A.2 (Cauchy-Schwarz Inequality) Let x, y ∈ Rn . Then,

|x, y | ≤ x y . (A.12)
Proof If y = 0, then both sides of (A.12) are equal to 0, and the assertion holds.
Suppose that y = 0, and let c denote the component of x along y. Hence, c is given
by (A.10). We then write
x = x − z + z = x − cy + cy. (A.13)
For an illustration of (A.13), consult Figure A.1. Applying the Pythagoras theorem
to (A.13), we obtain
x2 = x − cy 2 + cy 2 = x − cy 2 + c2  y 2 . (A.14)
Hence,
c2  y 2 ≤ x2 . (A.15)
On the other hand, by (A.10),
x, y 2 x, y 2
c2  y 2 =  y 2
= . (A.16)
 y 4  y 2
Combining (A.15) and (A.16) yields
x, y 2
≤ x2 . (A.17)
y 2

Multiplying both sides of (A.17) by  y 2 and taking the square root, concludes the
proof.
 A.3 MATRICES AND DETERMINANTS 629

Theorem A.3 (Triangle Inequality) Let x, y be two vectors in Rn . Then,

x + y  ≤ x +  y . (A.18)
Proof Both sides of (A.18) are nonnegative. Hence, to prove that (A.18) holds it suffices
to show that the squares of the both sides satisfy the desired inequality,
( x + y ) T ( x + y ) ≤ (x +  y ) 2 . (A.19)
Expanding ( x + y ) T ( x + y ) and applying the Schwarz inequality yields
( x + y ) T ( x + y ) = x2 + 2x, y  +  y 2
≤ |x2 + 2x y  +  y 2
= (x +  y ) 2 , (A.20)
which completes the proof of the theorem.

Theorem A.4 Let x, y be two vectors in Rn . Then,

|x −  y | ≤ x − y .
Proof We have
x = ( x − y ) + y  ≤ x − y  +  y .
Hence,
x −  y  ≤ x − y . (A.21)
The above implies that
x − y  =  y − x ≥  y  − x = −(x −  y ). (A.22)
Combining (A.21) and (A.22) yields
|x −  y | ≤ x − y ,
thus completing the proof.

A.3 Matrices and Determinants

Let m, n be two integers such that m, n ≥ 1. An array of numbers
⎡ ⎤
a11 a12 ··· a1n
⎢ a21 a22 ··· a2n ⎥
⎢ ⎥
⎢ .. .. .. .. ⎥
⎣ . . . . ⎦
am1 am2 · · · amn

is called a matrix. Thus a matrix is a set of scalars subject to two orderings. We say that it is an
m by n matrix, or m × n matrix. This matrix has m rows and n columns. If m = n, then we say
that it is a square matrix. If the elements of an m × n matrix A are real numbers, then we write
A ∈ Rm×n . If the elements of the matrix A are complex numbers, then we write A ∈ Cm×n . We
define the zero matrix, denoted O, to be the matrix such that ai j = 0 for all i, j. We define the
630 APPENDIX • MATH REVIEW

n × n identity matrix, denoted I n , as the matrix having diagonal components all equal to 1 and
all other components equal to 0.
Given an m × n matrix A. The n × m matrix B such that b ji = ai j is called the transpose of
A and is denoted AT . Thus, the transpose of a matrix is obtained by changing rows into columns
and vice versa. A matrix A that is equal to its transpose—that is, A = AT —is called symmetric.
We now define the sum of two matrices. For two matrices A and B of the same size, we
define A + B to be the matrix whose elements are ai j + bi j ; that is, we add matrices of the same
size componentwise.
Let c be a number and let A be a matrix. We define c A to be the matrix whose components
are cai j . Thus, to obtain c A, we multiply each component of A by c.
We now define the product of two matrices. Let A be an m × n matrix and let B be an n × s
matrix. Let aiT denote the ith row of A, and let b j denote the jth column of B. Then, the product
of AB is an m × s matrix C defined as
⎡ T ⎤
a1 b1 a1T b2 · · · a1T bs
⎢ T ⎥
⎢ a2 b1 a2T b2 · · · a2T bs ⎥
C = AB = ⎢ ⎢ .. .. ..
⎥
.. ⎥ . (A.23)
⎣ . . . . ⎦
am b1 am b2 · · · am bs
T T T

It is not always true that AB = B A; that is, matrix multiplication is not necessarily commutative.
Let A, B, and C be matrices such that A, B can be multiplied, A, C can be multiplied, and
B, C can be added. Then,
A(B + C) = AB + AC. (A.24)
The above property is called the distributive law. If x is a number, then
A(x B) = x( AB). (A.25)
Furthermore, matrix multiplication is associative because
( AB)C = A(BC). (A.26)
Suppose now that A, B can be multiplied. Then,
( AB)T = B T AT . (A.27)
Thus the transpose of a product is the product of transposed matrices in reverse order.
Let A be an n × n matrix. An inverse of A, if it exists, is an n × n matrix B such that
AB = B A = I n . (A.28)
If an inverse exists, then it is unique. A square matrix that is invertible is said to be nonsingular.
We denote the inverse of A by A−1 . Hence, we have
A−1 A = A A−1 = I n .
The transpose of an inverse is the inverse of the transpose,
( A−1 )T = ( AT )−1 , (A.29)
and we denote it as A−T .
We now present the Sherman–Morrison–Woodbury formula for inverting a special kind of
matrices.
 A.3 MATRICES AND DETERMINANTS 631

Lemma A.1 If A, D, and D−1 + C A−1 B are invertible, then

( A + BDC) −1 = A−1 − A−1 B( D−1 + C A−1 B) −1 C A−1 .
Proof We prove the Sherman–Morrison–Woodbury formula by verification. We have
( A + BDC)( A−1 − A−1 B( D−1 + C A−1 B) −1 C A−1 )
= ( A + BDC) A−1
− ( A + BDC) A−1 B( D−1 + C A−1 B) −1 C A−1
= I + BDC A−1
− ( B + BDC A−1 B)( D−1 + C A−1 B) −1 C A−1
= I + BDC A−1
− BD ( D−1 + C A−1 B)( D−1 + C A−1 B) −1 C A−1
= I + BDC A−1 − BDC A−1
= I,
as desired.

In the case when D = I, the Sherman–Morrison–Woodbury formula takes the form

( A + BC)−1 = A−1 − A−1 B(I + C A−1 B)−1 C A−1 . (A.30)
Formula (A.30) is sometimes referred to as the matrix inversion formula.
A set of nonzero vectors {q 1 , q 2 , . . . , q r } is said to be orthogonal if
q i , q j  = 0 for i = j.
If each vector in an orthogonal set is of unity length—that is, q i  = 1 for all i = 1, 2 . . . , r —
then this set of vectors is called orthonormal. A square matrix with orthonormal columns is
called a unitary matrix. Sometimes in the literature, a unitary matrix is called an orthogonal
matrix. If Q ∈ Rn×n is a unitary matrix, then
QT Q = Q QT = I n. (A.31)
For a square, n × n, matrix A, its trace, denoted trace A, is the sum of its diagonal elements,

n
trace A = aii . (A.32)
i=1
If B is also an n × n matrix, then
trace( AB) = trace(B A). (A.33)
The determinant of a square matrix A, denoted det A, can be evaluated using the Laplace
expansion as follows. Let ci j denote the cofactor corresponding to the element ai j of an n × n
matrix A. The cofactor ci j is (−1)i+ j times the determinant of the (n − 1) × (n − 1) submatrix
of A obtained by deleting the ith row and jth column from A. Then, for any fixed i, 1 ≤ i ≤ n,
we have

n
det A = ai j ci j . (A.34)
j=1
632 APPENDIX • MATH REVIEW

This is the expansion of the determinant along the ith row. In a similar fashion, we can expand
the determinant along a column. The determinant has the following properties:
D 1. det A = det AT .
D 2. Let A and B be square matrices of the same size, then det( AB) = det A det B =
det(B A).
D 3. det O = 0.
D 4. det I n = 1.
We now list additional properties of determinants. Let ai denote the ith column of A. Then,
we write
det A = det(a1 , a2 , . . . , an ).
D 5. Suppose that the ith column ai is written as a sum
ai = b + c.
Then, we have
det(a1 , . . . , ai , . . . , an ) = det(a1 , . . . , b + c, . . . , an )
= det(a1 , . . . , b, . . . , an ) + det(a1 , . . . , c, . . . , an ).
D 6. If x is a number, then
det(a1 , . . . , x ai , . . . , an ) = x det(a1 , . . . , ai , . . . , an ).
D 7. If two columns of the matrix are equal, then the determinant is equal to zero.
D 8. If we add a multiple of one column to another, then the value of the determinant does
not change. In other words, if x is a number, then
det(a1 , . . . , ai , . . . , a j , . . . , an ) = det(a1 , . . . , ai + x a j , . . . , a j , . . . , an ).
Properties D 5–D 8 hold also for rows instead of columns because det A = det AT .
Using the cofactors of A and its determinant, we can give a formula for A−1 . First, we define
the adjugate of A, denoted adj A, to be the matrix whose (i, j)th entry is the cofactor c ji ; that
is, adj A is the transpose of the cofactor matrix of A. Then,
 
−1 1 adj A
A = adj A = . (A.35)
det A det A
If A is n × n and v is a nonzero n × 1 vector such that for some scalar λ,
Av = λv, (A.36)
then v is an eigenvector corresponding to the eigenvalue λ. In order that λ be an eigenvalue of
A, it is necessary and sufficient for the matrix λI n − A to be singular; that is, det(λI n − A) = 0.
This leads to an nth-order polynomial equation in λ of the form
det(λI n − A) = λn + an−1 λn−1 + · · · + a1 λ + a0 = 0. (A.37)
This equation, called the characteristic polynomial equation, must have n, possibly nondis-
tinct, complex roots that are the eigenvalues of A. The spectral radius of A, denoted ρ( A), is
 A.4 QUADRATIC FORMS 633

defined as
ρ( A) = max |λi ( A)|,
1≤i≤n

where λi ( A) denotes the ith eigenvalue of A.

Any m × n matrix A can be represented as
A = U[S, O]V T if m ≤ n, (A.38)
or

S
A=U VT if m > n, (A.39)
O
where U is an m × m unitary matrix, V is an n × n unitary matrix, and S is a diagonal matrix
of the form
S = diag(σ1 , σ2 , . . . , σ p ), where p = min(m, n) and σi ≥ 0, i = 1, 2, . . . , p.
The nonnegative numbers {σi } are called the singular values of A; and (A.38), or (A.39), is
called the singular value decomposition (SVD) of A. Following the convention, we write
σ1 ≥ σ2 ≥ · · · ≥ σ p ,
so that σ1 = σ1 ( A) denotes the largest singular value of A. The singular values of A are the
square roots of the eigenvalues of A AT if m ≤ n or of AT A if m > n. If A is symmetric, its
singular values are the absolute values of its eigenvalues.

A.4 Quadratic Forms

A real quadratic form is the product x T Qx, where Q is a specified n × n matrix with real
coefficients and x is an n × 1 real, column vector. Without loss of generality, the matrix Q
can be taken to be symmetric, that is, Q = Q T . Indeed, if Q is not symmetric, we can always
replace it with a symmetric matrix without changing the quadratic form values. Indeed, because
z = x T Qx is a scalar, hence z T = z, and we have
(x T Qx)T = x T Q T x = x T Qx.
Therefore, we can write
 
1  Q + QT
x Qx = x
T T
2
Q+ Q
1
2
T
x=x T
x,
2
where 12 ( Q + Q T ) is a symmetric matrix. Thus, the quadratic form values are unchanged if
Q is replaced with the symmetric matrix 12 ( Q + Q T ). From now on, in our study of quadratic
forms, we assume that Q is symmetric.
A quadratic form x T Qx—or, equivalently, the matrix Q—is called positive semidefinite
if x T Qx ≥ 0 for all x. It is positive definite if x T Qx > 0 for all x = 0. If Q is positive
semidefinite, then we write Q ≥ 0; if it is positive definite, then we write Q > 0. We write
Q 1 ≥ Q 2 to mean that Q 1 − Q 2 ≥ 0. We say that Q is negative semidefinite if x T Qx ≤ 0
for all x, and it is negative definite if x T Qx < 0 for all x = 0. Note that Q ≤ 0 if and only if
− Q ≥ 0.
634 APPENDIX • MATH REVIEW

We now present tests for definiteness properties of symmetric matrices. We state these tests
in terms of principal minors of a given real, symmetric, n × n matrix Q with elements qi j . We
also give tests for definiteness in terms of eigenvalues of Q. The principal minors of Q are
det Q itself and the determinants of submatrices of Q obtained by removing successively an ith
row and ith column. Specifically, for p = 1, 2, . . . , n, the principal minors are
⎡ ⎤
qi1 i1 qi1 i2 · · · qi1 i p
  ⎢ q i 2 i 1 qi 2 i 2 · · · qi 2 i p ⎥
i1 , i2 , . . . , i p ⎢ ⎥
p = det⎢ . . . . ⎥, 1 ≤ i1 ≤ i2 ≤ · · · ≤ i p .
i1 , i2 , . . . , i p ⎣ . . .
. . . .
. ⎦
qi p i 1 qi p i 2 · · · qi p i p

The above are called the principal minors of order p.

In the following, we use the fact that if Q = Q T , then its eigenvalues are all real.

Theorem A.5 The following are equivalent:

1. The symmetric matrix Q is positive semidefinite.
2. Principal minors of Q are nonnegative; that is, for p = 1, 2, . . . , n,
 
i1 , i2 , . . . , i p
p ≥ 0, 1 ≤ i1 ≤ i2 ≤ · · · ≤ i p.
i1 , i2 , . . . , i p

3. Eigenvalues of Q are nonnegative, that is,

λi ( Q) ≥ 0, i = 1, 2, . . . , n.

We now give two tests for Q = Q T to be positive definite: one in terms of its leading principal
minors, and the other in terms of its eigenvalues. The leading principal minors of Q are
    
1 1, 2 q11 q12
1 = q11 , 2 = det ,
1 1, 2 q21 q22
⎡ ⎤
  q11 q12 q13  
1, 2, 3 1, 2, ..., p
3 = det⎣q21 q22 q23 ⎦ , . . . ,  p = det Q.
1, 2, 3 1, 2, ..., p
q31 q32 q33

Theorem A.6 The following are equivalent:

1. The symmetric matrix Q is positive definite.
2. Leading principal minors of Q are positive, that is,
 
1, 2, . . . , p
p > 0, p = 1, 2, . . . , n.
1, 2, . . . , p

3. Eigenvalues of Q are positive, that is,

λi ( Q) > 0, i = 1, 2, . . . , n.
 A.4 QUADRATIC FORMS 635

◆ Example A.1
We will find the range of the parameter γ for which the quadratic form

−5 − γ 6
xT x
0 −2

is negative definite. We first symmetrize the above quadratic form to get

 
−5 − γ 6 −5 − γ 3
f = xT x = xT x.
0 −2 3 −2

A quadratic form is negative definite if and only if its negative is positive definite.
So instead of working with f , we will find the range of the parameter γ for which
g = − f is positive definite, where

5+γ −3
g = xT x.
−3 2

To finish to problem, we will investigate when the matrix


5+γ −3
−3 2

is positive definite. The first-order leading principal minor of the above matrix is
1 = 5 + γ , which is positive if and only if γ > −5. The second-order leading
principal minor is

5+γ −3
2 = det = 2γ + 1,
−3 2

which positive for γ > −1/2. Thus the quadratic form f is negative definite if and
only if γ > −1/2.

If a given matrix Q ∈ Rn×n is symmetric, then its eigenvalues are real and there exists a
unitary matrix U such that
⎡ ⎤
λ1 0 · · · 0
⎢ 0 λ2 · · · 0⎥
⎢ .. ⎥
U T QU = ⎢ .. .. ⎥ = . (A.40)
⎣. . .⎦
0 0 ··· λn
Using the above, one can show that a symmetric positive
√ semidefinite matrix Q has a positive
1/2
semidefinite symmetric square root, denoted Q or Q, that satisfies

Q 1/2 Q 1/2 = Q. (A.41)

636 APPENDIX • MATH REVIEW

Indeed, it follows from Theorem A.5 that if Q = Q T ≥ 0 then its eigenvalues are all nonnegative.
Hence,
⎡ 1/2 ⎤
λ1 0 ··· 0
⎢ ⎥
⎢ 0 1/2
λ2 ··· 0 ⎥
1/2 = ⎢⎢ .. .. .. ⎥
⎥ (A.42)
⎣ . . . ⎦
0 0 ··· λ1/2
n

is well-defined. Using the fact that U T U = I n , we can define

Q 1/2 = U1/2 U T , (A.43)

which has the desired properties.

If Q = Q T is positive semidefinite and not positive definite, then some of its eigenvalues
are equal to zero. Suppose that Q has r nonzero eigenvalues. Then, there is a unitary matrix,
say U, such that
⎡ ⎤
λ1 0 · · · 0 0 · · · 0
⎢ 0 λ · · · 0 0 · · · 0⎥
⎢ 2 ⎥
⎢. . . . . ⎥
⎢ .. . . .. . . .. ⎥
⎢ ⎥
⎢ ⎥
U T QU = ⎢⎢ 0 0 · · · λ r 0 · · · 0 ⎥.
⎥ (A.44)
⎢ 0 0 · · · 0 0 · · · 0⎥
⎢ ⎥
⎢. . . ⎥
⎢. . . .. ⎥
⎣ . ⎦
0 0 ··· 0 0 ··· 0

Let V ∈ Rn×r be a matrix that consists of the first r columns of the matrix U, and let
⎡ 1/2 ⎤
λ1 0 ··· 0
⎢ ⎥
⎢ 0 1/2
λ2 ··· 0 ⎥
⎢ ⎥ T
C=⎢ . . . ⎥V . (A.45)
⎢ .. .. .. ⎥
⎣ ⎦
0 0 · · · λr
1/2

Note that the matrix C is of full rank, and

Q = C T C. (A.46)

We refer to (A.46) as a full-rank factorization of Q because C is of full rank. We can generalize

the above, using (A.38), to obtain a full-rank factorization of a nonsquare matrix. Specifically,
given an m × n matrix A of rank r , we can represent it as

A = BC,

where B is an m × r matrix and C is a r × n matrix such that

rank(B) = rank(C) = rank( A) = r.

 A.5 THE KRONECKER PRODUCT 637

A.5 The Kronecker Product

Let A be an m × n and B be a p × q matrices. The Kronecker product of A and B, denoted
A ⊗ B, is an mp × nq matrix defined as
⎡ ⎤
a11 B a12 B · · · a1n B
⎢a B a B · · · a B ⎥
⎢ 21 22 2n ⎥
A⊗ B =⎢ ⎢ . . .
. . . .. ⎥⎥. (A.47)
⎣ . . . . ⎦
am1 B am2 B · · · amn B

Thus, the matrix A ⊗ B consists of mn blocks. Using the definition of the Kronecker product,
we can verify the following two properties:

( A ⊗ B)(C ⊗ D) = AC ⊗ B D, (A.48)
( A ⊗ B)T = AT ⊗ B T . (A.49)

Let

X = [x 1 x2 ··· xm ]

be a n × m matrix, where x i , i = 1, 2, . . . , m, are the columns of X. Each x i consists of n

elements. Then, the stacking operator defined as
 T
s(X) = x 1T x 2T · · · x mT
= [x11 x21 ··· xn1 x12 x22 ··· xn2 ··· x1m x2m ··· xnm ]T

yields the column nm vector formed from the columns of X taken in order. Let now A be an
n × n, and let C and X be n × m matrices. Then, the matrix equation

AX = C (A.50)

can be written as

(I m ⊗ A)s(X) = s(C). (A.51)

If B is an m × m matrix, then the matrix equation

XB = C (A.52)

can be represented as

(B T ⊗ I n )s(X) = s(C). (A.53)

Using the above two facts, we can verify that the matrix equation

AX + X B = C, (A.54)

where A is n × n, B is m × m, and C is n × m, can be written as

(I m ⊗ A + B T ⊗ I n )s(X) = s(C). (A.55)

638 APPENDIX • MATH REVIEW

◆ Example A.2
Consider the matrix equation (A.54), where
  
0 1 −2 0 1 0
A= , B= , and C = .
0 1 −3 1 1 1

We will represent the above equation in the matrix–vector format as given by (A.55),
that is,

( I 2 ⊗ A − B T ⊗ I 2 )s( X ) = s( C).

Substituting into the above equation the given matrices yields

⎛⎡ ⎤ ⎡ ⎤⎞ ⎡ ⎤ ⎡ ⎤
0 1 0 0 −2 0 −3 0 x11 1
⎜⎢0 1 0 ⎥ ⎢
0⎥ ⎢ 0 −2 0 −3⎥⎟ ⎢x21 ⎥ ⎢1⎥
⎥⎟ ⎢ ⎥ ⎢
⎜⎢ + = ⎥.
⎝⎣0 0 0 1⎦ ⎣ 0 0 1 0⎦⎠ ⎣x12 ⎦ ⎣0⎦
0 0 0 1 0 0 0 1 x22 1

Performing simple manipulations yields

⎡ ⎤⎡ ⎤ ⎡ ⎤
−2 1 −3 0 x11 1
⎢ 0 −1 0 −3⎥ ⎢x21 ⎥ ⎢1⎥
⎥ ⎢ ⎥ ⎢
⎢ = ⎥.
⎣ 0 0 1 1⎦ ⎣x12 ⎦ ⎣0⎦
0 0 0 2 x22 1

The above system of linear equations has a unique solution because the matrix
⎡ ⎤
−2 1 −3 0
⎢ 0 −1 0 −3⎥
⎢ ⎥
⎣ 0 0 1 1⎦
0 0 0 2

is nonsingular.

Let λi , v i be the eigenvalues and eigenvectors, respectively of A, and μ j and w j the eigen-
values and eigenvectors of m × m matrix B. Then,
( A ⊗ B)(v i ⊗ w j ) = Av i ⊗ Bw j
= λi v i ⊗ μ j w j
= λi μ j (v i ⊗ w j ). (A.56)
Thus, the eigenvalues of A ⊗ B are λi μ j , and their respective eigenvectors are v i ⊗ w j for
i = 1, 2, . . . , n, j = 1, 2, . . . , m.
 A.5 THE KRONECKER PRODUCT 639

◆ Example A.3
For the given two matrices
⎡ ⎤
 4 6 −8
0 0
A= and B = ⎣0 −1 0⎦ ,
2 −2
0 2 7
we will find the eigenvalues of the matrix ( I + A ⊗ B T ), where I is the identity
matrix of appropriate dimensions. We note that
eig( I + A ⊗ B T ) = 1 + eig( A)eig( B),
because for any square matrix M, eig( I +M) = 1+eig( M) and eig( M T ) = eig( M).
By inspection,
eig( A) = {0, −2} and eig( B) = {4, −1, 7}.
Hence,
eig( I + A ⊗ B T ) = {1, 1, 1, −7, 3, −13}.

We will now use (A.56) to study the matrix equation (A.54), which we represent as
M s(X) = s(C), (A.57)
where
M = I m ⊗ A + BT ⊗ I n . (A.58)
The solution to (A.57) is unique if, and only if, the mn × mn matrix M is nonsingular. To find
the condition for this to hold, consider the matrix
(I m + ε B T ) ⊗ (I n + ε A) = I m ⊗ I n + ε M + ε 2 B T ⊗ A (A.59)
whose eigenvalues are
(1 + εμ j )(1 + ελi ) = 1 + ε(μ j + λi ) + ε 2 μ j λi (A.60)
because for a square matrix Q we have
λi (I n + ε Q) = 1 + ελi ( Q).
Comparing the terms in ε in (A.59) and (A.60), we conclude that the eigenvalues of M are
λi + μ j , i = 1, 2, . . . , n, j = 1, 2, . . . , m. Hence, M is nonsingular if and only if
λi + μ j = 0. (A.61)
The above is a necessary and sufficient condition for the solution X of the matrix equation
AX + X B = C to be unique.
Another useful identity involving the Kronecker product is
s( ABC) = (C T ⊗ A)s(B). (A.62)
For further information on the subject of the Kronecker product, we refer the reader to
Brewer [33].
640 APPENDIX • MATH REVIEW

A.6 Upper and Lower Bounds

Consider a set S of real numbers. A number M is called an upper bound of the set S if
x≤M for all x ∈ S.
A set of real numbers that has an upper bound is said to be bounded above.

The Least Upper Bound Axiom Every nonempty set S of real numbers that has an upper
bound has a least upper bound, also called supremum, and is denoted
sup{x : x ∈ S}.

Examples
 
1 2 n
1. sup , ,..., , . . . = 1;
2 3 n+1
 
1 1 1 1
2. sup − , − , − , . . . , − 3 , . . . = 0;
2 8 27 n
 √ √  √
3. sup{x : x < 3} = sup x : − 3 < x < 3 = 3.
2

Theorem A.7 If M = sup{x : x ∈ S} and ε > 0, then there is at least one number x in
S such that
M − ε < x ≤ M.
Proof The condition x ≤ M is satisfied by all numbers x in S by virtue of the least
upper bound axiom. We have to show that for any ε > 0 there is a number x ∈ S
such that
M − ε < x.
We prove the statement by contradiction. We suppose that there is no such number
in S. Then,
x ≤ M−ε for all x ∈ S
and hence M − ε would be an upper bound of S that is less than M, which is the least
upper bound. This contradicts the assumption, and thus the proof is complete.

To illustrate the above theorem, we consider the following set of real numbers:
 
1 2 3 n
S= , , ,..., ,... .
2 3 4 n+1
Note that
sup{x : x ∈ S} = 1.
Let ε = 0.1. Then, for example, the element of the set S that is equal to 99/100 satisfies
99
1−ε < ≤ 1.
100
 A.7 SEQUENCES 641

A number m is called a lower bound of S if

m≤x for all x ∈ S.

Sets that have lower bounds are said to be bounded below.

Theorem A.8 Every nonempty set of real numbers that has a lower bound has a
greatest lower bound, also called infimum, and is denoted as
inf{x : x ∈ S}.
Proof By assumption, S is nonempty and has a lower bound that we call s. Thus
s≤x for all x ∈ S.
Hence
−x ≤ −s for all x ∈ S;
that is, the set {−x : x ∈ S} has an upper bound −s. From the least upper bound
axiom, we conclude that the set {−x : x ∈ S} has a least upper bound, supremum.
We call it −s0 . Because
−x ≤ −s0 for all x ∈ S,
we have
s0 ≤ x for all x ∈ S,
and thus s0 is a lower bound for S. We now prove, by contradiction, that s0 is the
greatest lower bound, infimum, of the set S. Indeed, if there was a number x̃ satisfying
s0 < x̃ ≤ x for all x ∈ S,
then we would have
−x ≤ −x̃ < −s0 for all x ∈ S,
which contradicts the fact that −s0 is the supremum of {−x : x ∈ S}.

Using arguments similar to those we used in the proof of Theorem A.8, we can prove the
following theorem:

Theorem A.9 If m = inf{x : x ∈ S} and ε > 0, then there is at least one number x in
S such that
m ≤ x < m + ε.

A.7 Sequences
A sequence of real numbers can be viewed as a function whose domain is the set of natural
numbers 1, 2, . . . , n, . . . and whose range is contained in R. Thus, a sequence of real numbers
can be viewed as a set of ordered pairs (1, a1 ), (2, a2 ), . . . , (n, an ), . . . . We denote a sequence
642 APPENDIX • MATH REVIEW

a1

a3
g⫹␧
a5
a7
g a8
a6
g⫺␧ a4

a2
Figure A.2 An illustration of the notion
0 1 2 3 4 5 6 7 8 of the limit of a sequence.

of real numbers
a1 , a2 , . . . , an , . . .
as {an }.
A sequence {an } is increasing if a1 < a2 < · · · < an · · ·. In general, a sequence is increasing
if an < an+1 . If an ≤ an+1 , then we say that the sequence is nondecreasing. Similarly, we can
define decreasing and nonincreasing sequences. Nonincreasing or nondecreasing sequences are
called monotone sequences.
A number g is called the limit of the infinite sequence a1 , a2 , . . . , an , . . . if for any positive
ε there is a natural number k = k(ε) such that for all n > k we have
|an − g| < ε;
that is, an lies between g − ε and g + ε for all n > k. In other words, for any ε > 0 we have
|an − g| < ε for sufficiently large n’s (see Figure A.2), and we write
g = lim an ,
n→∞
or
an → g.
A sequence that has a limit is called a convergent sequence. A sequence that has no limit is
called divergent.
The notion of the sequence can be extended to sequences with elements in R p . Specifically,
a sequence in R p is a function whose domain is the set of natural numbers 1, 2, . . . , n, . . . and
whose range is contained in R p .
Suppose that a1 , a2 , . . . , an , . . . , denoted {an }, is a sequence in R p . Then, g ∈ R p is called
the limit of this sequence if for each ε > 0 there exists a natural number k = k(ε) such that for
all n > k we have
an − g < ε,
and we write
g = lim an ,
n→∞
or
an → g.
 A.7 SEQUENCES 643

Theorem A.10 If a limit of a sequence in R p exists, then it is unique.

Proof We prove this theorem by contradiction. Let us assume that a sequence {an }
has two different limits, say g 1 and g 2 . Then, we have
g 1 − g 2  > 0.
Let
ε = 12 g 1 − g 2 .
From the definition of a limit, there exist k1 = k1 (ε) and k2 = k2 (ε) such that for n > k1
we have an − g 1  < ε, and for n > k2 we have an − g 2  < ε. Let m = max(k1 , k2 ). If
n > m, then an − g 1  < ε and an − g 2  < ε. Adding an − g 1  < ε and an − g 2  < ε
yields
an − g 1  + an − g 2  < 2ε. (A.63)
It follows from the triangle inequality that
a − b ≤ a + b.
Using the above inequality, we obtain
−g 1 + g 2  = an − g 1 − an + g 2 
= ( an − g 1 ) − ( an − g 2 )
≤ an − g 1  + an − g 2 . (A.64)
Applying (A.63) to (A.64) yields
−g 1 + g 2  = g 1 − g 2  < 2ε.
However, the above contradicts the assumption that g 1 − g 2  = 2ε, which completes
the proof.

A sequence in R p is bounded if there exists a number M ≥ 0 such that an  ≤ M for all
n = 1, 2, . . . . The set of all bounded sequences is sometimes denoted as l∞ .

Theorem A.11 Every convergent sequence is bounded.

Proof Let
g = lim an .
n→∞
Choose ε = 1. Then, there exists a natural number k = k(ε) such that
an − g < 1 for all n > k.
By Theorem A.4,
a − b ≤ a − b.
Using the above inequality, we get
an  − g ≤ an − g < 1 for all n > k.
Therefore,
an  < g + 1 for all n > k.
644 APPENDIX • MATH REVIEW

Let
M = max(a1 , a2 , . . . , ak , g + 1),
then we have
M ≥ an  for all n,
which means that a convergent sequence is bounded, and the proof is complete.

Suppose we are given a sequence a1 , a2 , . . . in R p and an increasing sequence of positive

natural numbers
m1, m2, . . . , mn , . . . .
The sequence
am 1 , am 2 , . . . , am n , . . .
is called a subsequence of the sequence a1 , a2 , . . . in R p , and we have
m n ≥ n.
Indeed, this is obvious for n = 1; that is, m 1 ≥ 1 because m 1 is a positive integer. We now apply
the principle of induction. We assume that m n ≥ n for a given n. Then,
m n+1 > m n ≥ n.
Hence,
m n+1 ≥ n + 1.
Thus, indeed m n ≥ n for all n.
We can say that a subsequence is obtained from the given sequence by neglecting a number
of elements in the sequence. We denote a subsequence am 1 , am 2 , . . . , am n , . . . as
{am n }.

Theorem A.12 A subsequence of a convergent sequence in R p is convergent to the

same limit; that is, if
lim an = g,
n→∞
and if
m1 < m2 < · · · < mn < · · · ,
then
lim amn = g.
n→∞
Proof Let ε > 0 be given. Then, there exists a natural number k = k(ε) such that
an − g < ε for any n > k. From the fact that mn ≥ n, it follows that
mn > n > k,
 A.7 SEQUENCES 645

and thus
 
am − g  < ε
n

for any mn > n > k, which means that

lim amn = g.
n→∞
The proof is complete.

We now state, without a proof, an important result due to Bolzano and Weierstrass.

Theorem A.13 (Bolzano–Weierstrass) Every bounded sequence in R p contains a con-

vergent subsequence.

Theorem A.14 (Cauchy) A sequence {an } in R p is convergent if and only if for every
ε > 0 there exists an r = r (ε) such that
an − ar  < ε
holds for all n > r .
Proof (⇒) Let
lim an = g,
n→∞
and let ε > 0 be given. Then, for some l = l(ε),
an − g < 12 ε for all n > l.
Let r = l + 1, then
ar − g < 12 ε.
Adding the above inequalities, we obtain
an − g + ar − g < ε for all n > r.
On the other hand,
an − ar  ≤ an − g + ar − g.
Hence,
an − ar  < ε for all n > r.
(⇐) We now assume that for every ε > 0 there exists an r = r (ε) such that
an − ar  < ε for all n > r.
We then show that the above condition implies
lim an = g.
n→∞
First, we show that {an } is a bounded sequence. Let ε = 1. Then, an r = r (ε) exists
such that an − ar  < 1 for all n > r . Hence,
an  − ar  ≤ an − ar  < 1,
646 APPENDIX • MATH REVIEW

which means that

an  < ar  + 1.
Let
M = max{a1 , a2 , . . . , ar −1 , ar  + 1},
then we have
an  ≤ M for all n,
which implies that the sequence {an } is bounded. By the Bolzano–Weierstrass theorem
the sequence {an } contains a convergent subsequence. Let
lim amn = g, m1 < m2 < · · · .
n→∞

We will show that

lim an = g.
n→∞

From the hypothesis of the theorem it follows that for some r = r (ε/3) we have
an − ar  < 13 ε for all n > r. (A.65)
On the other hand, the assumption that the subsequence {amn } is convergent implies
that for some k = k(ε/3) we have
amn − g < 13 ε for all n > k. (A.66)
We can select k so that k > r . In such a case, (A.65) and (A.66) are satisfied for all
n > k. Because mn ≥ n > r , we have for all n > k the following:
 
 am − ar  < 1 ε,
n 3

or, equivalently,
 
 ar − am  < 1 ε. (A.67)
n 3

Adding (A.65), (A.66), and (A.67) yields

   
an − ar  +  amn − g  +  ar − amn  < ε.
Hence,
    
an − g = ( an − ar ) + ar − amn + amn − g 
   
≤ an − ar  +  am − g  +  ar − am 
n n

<ε for all n > k,

which implies that
lim an = g.
n→∞

The proof is complete.

 A.7 SEQUENCES 647

Theorem A.15 Every monotone bounded sequence in R is convergent.

Proof We will prove the theorem for nondecreasing sequences. The proof for non-
increasing sequences is analogous.
We assume that we are given a sequence {an } in R that is bounded and nonde-
creasing. Let Z be the set of numbers belonging to the sequence, and let g be the least
upper bound of Z. Hence
g ≥ an , n = 1, 2, . . . .
From the properties of the least upper bound (see Theorem A.7), it follows that for any
ε > 0 there exists a k such that
g − ε < ak .
Because the sequence {an } is nondecreasing, the inequality n > k implies that
ak ≤ an ,
and thus
g − ε < an .
We then conclude that for any ε > 0, there exists a k such that for n > k,
g − ε < an < g + ε.
Hence,
|an − g| < ε,
which means that
lim an = g.
n→∞
Thus, a monotone bounded sequence in R is convergent. The proof is complete.

Given a sequence {an } of real numbers. Then the sequence {s N } defined by

N
sN = an = a1 + a2 + · · · + a N
n=1

is called the sequence of partial sums of the series

∞

an . (A.68)
n=1

If the sequence of the partial sums has a limit—that is, s N → s as N → ∞—then we say that
the series converges to the sum s, and write
∞

s= an .
n=1

The above formula can only make sense when the series (A.68) converges.
648 APPENDIX • MATH REVIEW

∞
Theorem A.16 If the series n=1 an converges, then

an → 0 as n → ∞.

Proof Let s be the sum of the series ∞n=1 an . Then,

N
sN = an → s as N → ∞. (A.69)
n=1

We also have
s N−1 → s as N → ∞, (A.70)
and
a N = (a1 + a2 + · · · + a N ) − (a1 + a2 + · · · + a N−1 )
= s N − s N−1 .
Therefore, by (A.69) and (A.70),
a N → (s − s) = 0 as N → ∞,
and the proof is complete.

The converse of Theorem A.16 is false. The terms of the series

∞
1
n=1
n

tend to zero, but the series diverges.

A.8 Functions
We discuss functions of one and several variables and their basic properties. A function f from
a set A to a set B, denoted

f : A → B,

is a rule that assigns to each x ∈ A a unique element y ∈ B. The set A is the domain of f and
the range of f is a subset of B, not necessarily the whole of B.
Let f : S → R, where S is a subset of R. Then, f is said to be continuous at z 0 ∈ S if for
every ε > 0, there exists δ = δ(ε, z 0 ) such that if z ∈ S, |z − z 0 | < δ, then | f (z) − f (z 0 )| < ε.
This is also written as f (z) → f (z 0 ) as z → z 0 . If f is continuous at every point of S, then we
say that f is continuous on S.
We say that the function f is bounded on a set S if there is M ≥ 0 such that for any z ∈ S
we have

| f (z)| ≤ M.
 A.8 FUNCTIONS 649

Lemma A.2 If f is continuous on [a, b ] ⊂ R, then f is bounded on [a, b ].

Proof Consider the set
{x : x ∈ [a, b ], and f is bounded on [a, x]}.
This set is nonempty and bounded above by b. Let
c = sup{x : f is bounded on [a, x]}.
We now show that c = b. Suppose that c < b. From the continuity of f at c, it follows
that f is bounded on [c − δ, c + δ] for some sufficiently small δ > 0. Being bounded
on [a, c − δ] and on [c − δ, c + δ], it is bounded on [a, c + δ]. This contradicts our
choice of c. We therefore conclude that c = b. This means that f is bounded on [a, x]
for all x < b. From the continuity of f , we know that f is bounded on some interval
[b − δ, b ]. Hence, f is bounded on [a, b − δ] and on [b − δ, b ], which means that f
is bounded on [a, b ].

With the help of the above lemma, we can now prove the maximum–minimum theorem of
Weierstrass.

Theorem A.17 (The Maximum–Minimum Theorem) If f is continuous on an interval

[a, b ], then it takes on both its supremum and infimum values on [a, b ].
Proof By Lemma A.2, the function f is bounded on [a, b]. Let
M = sup{ f ( x) : x ∈ [a, b ]}.
We now show that there is a c ∈ [a, b ] such that f (c) = M. Let
1
g( x) = .
M − f ( x)
If f does not take on the value of M, then g is continuous on [a, b ], and by Lemma A.2
the function g is bounded on [a, b ]. But g cannot be bounded on [a, b ]. Thus, the
assumption that f does not take on the value M has led us to a contradiction. The
other case that f takes on its infimum value can be proved in a similar manner.

A sphere, ball, around x in R p is the set of the form { y : x − y < ε} for some ε > 0. A set
in R p is said to be open if around every point x in there is a sphere that is contained in .
If x ∈ R p , then any set that contains an open set containing x is called a neighborhood of x.
A sphere around x is a neighborhood of x.
A point x ∈ R p is called a boundary point of a set ⊆ R p if every neighborhood of x
contains a point of and a point that is not in . The set of all boundary points of is called
the boundary of . A set is said to be closed if it contains its boundary. Equivalently, a set
is closed if x k → g with x k ∈ implies g ∈ . A set is bounded if it is contained within
some sphere of finite radius. A set is compact if it is both closed and bounded. Any finite set
= {x 1 , x 2 , . . . , x k } in R p is compact.
We now generalize Lemma A.2 for real-valued functions of several variables.
650 APPENDIX • MATH REVIEW

Lemma A.3 Let f : R p → R be a continuous function on a compact set ⊂ R p .

Then, f is bounded on .
Proof We prove the lemma by contradiction. We assume that f is continuous on a
compact set , but f is not bounded on . Without loss of generality, suppose that f
is unbounded above on . Since f is unbounded above on , we can find a sequence
{x n } ⊂ such that f ( x n ) > r (n), where r (n) → ∞ as n → ∞, that is, f ( x n ) → ∞
as n → ∞. The sequence {x n } must be bounded because, by assumption, the set
is compact. Therefore, by the Bolzano-Weierstrass theorem, on page 645, there is a
convergent subsequence, say {x nk }, that converges to a point x ∗ ∈ . The function is
continuous on . Hence,

lim f ( x nk ) = f lim x nk = f ( x ∗ ). (A.71)
k→∞ k→∞

We arrived at a contradiction because, by assumption, the left-hand side of (A.71) is

divergent, while f ( x ∗ ) < ∞ because f is continuous on . Therefore, a continuous
function on a compact set in R p must be bounded.

We next prove the maximum–minimum theorem of Weierstrass for real-valued functions of

several variables.

Theorem A.18 (Weierstrass) If f : R p → R is a continuous function on a compact set

⊂ R p , then f achieves its supremum and infimum.
Proof The proof follows that of Theorem A.17, with the interval [a, b ] being replaced
with the compact set and by referring to Lemma A.3 rather than to Lemma A.2.

Definition A.1 (Uniform Continuity) Let f : X → Y. Then, f is called uniformly con-

tinuous on X if for every ε > 0 there exists δ = δ(ε) > 0, depending only on ε, such that
|x − x̃| < δ implies | f (x) − f (x̃)| < ε, where x, x̃ are in X .
Every uniformly continuous function is continuous, but not necessarily conversely. Indeed,
f (x) = 1/x is continuous on (0, 1) but not uniformly continuous.
Another type of continuous functions that we use in the book are absolutely continuous
functions.
Definition A.2 (Absolutely Continuous Function) Let f be a function defined on the
closed interval [a, b]. Then, the function f is said to be absolutely continuous on [a, b]
if for every ε > 0, there exists a δ > 0 such that

n
(bk − ak ) < δ (A.72)
k=1

for all a1 , b1 , . . . , an , bn such that a1 < b1 ≤ a2 < b2 ≤ · · · ≤ an < bn , we have

n
| f (bk ) − f (ak )| < ε. (A.73)
k=1
 A.9 LINEAR OPERATORS 651

Note that every absolutely continuous function is continuous, because the case n = 1 is not
excluded in the above definition. A function f that satisfies the Lipschitz condition,
| f (x) − f (y)| ≤ L|x − y|,
is absolutely continuous.

A.9 Linear Operators

A function
F : Rn → Rm
is said to be linear if it satisfies the following two properties:
LO 1. For any x, y in Rn , we have
F(x + y) = F(x) + F( y).
LO 2. If c is a number, then
F(c x) = c F(x).
Linear functions are sometimes referred to as linear maps or linear operators.
Linear functions from Rn into Rm can be identified with the m × n real matrices as follows:
F : Rn → Rm is linear if and only if F(x) = Ax,
where A ∈ Rm×n .
If F : Rn → Rm is linear, then there exists a constant L ≥ 0 such that
F(x) ≤ Lx (A.74)
for all x ∈ Rn . The smallest such constant L for which this is true is denoted F and called
the norm of the linear map F. If F is the linear map induced by the matrix A, then we define
 A = F. For any matrix A and vector norm  · , we can compute the matrix norm using
the formula
 Ax
 A = sup . (A.75)
x=0 x

The matrix norm—that is, the norm of the linear map F given by (A.75)—is equivalent to

 A = max  Ax (A.76)

x=1

A matrix norm produced using (A.75) or (A.76) is called the induced norm. Note that
 Ax − A y =  A(x − y) ≤  Ax − y;
that is, the linear map, F, represented by the matrix A is uniformly continuous.
We now comment on the notation used in (A.75) and (A.76). First, the same symbol is used
for the matrix norm as for the norm of a vector. Second, the norms appearing on the right-hand
side of (A.75) and (A.76) are the norms of the vectors x ∈ Rn and Ax ∈ Rm . Thus, if Rn is
equipped with the vector norm ·s while Rm is equipped with the vector norm ·t , then (A.76)
652 APPENDIX • MATH REVIEW

should be interpreted as

 A = max  Axt .
xs =1

Vector norms and a matrix norm are said to be compatible, or consistent, if they satisfy

 Axt ≤  Axs . (A.77)

We will show that (A.76) is well-defined. For this, we need the following lemma.

Lemma A.4 A vector norm  ·  : R p → R is an uniformly continuous function.

Proof By Theorem A.4 on page 629, we have
|x −  y | ≤ x − y .
Hence, for any ε > 0, if we take δ = ε, then for any x, y ∈ R p , if x − y  < δ, then
|x − y | < ε, as was to be shown.

By the continuity of the vector norm and the theorem of Weierstrass on page 650, a vector
x 0 can be found such that x 0  = 1 and  A =  Ax 0  = maxx=1  Ax. Thus, the expres-
sion (A.76) is well-defined.
Next, we will show that (A.76) satisfies (A.77). Suppose that we are given a vector y = 0.
Then,
1
x= y
 y
satisfies the condition x = 1. Hence,

 A y =  A( yx) =  y Ax.

Applying (A.76) to the right-hand side of the above yields

 A y ≤  y A.

Hence, any matrix norm produced by (A.76) is consistent.

We will now prove that a matrix norm obtained using (A.76) satisfies the following conditions:

MN 1.  A > 0 if A = O and O = 0.

MN 2. c A = |c| A, where c is a constant.
MN 3.  A + B ≤  A + B.
MN 4.  AB ≤  AB.

Proof of MN 1. Let A = O. Then a vector x, x = 1, can be found such that

Ax = 0, and thus  Ax = 0. Hence,  A = maxx=1  Ax = 0. If, on the other
hand, A = O, then  A = maxx=1 Ox = 0.
 A.9 LINEAR OPERATORS 653

Proof of MN 2. By definition, cA = maxx=1 cAx. Using a property of the

vector norm, we can write cAx = |c| Ax, and hence
cA = max |c| Ax = |c| max  Ax = |c| A.
x=1 x=1

Proof of MN 3. For the sum of two matrices, A+ B, there exists a vector x 0 , x 0  = 1,

such that
 A + B = ( A + B) x 0 .
Then, using properties of the vector norm, we obtain
 A + B =  Ax 0 + B x 0  ≤  Ax 0  + B x 0 .
Using now (A.76) yields
 A + B ≤  Ax 0  + Bx 0  =  A + B.

Proof of MN 4. For the product of two matrices AB, there exists a vector x 0 , x 0  = 1,
such that  AB =  AB x 0 . Then, using (A.76), we have
 AB =  A( B x 0 ) ≤  AB x 0 .
Employing (A.76) again yields
 AB ≤  ABx 0  =  AB.

The most frequently used matrix norms are the p-norms,

 A p = max  Ax p .
x p =1

The induced matrix norms corresponding to the one-, two-, and infinity-norm for vectors are:
•  A1 = max j a j 1 , that is,  A1 is the maximum of absolute column sum;
•  A2 = σ1 ( A), which is the largest singular value of A;
•  A∞ = maxi aiT 1 , which is the maximum of absolute row sum.
A matrix norm that is not induced by a vector norm is the Frobenius norm, defined for an m × n
matrix A as
! m n "1/2

 A F = ai2j . (A.78)
i=1 j=1

The Frobenius norm satisfies the relation

 A2F = trace( A AT ) = trace( AT A). (A.79)
The Frobenius norm is compatible with the Euclidean vector norm, that is,
 Ax2 ≤  A F x2 . (A.80)
654 APPENDIX • MATH REVIEW

A.10 Vector Spaces

A given set of vectors of the same dimension {x 1 , x 2 , . . . , x r } is linearly dependent if the zero
vector, 0, can be written as a nontrivial linear combination of the given vectors:
α1 x 1 + α2 x 2 + · · · + αr x r = 0, with αi = 0 for at least one i.
In other words, the set of vectors {x 1 , x 2 , . . . , x r } is linearly independent if
α1 x 1 + α2 x 2 + · · · + αr x r = 0, implies α1 = α2 = · · · = αr = 0.
A set S containing vectors of dimension p is a subspace of R p if for any scalars α and β,
x, y ∈ S implies αx + β y ∈ S. (A.81)
It follows from the above that a subspace must contain the zero vector, 0. The set containing
only the zero vector is a subspace. This set, {0}, is called a trivial subspace.
A matrix–vector product
⎡ ⎤
x1
⎢ x2 ⎥
⎢ ⎥
Ax = [a1 a2 · · · an ] ⎢ . ⎥ = x1 a1 + x2 a2 + · · · + xn an (A.82)
⎣ .. ⎦
xn
is a linear combination of the columns of the matrix. Thus, linear dependence of the columns
of A is equivalent to the condition
Az = 0 for some vector z = 0. (A.83)
Linear independence of the columns of A is equivalent to the condition
Az = 0 implies z = 0. (A.84)
The set of all vectors that are linear combinations of the columns of A is called the range, or
image, of A and is denoted range( A). Thus,
range( A) = {v : v = Ax for some x}. (A.85)
For a given m × n matrix A and an m-vector b, we have b ∈ range( A) if and only if there
exists an n-vector x such that Ax = b. We say that a system of linear equations represented by
Ax = b is compatible if b ∈ range( A).
The range of A is a subspace of Rm . Indeed, for any b1 , b2 ∈ range( A), we have b1 = Ax 1
and b2 = Ax 2 for some x 1 and x 2 . Using the fact that A represents a linear map, for any scalars
α1 and α2 , we obtain
α1 b1 + α2 b2 = α1 Ax 1 + α2 Ax 2 = A(α1 x 1 + α2 x 2 ). (A.86)
Hence, if b1 , b2 ∈ range( A), then α1 b1 + α2 b2 ∈ range( A), which means that the range of A is
a subspace of Rm .
Similarly, we define the range of AT to be the set
range( AT ) = { y : y = AT z for some z}. (A.87)
For a nontrivial subspace S there is a unique smallest positive integer called the dimension
of the subspace S and denoted dim S. The dimension of the subspace is equal to the smallest
number of linearly independent vectors of the subspace such that every vector of the subspace
 A.10 VECTOR SPACES 655

can be expressed as a linear combination of these vectors. Any such set of vectors is called a
basis of the subspace. The vectors forming a basis are not unique.
The number of linearly independent columns of a matrix A is called the rank of A and is
denoted rank( A). The rank of A is also equal to the number of linearly independent rows of A.
For the product of an m ×n matrix A and an n × p matrix B, we have Sylvester’s inequalities:
rank( A) + rank(B) − n ≤ rank( AB) ≤ min{rank( A), rank(B)}. (A.88)
For an m × n matrix A, the set of all n-vectors orthogonal to the rows of A is called the
kernel, or null space, of A and is denoted ker( A), that is,
ker( A) = {x : Ax = 0}. (A.89)
The kernel of A is a subspace of Rn . Indeed, if x 1 , x 2 ∈ ker( A), then Ax 1 = 0 and Ax 2 = 0.
Hence, for any scalars α1 and α2 , we have
A(α1 x 1 + α2 x 2 ) = α1 Ax 1 + α2 Ax 2 = 0. (A.90)
Thus, if x 1 , x 2 ∈ ker( A), then α1 x 1 + α2 x 2 ∈ ker( A), which means that ker( A) is a subspace
of Rn . Similarly, we define the kernel of AT to be the set
ker( AT ) = {w : AT w = 0}. (A.91)
The intersection of range( A) and ker( AT ) contains only the zero vector, that is,
range( A) ∩ ker( AT ) = {0}. (A.92)
For a given subspace S of Rn , the orthogonal complement S ⊥ of S is defined as
S ⊥ = {x ∈ Rn : x T y = 0 for all y ∈ S}. (A.93)
The vectors in ker( AT ) are orthogonal to the vectors in range( A), and vice versa. We say
that range( A) and ker( AT ) are orthogonal complements of one another and write
range( A)⊥ = ker( AT ) (A.94)
and
ker( AT )⊥ = range( A). (A.95)
Similarly,
range( AT )⊥ = ker( A) (A.96)
and
ker( A)⊥ = range( AT ). (A.97)
If A is an m × n matrix with linearly independent columns, which implies that m ≥ n, then
rank( A) = n = dim range( A)
and
dim ker( AT ) = m − n.
We now define a normed space.
656 APPENDIX • MATH REVIEW

Definition A.3 A vector space with a norm is called a normed space.

By the normedvector space Rn , we usually mean the Euclidean space, which is Rn with the
norm x2 = x12 + · · · + xn2 . The vector space Rn equipped with any of the p-norm forms a
normed vector space. The absolute value is a norm in R.
Let be a compact subset of Rn and let C( ) denote the set of continuous functions f :
R → R on . Thus, we view functions as vectors in the function space C( ). Then, the function
n

 ·  : C( ) → R defined as
 f  = max | f (x)| (A.98)
x∈
satisfies the norm axioms. The function space C( ), with scalars from R and with the norm (A.98),
is a linear normed space. In particular, the space of all real-valued functions, f (x), continuous
on some interval [a, b] of the real line with the norm
 f  = max | f (x)| (A.99)
x∈[a,b]

is an example of a linear normed space. This space is denoted C([a, b]).

We are also interested in sequences of elements of C([a, b]). Let f 1 , f 2 , . . . be a sequence
of elements of C([a, b]). Denote this sequence as { f k }. In the following, f k (x), refers to the
value of the function f k at x, while f k denotes the function itself. We say that the sequence
{ f k } converges (pointwise) on the interval [a, b] if there exists a function f such that for every
x ∈ [a, b] the sequence { f k (x)} converges to f (x). In other words, the sequence { f k } converges
(pointwise) if there exists a function f such that for a given x ∈ [a, b] and a given ε > 0, we
can find N = N (x, ε) such that
| f (x) − f k (x)| < ε for k > N . (A.100)
The sequence { f k } converges uniformly, on the interval [a, b], if there is a function f such that for
any ε > 0 there exists N = N (ε), depending on ε but not on x, such that (A.100) holds for every
x ∈ [a, b]. The distinction between convergence and uniform convergence is that in the later
case the same N can be used for any value of x to show the convergence of the sequence { f k (x)}.
One can show that the limit of an uniformly convergent sequence of continuous functions is
continuous. It can be verified that the norm (A.99) defines the uniform convergence because
{ f k } converges uniformly to f if and only if maxx∈[a,b] | f (x) − f k (x)| → 0. This is the reason
the norm (A.99) is sometimes called the norm of uniform convergence. It can be shown that
there is no norm on C([a, b]) that defines pointwise convergence to an element in C([a, b])—see
Debnath and Mikusiński [58, p. 12].
Definition A.4 (Banach Space) A normed linear space is complete if every Cauchy se-
quence with elements form the space converges to an element in the space. A complete
normed linear space is called a Banach space.
A sequence that satisfies conditions of Theorem A.14 is called a Cauchy sequence. The Euclidean
space is an example of a Banach space, as is C([a, b]) with the norm (A.99).

A.11 Least Squares

The theory of least squares was developed by Carl Friedrich Gauss (1777–1855) and published
by him in 1809. Gauss first applied the least squares to devise a method of calculating orbits
of celestial bodies. For more information on the history of the least squares development, we
recommend an account by Campbell and Meyer [40, Section 2.5].
 A.11 LEAST SQUARES 657

0 Ax
range (A)

Figure A.3 Geometrical interpretation of the least

squares solution to a system of linear equations.

Consider the system of equations

Ax = b, (A.101)
where A ∈ Rm×n , x ∈ Rn , and b ∈ Rm . A necessary and sufficient condition for the sys-
tem (A.101) to have a solution is
 . 
rank( A) = rank A .. b . (A.102)
In particular, if the matrix A is invertible, then there is a unique solution to (A.101) given by
x ∗ = A−1 b.
.
Here, we analyze the case when m > n and rank( A) < rank[ A .. b], that is, b ∈
/ range( A),
which means that the system (A.101) does not have an exact solution. We modify (A.101) by
incorporating an error vector e to obtain
Ax + e = b. (A.103)
This is illustrated in Figure A.3. Because
e = b − Ax, (A.104)
we have
e2 = eT e
= (b − Ax)T (b − Ax)
= x T AT Ax − 2x T AT b + bT b. (A.105)
The gradient of e2 with respect to x is
∇ x e2 = 2 AT Ax − 2 AT b. (A.106)
Solving
∇ x e2 = 0
yields candidate minimizer points of (A.105). If the matrix A has full column rank, then the
matrix AT A is invertible and the only critical point is

x ∗ = ( AT A)−1 AT b (A.107)
658 APPENDIX • MATH REVIEW

The above point satisfies the second-order sufficient condition to be the minimizer of (A.105)
because the Hessian of (A.105) is
2 AT A,
which is positive definite. The solution given by (A.107) is called the least squares solution of
the inconsistent linear system of equations, Ax = b, because it minimizes the sum of squares
m
e = e e = i=1
2 T
ei2 . For a graphical interpretation of the least squares solution, we refer to
Figure A.3.
Suppose now that we interpret the ith row of the matrix A and the ith component of the
vector b as the ith data pair. Thus, the ith data pair has the form (aiT ; bi ). Suppose that the k
data pairs are represented by the pair ( A; b). We know that the least squares solution to the
system of equations represented by the data pair ( A; b) is given by (A.107). If the new data pair,
(ak+1
T
; bk+1 ), becomes available, then we could find the least squares solution to the system of
equations
 
A b
T x= (A.108)
ak+1 bk+1

by applying the formula (A.107) to (A.108). Let x k+1 denote the least squares solution to the
above problem. Then,
! T "−1 T 
A A A b
x k+1 = T T T . (A.109)
ak+1 ak+1 ak+1 bk+1

In the case of large k the above method of recomputing the new least squares solution from
scratch may be time-consuming. We present a recursive method for computing least squares
solutions as the data pairs become available. To proceed, let
P k = ( AT A)−1 . (A.110)
In view of the above, we can write
T 
A A
P −1
k+1 = T T
ak+1 ak+1

  A
= AT ak+1 T
ak+1
= AT A + ak+1 ak+1
T

= P −1
k + a k+1 a k+1 .
T
(A.111)
Hence,
 −1
P k+1 = P −1
k + a k+1 a k+1
T
.
Applying to the above the matrix inversion formula (A.30), we obtain

T
P k ak+1 ak+1 Pk
P k+1 = P k − . (A.112)
1 + ak+1 P k ak+1
T
 A.12 CONTRACTION MAPS 659

Using (A.110), we write

x k = ( AT A)−1 AT b
= P k AT b. (A.113)
Employing (A.111), we obtain
! T "−1 T 
A A A b
x k+1 = T T T
ak+1 ak+1 ak+1 bk+1
= P k+1 ( AT b + ak+1 bk+1 ). (A.114)
Computing AT b from (A.113), we get
AT b = P −1
k xk . (A.115)
Substituting the above into (A.114) yields
x k+1 = P k+1 ( AT b + ak+1 bk+1 )
 
= P k+1 P −1
k x k + a k+1 bk+1 . (A.116)
Computing P −1
k from (A.111) gives

P −1 −1
k = P k+1 − a k+1 a k+1
T

Substituting the above into (A.116), we have

  
x k+1 = P k+1 P −1
k+1 − a k+1 a k+1 x k + a k+1 bk+1 .
T
(A.117)
Performing simple manipulations yields
 
x k+1 = x k + bk+1 − ak+1
T
x k P k+1 ak+1 . (A.118)

The two equations (A.112) and (A.118) constitute the recursive least squares (RLS) algorithm.
To initialize the RLS, we can set
 −1
P 0 = AnT An and x 0 = P n AnT bn ,
where the pair ( An ; bn ) represents the first n data pairs.

A.12 Contraction Maps

We now discuss the contraction maps that found interesting applications in control and stability
analyses of nonlinear systems. In particular, in 1964, Leibovic [180] presented an interesting idea
of how to apply contraction maps in nonlinear and adaptive control. The proposed application
required a significant computational power not available at that time. Leibovic’s idea looks like
a good challenge for today’s computers. More recently, Kelly [152] used the contraction map
theorem as well as Lyapunov functions to investigate stability properties of a class of neural
networks.
An operator F that maps Euclidean space R p into itself is called a contraction map if for any
two elements x and y of R p ,
F(x) − F( y) ≤ Lx − y, 0 < L < 1. (A.119)
660 APPENDIX • MATH REVIEW

A point x ∗ ∈ R p is called a fixed point of F if

F(x ∗ ) = x ∗ . (A.120)

The importance of the contraction operators lies in the fact that there is a unique fixed point.
Furthermore, this point may be found by successive iterations as stated in the following theorem.

Theorem A.19 Let F : R p → R p be a contraction map with the contraction constant

0 < L < 1. Then F has a unique fixed point, and the sequence
x (1) , x (2) , . . .
generated by x (n+1) = F ( x (n) ), n = 0, 1, 2, . . . , converges to x ∗ for any initial choice
of x (0) ∈ R p .
Proof To show the uniqueness of the fixed point x ∗ , suppose that there exist two
distinct fixed points x ∗ and y∗ . Then,
x ∗ − y∗  = F ( x ∗ ) − F ( y∗ )
≤ L x ∗ − y∗ .
Thus,
x ∗ − y∗  ≤ L x ∗ − y∗ , 0 < L < 1,
which is impossible unless x ∗ − y∗  = 0. Hence, x ∗ = y∗ which means that F has
at most one fixed point.
We will now show that a fixed point exists and that the sequence
x (1) , x (2) , . . .
generated by x (n+1) = F ( x (n) ), n = 0, 1, 2, . . . , converges to x ∗ for any initial choice
of x (0) ∈ R p . If F ( x (0) ) = x (0) , then the theorem is proved. In general, F ( x (0) ) = x (0) .
For any n, m with m > n, we have
 (m)      
 x − x (n)  =  F m x (0) − F n x (0) 
      
=  F F m−1 x (0) − F F n−1 x (0) 
    
≤ L  F m−1 x (0) − F n−1 x (0) 
..
.
 
≤ L n  x (m−n) − x (0) 
 
= L n  x (0) − x (m−n) . (A.121)
We write
 (0)  
 x − x (m−n)  =  x (0) − x (1) + x (1) − x (2) + x (2)

− · · · − x (m−n−1) + x (m−n−1) − x (m−n) . (A.122)
Substituting (A.122) into (A.121) and then applying the triangle inequality yields
 (m)       
 x − x (n)  ≤ L n  x (0) − x (1)  +  x (1) − x (2)  + · · · +  x (m−n−1) − x (m−n)  .
(A.123)
 A.12 CONTRACTION MAPS 661

Observe that
 ( j)      
 x − x ( j+1)  =  F x ( j−1) − F x ( j) 
 
≤ L  x ( j−1) − x ( j) 
..
.
 
≤ L j  x (0) − x (1) , (A.124)
We use the above to further evaluate the right-hand side of (A.123) to obtain
 (m)   
 x − x (n)  ≤ L n  x (0) − x (1) (1 + L + · · · + L m−n−1 )
 
≤ L n  x (0) − x (1) (1 + L + · · · + L m−n−1 + L m−n + · · ·)
Ln  
 x (0) − x (1) 
=
1−L
because 0 < L < 1. Recall that a sequence x (0) , x (1) , . . . is a Cauchy sequence if for
each ε > 0 there exists a positive integer n = n(ε) such that x (m) − x (n)  < ε for all
m > n. Given an ε > 0, let n be such that
ε(1 − L )
Ln <  
 x (0) − x (1)  .
Then,
 (m)  Ln  
 x − x (n)  ≤  x (0) − x (1)  < ε for all m > n. (A.125)
1−L
Therefore, the sequence x (0) , x (1) , . . . is a Cauchy sequence, and we will now show
that the limit g of this sequence is the fixed point x ∗ of the map F . Note that F
is uniformly continuous. Indeed, for each ε > 0 there exists a δ = δ(ε) such that if
x − y < δ, then F ( x) − F ( y) < ε. Since F ( x) − F ( y) ≤ L x − y, given an
ε > 0, we can choose δ = ε/L . Because F is continuous, we have
  
lim F x (n) = F lim x (n) = F ( g).
n→∞ n→∞

However, x (n+1) = F ( x (n) ). Therefore,

g = lim x (n+1)
n→∞
 
= lim F x (n)
n→∞

= F lim x (n)
n→∞
= F ( g).
Hence, g = x ∗ is a fixed point of F , and the proof is complete.

Using the above proof we can give an estimate of the norm of x (n) − x ∗ . Indeed, if we fix
n in (A.125) and allow m → ∞, then
 (n)  Ln   
x − x∗ ≤  x (0) − F x (0) . (A.126)
1−L
662 APPENDIX • MATH REVIEW

A.13 First-Order Differential Equation

Let R denote the set of real numbers and let D ⊆ R2 be a domain, that is, an open connected
nonempty subset of R2 . Let f be a real-valued function defined and continuous on D. Let
ẋ = d x/dt denote the derivative of x with respect to t. We call
ẋ = f (t, x) (A.127)
an ordinary differential equation of the first order. By a solution of the differential equa-
tion (A.127) on an open interval I = {t ∈ R : t1 < t < t2 }, we mean a real-valued continuously
differentiable function φ defined on I such that (t, φ(t)) ∈ D for all t ∈ I and
φ̇(t) = f (t, φ(t)) for all t ∈ I.
Definition A.5 Given (t0 , x0 ) ∈ D, the initial value problem for (A.127) is
ẋ = f (t, x), x(t0 ) = x0 . (A.128)
A function φ is a solution to (A.128) if φ is a solution of the differential equation (A.127) on
some interval I containing t0 and x(t0 ) = x0 .
We can represent the initial value problem (A.128) equivalently by an integral equation,
# t
φ(t) = x0 + f (s, φ(s)) ds. (A.129)
t0

To prove this equivalence, let φ be a solution of the initial value problem (A.128). Then x(t0 ) = x0
and
φ̇(t) = f (t, φ(t)) for all t ∈ I.
Integrating the above from t0 to t, we obtain
# t # t
φ̇(s)ds = f (s, φ(s)) ds.
t0 t0

Hence,
# t
φ(t) − x0 = f (s, φ(s)) ds.
t0

Therefore, φ is a solution of the integral equation (A.129).

Conversely, let φ be a solution of the integral equation (A.129). Then φ(t0 ) = x0 , and differ-
entiating both sides of (A.129) with respect to t gives φ̇(t) = f (t, φ(t)). Therefore, φ is also a
solution of the initial value problem (A.128).
The general ordinary differential linear equation of the first order has the form
ẋ(t) = a(t)x(t) + b(t), t ∈ (t1 , t2 ), (A.130)
where a(t) and b(t) are assumed to be continuous on the interval (t1 , t2 ). We associate with
equation (A.130) the initial condition
x(t0 ) = x0 , x0 ∈ R.
To find the solution to (A.130), we write it in the form
ẋ(t) − a(t)x(t) = b(t). (A.131)
 A.14 INTEGRAL AND DIFFERENTIAL INEQUALITIES 663

We then multiply (A.131) by the “integrating factor”

$t
− a(s) ds
e t0 (A.132)
to get
$t $t $t
− a(s) ds − a(s) ds − a(s) ds
ẋ(t)e t0
− a(t)x(t)e t0
=e t0
b(t). (A.133)
The above equation can be represented in an equivalent form as
 $t  $t
d − a(s) ds − a(s) ds
x(t)e t0 = e t0 b(t). (A.134)
dt

Integrating both sides of (A.134) yields

$t # t $τ
− a(s) ds − a(s) ds
x(t)e t0
− x0 = e t0 b(τ ) dτ. (A.135)
t0

Hence,
$t # t $τ 
a(s) ds − a(s) ds
x(t) = e t0
x0 + e t0
b(τ ) dτ , (A.136)
t0

or, in an equivalent form,

$t # t $t
a(s) ds a(s) ds
x(t) = e t0
x0 + e τ b(τ ) dτ (A.137)
t0

A.14 Integral and Differential Inequalities

A.14.1 The Bellman-Gronwall Lemma

Lemma A.5 Let v(t) and w(t) be continuous real functions of t. Let w(t) ≥ 0, and c
be a real constant. If
# t
v(t) ≤ c + w(s)v(s) ds, (A.138)
0
then
$t
w(s) ds
v(t) ≤ ce 0 . (A.139)
Proof Denote the right-hand side of (A.138) by x(t); that is, let
# t
x(t) = c + w(s)v(s) ds (A.140)
0
and let
z(t) = x(t) − v(t). (A.141)
664 APPENDIX • MATH REVIEW

Note that z(t) ≥ 0, and x(t) is differentiable. Differentiating both sides of (A.140) and
using (A.141) yields
ẋ(t) = w(t)v(t)
= w(t) x(t) − w(t) z(t). (A.142)
We use (A.137) to represent (A.142) in the equivalent integral form as
$t # t $t
w(s) ds w(s) ds
x(t) = e 0 x0 − e τ w(τ ) z(τ ) dτ. (A.143)
0
Because z(t) ≥ 0 and w(t) ≥ 0, we have
$t
w(s) ds
x(t) ≤ e 0 x0 . (A.144)
From the definition of x(t) given by (A.140), we have x(0) = c. On the other hand,
v(t) = x(t) − z(t) ≤ x(t), (A.145)
because z(t) ≥ 0. Hence,
$t
w(s) ds
v(t) ≤ ce 0 ,
and the proof is completed.

The Bellman–Gronwall lemma allows us to convert the investigation of integral equations

into integral inequalities.

A.14.2 A Comparison Theorem

We consider the differential inequality

ẋ(t) ≤ ω(t, x(t)), x(t0 ) = x0 ,

where ω : R2 → R is a continuous function in a domain  ⊂ R2 . We assume that in , the

function ω satisfies the Lipschitz condition,

|ω(t, x) − ω(t, y)| ≤ L|x − y|,

where L ≥ 0 is a Lipschitz constant.

Consider now an associated differential equation

x̃˙ (t) = ω(t, x̃(t)), x̃(t0 ) = x̃ 0 .

We refer to the above equation as the equation of comparison. Denote the solution to the equation
of comparison by

x̃(t) = x̃(t; t0 , x̃ 0 ).

The following theorem, which can be found, for example, in Corduneanu [55, p. 49], relates the
solutions of the differential inequality and the associated differential equation.
 A.15 SOLVING THE STATE EQUATIONS 665

Theorem A.20 If x(t) is a differentiable solution to the differential inequality and x̃(t)
is the solution to the associated differential equation with the corresponding initial
conditions such that
x0 ≤ x̃ 0 ,
then
x(t) ≤ x̃(t)
for all t ≥ t0 .
Proof We prove the theorem by contradiction. We suppose that x0 ≤ x̃ 0 but that
x(t) ≤ x̃(t) does not hold. Thus, there exists t1 > t0 such that
x(t1 ) > x̃(t1 ).
Let
y(t) = x(t) − x̃(t).
Note that y(t0 ) = x(t0 ) − x̃(t0 ) ≤ 0 and that y(t1 ) = x(t1 ) − x̃(t1 ) > 0. Denote by
τ the largest t ∈ [t0 , t1 ] such that y(τ ) = 0. Such a value of t exists by virtue of the
continuity of y. Thus,
y(t) > 0 for all t ∈ (τ, t1 ].
On the other hand, for t ∈ [τ, t1 ],
˙
ẏ(t) = ẋ(t) − x̃(t)
≤ ω(t, x(t)) − ω(t, x̃(t))
≤ |ω(t, x(t)) − ω(t, x̃(t))|
≤ L |x(t) − x̃(t)|.
In the interval [τ, t1 ],
y(t) = x(t) − x̃(t) ≥ 0.
Hence,
ẏ(t) ≤ L |x(t) − x̃(t)|
= L ( x(t) − x̃(t))
= L y(t).
Therefore, in the interval [τ, t1 ], the function y satisfies
y(t) ≤ y(τ )eL (t−τ ) = 0
because y(τ ) = 0. But this is a contradiction to the previous conclusion that y(t) > 0
for all t ∈ (τ, t1 ], which completes the proof.

A.15 Solving the State Equations

A.15.1 Solution of Uncontrolled System
Given a dynamical system’s time-invariant model of the form

ẋ(t) = Ax(t) (A.146)

666 APPENDIX • MATH REVIEW

subject to an initial condition

x(0) = x 0 , (A.147)
where A ∈ Rn×n . The solution to (A.146) subject to (A.147) is

x(t) = e At x 0

The above is a generalization of the situation when A is a scalar. The exponential matrix e At is
defined as
t2 t3
e At = I n + t A + A2 + A3 + · · · .
2! 3!
Observe that
d At
e = Ae At = e At A.
dt
Suppose now that the initial condition x(0) = x 0 is replaced with a more general condition
x(t0 ) = x 0 ,
where t0 is a given initial time instant. Then, the solution to ẋ(t) = Ax(t) subject to x(t0 ) = x 0 is

x(t) = e A(t−t0 ) x 0

The matrix e A(t−t0 ) is often written as

e A(t−t0 ) = (t, t0 )
and is called the state transition matrix because it relates the state at any instant of time t0 to the
state at any other time t. There are many methods for calculating the state transition matrix. For
an overview of some of them, we refer the reader to the classic paper on the subject by Moler and
Van Loan [208]. In the case when the matrix A is of low dimensions, we can use the formula that
results from applying the Laplace transform to the time-invariant system ẋ = Ax, x(0) = x 0 .
The Laplace transform of the above matrix differential equation is
s X(s) − x 0 = AX(s).
Hence
X(s) = (s I n − A)−1 x 0 ,
and the inverse transform yields

e At = L−1 {(s I n − A)−1 } (A.148)

where L−1 denotes the inverse Laplace transform operator.

A.15.2 Solution of Controlled System

Consider a model of a controlled dynamical system,
ẋ(t) = Ax(t) + Bu(t) (A.149)
 A.16 CURVES AND SURFACES 667

subject to a given initial condition x(0) = x 0 , where B is an n × m matrix, and m < n as is

usually in practice. Premultiplying both sides of (A.149) by the integrating factor, e− At , yields
e− At ẋ(t) = e− At Ax(t) + e− At Bu(t).
We then rearrange the above equation as
e− At ẋ(t) − e− At Ax(t) = e− At Bu(t). (A.150)
Noting that
d − At
(e x(t)) = − Ae− At x(t) + e− At ẋ(t),
dt
we can represent (A.150) in the form
d − At
(e x(t)) = e− At Bu(t).
dt
Integrating both sides of the resulting equation gives
# t
− At
e x(t) − x(0) = e− Aτ Bu(τ ) dτ.
0
Hence,
# t
x(t) = e x(0) +
At
e A(t−τ ) Bu(τ ) dτ.
0
If the initial condition is x(t0 ) = x 0 , then integration from t0 to t yields
# t
A(t−t0 )
x(t) = e x(t0 ) + e A(t−τ ) Bu(τ ) dτ . (A.151)
t0

As in the case of the uncontrolled system, we can also use the Laplace transform technique
to obtain the solution to (A.149) subject to the initial condition x(0) = x 0 . Indeed, taking the
Laplace transform of (A.149) and performing some manipulations yields
X(s) = (s In − A)−1 x(0) + (s In − A)−1 BU(s).
Hence,
x(t) = L−1 {(s In − A)−1 x(0)} + L−1 {(s In − A)−1 BU(s)}.
If x 0 = x(t0 ), then
x(t) = L−1 {e−t0 s (s In − A)−1 x(t0 )} + L−1 {e−t0 s (s In − A)−1 BU(s)}.

A.16 Curves and Surfaces

Consider a point moving along a curve in n-dimensional space. The position of the point at time
t can be described as
⎡ ⎤
x1 (t)
⎢ x2 (t)⎥
⎢ ⎥
x(t) = ⎢ .. ⎥ . (A.152)
⎣ . ⎦
xn (t)
668 APPENDIX • MATH REVIEW

1 cos t
sin t

t 1

Figure A.4 An example of a curve in two-dimensional

space.

For example, if the point is moving along a straight line passing through fixed points y and z in
Rn , then
x(t) = y + t z. (A.153)
As another example, consider
 
x1 (t) cos t
x(t) = = ∈ R2 . (A.154)
x2 (t) sin t
Then, the point moves around a circle of radius 1 in counterclockwise direction as shown in
Figure A.4. We used t as the variable corresponding to the angle corresponding to the position
of the point on the circle. We are now ready for a formal definition of a curve.
Definition A.6 Let I ∈ R be an interval. A parameterized curve defined on the interval I
is an association which to each point of I associates a vector, where x(t) denotes the vector
associated to t ∈ I by x. We write the association t → x(t) as
x : I → Rn .
We also call this association the parameterization of a curve. We call x(t) the position vector
at time t.
The position vector can be represented in terms of its coordinates as in (A.152), where each
component xi (t) is a function of time t. We say that this curve is differentiable if each function
xi (t) is a differentiable function of time t. We define the derivative d x(t)/dt to be
⎡ ⎤
d x1 (t)
⎢ dt ⎥
⎢ ⎥
⎢ d x (t) ⎥
d x(t) ⎢ ⎥
2
⎢ ⎥
ẋ(t) = = ⎢ dt ⎥
⎢ ⎥
⎢ ... ⎥
dt
⎢ ⎥
⎣ d x (t) ⎦
n
dt
and call it the velocity vector of the curve at time t. The velocity vector is located at the origin.
However, when we translate it to the point x(t), as in Figure A.5, then we visualize it as tangent
to the curve x(t).
 A.16 CURVES AND SURFACES 669

x(t) .
x(t) ⫹ x(t)

.
Figure A.5 Velocity vector of a curve at time t is parallel
x(t) to a tangent vector at time t.

We define the tangent line to a curve x at time t to be the line passing through x(t) in the
direction of ẋ(t), provided that ẋ(t) = 0. If ẋ(t) = 0, we do not define a tangent line. The speed
of the curve x(t), denoted v(t), is v(t) =  ẋ(t). To illustrate the above, we consider the point
moving on the circle. The point position at time t is
x(t) = [cos t sin t]T .
The velocity vector of the curve at time t is
ẋ(t) = [−sin t cos t]T ,
and the speed of the curve x(t) at time t is

v(t) =  ẋ(t) = (− sin t)2 + cos2 t = 1.
The set of points in R3 such that for some F : R3 → R we have
F(x) = F(x1 , x2 , x3 ) = 0
and

∂F ∂F ∂F
D F(x1 , x2 , x3 ) = ∇ F(x1 , x2 , x3 ) = T
 0T
=
∂ x1 ∂ x2 ∂ x3
is called a surface in R3 , where ∇ F = grad F. For example, let
F(x1 , x2 , x3 ) = x12 + x22 + x32 − 1;
then, the surface is the sphere of radius 1, centered at the origin of R3 .
Now let
C(t) = [x1 (t) x2 (t) ··· xn (t)]T
be a differentiable curve. We say that the curve lies on the surface described by
F1 (x1 , x2 , . . . , xn ) = 0,
..
.
Fm (x1 , x2 , . . . , xn ) = 0
if, for all t, we have
F1 (x1 (t), x2 (t), . . . , xn (t)) = 0,
..
.
Fm (x1 (t), x2 (t), . . . , xn (t)) = 0.
670 APPENDIX • MATH REVIEW

F1 ⫽ 0 grad F2

x0
F2 ⫽ 0
.
C(t0)
grad F1

Figure A.6 Tangent space to a surface is the set of

vectors orthogonal to the gradients of the functions
defining the surface.

Differentiating the above equations, using the chain rule, we obtain

D Fi (C(t)) Ċ(t) = ∇ Fi (C(t))T Ċ(t) = 0, i = 1, 2, . . . , m.
Let x 0 be a point of the surface, and let C(t) be a curve on the surface passing through x 0 ; that
is, for some t0 ,
C(t0 ) = x 0 .
For this t0 we have
∇ Fi (x 0 )T Ċ(t0 ) = 0, i = 1, 2, . . . , m,
where we assumed that Ċ(t0 ) = 0. Thus, the gradients ∇ Fi at x 0 are orthogonal to the tangent
vector of the curve at the point x 0 . This is true for every differentiable curve on the surface
passing through x 0 . For an illustration of the above properties in three-dimensional space, we
refer to Figure A.6.
Definition A.7 The tangent space to the surface {x : F(x) = 0} at the point x 0 is the set
of vectors at x 0 that are tangent to differentiable curves on the surface passing through x 0 .

A.17 Vector Fields and Curve Integrals

Definition A.8 Let U be an open subset of Rn . By a vector field on U we mean an association
F : U → Rn ,
which to every point of U associates a vector of the same dimension.
We can interpret a vector field as a field of forces. We can visualize a vector field as a field of
arrows, which to each point associates an arrow as shown in Figure A.7. If f : U → R is a
function, then
F=∇f
is a vector field, and we have
F(x) = ∇ f (x)
 A.17 VECTOR FIELDS AND CURVE INTEGRALS 671

Figure A.7 A vector field.

C(t)

U Figure A.8 A connected open set in which there

are points that cannot be joined by a straight line.

for all x ∈ U. We now consider an inverse problem: When is a given vector field in Rn a
gradient?
Definition A.9 Let F be a vector field on an open set U. If f is a differentiable function on
U such that
F = ∇ f,
then we say that f is a potential function for F.
We then can ask the question, Do the potential functions exist? To answer the question, we recast
the problem into the one that involves solving n first-order partial differential equations:
∂f
= Fi (x), i = 1, 2, . . . , n,
∂ xi
with the initial condition
F(x 0 ) = F 0 .
To answer the above question we need one more definition.
Definition A.10 An open set U is said to be connected if, given two points in U, there is a
differentiable curve in U that joins the two points.
If U = Rn , then any two points can be joined by a straight line. Of course, it is not always the
case that two points of an open set can be joined by a straight line, as illustrated in Figure A.8.
We say that the function F = [F1 F2 · · · Fn ]T is a C 1 function if all partial derivatives of all
functions Fi exist and are continuous.
672 APPENDIX • MATH REVIEW

Theorem A.21 Let F ( x) be C 1 function defined on an open connected region U of

Rn . Then, there is a solution to the differential equation
∇ f ( x) = F ( x), F ( x0) = F 0,
if and only if
∂ Fi ∂Fj
= ;
∂xj ∂ xi
that is, the Jacobian matrix of F is symmetric.
Proof We prove the theorem for n = 2, where the connected region U is a rectangle.
We select a point x 0 = [x10 x20 ]T in the rectangle. We assume that F = [F 1 F 2 ]T
and that
∂ F1 ∂ F2
D2 F 1 = D1 F 2; that is, = .
∂ x2 ∂ x1
Our goal is to find a potential function f ( x1 , x2 ). We assume that the potential function
has the form
# x1
f ( x1 , x2 ) = F 1 (s, x2 ) ds + u( x2 ),
x10

where u( x2 ) is a function of x2 only and will be determined later in the proof. We

illustrate the integration path in Figure A.9. Applying the fundamental theorem of
calculus, we obtain
# x1
∂f ∂ ∂u( x2 )
= F 1 (s, x2 ) ds +
∂ x1 ∂ x1 x10 ∂ x1
= F 1 ( x1 , x2 )
because
∂u( x2 )
= 0.
∂ x1
Hence, D 1 f ( x1 , x2 ) = F 1 ( x1 , x2 ) as required.

x2

Rectangle

[[
x1
x2

[ [
x10
x20

[ [
x1
x20

Figure A.9 Integration path used in the

x1 proof of Theorem A.21.
 A.17 VECTOR FIELDS AND CURVE INTEGRALS 673

We now differentiate f ( x1 , x2 ) with respect to x2 to get

# x1
∂f ∂ F 1 (s, x2 ) ∂u( x2 )
= ds +
∂ x2 x10 ∂ x2 ∂ x2
# x1
∂ F 2 (s, x2 ) ∂u( x2 )
= ds +
x10 ∂ x1 ∂ x2

because, by assumption, D 2 F = D 1 F . Therefore,

∂f ∂u( x2 )
= F 2 (s, x2 )|xx110 +
∂ x2 ∂ x2
∂u( x2 )
= F 2 ( x1 , x2 ) − F 2 ( x10 , x2 ) + .
∂ x2

Let
∂u( x2 )
−F 2 ( x10 , x2 ) + = 0,
∂ x2
∂u( x2 )
that is, ∂ x2
= F 2 ( x10 , x2 ). Then, D 2 f = F 2 as required. To conclude the proof, we let
# x2
u( x2 ) = F 2 ( x10 , x2 ) dx2 ,
x20

which yields
# x1 # x2
f ( x1 , x2 ) = F 1 (s, x2 ) ds + F 2 ( x10 , t) dt (A.155)
x10 x20

The proof is complete for n = 2.

We note that if the connected region U is an arbitrary open set, then the above proof may not
work. This is because we needed to integrate over intervals that were contained in the region U.
Suppose we are given a vector field F on an open set U in the plane. We can interpret F as a
field of forces. We wish to calculate the work done when moving a particle from a point C(t1 )
to a point C(t2 ) along the curve C(t) in U , as illustrated in Figure A.10. Let

F(x) = [F1 (x1 , x2 ) F2 (x1 , x2 )]T .

We assume that each component of F is differentiable; that is, F is a differentiable vector field.
We next assume that the curve C(t) is defined on a closed interval [t1 , t2 ] with t1 < t2 . For each
t ∈ [t1 , t2 ] the value C(t) is a point in R2 . The curve C lies in U if C(t) is a point in U for
all t ∈ [t1 , t2 ]. We further assume that the curve C is continuously differentiable and that its
derivative Ċ(t) = d C(t)/dt exists and is continuous; that is, we assume that C ∈ C 1 .
674 APPENDIX • MATH REVIEW

C(t2)
.
C (t)

C(t)

Figure A.10 A particle moving along a curve in

C(t1)
vector field.

We define the integral of F along C to be

# # t2
dC
F= F(C(t))T dt
C t1 dt
# t2
= F(C(t))T d C,
t1
which is the work done against the force field F along the curve C. Let
C(t) = [x1 (t) x2 (t)]T .
Then, we can write
# #
F= F1 d x1 + F2 d x2 . (A.156)
C C

Theorem A.22 (Green) Let the vector field F be given on a region A that is the interior
of a closed path C parameterized counterclockwise. Then,
# ##  
∂ F2 ∂ F1
F 1 dx1 + F 2 dx2 = − dx1 dx2 .
C A ∂ x1 ∂ x2

A.18 Matrix Calculus Formulas

Below, we list useful formulas from matrix calculus that involve computing the gradient (with
respect to x). Recall that in our convention the gradient is a column vector. We have
∇(x T y) = ∇( y T x) = y, (A.157)

∇(x T x) = 2x, (A.158)

∇(x T A y) = A y, (A.159)

∇( y T Ax) = AT y, (A.160)

∇(x T Ax) = ( A + AT )x. (A.161)

 EXERCISES 675

Let f : Rn → Rn and g : Rn → Rn be two maps, and let ∂ f /∂ x and ∂ g/∂ x be the Jacobian
matrices of f and g, respectively. Then,
 T  T
∂f ∂g
∇( f (x) g(x)) = ∇(g(x) f (x)) =
T T
g+ f. (A.162)
∂x ∂x

For an n × n symmetric matrix Q, we have

 T
∂f
∇( f (x) Q f (x)) = 2
T
Q f. (A.163)
∂x

Notes
For more on the subject of proof techniques, we refer to Velleman [288]. For further reading on
linear algebra, and in particular matrix theory, we recommend classical texts of Gel’fand [95]
and Gantmacher [93]. Analyses of numerical methods for matrix computations can be found in
Faddeeva [78], Householder [127], Golub and Van Loan [104], and Gill, Murray, and Wright [97].
Lang [175] and Seeley [255] are recommended for a review of calculus of several variables. A
straightforward approach to mathematical analysis is presented in Binmore [28]. Principles of
real analysis can be found in Natanson [215], Bartle [22], and Rudin [244]. Some facts from
basic functional analysis that we touched upon in this appendix are discussed, for example, by
Maddox [193] and by Debnath and Mikusiński [58]. Lucid expositions of differential equations
are in Driver [69], Miller and Michel [205], and Boyce and DiPrima [32].
G. H. Hardy (1877–1947) argues in reference 114 (p. 113) that in truly great theorems
and their proofs, “there is a very high degree of unexpectedness, combined with inevitability
and economy. The arguments take so odd and surprising a form; the weapons used seem so
childishly simple when compared with the far-reaching results; but there is no escape from
the conclusions.” He further states that “A mathematical proof should resemble a simple and
clear-cut constellation, not a scattered cluster in the Milky Way.”

EXERCISES
A.1 The open unit disc with respect to the norm  ·  p is the set

 p = {x ∈ R2 : x p < 1}.

The unit discs are quite different for different choices of the norm  ·  p . Sketch 1 ,
2 , 3 , and ∞ .
A.2 Given a linear combination of matrices,

M = α1 M 1 + · · · + αr M r , (A.164)
j
where for i = 1, . . . , r , αi ∈ R, and M i ∈ Rn×n . Let mi be the jth row of the matrix
676 EXERCISES

M i . Thus,
⎡ ⎤
mi1
⎢ 2⎥
⎢ mi ⎥
Mi = ⎢ ⎥
⎢ .. ⎥ .
⎣ . ⎦
min
Show that
⎡ ⎤
m1s1

r
r
⎢ . ⎥
det M = ··· αs1 · · · αsn det ⎢ . ⎥
⎣ . ⎦. (A.165)
s1 =1 sn =1
mnsn
A.3 Let X ∈ Rn×m and Y ∈ Rm×n . Show that
det(I n + XY ) = det(I m + Y X).
A.4 Let λi ( A) denote the ith eigenvalue of a matrix A ∈ Rn×n and let α be a scalar. Show
that
(a) λi (α A) = αλi ( A);
(b) λi (I n ± α A) = 1 ± αλi ( A).
A.5 Show that the formulas (A.75) and (A.76) for matrix norms are equivalent.
A.6 Show that for any square matrix A ∈ Rn×n , we have
 A ≥ max |λi ( A)|. (A.166)
1≤i≤n

A.7 Show that if P ∈ Rn×n is symmetric, then

λmin ( P)x22 ≤ x T P x ≤ λmax ( P)x22 .
A.8 Show that a sufficient condition for
lim Ak → O
k→∞

is
 A < 1.
A.9 Let A ∈ Rn×n . Show that a necessary and sufficient condition for
lim Ak → O.
k→∞

is
|λi ( A)| < 1, i = 1, 2, . . . , n.
A.10 Let eig(M) denote the set of eigenvalues of the square matrix M and let  be the open
unit disk in the complex plane. Prove that if eig(M) ⊂  ∪ {1}, then limk→∞ M k exists
if and only if the dimension of the eigenspace corresponding to the eigenvalue 1 is
equal to the multiplicity of the eigenvalue 1; that is, the geometric multiplicity of the
eigenvalue 1 is equal to its algebraic multiplicity.
 EXERCISES 677

A.11
(a) Show that the series

∞
1
n=1
n
diverges to +∞.
(b) Show that if α is a rational number such that α > 1, then the series

∞
1
n=1
nα
converges.
(c) Use MATLAB’s Symbolic Math Toolbox to compute

∞
1
2
.
n=1
n
A.12
(a) Let 0 < x < 1. Show that
ln(1 − x) ≤ −x.
(b) Let 0 < x ≤ 1/2. Show that
ln(1 − x) ≥ −2x.

A.13 Let 0 < ak < 1 for k ≥ 0. Use a proof by contraposition, Theorem A.16, and Exer-
cise A.12 to show that
%∞
∞
(1 − ak ) = 0 if and only if ak = ∞.
k=0 k=0

A.14 Prove Sylvester’s inequalities (A.88).

A.15 Compute e At for
(a)
⎡ ⎤
0 −3 0
⎣
A= 3 0 0⎦ ,
0 0 −1
(b)
⎡ ⎤
0 1 0
⎣
A= 0 0 1⎦ .
0 −2 −3
A.16 Find x(t) for the dynamical system model,
  
0 1 0 0
ẋ(t) = x(t) + u(t), x(0) = ,
0 0 1 1
where u(t) is the unit step function; that is, u(t) = 1 for t ≥ 0 and u(t) = 0 for t < 0.
678 EXERCISES

A.17 For

−2 −6
A= ,
2 −10
find constants M > 0 and α > 0 such that
e At 2 ≤ Me−αt , t ≥ 0.
A.18 Show that the function space C([a, b]) of real-valued functions, f : R → R, with the
norm defined by (A.99) is a normed linear space. Show that this space is a Banach
space.
A.19 Derive a sufficient condition for the convergence of the iteration process
x (k+1) = Ax (k) + b, k = 0, 1, . . . ,
for any initial x(0), where A ∈ Rn×n and b ∈ Rn .