Outline of the week

1 Similarity of matrices (lec 12)

2 Eigenbases and Diagonalization (lec 12)
3 An application of diagonalization (lec 13)
4 Linear independence of eigenvectors (lec 13)
5 Trace, determimants and other principal minors (lec 14)
6 Coefficients of characteristic polynomials (lec 14)
7 Examples (lec 14)

February 2, 2017 1 / 37
Similarity of matrices

Definition 1 (Similarity)
Let A and B be two n × n matrices. We say A is similar to B if there is an
invertible matrix P such that B = P −1 AP.

Theorem 2
If A and B are similar, then both have the same characteristic polynomials.
Thus they have the same eigenvalues with the same multiplicities.

Proof:

DB (t) = |B − tIn |
= |P −1 AP − tI|
= |P −1 (A − tIn )P| (∵ P −1 IP = I)
|P|−1 (∵ P −1  = |P|−1 )
 
=  
|P|
DA (t) = DA (t).

February 2, 2017 2 / 37
Diagonalizability and diagonalization
Diagonalization problem: Let A be a square matrix. Find an invertible P
such that P −1 AP = D is a diagonal matrix.
Equivalently, we can ask whether or not A is similar to some diagonal
matrix.
Observations: We make some observations to help us attack the
diagonalization problem.
1 First observe is that if D = diag{λ1 , λ2 , ..., λn } then λ1 , λ2 , ..., λn
must be the eigenvalues of A.
(Characterstic polynomials are same.)
2 Next observe that if we write P = [v1 v2 ... vn ] as a row of column
vectors, then {v1 , v2 , ..., vn } must be a basis of Rn .
(Column rank is n.)
3 And finally we make a deeper observation that
v1 , v2 , ..., vn are eigenvectors of A
with eigenvalues λ1 , λ2 , ..., λn respectively.
February 2, 2017 3 / 37
Computations

• Since A, is similar to D, their eigenvalues are same.

• Next,

P −1 AP = D =⇒ AP = PD
=⇒ A[v1 v2 ... vn ] = [v1 v2 ... vn ]diag{λ1 , λ2 , ..., λn }
=⇒ [Av1 Av2 ... Avn ] = [λ1 v1 λ2 v2 ... λn vn ]
=⇒ Avj = λj vj (1 ≤ j ≤ n),

as claimed.
The following definition is now natural.

Definition 3 (Eigenbasis)
For a square n × n matrix A, a basis B of Rn is called an A-eigenbasis if
each basis vector is an eigenvector of A.

February 2, 2017 4 / 37
Example

Example 4
 
1 2 −2
Let A = 2 1 −4. Find the eigenvalues and a basis of eigenvectors
1 −1 −2
which diagonalizes A. Also write down a matrix X such that X −1 AX is
diagonal.

Solution:
DA (λ) = −λ(λ2 − 9) =⇒ λ = 0, ±3. 
4 2 −2
λ = −3: Row reduction of A + 3I = 2 4 −4 yields
1 −1 1
   
1 0 0 0
0 1 −1. Hence v1 = 1 is an eigenvector.
0 0 0 1

February 2, 2017 5 / 37
Example contd.
 
1 2 −2
λ = 0: Row operations on A produces a matrix  0 1 0 . Hence
0 0 0
 
2
v2 = 0 is an eigenvector.
1
   
1 −1 1 1
λ = 3: A − 3I gives rise to  0 0 1  . Hence v 3 = 1 is an
0 0 0 0
eigenvector.  
0 2 1
This enables us to write X = [v1 , v2 , v3 ] = 1 0 1 .
1 1 0
   
−1 1 2 −3 0 0
1
X −1 =  1 −1 1  . Finally, X −1 AX =  0 0 0 .
3
1 2 −2 0 0 3
February 2, 2017 6 / 37
Existence/nonexistence of Eigenbases

Example 5
 
1 1
Let A = . Find the eigenvalues and eigenvectors.
0 1
Solution: The characteristic polynomial is D(t) = (t − 1)2 . There is only
one eigenvalue t = 1 which is repeated. Solving for the eigenvectors:
 
0 1
A − I2 = is in REF.
0 0
 
x
=⇒ y = 0 =⇒ v = , x 6= 0.
0

The span of the eigenvectors is only 1-dimensional, and hence they cannot
produce a basis of R2 .

February 2, 2017 7 / 37
Algebraic/geometric multiplicity, defect
Lesson: When λ is an eigenvalue of A, the nullity of A − λI has to be
positive; but may not be sufficiently positive.
This phenomenon leads to the following definitions:
Definition 6 (Algebraic multiplicity)
Let λ be an eigenvalue of A. The multiplicity of λ as a root of the
characteristic polynomial DA (t) is called the algebraic multiplicity of λ.
We write this number as mλ = mλ (A).

Definition 7 (Geometric multiplicity)

Let λ be an eigenvalue of A. The dimension of the eigenspace Eλ i.e. the
nullity of A − λIn is known as the geometric multiplicity of λ. We write
this number as gλ = gλ (A).

If λ is an eigenvalue then
1 ≤ gλ ≤ mλ .
February 2, 2017 8 / 37
Algebraic/geometric multiplicity, defect

Definition 8 (Defect)
Let λ be an eigenvalue of A. The difference mλ − gλ is known as the
defect of λ and is denoted δλ = δλ (A).
X
The sum δλ may be called the total defect of A. (Not a standard
definition.)

Theorem 9 (Existence of eigenbasis)

If the algebraic and the geometric multiplicities of an n × n matrix agree
for every eigenvalue λ of A, then there exists an A-eigenbasis of Rn .

In other words, if each eigenvalue is defect-free, equivalently the total

defect of A vanishes, we can have an A-eigenbasis.

February 2, 2017 9 / 37
Proof of the theorem
Proof:
Denote the distinct eigenvalues: λ1 , λ2 , ..., λk .
Geometric multiplicities: g1 , g2 , ..., gk respectively.

Let Bj = {vj1 , vj2 , ..., vjgj } be a basis of the eigenspace Eλj .

S
Then B = j Bj will be a L.I. set of eigenvectors of A. (∗)

In general,
#B = g1 + g2 + · · · + gk ≤ n
and equality will hold
X
⇐⇒ gj = mj ∀j ⇐⇒ δj = 0 ∀j ⇐⇒ δj = 0,

which is given.
Hence #B = n andtherefore B must be an A-eigenbasis of Rn .

(∗) This line in the above ’proof’ is NOT defect-free.

February 2, 2017 10 / 37
Diagonalization

Theorem 10 (Digonalization)
Let A be defect free i.e. each eigenvalue of A is defect-free. Then A is
similar to a diagonal matrix whose diagonal entries are the eigenvalues of
A, each occurring as many times as its multiplicity.

Proof:
Let
renamed
[
B= Bj = {v11 , ..., v1m1 , ..., vk1 , ..., vkmk } = {w1 , w2 , ..., wn }
j

be an A-eigenbasis.
Let
renamed
{λ1 , ..., λ1 , ..., λk , ..., λk } = {λ1 , λ2 , ..., λn }
| {z } | {z }
m1 times mk times

be the corresponding eigenvalues (with repetitions).

February 2, 2017 11 / 37
Diagonalization

Let
P = [w1 w2 ...wn ]
be the square matrix formed by the columns (eigenvectors) wj .

Then

AP = A[w1 w2 ...wn ] = [λ1 w1 λ2 w2 ...λn wn ] = Pdiag{λ1 , λ2 , ..., λn }

 
λ1 0 · · · 0
 0 λ2 · · · 0 
−1
=⇒ P AP =  .  = diag{λ1 , λ2 , ..., λn }
 
..
 .. . 0
0 0 · λn
-as claimed.

The defect or the gap in the proofs will be corrected later. [1.0]
February 2, 2017 12 / 37
An application
Before correcting the defect (∗) in the proofs, we give an application

Example 11
Consider the coupled system of equations satisfied by ẏ1 (t), ẏ2 (t), ẏ3 (t):

ẏ1 = 2y1 + y2 + y3
ẏ2 = y1 + 4y2 + y3
ẏ3 = y1 + y2 + 2y3 .

Determine an equivalent de-coupled system and hence find

yj (t), j = 1, 2, 3.

Solution:    
2 1 1 y1
In vector form ẏ = Ay, A = 1
 4 1 , y = y2  .
 
1 1 2 y3
The eigenvalues of A are λ = 1, 2, 5.
February 2, 2017 13 / 37
Application contd.
     
−1 1 1
A choice of the corresponding eigenvectors is  0  −1 2 .
1 1 1
 
−1 1 1
Let X =  0 −1 2 be the ’diagonalizer’ and make the ’change of
1 1 1
variables’
u(t) = X −1 y(t).
The equivalent de-coupled system is

u̇ = X −1 ẏ = X −1 Ay = X −1 AX (X −1 y) = diag{1, 2, 5}u

which is simply
u̇1 = u1 , u̇2 = 2u2 , u̇3 = 5u3 .

February 2, 2017 14 / 37
Application contd.

C1 e t
   
u1
The solution of the transformed system is u2  = C2 e 2t  .
u3 C3 e 5t

This in turn yields y = X u viz

C1 e t
    
y1 (t) −1 1 1
y2 (t) =  0 −1 2 C2 e 2t  =?.
y3 (t) 1 1 1 C3 e 5t

We used but did not calculate X −1 . [1.5]

February 2, 2017 15 / 37
Algebraic/geometric multiplicity, defect

Recall:
Definition 12 (Defect)
Let λ be an eigenvalue of A. The difference mλ − gλ is known as the
defect of λ and is denoted δλ = δλ (A).
X
The sum δλ may be called the total defect of A. (Not a standard
definition.)

Theorem 13 (Existence of eigenbasis)

If the algebraic and the geometric multiplicities of an n × n matrix agree
for every eigenvalue λ of A, then there exists an A-eigenbasis of Rn .

In other words, if each eigenvalue is defect-free, equivalently the total

defect of A vanishes, we can have an A-eigenbasis.

February 2, 2017 16 / 37
Proof of the theorem
Proof:
Denote the distinct eigenvalues: λ1 , λ2 , ..., λk .
Geometric multiplicities: g1 , g2 , ..., gk respectively.

Let Bj = {vj1 , vj2 , ..., vjgj } be a basis of the eigenspace Eλj .

S
Then B = j Bj will be a L.I. set of eigenvectors of A. (∗)

In general,
#B = g1 + g2 + · · · + gk ≤ n
and equality will hold
X
⇐⇒ gj = mj ∀j ⇐⇒ δj = 0 ∀j ⇐⇒ δj = 0,

which is given.
Hence #B = n andtherefore B must be an A-eigenbasis of Rn .

(∗) This line in the above ’proof’ is NOT defect-free.

February 2, 2017 17 / 37
L.i. of eigenvectors, removing defect from previous proofs
The defect in the proof was that even though
S vectors in each Bj are linearly
independent (basis of Eλj ), their union j Bj may have dependencies.
For example B1 = {[1 2 3 4], [5 6 7 8]} and B2 = {[2 3 4 5], [3 4 5 7]} are
two linearly independent subsets of R4 , their union does not form a basis
of R4 .

The case of eigenvectors in eigenspaces is however different. Such a

situation can not arise.
Theorem 14
Let λ1 , λ2 , ..., λk be distinct eigenvalues of a square matrix A. Let
v1 , v2 , ..., vk be a corresponding choice of eigenvectors. Then
{v1 , v2 , ..., vk } is a l.i. set.

Proof:
Suppose that
ck1 vk1 + ck2 vk2 + · · · + ckr vkr = 0, (1 ≤ k1 < · · · < kr ≤ k)
February 2, 2017 18 / 37
Proof of l.i. of eigenvectors

is one of the shortest linear relation (of length r ) possible.

Each ck1 , ck2 , ..., ckr 6= 0.
Applying A and multiplication by λk1 successively to this equation, we
obtain

ck1 λk1 vk1 + ck2 λk2 vk2 + · · · + ckr λkr vkr = 0

λk1 ck1 vk1 + λk1 ck2 vk2 + · · · + λk1 ckr vkr = 0.

Taking difference, we get a shorter relation (of length r − 1)

ck2 λk2 − λk1 vk2 + · · · + ckr λkr − λk1 vkr = 0
| {z } | {z }
6=0 6=0

with each scalar non zero -a contradiction.

February 2, 2017 19 / 37
L.i. of eigenvectors, removing defect from previous proofs

Corollary 15
S
The set B = j Bj in the theorem about A-eigenbasis is L.I.

Proof of the corollary:

Suppose that X
cv v = 0.
v∈B

Explicating, X X X
cv v + cv v + · · · + cv v = 0
v∈B1 v∈B2 v∈Bk
| {z } | {z } | {z }
w1 ∈Eλ1 w2 ∈Eλ2 wk ∈Eλk

February 2, 2017 20 / 37
L.i. of eigenvectors, removing defect from previous proofs

Can rewrite,
w1 + w2 + · · · + wk = 0, wj ∈ Eλj .
By the theorem above, each wj = 0. Therefore,
X X
∀jwj = cv v = 0 =⇒ cv v = 0 ∀j
v∈Bj v∈Bj
=⇒ cv = 0 ∀v ∈ Bj , ∀j
=⇒ cv = 0 ∀v ∈ B.

February 2, 2017 21 / 37
Simple eigenvalues

Corollary 16
If the characteristic polynomial has only simple roots (mλ = 1 for each λ),
then Rn admits a basis of eigenvectors of A viz an A-eigenbasis.

Proof: Since mλ = 1 for each characteristic root λ and

1 ≤ gλ ≤ mλ = 1 =⇒ gλ = 1 = mλ . Hence there is an A-eigenbasis of Rn .
(or sometimes Cn )
Example 17
   
1 1 0 1 1 0
Among the matrices A = 0 2 0 and B = 0 1 0, which ones are
0 0 3 0 0 3
diagonalizable?
Solution: A has simple eigenvalues λ = 1, 2, 3 each occurring once.
Hence A is diagonalizable.
For B, λ = 1, 1, 3 with m1 = 2 > g1 = 1. Hence non diagonlizable. [2.0]
February 2, 2017 22 / 37
Trace of a square matrix
Definition 18 (Trace)
The trace of a square matrix is defined to be the sum of its diagonal
entries.
If A is a square matrix, its trace is denoted trA.
In other words, if A = [ajk ] is n × n, then
n
X
trA = ajj .
j=1

For example if  
1 2 3
A = −4 −5 −6 ,
7 8 9
then
trA = 1 − 5 + 9 = 5.

February 2, 2017 23 / 37
Example

Example 19
Let A and B be two n × n matrices, show that tr(AB) = tr(BA).

Solution:Let A = [ajk ] and B = [bjk ], then

n n
" n #
X X X
(AB)jj = aj` b`j =⇒ tr(AB) = aj` b`j .
`=1 j=1 `=1

Similarly,
 
n n n
" n #
re-order sum
X X X X
tr(BA) =  b`j aj`  = b`j aj` = tr(AB).
`=1 j=1 j=1 `=1

February 2, 2017 24 / 37
Example
Example 20
Let A be a real n × n. Show that tr(AT A) = tr(AAT ) = kAk2 .

Solution:Let A = [ajk ] and AT = [bjk ], where bjk = akj . then

n
X
T
(AA )jj = aj` b`j
`=1
Xn
= aj` aj`
`=1
n
" n #
T
X X
2
∴ tr(AA ) = aj` .
j=1 `=1

Therefore tr(AAT ) is the sum of the squares of all the entries of A hence
equals kAk2 .
February 2, 2017 25 / 37
Principal sub-minors
The trace of a square matrix is generalized to the notion of the sum of principal
r × r sub-minors.

Let A be n × n and let J ⊂ n = {1, 2, ..., n} be an ordered subset with r elements,

say J = {ji1 < · · · < jir }.

Definition 21 (Principal submatrix)

The r × r submatrix AJJ obtained by selecting the rows and columns indexed by
the set J is called the principal submatrix of A indexed by J.

The submatrix AJK is already familiar to us. We only consider K = J.

Definition 22 (Principal sub-minors of order r )

The determinant of a principal r × r submatrix of a square matrix is called a
principal sub-minor of order r .

Remark: We encountered sub-minors in the definition of determinantal

rank, though we did not name them so.
February 2, 2017 26 / 37
r th -order trace

Definition 23 (Trace of order r )

The r th order trace of a square matrix is defined to be sum of the principal
subminors of order r .
If A is n × n, we write X  
trr A = AJJ .
J⊂n
#J=r
 
n
It should be obvious that the above sum has terms.
r

In particular,
tr1 = tr and trn = det.

February 2, 2017 27 / 37
Examples

Example 24
 
a b
Let A = . Then
c d
tr1 A = trA = a + d while tr2 A = |A| = ad − bc.
 
1 2 3
Let B = −4 −5 −6 . Then tr1 B = 5, while
7 8 9
     
1 2  −5 −6 1 3
tr2 B =  + +  = −6,
−4 −5  8 9  7 9
 
1 2 3 

tr3 B = −4 −5 −6 = 0.
7 8 9

February 2, 2017 28 / 37
Examples contd.

Example 25
Let B be as in the previous example. Determine the eigenvalues of B and
check for diagonalizability.
Solution: 
1 − t 2 3 

DB (t) =  −4 −5 − t −6  = −t 3 + 5t 2 + 6t = −t(t + 1)(t − 6)
 7 8 9 − t
The eigenvalues are simple and hence B is diagonalizable.

February 2, 2017 29 / 37
Connection with characteristic polynomial

Theorem 26
Let A be the a square matrix of order n. Its characterstic polynomial can
be expressed as
Xn
DA (t) = (trn−r A)(−t)r .
r =0

For convenience we have set tr0 ≡ 1 for all square matrices. Proof omitted.
Example 27
In the previous examples,

DA (t) = t 2 − (a + b)t + (ad − bc), DB (t) = −t 3 + 5t 2 + 6t.

DB (t) was also directly computed. It agrees with the theoretical

expression.

February 2, 2017 30 / 37
Connection with characteristic polynomial

Corollary 28
For a square matrix A, its r th order trace equals the sum of the produts of
its eigenvalues taken r at a time.

We separate out two special cases of the above corollary corresponding to

r = 1 and r = n.

Corollary 29
For a square matrix A, its trace equals the sum of its eigenvalues while the
determinant equals the product of the same.

February 2, 2017 31 / 37
Direct proof of the second corollary

Let λ1 , ..., λn be the eigenvalues (with repetitions).

Then OTO1 H
n
Y n
Y
DA (t) = (λj − t) =⇒ DA (0) = λj .
j=1 j=1

OTO2 H
DA (t) = |A − tI| =⇒ DA (0) = |A|.
n
Y
Next look at p(t) = t n DA (1/t) = (λj t − 1) = |tA − I|.
j=1

Its constant term is p(0) = (−1)n .

X
The coefficient of t is p 0 (0) = (−1)n−1 λj .
j

February 2, 2017 32 / 37
Direct proof of the corollary

OTO2 H,
 
−1 0 · · · a1j ··· 0

 
 0 −1 · · · a2j ··· 0

 
 .. .. .. 
0 d X  . . . 0 
p (0) =  (|tA − I|) = 
dt t=0  0 0 ··· ajj · · · 0 
j 
.. .. .. 
 . . . 0 

0 0 ··· anj · · · −1
X
= (−1)n−1 ajj (j th term by j th row.)
j

= (−1)n−1 trA.
X
Comparing, trA = λj .
j

February 2, 2017 33 / 37
Computation

For A = [A1 A2 · · · An ]- a row of columns, the derivative of the determinant is

A A · · · An 0 = (A1 )0 A2 · · · An  + A1 (A2 )0 · · · An 
 1 2     

+ · · · + A1 A2 · · · (An )0 .
 

0
∴ p 0 (t) = tA1 − e1 , tA2 − e2 , · · · , tAn − en 


= A1 , tA2 − e2 , · · · , tAn − en  + tA1 − e1 , A2 , · · · , tAn − en 

   

+ · · · + tA1 − e1 , tA2 − e2 , · · · , An 
 

Put t = 0 to see that

p 0 (0) = A1 , −e2 , · · · , −en  +  − e1 , A2 , · · · , −en  + · · · +  − e1 , −e2 , · · · , An 

     

= (−1)n−1 a11 + (−1)n−1 a22 + · · · + (−1)n−1 ann = (−1)n−1 trA.

February 2, 2017 34 / 37
Example

Example 30
 
1 2 4
Find whether or not A = 2 4 8  is diagonalizable. Use traces to find the
4 8 16
characteristic polynomial and eigenvalues.
Solution: trA = 21, tr2 A = 0 = tr3 A. Hence the char. polynomial is

DA (t) = −t 3 + 21t 2 .

The eigenvalues are 0,0, 21.   

1 2 4 1 2 4
λ = 0: A − λI = A = 2 4 8  7−→  0 0 0 .
4 8 16 0 0 0
Nullity i.e. geometric mult. of λ = 0 equals 2 equal to the algebraic multiplicity.
λ = 21 is simple. Hence both are defect-free and A is diagonalizable.

February 2, 2017 35 / 37
Computation of other higher traces trr

See Handout-2

February 2, 2017 36 / 37