Math 221: LINEAR ALGEBRA

Chapter 2. Matrix Algebra

§2-5. Elementary Matrices

Le Chen1
Emory University, 2020 Fall
(last updated on 10/26/2020)

Creative Commons License

(CC BY-NC-SA)

1
Slides are adapted from those by Karen Seyffarth from University of Calgary.
Elementary Matrices

Inverses of elementary matrices

Smith Normal Form

Elementary Matrices

Definition
An elementary matrix is a matrix obtained from an identity matrix by
performing a single elementary row operation. The type of an elementary
matrix is given by the type of row operation used to obtain the elementary
matrix.

Remark
Three Types of Elementary Row Operations
I Type I: Interchange two rows.
I Type II: Multiply a row by a nonzero number.
I Type III: Add a (nonzero) multiple of one row to a different row.
Example
     
1 0 0 0 1 0 0 0 1 0 0 0
 0 0 0 1   0 1 0 0   0 1 0 0 
E= ,F =  ,G =  ,
 0 0 1 0   0 0 −2 0   −3 0 1 0 
0 1 0 0 0 0 0 1 0 0 0 1

are examples of elementary matrices of types I, II and III, respectively.

Let  
1 1
 2 2 
A=  3 3 


4 4
We are interested in the effect that (left) multiplication of A by E, F and G
has on the matrix A. Computing EA, FA, and GA . . .
Example (continued)
    
1 0 0 0 1 1 1 1
 0 0 0 1  2 2  =  4 4 
  
EA =  
0 0 1 0  3 3   3 3 
0 1 0 0 4 4 2 2
    
1 0 0 0 1 1 1 1
 0 1 0 0   2 2   2 2 
FA =   = 
 0 0 −2 0   3 3   −6 −6 
0 0 0 1 4 4 4 4
    
1 0 0 0 1 1 1 1
 0 1 0 0   2 2   2 2 
GA =  −3
 = 
0 1 0  3 3   0 0 
0 0 0 1 4 4 4 4

Remark
Notice that EA is the matrix obtained from A by interchanging row 2 and
row 4, which is the same row operation used to obtain E from I4 . What
about FA and GA?
Theorem (Multiplication by an Elementary Matrix)
Let A be an m × n matrix.
If B is obtained from A by performing one single elementary row
operation,
then B = EA
where E is the elementary matrix obtained from Im by performing the
same elementary operation on Im as was performed on A.

A B
El. Op. =⇒ A = EB
I E
Problem
Let
   
4 1 1 3
A= and C=
1 3 2 −5
Find elementary matrices E and F so that C = FEA.

Solution
Note. The statement of the problem implies that C can be obtained from A
by a sequence of two elementary row operations, represented by elementary
matrices E and F.
     
4 1 → 1 3 → 1 3
A= E F =C
1 3 4 1 2 −5
   
0 1 1 0
where E = and F = .Thus we have the sequence
1 0 −2 1
A → EA → F(EA) = C, so C = FEA, i.e.,
     
1 3 1 0 0 1 4 1
=
2 −5 −2 1 1 0 1 3

You can check your work by doing the matrix multiplication. 

Inverses of Elementary Matrices

Lemma
Every elementary matrix E is invertible, and E−1 is also an elementary
matrix (of the same type). Moreover, E−1 corresponds to the inverse of the
row operation that produces E.

The following table gives the inverse of each type of elementary row
operation:

Type Operation Inverse Operation

I Interchange rows p and q Interchange rows p and q
II Multiply row p by k 6= 0 Multiply row p by 1/k
III Add k times row p to row q 6= p Subtract k times row p from row q
Note that elementary matrices of type I are self-inverse.
Inverses of Elementary Matrices

Example
Without using the matrix inversion algorithm, find the inverse of the
elementary matrix  
1 0 0 0
 0 1 0 0 
G=  −3 0 1 0 


0 0 0 1
Hint. What row operation can be applied to G to transform it to I4 ? The
row operation G → I4 is to add three times row one to row three, and thus
 
1 0 0 0
 0 1 0 0 
G−1 =  3 0 1 0 


0 0 0 1

Check by computing G−1 G.

Example (continued)
Similarly,
 −1  
1 0 0 0 1 0 0 0
 0 0 0 1   0 0 0 1 
E−1 =
 0
 = 
0 1 0   0 0 1 0 
0 1 0 0 0 1 0 0
and  −1  
1 0 0 0 1 0 0 0
 0 1 0 0 
F−1  = 0 1 0 0 

=
 
0 0 −2 0   0 0 − 12 0 
0 0 0 1 0 0 0 1
Suppose A is an m × n matrix and that B can be obtained from A by a
sequence of k elementary row operations. Then there exist elementary
matrices E1 , E2 , . . . Ek such that

B = Ek (Ek−1 (· · · (E2 (E1 A)) · · · ))

Since matrix multiplication is associative, we have

B = (Ek Ek−1 · · · E2 E1 )A

or, more concisely, B = UA where U = Ek Ek−1 · · · E2 E1 .

To find U so that B = UA, we could find E1 , E2 , . . . , Ek and multiply these

together (in the correct order), but there is an easier method for finding U.
Definition
Let A be an m × n matrix. We write

A→B

if B can be obtained from A by a sequence of elementary row operations. In

this case, we call A and B are row-equivalent.

Theorem
Suppose A is an m × n matrix and that A → B. Then
1. there exists an invertible m × m matrix U such that B = UA;
2. U
 can be computed
 by performing
 elementary
 row operations on
A Im to transform it into B U ;
3. U = Ek Ek−1 · · · E2 E1 , where E1 , E2 , . . . , Ek are elementary matrices
corresponding, in order, to the elementary row operations used to
obtain B from A.
Problem
 
3 0 1
Let A = , and let R be the reduced row-echelon form of A.
2 −1 0
Find a matrix U so that R = UA.

Solution
     
3 0 1 1 0 1 1 1 1 −1 1 1 1 1 −1
→ →
2 −1 0 0 1 2 −1 0 0 1 0 −3 −2 −2 3
   
1 1 1 1 −1 1 0 1/3 1/3 0
→ →
0 1 2/3 2/3 −1 0 1 2/3 2/3 −1

   
Starting with A I , we’ve obtained R U .

Therefore R = UA, where

 
1/3 0
U= .
2/3 −1


Example ( A Matrix as a product of elementary matrices )
Let  
1 2 −4
A =  −3 −6 13  .
0 −1 2
Suppose we do row operations to put A in reduced row-echelon form, and
write down the corresponding elementary matrices.
     
1 2 −4 1 2 −4 1 2 −4
−→ −→ −→
 −3 −6 13  E1  0 0 1  E2  0 −1 2  E3
0 −1 2 0 −1 2 0 0 1
     
1 2 −4 1 0 0 1 0 0
−→ −→
 0 1 −2  E4  0 1 −2  E5  0 1 0 
0 0 1 0 0 1 0 0 1
Notice that the reduced row-echelon form of A equals I3 . Now find the
matrices E1 , E2 , E3 , E4 and E5 .
Example (continued)
     
1 0 0 1 0 0 1 0 0
E1 =  3 1 0  , E2 =  0 0 1  , E3 =  0 −1 0 
0 0 1 0 1 0 0 0 1
   
1 −2 0 1 0 0
E4 =  0 1 0  , E5 = 0 1 2 
0 0 1 0 0 1
It follows that

(E5 (E4 (E3 (E2 (E1 A))))) = I

(E5 E4 E3 E2 E1 )A = I

and therefore
A−1 = E5 E4 E3 E2 E1
Example (continued)
Since A−1 = E5 E4 E3 E2 E1 ,

A−1 = E5 E4 E3 E2 E1
(A−1 )−1 = (E5 E4 E3 E2 E1 )−1
A = E−1 −1 −1 −1 −1
1 E2 E3 E4 E5

This example illustrates the following result.

Theorem
Let A be an n × n matrix. Then, A−1 exists if and only if A can be written
as the product of elementary matrices.
Example ( revisited – Matrix inversion algorithm)
 
  1 2 −4
A I =  −3 −6 13 I 
0 −1 2
   
  1 2 −4 1 2 −4 1 0 0
E1 A I = 0 0 1 E1  = 0 0 1 3 1 0 
0 −1 2 0 −1 2 0 0 1
   
  1 2 −4 1 2 −4 1 0 0
E2 E1 A I = 0 −1 2 E2 E1  = 0 −1 2 0 0 1 
0 0 1 0 0 1 3 1 0
 Example ( continued )
   
1 2 −4 1 2 −4 1 0 0
E3 E2 E1 [ A | I ] =  0 1 −2 E3 E2 E1  = 0 1 −2 0 0 −1 
0 0 1 0 0 1 3 1 0
   
1 0 0 1 0 0 1 0 2
E4 E3 E2 E1 [ A | I ] =  0 1 −2 E4 E3 E2 E1  = 0 1 −2 0 0 −1 
0 0 1 0 0 1 3 1 0
   
1 0 0 1 0 0 1 0 2
E5 E4 E3 E2 E1 [ A | I ] =  0 1 0 E5 E4 E3 E2 E1  = 0 1 0 6 2 −1 
0 0 1 0 0 1 3 1 0

 
1 0 2
A−1 = E5 E4 E3 E2 E1 =  6 2 −1 
3 1 0
Problem
 
4 1
Express A = as a product of elementary matrices.
−3 2

Solution
         
4 1 −→ 1 3 −→ 1 3 −→ 1 3 −→ 1 0
E1 E2 E3 E4
−3 2 −3 2 0 11 0 1 0 1

with
       
1 1 1 0 1 0 1 −3
E1 = , E2 = , E3 = 1 , E4 =
0 1 3 1 0 11
0 1

Since E4 E3 E2 E1 A = I, A−1 = E4 E3 E2 E1 , and hence

A = E−1 −1 −1 −1
1 E2 E3 E4
Solution (continued)
Therefore,
 −1  −1  −1  −1
1 1 1 0 1 0 1 −3
A= 1
0 1 3 1 0 11
0 1

i.e.,     
1 −1 1 0 1 0 1 3
A=
0 1 −3 1 0 11 0 1
Check your work by computing the product. 
One result that we have assumed in all our work involving reduced
row-echelon matrices is the following.

Theorem ( Uniqueness of the Reduced Echelon Form )

If A is an m × n matrix and R and S are reduced row-echelon forms of A,
then R = S.

Remark
This theorem ensures that the reduced row-echelon form of a matrix is
unique, and its proof follows from the results about elementary matrices.
Smith Normal Form

Definition
 
Ir 0
If A is an m × n matrix of rank r, then the matrix is called the
0 0 m×n
Smith normal form of A.

Theorem
If A is an m × n matrix of rank r, then there exist invertible matrices U and
V of size m × m and n × n, respectively, such that
 
Ir 0
UAV =
0 0 m×n
Proof.
1. Apply the elementary row operations:
e.r.o.
[A|Im ] −→ [rref (A) |U]

2. Apply the elementary column operations:

  
  Ir 0
rref(A) e.c.o. 
−→ 0 0 m×n 
In
V 2m×n

Remark
The elementary column operations above are equivalent to the elementary
row operations on the transpose:
"  #
h
T
i
e.r.o. Ir 0 T
rref(A) In −→ V
0 0 n×m
n×2m
Problem
 
3 0 1
Find the decomposition of A = into the Smith normal form:
2 −1 0
A = UN
e V,e where N is the Smith normal form of A and U,
e V
e are some
invertible matrices.

Solution
We have seen that
   
3 0 1 1 0 1 0 1/3 1/3 0
[A|I2 ] = → = [rref(A)|U]
2 −1 0 0 1 0 1 2/3 2/3 −1

Now,
   
  1 0 1 0 0 1 0 1 0 0 h i
rref(A) T
I3 = 0 1 0 1 0 → 0 1 0 1 0  = NT V T
1 2
3 3
0 0 1 0 0 − 13 − 23 1
Solution (Continued)
Hence, we find N = UAV, namely,
 
     1 0 −1/3
1 0 0 1/3 0 3 0 1 
= 0 1 −2/3
0 1 0 2/3 −1 2 −1 0
0 0 1

Finally, since U and V are invertible, we see that

A = U−1 NV−1 ,

namely,

 1 0 −1/3 −1
 
   −1 
3 0 1 1/3 0 1 0 0 
A= = 0 1 −2/3
2 −1 0 2/3 −1 0 1 0
0 0 1
 
   1 0 1/3
3 0 1 0 0 
= 0 1 2/3
2 −1 0 1 0
0 0 1
= UN
e V.
e