Matrices - Basic Definitions and Notation

Definitions
Let m and n be positive integers.
An m × n matrix is a rectangular array of numbers having m rows and n
columns. Such a matrix is said to have size m × n.
A row matrix (or row) is a 1 × n matrix, and a column matrix (or column) is
an m × 1 matrix.
A square matrix is an n × n matrix.
The (i, j)-entry of a matrix is the entry in row i and column j. For a matrix
A, the (i, j)-entry of A is often written as aij .

General notation for an m × n matrix, A:

 
a11 a12 a13 ... a1n
 a21 a22 a23 ... a2n 
 
A =  a31 a32 a33 ... a3n  = [aij ]
 
 .. .. .. .. 
 . . . . 
am1 am2 am3 ... amn
Matrices: Matrix Arithmetic Matrices Page 4/90
 Matrices – Properties and Operations

1 Equality: two matrices are equal if and only if they have the same
size and the corresponding entries are equal.
2 Zero Matrix: an m × n matrix with all entries equal to zero.
3 Addition: matrices must have the same size; add corresponding
entries.
4 Scalar Multiplication: multiply each entry of the matrix by the
scalar.
5 Negative of a Matrix: for an m × n matrix A, its negative is denoted
−A and −A = (−1)A.
6 Subtraction: for m × n matrices A and B, A − B = A + (−1)B.

Matrices: Matrix Arithmetic Matrices Page 5/90

 Matrix Addition

Definition
Let A = [aij ] and B = [bij ] be two m × n matrices. Then A + B = C where
C is the m × n matrix C = [cij ] defined by

cij = aij + bij

Example
   
1 3 0 −2
Let A = ,B = . Then,
2 5 6 1
 
1 + 0 3 + −2
A+B =
2+6 5+1
 
1 1
=
8 6

Matrices: Matrix Arithmetic Matrix Addition Page 6/90

 Theorem (Properties of Matrix Addition)
Let A, B and C be m × n matrices. Then the following properties hold.
1 A + B = B + A (matrix addition is commutative).
2 (A + B) + C = A + (B + C ) (matrix addition is associative).
3 There exists an m × n zero matrix, 0, such that A + 0 = A.
(existence of an additive identity).
4 There exists an m × n matrix −A such that A + (−A) = 0.
(existence of an additive inverse).

Matrices: Matrix Arithmetic Matrix Addition Page 7/90

 Scalar Multiplication
Definition
Let A = [aij ] be an m × n matrix and let k be a scalar. Then kA = [kaij ].

Example
 
2 0 −1
Let A =  3 1 −2 .
0 4 5
Then
 
3(2) 3(0) 3(−1)
3A =  3(3) 3(1) 3(−2) 
3(0) 3(4) 3(5)
 
6 0 −3
=  9 3 −6 
0 12 15

Matrices: Matrix Arithmetic Scalar Multiplication Page 8/90

 Theorem (Properties of Scalar Multiplication)
Let A, B be m × n matrices and let k, p ∈ R (scalars). Then the following
properties hold.
1 k (A + B) = kA + kB.
(scalar multiplication distributes over matrix addition).
2 (k + p) A = kA + pA.
(addition distributes over scalar multiplication).
3 k (pA) = (kp) A. (scalar multiplication is associative).
4 1A = A. (existence of a multiplicative identity).

Matrices: Matrix Arithmetic Scalar Multiplication Page 9/90

 Example
       
−1 0 −2 1 6 8 −16 −4
2 +4 − =
1 1 3 0 1 −1 13 3

Problem
Let A and B be m × n matrices. Simplify the expression

2[9(A − B) + 7(2B − A)] − 2[3(2B + A) − 2(A + 3B) − 5(A + B)]

Solution

2[9(A − B) + 7(2B − A)] − 2[3(2B + A) − 2(A + 3B) − 5(A + B)]

= 2(9A − 9B + 14B − 7A) − 2(6B + 3A − 2A − 6B − 5A − 5B)
= 2(2A + 5B) − 2(−4A − 5B)
= 12A + 20B
Matrices: Matrix Arithmetic Scalar Multiplication Page 10/90
 Vectors

Definitions
A row matrix or column matrix is often called a vector, and such matrices
are referred to as row vectors and column vectors, respectively. If X is a row
vector of size 1 × n, and Y is a column vector of size m × 1, then we write
 
y1
   y2 
X = x1 x2 · · · xn and Y =  . 
 
.
 . 
ym

Matrices: Matrix Arithmetic Vectors Page 11/90

 Vector form of a system of linear equations
Definition
Consider the system of linear equations

a11 x1 + a12 x2 + ··· + a1n xn = b1

a21 x1 + a22 x2 + ··· + a2n xn = b2
.. .. .. ..
. . . .
am1 x1 + am2 x2 + · · · + amn xn = bm

Such a system can be expressed in vector form or as a vector equation by

using linear combinations of column vectors:
       
a11 a12 a1n b1
 a21   a22   a2n   b2 
x1  .  + x2  .  + · · · + xn  . = ..
       
.
 .  .
 .   ..   .


am1 am2 amn bm

Matrices: Matrix Arithmetic Vectors Page 12/90

 Vector form of a system of linear equations

Problem
Express the following system of linear equations in vector form.

2x1 + 4x2 − 3x3 = −6

− x2 + 5x3 = 0
x1 + x2 + 4x3 = 1

Solution
       
2 4 −3 −6
x1  0  + x2  −1  + x3  5  =  0 
1 1 4 1

Matrices: Matrix Arithmetic Vectors Page 13/90

 Matrix Vector Multiplication
Definition
Let A = [aij ] be an m × n matrix with columns A1 , A2 , . . . , An , written
A = A1 A2 · · · An , and let X be an n × 1 column vector,
 
x1
 x2 
X = . 
 
 .. 
xn
Then the product of matrix A and (column) vector X is the m × 1 column
vector given by
 
x1
n
 x2 

  X
A1 A2 · · · An  .  = x1 A1 + x2 A2 + · · · + xn An = xj Aj
 .. 
j=1
xn

that is, AX is a linear combination of the columns of A.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 14/90
 Matrix Vector Multiplication

Problem
Compute the product AX for
   
1 4 2
A= and X =
5 0 3

Solution
            
1 4 2 1 4 2 12 14
AX = =2 +3 = + =
5 0 3 5 0 10 0 10

Matrices: Matrix Arithmetic Multiplication of Matrices Page 15/90

 Matrix Vector Multiplication

Problem
Compute AY for
 
 2 
1 0 2 −1  −1 
A =  2 −1 0 1  and Y = 
 1 

3 1 3 1
4

Solution
         
1 0 2 −1 0
AY = 2  2  + (−1)  −1  + 1  0  + 4  1  =  9 
3 1 3 1 12

Matrices: Matrix Arithmetic Multiplication of Matrices Page 16/90

 Matrix form of a system of linear equations
Definition
Consider the system of linear equations

a11 x1 + a12 x2 + ··· + a1n xn = b1

a21 x1 + a22 x2 + ··· + a2n xn = b2
.. .. ..
. . .
am1 x1 + am2 x2 + ··· + amn xn = bm

Such a system can be expressed in matrix form using matrix vector multiplication,
    
a11 a12 · · · a1n x1 b1
 a21 a22 · · · a2n   x2   b2 
 .. .. ..   ..  =  .. 
    
 . . .  .   . 
am1 am2 ··· amn xn bm
Thus a system of linear equations can be expresses as a matrix equation AX = B,
where A is the coefficient matrix, B is the constant matrix, and X is the matrix of
variables.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 17/90

 Matrix form of a system of linear equations

Problem
Express the following system of linear equations in matrix form.

2x1 + 4x2 − 3x3 = −6

− x2 + 5x3 = 0
x1 + x2 + 4x3 = 1

Solution
    
2 4 −3 x1 −6
 0 −1 5   x2  =  0 
1 1 4 x3 1

Matrices: Matrix Arithmetic Multiplication of Matrices Page 18/90

 Matrix and Vector Equations
Theorem
1 Every system of m linear equations in n variables can be written in the
form AX = B where A is the coefficient matrix, X is the matrix of
variables, and B is the constant matrix.
2 The system AX = B is consistent (i.e., has at least one solution) if
and only if B is a linear combination of the columns of A.
 
x1
 x2 
The vector X =  .  is a solution to the system AX = B if and
3
 
 .. 
xn
only if x1 , x2 , . . . , xn are a solution to the vector equation

x1 A1 + x2 A2 + · · · xn An = B

where A1 , A2 , . . . , An are the columns of A.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 19/90

 Proof of the Theorem (a sketch)

Every statement that deserves to be called a theorem deserves a proof, and

the theorem from the previous slide is no exception. In this particular case
the proof is straightforward (i.e. uneventful).

Proof.
(a) One first checks
 x that
 (x1 , . . . , xn ) is a solution to the original system if
1
x2
and only if X =  .
.
 is a solution to AX = B.
.
xn

This depends on the way that the matrix arithmetics (addition,

multiplication by scalars, multiplication) was defined.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 20/90

 Proof continued

Proof.
(b) Once (a) is taken care of, it gives a one-to-one correspondence between
the set of solutions to the original system and the set of solutions to
AX = B:   x1
x2
(x1 , . . . , xn ) 7→  .
.
.
.
xn

This is (3), and it implies that the two sets have the same cardinality, and
(2) follows.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 21/90

 Problem
   
Let 1 0 2 −1 1
A =  2 −1 0 1  and B =  1 
3 1 3 1 1
Express B as a linear combination of the columns A1 , A2 , A3 , A4 of A, or show
that this is impossible.

Solution
Solve the system AX = B where X is a column
 vector with four entries. Do so by
putting the augmented matrix A B in reduced row-echelon form.
1
 
  1 0 0 1
1 0 2 −1 1 7
0 1 0 1 − 57
 
 2 −1 0 1 1  → ··· → 



3 1 3 1 1 0 0 1 −1 3
7

Since there are infinitely many solutions (x4 is assigned a parameter), choose any
value for x4 . Choosing x4 = 0 (which is the simplest thing to do) gives us
       
1 1 0 2
1 5 3 1 5 3
B =  1  =  2  −  −1  +  0  = A1 − A2 + A3 + 0A4 .
7 7 7 7 7 7
1 3 1 3

Matrices: Matrix Arithmetic Multiplication of Matrices Page 22/90

 Matrix Multiplication

Definition (Product of two matrices)

 
Let A be an m × n matrix and let B = B1 B2 · · · Bp be an n × p
matrix, whose columns are B1 , B2 , . . . , Bp . The product of A and B is the
matrix
   
AB = A B1 B2 · · · Bp = AB1 AB2 · · · ABp

i.e., the first column of AB is AB1 , the second column of AB is AB2 , etc.
Note that AB has size m × p.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 23/90

 Problem
Find the product AB of matrices  
  −1 1 2
−1 0 3
A= and B =  0 −2 4 
2 −1 1
1 0 0

Solution
AB has columns
   
  −1   1
−1 0 3  0  , AB2 = −1 0 3 
AB1 = −2  ,
2 −1 1 2 −1 1
1 0
 
  2
−1 0 3  4 
and AB3 =
2 −1 1
0
 
4 −1 −2
Thus, AB = .
−1 4 0
Matrices: Matrix Arithmetic Multiplication of Matrices Page 24/90
 Compatibility for Matrix Multiplication

Definition
Let A and B be matrices, and suppose that A is m × n.
In order for the product AB to exist, the number of rows in B must be
equal to the number of columns in A, implying that B is an n × p
matrix for some p.
When defined, AB is an m × p matrix.
If the product is defined, then A and B are said to be compatible for
(matrix) multiplication.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 25/90

 Example
As we saw in the previous problem
3×3
2×3

 −1
 2×3
 1 2  
−1 0 3  4 −1 −2
0 −2 4 = 
2 −1 1 −1 4 0
1 0 0

Note that the product

3×3
  2×3
−1 1 2  
 0 −2 4  −1 0 3
2 −1 1
1 0 0

does not exist.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 26/90

 Multiplication by the Zero Matrix
Example
Compute the product A0 for the matrix
 
1 2
A=
3 4
 
0 0
and the 2 × 2 zero matrix given by 0 =
0 0

Solution
In this product, we compute
    
1 2 0 0 0 0
=
3 4 0 0 0 0

Hence, A0 = 0.

Matrices: Matrix Arithmetic Multiplication of Matrices Page 27/90

 Definition (The (i, j)-entry of a product)
Let A = [aij ] be an m × n matrix and B = [bij ] be an n × p matrix. Then
the (i, j)-entry of AB is given by
n
X
ai1 b1j + ai2 b2j + · · · + ain bnj = aik bkj
k=1

Example
Using the above definition, the (2, 3)-entry of the product
 
  −1 1 2
−1 0 3 
0 −2 4 
2 −1 1
1 0 0

is computed using the second row of the first matrix, and the third column
of the second matrix, resulting in

2 × 2 + (−1) × 4 + 1 × 0 = 4 − 4 + 0 = 0.
Matrices: Matrix Arithmetic The (i, j)-Entry of a Product Page 28/90
 Questions on Matrix Multiplication

Given matrices A and B, is AB = BA?

Suppose A is an m × n matrix and B is an m0 × n0 matrix.

The product AB is defined if and only if n = m0 .
The product BA is defined if and only if m = n0 .
Therefore the equation AB = BA makes sense if and only if A is an m × n
matrix and B is an n × m matrix for some—possibly different—m and n.

So the right question is:

Given matrices A and B such that both AB and BA are defined, is

AB = BA?

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 29/90

 Matrix Multiplication is Not Commutative
Problem
Let 
1 2  
1 −1 2 0
A = −3
 0  and B =
3 −2 1 −3
1 −4

Does AB exist? If so, compute it.

Does BA exist? If so, compute it.

Solution
 
7 −5 4 −6
AB =  −3 3 −6 0 
−11 7 −2 12

BA does not exist

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 30/90

 Problem
Let  
1  
G= and H = 1 0
1

Does GH exist? If so, compute it.

Does HG exist? If so, compute it.

Solution
 
1 0
GH =
1 0
 
HG = 1
In this example, GH and HG both exist, but they are not equal. They
aren’t even the same size!

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 31/90

 Problem
Let    
1 0 −1 1
P= and Q =
2 −1 0 3

Does PQ exist? If so, compute it.

Does QP exist? If so, compute it.

Solution
 
−1 1
PQ =
−2 −1
 
1 −1
QP =
6 −3
In this example, PQ and QP both exist and are the same size, but
PQ 6= QP.

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 32/90

 Fact
The three preceding problems illustrate an important property of matrix
multiplication.
In general, matrix multiplication is not commutative, i.e., the
order of the matrices in the product is important.
In other words, in general AB 6= BA.

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 33/90

 Problem
Let    
2 0 1 2
U= and V =
0 2 3 4

Does UV exist? If so, compute it.

Does VU exist? If so, compute it.

Solution
 
2 4
UV =
6 8
 
2 4
VU =
6 8
In this particular example, the matrices commute, i.e., UV = VU.

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 34/90

 Properties of Matrix Multiplication

Theorem
Let A, B, and C be matrices of the appropriate sizes, and let r ∈ R be a
scalar. Then the following properties hold.
1 A(B + C ) = AB + AC .
(matrix multiplication distributes over matrix addition).
2 (B + C )A = BA + CA.
(matrix multiplication distributes over matrix addition).
3 A (BC ) = (AB) C . (matrix multiplication is associative).
4 r (AB) = (rA)B = A(rB).

This applies to matrix-vector multiplication as well, since a vector is a row

matrix or a column matrix.

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 35/90

 Problem
Let A = [aij ], B = [bij ] and C = [cij ] be three n × n matrices. For
1 ≤ i, j ≤ n write down a formula for the (i, j)-entry of each of the
following matrices.

1 AB 5 (AB)C
2 BA 6 (A+B)
3 A+C
4 A(BC) 7 C(A+B)

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 36/90

 Elementary Proofs

Problem
Let A and B be m × n matrices, and let C be an n × p matrix. Prove that
if A and B commute with C , then A + B commutes with C .

Proof.
We are given that AC = CA and BC = CB. Consider (A + B)C .

(A + B)C = AC + BC
= CA + CB
= C (A + B)

Since (A + B)C = C (A + B), A + B commutes with C .

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 37/90

 Problem
Let A, B and C be n × n matrices, and suppose that both A and B
commute with C , i.e., AC = CA and BC = CB. Show that AB commutes
with C .

Proof.
We must show that (AB)C = C (AB) given that AC = CA and BC = CB.

(AB)C = A(BC ) (matrix multiplication is associative)

= A(CB) (B commutes with C )
= (AC )B (matrix multiplication is associative)
= (CA)B (A commutes with C )
= C (AB) (matrix multiplication is associative)

Therefore, AB commutes with C .

Matrices: Matrix Arithmetic Properties of Matrix Multiplication Page 38/90

 Definition (Matrix Transpose)
If A is an m × n matrix, then its transpose, denoted AT , is the n × m
whose i th row is the i th column of A, 1 ≤ i ≤ n; i.e., if A = [aij ], then

AT = [aij ]T = [aji ]

i.e., the (i, j)-entry of AT is the (j, i)-entry of A.

Theorem (Properties of the Transpose of a Matrix)

Let A and B be m × n matrices, C be a n × p matrix, and r ∈ R a scalar.
Then

1 (AT )T = A 3 (A + B)T = AT + B T
2 (rA)T = rAT 4 (AC )T = C T AT
To prove each these properties, you only need to compute the (i, j)-entries
of the matrices on the left-hand side and the right-hand side. And you can
do it!
Matrices: Matrix Arithmetic The Transpose Page 39/90
 Problem
 
  T 2 1
1 −1 0
Find the matrix A if A + 3 =  0 5 .
1 2 4
3 8

Solution
 
  T 2 1
1 −1 0
A+3 =  0 5 
1 2 4
3 8
   
1 −1 0 2 0 3
A+3 =
1 2 4 1 5 8
   
2 0 3 1 −1 0
A = −3
1 5 8 1 2 4
 
−1 3 3
A =
−2 −1 −4

Matrices: Matrix Arithmetic The Transpose Page 40/90

 Symmetric Matrices
Definition
Let A = [aij ] be an m × n matrix. The entries a11 , a22 , a33 , . . . are called
the main diagonal of A.

Definition
The matrix A is called symmetric if and only if AT = A. Note that this
immediately implies that A is a square matrix.

Examples
 
  0 2 5 −1
  −1 0 5
2 −3  2 1 −3 0 
,  0 2 11  ,  
−3 17  5 −3 2 −7 
5 11 −3
−1 0 −7 4
are symmetric matrices, and each is symmetric about its main diagonal.

Matrices: Matrix Arithmetic The Transpose Page 41/90

 Problem
Show that if A and B are symmetric matrices, then AT + 2B is symmetric.

Proof.

(AT + 2B)T = (AT )T + (2B)T

= A + 2B T
= AT + 2B, since AT = A and B T = B

Since (AT + 2B)T = AT + 2B, AT + 2B is symmetric.

Matrices: Matrix Arithmetic The Transpose Page 42/90

 Skew Symmetric Matrices
Definition
An n × n matrix A is said to be skew symmetric if AT = −A.

Example (Skew Symmetric Matrices)

 
  0 9 4
0 2
,  −9 0 −3 
−2 0
−4 3 0

Problem
Show that if A is a square matrix, then A − AT is skew-symmetric.

Solution
We must show that (A − AT )T = −(A − AT ). Using the properties of matrix
addition, scalar multiplication, and transposition

(A − AT )T = AT − (AT )T = AT − A = −(A − AT ).
Matrices: Matrix Arithmetic The Transpose Page 43/90
 The n × n Identity Matrix
Definition
For each n ≥ 2, the n × n identity matrix, denoted In , is the matrix having ones
on its main diagonal and zeros elsewhere, and is defined for all n ≥ 2.

Example
 
  1 0 0
1 0
I2 = , I3 = 0
 1 0 
0 1
0 0 1

Definition
Let n ≥ 2. For each j, 1 ≤ j ≤ n, we denote by Ej the j th column of In .

Example
    
1 0 0
When n = 3, E1 =  0  , E2 =  1  , E3 =  0 .
0 0 1
Matrices: Matrix Arithmetic The Identity and Inverse Page 44/90
 Theorem
Let A be an m × n matrix Then AIn = A and Im A = A.

Proof
The (i, j)-entry of AIn is the product
 of the i th row of A = [aij ], namely
ai1 ai2 · · · aij · · · ain with the j th column of In , namely Ej . Since Ej
has a one in row j and zeros elsewhere,
 
ai1 ai2 · · · aij · · · ain Ej = aij

Since this is true for all i ≤ m and all j ≤ n, AIn = A.

The proof of Im A = A is analogous—work it out!

Matrices: Matrix Arithmetic The Identity and Inverse Page 45/90

 Instead of AIn and Im A we often write AI and IA, respectively, since the size of
the identity matrix is clear from the context: the sizes of A and I must be
compatible for matrix multiplication.
Thus
AI = A and IA = A
which is why I is called an identity matrix – it is an identity for matrix
multiplication.

Matrices: Matrix Arithmetic The Identity and Inverse Page 46/90

 Matrix Inverses

Definition
Let A be an n × n matrix. Then B is an inverse of A if and only if AB = In and
BA = In . Note that since A and In are both n × n, B must also be an n × n
matrix.

Example
   
1 2 −2 1
Let A = and B = 3 . Then
3 4 2 − 12
 
1 0
AB =
0 1

and  
1 0
BA =
0 1
so B is an inverse of A.

Matrices: Matrix Arithmetic The Identity and Inverse Page 47/90

 Does every square matrix have an inverse?
No! Take e.g. the zero matrix 0n (all entries of On are equal to 0)

A0n = 0n A = On
Pn
for all n × n matrices A: The (i, j)-entry of On A is equal to k=1 0akj = 0.

Does every nonzero square matrix have an inverse?

Matrices: Matrix Arithmetic The Identity and Inverse Page 48/90

 Example
Does the matrix  
0 1
A=
0 1
have an inverse?
No! To see this, suppose  
a b
B=
c d
is an inverse of A. Then
    
0 1 a b c d
AB = =
0 1 c d c d

which is never equal to I2 . (Why?)

Matrices: Matrix Arithmetic The Identity and Inverse Page 49/90

 Uniqueness of an Inverse

Theorem
If A is a square matrix and B and C are inverses of A, then B = C .

Proof.
Since B and C are inverses of A, AB = I = BA and AC = I = CA. Then

C = CI = C (AB) = CAB

and
B = IB = (CA)B = CAB
so B = C .

Matrices: Matrix Arithmetic The Identity and Inverse Page 50/90

 Example (revisited)
   
1 2 −2 1
For A = and B = 3 , we saw that
3 4 2 − 12
   
1 0 1 0
AB = and BA =
0 1 0 1

The preceding theorem tells us that B is the inverse of A, rather than just an
inverse of A.

Matrices: Matrix Arithmetic The Identity and Inverse Page 51/90

 Definitions
Let A be a square matrix, i.e., an n × n matrix.
The inverse of A, if it exists, is denoted A−1 , and

AA−1 = I = A−1 A

If A has an inverse, then we say that A is invertible.

Matrices: Matrix Arithmetic The Identity and Inverse Page 52/90

 Finding the inverse of a 2 × 2 matrix
Example
 
a b
Suppose that A = . If ad − bc 6= 0, then there is a formula for A−1 :
c d
 
−1 1 d −b
A =
ad − bc −c a

This can easily be verified by computing the products AA−1 and A−1 A.
   
−1 a b 1 d −b
AA =
c d ad − bc −c a
  
1 a b d −b
=
ad − bc c d −c a
   
1 ad − bc 0 1 0
= =
ad − bc 0 −bc + ad 0 1

Showing that A−1 A = I2 is left as an exercise.

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 53/90

 Finding the inverse of an n × n matrix

Problem
Suppose that A is any n × n matrix.
How do we know whether or not A−1 exists?
If A−1 exists, how do we find it?

Solution
The matrix inversion algorithm.

Although the formula for the inverse of a 2 × 2 matrix is quicker and easier to use
than the matrix inversion algorithm, the general formula for the inverse an n × n
matrix, n ≥ 3 (which we will see later), is more complicated and difficult to use
than the matrix inversion algorithm. To find inverses of square matrices that are
not 2 × 2, the matrix inversion algorithm is the most efficient method to use.

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 54/90

 The Matrix Inversion Algorithm
Let A be an n × n matrix. To find A−1 , if it exists,
take the n × 2n matrix  
A In
obtained by augmenting A with the n × n identity matrix, In .
 
Perform elementary row operations to transform A In into a reduced
row-echelon matrix.

Theorem (Matrix Inverses)

Let A be an n × n matrix. Then the following conditions are equivalent.
1 A is invertible.
2 the reduced row-echelon form on A is I .
A In can be transformed into In A−1 using the Matrix Inversion
   
3

Algorithm.

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 55/90

 Problem
 
1 0 −1
Find, if possible, the inverse of  −2 1 3 .
−1 1 2

Solution
Using the matrix inversion algorithm
   
1 0 −1 1 0 0 1 0 −1 1 0 0
 −2 1 3 0 1 0  → 0 1 1 2 1 0  →
−1 1 2 0 0 1 0 1 1 1 0 1
 
1 0 −1 1 0 0
 0 1 1 2 1 0 
0 0 0 −1 −1 1

From this, we see that A has no inverse.

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 56/90

 Problem
 
3 1 2
Let A =  1 −1 3 . Find the inverse of A, if it exists.
1 2 4

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 57/90

 Solution (continued)
Using the matrix inversion algorithm
   
  3 1 2 1 0 0 1 −1 3 0 1 0
A I =  1 −1 3 0 1 0  → 3 1 2 1 0 0 →
1 2 4 0 0 1 1 2 4 0 0 1
   
1 −1 3 0 1 0 1 −1 3 0 1 0
 0 4 −7 1 −3 0  →  0 1 −8 1 −2 −1  →
0 3 1 0 −1 1 0 3 1 0 −1 1
   
1 0 −5 1 −1 −1 1 0 −5 1 −1 −1
 0 1 −8 1 −2 −1  →  0 1 −8 1 −2 −1  →
3 5 4
0 0 25 −3 5 4 0 0 1 − 25 25 25
10 5
1 0 0 0 − 25
 
25
1
− 10 7
A−1
 
 0 1 0 = I
25 25 25
3 5 4
0 0 1 − 25 25 25

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 58/90

 Solution (continued)
Therefore, A−1 exists, and
 10 5
0 − 25
  
25 10 0 −5
1
A−1 =  1
25 − 10
25
7
25
=  1 −10 7 
25
3
− 25 5 4 −3 5 4
25 25

You can check your work by computing AA−1 and A−1 A.

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 59/90

 Systems of Linear Equations and Inverses

Suppose that a system of n linear equations in n variables is written in matrix

form as AX = B, and suppose that A is invertible.

Example
The system of linear equations

2x − 7y = 3
5x − 18y = 8

can be written in matrix form as AX = B:

    
2 −7 x 3
=
5 −18 y 8
 
18 −7
You can check that A−1 = .
5 −2

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 60/90

 Example (continued)
Since A−1 exists and has the property that A−1 A = I , we obtain the following.
AX = B
A−1 (AX ) = A−1 B
(A−1 A)X = A−1 B
IX = A−1 B
X = A−1 B
i.e., AX = B has the unique solution given by X = A−1 B. Therefore,
      
3 18 −7 3 −2
X = A−1 = =
8 5 −2 8 −1

You should verify that x = −2, y = −1 is a solution to the system.

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 61/90

 The last example illustrates another method for solving a system of linear
equations when the coefficient matrix is square and invertible. Unless that
coefficient matrix is 2 × 2, this is generally NOT an efficient method for solving a
system of linear equations.

Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 62/90

 Example
Let A, B and C be matrices, and suppose that A is invertible.
1 If AB = AC , then

A−1 (AB) = A−1 (AC )

(A−1 A)B = (A−1 A)C
IB = IC
B = C

2 If BA = CA, then

(BA)A−1 = (CA)A−1
B(AA−1 ) = C (AA−1 )
BI = CI
B = C

Problem
Find square matrices A, B and C for which AB = AC but B 6= C .
Matrices: Matrix Arithmetic Finding the Inverse of a Matrix Page 63/90
 Inverses of Transposes and Products
Example
Suppose A is an invertible matrix. Then

AT (A−1 )T = (A−1 A)T = I T = I

and
(A−1 )T AT = (AA−1 )T = I T = I
This means that (AT )−1 = (A−1 )T .

Example
Suppose A and B are invertible n × n matrices. Then

(AB)(B −1 A−1 ) = A(BB −1 )A−1 = AIA−1 = AA−1 = I

and
(B −1 A−1 )(AB) = B −1 (A−1 A)B = B −1 IB = B −1 B = I
This means that (AB)−1 = B −1 A−1 .
Matrices: Matrix Arithmetic Properties of the Inverse Page 64/90
 Inverses of Transposes and Products

The previous two examples prove the first two parts of the following theorem.

Theorem
1 If A is an invertible matrix, then (AT )−1 = (A−1 )T .
2 If A and B are invertible matrices, then AB is invertible and

(AB)−1 = B −1 A−1

3 If A1 , A2 , . . . , Ak are invertible, then A1 A2 · · · Ak is invertible and

(A1 A2 · · · Ak )−1 = A−1 −1 −1 −1

k Ak−1 · · · A2 A1

(the third part is proved by iterating the above, or, more formally, by using
the mathematical induction)

Matrices: Matrix Arithmetic Properties of the Inverse Page 65/90

 Properties of Inverses

Theorem
1 I is invertible, and I −1 = I .
2 If A is invertible, so is A−1 , and (A−1 )−1 = A.
3 If A is invertible, so is Ak , and (Ak )−1 = (A−1 )k .
(Ak means A multiplied by itself k times)
4 If A is invertible and p ∈ R is nonzero, then pA is invertible, and
(pA)−1 = p1 A−1 .

Matrices: Matrix Arithmetic Properties of the Inverse Page 66/90

 Example
 
T −1 1 1
Given (3I − A ) = 2 , we wish to find the matrix A. Taking inverses
2 3
of both sides of the equation:
  −1
T 1 1
3I − A = 2
2 3
 −1
1 1 1
=
2 2 3
 
1 3 −1
=
2 −2 1
 3
− 12

= 2
1
−1 2

Matrices: Matrix Arithmetic Properties of the Inverse Page 67/90

 Example (continued)

3
− 21
 
T 2
3I − A = 1
−1 2
3
− 12
 
−AT = 2
1 − 3I
−1 2
3
− 12
   
T 2 3 0
−A = 1 −
−1 2
0 3
− 32 − 12
 
−AT =
−1 − 52
 3 
2 1
A = 1 5
2 2

Matrices: Matrix Arithmetic Properties of the Inverse Page 68/90

 Problem
True or false? Justify your answer.

If A3 = 4I , then A is invertible.

Solution
If A3 = 4I , then
1 3
A =I
4
so
1 1
( A2 )A = I and A( A2 ) = I
4 4

Therefore A is invertible, and A−1 = 14 A2 .

Matrices: Matrix Arithmetic Properties of the Inverse Page 69/90

 A Fundamental Result

Theorem
Let A be an n × n matrix, and let X , B be n × 1 vectors. The following
conditions are equivalent.
1 The rank of A is n.
2 A can be transformed to In by elementary row operations.
3 A is invertible.
4 There exists an n × n matrix C with the property that CA = In .
5 The system AX = B has a unique solution X for any choice of B.
6 AX = 0 has only the trivial solution, X = 0.
7 There exists an n × n matrix C with the property that AC = In .

Matrices: Matrix Arithmetic Properties of the Inverse Page 70/90

 Proof of Theorem:
(1) ⇒ (2) The rank of A is the number of leading 1s in the RREF of A. Since the
size of A is n × n, rank (A) = n is equivalent to A being row-equivalent to In .
(2) ⇒ (3): Matrix inversion algorithm.
(3) ⇒ (4): C = A−1 .
(4) ⇒ (5): X = CB.
(5) ⇒ (6): Take B = 0.
(6) ⇒ (1): If rank of A is < n, then there are non-leading variables in the RREF
of [A|0]. Hence AX = 0 has infinitely many solutions.
(4) ⇔ (7): CA = I if and only if AT C T = I ; hence (4) for A is equivalent to (7)
for AT .
We already know that A−1 exists if and only if (AT )−1 exists.

Matrices: Matrix Arithmetic Properties of the Inverse Page 71/90

 The following is an important and useful consequence of the theorem.

Theorem
If A and B are n × n matrices such that AB = I , then BA = I . Furthermore, A
and B are invertible, with B = A−1 and A = B −1 .

Important Fact
In the second Theorem, it is essential that the matrices be square.

Matrices: Matrix Arithmetic Properties of the Inverse Page 72/90

 Theorem
If A and B are matrices such that AB = I and BA = I , then A and B are square
matrices (of the same size).

Matrices: Matrix Arithmetic Properties of the Inverse Page 73/90

 Example
 
  1 0
1 1 0
Let A = and B =  0 0 . Then
−1 4 1
1 1
 
  1 0  
1 1 0  0 1 0
AB = 0  = = I2
−1 4 1 0 1
1 1
and   
1 0   1 1 0
1 1 0
BA =  0 0  =  0 0 0  6= I3
−1 4 1
1 1 0 5 1

This example illustrates why “an inverse” of a non-square matrix doesn’t make
sense. If A is m × n and B is n × m, where m 6= n, then even if AB = I , it will
never be the case that BA = I .

Matrices: Matrix Arithmetic Properties of the Inverse Page 74/90

 Elementary Matrices

Definition
An elementary matrix is a matrix obtained from an identity matrix by performing
a single elementary row operation.

The type of an elementary matrix is given by the type of row operation used to
obtain the elementary matrix. Recall the elementary row operations.

Elementary Row Operations.

Type I: Interchange two rows.
Type II: Multiply a row by a nonzero number.
Type III: Add a (nonzero) multiple of one row to a different row.

Matrices: Matrix Arithmetic Elementary Matrices Page 75/90

 Example
     
1 0 0 0 1 0 0 0 1 0 0 0
 0 0 0 1   0 1 0 0   0 1 0 0 
E = ,F =  ,G =  ,
 0 0 1 0   0 0 −2 0   −3 0 1 0 
0 1 0 0 0 0 0 1 0 0 0 1

are examples of elementary matrices of types I, II and III, respectively.

Let  
1 1
 2 2 
A=  3 3 


4 4
We are interested in the effect that (left) multiplication of A by E , F and G has
on the matrix A. Computing EA, FA, and GA . . .

Matrices: Matrix Arithmetic Elementary Matrices Page 76/90

 Example (continued)
    
1 0 0 0 1 1 1 1
 0 0 0 1   2 2   4 4 
EA =  0
 = 
0 1 0  3 3   3 3 
0 1 0 0 4 4 2 2
    
1 0 0 0 1 1 1 1
 0 1 0 0  2 2  =  2 2 
  
FA = 
 
0 0 −2 0   3 3   −6 −6 
0 0 0 1 4 4 4 4
    
1 0 0 0 1 1 1 1
 0 1 0 0   2 2  =  2 2 
  
GA =  
−3 0 1 0  3 3   0 0 
0 0 0 1 4 4 4 4
Notice that EA is the matrix obtained from A by interchanging row 2 and row 4,
which is the same row operation used to obtain E from I4 . What about FA and
GA?

Matrices: Matrix Arithmetic Elementary Matrices Page 77/90

 Multiplication by an Elementary Matrix

Theorem
Let A be an m × n matrix, and suppose that B is obtained from A by performing
a single elementary row operation. Then B = EA where E is the elementary
matrix obtained from Im by performing the same elementary operation on Im as
was performed on A.

Matrices: Matrix Arithmetic Elementary Matrices Page 78/90

 Problem
   
Let 4 1 1 3
A= and C =
1 3 2 −5
Find elementary matrices E and F so that C = FEA.

Solution
Note. The statement of the problem implies that C can be obtained from A by a
sequence of two elementary row operations, represented by elementary matrices E
and F .      
4 1 → 1 3 → 1 3
A= E F =C
1 3 4 1 2 −5
   
0 1 1 0
where E = and F = . Thus we have the sequence
1 0 −2 1
A → EA → F (EA) = C , so C = FEA, i.e.,
     
1 3 1 0 0 1 4 1
=
2 −5 −2 1 1 0 1 3

You can check your work by doing the matrix multiplication.

Matrices: Matrix Arithmetic Elementary Matrices Page 79/90
 Inverses of Elementary Matrices

Example
Without using the matrix inversion algorithm, find the inverse of the elementary
matrix  
1 0 0 0
 0 1 0 0 
G =  −3 0 1 0 


0 0 0 1
Hint. What row operation can be applied to G to transform it to I4 ? The row
operation G → I4 is to add three times row one to row three, and thus
 
1 0 0 0
 0 1 0 0 
G −1 = 
 3 0 1 0 


0 0 0 1

Check by computing G −1 G .

Matrices: Matrix Arithmetic Elementary Matrices Page 80/90

 Example (continued)
Similarly,
 −1  
1 0 0 0 1 0 0 0
 0 0 0 1   0 0 0 1 
E −1 =
 0
 = 
0 1 0   0 0 1 0 
0 1 0 0 0 1 0 0
and  −1  
1 0 0 0 1 0 00
 0 1 0 0   0 1 00 
F −1 =  = 
 0 0 −2 0   0 0 − 12 0 
0 0 0 1 0 0 0 1

Matrices: Matrix Arithmetic Elementary Matrices Page 81/90

 The Form B = UA

Suppose A is an m × n matrix and that B can be obtained from A by a sequence

of k elementary row operations. Then there exist elementary matrices
E1 , E2 , . . . Ek such that

B = Ek (Ek−1 (· · · (E2 (E1 A)) · · · ))

Since matrix multiplication is associative, we have

B = (Ek Ek−1 · · · E2 E1 )A

or, more concisely, B = UA where U = Ek Ek−1 · · · E2 E1 .

To find U so that B = UA, we could find E1 , E2 , . . . , Ek and multiply these

together (in the correct order), but there is an easier method for finding U.

Matrices: Matrix Arithmetic Elementary Matrices Page 82/90

 Definition
Let A be an m × n matrix. We write

A→B

if B can be obtained from A by a sequence of elementary row operations.

Theorem
Suppose A is an m × n matrix and that A → B. Then
1 there exists an invertible m × m matrix U such that B = UA;
 
2 U can be computed by performing
 elementary row operations on A Im
to transform it into B U ;
3 U = Ek Ek−1 · · · E2 E1 , where E1 , E2 , . . . , Ek are elementary matrices
corresponding, in order, to the elementary row operations used to obtain B
from A.

Matrices: Matrix Arithmetic Elementary Matrices Page 83/90

 Problem
 
3 0 1
Let A = , and let R be the reduced row-echelon form of A.
2 −1 0
Find a matrix U so that R = UA.

Solution
     
3 0 1 1 0 1 1 1 1 −1 1 1 1 1 −1
→ →
2 −1 0 0 1 2 −1 0 0 1 0 −3 −2 −2 3
   1 1 
1 1 1 1 −1 1 0 3 3 0
→ →
0 1 23 23 −1 0 1 2
3
2
3 −1
   
Starting with A I , we’ve obtained R U .
Therefore R = UA, where  1 
0
U = 32
3 −1

Matrices: Matrix Arithmetic Elementary Matrices Page 84/90

 A Matrix as a Product of Elementary Matrices
Example
Let  
1 2 −4
A =  −3 −6 13 
0 −1 2
Suppose we do row operations to put A in reduced row-echelon form, and write
down the corresponding elementary matrices.
     
1 2 −4 1 2 −4 1 2 −4
 −3 −6 13  E1  0 −→ −→ −→
0 1  E2  0 −1 2  E3
0 −1 2 0 −1 2 0 0 1
     
1 2 −4 1 0 0 1 0 0
−→ −→
 0 1 −2  E4
 0 1 −2  E5
 0 1 0 
0 0 1 0 0 1 0 0 1
Notice that the reduced row-echelon form of A equals I3 . Now find the matrices
E1 , E2 , E3 , E4 and E5 .

Matrices: Matrix Arithmetic Elementary Matrices Page 85/90

 Example (continued)
     
1 0 0 1 0 0 1 0 0
E1 =  3 1 0  , E2 =  0 0 1  , E3 =  0 −1 0 
0 0 1 0 1 0 0 0 1
   
1 −2 0 1 0 0
E4 =  0 1 0  , E5 = 0 1 2 
0 0 1 0 0 1
It follows that

(E5 (E4 (E3 (E2 (E1 A))))) = I

(E5 E4 E3 E2 E1 )A = I

and therefore
A−1 = E5 E4 E3 E2 E1

Matrices: Matrix Arithmetic Elementary Matrices Page 86/90

 Example (continued)
Since A−1 = E5 E4 E3 E2 E1 ,

A−1 = E5 E4 E3 E2 E1
−1 −1
(A ) = (E5 E4 E3 E2 E1 )−1
A = E1−1 E2−1 E3−1 E4−1 E5−1

This example illustrates the following result.

Theorem
Let A be an n × n matrix. Then, A−1 exists if and only if A can be written as the
product of elementary matrices.

Matrices: Matrix Arithmetic Elementary Matrices Page 87/90

 Problem
 
4 1
Express A = as a product of elementary matrices.
−3 2

Solution
         
4 1 −→ 1 3 −→ 1 3 −→ 1 3 −→ 1 0
E1 E2 E3 E4
−3 2 −3 2 0 11 0 1 0 1

and
       
1 1 1 0 1 0 1 −3
E1 = , E2 = , E3 = 1 , E4 =
0 1 3 1 0 11
0 1

Since E4 E3 E2 E1 A = I , A−1 = E4 E3 E2 E1 , and hence

A = E1−1 E2−1 E3−1 E4−1

Matrices: Matrix Arithmetic Elementary Matrices Page 88/90

 Solution (continued)
Therefore,
 −1  −1  −1  −1
1 1 1 0 1 0 1 −3
A= 1
0 1 3 1 0 11
0 1

i.e.,     
1 −1 1 0 1 0 1 3
A=
0 1 −3 1 0 11 0 1
Check your work by computing the product.

Matrices: Matrix Arithmetic Elementary Matrices Page 89/90

 Problem
Is the n × n identity matrix an elementary matrix? Justify your answer.

One result that we have assumed in all our work involving reduced row-echelon
matrices is the following.

Theorem
If A is an m × n matrix and R and S are reduced row-echelon forms of A, then
R = S.

This theorem ensures that the reduced row-echelon form of a matrix is unique,
and its proof follows from the results about elementary matrices.

Matrices: Matrix Arithmetic Elementary Matrices Page 90/90