LECTURE 12 RECAP

1) A theorem on the usefulness of knowing the dimension

of a vector space.
2) Two more equivalent statements to "A is an invertible
square matrix of order n."
3) Transition matrices: how to find and how to use.

4) Transition matrices are invertible and their inverses

are also transition matrices.
 LECTURE 13
ROW SPACES AND COLUMN SPACES
 DEFINITION
Given any m  n matrix A,
n

 a11 a12 ... a1n  r1   a11 a12 ... a1n 

a 
a ... a r2   a21 a22 ... a2 n 
A   21 22 2n

 
a  rm   am1 am 2 ... amn 
 m1 m 2
a ... amn 

n
The rows of A can be considered as vectors in .
 span{r1 , r2 ,..., rm } is a subspace of n
,
This subspace is called the row space of A.
 DEFINITION
Given any m  n matrix A,

 a11 a12 ... a1n  The columns of A can be

a a ... a  m
considered as vectors in .
A   21 22 2n
 m
   span{c1 , c2 ,..., cn } is
a 
 m1 m 2
a ... amn 
a subspace of m
,
 a11   a12   a1n 
a  a   a  This subspace is called
c1   21  c2   22  cn   2 n  the column space of A.
     
a  a  a 
 m1   m2   mn 
 REMARK
1 0 0
0 1 0
 1 0 1 1 4   
A  0 1 4 2 1 A   1 4 2 
T

  1
 0 0 2 0 1  2 0
   
4 1 1 


The row space of A is the column space of AT

The column space of A is the row space of AT

 EXAMPLE
 2 1 0
 1 1 3
A 
 5 1 0
1 0 
 1
3
The row space of A is a subspace of .
4
The column space of A is a subspace of .

Note that if A is not a square matrix, then the row space

and column space of A contains totally 'different type'
of vectors.
 EXAMPLE
 2 1 0 r1
 1 1 3 r2 We write r1  (2, 1, 0) (as a vector)
A 
 5 1 0 r3 rather than a row matrix  2 1 0 .
1 0 1 
 r4
3
The row space of A is a subspace of .
 span{ r1 , r2 , r3 , r4 }
 {a(2,1, 0)  b(1, 1, 3)  c(5,1, 0)  d (1, 0,1)| a , b, c, d  }
 {(2a  b  5c  d , a  b  c , 3b  d )|a ,b , c , d  }
 EXAMPLE
 2 1 0
 1 1 3
A 
 5 1 0
1 0 1 

c1 c2 c3
4
The column space of A is a subspace of .
 span{c1 , c2 , c3 }
  2   1  0    2a  b  
  1   1  3    a  b  3c  
         | a , b , c  
 a b c |a ,b, c   
  5   1   0    5a  b  
  1   0   1    a  c  
 NOTATION
n
We have observed that a vector in can be identified
as a row or matrix.
Henceforth, when we write Identified with:
(u1 , u2 ,..., un ) row vector  u1 u2 ... un 
 u1 
u 
(u1 , u2 ,..., un )T column vector  2
 : 
u 
 n
 EXAMPLE
 1 0 1 1 4 
A  0 1 4 2 1 How to find a basis for the
 
 0 0 2 0 1 
  row space or column space of A?
5
The row space of A is a subspace of .
3
The column space of A is a subspace of .
 EXAMPLE
 1 0 1 1 4 
A  0 1 4 2 1 How to find a basis for the
 
 0 0 2 0 1 
  row space or column space of A?

row space of A  span{(1, 0, 1,1, 4),(0,1, 4, 2,1),(0, 0, 2, 0,1)}

If (1, 0, 1,1, 4),(0,1, 4, 2,1),(0, 0, 2, 0,1) (that is, the rows of A)
are linearly independent, then obviously they will form
a basis for the row space of A.
 EXAMPLE
 1 0 1 1 4 
A  0 1 4 2 1 How to find a basis for the
 
 0 0 2 0 1 
  row space or column space of A?

row space of A  span{(1, 0, 1,1, 4),(0,1, 4, 2,1),(0, 0, 2, 0,1)}

(0, 0, 0, 0, 0)  a(1, 0, 1,1, 4)  b(0,1, 4, 2,1)  c(0, 0, 2, 0,1)

 a  0 ,b  0 ,c  0
So the three rows of A are linearly independent and
thus form a basis for the row space of A.
 EXAMPLE
 1 0 1 1 4  Note that A is in row echelon form.
A  0 1 4 2 1
  What if we want to find a basis
 0 0 2 0 1 
  for the row space of a matrix R
that is in row echelon form?
  * *
 non zero
 * *
  rows
 *
R  0 
 * *
 
 0 ... ... ... 0 0 zero rows
  (if any)
 
 EXAMPLE
 1 0 1 1 4  Note that A is in row echelon form.
A  0 1 4 2 1 The non zero rows of R are always
 
 0 0 2 0 1 
  linearly independent and thus forms
a basis for the row space of R.
  * *
 non zero
 * *
  rows
 *
R  0 
 * *
 
 0 ... ... ... 0 0 zero rows
  (if any)
 
 EXAMPLE
 1 0 1 1 4 
A  0 1 4 2 1 How to find a basis for the
 
 0 0 2 0 1 
  row space or column space of A?
 1   0   1   1   4  
          
column space of A  span  0 , 1 , 4 , 2 , 1 
     
 0   0   2   0   1  
          
column space of A is a subspace of 3 ,
 the dimension of this subspace is at most 3
So if we can identify 3 linearly independent vectors
(out of the 5) from the set above...
 EXAMPLE
 1 0 1 1 4 
A  0 1 4 2 1 How to find a basis for the
 
 0 0 2 0 1 
  row space or column space of A?
 1   0   1   1   4 
column space of A  span{ 0  ,  1  ,  4  ,  2  ,  1 }
     
 0   0   2   0   1 
     
3
linearly
column space of A is a subspace of ,
independent
 the dimension of this subspace is at most 3
3
Column space of A is the entire .
 DISCUSSION
 1 0 1 1 4 
A  0 1 4 2 1 How to find a basis for the
 
 0 0 2 0 1 
  row space or column space of A?
That was based on Yes, you are right.
observations... A more systematic
approach is needed.
 DISCUSSION
Let S be the set of all matrices of the same size (say m  n).
Recall the definition of row equivalent.
Series of e.r.o A and B are
A B
row equivalent

Row equivalence is an equivalence relation on S .

(Reflexive) A is row equivalent to A
(Symmetric) If A B , then B A
(Transitive) If A B and B C then A C
 DISCUSSION
Let S be the set of all matrices of the same size (say m  n).
Recall the definition of row equivalent.
Series of e.r.o A and B are
A B
row equivalent

Two matrices A and B (of the same size) are row equivalent
if and only if they have a similar row-echelon form (or they
have the same unique reduced row-echelon form).
 THEOREM

Let A and B be row equivalent matrices. Then the row

space of A and the row space of B are identical.
Series of e.r.o
A B

same row space

That is to say, performing elementary row operations

on A does not change its row space.
 EXAMPLE

0 0 1 R  R  12 1 2 
0 2 4 1 3
0 2 4
    span{(0, 0,1),(0, 2, 4),( 12 ,1, 2)}
 1 1 2 0 0 1
2   
 span{(12 ,1, 2),(0, 2, 4),(0, 0,1)}
2R1
 span{(1, 2, 4),(0, 2, 4),(0, 0,1)}
 1 0 0  R1  R2 1 2 4
0 2 4 0 2 4  span{(1, 0, 0),(0, 2, 4),(0, 0,1)}
   
0 0 1 0 0 1
   
 BACK TO THIS QUESTION
Question : How to find a basis for the row space of a matrix A?

  * * The non zero rows of R are always

  * *
  linearly independent and thus forms
 *
R  0 a basis for the row space of R.
 * *
 
 0 ... ... ... 0 0 Answer :
  Find a row-echelon form R of A.
 
A basis for the row space of R is also a basis for the
row space of A.
Let A and B be row equivalent matrices. Then the row
space of A and the row space of B are identical.
 EXAMPLE
Find a basis for the row space of the following matrix.
 2 2 1 0 1   2 2 1 0 1
 1 1 2 3 1  Gaussian 0 0 3 3 32 
A   2

 0 0 1 1 1  Elimination  0 0 0 3 0
 1 1 2 0 1 0 0 0 0 0 
  
Performing Gaussian Elimination on A :
A basis for the row space of A is
{(2, 2, 1, 0,1),(0, 0, 32 , 3, 32),(0, 0, 0, 3, 0)}
FINDING BASIS FOR COLUMN SPACES
 1 0 1 1 4  1 0 0
0 1 0
A   0 1 4 2 1  column space of A  
 
 0 0 2 0 1 
  row space of A A   1 4 2 
T

T

1 2 0
Ok, what about  
4 1 1
column spaces?  
Note the
relationship
between column
space of A and
row space of AT ...
FINDING BASIS FOR COLUMN SPACES
 1 0 1 1 4  1 0 0
0 1 0
A   0 1 4 2 1  column space of A  
 
 0 0 2 0 1 
  row space of A A   1 4 2 
T

T

1 2 0
 
4 1 1 

So to find a basis for the column space of A, we can
use the previous method to find a basis for the row
space of AT .
In what follows, we will discuss another method.
 IMPORTANT TO NOTE

Elementary row operations preserve the row space of a

matrix but NOT the column space.
1 0 R1  R2 0 0
A  1 0  B
 0 0   
column space of A column space of B
1 y 0
 span{ }  span{ }
0 1
 x  axis  y  axis
x
 THEOREM
Let A and B be row equivalent matrices. Then the following
statements hold:
* * * * * * * * * *
* * * * *  Series of * * * * *
   
A: : : : : e.r.o : : : : :B
: : : : : : : : : :
   
* *  * * 
 * * *  * * *

A given set of columns of A is linearly independent

if and only if the corresponding columns of B is linearly
independent.
 THEOREM
Let A and B be row equivalent matrices. Then the following
statements hold:
* * * * * * * * * *
* * * * *  Series of  * * * * *
 Remember: Column space of A Column space of B 
A: : : : : e.r.o  : : : : :B
: : : : : : : : : :
   
* *  * * 
 * * *   * * * 
A given set of columns of A forms a basis for the column
space of A if and only if the corresponding columns of B
forms a basis for the column space of B.
 OBSERVATION
If R is a matrix in row echelon form,
  * *
the pivot columns of R always form   * *

a basis for the column space of R. R   0 *
 * *
 
Question : How to find a basis for the  0 ... ... ... 0 0
 
column space of a matrix A?  

Let A and B be row equivalent matrices.

A given set of columns of A forms a basis for the column
space of A if and only if the corresponding columns of B
forms a basis for the column space of B.
FINDING A BASIS FOR COLUMN SPACE
If R is a matrix in row echelon form,
  * *
the pivot columns of R always form   * *

a basis for the column space of R. R   0 *
 * *
 
Question : How to find a basis for the  0 ... ... ... 0 0
 
column space of a matrix A?  
Answer
Let R be a row echelon Remember NOT to take the
form of A. columns of R as your answer!
A basis for the column space of A can be obtained by taking
the columns of A that correspond to the pivot columns in R.
 EXAMPLE
Find a basis for the column space of the following matrix.
 2 2 1 0 1   2 2 1 0 1 
 1 1 2 3 1  Gaussian  0 0 3 3 3 
A   2 2

 0 0 1 1 1  Elimination  0 0 0 3 0 
 1 1 2 0 1 0 0 0 0 0
   
Performing Gaussian Elimination on A :
 2   1   0  
 1  2   3  
      
A basis for the column space of A is  , , 
 0   1   1  
 1   2   0  
APPLYING THIS KNOWLEDGE
Let u1  (1, 2, 0, 4), u2  (0,1, 5, 0), u3  (1, 3, 2, 4),
u4  (2,1, 0,8), u5  (3,1, 1,12) and V  span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V . We can construct a
There is no matrix matrix ourselves!
in this question!
Let me show you
two methods...
 METHOD 1
Let u1  (1, 2, 0, 4), u2  (0,1, 5, 0), u3  (1, 3, 2, 4),
u4  (2,1, 0,8), u5  (3,1, 1,12) and V  span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V .
Put the vectors as
same as "Find a basis for the
rows. So V is the
row space of A".
row space of the
1 2 0 4 matrix A.
0 1 5 0
 
Let A   1 3 2 4 .
2 1 0 8
 
3 1 1 12 

 METHOD 1
Let u1  (1, 2, 0, 4), u2  (0,1, 5, 0), u3  (1, 3, 2, 4),
u4  (2,1, 0,8), u5  (3,1, 1,12) and V  span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V .
{(1, 2, 0, 4),(0,1, 5, 0),(0, 0, 23, 0)}
same as "Find a basis for the
is a basis for V .
row space of A".
1 2 0 4 1 2 0 4
0 1 5 0 0 1 5 0
  Gaussian  
A   1 3 2 4  0 0 23 0  R
2 1 0 8
Elimination 0
  0 0 0
 
3 1 1 12  0 0 
  0 0
 METHOD 2
Let u1  (1, 2, 0, 4), u2  (0,1, 5, 0), u3  (1, 3, 2, 4),
u4  (2,1, 0,8), u5  (3,1, 1,12) and V  span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V . Note that Put the vectors as
same as "Find a basis for the
B  AT columns. So V is
column space of B ".
( A from the columns space
 1 0 1 2 3  Method 1) of the matrix B.
2 1 3 1 1 
Let B   .
 0 5 2 0 1
 4 0 4 8 12 
 
 METHOD 2
Let u1  (1, 2, 0, 4), u2  (0,1, 5, 0), u3  (1, 3, 2, 4),
u4  (2,1, 0,8), u5  (3,1, 1,12) and V  span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V .
{(1, 2, 0, 4),(0,1, 5, 0),(1, 3, 2, 4)}
same as "Find a basis for the
is a basis for V .
column space of B ".

1 0 1 2 3 1 0 1 2 3 
2 1 3 1 1  Gaussian 0 1 5 3 5 
B    R
0 5 2 0 1 Elimination 0 0 23 15 24 
4 0 4 8 12  0 0 0 
  0 0
 METHOD 1 VS METHOD 2
Let u1  (1, 2, 0, 4), u2  (0,1, 5, 0), u3  (1, 3, 2, 4),
u4  (2,1, 0,8), u5  (3,1, 1,12) and V  span{ u1 , u2 , u3 , u4 , u5 }.
Find a basis for V .
Method 1 Method 2
{(1, 2, 0, 4),(0,1, 5, 0),(0, 0, 23, 0)} {(1, 2, 0, 4),(0,1, 5, 0),(1, 3, 2, 4)}
is a basis for V . is a basis for V .
{u1 , u2 , u3 } is a basis for V .

The basis found using Method 2 will always be a subset

of { u1 , u2 , u3 , u4 , u5 }.
 EXAMPLE

S  {(1, 4, 2, 5,1),(2, 9, 1,8, 2),(2, 9, 1, 9, 3)}.

1) Show that S is a linearly independent set.

5
2) Extend S to a basis for .
Here we mean add vectors to the set S so that the
5
resulting set becomes a basis for .
How many vectors do we need to add?
COLUMN SPACE AND LINEAR SYSTEMS
 2x  y  1
 x  y  3z  4
  Ax  b where

5 x  y  2
 2 1 0
 x  z  3  1 1 3
A 
 2   1  0   1   5 1 0
 1   1  3   4  1 0
 1 
 x   y   z    
 5   1   0   2   1 
 1   0  1  3   x  4
        x  y b 
   2 
z
   3
 
COLUMN SPACE AND LINEAR SYSTEMS
 2x  y  1
 x  y  3z  4
  Ax  b

5 x  y  2
 x Ax  b is consistent means
 z  3
x , y , z can be found
 2   1  0   1  to satisfy (*)
 1   1  3   4 
 x   y   z      b is a linear combination
 5   1   0   2  of the columns of A.
 1   0  1  3 
        That is, b belongs to
(*)
the column space of A.
COLUMN SPACE AND LINEAR SYSTEMS
 2x  y  1
 x  y  3z  4
  Ax  b

5 x  y  2
 x  Ax  b is consistent
 z  3
 x , y , z can be found
 2   1  0   1  to satisfy (*)
 1   1  3   4 
 x   y   z      b is a linear combination
 5   1   0   2  of the columns of A.
 1   0  1  3 
        b belongs to
(*)
the column space of A.
 THEOREM

Let A be a m  n matrix. Then the column space

of A is
  u1  
 u  
  2 
 A u , u ,..., u     Au u  n

  : 
1 2 n

  un  
A system of linear equations Ax  b is consistent if and
only if b lies in the column space of A.
 END OF LECTURE 14

LECTURE 15: RANKS

NULLSPACE AND NULLITY(END OF CHAPTER 4)