Vector Spaces

Axioms of a Vector Space

Definition : Let V be a non empty set of vectors with 2 operations :
i. Vector addition : u, v є V  u + v є V
c osu e
closure
ii Scalar
ii. S l multiplication:
lti li ti u є V  k u є V where h k iis scalar.
l Th Then, V
is called a Vector Space if the following 8 axioms hold for any
u, v, w є V

1) (u  v )  w  u  ( v  w ) associativ e
2) u  0  0  u  u zero vector
3) u  ( u)  u  u  0 additive inverse
4) u  v  v  u commutativ e
5) k (u  v )  ku  kv k is scalar
6) ( a  b)u  au  bu a, b scalars
7) ( abb )u  a (bu )
8) 1u  u too many conditions to check !!
 Subspaces
Definition : Let V be a vector space and let W be a subset of V. Then, W
is a subspace of V if W itself is a vector space
Theorem : Suppose W is a subset of V. Then, W is a subspace if
a) The zero vector 0 belongs to W We don’t need to verify that the 8
axioms of a vector space hold !
b) For every u, v є W  (au + bv) є W
Remark : Two trivial subspaces of V are { 0 } and V itself
Example : Consider V  R 3 . Let U consist of all vectors in R 3 with
equal entries, i e U  {(a,
entries i.e. b c) : a  b  c}
{(a b,
u  (a, a, a) u   v  (a   b)(1,1,1)
(i) 0  U (ii) 
v  (b, b, b) U
U is a subspace
“Abstract”
Abstract Subspaces Examples
Examples : (prove by showing that the 2 conditions of a subspace hold)

Pn(t): polynomials of degree less than or equal to n. Note that it is no longer

a subspace
b if we lilimit
it th
the d
degree off th
the polynomial
l i l tto n ; why
h ?

Rn : n-tuple
n tuple (vector) of real numbers (a1, a2, ……, an)

Mm,n : m x n real matrices

F : all real functions f(x).

( ) What about continuous or integrable
g or
differentiable functions ?
 More Subspaces Examples
Example : Let V = Mn,n. Let W1 be a subset of all upper (or lower) triangular
matrices
ti
 W1 is a subspace
Let W2 be the subset of all symmetric matrices (all
(all-zero
zero matrix is
symmetric, linear combination of symmetric matrices is symmetric)
W2 is subspace
Example : Subset of all polynomials of even-power is subspace of the
space
p of n-th degree
g p
polynomials
y
Example : Subset of all continuous real functions and subset of all
differentiable real functions are subspaces of the space of all real functions
 Span
Definition : Let V be a vector space. Vectors u1, u2, ….un in V span V if every
vector v in V is a linear combination of {u1, u2, ….un}, i.e.
v = a1u1 + a2u2 + …. + anun
Remarks :
Find a span for X-Y plane.
Is it unique ?
1) Suppose {u1, u2, ….un} span V. Then, for any vector w, (assign zero
weight to w) the set {w, u1, u2, …., un} also spans V
2) Suppose {u1, u2, …., un} span V and suppose that uk is a linear
combination of some of the other u’s. Then,the u’s without uk also span V

Example : Verify that the vectors (1,1) and (-1,-1) don’t span R2 while the
vectors (1,1) and (1,0) do
 Span Examples
1) Consider t he vector space R 3
a) e1  1 0 0 , e2  0 1 0 , e3  0 0 1 span R 3
( a , b , c )  v  ae1  be 2  ce 3

b) w1  1 1 1, w2  1 1 0 , w3  1 0 0  also span R 3

v  ( a , b , c )  cw1  (b  c ) w2  ( a  b ) w3

c) However , u1  [1, 2, 3], u 2  [1, 3, 5], u 3  [1, 5, 9 ] does not span R 3

(show
h th t any vector
that t b in t b1  b 3  2b 2 can'' t be
i R 3 s.t. b written
itt as
linear combinatio n of u 1 , u 2 , u 3 )
 More Span
p Examplesp for
“Abstract” Subspaces
2) Consider Pn ( t) : all polynomial s of degree  n
Every
yppolynomial
y can be expressed
p as a linear combinatio n of
(expand about "0" ) {1, t, t 2 , ....., t n }

3) Consider v ector space M 2,2, spanned by the 4 matrices

 1 0   0 1   0 0   0 0  
  ,   ,  ,  
  0 0   0 0  1 0   0 1  
 Intersections & Unions of Subspaces
p

u 0
Vector Space
su+tv
su v tv

Theorem : The intersection of any number of subspaces of a vector V is

a subspace of V. Question : what about their union ? not a subspace!
Gi a counter
Give t example,
l (hint
(hi t : consider
id the
th x and
d y axes))
Proof:
Let U & W be subspaces of V. Clearly 0  U, 0  W  0  U  W
Now, suppose u and v  U  W then u, v  U and u, v  W.
Furthermor e,
au  bv  U and au  bv  W (since U and W are subspaces)
 au  bv  U  W  U  W is a subspace of V
 The Null Space
Theorem : The solution set W of a homogenous
g system
y Ax  0
in n unknowns is a subspace of R n called null space of A.
Proof : Let W be the solution set.
A0  0  0  W suppose u, v  W
 Au  0 and Av  0  A(au  bv)  aAu  bAv  0
 au  bv  W
Remark : Solution set of non - homogenous set Ax  b is not
a subspace since 0 doesn' t belong to it.
Connection to Linear Equations
Amxn matrix
rows : “m” n-dimensional vectors
columns : “n” m-dimensional vectors
Systems of Equations Ax=b can be thought of geometrically in as
1) The “A” matrix transforms vector “x” to vector “b”
2) The vector “b” is a linear combination of columns of “A” (lies in a
vector space called the column space of A)A). So,
So what is a vector
space ? x 
1
x 
v 1 v 2  v n  2   b
  
 
 xn 
 x1 v 1  x 2 v 2    x n v n  b
Row
o aand
d Co
Column
u Space o
of a Matrix
at
Let A be m x n matrix. The rows (columns) of A may be viewed as
vectors in Rn (Rm); hence they span a subspace of Rn (Rm) called the
row space (column space) of A
Linear combination R(A)  span (R1 , R 2 , ...... , R m )

of rows (columns) of A C(A)  span(C1 , C 2 , ...... , C n ) 

Example : Describe the column space of :

1 0 0 Exercise : describe the row and null
(i) 0 1 0  R 3 spaces off these
th 2 matrices
ti as wellll

0 0 1
1 2 - 1 1
(ii)    C ( A)     line
3 6 - 3 3
Connection to Linear Systems of Equations

Definition : the column space of A is the

subspace of spanned by columns of A

The systems
Th t off lilinear equations
ti A
Ax=b
bhhas
1) No solution if and only if b is not in the
span of the column space of A
2)) Unique
q solution if b is in the span p of
column space of A in a unique way
3) Infinite solutions if b is in the span of the
column space of A in infinite ways
 Linear Dependence
p & Independence
p
Definition : We say that the vectors v1 , v2 ,......, vn in V are linearly
dependent if  scalars a1 , a2 ,......, an , not all zero, such that
a1v1  a2 v2  ......  an vn  0
Otherwise the vectors are linearly independen t, i.e.
Relation to null
n
space of a matrix
a v
i 1
i i  0 implies ai  0  i  1, ......, n

Example : Is the set of vectors {1, t, t 2 ,1  t  t 2  t 3 }

linearly independen t ?
solution : x 1 (1)  x2 (t )  x3 (t 2 )  x4 (1  t  t 2  t 3 )  0
By equating coefficients for powers of (t) :
x4  x3  x2  x1  0; hence set is linearly independen t.
 Examples on Linear Independence

1) Let u  (1,1,0), v  (1,3,2), w  (4,9,5)

Since 3u  5v  2 w  0  linearly dependent

2) u  (1,2,3), v  (2,5,7), w  (1,3,5) linearly independent

1  2 1 0 x  2y  z  0
x 2  y 5  z 3  0  2 x  5 y  3z  0
3 7  5 0 3 x  7 y  5 z  0
x  2 y  z  0
 
y  z  0  x  y  z  0
2 z  0 
Linear dependence in R3 (geometric view)
v
a) Any 2 vectors in R 3 are linearly dependent u
if and only if they lie on the same line through origin
0
b) Any 3 vectors in R are linearly dependent
3

if and only if they lie on the same plane

Theorem : Suppose 2 or more non - zero vectors v1, v 2 ,  .., v m

are linearly dependent. Then, one of the vectors is a linear combinatio n
of the preceding vectors, i e  k  1 s.t
vectors i.e. st
vk  c1v1  c2 v2  ......  ck 1vk 1
Theorem : The non - zero rows of a matrix in echelon form are linearly
independen t
 Basis and Dimension
Definition : A set S  u1 , u 2 ,....., u n  of vectors is a basis
ffor V ( vector
t space)) if
(i) S spans V (ii) S is linearly independen t
Exercise : give example of sets of vectors that satisfy only one of these 2 conditions
Theorem : Let V be a vector space such that one basis has
" m" elements and another basis has " n" elements, then m  n
and we write dim( V )  n.
The basis of a vector space is not unique
but the number of vectors in the basis
i unique
is i
Notes :
1) The vector space 0 has dimension 0 by definition prove (2)
by contradiction !
2) For
F a given
i b i every vectt or can be
basis, b written
itt in
i
only one way as a linear combinatio n of the basis vectors
 Basis and Dimension Examples
1) R n : The unit vecto rs e1 , e 2 ,......, e n  are linearly independen t
& span R n  basis for R n , dim( R n )  n . This is called the standard
basis. Find other bases.
2 ) M m x n : The mn matrices E ij (has 1 in (i, j) position and zeros
else where for 1  i  m & 1  j  n) form a basis of M m x n
 dim(
di ( M m x n )  mn
 
3 ) Pn ( t ) : 1, t , t 2 ,....., t n is a basis  dim  n  1
4) The dimension of the solution space W of the homogeneou s
system AX  0 (i.e. null space of A) is (n - r) which is
the number of “free variables ”,, where n is the number of unknowns
and “r” is the rank of A (number of pivots)
 Theorem

Lett V be
L b a vector
t space where di (V )  n. Then,
h dim( Th
(i) Any (n  1) or more vectors in V are linearly dependent.
For example, (n  1) vectors in R n are linearly dependent
(ii) Any linearly independent set S  u1 , u 2 ,......, u n  with n
elements is a basis of V
(iii) Any spanning set T  v1 , v2 ,......, vn  of V with n elements
is a basis of V
(iv) Any set of (n - 1) or less vectors in V does not span V
Example : Let V  R 3  dim(V )  3. Let W be a subspace of V .
Th di (W )  3.
Then, dim( 3
a) dim(W)  φ  W  0 (a point)
di (W)  1  W is
b) dim i a line h origin
li through
th i i
c) dim(W)  2  W is a plane through origin
di (W)  3  W  V
d) dim
Example :
1) Dim ( M n,n )  n 2
n2  n n2  n
2) Dim (upper triangular )  n
2 2
n2  n n2  n
3) Dim ( symmetric matrices)  n
2 2
4) Dim (diagonal)  n
 Four Subspaces of A m x n
1) Row space R(A) or C(A T )  R n
2)) Column space
p C(A) ( T )  Rm
( ) or R(A
3) Null space N(A)  R n
4)) Left Null space
p ( T )  Rm
N(A
Facts : 1) dim (R(A))  dim (C(A))  rank " r "
2) dim (N(A))  n - r
A  AT
n  m  3) dim (N(A T ))  m- r
mn Fundamental Theorem of Linear Algebra, Part I

4) dim (R(A))  dim (N(A))  dim ( R n )  n

5) dim (C(A))  dim (N(A T ))  dim ( R m )  m

 Row and Column Subspaces
1) The columns of mxn matrix A are linearly independent when the rank r = n.
There are “n” pivots and no free variables. Only x = 0 is in the null space
2) The vectors v1, v2, ….., vn are a basis for Rn exactly when they are the
columns of an n x n invertible matrix. Hence, Rn has infinitely many
diff
different
t bases
b ((allll with
ith th
the same di
dimension
i “n”)
“ ”)
3) The pivot columns of A are a basis for its column space C(A). The pivot
rows of A ((or its echelon form)) are a basis for R(A)
( )

Note : Ax  0  Ek ......E2 E1 Ax  0  Ux  0
Therefore, null spaces of A and U are same
Therefore
4) The maximum number of linearly independent rows
of A is equal to the maximum number of linearly independent
columns of A. Hence, dim( R(A))  dim(C(A))  rank ( A)
 Computing Bases for the Row and Column Spaces

Procedure 1(use to find a basis for the row space of A)

i Form the matrix M whose rows are the given vectors
i.
ii. Row reduce M to echelon form
iii. The basis is given by non
non-zero
zero rows of echelon matrix
(or the corresponding rows in the original matrix)
Procedure 2 ((use to find a basis for the column space)
p )
i. Form matrix M whose columns are the given vectors
ii. Row reduce M to echelon form
iii. For each column Ck in the echelon matrix without pivot, delete
vector uk from the given list S
i Basis = remaining vectors
iv. ectors in S
 Example 1
Let W be a subspace of R 5 spanned by :
u1  (1, 2,1, 3, 2 ) , u 2  (1, 3, 3, 5, 3)
u 3  (3, 8, 7 ,13, 8) , u 4  (1, 4, 6, 9, 7 )
u 5  (5,13,13, 25 ,19 )
Find a basis for W from the set of S  u1 , u 2 ,....., u 5 
(in other word s which u's are independen t)
1 1 3 1 5  1 1 3 1 5
 
2 3 8 4 13   0 1 2 2 3 
Sol : M  1 3 7 6 13  ~  0 0 0 1 2
   
3 5 13 9 25   0 0 0 0 0
2 3 8 7 19   0 0 0 0 0 

delete u 3 & u 5  basis for W  u1 ,u 2 ,u 4  basis for column
space of M
dim( W )  3
 Example 2
1 2 1 2  1
3 1 2 1 3 1 2
2 5 5 5  0
6 4 1 3 1 2 1 

A  3 7 6 11 6 9  ~ 0 0 0 1 1 2  B
   
1 5 10 8 9 9  0 0 0 0 0 0
2 6 8 11 9 12 0 0 0 0 0 0
Gaussian elimination applies elementary
operations (linear combinations) on rows of A;
a ) Find a basis for R(A) Hence row subspace stays the same
Hence,
B and A have same row space . Non zero rows of B are linearly
independent  form a basis for R(A)
 1 2 1 3 1 2, 0 1 3 1 2 1, 0 0 0 1 1 2 
 Example 2 (Cont’d)
b) Find a basis for C(A)
C l
Columns with
ith pivots
i t C1 , C 2 , C 4 are linearly
li l independen
i d d t
 form basis for C(A)
 12 3 1 2, 2 5 7 5 6, 3 6 11 8 11 
Columns of A that contain pivots in echelon form of A

c ) Find rank( A)
(i) 3 pivots  rank( A)  3
( ))  3  rank
((ii)) dim ( R(A)
(iii) dim(Col(A))  3  rank
 Example 3
1) Find the rank and a basis for the row space and column space of
1 2 0  1
 2 6  3  3
 
 3 10  6  5 
Solution : Reduce A to echelon form
1 2 0  1  1 2 0  1
A ~  0 2  3  1  ~  0 2  3  1
 0 4  6  2   0 0 0 0 
- The 2 non - zero rows of the echelon form of A ( or first 2 rows
of A itself ! ) form a basis for its R(A) and rank( A )  2
- Pivots
i are in
i columns
l 1 & 2. Hence, a basis
b i for
f C(A)( ) is
i
given by first 2 columns of A (not its echelon form ! )
Example 4
Repeat previous problem for this matrix
1 3 1 2  3
1 4 3 1  4 
A 
2 3  4 7  3
 
3 8 1  7  8
1 3 1  2  3 1 3 1 2  3
0 1 2 1  1   0 1 2 1  1 
~  ~ U
0 3 6 3 3  0 0 0 0 0 
   
0 1  2 1 1  0 0 0 0 0 

r  2 , first 2 rows of U or A are basis for R(A) ;

first 2 columns of A are basis for C(A) ;
dim( N (A))  n  r  5  2  3
Find a basis for N(A)
free variables are x 3 , x 4 , x 5 
 Complete Solution to Ax=b
Example 3 : Find Complete Solution to
 x1 
1 1 2 3    1
2 2 8 10  x2   4
  x   
3 3 10 13  3  5
 x4 
1 1 2 3 1
Step 1 : 
Put in echelon form : U  0 0 4 4 2 
0 0 0 0 0
Step 2 : Determine pivot variables : x1, x3
Determine free variables : x2 , x4
Step 3 : Write pivot variables in terms of free variables
x3  0.5  x4 ; x1   x2  x4

 x1   1  1  0 
x  1 0  0 
Step 4 : xcomplete   2   x2    x4     
 x3  0  1 0.5
       
 x4  0 1  0 
 xnull  x p

Two basis vectors for null space of A

 Complete Solution to Ax = b
xcomplete  x particular  xnull


p n

Axcomplete  Ax p  Axn  Ax p  b

Procedure for computing the complete solution to Ax=b

1) Put augmented matrix A b in echelon form

2) Determine pivot and free variables
3) Express the pivot variables as linear combinations
of the free variables((this is xn ) pplus a constant vector
( this is x p )
 The 4 Fundamental Subspaces
Fundamental Theorem of Linear Algebra (Part II)
Consider an m x n matrix A with rank r. The nullspace is the orthogonal
complement (will be defined in Ch. 4) of the row space (in Rn). The left
null space is the orthogonal complement of the column space (in Rm).

dim = r
AXp = b dim = r
row space column space

Rn X = Xp + Xn Rm
AXn = 0
left null space
null space
dim = n - r dim = m - r
 Finding Basis for N(A)
E
Example
l : Find
Fi d basis
b i and
d dimension
di i off N(A) for
f :
1 2 2  1 3 1 2 2 1 3 
(i) A  1 2 3 1 1  ~ 
0 0 1 2  2 
3 6 8 1 5   0 0 0 0 0 
r  2, nr  52  3 free variables
x 2 , x 4 , x5   dim( N ( A))  3
x3   2 x 4  2 x5 ; x1   2 x 2  2 x3  x 4  3 x5   2 x 2  5 x 4  7 x5
 x1   2  5   7 
x   1   0   0 
 2      
x null   x3   x 2  0   x 4   2   x5  2 
       
x
 4  0   1   0 
 x5   0   0   1 
ii) Write down the complete solution for Ax = b for b =

x3  2 x4  2 x5 ; x1  3  2 x2  2 x3  x4  3 x5  3  2 x2  5 x4  7 x5
 x1    2 5   7   3
x  1 0  0  0 
 2        
xcomplete   x3   x2  0   x4  2  x5  2   0
         
 x4  0 1  0  0 
 x5   0   0   1  0
Find the basis and dimension of the left null space  N(A T ) of
 1 2 3 
 2 4 6  x1  2 x 2  x3  2 x 4  0
A  
 1 4 7  2 x3  x 4  0
 
  2  3  4 
1 2 1  2  1 2 1  2 
A T   2 4 4  3  ~  0 0 2 1 
 3 6 7  4   0 0 0 0 
r  2  dim( N ( A T ))  4  2  2  2 free variables x 2 , x 4 
x3   0 .5 x 4 ; x1   2 x 2  x3  2 x 4   2 x 2  2 .5 x 4
 x1   2  2 .5 
x   1   0 
xleft  null   2   x2    x4  
 x3   0    0 .5 
     
x
 4  
0  1 
 Linear Transformations
• Definition : A transformation T is linear iff :
i) T(u+v)=T(u)+T(v)
T(u+v) T(u)+T(v)
ii) T(cv)=cT(v)
where c is a scalar and u,v
, are vectors

• Definition : Let A be an m x n matrix. The function

T(x)=y=Ax is called a matrix transformation from
n m
R to R
n
• Fact : Every matrix transformation on R is linear
(prove it!)
 Linear Transformation
• How to determine its matrix representation ?
u1  Tv 1 ; u 2  Tv 2 ;.....;
; ; u n  Tv n
 u1 u2 u3 u 4   Tv 1 v2 v3 v4 
 U  TV  T  UV 1
• For V to be invertible, its “n” columns must be
li
linearly
l iindependent
d d t (i.e.
(i fform a basis)
b i )
• Idea : if you know result of applying T(.) to
b i vectors,
basis t you kknow resultlt off applying
l i T(T(.))
to any vector ! But which basis to choose ?
• To
T avoid id matrix
t i iinversion,
i V = identity
id tit matrix
ti !
 Representation
p Theorem
• Let T be a linear transformation from
n m
R to R . The m x n matrix
A  T (e1 ) T (e 2 )  T (e n )

(where ei is the ith unit vector) has the property

that T(x)=Ax and A is unique.
Applying successive transformations T1 ( x), T2 ( x),....
corresponds to successive matrix multiplications
....A 2 A1 x
 Examples
• Rotation by angle  counter clockwise
cos( )  sin( )
A 
 sin( ) cos( ) 

• Reflection about origin (mirror image)

 1 0 
A 
 0  1
• Projection on xx-axis
axis (drop a perpendicular)
Not invertible ! Will
be studied in detail in 1 0
Chapter 4 A 
0 0 