Chapter 3.

Vector spaces
Lecture notes for MA1111

P. Karageorgis

[email protected]

1 / 22
Linear combinations

Suppose that v1 , v2 , . . . , vn and v are vectors in Rm .

Definition 3.1 – Linear combination

We say that v is a linear combination of v1 , v2 , . . . , vn , if there exist
scalars x1 , x2 , . . . , xn such that v = x1 v1 + x2 v2 + . . . + xn vn .

Geometrically, the linear combinations of a nonzero vector form a line.

The linear combinations of two nonzero vectors form a plane, unless
the two vectors are collinear, in which case they form a line.

Theorem 3.2 – Expressing a vector as a linear combination

Let B denote the matrix whose columns are the vectors v1 , v2 , . . . , vn .
Expressing v = x1 v1 + x2 v2 + . . . + xn vn as a linear combination of the
given vectors is then equivalent to solving the linear system Bx = v.
2 / 22
Linear combinations: Example
We express v as a linear combination of v1 , v2 , v3 in the case that
       
1 1 3 3
v1 = 1 , v2 = 2 , v3 = 1 , v = 2 .
2 4 2 4
Let B denote the matrix whose columns are v1 , v2 , v3 and consider
the linear system Bx = v. Using row reduction, we then get
   
1 1 3 3 1 0 5 4
[B v] = 1 2 1 2 −→  0 1 −2 −1 .
2 4 2 4 0 0 0 0
In particular, the system has infinitely many solutions given by
x1 = 4 − 5x3 , x2 = −1 + 2x3 .
Let us merely pick one solution, say, the one with x3 = 0. Then we
get x1 = 4 and x2 = −1, so v = x1 v1 + x2 v2 + x3 v3 = 4v1 − v2 .
3 / 22
Linear independence

Suppose that v1 , v2 , . . . , vn are vectors in Rm .

Definition 3.3 – Linear independence

We say that v1 , v2 , . . . , vn are linearly independent, if none of them is
a linear combination of the others. This is actually equivalent to saying
that x1 v1 + x2 v2 + . . . + xn vn = 0 implies x1 = x2 = . . . = xn = 0.

Theorem 3.4 – Checking linear independence in Rm

The following statements are equivalent for each m × n matrix B.
1 The columns of B are linearly independent.
2 The system Bx = 0 has only the trivial solution x = 0.
3 The reduced row echelon form of B has a pivot in every column.

4 / 22
Linear independence: Example

We check v1 , v2 , v3 , v4 for linear independence in the case that

       
1 1 4 3
v1 = 1 , v2 = 2 , v3 = 5 , v4 = 1 .
2 3 9 2

Letting B be the matrix whose columns are these vectors, we get

   
1 1 4 3 1 0 3 0
B = 1 2 5 1 −→  0 1 1 0 .
2 3 9 2 0 0 0 1

Since the third column does not contain a pivot, we conclude that the
given vectors are not linearly independent.
On the other hand, the 1st, 2nd and 4th columns contain pivots, so
the vectors v1 , v2 , v4 are linearly independent. As for v3 , this can be
expressed as a linear combination of the other three vectors.
5 / 22
Span and complete sets

Suppose that v1 , v2 , . . . , vn are vectors in Rm .

Definition 3.5 – Span and completeness

The set of all linear combinations of v1 , v2 , . . . , vn is called the span
of these vectors and it is denoted by Span{v1 , v2 , . . . , vn }. If the span
is all of Rm , we say that v1 , v2 , . . . , vn form a complete set for Rm .

Theorem 3.6 – Checking completeness in Rm

The following statements are equivalent for each m × n matrix B.
1 The columns of B form a complete set for Rm .
2 The system Bx = y has a solution for every vector y ∈ Rm .
3 The reduced row echelon form of B has a pivot in every row.

6 / 22
Bases of Rm

Definition 3.7 – Basis of Rm

A basis of Rm is a set of linearly independent vectors which form a
complete set for Rm . Every basis of Rm consists of m vectors.

Theorem 3.8 – Basis criterion

The following statements are equivalent for each m × m matrix B.
1 The columns of B form a complete set for Rm .
2 The columns of B are linearly independent.
3 The columns of B form a basis of Rm .
4 The matrix B is invertible.

Theorem 3.9 – Reduction/Extension to a basis

A complete set of vectors can always be reduced to a basis and a set
of linearly independent vectors can always be extended to a basis.
7 / 22
Subspaces of Rm

Definition 3.10 – Subspace of Rm

A subspace V of Rm is a nonempty subset of Rm which is closed under
addition and scalar multiplication. In other words, it satisfies
1 v ∈ V and w ∈ V implies v + w ∈ V ,
2 v ∈ V and c ∈ R implies cv ∈ V .

Every subspace of Rm must contain the zero vector. Moreover, lines

and planes through the origin are easily seen to be subspaces of Rm .

Definition 3.11 – Basis and dimension

A basis of a subspace V is a set of linearly independent vectors whose
span is equal to V . If a subspace has a basis consisting of n vectors,
then every basis of the subspace must consist of n vectors. We usually
refer to n as the dimension of the subspace.
8 / 22
Column space and null space

When it comes to subspaces of Rm , there are three important examples.

1 Span. If we are given some vectors v1 , v2 , . . . , vn in Rm , then their
span is easily seen to be a subspace of Rm .
2 Null space. The null space of a matrix A is the set of all vectors x
such that Ax = 0. It is usually denoted by N (A).
3 Column space. The column space of a matrix A is the span of the
columns of A. It is usually denoted by C(A).

Definition 3.12 – Standard basis

Let ei denote the vector whose ith entry is equal to 1, all other entries
being zero. Then e1 , e2 , . . . , em form a basis of Rm which is known as
the standard basis of Rm . Given any matrix A, we have

Aei = ith column of A.

9 / 22
Finding bases for the column/null space

Theorem 3.13 – A useful characterisation

A vector x is in the column space of a matrix A if and only if x = Ay
for some vector y. It is in the null space of A if and only if Ax = 0.

To find a basis for the column space of a matrix A, we first compute

its reduced row echelon form R. Then the columns of R that contain
pivots form a basis for the column space of R and the corresponding
columns of A form a basis for the column space of A.
The null space of a matrix A is equal to the null space of its reduced
row echelon form R. To find a basis for the latter, we write down the
equations for the system Rx = 0, eliminate the leading variables and
express the solutions of this system in terms of the free variables.
The dimension of the column space is equal to the number of pivots
and the dimension of the null space is equal to the number of free
variables. The sum of these dimensions is the number of columns.
10 / 22
Column space: Example
We find a basis for the column space of the matrix
 
1 2 4 5 4
A = 3 1 7 2 3 .
2 1 5 1 5
The reduced row echelon form of this matrix is given by
 
1 0 2 0 0
R = 0 1 1 0 7 .
0 0 0 1 −2
Since the pivots of R appear in the 1st, 2nd and 4th columns, a basis
for the column space of A is formed by the corresponding columns
     
1 2 5
v1 = 3 , v2 = 1 , v4 = 2 .
2 1 1
11 / 22
Null space: Example, page 1
To find a basis for the null space of a matrix A, one needs to find a
basis for the null space of its reduced row echelon form R. Suppose,
for instance, that the reduced row echelon form is
 
1 0 −2 3
R= .
0 1 −4 1
Its null space consists of the vectors x such that Rx = 0. Writing the
corresponding equations explicitly, we must then solve the system
x1 − 2x3 + 3x4 = 0, x2 − 4x3 + x4 = 0.
Once we now eliminate the leading variables, we conclude that
   
x1 2x3 − 3x4
x2   4x3 − x4 
x= x3  = 
  .
x3 
x4 x4
12 / 22
Null space: Example, page 2

This determines the vectors x which lie in the null space of R. They
can all be expressed in terms of the free variables by writing
     
2x3 − 3x4 2 −3
 4x3 − x4  4 −1
x=  = x3   + x4   = x3 v + x4 w
1  0
 x3 
x4 0 1

for some particular vectors v, w. Since the variables x3 , x4 are both

free, this means that the null space of R is the span of v, w.
To show that v, w form a basis for the null space, we also need to
check linear independence. Suppose that x3 v + x4 w = 0 for some
scalars x3 , x4 . We must then have x = 0 by above. Looking at the
last two entries of x, we conclude that x3 = x4 = 0.

13 / 22
Vector spaces

Definition 3.14 – Vector space

A set V is called a vector space, if it is equipped with the operations of
addition and scalar multiplication in such a way that the usual rules of
arithmetic hold. The elements of V are generally regarded as vectors.

We assume that addition is commutative and associative with a zero

element 0 which satisfies 0 + v = v for all v ∈ V . We also assume
that the distributive laws hold and that 1v = v for all v ∈ V .
The rules of arithmetic are usually either obvious or easy to check.
We call V a real vector space, if the scalars are real numbers. The
scalars could actually belong to any field such as Q, R and C.
A field is equipped with addition and multiplication, it contains two
elements that play the roles of 0 and 1, while every nonzero element
has an inverse. For instance, Q, R and C are fields, but Z is not.
14 / 22
Examples of vector spaces

1 The zero vector space {0} consisting of the zero vector alone.
2 The vector space Rm consisting of all vectors in Rm .
3 The space Mmn of all m × n matrices.
4 The space of all (continuous) functions.
5 The space of all polynomials.
6 The space Pn of all polynomials of degree at most n.
.......................................................................
The set of all matrices is not a vector space.
The set of polynomials of degree n is not a vector space.
Concepts such as linear combination, span and subspace are defined
in terms of vector addition and scalar multiplication, so one may
naturally extend these concepts to any vector space.

15 / 22
Vector space concepts

P
Linear combination: v = xi vi for some scalars xi .
Span: The set of all linear combinations of some vectors.
Complete set for V : A set of vectors whose span is equal to V .
Subspace: A nonempty subset which is closed under addition and
scalar multiplication. For instance, the span is always a subspace.
P
Linearly independent: xi vi = 0 implies that xi = 0 for all i.
Basis of V : A set of linearly independent vectors that span V .
Basis of a subspace W : Linearly independent vectors that span W .
Dimension of a subspace: The number of vectors in a basis.
..................................................................
The zero element 0 could stand for a vector, a matrix or a polynomial.
In these cases, we require that all entries/coefficients are zero.

16 / 22
Example: Linear combinations in P2

We express f = 2x2 + 4x − 5 as a linear combination of

f1 = x2 − 1, f2 = x − 1, f3 = x.

This amounts to finding scalars c1 , c2 , c3 such that

f = c 1 f1 + c 2 f2 + c 3 f3 .

In other words, we need to find scalars c1 , c2 , c3 such that

2x2 + 4x − 5 = c1 x2 − c1 + c2 x − c2 + c3 x.

Comparing coefficients, we end up with the system

c1 = 2, c2 + c3 = 4, −c1 − c2 = −5.

This has a unique solution, namely c1 = 2, c2 = 3 and c3 = 1. One

may thus express f as the linear combination f = 2f1 + 3f2 + f3 .
17 / 22
Example: Linear independence in P2

We show that f1 , f2 , f3 are linearly independent in the case that

f1 = x2 − x, f2 = x2 − 1, f3 = x + 1.

Suppose c1 f1 + c2 f2 + c3 f3 = 0 for some scalars c1 , c2 , c3 . Then

c 1 x2 − c 1 x + c 2 x2 − c 2 + c 3 x + c 3 = 0

and we may compare coefficients to find that

c1 + c2 = 0, c3 − c1 = 0, c3 − c2 = 0.

This gives a system that can be easily solved. The last two equations
give c1 = c3 = c2 , so the first equation reduces to 2c1 = 0. Thus, the
three coefficients are all zero and f1 , f2 , f3 are linearly independent.

18 / 22
Example: Basis for a subspace in P2
Let U be the subset of P2 which consists of all polynomials f (x) such
that f (1) = 0. To find the elements of this subset, we note that

f (x) = ax2 + bx + c =⇒ f (1) = a + b + c.

Thus, f (x) ∈ U if and only if a + b + c = 0. Using this equation to

eliminate a, we conclude that all elements of U have the form

f (x) = ax2 + bx + c = (−b − c)x2 + bx + c

= b(x − x2 ) + c(1 − x2 ).

Since b, c are arbitrary, we have now expressed U as the span of two

polynomials. To check those are linearly independent, suppose that

b(x − x2 ) + c(1 − x2 ) = 0

for some constants b, c. One may then compare coefficients to find

that b = c = 0. Thus, the two polynomials form a basis of U .
19 / 22
Example: Basis for a subspace in M22
Let U be the subset of M22 which consists of all 2 × 2 matrices whose
diagonal entries are equal. Then every element of U has the form
 
a b
A= , a, b, c ∈ R.
c a
The entries a, b, c are all arbitrary and we can always write
     
1 0 0 1 0 0
A=a +b +c .
0 1 0 0 1 0
This equation expresses U as the span of three matrices. To check
these are linearly independent, suppose that
     
1 0 0 1 0 0
a +b +c =0
0 1 0 0 1 0
for some scalars a, b, c. Then the matrix A above must be zero, so its
entries a, b, c are all zero. Thus, the three matrices form a basis of U .
20 / 22
Coordinate vectors in a vector space

Definition 3.15 – Coordinate vectors

Suppose V is a vector space that has v1 , v2 , . . . , vn as a basis. Then
every element v ∈ V can be expressed as a linear combination

v = x1 v1 + x2 v2 + . . . + xn vn

for some unique coefficients x1 , x2 , . . . , xn . The unique vector x ∈ Rn

that consists of these coefficients is called the coordinate vector of v
with respect to the given basis.

By definition, the coordinate vector of vi is the standard vector ei .

Identifying each element v ∈ V with its coordinate vector x ∈ Rn ,
one may identify the vector space V with the vector space Rn .
Coordinate vectors depend on the chosen basis. If we decide to use a
different basis, then the coordinate vectors will change as well.
21 / 22
Coordinate vectors in Rn
Theorem 3.16 – Coordinate vectors in Rn
Suppose that v1 , v2 , . . . , vn form a basis of Rn and let B denote the
matrix whose columns are these vectors. To find the coordinate vector
of v with respect to this basis, one needs to solve the system Bx = v.
Thus, the coordinate vector of v is given explicitly by x = B −1 v.

In practice, the explicit formula is only useful when the inverse of B is

easy to compute. This is the case when B is 2 × 2, for instance.
The coordinate vector of w is usually found using the row reduction
   
v1 v2 · · · vn w −→ In x .
One may similarly deal with several vectors wi using the row reduction
   
v1 v2 · · · vn w1 · · · wm −→ In x1 · · · xm .
The vector v is generally different from its coordinate vector x. These
two coincide, however, when one is using the standard basis of Rn .
22 / 22