Reminders

Let V be a vector space over F, let v1 , . . . , vn ∈ V be vectors.

Definition
For any α1 , . . . , αn ∈ F, the vector

α1 v1 + α2 v2 + . . . + αn vn

is a linear combination of v1 , . . . , vn .

Let S ⊆ V be a set of vectors.

Definition
The linear span of S (denoted by span(S)) is the set of all linear
combinations of elements of S.
We say that S generates a subspace U if U = span(S).
Lemma
Let S be a subset of a vector space V. If v ∈ span(S), then

span(S ∪ {v }) = span(S).

Proof.
If v ∈ span(S), then

v = α1 v1 + . . . + αn vn

for some v1 , . . . , vn ∈ S.
Clearly, span(S) ⊆ span(S ∪ {v }). If x ∈ span(S ∪ {v }), then
w.l.o.g.
x = βv + β1 v1 + . . . + βn vn .
Hence,

x = (β1 + βα1 )v1 + · · · + (βn + βαn )vn ∈ span(S).

Spans generating a subspace

{(x, y , z) : 3x − 3y + z = 0} = span({(x, y , z) : 3x − 3y + z = 0})

= span((1, 1, 0), (1, 2, 3), (3, 4, 3))
= span((1, 1, 0), (1, 2, 3))
Non-minimality

In
span((1, 1, 0), (1, 2, 3), (3, 4, 3)),
the vector (3, 4, 3) is redundant—a linear combination of other
vectors.

(3, 4, 3) = 2(1, 1, 0) + (1, 2, 3).

Equivalently,

(3, 4, 3) − 2(1, 1, 0) − (1, 2, 3) = o.

A linear combination

α1 v1 + α2 v2 + . . . + αn vn is

trivial if α1 = . . . = αn = 0,
non-trivial otherwise.
Let V be a vector space.
Definition
A set S ⊆ V is linearly independent if no non-trivial linear
combination of elements of S is equal to o.
Examples

The set {(1, 1, 0), (1, 2, 3), (3, 4, 3)} in R3 is not linearly
independent, since

(3, 4, 3) − 2(1, 1, 0) − (1, 2, 3) = o.

The set {x 2 + x + 1, x 2 + 2x + 3, x 2 + 2.5x + 4, x + 2} in

P is not linearly independent, since

−(x 2 + x + 1) + 3(x 2 + 2x + 3) − 2(x 2 + 2.5x + 4) = o.

The set {(1, 1, 0), (1, 2, 3)} in R3 is linearly independent.

If o ∈ S, then S is not linearly independent, since

1o = o.

The empty set is linearly independent.

Testing linear independence
Problem
Is {(0, 1, 0, 1), (1, 0, 1, 0), (1, 2, 3, 4)} linearly independent?

Decide whether there exist α1 , α2 , α3 6= 0, 0, 0 such that

α1 (0, 1, 0, 1) + α2 (1, 0, 1, 0) + α3 (1, 2, 3, 4) = (0, 0, 0, 0).
Equivalently  0 1 1     
 1 0 2  α1 0
   α2  =  0 
 0 1 3 
α3 0
1 0 4
   
0 1 1 1 0 0
 1 0 2   0 1 0 
RREF   0 1 3  =  0 0 1 ,
  

1 0 4 0 0 0
the only solution is (α1 , α2 , α3 ) = (0, 0, 0). The vectors are
linearly independent.
Linear independence and RREF

Lemma
Vectors a1 , . . . , ak ∈ Fn×1 are linearly independent if and only if
all columns of
RREF(a1 |a2 | . . . |ak )
are basis columns.
Linear independence and minimality of span
Lemma
A set S ⊆ V is linearly independent if and only if for every
T ( S,
span(T ) 6= span(S).

Proof.

⇒ If span(T ) = span(S) for some T ( S, then consider

v ∈ S \ T . Since v ∈ S ⊆ span(S) = span(T ), we have

v = α1 v1 + . . . + αn vn

for some v1 , . . . , vn ∈ T . But

α1 v1 + . . . + αn vn − v = o

contradicts the independence of S.

Linear independence and minimality of span
Lemma
A set S ⊆ V is linearly independent if and only if for every
T ( S,
span(T ) 6= span(S).

Proof.

⇐ Suppose for a contradiction that

α1 v1 + . . . + αn vn = o

for some v1 , . . . , vn ∈ S, with α1 6= 0. Then

v1 = −α1−1 (α2 v2 + . . . + αn vn ) ∈ span(S \ {v1 }),

and thus
span(S \ {v1 }) = span(S).
Basis

Let V be a vector space.

Definition
A set S ⊆ V is a basis if
S generates V, i.e., span(S) = V, and
S is linearly independent.
Example

Each of the following sets

{(1, 1, 0), (1, 2, 3)}
{(1, 1, 0), (3, 4, 3))}
{(1, 2, 3), (3, 4, 3))}
is a basis of the plane {(x, y , z) : 3x − 3y + z = 0}.
Standard basis

The vectors
(1, 0, 0), (0, 1, 0), (0, 0, 1)
form a basis of R3 .

More generally, the vectors

(n) T
 
(1, 0, 0, . . .) = e1
(n) T
 
(0, 1, 0, . . .) = e2
...
(n) T
 
(0, 0, . . . , 0, 1) = en

form the standard basis of Rn .

Does every vector space have a basis?

“It depends.”
Equivalent to the Axiom of Choice.
E.g., no constructive way to obtain basis for the space of
functions R → R.
Basis in a “nice” space

Definition
A vector space V is countably generated if there exists a (finite
or infinite) sequence v1 , v2 , . . . ∈ V such that

V = span(v1 , v2 , . . .).

All spaces we considered are countably generated, except for

the space of functions/continuous functions R → R
the space of infinite sequences
R as a vector space over Q
Basis in a “nice” space

Lemma
Every countably generated space has a basis.

Proof.
If V = span(v1 , v2 , . . .), let

B = {vi : vi 6∈ span(v1 , v2 , . . . , vi−1 )}.

We have span(B) = span(v1 , v2 , . . .) = V.

If α1 vi1 + . . . + αk vik = o for some vi1 , . . . , vik ∈ B, where αk 6= 0

and i1 < i2 < . . . < ik , then

vik = −αk−1 (α1 vi1 + . . . + αk −1 vik −1 ) ∈ span(v1 , . . . , vik −1 ),

which contradicts the construction of B.

Transfer lemma
Lemma (Transfer lemma)
Suppose that S ⊂ span(T ) is a linearly independent set. If
span(S) 6= span(T ), then there exists v ∈ T \ S such that
S ∪ {v } is linearly independent.

Proof.
Since span(S) 6= span(T ), there exists v ∈ T \ span(S).
Suppose that
αv + α1 v1 + . . . + αn vn = o
for some v1 , . . . , vn ∈ S. If α 6= 0, then

v = −α−1 (α1 v1 + . . . + αn vn ) ∈ span(S), a contradiction.

Hence α = 0, and since S is linearly independent,

α1 = . . . = αn = 0. Therefore, S ∪ {v } is linearly
independent.
Extension to basis

Corollary (Extension lemma)

If a vector space V has a finite basis B, then every independent
set is a subset of some basis.

Proof.
Let S be an independent set. Keep adding elements of B to S
by the Transfer lemma until S becomes generating.
Exchange lemma

Corollary (Exchange lemma)

Let S ⊂ span(T ) be a linearly independent set. For every
s ∈ S \ T there exists t ∈ T \ S such that

(S \ {s}) ∪ {t}

is linearly independent.

Proof.
Since S is linearly independent,

span(S \ {s}) ( span(S) ⊆ span(T ).

By the Transfer Lemma, there exists t ∈ T \ S such that

(S \ {s}) ∪ {t} is linearly independent.
Generating and independent sets

Lemma (Generating-independent inequality)

Suppose that S and T are sets of vectors, where T is finite. If
S ⊆ span(T ) is linearly independent, then S is finite and
|S| ≤ |T |.

Proof.

Using the Exchange lemma, replace elements of S \ T by

elements of T \ S in S as long as possible (at most |T |×).
In the end, S ⊆ T , and thus |S| ≤ |T |.
Sizes of bases

Lemma
If a vector space V has a finite basis, then all its bases are finite
and have the same size.

Proof.
Let B1 and B2 be two bases of V, where B1 is finite.
span(B1 ) = V and B2 ⊆ V is linearly independent.
By the Generating-independent inequality, |B2 | ≤ |B1 |.
Symmetrically, |B1 | ≤ |B2 |.

Definition
The dimension dim(V) of a vector space is the size of its basis.
Examples

Rn has dimension n.
Rn×m has dimension nm.
Complex numbers as a vector space over R have
dimension 2.
The space of polynomials has infinite dimension.
The space of polynomials of degree at most n has
dimension n + 1.
The trivial space {o} has dimension 0.
Dimension, independent and generating sets

Lemma
Let V be a vector space of a finite dimension n.
Every independent set in V has size at most n, and all
independent sets of size n are bases.
Every set that generates V has size at least n, and all
generating sets of size n are bases.

Proof.
Let S be independent, G generating.
|S| ≤ n ≤ |G| by the Generating-independent inequality.
If |S| = n, then no proper superset of S is independent.
By the Transfer lemma, span(S) = V.
Dimension, independent and generating sets
Lemma
Let V be a vector space of a finite dimension n.
Every independent set in V has size at most n, and all
independent sets of size n are bases.
Every set that generates V has size at least n, and all
generating sets of size n are bases.

Proof.
Let S be independent, G generating.
|S| ≤ n ≤ |G| by the Generating-independent inequality.
If |G| = n, then no proper subset of G is generating.
Hence, span(A) 6= span(G) for every A ( G.
Implies that G is linearly independent.
Dimension and subspaces

Lemma
Suppose that V has finite dimension, and U b V.
dim(U) ≤ dim(V)
If dim(U) = dim(V), then U = V.

Proof.

Let BU be a basis of U.
By the Extension lemma, we have a basis BV ⊇ BU of V.
dim(U) = |BU | ≤ |BV | = dim(V)
If dim(U) = dim(V), then BU = BV and

U = span(BU ) = span(BV ) = V.
Example: Dimension and subspaces
Subspaces of R3 :
Dimension 3: R3
Dimension 2: spans of 2 independent vectors = planes
containing (0, 0, 0).
Dimension 1: spans of vectors = lines containing (0, 0, 0).
Dimension 0: {(0, 0, 0)}
Example: bases of polynomials

Pn has dimension n + 1
Basis 1, x, x 2 , . . . , x n .
Lagrange polynomials

Let a0 , . . . , an ∈ R be pairwise distinct.

For k = 0, . . . , n, let

(x − a0 ) · · · (x − ak −1 )(x − ak +1 ) · · · (x − an )
pk (x) = .
(ak − a0 ) · · · (ak − ak −1 )(ak − ak +1 ) · · · (ak − an )
(
1 if i = k
We have pk (ai ) =
0 if i 6= k
The set B = {p0 , . . . , pn } is another basis of Pn .
|B| = dim(Pn )
B is linearly independent:

(α0 p0 + . . . + αn pn )(ai ) = α0 p0 (ai ) + . . . + αn pn (ai ) = αi ,

hence if α0 p0 + . . . + αn pn = o, then αi = o(ai ) = 0 for

i = 0, . . . , n.
Polynomial interpolation
Corollary (Polynomial interpolation lemma)
A polynomial p of degree at most n is uniquely determined by
its values in n + 1 distinct points.

Proof.

Since B generates Pn , there exist α0 , . . . , αn ∈ R such that

p = α0 p0 + . . . + αn pn .

For i = 0, . . . , n,

p(ai ) = α0 p0 (ai ) + . . . + αn pn (ai ) = αi .

Therefore,
p = p(a0 )p0 + . . . + p(an )pn
is uniquely determined by the values of p in a0 , . . . , an .
Example
Problem
Find the equation of a quadratic function through points

(−2, 9), (−1, 2), and (1, 6)

y
9
8
7
6
5
4
3
2
1
x
−2 −1 1

(x + 1)(x − 1) (x + 2)(x − 1) (x + 2)(x + 1)

9 +2 +6
(−2 + 1)(−2 − 1) (−1 + 2)(−1 − 1) (1 + 2)(1 + 1)
= 3x 2 + 2x + 1
Vandermonde matrix

Definition
For distinct real numbers a0 , . . . , an ,

1 a0 a02 . . . a0n
 

 1 a1 a12 . . . a1n 
 
(a0 ,...,an )
V =
 ,
 . . . 

1 an an2 . . . ann

is a Vandermonde matrix.
For any polynomial p(x) = β0 + β1 x + β2 x 2 + . . . + βn x n ,
   
β0 p(a0 )
 β1   p(a1 ) 
V (a0 ,...,an ) 
 ...  =  ... 
  

βn p(an )
Vandermonde matrix and polynomial interpolation
For b0 , . . . , bn , if a polynomial
p(x) = β0 + β1 x + β2 x 2 + . . . + βn x n
satisfies p(a0 ) = b0 , p(a1 ) = b1 , . . . , p(an ) = bn , then
   
β0 b0
 β1   b1 
V (a0 ,...,an ) 
 ...  =  ... 
  

βn bn
By the Polynomial interpolation lemma, this system always has
a solution,
β0 + β1 x + β2 x 2 + . . . + βn x n = b0 p0 (x) + b1 p1 (x) + . . . + bn pn (x).

Corollary
Every Vandermonde matrix is regular.
Linear recurrences

Problem
Describe all infinite sequences a0 , a1 , . . . that satisfy

an+2 = 5an+1 − 6an for every n ≥ 0. (1)

Let S be the vector space of infinite sequences, and let U ⊆ S

consist of those satisfying (1). Then U is a subspace:
(0, 0, . . .) ∈ U
If A = (α0 , α1 , . . .) ∈ U and B = (β0 , β1 , . . .) ∈ U, and
γ ∈ R, then

αn+2 + βn+2 = 5(αn+1 + βn+1 ) − 6(αn + βn )

γαn+2 = 5γαn+1 − 6γαn ,

and thus A + B, γA ∈ U.
Linear recurrences

Problem
Describe all infinite sequences a0 , a1 , . . . that satisfy

an+2 = 5an+1 − 6an for every n ≥ 0. (1)

The choice of a0 and a1 uniquely determines the rest of the se-

quence. Hence, dim(U) = 2. “Standard” basis:
a0 = 0, a1 = 1 → (0, 1, 5, 19, 65, . . .)
a0 = 1, a1 = 0 → (1, 0, −6, −30, −114, . . .)
Linear recurrences

Problem
Describe all infinite sequences a0 , a1 , . . . that satisfy

an+2 = 5an+1 − 6an for every n ≥ 0. (1)

Nicer basis:
an = 2n → (1, 2, 4, 8, 16, . . .)
an = 3n → (1, 3, 9, 27, 81, . . .)

2n+2 = 4 · 2n = 10 · 2n − 6 · 2n = 5 · 2n+1 − 6 · 2n
3n+2 = 9 · 3n = 15 · 3n − 6 · 3n = 5 · 3n+1 − 6 · 3n

Therefore, for α, β ∈ R, an = α2n + β3n is a solution, and no

other solutions exist.