BASIC LINEAR

ALGEBRA

CEMALKO~

Department of Mathematics

Middle East Technical University

Ankara

1996

 MATH 2;30, Linear Algebra

\Veekly Teaching Plan
(Spring term)

VVeeks Sections

1) Feb 26 March 1 1.1, 1.2

2) lVIarch 4 8 1.3, 1.4
3) ll •j
~.o
~

1.5,
4) 18 22 Cont: 1.5, 1.6
5) ')­
_.') 29 Cont: 1.6. 1.7
6) April .) Cont: 1.7, 2.1
,-i
7) (. - 12 ') ')

6) F;u 19 1.-\prilli. l.Interm; Cont: :2.2

9) 22 26 (April 23, Bayram) 3.1
10) 29 May 3 (April 29,30, May 1: Bayram)
11) May 6 10 3.2, 3.3
12) 13 17 Cont: 3.3
13) 20 2L1 (:VIay 22, II.Interm) 4.1
1Lt) 27 31 4.2, 5.1
15) June 3 7 5.2

Book: Basic Linear Algebra

By Prof.Dr.Cemal I\o<:;

.•
 BASIC LINEAR

ALGEBRA

CEMAL KOC,
Department of Mathematics

Middle East Technical University

Ti'c" " ­
. J\EJ\0~,

Ankara
1996
/
1. MATRICES AND SYSTEl\fS OF LINEAR EQUATIONS ........... 1

1.1. Scalars ........................................................ 1

1.2. :tviatrices ....................................................... G

1.3. Operations on Matric0s ....................................... 11

1.4. Special Type of Squ<~re .\I a trices .............................. 25

1..5. Row Equivalence, Inwrtihility ................................ 3:3

1.6. Systems of Linear Eqnations .................................. .53

1.7. Detenninants ................................................. 7.5

2. VECTOR SPACES ................................................ 99

2.1. Vector Spaces ................................................ 99

2.2. Linear Dependence, Ind<'Jwndence, B<~ses .................... 117

3. INNER PRODUCT SPACES ..................................... 157

3.1. Inner Products .............................................. 159

3.2. Norm and Orhogonality ..................................... 167

3.3. Orthogonal and Orthonorntal Bases .......................... 177

4. DIAGONALIZATION AND ITS APPLICATIONS ............... 197

,1.1. Eigenvalues, Eigenvectors and Dia.gonalizat ion ............... 197

4.2. Diagonalization of R('al Symmetric Matric0s ................. 218

5. LINEAR TRANSFORNL\TTO:'\S ................................. :2~)q

5.1. Definition and Exam i>IPs ....•......•.•....•...............•. :2~~)

5.2. Matrix Repres<'nt.at ions or LinPar Transformations ........... 2·1:3

1\PPENDIX A ................................................... 2.11

APPENDIX B ................................................... :lG.-1

INDEX .......................................................... 2G6

Chapter 1

MATRICES
AND

SYSTEMS OF LINEAR EQUATIONS

In this chapter we shall give ha:;;ic properties of matrices and solutions of

systems of linear equations to which <'Very read<'f is aqua.inted at least at the
introductory level.

I l l SCALARS

Concepts oflinear algebra such as vectors, matrin's, linear transformations

etc. have indispensible companions called scalars. At the introductory lcv<'l
these scalars are usually taken to h<' real numbers and seldom they are tak<'n
to be complex numbers. But even in engineering applictions real and com­
plex numbers are not adequate to h<' taken as scalars. For example in coding

~~=---~----~-------------------------------------------------
I
_j
(i)

theory the primary scalars to b0 us0d <11"<' 0 and 1 with binary addition a11d
mult.ipliration. For this rt'ason, we arr' ololig<'d to g<'neralize the concept of
scalar by iutroducing fi<'lds.

Definition 1.1.1. Dy, a Jilld we nwan a s<'l F together \Vit.h two operations
+ and · (usually ralh'd O]H'rations of addition and mu.llip/i('(l/ion) for which
the following properti<'S hold:

l. a+ (b +c)= (a+ b)+ c for all a. band c in F.

2. There exists a.u <'lelll<'llt 0 ( calkd the ::rm clrmcnt) snell 1h:d

a +0 = 0 +a =a for all a E F.

:3. For each element o ofF there 0xis1s an <'lcmen.t -a (call<'d I h<' addilit·e
im•crse of a) in r snell that

a + (-a) = (-a) + a = 0.

4. a+ b = b +a for all a and bin F .

.5. a · ( b · c) = (a · b) · (' for all a, b and c i n F.

6. a · ( b + c) = (a · l1) + (o · c) for all o. b and c in F.

7. a· b = b ·a for all o and bin F.

8. There exists an cl<'IIH'n1 1 (called 111<' unity or identity dement) such that

a· I =1·a = a for a 11 a m F
 9. For each a different from th<' ZNO element 0 thNe exists an element a- 1
(called the multiplicative inl'rT.<~r' of a) such that

In our investigations in linear al~Phra we always fix a field and we call its
<'lements scalars.

Note that the properties list<•d ahove are natural properties of real and
complex numbers and thereby thos<' who are not familiar with other examples
of fields may be contented with considering scalars to be real or complex
numbers.

Example 1.1.2. Natural exampks of fields arc the following sets together
with the usual operation of addition and multiplication

Q the sd of rational numhcrs

R the SC't of real numlH'rs

C = the s<'t of complex n llllllwrs

Z 2 = {0, 1} with the operations <kfitwd by

+ 0 1 • 0 1
0 0 1 0 0 0
1 1 0 1 0 1

Similarly one can deftne Z 3 , Z 5 and in general Zp for any prime number p
which are the most elementary examples of fields.
181 EXERCISES

I. fn each cas<' ht•low dl'tPrmin<' wlwt l11•r t l1e given sd is a fi('ld or not.

(a.) The set of nonnPp;ative real nnmiH•rs Ulllh'r usual addition and nJul­
tiplication.

(b) The s<•t of natnralllllllllwrs llll<iPr usual addition and multiplication.

(e) Tht• set of odd ini<')!;NS nnd<'r usual addition and nndtiplicat.ion.

(d) The set of rational numlwrs with odd denominator undPr usual
addition and multiplication.

(e) The subset {a+ ib: a,b E Q} of C under usnaJ op<'rations.

2. Solve the follmving <'CJnations in z~ in Z: 3 and in Z5

(a) G.z·=2 (b) 2:r~+:3.r-2=0 (c) :r 4 =1

;~. Given :1: 6 = 1. Find its solution sPt ill

(a.) R (h) C (c) Z7 .

4. Find the complex numbers :r and y satisfying the equations

(2- i)x + iy = -3 and - ix +y = 0.

5. Find x, y and z in Z 5 satisfying the equations

2x + 3y + z = 4, 2;r + y- z = 0, x- y- z = -1
(j

llfJ MATRICES
A matrix is a rectangular array of clements taken from a field of scalars.
To make it precise we giw thl' follo\\'ing

Definition 1.2.1. Let F he a fi<·ld. An 1n x n matri:r Jt OL'CT F is a function

from the set
{(i,j): i = 1,2,. .. ,m j = 1,2,···,n}

into the field F. The image A(i.j) of (i,j) under A is called the (i,j)-entryof
A and it is usually cll•notPd hy .. \ij and A itsdf is denoted by

....1,1 il,2 Atn

,-\~I ....12:? A:!n

"\ =
...11111 ,\,:? Am.n

The set of all m X n ma I rir·rs orr r Ihe fir· hi F is denoted by F"" x 11 •

For any 1::; i::; m the 1 X 11 matrix

is called the ilh mw of J, and for any 1::; j::; n then X 1 matrix

Amj

is called the j-th column of A. The cntriC's A 11 , A 22 , .•• , Au, ... arc called
entries on the main diagonal.. \\'lH•n n = m. the matrix is said to ])(' square.
 7

When we take the field F to be the field of real or complex numbers-as we

usually do-we speak of real or complex matrias.

Example 1.2.2. The function A: {1, 2, 3} x {1, 2}-+ R given by

A(1, 1) = -2, A(1, 2) = V2, A(2, 1) = 1, A(2, 2) = 0, A(3, 1) = -1, A(3, 2) =0

[-~ ~ l
is the 3 x 2 matrix which is denoted by the rectangular array

-1 0

Example 1.2.3. For the matrix

-2 2 -1
[
1 0 1
we have

A(1, 1) = A11 = -2, A(l, 3) = A = -1,

13 A(2, 2) = A22 = 0
Example 1.2.4. The function 8 defined on pairs of natural numbers by

. .
o(z,]) = 0;3· ={ 1 if i =j
0 if i ::/= j
is referred to as Kronecker delta and the n x n matrix I defined by

I;i = O;j, for i = 1,2, . .. ,n; j = 1,2, . .. ,n

is called the identity matrix as it acts as identity with respect to multiplication

that will be introduced later. It's rectangular array form is

1 0 0
0 1 0
I=

0 0 1
 \ot<' that two matricP:-; .\and lJ an' <'qual ifl'tii('S<' I\\'O functions ha\'(' th0
sallll' domain and th<' s;tlll~> \·alll('s: in otll('r words
A = 11 ~ i\;j = /J;; for i = 1, 1, ... , m; j = I. 2, ... , 11

Definition 1.2.5. II' . I is an m X 11 matrix t h<' lmnspo.w .-I'~' of. I IS t l1<'

n X 111 matrix whirh is ~i\'"11 hy

T
.I (i.))=.lji·

'I II ,\~1 .. I Ill I

:11 l .. 11~ /1,"
.. 11:! :I:!:!
[
. 'HI:!
.·1 = :I~ I .. t'..!, .lr
,·1,,,
A:!:!

..- \ 111:2 : \ 11111

j
.. I '" :1:!11 .! \11111

II illll ('I.\' rows of 1l an' rollllllllS of ,I and \'I('(' \'l'rsa.

ii'J EXERCISES

I. \\"ritl' ('arll lllat rix IH'Iow in n'rlall).!;lllar arra.\· for1n

 9

(a)A;j=i 2 +j 2 -1; i=1,2,3 and j=1,2

2-J
(b) A;j = i + j; i = 1, 2 and j =1

i +j if i 2: j

(c) A;j = { . 0 i=1,2,3,4 and j = 1, 2

if i < j

1 if i =I= j

(d) A;i = { 0 i = 1, 2, 3 and j = 1, 2

if 'l=J

l
2. For each matrix below find the indicated value:

(a) A= [ ')0. ,1 -011 A(2,2)+A(3,1)=?

(b) A= [ -~ ~ ~
3 1 -1
l A12 + A21 + As2 =?

3. Find a if

[ x+y y+z]
z+t t+x -
[1 a]a .
3

4. Write a 3 X 3 matrix A whose f'ntries are distinct nonzero digits so that

3 3 3 3

LAij = LAA·j =LAir= LAs

j=l j=l i=l i=l

for any choice of l, k, r, s E { 1, 2, 3}. What is the common value of this

sum? Change rows so that the sum of entries on each diagonal are equal
to this common value. (The resulting matrix is called a magic square.)

5. For
10

G. Prm·p that (:IT j"l' = .·1 for a11y 1na1 rix :I.

1. Tl/( nodal inr·irh /IN 1/l(f/ri.r of" a diri'C11'd gr;q>ll ronsistinp; of" ~!IIIII' \'<'!"­

tin's \'1. \~ ..... 1·i···· a11d Sllllll' f'dg1'~ (diri'Ctl'd pa1hs) r 1 .r~ ..... r 1 , ...

is d<'fi 1wd h.v

ir (j It'll ,.('S I.
I

·"ij =
{ -- I
()
if

ir
(

(.
J

J
. I'll t I' rs

<I lid I.
I
I .i

(I J"(' not COilii('Ct I'd.

( i) Find t II<' illrid<'ll('(' Ill <II rin's or tIll' f"ollo\\'ill,(!,:

(a) v 3 (h)

/1e~
F 4 \' s \' 2

(~}~~
\. I

l
() -I
(ii) Ski'lrli II"' g1·a ph n>IT<''P"" d; 11g I" [ () -I

-I -I () ()

 11

OPERATIONS ON MATRICES

Now we shall define some operations to produce new matrices out of some
given matrices and scalars.

A. Matrix addition. Th0 sum A+D of two m x n matrices A and B is

defined as the matrix with entries

(A + B);i = A;i + B;i; i = 1, 2,, .. , m; j = 1, 2, ... , n.

Nate that this is in accordenc0 with the usual definition of the sum of real
valued functions, namely it is ess0ntia.lly in the form

(J + g)(.r) = f(x) + g(x)

where f = A and g = B and x is t h0 pair ( i, j) . But this time the values may
be taken in an arbitrary field of scalars in place of reals. The operation can
be carried out by adding corresponding entries of the arrays of the same size.
For example we have

[~ -~ ~ :l
1 -1 1 4
+[
-1
~ 1 :
0 1
=~4
l = [:
0 -1
~ ~ :
2 8
]·

The properties of the operation of addition on pnxn are very natural and
is given in

Theorem 1.3.1. The followin?; are satisfied in pnxn:

(1) A+ (B +C)= (il +B)+ C: for all A,B,C E Fmxn

(2) For the m X n matrix 0 wl1ose entries are all zero we have

A +0 = 0 + . -1 = A for all A E Fmxn

12

(3) For any m. X n matrix A the matrix -A defined hy (-A )ij = -A;i
satisfies
,.\ + (-A)= (-A)+ A= 0.
-A is called t.hc tugalit·(· of..-\.

(4) We have

il + 11 = 11 +A for all il, R E pmxn.

Pmof: The proof is st raight.forward. For example to prow tl1e fl rst. prop­
erty it is enough to V<'rif.'· that (A+ (13 + C'));j = ((il + IJ) + C')ij and it is
obta.hwd a.t once hy using 1h<> definition of addition:

[(A+ (lJ + C));i ,.\ii + (lJ + C');i = A;i + (Tl;i + C;i)

(,-\ii + /Jij) + C'ii = (;l + B);i + Ci = ((~\ + fl) + C)ii·

All others ca.n b0 provc'd just. in the same manner.

B. Multiplication by scalars. Given any sc<llar c and any m. X n

matrix A t.hc matrix (·,-\ is dPfltH'd hy

(cA)ij = (·,·\ii for all i = l,2, ... ,m; j = 1,2... . ,n.

Again it is essential]~· nothing hut the well-known formula

(cf)(x) = cf(:l:)

where f =A and :r = (i.j). The operation is carried out by multiplying each

entry of the given matrix hy I he given scalar. For example, if we tak<'

..-\ = [ ·I
:J
0 2 3] c = -3
1 5 3 '
0 -1 2 8

 JS

we have
r
-3A =
-0
-12
0
-3
-6
-15
-9]
-9 .
0 3 -6 -24

Theorem 1.3.2. Let c, c~o c2 lw scalars and let A, A 11 A 2 lw Til X n ma­

trices. Then we have

(3) (c1cz)A = c1(c2A).

(4) l·A=A.

Proof: The first of them follows from

and the others are proved similarly.

C. Matrix multiplication. If ..l is an m X n matrix and n is an n X r

matrix then their product AB is dPfi11ed by giving its (i,j)-entry as

(AB)ij = L" A;kBkj·

k'=l

Note that the product AB is ddined only in the case

the number of columns of r1 = the number of rows of B .
The following figure shows how the operation is carried out:
14

Exan1ples 1.3.3. a)

A= [
l I

.:
For tlH' matrices

-2 :1

.~
and IJ =[
:l:
z
11
y
t

v
l
we have

AJJ =[:
·1
-~ l[·: ~ l
:
·) II 1'
[
x - 2.:
:r+z+·tt
<Lr
+ 3u

+;:; + 5n
y - 2l

•!y
+ :"11·
'IJ+I+v
+ t + 5v
l
.

vVlwi·eas IJA is not <h•finPd.

we have

From these examples w0 nhsPrw that 1\JJ and JJil may not be equal. r·1•rn
more one of them may not fJ( dr.fincd, and the pmduct of non.::cm ma I ric('H
need not be nonzero.

Now, consider three matrin•s ,\.JJ and C snch that the product (.\TJ)C is
defined, then since_,\[] is dPfined A and B must be m X nand n X r ma1rici'S
 15

respectively. Since the product (A R)C is also d('fi ned C must be an r X s

matrix. Therefore the products BC and fl(BC) are defined. Further, we have

r r n r n

((AB)C);i = 2)AB);~.;Cki = 2:\2.: AaB11. )C~.;1 = L L AilBikCki·

k=l k=1 1=1 k=l 1=1

n n r r n

(A(BC); 1 = 2:At(BC)t1 = L:Aii(L Ru,Ck 1 ) = :L;L:AilBikckJ·

1=1 1=1 k=l A:=l 1=1

showing that (AB)C = A(BC). Thus we S('<' that if the product (AB)C is
defined, so is A(BC) and the equality A(RC) = (AR)C holds, that is matrix
multiplication is associative. In a similar way we prove distributive laws and
that the identity matrix acts as identity. So the following theorem has been
established:

Theorem 1.3.4. If A, B and C arf' matrices such that left sides of the
following are defined then right sides are also defined and the equalities hold:

1. (AB)C = A(BC).
2. fl(B +C)= AB +ftC and (.1 + /J)C = AC + BC.

3. IA = A and AI = A, where I stands for identity matrices.

4. c(AB) = (cA)B = A(cB) for any scalar c.

Note that when we take square matric('s, the sum and the product of two
matrices of the same size are always defined, and the above equalities can be
used in computations. We can also form positive powers of a matrix A with
usual rules
J(j

Examples 1.3.6. a) Ci\'<'11

2
-]
.I= 0
1
1

Comp11t<' tiH' product .1/J('.

\V<> ran compute this product as (i\ll)C or il(DC). Not(' that tl1<' quicker
choic<' is ,\( /JC) and it gin's at once

2 2 10

1][_~]~~
-I -I -!i
,-\(TJ(') = (} ( [ :~ ·I 0 [!i] = 0
!")

;)

b) For A ~ [~ ~] n ~ [~ 0 ]
()
and (' = [ 0 0]
] 0
com­

pute An, ilC, nc and ( "' + 211)( -B +C).

Computation giws

A II ~ ~ ~
[ ] "'"I AC ~ ~ ~
[ ]

and lwnc<'

(A+ 2/J)(-fl + C') = -.-IJJ + AC- 2JJ + 2BC

2

0 +[' o]- 2[1 o]+o-[1 o]=-JJ.

() 0 () 0 0 0
 17

Finally we give the following

Theorem 1.3.7. If A and Ban' matrices over the same field we have

(a) (A+ B)T =AT+ BT

(b) (cA)T =cAT

(c) (ABf = BT AT
(d) (AT)T =A

Proof: Each equality is proved at once by computing the (i,j) entries of

both sides. For example to prove (c) we note that
n n
(AB)T(i,j) = (AB)ii = LAikBk; = LBk;Aik
k=l k=l
n
L JJ~,,tii = (BT AT)ii
k=l

MULTIPLICATION OF PARTITIONF,D l\1ATRICES

Sometimes we need to consider rectangular arrays in partitioned form. For

example when we are to deal with rows or/and columns of a matrix we consider
the rectangular array as a collection of (disjoint) rectangular arrays. This can
be done in general by considering s11hmatrices. A submatrix of a matrix is one
obtained by deleting some, but not all, of its rows and columns. Although it
is possible to consider various part it ions of a matrix into submatrices all we
need ·will be those obtained in the following manner:

Given an m x n matrix A,we fix two sequences

18

and ddinc the matrin's .\!,. 1 by

where h: = 1,···,8 and I= l,···,t. The collection ofthC's<' suhn1atriccs Af;j

completely determines .-1 . II' this is the case we write

J ,11 Aft2 M1t

Jf:!l Jl:l:l M'.!t

Jf,l M,,2 M,t

For example some p:nt it ion<'d forms of a ·I X :1 matrix ar<'

All A12 AJ:J All ;112 ill:l .4]1 .·It" .II :1

A2t fl22 il2:1 A21 An ...123 A:ll il :!:! A:!:l

A:n Aa2 An .·l:~t A32 . .1:!3 /l:n +l'.! ... 1:1:1

A41 ;\42 A4:l i\41 A4'.! J\43 il41 A42 A43

Let .4 and n be part it ionPd as

N11 Nti N1"

A/11 Af12 JUu
N21 N'.Ji ;.:ft·{:?tt

il= ~Mit M;2 M;t ,B =

N11 .T\rlj 1V1"

Afsl AJ,, 2 i\ f..t

Nn Nti 1Vt"

where each Af;j has 1'i rows and 8j columns and each N1.:1 ha.s 8k rows and (]1
columns. Then for each pair ( i, j) the matrices
 19

are defined and each of the entries of P;i is obtained by fixing a row of all
Nf;. 'sand a column of all N.j's, thus they constitute al1 entries of the product
AB. So we see that AB is partitioned by Pij 's, that is to say we have

The formal proof is rather tedious and will be omitt0(l.

Example 1.3.8. To illustrate the multiplication of partitioned matrices

we shall describe rows and columns of product matrices in terms of those of
factors.

(a) Rows of AB are obtainul by postmultiplying Ntrh row of A by B:

AB= B=

(b) Rows of AB are linear combinations of rows of B , more precisely

we have

A11R1 + ···+ AtnRn

A21R1 + .-. · + A2nRn

R,

(c) Columns of AB are obfainul by premulliplying columns of B by A:

AB = A [ C 1 C'2 ••• C, ] = [ ACr AC~- · · · ACn ]

20

(d) Columns of An arr linear combination<~ of rolumns of 11:

B11 Blr
B21 B2r
AB = [ (' 1 (''2 ... Cn ]

Bn1 Bnr
 21

11@1 EXERCISES

1. Given

A= [ : -~ -~] ,B = [ : -~ ~] and C = [ -~ : -~]

over Z5 • Find all values of x and y sud1 that xA + yB =C.
2. Given

Find all possible triple products.

~
J:f

~
!;

----------------------------------------------------- ~
22

3. Sintplify the following expr<'ssions as much as possih]f':

(a) ;\[(2A- 5fl)- ;~.4] +,·\(..\- JJ),

(h) (A+fl) 2 -(A-!Jf,

(c) (il + !J) 2 - A2 - 2AJJ- lJ 2 •

4. Th<' following is a list. of stat<'rn<'nts about squar<' rnat ricPs. In each case,
<'it h<'r show that the stal.('lll<'nt is ~<'IH'rally trne or find sp<'dfic matrices
for which th<' statellH'Ill is not t rll<':

(a) A 2 =I=> A= I or ,\=-I,

(h) (A-JJ)(A+IJ)=,\"-JJ 2 <==> AB=IJ..\,

(c) B=il 2 -5A+l=> .1/J= TJA

(d) AB=0=>11:1=0

5. C:iven

-1
:J
l and X= [
:l't
.?·"
:ra
l
.

Express AX as a linear cornhination of columns of ,1.

6. Given

A =[
-1
.j
G
-:3

1
1
2 -2
0
l
If B is a matrix v-:ith rows R 1 , 1? 2 , R 3 expr<'ss the S<'cond row of ;lfl in
terms of rows of B.
 2:1

7. Compute ;F - A+ 3! for

8. Determine all matrices of the form

satisfying 1P- I= 0. Wha.t is the nmnlwr of solutions of the equation

x2 - 1 = 0 in 2 X 2 matrices.

9. Compute A 6 for

10. Find all solutions of the equation x 2 + 1 = 0 in 2 X 2 matrices over Z 2 •

11. Given
-1 0 X y ..;,

0 -1 1l v 1l'

A= 0 0 1 0 0
0 0 0 1 0
0 0 0 0 1

Use its partitioned form

A= [-I 0
(~ l
to show that A 2 =I.
24

12. Show that A= [

-~
.1
-1
1 l ;md B
-4 -1

=[
is to say AlJ = -11.1. Compn1<' (il + B) - A 2 - B 2 •

2
l
1 - 1 anti-commut<', that

13. A matrix A is said to commul< with B if AB = BA. Determilw all 2 X 2

matrices which rom mute with <'arh of the following matrices:

E 11 =[ 1 0
0 0
l· F., 2 =[ 0 1
0 0
l, £ 21 =[ 0 0
1 0
l, En = [ 00 0
1
l
 25

11'#1 SPECIAL.TYPES OF MATRICES

In ord<'r to facilitate the communication we give special names to matrk<>s

which have ratl1('r particular appearence.

A. DIAGONAL MATRICES

A square matrix A for which Aii = 0 for all i ::J j is called a diagonal
matrix. So a diagonal matrix is one whose entries off the main diagonal art>
all equal to zero.

If A is a diagonal matrix with diagonal entries

then we write

For example the matrices

-1 0 0 0
0 1 0 0
0 0 0 0
[~ ~],
0 0 0 2

are diagonal matrices that can h<> given by

diag( -1, 1, 0, 2), diag(O,O), diag(1,1,1)2

I
26

respectively. Note that any 11 x n zero matrix and identity matrix are diagonal.

It is easy to sec that if D 1 and D"J an.' diagonal n X n matrices and c

IS a scalar, then so are D 1 + D 2 , cD 1 and D 1 D 2 , namely the set of n X n

diagonal matrices is closed under addition. scalar multiplication and matrix

multiplication.

When all diagonal <'111 ri!'s of a diagonal matrix are equal it takes the form

d
d 0
/)= = dl

0 d

such a matrix is cotnpiP1PI,V dctNniillPd h.Y the scalar d and thcrPhy it ts

called a scalar malri.r. TI1P correspondr'ncc

d- dl

provides a one-to-one correspondence hetwecn scalars and scalar matrices.

Note that this correspond<'!Ice also pr<'SNY~'S matrix operations, namely we
have

So this correspondence allows us to idPnt if:v· scalars and corresponding scalar

matrices.

B. TRIANGULAR l\1:\TRTCES

A square matrix A is said to be upprr triangular if

..-lij =0 for all i >j

 27

and it is said to be lower trim1gu!or if

A 1j for all i < j.

For example

and [
-1 2l
0 0 .JI

are upper triangular matrices wlH'r<'ils

f2
[ 4 0
1 -1
1 :1nd
l 0 0 ]
0 -1 0
0 0 3

are lower triangular. The last milt rix and in general any diagonal matrix is
both upper and lower triangular.

Theorem 1.4.1. The set of nppN triangular matrices is closed undf'r

addition, scalar multiplication and matrix multiplication. The same is true of
lower triangular matrices.

Proof: If A and B are n X n nppN triangular matrices then for i > j we

have A,j = B;i = 0 and hence
(A+ B);i A;i + B;i = 0 + 0 = 0
(cA);i cA;i = c · 0 = 0
n i-l n
(AB);i L A;kflkj = L .t1ik Bkj +L Aik Bkj =0
k=l k=l y k=i y
which proves the first half of th(' 1h('or('m. The semnd half is proved in th<>
same manner.
0
28

C. SYMMETRIC AND SKEW-SYMMETRIC !VfATRICES

A matrix A for which .IT =A is called a symmrlric matri.r. If ,\r -A

is satisfied thC'n A is said to be skr:w-symmelrir.

Symmetric and skC'w-symmetric matrices pia~· an important part 111 lir1C'ar

algC'bra and they haw t hP following elementary propC'rties:

1. Symmetric and sh ll"-H.lJmmclric nwtrirc;:; arr nrf'fssarily sq11aN malrircs.

Indeed, it follows from _.p· = =j=A that ;\ and ,p· are oft h~> "'lllH' size.

2. A is symmetric iff .\ji = ,\ii for all i,j.

il is skcw-symnwt ric i rr ,\ji = - Aij for all I',].

D. ECHELON T\[ATHICES

As we shall SC'c in th<> sul>s<'qll<'nt S<'ctions th<' k<>y st<>p in t lw snlution of

systems of linear l'fplatious is to bring the system into echelon fortn. Of ronrsc
rn this reduction ecbPlon mat rir<'s will play th<> main part.

Let A be an m. X 11 matrix. The first non-z<>ro C'lltry (if any) of tb(• ith row
of ;1 is called the lcadiny (1//ry of the ith row. If the leading entries (if any) of
the rows of A arc of tlH' form

Alit, ,\~j 2 • •••• A,.ir with .J1 < j2 < · · · < jr

then A is cai!Pd an u·hdon nwlrh·. Its shap<' is of the form

 2.9

An echelon matrix is called a row-rnlurrrl echelon matrix if

(i) each leading entry is 1

(ii) each leading entry is the unique non-zNo entry of its own column.

For example

0 0 0
000 0 0 0
000 0 0 0

are all echelon matrices, and

[
~ 0 0 0
0 0 0 0

are row-reduced echelon matrices.

so

iitl EXERCISES

1. Given the matric<'S ..-1 diag(l,-1.3,,1) and B diag(2, -2, -3, 0)

compute

2. Prove that if D is a diagonal n X n matrix, for any n X n matrix A

the product DA is oht ained by multiplying rows of A by corresponding
diagonal entriPs of D. State and prove a similar result for A D.

:3. Find the number of -1 X 4 complex diagonal matrices D which satisfy

D 3 = I. How many of t hPm are scalar matrices?

4. Let E;j denote tlw n X n matrix whose (i,j)-entry is 1, all other entries
are O.Prove that if an 11 Xn matrix A commutes with E;j ,then A,; = Aii
 31

,and all other entries of its i-th column and all other entries of its j-th
row are zero.Use this to prove that if A is an n X n matrix commuting
with all n x n matrices , th('ll A is a sca.lar matrix.

5. If A is an upper diagonal n x n matrix whose diagonal entries are all zero,

prove that An = 0. State and prove a similar r('sult for lower triangular
matrices.

6. Determine whether the following are symnwt.ric, skew-symmetric or

none.

-1
4
-7
-~ l'
0
(b) [ ~ ~~
-3
-
-7 0
l' (c)
0 -1
1
-1 -3
-1
0

2
1
3 -2
0
-4
1

4
0

7. ·what must be x, y and z if thf' matrix is sk0w-symmetric?

y+z-2
-1

1
-1
1
z+:r-3
l.

8. Prove that the diagonal C'nt rks of a real or complex skew-symmetric

matrix are all equal to zero. Is it true for a skew-symmetric matrix over
Z:.!?

9. Show that for any square matrix A the mat rices A+ AT, AAT and .r1? A
are symmetric and A- AT is skC'w-syrnnwtric.

10. Prove that every real or complex square matrix A can be written
uniquely in the form
A=S+I\
3.
'­
~

where Sis a symmetric matrix and h" is a skew-symmetric matrix. Apply

this result to

A= [5-2
~
3]
-~ 0

11. Prm·e that the set of symmetrk mat rir<'s is closed und£>r <ultlit ion and
sra lar multiplication and that it. is not dosed under matrix mull i plica­
l ion.

12. Fiud the number of2x3 cc:hdon mat riel's over Z 2 which ha\'(' two ]Pading
<'ntries. How many of them are row-r<'duced'?

13. Dl't<'rmine all l'OW-I'I'dllred echelon 2 X 2 matrices over z:l.

14. Show that the only l'<'al n X n matrix which is row-reduced <'rh,Jon wHh
11 lPading entri<'s is tlu.' idPntity matrix I.
 3.'J

ll!e11 ROW EQUIVALENCE, INVERTIBILITY-

Somdimes the reduction of matrices to echelon matrices may be unexpect­

edly fruitful. This reduction is carried out by means of certain row operations
that v.r<' a.re about to introduce.

ELEl\fBNTARY Row OPERATIONS, Row EQUIVAT.ENCE

Definition 1.5.1. Any one of the follwing opf'rations is called an elemen­

tary row operation and is denoted hy the indicated notation:

Type I. Add c times_row ito row j for .~orne pair i i: j. (Notation: cRi + R1)

Type IT. Interchange row i and rott' j. (Notation: Ri .- R 1 )

Type III. Multiply row i by a nonzero .~calar c. (Notation: c Ri)

Examples 1.5.2.

(<1) [ -~
1
-1 0
3 -1

1 4 -2

3
l (-2)R 1 + Ra
-:
[-~ -1
10 -1
1 3

0
-2
-:]
2 -3 6 2 -3 6

-1 5 1 R2 <-+ Ra 0 1 -2
-!
(b)
0 1 -2 ~ -1 5 1
2 -5 1 2 -5 1
34

2 -3 6 2 -3 6
0 1 -2 5R3
0 1 -2
(c)
-1 5 1 -5 25 5
2 -5 1 2 -5 1

-~
~l u ~l [-~ :l
1 -1 -1
-1
(1 )Rt + R3 2R2
"2
(d) [
-----+
1 2 2

2 -3 6 -2R3 0 1 -2

(e)
0 1 -2 (-l)Rt + R4 2 -3 6
-1 5 1 Rt +-+ R2 2 -10 -2
2 -.5 1 0 -2 -5
Theorem 1.5.3. r:\'('ry <'lnwnt.a.ry row operation£: F"'x" - rmxn has
an inverse of the sam<' t~·p<'.

Proof: The invNS~' of the operation cR; + Ri is ( -c)R; + Ri, for if we

write the matrices in t~'nns of tlwir rows we s0e tha.t

R; R; R;
rR; + Ri -cR; + Ri
-----+
Ri cR; + Ri R·J

Similarly the inverses of R; ,...... Rj and cR; a.re R; +-+ Rj a.nd c- 1 R; r0spectively:
 35

Ri Ri
R-
1. f-+ Rj Ri R; - Ri

-:

Ri Ri
Ri:

[ R; I~ [ I,-, I cR; R, [ R,
0

Definition 1.5.4. Let A a11<1 lJ be m x n matrices over a field F. We say

that A is row equivalent to D if JJ can be obtained by applying a sequence of
elementary row operations to ..\.

Example 1.5.5. The mat rkf's

2 G 0 -2
0 -2 2 6
and
-1 2 -2
2 1 0 -5
are row equivalent to one a.notlwr, IH•cause

-2R3

(-1)Rl + R4

2 6 R1 f-+ R2
0 -2
0 -2 --;. 2 6
-1 1 +----­ 2 -2
2 1 R1·f--)- R2 0 -5
(1 )R1 + R4
-1/2R3
36

Theorem 1.5.6. '·now equivalence" is an equivalence rdation. Namely

it is reflexise symrne1 rir and transtive more precisely we have:

(a) Each matrix A is row equivalent to itself

(b) If A is row eqnindl'rlt to B, then B is row equivalent toil

(c) If .-1 is row eqniYalPnt to B and JJ is row equivalent to C then .-l is row
equivaknt to C.

Proof: (a) Ohviou~ly <'arh matrix ..:1 is row equivalent to its<'lf, namely
the relation is refl<•xiw.

(b) If A is row eq 11 i n1 knt to B, say

then by the above thi'Orc'rn each elem<'ntary operation IS inv<.>rtihl<' and we

have

that is t.o say JJ is row equivalent to A. Therefor<' the relation is symmetric.

(c) If A is 1"0\V equi\·aknt to Band B is row equivalent to C say

and

then

C - "
- (.._, e • • "
(•p• • • • ,. " (A).
(•:!L•l

Hence Cis row equi\·al<•ll1 1o A and the r<'la1ion is transitive.D

 .77

This theorem allows us to usc th0 phrase "A and Bare equivalent" w]tl'll .-l
is row equivalent to B. Now we shall see that each class consisting of equivalent
matrices contains a row reduced Prlt<•lon matrix.

Lemma 1.5.7. Every matrix is row equivalent to a row reduced ecltPlon

matrix.

Pmof: Let A be any matrix, by applying some elementary row operations to

this matrix we shall reach to a row reduced echelon matrix. Consider tlH' first
nonzero column of A and consider a nonzero entry of this column, if necessary
interchange the tow of this nonzNo entry and the first row to bring tlw entry
to the leading position ( 1 ,j) of t 1H' first row and call it A 1j,. Then for <'arh
other row multiply the first row hy A!/, times its leading entry and add to
this row. Thus reduce all other entries of the first column to zero. Thus our
matrix is converted to a matrix of the form
0 0 0 Alj, ...
* *
0 0 0 0 ...
* *
0 0 0 0 ...
* *
Now fix row 1 and proceed with tlw suhmatrix consisting of remaining rows
just in the same manner by noting that the first j 1 columns will remain zero
at each further step. Apply the proc0dnrc successively. Eventually we reach
an echelon matrix. Multiply each nonzero row of this echolon matrix by the
· inverse of its leading entry to get an echelon matrix whose leading entries are
all equal to 1. Now starting with the 1a.st leading entry, successively multiply
the row of the leading entry by the negative of each other entry of its column
and add to the row of the entry. This will convert the matrix to a row reduced·
echelon matrix. 0

Example 1.5.8. Find a. row rt'dnced echelon matrix which is row equiv­
.JB

alent to
0 -1 :3 -2
0 0 -:~ 1
0 2 -:3 4
0 1 -2 1
0 1 -!} 2
Dy applying the indicat<>d row opNations W<' obtain

0 -1 :3 -2 0 -1 3 -2
(2)U 1 + R:1
0 0 -:~ 0 0 -3
(I)R 1 + H..1
0 2 -:3 .J 0 0 3 0
(l)R 1 + Rr.
0 1 -2 () 0 1 -1

0 -!} 2 0 0 -2 0

() -1 :3 -2
(l)R~+Ra
0 0 -3 1
(Itnn:! + u..,
0 0 0
( -2j;3)R~ + R5
0 0 0 -2/3
() 0 0 -2/3

0 -1 3 -2
(-2j:3)Ra+ H.·t 0 0 -3 1
2/:H?a + R., 0 0 0 1
0 0 0 0
0 0 0 0

0 1 -3 2
(-l)R1 0 0 1 -1/3
(-Ij:l)R2 0 0 0 1
0 0 0 0
0 0 0 0
 39

0 1 -3 0
(1/:l)R3+ R2 0 0 1 0
(-2)R3+Rl 0 0 0 1
---+ 0 0 0 0
0 0 0 0

0 1 0 0
0 0 1 0
(:1)R:! + R1
0 0 0 1
0 0 0 0
0 0 0 0

Messy fractions appearing in this example would be avoided by suitably

arranging the steps in the procedur<'.

ELE/\IENTARY MATRICES

Definition 1.5.9. Any matrix obtained from the identity matrix by

applying one single elementary rmv operation is called an elementary matrix.

The following theorem shows that the application of elmentary matrices

amounts to premultiplication by corresponding elementary matrices so that
when dcsir<'d elementary row opNat ions can be replaced by elementary ma­
trices.

Lemma 1.5.10. For any matrix A and any elementary row operation [
we have

£(.-1) = [(I)A
40

Proof: The result can be seen imnwdiately by writing out the Nplality in
terms of rows R 1 , R 2 , • • ·, Rm of A and r 1 , r 2 , • ··,em of I for a specifir t :·pe of
operation. For example, if[. is the operation cR; + Ri then by using Example
1.3.7 we get

R; f'; e;A R,
£(A)= and f.(J)A = A=
cR;+Rj ce;+ej cc;A+eiA r·R;+ Rj

and the C'qnality follows. ThC' same' argument appliC's to other t.ypC':-;.0

Theorent 1.5.11. If the matrix n is obtained from A by applying <'1<'­

nwntary row operations £ 1 , [..h ... , [,. 1hen

n= P ..i

whC're

is a product of elementary matricC's.

Proof: \Ve use induction on r,the nnmlH'r of operations. Let us prow th{'
particular case first. For r = 1 we haw f.d..l) = [ 1 (/)A and its validity follows
from the F)· '---------~
Assuming the equalit:·

\);1,., .... ,-,,

-,.,;.{; ,.,, ., -+ - ·, _.- ,_
" P1
L-r- 1 .•• ...... (A) =F. .......
1 (I)A
... ' - 1 (T)···F

and using the tlworC'm we get at once

~
iL,.QN\:\ !" - 4'/\
 41

The 'i'quality

is obtained by considNing the particular case A = I and we note that each

£k(I) is an elementary matrix. 0
. __....-·'"·--·---r:Jl ~'";' t
Tlds ;~orollary)says that if we apply some elementary row operations to an
m X n m~,t~i)c Aand to the m X m identity matrix I to obtain the matrices
B and P, then B = PA. Another interpretation is that if we apply some
el<:'nwnary row op<'rations to the partitioned matrix [Ali] yielding [BIP) then
B = r 1\ and tha.t P is a. product of elementary matrices. Symbolically,

[Ail]---+ [BIP] ===} B = PA

Example 1.5.12. For

[ 2 3 -1 06]
A= 1 -3 1 1 8
0 2 0 0 1
find a row reduced echolon matrix R and a product P of elementary matrices
such that R = P A. Reducing the partitioned matrix [Ali] we obtain

A=
[ 2
~ -3
3 -1 0 6 1 0

2
1 1 8 0 1
0 0 1 0 0 ~ l- [~
-3
1
9
1
0
-3 -2
1
0
8 0
1/2 0
-12
1
0
1 -2
1/; l
~ [~
-3 1 1 8 0 1
1 0 0 1/2 0 0 0]
1/2
0 -3 -2 -33/2 1 -2 -9/2

~
f~
 ~
0 1 1 19/2 0 1
3/2]
--+ [ 1 0 0 1/2 0 0 1/2
. 0 0 -3 -2 -33/2 1 -2 -9/2

INVERTIBLE 1\-IATRICES

Definition 1.5.13. /\ matrix is said to have a left im1erse if tlwre exists

a matrix A' such that _,\'.-1 = T; it is said to have a r·ight int1fTSe if th<>re exists
A' such that AA' =I. "\ rna1 rix is called an itwlrlible matrix if there exists a
matrix A' such that AA' = 1l' A =I.
vVe note first of all that an invNtible matrix A must be a squa.r(' matrix
])('cause of the equality ,\.1' =_,\'A. S('corHily tll<'re is at most one A' satisfying
rL1' = A' _.:1 = I, for if W<' had another matrix A" with AA" = A" A =I we
would have
..-1'.·1. I" = (A' A )A" = IA" = A"

,I' ,I.. I" = A'( !lA") = A'! = A'

showing that A' = A". Tlnrs t lw invcr·Hc of an i1wc1'lible matrix is uniquely

determined. It is denoted by ,.t- 1 •

Theorem 1.5.14. If .-1 1 • .-1 2 , • • ·, Ar are inv<'l'tible ma.tric<>s with inverses

A1 1 , A;- 1 , • • ·, A;:- 1 r('SIH'('t ivPI.V tlr(' product A 1 A 2 ···A,. is invertible and its
inverse is
 Proof: In the case r = 2 we have

and

establishing the assertion. The proof is completed by induction.D

Corollary 1.5.15. Every elementary matrix and every product of ele­

mentary matrices is invertible.

Proof: It is enough to prove that every elementary matrix is invertible,

because then by the theorem every product of them becomes invertible. Now,
if E is an elementary matrix say E = f(J) by Theorem 1.5.10 the inverse
elementary opera.tion can be considered and we obtain

Theorem 1.5.16. For an X n matrix A the following are equivalent:

(i) A is a product of elementary matrices.

(ii) A is invertible.

(iii) A is not row equivalent to a matrix with a zero row.

(iv) A is row equivalent to the identity matrix.

44

Pmof: We shall give a cydic proof: (i) :::> (ii) :::>(iii):::> (iv) :::> (i).

('i) :::> (ii): This is the statement of the above corollary.

(ii) :::> (iii): Suppose A is invertible and is row equivalent to matrix B

whose one row consists of zNos only. Let R; = 0 be tlte i-th row of B. Then hy
Corollary 1..5.11 we have B = P A for some P, a product of invNtible m<~t rices,
and as a prod net of invertihk matrices IJ is also invertible. \Vriting J = nn- 1

and using Example 1.3.7 we see that the i-th row of I is R;n- 1 = 0./J- 1 = 0
which is nonsense. So whPn A is invertible it cannot be row equiYaiPnt to a
matrix with a zero row.

(iii) :::> (iv): It follows from Theorem 1..5.7 that A is row equi\·aiPnl to a
row reduced echelon matrix R, and this matrix has no zero row by hypothesis.
So it has n leading entries R 1j,, R 2h, · · ·, R,j,. on its n nonz<'l"o rows. Since
R is row reduced echelon we have 1 ~ j1 < j 2 < · · · < jn ~ n and t !tis
is possible iff 1 = jll 2 = j 2, · · ·, n = jn. Thus tho loading entries or R are
R 11 = 1, R 22 = 1, · · ·, R,, = 1 and all other entries are necessarily Zf'ro, that
is to say R = I. Hence .4 is row equivalent t.o I.

(iv) :::> (i): If ;1 is assunwd to be row equivalent to I tlwn by Corollary

1.5.11 we have A = PI = P where P is a product of elementary mat rices. 0

This theorem provides us an algorithm to determine whether a given matrix

A is invertible or not and to find its inverse when it is invertible:

We write down the partitioned matrix [Ali] and apply flcmentm·y rmr op­
erations to nach mw reduced echelon matrix. Then either we ob/(Jin a ::rro
mw in the first pad at sonu· step and conrlude by (iii) that A is no! inlYrl­
iblc or we reach the identity malri:r and conclude by {iv) that ;l is inl'erliblr
 45

whose inverse is the matrix appra ring in the second block of the reduced matrix.
Symbolically we have either

[Ali] ~ [a zero row *] which means A isnoninvertible

/
or

Corollary 1.5.17. Two m X n matrices A and Bare row equivalent iff

B = PA for some invertible matrix P.

Proof: =?:If A and B are row equivalent by Corollary 1.5.11 we can write
B = P A where P is a product of dementary matrices and thus invertible.
-¢::: If B = PA for some invertible matrix P, by the fourth statement
of the above theorem P is row equivalent to I, say P = fr · · ·£2 £1 (!) then
B =:= fr · · ·£~£1 (/)A and hence hy Corollary 1.5.11 B = fr · · ·£2 £1 (A) showing
that A and B are row equivalent. 0

Theorem 1.5.18. Every matrix is row equivalent to a uniquely deter­

mined row reduced echelon matrix.

Proof: Lemma 1.5.7 a.sserts the existence part and so it remains to prove
the uniqueness. Let A be row <'quivalent to the row reduced echelon matrices
R and R' with columns J( 1 , • · ·, /\'" and J(~, • · ·, J(~ respectively. From tran­
sitivity it follows that R and R' arc equivalent and therefore by the above
·corollary R' = P R for some inwrtihle matrix P.Writing Rand R' in terms of
their columns this.yields:

(a) Ki = PK; and A'i = P- 1 Kf for each i;

(b) K, = 0 K; = 0 and h<>nre the first nonzNo columns (if any) of the
row reduced echelon matrices R and R' are the same.Thus this column
46

IS

0
= PE 1 = !\'
••
0

Further if the columns of Ih(' first k leading entries of Rand R' are identical,this
means that

0
0
1
1 0
0
E1= = PE 1 F.'2 = 0 p ~~-2 Ek = 1 = PEkl
0
0
0
0

hence any column matrix oft he form

satisfies
PI\= K = p- 1 I\·.

It follows from this thaI if one of our row n'd need echelon matrices has an­
other column Ek+l contai11ing a leading entry the other one must also contain
 47

a leading entry in the same numbered column. Now,by induction we conclude

that the columns of leading entries are corresponding columns of R and R'
,and that each column ]( of R is equal to the corresponding column P ]( of
R'. This completes the proof of R = R'. D

This theorem provides us a. systematic wa.y to determine whether two ma­

trices are row equivalent or not: em \\"hen we reduce the matrices to row
reduced echelon form if we obtain the same row reduced echelon matrix they
are row equivalent, otherwise they are not.This is proved in

Corollary 1.5.19. Two m x 11 mat rices are row equivalent iff they are
row equivalent to the same row reduced echelon matrix.

Proof. If A and B are row equiva.lent to the same row reduced echelon
matrix ,from transitivity, they are obviously row equivalent. Conversely assume
A and B are row equivalen(By the existence part of Theorem 1.5.18 ,A is row
equivalent to a row reduced echelon matrix R and B is row equivalent to a
row reduced echelon matrix R'. Tlwn W(' obtain

Jl "" R and A ,...., B ,...., R'

namely, A is row equivalent to both Rand R' .The uniqueness part of Theorem
1.5.18 shows that R = R' ,that is to sa.v .tl and B are row equivalent to the
same row reduced echelon matrix.

bf Example 1.5.20 Let us use the above result to show that

[ -11 -22 01 -25]

0 0 1 3
and l -1
-1
1 -2
2
2
0-5]

2
-1
1
2

are row equivalent matrices. vVhen we r<'d uce them we see that they are both
48

row equivalent to til(' row rC'duced echelon matrix

1 -2 0
[ 0 0 1 3
5]
0 0 0 0

and hence they are row equivalent.

 49

119j EXERCISES

1. Let
0 0 1 3 5
0 -1 1 1 2
A=
0 2 -1 1 1
0 1 0 2 3
Find an echelon matrix E which is row equivalent to A.

2. Find a row-reduced echelon matrix R which is row equivalent to

1 -1 1 3 1
1 2 -1 1 0
A=
0 3 -2 -2 -1
2 0 1 7 2

3. Find the 4 X 4 elementary matrk<'s corresponding to the elementary row

operations
so

c) R2 + R1
f) R2 ...-+ R3

4. Apply the elem<'ntary row operation

a) -2R 2 + R 1 tot h<• f'kmentary matrix corresponding to 2R 2 + R 1

b) -tR3 to the <'l<'llH'ntary matrix corresponding to -2R 3
c) R 2 ...-+ R 3 to tlH• <'kmentary matrix corresponding to itself.

l
5. Let

A= [ -: =: ~ -:
Find elementary mat ric('s F- 1 , ••• , Ek and a row redun•d <'clwlon matrix
R such that
R = Ek · · ·E1A.

6. \Vhich of the follovv·in~?; matrices are invertible?

[: -: l, [~
-1

a) h) -1
-2
:r
2
2 -1 1 -1 1 2
1 -1 2 2 1 -1 1
c) d)
0 1 -1 -;3 1 1 -1 1
1 2 -1 0 1 0 1

7. Determine wheth<'r
-1 1 2 1
1 -1 1 1
3 1 -1 1
1 1 2 -1

 is invertible or not. If it is iuvPrtihle find its inverse.

8. What must be k if
-1 k 2
2 -1 2k- 1 5
-1 1 1 -3
1 -1 k 4
is invertible. Find the inverse when k = -2.

9. If possible, write

[
1 -1

-~ -~ -~
0 l
as a product of elementary matrices corresponding to elementary row
operations of type cR; + Ri? Is it possible for elementary row operations
. of type cR; or R; .,..... Ri?

10. Prove that every elementary matrix is a product of elementary matrices

corresponding to row op('rations of type R; + cRi and cR;.

11. Let

A =[: ~: ~ ] and B =[:

-1 1

~ -~ ~ .
1 l
Solve the equation
AX=B.

l
12. Let A be an invertible matrix with

A- 1 = [ ~ -~ ~
1 -2 .t
and B = [ -~ ~ ~
-1 -1 1
]·
52

Solve the equations

a) AXJ1~ 1 = B b)A- 1 XA=B

13. Write the matrix

2 :-1 1
A= [ 3 1 -1
-1 0 1

in the form
A= P
R

where P is an inwrt ibl0 matrix and R is a row-reduced echelon ma1 rix.

l
14. Prove tl1at

A~ [ _;
2
-2
2 0 -1

.] 0
4
-2 2
,: l and n =
[ -1

0
1
-2 0 1
2 1 2 -15
0 1 3 4

arc row equiva.l<•nt and find an invertible matrix P such that 11 = P B.

15. Prove tl1at if A= PH. whH<' R is a row reduced eclwlon matrix with r
leading entries t IH'n t h<' first r <:olumns of Pare the sa.nw as columns of
A wrresponding to rolnmns of le;:.ding 0ntrics of R.Use this to wmpute
Pin

[ (~ ~ -~ ~ ~]
I 0 2 1 0
=PR,

where R is a row n•duc<'d <'chelan matrix.

 SYSTEMS OF LINEAR EQUATIONS Ei1m

A system of linear equations is a rollection of linNlr equations to be solved

simultaneously:

In order to give -a prerise definition we shall say t l1at a system of m linear

equations in n tmknowns is a pair [A, B] of matrires where A is an m X n
matrix and B is an m X 1 matrix over a field F. Surh a pair is completely
determined by by tlw partitioned matrix [AlB] wllirh is called the augmented
matrix of the systPm.Tts first part A is called the ('o<.[Jir-ient matrix for obvious
reasons. An m x 1 matrix S is called a solution of the system [AlB] if the
matrix equality AS= B is satisfied. When the system has at least one solution
we say that it is con8i8fcnt, otherwise it is said to lH' inronsistent. To indicate
the system [A-D] we u:;;ually write AX =B or we write out equations in
displayed form as ahove and thus we emphasize that t.he main problem is to
find the solutions.

Two systems are said to be equivalent if they h aYe row equivalent augmented
matrices [AlB] and [rl'l B']. If this is the case arrording to the results of the
previous section we have [A'IB'] = P[AIB] = Q[A'IB'] for some
and [;\j B]
invertible matrices P and Q. Therefore, we have A' = PA, B' = P B, A = QA'
and B = Q B' showing that

AS = B -¢:::::} PAS = PB -¢:::::} !1' S = B'

54

A'S = B' ~ QA'S = QB' ~ AS =B

This shows that eqivalent systeins have the same solutions. However the con­
verse of this statement is not true, for the systems

x+y=2 x-y=2
and

X+ y::::: -2 X- y::::: -2

have the same solution set but they are not equivalent.

The fact 1hat equivalent systems have the sanw solutions allows us to reduce
systems to si m pier ones to determine their solutions. Given a system of linear
equations we reduce its augmented matrix to row reduced echelon form (or
simply to eriH'Ion form) whose system is of the form

_.-\Ij,:l"j 1 +... : : : !J1

A2h;rh + ... = b'2

Jl,.jr:t_'jr + ••• = br

0 = br+l
0=0

0=0

This systf'III is achieved symply by applying operations corresponding to el­

ementary rmv operations to equations of the system to eliminate some un­
knowns fro111 some equations:

• add a multiple of an equation to another equation,

• int0rchange two equations,

• multiply an equation through a non z0ro scalar.

 55

The method where the system is r('(ltH·<'d to a system with an echelon matrix
(or row reduced echelon matrix) i~ calkd GAUSSIAN elimination (or GA USB­
JORDAN reduction.) Looking at 1h<' reduced system above we can get all the
information about solutions to th<' syst<'m:

(a) If br+l # 0 the equation 0 # br+l has no solution and hence the
system is inconsistent.

(b) If br+l = 0, we can expr0ss :rh, xh, · · ·, Xjr,( called basic variables) in
terms of other variables, that are call<'d free variables, by using back substitu­
tion. Now there are two cases to consi<l<'r:

(i) In the case all unknowns ar<' ha.sic variables there is no free variable
to assign arbitrary values and h<'nce there is a unique solution.

(ii) In the case there is at l0ast. one free variable we can assign them
arbitrary elements of the fi<'ld and we obtain more than one solution,
further over the field of real or complex numbers we obtain infinitely
many solutions.

Example 1.6.1. Solve th<' syst<'rn

x+y-z+t+u= 1
-x + 2y + :3.: - t + 2u = -1

2x + y - z + 2t - u = 2

x + 6y + 4.: + t + 4u = 1

8y+7z+6u= 0
3x + 7y + 3.: + 3l + 3u = 3.
56

The augmented matrix of the syste;n is

1 1 -1 1 1 1
-1 2 3 -1 2 -1
2 1 -1 2 -1 2
1 6 4 1 4 1
0 8 7 0 6 0
3 7 ;3 3 ;3 3

and by a.pplyi ng <'lcnH'Ii1 a ry row operations it can be reduc0d for <-'X am plc to

1 -1 1 1
0 -1 1 0 -3 0
0 0 5 0 -6 0
0 0 0 0 0 0
0 0 0 0 0 0
0 0 0 0 0 0

Thus the Gaussian eli 111 i 11 a lion reduces the system to

;r+y-z+t+u=1
-y + z- 3u = 0
5z- Gu = 0
0=0
0=0
0 = 0.

\Ve see that the syst0m is consistent with basic va.riabl0s x, y, z and free
 57

variables t, tt. By back substitution WC' obtain

z = 6/5u
y = z- 3u = (6j.5n)- 3u = -9/5u
x = 1- y + z- t - 11 = 1 - ( -9/5)u + (6j5u)- t- tt
= 1- t + 2u.
Thus the general solution of th0 syst0m is of the form

1- t + 2u
1 -1 2
-9/5u
0 0 -9/5
6/5u
0 +t 0 +u 6/5
t
0 1 0
u
0 0 1

Over the field of real numhNs, for any choice of t and tt we obtain a solu­
tion, so the system has infinitely many solutions.

Example 1.6.2. Find a:, y and z if

1 1 1 1
1 -1 1 0
X +y +z
3 1 3 2
1 1 -1 5

This matrix equation gives us a syst0m with augmented matrix

1 1 1 1
1 -1 1 0
3 1 3 2
1 1 -1 5
/j8

\vltich can he reduced to

1 1 1 1
0 -2 0 -1
0 0 1 -2
0 0 0 0

Thus t.he system reduces to

:r+y+z=1
-2y = -1
z = -2,

and gives us
X= 5/2 , y = 1/2 , Z = -2.

Example 1.6.3. Find the values of a for which the system

2;r - y + 2az +t = b 2x- y + (2a + 1)z +(a+ l )l = 0

-2x + y + (1- 2a)z--:- 2t = -2b- 2

has:

(a) A unique solution.

(b) Infinitely many solutions.

(c) No solution.

Forming its augmented matrix and applying elementary row operations one
g<•ts
 1
a
a+1

VVe see that the system is inconsistent when a= -1, it is consistent and has
infinitely many solutions when o :;!:. -1. a has no value for which the system
has a unique solution.

Example 1.6.4. Find tlH' rP<li numbers x,y,z and t if

+ y + zt = 0
-3~-r

3.1: + y(l - 1) - z( t + 1) = 0

<l.r + y + zt = t + 2
-3:r + y(l + 1) + zt = t + 5.

Note that this system of equations ran be considered to be a system of linear

equations with augmented matrix

-3 1 0
3 (/- 1) -t- 1 0
-3 1 t t+2
-3 (I+ 1) t t+5

which can be reduced to

-3 1 t 0
0 -1
0 0 1 3
0 0 0 t + 2.
60

As is seen,the systnn is consistent only in the case t = -2. For this value we
ha.ve x = -5/2, y = -:l/2 and z = 3. Thus the unique solution of the system
is obtained as

X= -G/2, y = -3/2, z = 3, t = -2.

SYSTE!viS OF liOIIIOGBNBOUS EQUATIONS

A system of litwar <'qnations of the form AX =0 is called a homogeneous

system or a system of homogeneous equations. Obviously, AO =0 that is to
say

0
0

is always a. solution to 1he homogeneous system; it is called lhr lrh,ial.~olution.

Other solutions nam<'I.Y solutions having at kast one nonzero <'ntry are called
nontril'ial' solutions. i\on1 rivia.l solutions exist only in the case tlte system
has more than one solution, that is to say in the case there is at kast one free
variable. If this is the cas<' we fix each free variable once, assign it 1 and assign
all other free variahks z<>ro ~o get the so called fundamental .<;ofution.<:.
 Example 1.6.5. Solve the homogeneous system

:r+y-z+t+'U=O
-x + 2y + 3z- t + 2'U = 0
2J' + y - z + 2t - 'U = 0
x + Gy + 4z + t + 4'U = 0
8y+7z+6'U=0
3.t: + 7y + 3z + 3t + 3'U = 0.
vVe sha.ll reduce its augmented matrix, but for a homogeneous system the last
column consists of zeros and does not change under elementary row opera.tons
and so may be neglected. Now, neglecting the last column we reduce

1 1 -1 1 1 1 1 -1 1 1

-1 2 3 -1 2 0 -1 1 0 -3
2 1 -1 2 -1 0 0 5 0 -6
to
1 6 4 1 4 0 0 0 0 0
0 8 7 0 G 0 0 0 0 0
3 7 3 3 3 0 0 0 0 0

Thus the Gauss-Jordan reduction reduces the system to

x+y-z+t+u=O
-y + z- 3u = 0
5z- 61t =0
0=0
0=0
0 = 0.
giving
:::=6j':)u

y = -9/Gn x = -t + 2·u.

62

and the general solution

-t + 211 -1 2
-Hj':iu 0 -9j.5
Gj;iu = t 0 +u
6/5
1 0
'II 0 1.

Here, the free variahi!'S a rP u and t, the assignment ·u = 1, t = 0 yiPlds the

fundanwntal solution
2
-9/.5
6/5
0

the assignment t = L u = 0 yi<>lds the fundamental solution

-1
0
0
1
0

Examples 1.6.6. \Yl1at must he k if the following homog0neous system

has nontrivial solutions?

x- + 2z - t = 0
~:y

- 2.t + 2ky - 2z + 3t = 0
-:r + (~: + 1 )y + (k + 1 )t = 0
-:l.r + :Jky - 2z + (k + 4 )l = 0
 68

Applying the Gaussian elimination its coefficient matrix is reduced to

1 -k 2 -1
0 1 2 k-1
0 0 2 1
0 0 0 k-1

This shows that the system has nontrivial solutions only in the case k = 1.
· By back substitution we obtain the solutions
X 3
y 1
.X= =t
z -1/2
1

lNVERTIBILITY AND SYSTEMS OF LINEAR EQUATIONS

We can characterize invertible matrices conveniently by means of systems

of linear equations.

Theorem 1.6.7. For a square matrix A the following are equivalent:

(i) A has a left inverse.

(ii) The homogeneous system AX = 0 has only the trivial solution.

(iii) A is invertible.

(iv) A has a right inverse.

Proof. ( i) :::} ( ii) : Let A have a left inverse A' and let S be a solution of
A.Y = 0. Then AS= 0 implies A' AS= A'O =0 giving IS= S = 0 that is
to say I he only solution is the trivial solution.
64

(ii) ::::} ('iii) : Suppose the homogf'JH'ous sytem ha.s only the tri\·ial so­
lution. Let R be the row reduced PrhPlon matrix which is row eqniYaJ,.nt to
A. Then RX =0 has is equivaknt to AX =0 and hence it also has only
the trivial solution; this m0ans that. all rows of the row reduced matrix R <nP
nonzero consequently R = I. Thus A is row equivalent to J and by Tlii'Ol"l'lll
1.5.16 ( iv) it is invertible.

(iii) ::::} ( i) : Obvious.

(iii)::::} (iv): Obvious.

(iv)::::} (iii): Let A hav0 a right inwrse A', then A' has kft inv0rs1' .-\.so
by the equivalence of (i) and (iii) proYNl above A' is invertible wi1l1 in\"I'I"SI'
say A". Now, we have

A= AT= A(A'A") = (~IA')A" = IA" =A"

giving .il'A = A'A" =I= AA'.D

As a consequence of this theor0m W<' ran also give

Corollary 1.6.8. For a. square matrix ..-1, the following are equvaJ,nt:

(i) .4 is invertible.

(ii) Every system AX= B has a 11nicpw solution.

(iii) Every system AX= B has at most one solution.

(iv) Tllf're is a system AX= B which has a unique solution.

Proof: ( i) ::::} (ii) : If A is an invertible matrix then, A( A - l fl) = n sl10ws

that S = A- 1 B is a. solution. Further, if Sis any solution, from :IS= nit
follows that S = A- 1 JJ, that _is, Sis the unique solution.
 65

(ii) :::::? (iii) : Obvious.

(iii) :::::? ( iv) : Supposing every system has at most one solution we con­
clude that AX= 0 l1as a unique solution (i.e.1l1e trivial solution).

(iv) :::::? ( i) : Let B be a column matrix for which the system AX = B

has a unique solution S, then we claim that tlH'! Jwmogeneous system AX= 0
cannot have nontrivial solutions because otherwise any non-zero S' satisfying
AS'= 0 would give J\(8 + 8') =AS+ AS'= IJ +0 =Band hence it would
yield the solution S + S' of AX = B ot.lH'r than S. Now it follows from the
above theorem tl1at :1 is invertible. D

Example 1.6.9. tct us use Theorem 1.6.6 to prove that the matrix

1 1/2 1/3 1/n

1/2 1/3 1/-1 1/n -1
A=

1/n 1/n+1 1/n+2 ... 1/2n-1

is invertible. For t.his purpose it suffices to show that the only solution of the
homogeneous system il.Y = 0 is tl1c tri\·ial solution. Suppose

l'n-1

is a solution of AX= 0. Then for earh k = 1, 2, · · ·, n we have

1 1 1 1
-co
k
+ -k+1
- c 1 + --c., + ·· · +
k+2- k+n-1
Cn-1 = 0

and using the fact 1I. at

1
xrd.r= - ­
r+1

~----------------------------------------------------------
~

i:"
~
66

these <'qua lit i<>s ran hc> put. into t.he form

or
t xk- 1(r·o + +r 1x + c x
Jo 2 2 + ·· · + Cn_ 1x"- 1)dJ: = 0.

1 1
J(:t)d.e = 0,1 1
J:f( :r )tl.r = 0,1 1
a.· 2 f( x )dx = 0, ···, 1 1
x"- 1 f(x )d:r = 0
Jv1ult.ipl,\"ing th<'m by r· 0 .c· 1.· .. ,c,_ 1 r<>spect.ively and adding we obtain

which is possible only if f(.r) = 0, <'quivalently c0 = c1 = · · · = c,_ 1 = 0. This

rom pl<'t.<'S t h<' proof.
D

LU -f'ACTORIZATIO:\

In som<' cases au in\'<'1'1 ihl<' matrix A can be written in the form

A= LU

where L is a lower triangular matrix whose diagonal entries are all equal to
· 1 and F is an upp<'r triangular matrix. If this is the case A is said to have
an LU -far-tori:::ation or a11 ur -t!r-romposition. Note that, if i1 can he reduced
to an echPJ.on matrix F h~· applying elem<'nt.ary row operations .of the form
cR; + Ri wit It i >j t.hPn .-I has an LU decomposition, because then U =KA
where [,· is obtained frolll I by applying th<' same operations and hence J(
and its· invNS<' L = A'- 1 ar<' uppN triangular matrices whos<' diagonal entries
 67

are equal to 1, and tlwy satisfy;\= l\"- 1 U = LU. Conversely, if A has an LU­
decomposition A = LU then L ran h<' reduced to I by applying operations
of type cR; + Ri, (i > j), and hy th!.' algorithm following Theorem 1..5.16 the
same operations applied to I yields L- 1 • So, if we apply these operations to
A we get L- 1A = L- 1 LU = U,t.hat is to say A can be reduced to an (echelon)
upper triangular matrix by means of elementary row operations of the form
cRi + Ri with i > j. In this way \V<' also see that to determine L and U we
reduce the partition<'<! matrix (AJI] hy applying elementary row operations of
type indicated ahovf' and get [UIL- 1]. Inverting the second part we ohtain U.
They can also be computed effici<'ntly by writing out

A11 ;112 ;\In 1 0 0 Uu [112 U1n

An A22 A2n [,21 1 0 0 [122 U2n

An1 An2 ... Ann Lnt Ln2 ... 1 0 0 . .. Unn

and solving the r<'snlt ing equations.

The use of LU-<kromposition li<'S in the fact that,given a system AX= B

with A = LU WC' ran consider tlw syst<'ms

LY =n UX=Y

of which the former can be soln'd hy back substitution and the latter by
forward substitution. For solving large systems computer programs for LU­
decomposition are available.

Example 1.6.10. Find the LU-d<'composition of

68

r -~4 -~ ~ ]·

3 -1
and use this to solve
x- y + 2z = 4
-2x + y = -1
:t.r + 3y - z = 2.
To find the LU-decomposi1 ion we make the following reduction:

.
r -~
4
-~ ~ ~ ~ ~ 1 r ~ =~
3 -1 0 0 1 0 7

~]
1 -1 2 1 0
[ 0 -1 :J 2 1
0 0 19 10 7
Thus

r 10~ ~ ~]·
u = r ()
l -1 2]
-I -1
() 0 19 7 1
Inverting L- 1 we obtain

[, = r -~ ~ ~ J.
4 -7 1
Now, to solve AX= JJ W(' solve the equations LY =Band UX = Y, namely

u =4 X- y +2z = ll

-2ll + l: = -1 and -y + '1Z = v

4·u- fl' + IP = 2. 19z = 'W.
 6.9

They yield u = 4, v = 7, w 14; z 14/19, y = 77/19, X -25/19.

70

EXERCISES ¥9

1. Find all solutions,if any exists, of the system

-3x + 2y - z + t = 5
x-y+z-t=1
2x- 3y + 3t = 0
-:r - y- z + 4t = 5
.1: - 4y - z + 7t = 5

2. Solve the following system if it is consistent:

x+y-z+t=2
2x - y + 2z - t = -1
-x - 4y + 5z - 4t = 0
:r - y + 4z - t = 1

3. Find real numbers .T, y and z satisfying

1 1 1
-1 1 0
:I; +y +z

:3 1 3 2
1 -1 5

4. Solve the system whose augmented matrix is

3 -2 3 -4 1 3
2 1 -1 -2 1 0 0
-1 4 1 -.5 3 G 1
2 8 -2 -4 0 7 4
5. Find all solutions of the following system over Z 2 :

:r+y+z=1
x+y+t=O
y+z+t=1
y+t=1
:r+z+t=O

6. Solve
x+y-z+t=2
2;t·- y + 2z - t = -1
-x-·1y+5z-4t=0
x - y + ·Lo - t = 1

7. \Vha.t must be the value of a if t hNe are numbers x, y and z such that

1 2
1 a+1
-1 0 a
X +y +z
3 -1
1 -1
2 0 -a

8. Find the values of a for which the system

2x- y + 2az +t = b
-2x + ay- 3z = 4
2x- y + (2a + l)z +(a+ l)t = 0
-2;r + y + (1 - 2o )z- 2t = -2b- 2

has:

(a) A unique solution.

 (b) Infinitely man:• solutions.

(c) No solution.

9. Solve the homog<'IH'ous system

-X + 2y + Z - 2t = 0
-X- y + Z + t = 0
x+y+z+l=O
-:r + 5y + 4z - 5t = 0
2:1' - y + z +t = 0

10. Find the general sol11tion of the following homogeneous system in terms
of its fundamental solutions:

a.:+y-z+t+u=O
+ 2y - 2z + 3t = 0
x
- . T - 2y + 4z- 7l + u = 0

x-t+u=O
2.1' + 2y - 2z + 2/ + u = 0

11. Find fundamental solutions of

+ x 2 - x 3 - x 4 + X5 = 0
:r 1

2xt + 2a:2 - + X5 = 0
X4

:l't + a:2 + x 3 - 2:1: 4 + X5 = 0

-:t, - ;1" 2 + 3x 3 + 2;1: 4 - 2:r5 = 0

2:r 3 - x4 = 0
.r 1 + 2 + 5:r 3 + 2.1: 4 -
J" 2x5 = 0

12. Find fundamental solutions of the homogeneous systems whose coeffi­

 dent matrices are

0 0 0 3 -4 1 3 0 0 0 0 1 -2 3 -1
0 0 0 -2 1 0 0 0 0 o· o 2 -4 6 -3
(a) (b)
0 0 0 -5 5 -1 -3 0 0 0 0 1 -2 1 2
0 0 0 -4 2 0 0 0 0 0 0 0 0 0 7

13. What must be the values oft if the following homogeneous syst('m has
nontrivial solutions?

+ tx 2 - 3x 4 =
x1 0
(t- 1)xl- (t + 1)x2 + 3x4 = 0
x 1 + tx2 + (t + 2)xa- 3x4 = 0
(t + 1)x + tx 2 + (t + 5)xa- 3x4 =
1 0

14. What must be kif the matrix

-1 /.; 3 -2
2 1 1 k
is invertible?
1 k+l -l k-2
2 1 1 k~ +1
15. Solve the equation AX = n for

-1 2 5 ;3 2 1 0 -1 1 2
5 3 1 -3 1 -1 -1 1 -2 1
A= fl=
1 -1 2 -2 1 1 1 -2 -2 -2
0 1 0 1 1 0 1 0 1 ·o

16. Prove:

(a) An m X n matrix with m < n has no left inverse.

(b) An m X n matrix wit 11 m > n has no right inverse.

 Deduce that a. matrix which.ha.s a. right inverse and a. left illVI'J"SP should
be an (square) im·Prtihle matrix.

17. Prove that if . . 1 is an n X n matrix which is not invertihl<' t lt<'n there is

a. square matrix lJ wi t.h a.t least n nonzero entries such that

AB = 0. Dcterm i I}(' all such matrices over z3 for

18. Find the L[T -d<'rnmposition of

2 1 -1 1
-6 0 -1 -4
A=
4 -1 2 3
0 -3 0 9

and use this d<'romposition to solve the system AX =B where

9
4
B=
-3
0

19. Show that B = [ ~ ~] has no LU-decomposition.

 75

ID DETERMINANTS
Orw of the most useful devices oflinear algebra is the determinant function
that assings a scalar to each square matrix and even to some other concepts
wldch can be represented by matrices. vVe shall introduce them ind nctively
(in the way almost all readers are aquainted with from introductory courses
without using the adverb "inductively"), namely, first we definC' detNminants
of 1 X 1 matrices, then using this definition we define determinants of 2 X 2
matrices, then use this to define it for 3 X 3 matrices etc.... The JHOCC'dure will
bC' formalized below by considering submatrices obtained by delC'ting the first
row and one of the columns.

By 1lU) we mean the ( n- 1) X ( n - 1) matrix obtained from A by deleting

t lw row and column of A 1i and we call it the minor of A 1i.

For example if we take

A~[
l

;\ ll A12 At3

.:121 An A23

l\ :n A32 A33

thC' minor of A11 is

76

and similarly minors of i1 1:! and A 13 are

A_(3) =

Definition 1.7.1. Tit<• function D,. defined on n X n matric<'s illdnctiY<•I,v

by the following equa.litiPs is railed the determinant function:
n

Dl([Au]) = A11 and D11 (A) = 2::( -l)l+j A1jD 71 _ 1 (;1Ul) for 11 >1
j=l

where }iUl is the minor of :lti·

The value of D .. at ;\ is <lPnoted by any one of the following not at ions:

det(A) , lA I,

(Here Rh ... , Rn stand fort h0 rows of A.)

Example 1.7.2. L<•t ns use this definition to give the well-known explicite
formulae for determinants of 2 X 2 and 3 X 3 matrices. From the defining
equality we obtain for a 2 x 1 matrix A that

Thus
 77

Now using this determinant function for 2 X 2 matrices we shall obtain the
formula for the determinant function on 3 X 3 mat rices again by using the
defining relation above:

An A12 A13
A22 A23 A21 i\23 A21 An
A21 A22 A23 =An - A12 + A13
A32 A33 A31 A33 A31 A32
A31 A32 · A33

The computation can be made as the above arrows Rnggest, but this method
of computation, which is known as Sarrus' rule, is valid for the determinants
of 3 X 3 matrices only and it shouldn't be overeslimalcd.

Example 1.7.3. Let us use the definition to show that the determinant
of a triangular matrix is the product of its diagonal entries.We handle lower
triangular matrices and leave upper triangular matrices to the reader. The
statement is obvious for 1 x 1 matrices. Supposing it is true for ( n -1) x ( n -1)
lower triangular matrices we obtain

0
An 0
0
=An +0. +0. + ...
* *
A32 A33
An3
= Au(An···Ann) = AnAn···Ann as required. In par­
ticular we have
det(J) = 1.
78

Now, we shall derive a. formula to express Dn(A), for n > 2, in terms of the
first two rows of A. Using the defining relation we get

n n n-1 - (k)
Dn(A) = L(-1) 1+i A1i Dn-d _:pn) = LH) t+i Ati(L(-1) 1+k(.:f(n)lkDn-1 (AU)) ).
i=l j=1 k=l

But by considering tlw f'nt ri<'s of .~Tu l we s<>e that the ('ntries of the first row
are the entries of the second row of A and so they can be expressed by

( .f':'l(j)). lk = ·t 2k
.f if k<j and ( -:-tU)) 1k-1 -- A 'lk
j' if k >j

Further the (l,j)-minor of this matrix is obained by deleting rowl, row2,

columnk and columnj in <>ith('r cas<> k < j or k > j; d<'note it by A(ki),

Therefore we have

n j-1
D,(A) = L L( -1) 1 +i+l+~·AtjA'2kDn-'2(fp~·j))
j=l A·=l

n n
+L L (-1 )l+i+I+(A·- 1 ) A1jA2kDn-2(rT(kj))
i=l k=i+1

-- """'(
L..J - l)i+k .t-llj~'~2k
J " n· 11-2 (,--:--t'•il) + ""c·
L..J - l)l+Hk
. i t lj A 2k D n-2 (A-(kj))
·
k<j k>j

Now, interchanging k and j in he s<'cond snm we can combine them a.s

D,;(A) = L( -1)i+"'(:1w\2k- A1kA2i)Dn-2(A(ki>)

k<j

and get the required fonmlla.

Theorem 1.7.4. (A) D, is an n-linrar function on n x n matrices, that

is to say if the i-th row of a square matrix is of the form cR; + dR~ \vhere R;
 7.9

and R~ are 1 X n matrices then we have

cR; + dR; +d R~
'

(B) D, is alternating, that is to say, the determinant of any matrix with

two identical rows is equal to ZNO.

Further if a multiple of one row is added to another row the determinant

does not change; and if two rows a r0 interchanged the determinant is multiplied
by (-1).

Proof: If we can prove (A) a11d (B) the last statement will follow at once
from

RI Rt R1

R;
R; R;
= c +

cR;+ Hj
R; R-J

Rn
R, Rn
"-v-"

,,!:;.$!

~
80

and

0= + + +
R;

R,
where undNhraccd dctNminants are equal to zero. So, all we have to do is to
prove (A) and (B). This \Vill be done by induction. The case n = 1 is obvious
for both statements. So to prove (A) assume Dn_ 1 is an (n-1)-lincar function
and take A to be an 11 X n matrix whose i-th row is of the form cR; + d R~. Let
B and B' be matrices obtained from A by replacing its i-th row by R; and R;
respectively. If i = 1 the defining relation takes the form
n
Dn(,.\) = 2)-1)1+i(cAli + dA~i)Dn-1(JIUl)
i==l
and decomposing it gin's D,(A) = cDn(B) + dD,(B') as required. If i > 1
one row of each fl(j) will he of the form cR; + dR~ and will he decomposed
into two parts of the required type, consequently (A) will follow.

To prove (B) forD" hy assuming it is true for Dn_ 1 we take ann X n matrix
whose i-th and k-th rows are identical and consider three cases separately:

(a) If i > 1 and k > 1 in the defining equality of Dn(A) each matrix JlU)
is to contain two identical rows and by induction their determinants are
all equal to zero, hence Dn (A) = 0.
(b) If i = 1, k = 2,t ll<'ll D,(A) = 0 follows at once from the formula estab­
lished above givi11g the determinant in terms of its first t\VO rows and
 81

their complements.

(r) If i = 1, k > 2 we form ~ ll<'W matrix B from A by interchanging its

second and k- th rows and we observe that

n n
Dn(B) = L( -l)l+i(Bli )Dn-t(_B(i)) = L( -1) 1+i(Ali)( -Dn-t(fi'U))) = -Dn(A)
i=1 i=1

since each _8(i) can be oht ained from A(j) by interchanging rowl and
row(k-1) and'since the assertion is assumed to be true for Dn-l· Now,
we get Dn(A) = -D,(B) = 0 hy the case (b). D

Especially the last part of this t h<'orem shows the effect of elementary row ·

operations on determinants and allows us to state

Corollary 1.7.5. If£ is an <>l<>mentary row operation corresponding to

the elementary matrix E, then

(a) det(E) =I= 0

(b) det(E) = det(ET)

(c) det(f.(A)) = det(EA) = dP1( E) det(A) for any square matrix A.

Proof: (i) If £ is the el<'m<'ntary row operation cR; + Ri then by the

last part of the theorem det(F.) = det(J) = 1 and det(£(R)) E = det(A)

showing that the validity of (a) and (c). As for (b), ET is the elementary

matrix corresponding to cRi + Ri and it is of determinant 1 as well.

l
i

.
 82

(ii) If t: is the <'I<'Jil('Jltary row operation R; H- Rh then ET =E and the

rest follows from the last part of the theorem as above.

(iii) If t: is the <'klll('ntary row operation cR;, then ET =E and using

(A) with d = 0 one ohtains (a) and (c).

Corollary 1. 7 .6. For a. square matrix A we have

(a) det(A) = 0 if A has a zero row,

(c) A is invertible iff det( A) ¥: 0,

(d)

det(AT) = det(A).

Proof: (a.) It is d('<H from Theorem 1. 7.5 (A) for c = d = 0.

(b) This is clear from Corollary 1.7.4 (c).

(c) We apply som<' <'l<'mcnta.ry ro)V operations corresponding to el<'mentary

matrices Et, E 2 , • • ·, F:,. say, to reduce A to a. row-reduced echelon matrix R.

Then, by Corollary 1.7.5

det(A) = det(ErEr-1 · · · E~E 1 R) = det(Er) det(Er-d · · ·det(E2) det(Et) det(R).

·Therefore, when A is inwrtible R =I giving det(R) = 1 and det(A) ¥: 0;

however when A isnot invertible R has a zero row, by (a.) det(R) = 0 implying
det(A) = 0.
(d) Suppose first that A is not invertible. Then AT is not invertible

and hence det( A) = ckt( ;1T) = 0. Secondly, if A is invertible we have

A = E1E 2 • • • Er for some el~mentary matrices Et, E 2 , • • •, Er and therefore

 8S

using (b) and Corollary 1.7.5 (h) we get

det(A) = det(EI) det(E2 ) .. • <kt( Er) = det(Er) .. ·det(E2 ) det(E!)

det(E;) .. · det(E{) <l<'t(E{j = det(E; .. · Ef E[) = det(AT)

Corollary 1.7.7. (a.) The ddwminant of a product is the product of

determinants:
det( A B) = <l<'t( A) det( B)

(b )If A and B are square matrices,t Ill'II for the matrices in block form we have

det [ CA0] = dtl [A Cl = det(A)det(B)

B 0 B

Proof: For the first part we consider two cases separately:

Case 1. If A is invertible, it can be written as a product of elementary

matrices, say A= E 1 E 2 • • ·E,., and hy the above corollary we have det(AB) =
det(E 1 E 2 ···ErE)= det(E 1 )det(E2 ) • .. det(Er)det(B) = det(A)det(B).

Case 2. If A is not invertibl('. tl1Pn A B cannot be invertible and therefore

their determinants are both zero a\cording to the above corollary and the as­
sertion follows from det(AB) = 0 and det(A) det(B) = 0. det(A) = 0.0

As for the second part letting [) = [ 0A C

B l and considering inverses of

elementary row operations reducing A. l3 and D to echelon form we note that

84

where A-, B- and D- an' upper triangular (in fact,echclon) ma.trices.Furth<>r

'<·.
we have det( n-) = dil( '"1- )del( B-). Ndw the proof is completed at once hy
using Part(a).

Theorem 1.7.'1 can also he used to give an explicit expression of th<' d<>­
termina.nt of a. matrix in terms of its entries.In fact using this th<>on•m we
conclude that

• if one row is oft h<' form c 1 R 1 + c2 R 2 + · · ·+ c,.Rn,then the cktNminant.

of the matrix is ·

* * * *

= Cl + ···C,
R,

• if m intcrchang<>s ar<' applied the det<>rminant is multiplied by ( -l)m .

• if there is a. reJwating row ,the det<>rminant is zero.

Thus letting 1 1 , I".· · ·, ! 11 for the rows of the identity matrix I earl1 row of
an n X n matrix"' can be written in the form
n
Rk = .-1,.1/1 + Ak2f2 + · •· + Aknln = L Ak;J;k
ik=l
 85

and hence repeated application of tlw above proc<>d ure yields

Rt

2:::7~=1 AJ;,l;. R2
R'2

R3
det(A) = Ra

Rn
R,

Tl

L·Ali 1 R3
it=l

L A!i,A2i2 R3
i1,i2=1

I;l

I;,
= L Ali,A;,···Anin J;3
i1,i2 1 ···,in
86

On tlH' otlH'r hand, the determinant

is either 0 or it is of the form ( -1 )111 .Ir there are any two identical indices ik
and i 1 ,then two rows are identical and t h<' determinant is O,if all indices arc
distinct by a number of interchan)!;<'S it can he converted to the determinant
of the i<l<'ntity matrix I and thus it is Pqual to ( -1 )"',where m is the numlwr
of interchanges to bring i 1 , £2 , • • ·, i" to t h<' nat ural order 1, 2, · · ·, n,for which
we also conclude that its parity is ind<'fli'IHknt of the way the interchanges are
carried out.Therefore b.Y coHsid<'rin)!; thi' function a from the set {1,2, .. ·,n}
into itself defined by

O"(n)

and letting
.sng( O") = (- 1)'"
we obtain
dct(A) = 2:sgn(O").I," 1 'l+la(:?)'''/l,,.(,)
"
where O" ranges over all functions from { l. '2, · · ·, 11} into itself such that a(k) f:.
O"(l) when k f:.l. Such functions are callPd jJ(I'mutations on {1,2, .. ·,n} and
.sng(O") is called the sign of the pcrnwfalion.ln order to indicate tlre images in
an explicit form we write

(]' = ( 1
Jl /2 ::)
 87

Thus we have proved the following

Corollary 1.7.8. The determinant of ann X n matrix A is equal to

when> <T ranges ovN all perm11t.ations on {1,2,···,n}.D

For example on the set {1, 2} tl1NP are two permutations:

<=(: ~) and a=(1

2 1
2)
and we have

COFACTOR EXPANSIONS AND Cm.IPUTATIONAL PROPERTIES

Consider a square matrix rl and pick up one of its rows, say rowk. In­
terchanging this row with the otw prior to it successively eventully we get a
matrix B whose first row is tlw k-1 h mw of 11, and its determinant becomes
(-1)k-l by Theorem 1.7.4 on one l1and and on the other hand it is

n -(A·j)
det(B) = L .\kj( -1) 1 +i det(A ),
j=l

for its first row consists of Akj's and their minors in Bare the same as tlH)se
in A. This shows that

n . -::_(kj)
det(A)=L·'kj(-1)"'+1 det(A )
_j=l
88

In this sum tlt0 factor

·\In

which is mnltipli<'<l I>~· . \kj is call<'d th<' <·ofl/("/or of Akj· The matrix adj(.·\)
ddi1wd by

is called the (l(Uoint of .I. In terms of tlt<'S<' rofactors the expansion takes the
form
11

<lPt( A) = L. \kj(/kj
j=l

\Yh<'ll the <:'ntrics of th<' /,·-throw of A is rPplacC'd hy :r1,:r 2 , ••• ,:r 11 th0 dctPr­
minant of the r0su1t in!!; matrix h0comPs
1l

2:::: :rjakj
i=l

m particular the expr<'ssion

n

L A;jakj for i =/:. k

j=l

is the det<'rminant of a matrix with two id<'nt ira] rows R; = R.., IH'nce we have
11

L ..-\;jfl!-j = 0 for i =f k.
j=l

Thus we see that

7l
if i =k
L ,·\;ja!·j
if i =/:. k
j=l
Sinre n n
L:A;jakj == I.>tii(adj(A))jk == (A(adj(A)))ik
j=l j=l

this PC]ll ality is equiva.lent to

Aorlj( A)= det(A)J

By intNpreting Corollary 1.7.G(d) that any property given for rows can aL<>o
be gircn for columns, we obtain the column expansions:
n
d('t (A) = ".L:>-hjakj
k=l

0 = LA;jakj,(i -:f. k)
k=l

and hence the matrix equation

arlj( J )il == det(A)J

In particular if det(A) -:f. 0 then matrix equalities tha.t we have obtained can
be put to the form

and it yields
A- 1 = (Ict~A) adj(A).
Thus we have proved the following

Theorem 1.7.9. For any sq11are matrix A we have

n { det(A) if i = k
'L:A;ja!·j =
j=l 0 if i -:f. k

equivalently
Aadj(A) = adj( A)A = det(A)I.
.?0

In particular the cofactor expansions with respPrt to any row (and any column
an') valid:

If dd( A) f. 0 the inverse of A is

1 l OCJ(.!
,\ -1 = (f.r:l(.! [' l'). [=>
JOJ.~
)

An Application: CtL<HlEH's RULE

Let AX= B be a syst<'lll of linear ('quat ions so that A is a. square matrix.

Now, the above theor<'lll shows that for an.v solution X we have

AX= n => rf(fj(.\ ).\X= adj(.4)!J => drt(il)X = adj(A)TJ

Now, letting Dk = b 1aa + b'.!a:H: + · · · + bna,k and noting that it is tlw deter­
minant of the matrix oht a i lll'd from A by n'placing the k- th col u 111 n by 17 W<'

obtain the explicit fonn11la

k = 1,2, ... ,n
or
1);.
./k- / ) k = 1,2, ... ,n

where D = drt(A). This nwtl1od is known as CRAMER's nile.

Example 1.7.10. L<'l us solv<' the S!'St<'m

2:r- y + z = 0
.r+y-z =G
l.r - !J.IJ + ;Jz = 28
 91

We have
2 -1 1

D= 1 1 -1
= -6,
4 -5 3

0 -1 1

D!= 6 1 -1
= -12,
28 -5 3

2 0 1

D3= 1 6 -1
= 96,

4 28 3

2 -1 0

Da= 1 1 6
= 120.
4 -5 28

Therefore,

D1 -12 D., 96

x- -D- -- - z = Da = 120 = -20.

-6- -2 '
y=-~=-=-16
D -6 ' D -6
92
IWI EXERCISES

1. Compute the following determinants

'
1 0. 0 0 0
7 -5 0 0 0
2 2 a -3
(a) (b) (c) -9 6 4 0 0
-3 6 ' X b
3 -2 0 1 0
-2 3 2 5 -6

2. Solve the following equations in Z 5

X 0 0 0
2x x2 + 1 X
X x2 + 1 0 0
(a) =0 (b) X x2 + 1 X =0
3x- 5 x+1 X 0
3x- 5 x+1 4x- 5
1 -x -3 x+2
94

3. Given that
1 a 2
-1 1 b = 4,
a 2 3b

compute the following determinants by using properties of det function:

x a+ bx 2
(a) -x 1 b (b) -x 1-bx b
ax 2 3b ax 2 + abx 3b

-1 -2a -2 a+ 1 a+ 2 2 + 3b
(c) -2 4 2b (d) -1 1 b
a 4 3b a· 2 3b

4. Compute the determinant of each of the following permutation matrices

and interpret the r<'sults:

0.1
0 0
1 0
l (b)
0 1 0 0
1 0 0 0
0 0 0 1
0 0 1 0
(c)
0 0 0 0 1
0 0 0 1 0
0 0
0 1 0 0 0
1 0 0

1 0 0 0 0

5. Evaluate the detNminants of

2 -5 4 -1 1
1 -3 3 -4
-7 3 -2 1 0
2 1 -1 -5
(a) 0 0 0 0 0 (b)
-3 9 -9 12

0 ·11 10 0 0
7 1 -1 5

12 10 0 0 -20

 95

7. Solve the following equation over Z 7

x+1 x+1 1
-2.1:-3 -X -3 = 3
1 0 1

8. Let E be an idempotent matrix (i.e E 2 = E) different from I. Prove

that det(E) = 0.

9. Show that if c is a scalar and A is an n X n matrix, then dd( cA) =

cndet(A).

10. Show that the determinant. of any skew-symmetric matrix of odd order
is zero.

11. If Q is an orthogonal matrix (i.e. QT = Q- 1 ), then det(Q) = +1.

96

12. Verify that

b c o d e f
a 0 c X y z
b2 + ac be c2
0 a b u v w
ab 2ac be = b 0
., 0 0 0 c
a- ab b2 + ac
0 0 0 a 0 c

0 0 0 0 a b

and compute th<' common value.

13. Compute:
1 + :l"J a:1 Xt Xt

.7'2 1 + ,1:2 x2 x2

:Z':J x3 1 + X3 x3

;r" Xn Xn ... 1 + Xn

14. Compute:
a b b b
b a b b

b b ... a b
b b b a

15. Evaluate
0 1 n-2 n-1
1 0 n-3 n-2

n-2 n-1 0 1
11- 1 n-2 1 0
 97

16. Evaluate the following and factorize the result:

a b c d
b c d a
c d a b
d a b c

17. Cornpute the adjoint of

2 0 0 1
2 -3 1
-2 1 0 1
(a) 0 1 -1 (b)
0 0 1 -2
3 1 7
1 0 2 0

18. The inverse of a certain matrix A is given as

2 2 3 3
-1 -2 1 6
A-t=
1 -3 -2 5
4 4 5 4

use this information to find det(A) and adj(A).

19. For ann x n matrix prove that

(a) adj(AT) = (adj(A)f (b) adj(cA) = cn- 1adj(A) for any scalar.

20. Let A be an n X n matrix with n > 1.

(a) Prove that A is invertible iff adj (A) is invertible. Find a formula for
the inverse of adj(A) when it is invertible.

(b) det(adj(A)) = (det(A))n-l

98

Use this to compute (adj(A))- 1 and det(adj(A) for

1 -3 0 0 0
3 -6 0 0 0
2 -4 2 4 0
-2 3 5 1 -5
0 9 3 4 7

21. Show that the matrix

cannot be the adjoint of any real matrix.

22. Compute a.dj( a.dj( ,\)) for

4 -3 9 2
1 -3 1 -1
0 0 1 1
0 0 2 -1

23. Establish a. formula for the l'andermonde determinant

1 1 1
Xr x2 x3 Xn
ry ry ry
J"i X~ X3 x;.

Use this to evaluate the complex determinant

1 1 1 1
1 -1 'l -'l

1 1 -1 -1
1 -1 -/.
 99

~ ~ - -
·c;_hapter 2 _
=, ;~ ;> ' ' ~ - ~ 1 ~. ' ~ ~ >

VECTOR SPACES
The n<'ccssity of vector qnantiti<'s as well as scalar quantities is very Wf'll­
known to all readers. One usually nf'eds r<'sultants ( a.lgebraic sums) and scalar
multiples of vector quantities. The same sort of operations apply to many ot lwr
qnantitif's, for example matrices can 1><' considered to be vectors, althouglt it
docs not sound that natural. They ohcy all characteristic laws of vectors.

t,.f8 VECTOR SPACES

Den nition 2.1.1. Let F h<' a fi<>ld and V a nonempty set. If there is

(a) an operation + on ll

(b) a function

FxV ----+ V
(r, r) ~--+ cv
100

Ruch that the followin~ axioms are satisfied, then V is called a vcrlor spare
over F:

(F2 ) There exists an <'l<'nwnt 0 in l1 such that v +0 = 0 +v = r for all

vEl'.

(V3 ) For cadt v E Y t hPn' exists an element v' E V such tlwt v + r' = r' + v =
0.

(V8) 1 v =v for all 1' EV

for all c,c 1 ,c 2 E F and r·.r· 1 ,v2 E l'.

Elements of a vector spare are called l'ccfors whereas el<•nwnts of the field
Fare called scalars. Tlte n~apping (c, v) ~----+ cv is called scalar multiplication
or multiplication by s('([lnrs: and the eight axioms (VI)- (l&) arecalhl vector
space m:ioms. ThP particular type clement given in (V2 ) is unique (rf Exercise
7) and it is calkd tlt<' ::rro vector, it is denoted by 0 11 or simply by 0 if there
is no danger of confusion. For each v E V the element giH•n in (l;) is also
uniquely determined ( cf Exercise 7) and it is called the negative of v, it is
customary to denot<' it by -v. Vector spaces over the field of real numbers
are ca.JJed real vector spares and those OVE'r t]w field of complex nnmbcrs are
 101

called complex vector spaces. In the subsequent sections our main concern trill
be real (or complex) vector spacc~'l.

Before considering specific <'Xamples of vector spaces it may be useful to

derive in detail some elementary properties that will often be needed in com­
putations.

Theorem 2.1.2. (Elementary Properties) For an arbitrary scalar c

and an arbitrary vector v we haY<':

(i) cO= Ov = 0,
(ii) ( -1)v = -v,

(iii) (-c)v = c( -v) = -(cv).

Proof: (i) To prove the equality cO= 0 we have

cO =c(O+O) by(V2)
=cO +cO by(Vs)
cO+ (-cO) =(cO+ cO)+ (-cO) by using(%)
0 =cO+ (cO+ (-cO)) by{Vt)
=c0+0 by(Va)
=cO by(V2)

The other equality is proved similarly.

(ii) Using the second equality in ( i) we get

0 = 0 v = (1 + (- 1)) t' = 1v + (-1 )v = v + (-1 )v

and then

-v = .:...v + 0 = -v + (v + (~l)r) = (-v + v) + (-l}v = 0 + (-l)v = (-1)v.

102

Here the reader is invitC'd to indicate the axiom to be used at each st0p.

(iii) This is obtained at onrc by using (ii) and (V7 ) from

(-r)1· = ((-l)c)v = (-l)(cv) = -(cv)

and

c( -v) = d( -l)v) = (c( -l))v = (-c)v. D

NATURAL EXAMPLBS OF \'BCTOR SPACES

A) The vector space of m x n matrices. For any field F tlw sC't pnxn
of m X n matrices is a \'0rtor space over F under usual matrix addition and
scalar multiplication sinrC' t h(' axioms (Vi)- (l18 ) are satisfied as we prov('d in
Theorems 1.3.1 and l.:l.2. In particular realm, X n matrices form a rC'a( V<'rtor
space and complex m X n matrices form a complex vector space.

Further, we can considPr m<1t.rices whose entries are vectors of a gi\'l'll VC'ctor
space V:

v= , vii E V.

All the statements givC'n in ThC'orems 1.3.1 and 1.3.2 hold for thesemat ricC's.
In addition if A is an p X rn matrix whose entries are scalars in the hase fip(<f
of lf and ifF is a matrix whosp entries arc vectors in V then thP product. A I'
can be defined in a natmal way an analog of Theorem 1.3.4 can bC' proved hy
making natural modifical inns.
------------------------------------------------~1Q1

For example we have

r2 -1 3]rl(l,-2) (-1.1)] [(2,-1) (1,-1)]

l1 2 1
(1, 2)
(-1,3)
(2. 1)
(1,0)
+2
(2,0) (-2,1)
=

[
(-2,3) (-1,1)
(2,5) (4,3)
l[ +
(:l,-2) (2,-2)
(4,0) (-4,2)
l[
=
(2,1) (1,-1)
(6,5) (0,.5)
l .

B) The n-tuple space. For any field F the finite sequence (x 1 , •.•• ;r")
with (xb ... , Xn) with X; E F is called an n-tuple. These

is a vector space over F under the operations defined by

Indeed, the axioms (Vi)- (V8 ) can E.'ither be verified directly or they can be
justified by identifying n-tuples (:~· 1 , • .. ,:tn) with 1 X n matrices [xt, ... ,x,,].
This vector space pn is known as the n-tuple space.

Note that elements of R 2 are vectors represented by directed line segments

of the plane and those of R 3 are \'('\tors in 3-spa.ce and they can also repre­
sented by directed line segments.

............................. ; (X I, X2)
(X I , X2, X3)

--0;-=::-----'-;-- X I

104

Th<' l'!'ad<'l' is advi<'ed not to try to <risurdi::e drmcnts of R" for n > 3 to
be "dircdrrlline segments'' or to be "some creatures whafe<,cr they are". They
are nothinp; but finite sequences of the form (x 1 , •.. , x,) with components
:r 1 , .•. , :r,. In various applications these components :z: 1 , ..• , :rn may appear
to be tlH' numlH'r of votes given to the parties in an election, they may appear
to be results of an exam, etc.

C) Field as vector space over itself. vVhen we take scalars and vectors
to be ('lC'm<'nis of a field F with operations ofF, the axioms (l'1 ) - (V8 ) are
obviously satisfied. Hence F is a. vector span' over ittwlf. Note that this vector
space can IH' id('ntified with the !-tuple space F 1 •

D) The space of functions. Let .F[a, b] be the set of all r0al valued
functions ddinPd on an int0rval [(/, b]. 1t forms a real v0ctor span' w.r. to
operations ddlned by

(J+g)(;l:) f(:r) + g(:z·)

(cf)( ;r) cf(.r)

where f and g are functions ddined ou the interval [a, b] and cis a real number.
The verification of axioms (l1d- (l8) is straightforward and can b(' kft to the
reader.

E) The space of polynomials over F

An expression of the form

l'{;r) = llo + a 1;z: + · · · + a,xn

where a 0 , ah ... , a, are elenwnts ofF is called a polynomial with co<'fficients

Ill F and a 0 , a 1 , ..• , a, are called coefficients of the polynomial. A polynomial
is uniquely determined by the sequence ( a 0 , a 1 , ••• , a 11 , ••• ,) of rodficient.s,
 105

namely polynomials

are equal iff a0 = b0 , a1 = b1 , ••• , oA: = bk, . . .. If P( x) is a polynomial with

an -:/: 0 and am = 0 for all m > n we say that P( x) is of degree n and we write
n = 8P. The set of polynomials with coefficients in F is denoted by F[x].
This set is a vector space over F tmdN the operations

Given a polynomial p( x) = a0 + o 1:r + · · ·+an xn with real coefficients, we can

use it to define a function sending any real number t to the real number

This function pis ca.lled the real polynom-ial function defined by the polynomial
P(x ). Note that real polynomial func1ions are defined in the interval ( -oo, oo)
and they form a real vector space.

SUBSPACES

Definition 2.1.3. Let V be a vector space over a field F and let W be

a nonempty subset of V.

(a) rv is said to be closed under addition if the sum of two vectors in lV

is also contained in W.

(b) W is said to be closecl undrr sralar multiplication if for any scalar c

and for any w in Hl the product c·w is contained in lV

Examples 2.1.4.( a) The set of vectors in the first quadrant of the plane
is closed under addition but it is not dosed under scalar multiplication because
106

as the figure shO\vs if Vl'ctors x and y lie in the first quadrant so does the
parallelogram with si<ks :r anJ. y and therefore its diagonal :t +y lies in the
first quadrant. Ilowewr ( -1 )x is not in the first quadrant. To see all this
algebraically we denot<' 1hP rirst quadrant by W and describe it by

It is closed under add i1ion l><'ra use if x = ( :rr, x 2 ) and y = (y 11 y~) hav<' positive
componf:'nts x t. x 2 , y 1 , lh t hPH the components x 1 + y 1 and :r:! + Y:2 oft he vector
:t +yare a.lso positiv0. Ilm\'('\'<'r, lr' is not closed under scalar multiplication,
for example the product ( -l )(5, 2) = ( -5, -2) is not in TV, although (5, 2) E
IV.

(h) The subset

of R 2 is closed un<kr scalar lllldtiplication, for when WP take an arbitrary

scalar c E R and any Pl!'lll!'llf ( :z: 1 ,;r~) of (!. t.IJ<'ll the prod urt of com poneuts
of c(.rJ,:t':d = (c:t·t,c:r~) IH'COill<'S c~.r 1 .1: 2 2::0 and hence c(:rt..r:J is in U. This
set is not closed und<'r addition IH'cause WP have :r = (1,2) andy= (-3,-1)
in U such that :t + y = (-:2. l) is not contained in U.
 107

(c) The set of vectors along a fixed line ax 1 + bx 2 = 0 passing through

the origin (0, 0) ,namely the set

is closed under both addition and scalar multiplication since

(x1, x2), (Yb Y2) E L ~ ax1 + bx2 = 0, ay1 + by2 = 0

~ a(x1 + yl) + b(x2 + Y2) = 0

~ (xt + Y11X.~ + 'fh) E Lc E R, (x11x2) E L

~ c(ax 1 + bx2) = 0
~ a(ca-d+b(cx 2 )=0

Theorem 2.1.5. Let V be a. vector space over a field F. If W is a.

nonemty subset of V which is closNl under addition and scalar multiplication
then

(a) W contains the zero vector 0 of V,

(b) for each w E lV its negative -U' is contained in fV,

and further W is a vector space with respect to the operations of V.

Proof. We first note that since lV is closed under addition and scalar
multiplication these operations can he considered to be operations of lV. For
108

each wE W we have (-l)w E W since H' is closed under srala1· multipli­

cation.By Theorem 2.1.2 one obtains -w = (-l)w E W ;furtlwr.using the
hypotheses lV =j; ~?,say ll' E IV, and that H' is closed under add it ion one has
0 = w + (-w) E lr.Tlds proves (a) and (b) .Thus,we also verifi1•d ,-\xioms
(V2 ) and (%) .Now ,lh<' other axioms arc satisfied for all clem('nts of lV as
they are satisfied hy I hos<' of V .0

This theorem sl10ws how appropriate the following definition is:

Definition 2.1.6. A subset W of a V('ctor space ll over a. fi(•l<l r is called

a. subspace of ll if
(a.) W is nonempty,
(b) \Vis closed und(•r addition,
(c) W is closed und<•r sralar multiplication.

In order to test (a) \\'<' usually check whPther 0 is contained in 1r or not,

if the result is not ~ffirma1 ive we can conclude that H' is not a su hspace by
Theorem 2.1.7, in cas<' it is affirmative, however, (b) and (c) should IH' t('stcd.

Example 2.1.7. For every vector space ll the subsets 0 a11d l/ are
nonempty subsets \vhirh are closed under addition and scalar multiplication,
namely they are suhspac<'s.TIH'y arc called lr·ivial subspaccs.

Example 2.1.8. V<'ct.ors in the direction of any line ax 1 + b:t: 2 = 0 more

precisely the suhs<'t

of R 2 is a subspace ,for we have seen in Example 2.1.4 (c) that this subset
is closed under addition and scalar multiplica.tion.Ilowcver, the subsets given
in Example 2.1.4 (a) a11d (b) are not subspaccs, because one of the closure
properties is not sat bfiPd.
 109

LINEAR SPAN

Definition 2.1.9. Let V be a vector space over a field F.

(a) If v1 , v2 , • • ·, vr are elements V , then every vector of the form

is called a linear combination of -t• 1 , v2 , • • ·, vr.

(b )For a subset S of V the set of all linear combinations of elements of Sis

called the linear span of S and is denoted by < S > or Span(S).When S = ¢
we define < S >= 0.
(c) If V =< S > that is to say if every vector of 11 is a linear combination
of elements of S ,then we say that V is spanned or generated by S .In this case
S is called a set of generators or a spanning set of 11.

Example 2.1.10. Let S = {(3, -4, 1, 0), (0, 6, 1, -3)}.Regarding this set
as ~ subset of the space R 4 we can form the linear combinations

-8(3, -4, 1, 0)+4(0, 6, 1, -3)' 3(3, -4, 1, 0)+2(0, 6, 1, -3)' 0.(3, -4, 1, 0)+(0, 6, 1, -3)'

of elements of S .Further a quadruple (a, b, c, d) is in < S > iff we can find x1

and x 2 such that

(a, b, c, d)= x 1 (3, -4, 1, 0) + x 2 (0, 6, 1, -3).

This yields the system with augmented matrix

3 0 a
-4 6 b
1 1 c
0 -3 d
110

By row reducing we obtain

1 1 c
0 -3 d
0 0 a- 3c- d
0 0 b + 4c + 13°d

and conclude that (a, b, c, d) is in (S) iff a- 3c- d = 0 and

:lb + 12c + lOd = O.Finall~· note that R 4 is not genNa.ted by S,heca.use it has
elements not contained in < S >,for example (I, 0, 0, 0) does not satisfy the
above equations and lwnc<' it is not contained in < S >.

Example 2.1.11. l.<'t us show that {(1,2,-1),(-3,1,0),(2,-3,2)} is a.

set of generators of 1{1 tl1at is to say every t:ripl<• (a, b, c) can be writ.ten as a
linear combination

l
This is equivalent to sayin)!; that the system wllith augmented matrix

1 -3
[ 2 1 -32 (/b
-1 0 2 ('

is consistent. A reduct ion of this matrix gin's

I
[ 0

()
-3
1 -1
0
2

7 1 a- 3b- ic
b-_2aa
1
l
'and shows that this is in fact the case.

Theorem 2.1.12. If S' is any subset of a. vector space l' over a. field F
.then < S > is a subspace . .
 111

Proof. If S = </> we have by definition (S) = {0}. So we may assume

S 'f:. </>, say v E S. Then

(a) < S > is nonempty since the linear combination l.v is contained in
< S>.
(b) ·when we ta.ke two linear combinations of elements of S we can write
them as

a.nd their sum becomes the linear combination

Therefore the sum is in < S >.

(c) A scalar c multiplied by a linear combination a 1 v1 + a 2 v 2 + · · · + an Vn
of <'l<'nwnts of S gives us the linear combination

of (']('111<'nts of S.Hence it is in< S > .0

Example 2.1.13. Let us describe the space

< [ 12 -3]
1
[7
2 -1
3] [ -63 -51-5] >
by mf'ans of a homogeneous system.Now, a matrix

[: :l

112

is in the given space iff the system with augmented matrix

1 7 3 a
-3 3 15 b
2 2 -6 c
1 -1 -5 d

has a solution.By reducing we sec that this is the case iff a.,b,c,d satisfy

-a + 2c - 3£1 = 0 , b + 3d = 0.
 113

f'JII EXERCISES

In Exercises 1 through 6 determine whether the given set, together with

giwn opNations, is a vector space over R or not. If it is not a vector space
indicatC' the first axiom that fails to hold.

1. The set of all symbols of the form << x, y >>,where x and y are real
numbers, together with the operations defined by

c << x,y >>=<< c,xy >>

2. The set .of symbols of the form )x, y( where x and y are real numbers
together with the operations defined by

)xl!Yt( + )x2,y2( = )xl + X2,Y1 + Y2(

c )x 1, x 2 ( = )ci,O{"- t\.
3. R with ordinary addition and (scalar) multiplication.

4. C ,the set of complex numbers with usual addition and "multiplication

by real numbers".

5. 2 X 2 real matrices with operations

[ a b
c d
l +[ a' b'
c' d'
l[ a + a'
c + c'
b+ b'
d+ d'
l
c[ x Y] [ ex Y]
z t z ct
114

6. The set of positive 1·cal numbers together with operations EB and 8 defined
by
pEf)q=pq

c 8 p =PC

7. Prove .that in a vector space V there is only one element z satisfying

v + z = z + = v for all v in V
'lJ and for each v in· V there is only one v'
such that v + v' = v' + v = 0.

8. Is the subset
W = {(a,b,c) I a.,b,cE Z}

of R 3 dosed under addition? Is it closed under scalar multiplication?

9. Show that the subsl't {(a:, y) I x, y E R , a:y = 0 } of R 2 is not closed

under addition .Is it closed under scalar multiplication?

10. Prove that the following subsets of R 4 are subspaces:

(a) wl = {(;r,O,y,O) I x,y E R}
(b) lV2 = {(X, y' z' t) E R4 I X + y = z + t}
(c) w3 = {(x,x,y,O) I x,y E R}

11. Which of the following are subspaces of the vector space of real n X n
matrices?
(a) Upper triangular matrices.
(b) Diagonal matrices.
(c) Invertible matrices.
(d) Symmetric matrices.
(e) Skew-symmetric matrices.
(f) Singular matrices.
 115

12. Show that the following are subspaces of the real vector space of real
valued functions defined in ( -oo, oo) :
(a) Polynomial functions of the form J( x) = a + bx.
(h) Functions of the form acosx + bsinx.
(c) Functions satisfying y' + 2y = 0.
(d) Even functions.
(c) Odd functions.
(f) Periodic functions of period 1r.

1:3. Find the relation satisfied by a,b and c if (a,b,c) is a linear combination
of
(a) (1, -1, 1), (2, 1, -1) (4, -1, 1)

(b) (3, 1, -2), (-2, 1, 1) , (5, 0, -3).

U. Find k if (k, 2~~ + 1, 3k - 1, P + 3k- 9) is in the subspace of R 4

spanned by

(1,1,1,1), (-1,-1,-2,-1), (1,2,3,2), and (1,0,4,2).

15. D<>terminc the polynomial 1 + x + c:r 2 + dx 3 if it is a. linear combination

of

10. \Yhat must be a if ( -b, 4, 0, 2b- 2) is not contained in the subspace

< (-1, 1, -1,1 ), (1, a, 1,-1 ), (-2a, -a, -2a+ 1,1 +2a), (-1, 3, a-1, 2) >

17. Can (1, -2, 4) he written as a linear combination of (1, 0, 1), (3, -4,1)
and (4, -6, -2)?
116

18. Is ( 6, 11 , 4, 1 ) in Spa 11 { ( :1, - 1 , 1 , 1 ). ( ·1. :1, 2, 1 ) , ( 1, ·I. 1. 0 )} ?

19. Find the relations satisfied by a,b and c if (a,b,c) is in the subspace of
R 3 spanned hy (1.-1.0),(0,1,1)<111<1(:3,-2,1)

20. Expr<'ss Span { (I, -I, 0 ), (0, 1, 1 ), (:J. -5, -2)} as the solution space of a
homogeneous s_vs1<'Jll of linear equations.

21. Express the spacp spann<'d by

1 :J -2
-1 -2
2 -I 3
·~ 3

as thC' :;C't of solu1io11s of a homop;<'n<'ons system.

22. Find th<' value of .r if

[ +4 --Lt-+ 2]
:r 10 '1 x G E<
[ 1 -x
:3 1 + .r
l[ '
-x
-:3.-r
a: 3

-:z: 2 - :z:
l[ l
'
3 2 - 3x
9 3:r +1 >

~~----~---- - -----~------~-~~---------~~-~-~-~-...-.,

117

t.J:J LINEAR DEPENDENCE, INDEPENDENCE,~ BASES

In the previous section we ohs0rved that vector spaces may be spanned by
a number of vectors. Now we want. to handle the prohl<'m of spanning vector
spaces by the least number of \'<'dors. Obviously to be able to do this we
must pic.k up generators in sud1 a way that no one of 1hem can be written as
a linear combination of other g<'nNators, namely a kind of independence will
be obliged.

Definition 2.2.1. Let Y h<' a vector space ov<.'f a field F and let
{ Vt. v2 , • • ·, Vm} be a finite set of vectors in V. \V<' say that v1 , v2 , • • ·, Vm
are linearly dependent if there <>xist scalars c1 , c2 , • • ·, cm, at least one of whkh
is nonzero, such that the (nontrivial) linear relation

is satisfied. Otherwise, namely if

'

implies
C1 = l'2 = ·· · = Cm = 0,
the vectors vb v2 , • • ·, Vm are said to be linearly ituh pf1ulent. A subset S of
V is called a· linearly dependent sf'f if it contains a. fl nite non empty subsetset
·consisting of linearly dependent v<>ctors. Otherwis<', it is called a. linearly
independent set.

Example 2.2.2. The vectors (1, -3, 2), (2, 2,-1 ). ( 1, 5, -3) in R 3 are lin­
early dependent, because the existence of c1 , c2 , ca, not all zero, such that

c 1 (1,-3,2)+ c 2 (2.2,-1)+ c3 (1,5,-3) = (0,0,0)

118

is equivalent to th0 exi:-;t <'nee of nontrivial solutions oft he homog('neons syst0m

x 1 +2:r 2 +:r 3 0

--:Lr 1 + 2x 2 + 5x 3 0

2:r 1 - x2 - 3.ra 0

and indeed this systC'tll has nontrival solutions.

Example 2.2.3. Th0 r0al valued functions l,cos:t,cos2.r.ros 2 :r are lin­

early dependent, for t h<' nontrivial relation

1 + cos 2.r - 2 cos 2 :r = 0

is satisfied.

Example 2.2.4. For any vector spare t lw S<'t { 0} consisting of the zero
vector alon<' is lin<'arly dqwnd0nt:

1.0 = 0.
Further any set contait1ing the zero vector is a linearly depcnd<'nt set. A single
vector vis linearly d<'JWn<knt iff rv = 0 for some nonzero scalar c and hence
iff v = 0.
Example 2.2.5. Tit<' empty subset¢> of a vector space is linearly indepen­
dent since it has no nonernpty linearly dependent subset.

Example 2.2.6. Tit<' infinite set of real valued functions giwn by

is linearly independent. Indeed, if we have real numbers c0 , c" ···,en such that

r·ofo + c,J, + · · ·+ c,J, = 0,

 119

then we have

f(x) = c0 + c 1 .r + · ··+ CnXn:::: 0 for all x (1)

which also implies that

J'(x) Ct + 2t::~:r + ••· + nCnXn-l = 0

J"(x) 2c2 + 6t 3 x + · · · + n(n- l)cnxn- 2 = 0

(2)

At x = 0, (1) and (2) yield

Co = 0, Ct = 0, · · •, Cn = 0.

Tlwrdore / 0 , / 1 , • • ·, fn are lin<'arly independent.

This example shows also tl1at two real polynomial functions are equal iff
the polynomials are equal. In fact, iff= g for

t}H'n

J(x)- g(x) = (ao- bo) + (a 1 - bt)x + .. · = 0

for aJl :l' and hence ao - bo = a1 - b since 11 X, • • ·, Xr are linearly independent.

Theorem 2.2.7. Let V be a vector space spanned by m vectors. A linearly

indf'pendent subset of V cannot contain more then m elements.

Proof: Let V be spanned by { l't. v2 , · · ·, vm}· If S is any set containing

more than m elements, say { u· 1 , v• 2 , • • ·, w,J with n > m, expressing the w; in
120

terms of the vk one ob1 a ins

'Wt A11v1 + A21V2 + · · · + AmlVm

tl''.! A12V1 + A22V2 + · · · + A,.2vm

which has the matrix form

Since, n > m the hontnp;<'lwous system AX =0 has a. nontrivial solution

C=

For this solution, the <'quality AC = 0 implies that

namely wh w 2 , • • ·, 11' 11 are linearly dependent.

0

Example 2.2.8. Tlte polynomials

are linearly dependent, b0cause they belong to the space of polynomials of the
form
 121

and this space is spanned by 1, x, :z: 2 , x 3 , so the above theorem applies with
m = 3 and n = 4.
Example 2.2.9. Let us show that the vectors (1, -2, 4), (0, 1, 2), (0, 0, -3)
are linearly independent and deduce that neither of the sets

¢, {(1, -1, -2)}, {( -1, 1, 3), (2, -2, 1)}

can span R 3 • Indeed, if we can show that the given vectors are linearly inde­
pendent, the theorem says that R 3 can be spanned by at least three vectors,
however, the given s<'ts have fewN <'h•ments. To verify linear independence we
form
c 1 (1, -2,4) + c 2 (0, 1, 2) + c3 (0, 0, -3) = (0, 0, 0)

and see that it implies

c1 0

-2c 1 + c2 = 0
4cl + 2c 2 - 3ca 0

namely c 1 = c2 = c3 = 0 as required.

BASIS AND DIMENSION

Now we shall introduce genC'fators without any redundant member. For

example, 1,x,x 2 ,x3 are generators of the space of polynomials of degree :S 3
and if we omit any of them we cannot €'xpress some polynomials in our space
as linear combinations of remaining polynomials.

Definition 2.2.10. Let V be a vector space. A subset B of V is called a

basis for V if
j')'~

(a) B spans V, that is to say every element of Vis a linear combination of

elements of B

(b) B is linearly indq>P!ld('nt.

In order to determin<' wltdher a set is a basis or not all we have to do is to

check conditions (a) and (h). The second condition says that no member of
a basis is a linear corn hi nat ion of the others. So a basis is a set of generators
whose clements are all nr·rr.~.«rtry to span V.

Example 2.2.11. ThP Y<'ctors (1, 1, -1),(1,2,3) and (0, 1, 1) form a basis
for R. 3 . In order to verify tllis we must check conditions (a) and (h) above:

(a) For any (a, b, c), t h<' Pquality

leads us to the syst <'lll

Its augmented matrix can be reduced to

l~
001
0
1 b-
a a
0 -3 c + 5a- 4b
l
and hence the syst<'m has a solution. Therefore the given vectors span
v.
 123

(b) To see the linear independence we consider the relation

c1 (1, 1, -1) + c2(1, 2,3) + c3 (0, 1, 1) = (0, 0, 0).

We solve the resulting homogeneous system and see that we must have

The basis for R 3 that we considered above is of course not the only one, for
instance it is very easy to verify that the set {(1,0,0),(0,1,0),(0,0,1)} is also
a basis and more natural than the one given above. In the subsection below
we describe this sort of natural bases.

STANDARD BASES FOR SOMF: VECTOR SPACES

Now we shall give examples of standard bases for natural vector spaces that
we usually need to consider.

A) The Standard Basis for the n-tuple Space

For the n-tuple space consi<kr 1he vectors

e1 = (1,0, ... ,0)

e2 (0,1, ... ,0)

en (0,0, ... ,1).

Since we have
124

for each n-tuple (:r 1 , :r'.! .... , .r, ), on one hand we see that { r 1 , c 2 , ••• , c,} spans

then-tuple space and on t h<' .ot.her hand

implies that x 1 = :r:! = ··· = x,. = 0, namely {e 1 , e 2 , . . . , t- 11 } IS linearly

independent. Thus we S<'<' 1 hat the set

is a basis for the n-tupl~' span•. This basis is known as the s!arulard (or usual)
bas.Zs for then-tuple span•.

B) The standard basis for m x n matrices

As is done for the n-tupl(' space we can irnmediat<'ly verify that tlw matric<'S

00···000···0

R;j = 0 0 ··· 0 1 0 ··· 0

00 000···0

whose (i,j)-entry is 1 all other entries are zero, form a basis which can he
considered to be the standard basis.

C) The standard basis for polynomials

\Ve know that every polynomial has a unique expression in the form

l'(.r) =Co+ c 1x + ···+ c 11 :z: 11 •

This shows that the span• of polynomials is spanned by

l,:z·, .. . ,X 11 , •••
 125

and these spanning vectors are lirwarly independent because any finite linear
relation

implies c0 = c1 = · · · = Cn = 0. Thus the set

{l,x, ... ,xn, ..,.}

is a basis for the space of polynomials.

D) T he standard basis for UC-functions

Linear combinations of non-zero functions of the form

where n is a natural number, a is a. real number and b is a nonnegative real

number are called UC-functions. They play the main part in the solution of
linear differential equations with constant coefficients. It can be proved that
these functions are linearly independent and hence they form a basis for the
space of UC-functions (See Appendix A).

CONSTRUCTION OF BASES, DIMENSION

A vector space may have various bases but it turns out that all of them
have the same number of elements, as is established in the following theorem
·which also assures the existence of bases.

Theorem and Definition 2.2.12. Let V be a vector space spanned by

m vectors. Then:

(i) V has a basis and it can be obtained by deleting generators which are lin­
ear combinations of its pred<'!'essors. Further, any linearly independent
 1:26

set can h0 extended to a basis for V.

(ii) Th,, number of elements contained in each basis is at most m.

(iii) Any two bases for\' have the same number of elements.

The nunilH'r of elcnwnts contained in any basis is called the dimension of

F and it is denoted hy dirn(V).

Proof: (i) Let w 1 , ·w", · · ·, tl'm be g<'nerators of V. If 'Wm is a linPar combi­

nation of its pred('(·essnrs ·w 1 , ·u, 2 , • • ·, w,,_ 1 then we can omit it to reduce the
generators to wh H' 2 , · • ·, '11' 111 _ 1 , lwcausc an.v linear combination in which 'Wm

appears can be wri 1(('n as a linear com hination of i 1s predecessors si nee Wm can
be replaced by a linear combination of them. If w, is not a linPar combination
of its predecessors WP ke('P it in the set of genNators. Thus in both cases we
pass to the set { w 1 , 1/':2, • • ·, W 711 _ 1 } and apply the same proced me. I 11 this way
we delete the generators which are linear combinations of th(' preceding ones
and get a generating set

consisting of those which are not linear combinations of tlwir pred('cessors.

Further, if we had a linear relation

·the last nonzero term would give one w;k as a linear combination of its prede­
cessors, but this is impossible. To prove the rest of the sta.tPment it is enough
to note that given a linearly independent set and a set of generators their
union is also a set of generators and that if linearly independent vectors are
taken to be first vectors none of them will be omitted because none of them
will be linear combinations of their predecessors.
 127

(ii) Since a basis is linearly independent this assertion follows at once from
Theorem 2.2.7.

(iii) Let B 1 and B2 be two basis for V containing n 1 and n 2 vectors respec­
tively. They are linearly independent generating sets. Now, considering B 1
to be a basis and B 2 to be a generating set we have n 1 s; n 2 • Reversing the
argument we have n 2 s; n 1 and thus we get n 1 = n 2 by ( ii).
0

This theorem is of utmost importance. It enabl('S HS

• to define dimension by showing the invariance of elements in vanous

bases.

• to construct a basis out of a finite generating set of a vector space and

thus show that it has a \veil-defined dimension.

• to extend any linearly ind0pendent set to a basis.

For example considering standard bases we observe that the n-tupk space
and the space of m. X n matrices arc finite dimensiona1 vector spaces with
dim(!R.") = n, dim(!R.mxn) = mn respectively. The space of polynomiaL how­
ever, is infinite dimensional. The space of polynomials of degree s; n has a

basis {1, :r, ... , x"} so it is of dimension n + 1.

As a snpplement to the above theorem we give two corollaries:

Corollary 2.2.13. Let V be a finite dimensional vector space of dimension

n. Then:

(i) A subset more than n clements cannot be linearly independent.

(ii) A subset fewer than n elements cannot span V.

128

Proof: Since dim(V) = n means that V has a basis of n elements (genera­

tors) and therefore every linearly independent set has at most n elements hy
Theorem 2.2.7. This proves the first statement. The second one is a rE-state­
ment of Theorem 2.2.12 (ii).
D

Corollary 2.2.14. Let V be a finite dimensional vector space of dimen::;ion

nand let W be a subspace of V. Then dim(W)::; dim(V) and every basis for
W can be extended to a basis for V.

Proof: Since any basis for lY is a linearly independent subset of V the

result follows at once from Theorem 2.2.12 (i) and the corollary above.
D

Corollary 2.2.15. Let V be a finite dimensional vector space of dimension

n. Then:

(i) Any linearly independent set containing n vectors -is a basis.

(ii) Any spanning set containing n vectors is a. basis.

Proof: _(i) If S is a. linearly independent set of n vectors by Theorem 2.2.12

(i) we can extend this set to a basis B which also contains n vectors. Thus it
will not be a proper extent ion, namely we have S = B, a basis for V.

(ii) If Sis a spanning set of n vectors, then by Theorem 2.2.12 (i) we find
a subset of S which is a. basis for V. Since this subset contains n = llim(l·r)
elements it cannot be proper and it must be S itself.
D
 1':!.?

Example 2.2.16. Consid0r 11t0 subspace of 4 X 1 matrices spanned by

-1 :2 1 3 1
2 3 4 0
Xt = x2 = :r:1 =
'
:1.'4 =
'
J.~5 =
1 -1 0 -1 0
3 0 3 3 1

vVe shall use the above theorPill 1() find a basis. For this purpose we form the
matrix whose columns are tlH' ?;i\'('fl column matrices and we reduce it to gd.
for <'xarnplc the echelon ma1 rix

-I :2 :3 1
0 2 1
0 () 0 () 1
0 () 0 () 0

This shows that x 2 is not a lin<'<ll' combination of :r 1 because otlwrwise :r 1

would l<•ad us to the syst0m

-(' 2

:2c 1

(' -1

:k ()

·with augmented matrix

-I :2
:2 1
-1
:l ()

which is row ('quivalent to the ma1 rix obtain<'<~ by taking the first two rolnmns
1:'10

a hove namely to
-1 2
0 1
0 0
0 0
which is seen to be inconsistent. But x 3 is a linear combination of x 1 and x 2
since x 3 = c 1 x 1 + c2 x 2 leads us to the system with augmented matrix
-1 2 1
2 1 3
1 -1 0
3 0 3

which is equivalent to tlw matrix

-1 2 1
0 1 1
0 0 0
0 0 0

obtained by taking the first three columns of the above big echelon matrix.
As is seen the system has a solution (c 1 , c2 ).

Similarly x 4 is a linear combination of its predecessors x 1 and x 2 but x 5 is

not a linear combination of its predecessors x 1 , x 2 , x 3 , x 4 • Thus we can delete x 3
and x 4 and we obtain the linearly independent spanning set {x 1 , x 2 , xs}. Note
that the resulting column matrices are those corresponding to leading entries
of the big echelon matrix and not their own columns of leading entries.
 .•

1."11

a) 2 + ;t = 1 + ( 1 +;c), a linear combination of its predecessors, so we delete

2 + :r;
h) 2 + x 2 = c 1 1 + c 2 ( 1 + x) + c3 (2 + x) implies by comparing the coefficients
of :r 2 that 1 = 0 so it is impossible to write 2+x 2 as a linear combination
of its p red ('CC'SSOrS.

According to TIH'orem 2.2.12 (i) a basis for the space is

Example 2.2.18. We know that R 3 has the standard basis of three ele­
nwnts. So it has dimension three. Now, let us show that the sets

a) {(1,1,1),(1,-1, 1),(2,0,1)}

a11d

b) {(1,1,1),(1,1,0),(1,0,0)}

are has0s for JR 3 by using Corollary 2.2.1{) and verifying that the first set is
linearly independent and the second one spans R 3 •

In fact,

implies that
132

which yields c 1 = c2 = c3 = 0.
To verify that (1, 1, 1),(1,1,0) and (1,0,0) span R 3 , we write

(a, b, c)= c1 (1, 1, 1) + c 2 (1, 1, 0) + c3 (1, 0, 0)

and obtain the system

c1 + c 2 + c3 a
c1 + c2 b

and observe that it has solutions

c1 = c, c2 =b- c, c3 =a - b.

Therefore every (a,b,c) in R 3 is a linear combination of (1, 1, 1),(1.1,0) and

(1, 0, 0) coordinates.

COORDINATES

Theorem 2.2.19. Let V be a vector space and B = { vh .. . , t'n} a. subset

of V. Then B is a basis for V iff every element v E V can be written uniquely
in the form

where ell ... , en are scalars and v1 , v 2 , ••• , Vn are elements of V.

Proof: ::;.: Suppos<' B is a basis for V. Then each vis a linear combination
of vectors in B, say

Suppose now that v has two expressions

 J;'].']

Then we obtain

Linear independence of elements of B shows that

and hence

that is two expressions are the same.

~ : Suppose now that 8 is a subset of V so that each vector v in l' has a

unique expression
V = C1 Vr + · ·· + Cn'Vn
with scalars c1 , ••• , c11 and vectors v1 , ••• , 'V 11 in B. Then every vector is a linear
combination of elements of 8, namely 8 opens l'.

then uniqueness of this expression for the zero vector

OVJ + · ••+ OV 11 = 0

yields that c 1 = 0, ... , Cm = 0. That is to say 8 is linearly independent. As a

linearly independent generating set B is a. basis.
0

Corollary and Definition 2.2.20. If l' is a.n n-dimensional vector space

with a basis 8 = {v 1 , ••• , v11 } , then for each vector v in l' there is a uniquely
determined n X 1 matrix

[v]B = [ -:~· ]
134

so that [v + w] 13 = [v] 13 + [w] 13 and [cv] 13 = c[v] 13 • The matrix [v] 13 is called the
coordinate matrix of v relative to the basis B.

Proof: By the theorem, for each tJ in Y we can find uniquely determined

scalars c1 , ••• , en such that

l
Thus we obtain a uniquely determin<'d n x 1 matrix

c1
[v]a = [ : .
c,.

l
If we take ~mother vector w with coordinate matrix

[w]Li = dt:
[
dn

then we have

and hence

l [l [l
Therefore

[v + w]s =[ c1 +: d1 c1
: + d1
: = [v]s + [w]s.
Cn + dn C,. dn

Further for a scalar c we have

 [cv]s = [ ~ .~ ] = c[ ~~ ] = c[v]s 0

This theorem allows us to visualize vectors of an n-dimensional space to be

n X 1 matrices or n-tuples by means of the correspondences

v ...._. [v]s = [
Cl
: . . . .,. (cl, ... ,c,).

c·,

Since addition and scalar multiplication is in accordance with this correspon­

dence as far a.s linear algebra conrNned all properties under this correspon­
dence and will be preserved all these three quantities can be identified. This
will be effectively used in computations.

Theorem and Definition 2.2.21. Let B = {v~> ... , v,} and C =

{ w1 , ••. , wn} be two ordered bases for a finite dimensional vector space F.
There is a unique invertible matrix P such that

for all r 111 V. This matrix is called the transition matrix (or the change of
coordinate rnatrLz:) from B to C.

Proof: In order to see the uniqueness first we take ·p to be v; in the equa.t ion
( 1) and ohtain
136

Noting that
0

0
[v;]B = 1
0

0
by using the product of partitioned matrices we obtain

[[vdc · · · [vn]c] [P[vdB · · · P[vn]B] = P[[vt]B · · · [vn]B]

P.

Thus the matrix P in ( 1) is uniquely determined as

/'~

P = [[vt]c · · · [vn]c]. ft(4)

I . J
I
\_____./

To see that (1) is satisfied by the matrix in (2) we note that for any
·-­ --. 'I rc,i
"~ \~(
~ ' .u db lc
::::1
,,
we have

[v]c = ct[vdc + ·· · + c,[vn]c [[v,]c[v,]c · · ·[v.]c] [ ~~ ]

P[v]B

smce
 1.'J7

Thus to find the B to C change of coordinate matrix we write elements of

B in terms of clements of C and lake the coefficients to bf' f'nfries of rcspcdh'e
columns:

The C to B change of coordinate matrix Q satisfies

by ( 1) and hence
_ [v]t., = QP[ I!]
implying that QP = I; ~)or all v/onsequently Q = p-l. Thus each chrmgf'
of coordinate matrix is invertible.

Example 2.2.22. Find the transition matrix

(a) from B = {(1, 1, 1 ), ( l, 1, 0), (1, 0, 0)} to [ = {(1, 0, 0), (0, 1, 0 ), ( 0. 0. 1)}
(b) from [ = {(1,0,0),(0.1.0).(0.0, 1)} to B = {(1, 1.1),(1, 1,0),(1,0.0)}
(c) from B = {( 1, 1, 1 ), ( l, l, 0 ), ( 1, 0, 0)} to C = {(1, 1, 1 ), ( 1, -1, 1 ). ( 1. 0. 0)}

a) \Ve find coordinate matrices of dements of B relative to [:

[( l, l, 1)], =[ l [(!,1,0)], =[ J [(1,0,0)], =[ ~ l

 138

Then we write these to be columns of the transition matrix P, namely

b) As we observed a hove the transition matrix from B to f. is obtained by

l
inverting P. We have

p-1 = [ ~ -~
_:

c) Letting

v1 = (1, 1, 1). v2 = (1, 1, 0), Va = (1,0,0)

and

Wt = (1, 1, 1), w2 = (1, -1, 1), w3 = (1,0, 0)

and then expressing v 's in t<'rms of w 's we get

1'1 wl
1 1
1'2 -w 1 - -w2
2 2
+ va
t'a wa.

· Therefore

P=
[~
1/2
-1/2
1 :]·
Now, we can explain how the change of coordinate matrix changes the
coordinates.
 1:'1.9

In general we can use the follo\\'ing procedure to determine the transition

matrix:

e \Vrite the partitioned matrix [ w1, • • • Wn I v1 • • • vn] whose columns arc co­
ordinate matrices of tv's and t' 's.

• Hednce it to row reduced ech<>lon form and get [liP] and thus obtaiu P
from the second part, because /) is the matrix of coefficients when v 's
are expressed in terms of tL' 's.

SUDSPACES ASSOCIATED \YTTII A MATRIX

A) Row space

Let/\ be an m x n matrix. TIH'll its rows can be considered to ben-tuples.

The subspace of n-tuples spanned by 1hcse row vectors is called the row space
of A and its dimension is calkd th(' mu• ·rmtk of A.

First we note that two row-cquivaknt matrices A and B have the same row
space, in particular they are of thC' same row rank. Because we have

n = PA

for some invertible matrix P so 1 hat

R~
B=
140

where R 1 , ••• , Rm are rows of A and this gives

PuR1 + · · ·+ P1mRm

B=
P21R1 + · ··+ P2mRm

which shows that rows of B are linear combinations of rows of A in other words
row space o:C B is contained in the row space of A. Reversing the argument
we also obtain row space of A is contained in row space of B and the result
follows.

Secondly non-zero rows of an echelon matrix

R=[~
... 0 A1h
*
... . .. 0
0
0 ... 00
A2h
*
0 Arir* .. ]
are linearly independrnl because when we multiply the rows by ch c2 , . . . , Cr
respectively and add we get

+ (0 .. ·0 0 .. ·0 c2A2h * ·· · )
+ (0 .. ·0 0 .. ·0 0···0 CrArir*···)=(O,O, ... ,O)

showing that c1 = 0 first, c 2 = 0 secondly, c3 = 0 thirdly, etc.

Suppose now that A is an m x n matrix and R is an echelon matrix. Then
the first result above shows that the row space of A is the row space of R and
the abond result shows R is spanned by linearly independent vectors namely
nonzero rows of R form a basis for the row space of A. Thus to find a basis for
 141

the subspace spanned by some n-luplrs, we consider the matrix whose rows are
given n-tuples and !.hen reduc£. it to an echelon matrix, and pick 11p non-zero
mws of this echelon matriJ:. Whr.'n l' is an arbitrary finite dimensional vector
space to apply the above pron'dure we use then-tuples of coordinates.

Examples 2.2.23. 1) Find a basis for the subspace of R 5 spanned by

(1,-1,2,0,-1), (2,-1,-2,0.1), (-1,0,4,0,-2), and (0,-1,6,0,-3).

Taking these to he rows of a. matrix we have

-1 2 0 -1
2 -I -2 0 1
-1 0 4 0 -2
0 -1 6 0 -3

and reduce it to
1 -1 2 0 -1
0 -6 0 3
() 0 0 0 0
0 0 0 0 0
Therefore a basis for H' is

B = {(l,-L2.0,-l), (0, 1,-6,0,3)}.

2) Find a basis for the span' of polynomials spanncd by

Considering the correspondence

142

we form
1 0 1 0 0
1 1 0 1 0
1 1 -1 1 0
-1 0 1 0 1
and find a basis for its row space by reducing it to

1 0 1 0 0 ,

0 1 -1 1 0
0 0 2 0 1
0 0 0 0 0

Thus a basis for the gin•n space of polynomials is

B) Column space: Tl1e subspace of m. X 1 matrices spanned by columns

of the matrix A is calhl the column space of A. A column matrix

B=

is in the column span• of A iff

where At, A 2 , ••• , An ar<' columns of A and c1 , c2 , ••• , Cn are scalars. \Vriting
out explicitely we see that it is equivalent to
 lf.l

Therefore a column matrix

IJ=

is in the column space of A iff//!( system

AX= IJ

is consistent. This fact is <'Xtn'nH{v useful in constructing bases and it makes

the column space the most import ant space associated with 11. Ivla.ny problems
concerning computations on bas<'S, linear dependence, generators etc. can he
reduced to the reduction of mat rirPs whose columns are coordinate matrices
of vectors relative to rea.sonahl~· stan <lard ha.ses.

Example 2.2.2.4. Show that the vectors (l,-1,2),(2,1,1),(0,:3,-3) are

linearly dependent.

In order to show this we can form the matrix whose columns are coordinate
matrices of given vectors relatiw tot he standard basis and reduce it to echelon
form:

[-~ ~ ~] [:) ~ ~ ] [~ ~ ~ ]
2 1 -3 0 -3 -3 0 0 0
.
144

l
We see that the system with augmented matrix

1 2 0
[ -1 1 3
2 1 -3

l
is equivalent to the one with augmented matrix

1 2 0
[ 0 3 3
0 0 0

and hence it is consist0nt. According to the result above this means that
(0, 3, -3)is a linear combination of (1, -1, 2) and (2, 1, 1). Thus these three
vectors are linearly dl.'pendent.

We use Theorem 2.2.12 (i) to find a basis for the column space. \Ve apply
elementary row operations to reduce the matrix to a matrix in echelon form
and see in this way columns which are not linear combinations of their prede­
cessors, they are exactly columns of leading entries. Thus bf colvmns of the
given matrix A (not columns of the resulting reduced matrix) corresponding to
columns of leading entries of the resulting matrix form a basis for the column
space of A. By using the coordinate matrices this procedure is used in the
construction of bases for subspaces spanned by a finite number of vectors in
an arbitrary vector space.

Examples 2.2.25. 1) Find a basis for the subspace of R 5 spanned by

v1 (1, -1, 1, 0, 1), v2 = (2, 1, -1, 1, 1),

v3 (0, 3, :-3, 1, -1) and v 4 = (0, 3, -3, 1, -1).
 145

\Ve form the matrix by considPri n)!; coordinate matrices relative to the stan­
dard basis and reduce to echelon form.

1 2 0 1 1 2 0 1 1 2 0 1

-1 1 3 0 () :3 3 1 0 1 1 0
1 -1 -3 1
---+ 0 -3 -3 0 ....... 0 0 0 1

0 1 1 0
0 1 1 0 0 0 0 0

1 1 -1 1
0 -1 -1 0 0 0 0 0

Thus the first, the second and th<> last columns are not linear combinations of
their predecessors namely v1 , 1' 2 and r· 4 form a hasis forth<' given subspace.

2) Find a basis containin~ :r and :z· + :z: 2 for the space of polynomials of
degree :s; 3 we know that { 1, x. :1· 2, .r:l} is a s<>t of genNa.tors for polynomials of
degree :s; 3. By adding a· + :r 2 to tlds s<'t we find a generating set

{,..I, .I,. + .I,:.! , 1, :!.,2 , J,,3}

containing the given polynomials. Now considering coordinates relative to

vectors in {1,x,.x 2 ,;1: 3 } we form

0 0 0 0
() () 0
0 0 1 0
() 0 () 0 1
l
which is row equivalent to

1 0 0 0
0 0 1 0
() () () 0
() 0 0 () 1
 146

Thus the columns to be considered are columns 1, 2, 3 and 5 of the first matrix
a.nd the required basis is {:z·, x + x 2 , 1, x3 }.
C) The solution space of AX =0
Consider the homogeneous system AX = 0. If x 1 and x 2 are solutions of
this system then AX1 = 0 and AX2 = 0 an hence

= cAX1 =cO= 0
A(cX1 )
that is to say all solutions of AX = 0 form a subspace of n X 1 matrices. Recall
that to solve the homog('n<'ons system AX = 0 we reduce the coefficient matrix
A and we determine hasic variables and free variables, by a.ssigning 1 to each
free variable and 0 to all other variables we find fundamental solutions

* * *
0 0

0 1 0
x2 = , ... ,xs =

0
note that they are linearly independent and any solution is a linear combina­
·. tion of them. So fundamC'nl al solutions form a basis for the solution space.

Example 2.2.26. Find a basis for the solution space of

x+y+z-t 0
2x + 2y- z + t 0
-x - y - 4z + 4t = 0.
 147

l [~ ~ l [~ ~ -: l
RPdncing the coefficicut matrix we have

[ 2
-1 -1
~ -: -:
-4 4
0 -3
0 0 -3
­
3 0 0 0
-11

0
.

So t lH' ~y~tem is reduced to

x+y+z-t 0

-z+t 0

wit II frl'P variable y and t. Now, taking y = I and t = 0 W(' get the solution

-2

and t h(' choice y = 0 and t = 1 givf's

0

So a basis for tl1e solution span' is

0
0
B=
1
0 1
148

SUMMARY
Let l' be a vector space over a field F .A subset S of V is said to be linearly
depend(n/ if we can find v 11 v2 , • • ·, Vm E V and c 11 c 2 , • • ·, Cm E F (not all
zero) such that
150

f»J EXERCISES
1. Determine whether (2, -1, 1, 4), ( -1, 1, 5, -2), (-3, 2, 4, -6) are linearly
independent .If not,write one vector as a linear cornbinetion of the others.

2. Show that if u, v and w are linearly independent vectors of a vector

space, so are u + v , u - v and u + v + w.
· 3.) Show that the functions

are linearly independent .

4. Prove that the functions

1x etdt ' 1x et2 dt ' 1x et3 dt

are linearly independent.

 151

S. \\'hat must be n if the vectors

(l.-n,1,1), (-1,n+2,-2,n-1), (0,2,-l,n 2 +n-4)
<m' linearly dependent?

(). \Yitich of the following sets of vectors span R 3 :

(a) {(:J,-1,4) , (5,-1,7) , (4,-2,5) , (1,-1,1)}

(b) {(2,-1,1) ' (1,4,-1) ' (-1,.),-2) , (0,9,-3)}
(c) {(0,0,0) , (1,-1,1) , (4,2,-l)}

1. D0tcrmine whether the following subsets of R 4 are linearly dqwnd<.'nt

or independ<•nt.In case it is linearly dcpe!Hient express one vector as
a li110ar combination of the preceding vectors,in the case it is linearly
ind(•Jwndent find a basis for R4
(a) {( 1,-1,2,3),(-1,4,1,1 ),( 1,2,5,7)}
(h) {(1,-1,1,1),(2,1,-1,1 ),(1,0,0)}
(c) {(2,-1,1,1),(1,-l,l,-1)}

8. rind a. basis for the su bspa.ce of R 3 generated by

(1,-1.,4), (;J,-1,4)' (1, 1,-4), (4,-2,8).

9. (a)Find a basis for the subspace of R4 generated by

(-2,1,-2,1), (1,0,1,0), (-1,1,-1,1).

(h)Sitow that (l,l,IJ) is contained in this subspace and express it in

t Nms of basis vectors in (a).

(c)What condition must be satisfied h:-· a,b,c and d if (a,b,c,d) is in this

subspace'!

10. Find a basis for the vector space of 2 X 2 symmetric matrices.

 152

11. Find a basis for the row space of the following matrices

2 -1 1 3 1 -1 1 3 4

-~
l
1 3 1 4
4 1 1 -1 -1 1 1 4 1
(a) (b) (c) [ 1 -1 1 0
1
0
2
3
-1
-1
3
-7
3
3
-1
1 4
2 0 1
7 6
-1 4 8 4 12 1:
12. Find a basis for the column space of each of the following

2 5 3 1 0 -1 0 1 -1 3
-1 1 1 0 1 0 -1 2 1 1
(a) (b) (r-)
1 1 -1 2 1 -1 1 1 1 -1
4 -1 1 1 -1 1 1 1 -1 1

13. Find the rank of each matrix below :

1 -1 1 1 4 2 1 -1 4
2 1 4 -5 6 1 1 3 4
(a) -1 4 1 -8 -6 (b) 3 2 2 8
2 3 6 -12 4 1 0 1 0
1 7 7 -20 -2 4 3 5 12

14. Find the order of the largest inwrtihle submatrix of

1 1 -1 1
4 2 1 0
2 0 3 -2
7 3 3 -1

(~ Let S = {(1, 1, 1), ( -1, 1, -1), (3, 2,3), ( 4, 6, 1), (1, 0, 4 )}. Find all subsets
of S which are bases for < S > by considering
(a) S as a subset of R 4
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ____!.l:i:l

(h) S as a subset of Zi

v'( c) S as a subset of Zj

16. Is (0,-1,2,-7) in Span{(1,2,-3,4),(1, 1,-1,-3),(2,3,-4, 1)}.

· 17i Find the number of subsets of

/
/

s= { ( 1, -1' 1), ( -1' 1, -1 ), (3, -4, 2), ( 4, -3, 5)}

p
which arc hases 1 <S > over R.
lr''r­

18. Find thl' number of all bases for Z~·

19. Expr<'ss

Span { (1, -3, 2, 1), (1, 1, 1, 1), (3, -5, .5, 3), (2, -6, 4, 2), (2, -2, 3, 2)}

as lit<' solution space of a homogeneous system of linear equations.

20. Express the space spanned by

1 3 -2
-1 -2
2 -1 3
4 1 3

as the solution space of a homogeneous system.

21. Find a basis for the column space of

0 1 3 -2
2 4 0 4 2
1 4 1 4
1 1 -1 0 0
-1 1 -1 1
(a) -1 -1 1 0 0 (b)
1 -1 4 -4
3 1 -.5 -4 -2
-1 1 5 -.5
4 0 -8 -8 -4
3 0 1 2
154

22. Find a basis for the solution space of the homogeneous syst('m

x+y+z-t 0

2x - y - 2z + 3t 0

-x + 2y - 3z + 4t 0

2x + 2y - 4z + 6t 0

23. Find a basis for the solution space of

2x- y + 3z +t 0

-5x+ y + 4z- t 0

-x- y + 10z + t 0

24. Find a basis for the solution space of

3 -1 1 2 1 xl 0
4 4 -2 1 2 x2 0
0 1 0 1 -1 x3 0
1 5 -3 -1 1 x4 0
0 0 0 1 -1 x5 0

25. Find a basis for the subspace of R 4 given by

{(a,b,c,d): a-b+c+d=0,2a+3b-d=0}

26. Find a basis for the plane

-3x + y + 3z = 0
in space.
 155

27. Find a basis for the vector space of polynomials given by

{a+ bx + cx 2 + dx 3 : a+ b + c = 0, a+ b + d = 0}
28. Find a basis for the space of 2x2 matrices

satisfying x + t = 0.
[: :l
29. Find the dimension of the vector space of polynomials generated by
x +x 2 , x - x2 +x3 , 2 - x - x3 , x + 1.
30. Find a basis for R 4 containing the vector (1,2,-3,0).

31. Find a.ba.sis for R 5 containing the vectors

(1,-1,1,1,-1) and (0,1,1,1,0).

32. Find a. basis for the vector space of 2 X 2 matrices containing

[ -1 2 1
4 1 -1
l and [ 1 0 1
0 1 0
l
33. Find a. basis for the vector space of polynomials of degree ::; 3 containing
1 +x + x 2 a.nd x + x 2 .
34. Find a. basis for R 3 containing {(1,-3,2),(4,-5,0)}

35. Complete the set {(1,-1,Ei,3),(-1,2,-l,O)} to a. basis for R 4 •

36. Supplement the set of matrices

[
to a. basis of the space of 2
~1 : l,[~ ~ l,[~ -: l
X 2 matrices.
 156

37. Find a basis of E 3 which is a sn hs<'t of

{(1, -2, 1), ( -2, 4, -2), (1, 1, 1), (2, -1, 2), (1, 1, 0)}

38. Find a basis for polynomials of d<'gree ::;3 consisting of polynomals from

(3~) Find the coordinate matrix of tlw function cos 2 x relative to tlH' ord<'fed
/ basis

{1, cosx, s1n:r., cos2x, sin2x}

i"'\
· 40. Find a basis for the solution space of y" + y' = 0.
41. Find a basis for the solution spac(' of y" - y' = 0.

42. What must be k if the functions

cosx+2sinx-3cos2x+ksin2:z:, kcosx-sin:z·+kcos2x+2sin2x, -cosx+sinx-5cos2x

are linearly dependent.For this valne express the last function as a linear
combination of the first two functions.

43. What is the dimension of t]w solution space of y111 + 2y" = 0 ?

-ctlagte•t a­
-,~ { h < " ! •

INNER PRODUCT SPACES

vVe have had some experience with real vector spac('s in analytic geometry.
We l1ad the idea of length and orthogonality-what is more the idea of angle
between two vectors- which can be carried over to a more abstract S('tting as
we shall do in this chapter.

For vectors v = (.1~1> x 2, x 3 ), and w = (y1 , Y2· y3 ) in space in addition to

"vector addition" and "multiplication by scalars" giv<'n by

v+ w
cv

we define the "dot product" by

This allows us to express,

158

II I'll= vv:v = Jxi +X~+ X~;

b) the distance between two points determined by the located Yf'dors
v = (x1,X2,x3), w = (Yt·Y2,Y3) by

c) the angle e between two wctors v and w by

V·W
cos e = llvllllwll"
(see Figure).

X3

v \.llv-wll
\

The formal properties that this dot product enjoys are:

(i) v · v;::: 0 and v · v = 0 iff v = 0, for v E R 3 ;

(ii) v · w = w · v for v, tl' E R 3 ;

(iii) (av + bw) · p = a(v · p) + b( w · p).

 159

All others are dedur('d from tlws<> and some other concepts can be introduced
by using them.

In this chapter we shall genNa.lb:e these ideas to arbitrary real vector spaces.

@II INNER PRODUCTS

\Ve begin with g<>neralizing tlte concept of dot product as inner product.

Definition 3.1.1.
A) Let V be a real vector space. A function from V XV to R assigning ( vlw)
to the ordered pair ( 1.1, w) is cal!HI an inner product on V if it sati.<;fies the
following axioms:

(Pl) (at'+ bwlzJ) = a(vlp) + b(wlp) for all v, w,p E V and a, bE R.

(P3) (vlv) > 0 for all non-zero 1' in ll.

B) A real vector space together wilh a specified inner product is called a real
inner product space.

A finite-dimensional real inner product space is called an Euclidean

space.

Theorem 3.1.2. Let ( I ) be an in·ner pm<luct on a vector space over F

(i) ( vlbw + cp) = b(vlw) + c( '~'IP) for all v, w,p E V and b, c E F.

(ii) (vlw) = 0 iff(wlv) = 0.

(iii) For any v in V (vlv) =: 0 iffv = 0.
160

Proof.
(i) Using (P2) and (Pl) W<' obtain

(vlbw + cp) = (bw + cplv) = b( wlv) + c(plv)

(ii) Using (P2) we obtain at once that

(vile)= 0 {:? (wlv) = (vlw) = 0

(iii) When v =f. 0 we have ( vlv) > 0 from (P2), and when v = 0, we have
(OIO) = (0. OIO)
= o · (OIO) = 0. Thu"s (vlv) ~ 0 for each v and v =f. 0 implies (vlv) ¥= 0. v = 0
implies ( vlv) = 0.
0

Example 3.1.3. Show that the pairing given by

is an inner product on R 3 .

In fact letting v = (x 1 ,x:J),w = (y 11 y2 ) and p = (z 11 z2 ) we have at•+ bw =

(ax 1 + by11 ax 2 + by2) and hE"nce the above formula gives

(av + bwlp) = (ax 1 + by1 )zt+ (ax1 + byi)z2

+ (ax2 + by2)z1 + 3(ax2 + by2)z2
a(xtZt + X1Z2 + x2z2 + Jx2z2)
+ b(y1z1 + Y1Z2 + Y2Z1 + Jy2z2)
a(vlp) + b( wlp).

and ( P 1 ) is verified. As for ( n) we have

 161

Finally

(vJv) xi+x1x 2 +x2x1+3x~

= :ri + 2x 1 x 2 + x; + 2x;
(a· 1 + :7' 2)2 + 2x~
shows that (vJv) 2:: 0 and if ( vJu) = 0 we have x 1 + x 2 = 0 and x2 = 0 which
means that v = (:1· 1 , :r 2 ) = 0.
Example 3.1.4. For v = (x 1 , ••• ,.r 11 ) and w = (y1 , • •• ,y,) ERn we define

(vJw) = X1Y1 + .Y2Y2 + · · · + XnYn·

Show that it is an inner product on Rn. It is called the standart mner

product (also called dot product or scalar product) on Rn. Unless another
inner product is specified, by "the inner product spaceR"" we shall mean
Rn with the standard inner product.

Solution. Let v = (x 1 , ••• , Xn), 11' = (yl, ... , Yn) and p = (z1, ... , Zn), then

(1) for any a,b E R,

n n n
(av + bwjp) = '2.::( ax;+ by; )z 1 = n '2.:: x;z; + b LY;Z; = a( vjp) + b( wJp);
i=l i=l

n n
(2) ( wJv) = LY;X; = L x;y; = (~·Jw) ;
i=l i=l

(3) (vJv) =xi+···+ x;l, > 0 when X; =j:. 0 for some i, equivalently when
1J =1- 0.
 162

Example 3.1.5. Let V be the space of real m x n matrices. Show that

n m
(AlB)= I: I: B~:iAJ:i·
i=l 1:=1

is an inner product on Rmxn.

Note that (AlB) is nothing but the standard inner product obta.in€'d by
considering the matrices A= [Ai~:] as vectors with mn components A;J:.

Example 3.1.6. On Rnxl there is an inner product which can be defined as

(XIY) = yrx

by identifying the 1 x 1 matrix yr X with its unique entry

This example is a. special case of the inner product given above

Example 3.1.7. Let C[a, b] be the real vector space of real continuous func­
tions defined on a closed interval [a, b]. The following is an inner product on
V:
(Jig)= 1b f(x)g(x)dx.
This inner product will be taken to be the standard inner product on C[a, b].

In fact, if j, g and h are continuous functions on [a, b] and c and d are real
numbers we have

(1)

(cf+dglh) = 1b(cf(x)+dg(x))h(x)dx

= c 1b f(x)h(x)dx + d 1b g(x)h(x)dx
= c(flh) + d(glh)
 16.1

(2) (.fJ.q) = 1b f(:r)g(x)dx = 1b g(:r:)f(:r)dx = (gif)

(3) To \'<'rify ( P:l) take a continuous function on [a, b] such that f( :r 0 ) :j: 0
for some :r 0 E [a, b]. Letting L = f(x 0 ), from continuity off\ we see
that there exists o > 0 such that

~ . ~~
IJ-(:r)-L- < F·
2 for :ro-{!<x<x 0 +o
. /,'2 ., ., L:!. .
that 1s - - < j-(;r)- L- <-and eqtuvalcntly
2 2

J/ ., ;~U
-. < J-(:r) < - for :r 0 - b < :r < :r 0 + o.
2 2

Thus 1hNP is a subinterval [c. d] of the int<'rval [a, b] surh that c < d and
/} .,
T < J- (:r) if c ::; :r ::; d.

Th<'n

1b •)J-( .r )d.r = ~r J-(.r )d.t: + jd J-(.r

., ., )d.t: + lb ., J}
j-(.r )d.r ~ 0 + -, (d- <') + 0 > 0.
a • " r d 2
(see Fip;ur<').

y
3 L2/2 1-----..,,c__ __:~

a b
164

fill EXERCISES
1. Show that the following functions from R 2 xR 2 into Rare inner products
on R 2 :

a) (vlw) = 2a'IY1 + X2Y1 + X1Y2 + 2x2Y2

b) (viw) = 3a~IYt- 2x2Y1- 2XlY2 + 2X2Y2

2. For each of the following, either show that the given function is a.n inner
product onR2 or provide a.n example violating one of the axioms (Pl),
(P2), (P3).

a.) (vjw) : :': : 2xi + X1Y1 + X2Y2

b) (vlw) = X1Y1- XtY2- X2Y1 + 4X2Y2
 165

c) (rJw) = J:lyl- XJ:th + .l''2.1JI + 4:7:2.1J2

d) (PJw) = XJ.IJl + 2.r1.1h+ 2.r:>..IJJ + 4:1:2Y2

:3. Show that the function ddinPd by

()
(
for v = (x 1,x 2) ami w = (y 1 ,,1}'2) is inner product on R 2 iffb = c, a> 0
and ad- be > 0.
.1, Given v = (1, -2, 1,3), w = (2. 0, 1, -1) in R 4 with the standard inner
product. Evaluate:

a) (vJv) b)(vJw) c l ( wIt' l d)(wJw)

5. The trace of a matrix .-1 is t h<' sum of its diagonal entries Jlii and it is
dPnoted by Tr( A). For mat ric<'s

A=
1 -1
[ .
1 0
I
2
l and B = [1
2
1
-1
-1
1
l
evaluate:
a) Tr(,lT H) b) Tr(JJTt\) c) Tr(AT A)

d)Tr(BTfl) e)Tr(AT(IJ+:I)) f)Tr((A+2B)Tfl)

Derive a. formula. for Tr(illl) and show that Tr(ATB) = Tr(BTA) for
arbitrary m x n matricPs . I and /1.

G. Consider the functions f and g ckfined by

f(:r) = (.r + !)", g(x) =x

in the inner product space C[-1.0]. Evaluate (fJg).

1. Considering the polynomia 1 funrt ions f( .r) =a +x and g( x) =x in the

ilin<'r product space C[O.G] d!'lPrmine a such that (fJg) = 0.
166

8. Let V be any in n0r product space over F. If v f 0 and w an- two V<'ctors
in V determinE' c E F such that (cvlw- cv) = 0. For this value of c
prove that
(wlw) = (cvlcv) + (w- cvlw- cv).
Show that for Y = R 3 the vector cv is the orthogonal proj0rtion of w
along v.
 Jfi7

@!f.J NORM AND ORTHOGONALITY

Inner product axioms (PI), (P2), (P:J) and tlwir consequences giYPil 111

Theor('m 1 allow us to define "le11gt h" and "orthogonality''.

Definition 3.2.1. Let (I ) be an inner product on a 1'frtor space V orrr r.

a) For v E F, flu· length (or norm) of r, ll'rillrn as !lull is defined hy

11 1 ·11=~.
If llrll = 1, then u is called a unit vector (or normalized vector).

b) Tu•o vectors v and w in V an said to be orthogonal if

(/'Ill')= 0.

c) For any sub,c;et S of V thr sri of all l'rcfors u•hich are orthogonal to all
vectors in S is called the· orthogonal complement of S and is denoted
by s1. (read ··s pcrp ").

Example 3.2.2. The norm of a \'('<"tor r = (J:I! ... , :r,) in R" with respect
to the standard innN product is

II vii= j.ri + · · · + .r?, if v E R".

For instance WP have

II( V:"i, -:3,-2 lll = J( V:"i l" + (-:3) 2 + (-2F = ,~,

168

Example 3.2.3. In the inner product space of continuous functions on the

interval [-11, 11] with the integral inner product, find the norms off(x) = sin x
and g( a:) = 1 and show that they are orthogonal.

Solution. We have

a.)

= (sin xI sin x) =1 1r
sin 2 x dx = -111r (1 - ms 2.1· )dx
-1r 2 -1r
~ (x _ sin2x)l1r = 11 (l)
2 2 -1r '

giving II sin xll = .ji;

b) 111 If= (111) = 1: dx = 211 giving IIlii = $ ;

c) (sin.rll) = J:.1rsinxdx = -cosx[1r = 0.

The basic properties of norm and orthogonality are given in the following
theorem. These are the most important properties in all mathematics.

Theorem 3.2.4. Let (I) be an inner product on a vector space lf.

(i) For any v E l', we have llvll 2': 0 and

II vii = 0 iff v = 0.

(ii) For any v E V and c E F, we have

llcvll = lclllvll·
(iii) If n and w are orthogonal vectors in F, then

llv + wll 2 = llvll 2 + llwll 2 (Pythagoras theorem).

 16.9

(iv) For any v,1t' E V, we have

J(rJu·)J::; !lvllllwll (Cauchy- Schwarz ineqttality).

(v) For any t', u· E l", we have

ll1• + ·u·ll :S llvll + llwll (triangle inequality).

(vi) For any subsrt S ofV,Sj_ is a subspace ofV.

Proof:

(i) It is obvious lhat

llvll = 0*) 2: 0.

The fact llt·ll = 0 iff v = 0 follows from Thcorcm 1 (iii).

(ii) By using the dPfinition of norm, inner product axioms and Theorem 1
we get

(iii) From (vJ'!I') = (wJr) = 0, we obtain

(iv) If v = 0 or 1r = 0, the inequality is obvious, so we may assume w =f 0.

( uJw)
Now, letting c = -11 w II~
~
wc observe that the vectors cw and v- cw are
orthogonal:

( vjw)
(v-nujnl') = d t•-cwJw) = c((vJu·)-c( wJw)) = c((vJw)---· (wJw)) = 0.
. llwll 2
170

Thus using the Pyhagorean theorem proved above we get

which implies by (i) and (ii) that

0~ llv- cwll 2= llvll 2 -llcwll 2 = llvll 2- Jcl 2 Jlwll 2

o ~ llvW- I( vJw Wllwll 2
llwJJ4

therefore J(vJw) ~ JlvJJIIwiJ as required.

(v) We have

llv + wll 2 = (v + wJv + w) = (vJv) + (vlw) + (wJv) + (wlw)

= llvll 2 + 2J( vJw )J + llwJJ 2 (2)
Now, from the Ca.uchy-Schwarz inequality it follows that

that is to say
llv + wJJ ~ llvll + llwll.
(vi) By definition,

S.L = {w E VJ(wJv) = 0 for Vv E S}.

Now. (OJt') = (0 · OJv) = O(OJv) = 0 for each v E S showing that 0 E S.L.

If 1L'h w2 E S.L and c1 , c2 E F, then for any v E S we have

(c1W1 + C:;W2Jv) = Ct(wtlv) + c2(w2Jv) = 0 that is to say CtWt + C:;W2 E S.L.

Thus S.L is a subspace of V.
0
 171

Corollary 3.2.5. If V is an inner product space, the distance function d

defined by
d( v, w) = llw - t'll sal i8]ies the following:

a) d(v,w) ~ 0 and d(v,w) = 0 iff v = w

b) d(v,w)= d(tl',1')

c) d(v,u):Sd(l',ll')+d(w,tt)

Proof. By using the properties (i), (ii) and (v) respectively we have

a) d(v, w) = llu'- vii ~ 0 and llw- vii = 0 iff w- v = 0.

b) d(v, w) = 1111'- rll = ll(v- w)ll = 1- llllv- wll = d(w, v)
c) d(v,u) = llu-rll = llu-w+w-vll :S llu-wll+llw-vll = d(w,u)+d(v,w).
0

Corollary 3.2.6. If l' is a real inner product space, then for arbitrary non­
zero oectors v and w in V, we hcwe

-1 < (vlw) < 1.

- llollllwll -
Proof. 1t follows from ( iv) in the previous theorem that

0 )-'-:-I
'l,---:
.,.:-I(..::..,.v,..:.,-1 < 1
llvllllwll-
Since ( vlw) is a r0al number it yields
(vlw)
-1 < < 1.
- llvllll·wll ­
0
172

This corollary shows that the following definition is reasonable in the real
case.

Definition 3.2. 7. Jf v and w are vectors of a real inner product spnce, the
angle e brlu•pen them is defined by
(vlw)
cos e = llvllllwll; 0 s () s 1f

Example 3.2.8. Find the angle between v = (2, -2, 0,-1) and
w = (1, -1, -1, 1) in R 4 with standard inner product.

Solution: "'<' have

(viw) = 2+2+0-1=3
llvll v'4+4+0+1=3

llwll v'1 + 1 + 1 + 1 = 2 (3)

(viw) 3 1
cosO= llvllllwll 3.2
- and 0
2
s es 7r.

Thus e= ~·

Example 3.2.9. For arbitrary real numbers a, band e prove that

iacos() + bsin81 s; v'a 2 + b2

Solution. Taking v =(a, b), w =(cosO, sinO) in R 2 (with the standard inner
prodnrt) and applying tl1e Cauchy-Schwarz inequality we get

Example 3.2.10. Find the orthogonal complement s1. of the set

s = {(1, -1, 1), (1, 1, 0), (2, 0, 1)}

 173

in the inner product space R 3 .

Solution. A vector (xbx 2,x3) in R 3 is in Sl. iff it is orthogonal to each

element of S, so (x:h x:11 x3) E SJ.. iff

(4)

Thus elements of Sl. are of the form (t, -t, -2t), i.e., Sl. is the subspac<'
spanned by {1, -1, -2).
174

BlfJ EXERCISES

1. Given v = (-2, 1) in R 2 • Find the norm II vii with resp0ct to:

a) the usual inner product

b) the inner product given by

2. Evaluate the norm of

A= [ 2
1 -1
-1
1
. -1 -2

with respect to the standard inner product on 3 X 2 matrices.

3. Show that ll~ll v is a unit vector for any non-zero vector of a real inner
product space lf.

4. Normalize:

a) (2, -1, 1, -2) in R 4 b) (1,1,2,2)in R 5

c) [ 10 -J3
-1 l in R2x2

5. Let F be the space of polynomials of the form ax +b with the inte­

gral inner prodti.ct on [0, 1]. Find normalized elements of l1 \vhich are
orthogonal to x.
 175

6. Let V be an inner product space. Prove that if v and w are non-zero

vectors of V satisfying

l(vlw)l = llvllllwll, then v = f:},~w and w = ~:,':jv.

7. If v and w are two vectors of an inner product space prove that

8. If v and w are in a real inner product space prove that llvll = llwll iff
( v + wjv- w) = 0. Interpret this geometrically in R 3 •

9. Let v and w be vectors of a real inner product space. Prove that

llv + wll = llvll + llwll

iff v = cw or w = cv for some non-negative real number c. If this is the
case prove that (vjw) h; a non-negative real number.
10. J,et A, B, C, D be four points in R 3 and let P and Q be the midpoints
of [AB] and [CD] respectively. Prove that AD I I BC iff

IPQI = IADI + IBCI.

2

11. Prove the following inequalities:

12. If v and w are non-zero orthogonal vectors of an inner product space show
that llv + wll > ll·vll and llv + wll > llwll. Interpret this geometrically in
R3.

~~-----------------------~
I
 176

13. Find the angle lwtweE>n the following vectors of the given real inner
product space:

a) v = (1, -2, -3), w = (2,3, 1) in R 3

b) v=t+1, w=tinC[-1,1]

c) A= [ 2 1
0 '-2
l' B = [ -1 1
1 1
l in R2x2

14. Find a basis forth<' orthogonal complement of the set:

a) S = {(4,-3,2,-1),(8,-7,5,-30)} in R 4

b) S={(1,1),(1,2)}inR2

c)S={[: :]la,bER} inR"'

15. Which conditions should be placed on the coefficients of a polynomial
function f(x) = a0 + a1 x + · · · + anxn in C[O, 1] so that f is in the
orthogonal compl<'ment of the space of polynomials of degree ~ 1?

16. Let S 1 and S2 be suhspaces of an inner product space such that S1 ~ S2.
Prove that Sf ;;2 Sd-.

17. If 8 is a. subset of an inner product space V, prove that Sl. = (S)l. and
(S} ~ Sl.l. wherE' (.'i} is the subspace spanned by S.

/
 177

ECISI ORTHOGONAL AND ORTHONORMAL BASES.

As we do in the plane R 2 and spare R 3 , it will be useful to describe an
inner product space by means or hasf's consisting of mutually orthogonal unit
vectors. We begin by defining orthogonality and orthonormality of sets.

Definition 3.3.1. Let V be an inwr product space and S a subset of 11.

r:/.1

(i) S is callf'd an orthogonal set if (nJ,6) = 0 for any pair~! ,;lislincl

vectors in S. If in addition S i,.,. a basis it is called an orthogonal basis
for V.

(ii) S is called an orthonormal set if it is an orthogonal set consisting

of unit vectors. A basis ll'hich is an orthonormal set is called an or­
thonormal basis for V.

Note that by normalizing vectors or an orthogonal set of non-zero vectors we

obtain an orthonormal set.

Example 3.3.2. In R 3 with the standard inner product, the sets

s1 = {(L1,1),(-2,1,1),(o.-Ll)}, s2 = {(3,~,-1),(-1,2,-2)}
are orthogonal sets and the sets

s3-- {(-1
/3' _!__ J..-)
y'3' J3 '

84 = {(1.0,0),(0,1,0),(0,0,1)}

178

are orthonormal sets. The orthonormal set 8 3 has been obtained from S1 by
normalizing its non-zero vectors.

Example 3.3.3. Show that the set consisting of

1, cos x, cos 2x, ... , cos 71 x, ... , a 71 d sin x, sin 2x, ... , sin mx, ...

IS an orthogonal subset of the real space of continuous functions on [-1r, 1r]

with respect to the integral inner product. Normalize these vectors to obtain
an orthonormal set

Solution. If n and m arc distinct integers, then

(cos nxj cos mx) = 1:,. cos 71X cos mx dx

1 1
-[cos(71 +m)x + cos(n- m)x]dx
_,. 2
~ [sin(71 + m)x + sin(n- m)],.
2 71 + m n- m _,.
0

(sin nxl sin mx) 1,.

_,.
sin n.1· sin mx d:t=--:-
11,.
2 _,.
[cos(n+m )x-cos(n-m)x]dx

1 [sin(n+m)x sin(n-m)x],. _
-- - -0.
2 71 + m n- m _,.

For arbitrary natural numbers m and n

(cos nxl sin mx) = j,.

_,. cos 71X sin mx d;r = 0
since the integrand is an odd function. Th<'rdore the giv<'n s0t is an orthogonal
set of non-zero functions. To normaliz<' its members W<' compute the norms.
We have
( 1j1 ) = I: d:r = 2rr
 179

and for each positive integer n W<' oht ain

(cos nx I cos n:r) ! .- 1

cos 2 nx dJ· = :--
2
j,. (1 +cos 2nx )dx = 1r +
sin 2nx
4n
I" = 1r
1.- 11"' (1
-1r -11'" -11'

(sin nx I sin nx)

_,.
sin 2 m: d:r = :-­2 _,.
+cos 2nx )dx = 1r­ sin4n2nx I~,. = 7r
Thus

IIlii = V'fi and II cos n;rll = II sin nxll =Vi for each n EN+

and the set

1 \OS .r s1n x sin 2x
V'fi' Vi' fi' Vi, ...
is an orthonormal set.

Our next theorem shows that ort l10gonality implies linear independence (of
course not conversely). This is an ohvious fact in R 3 , for example three mutu­
ally orthogonal vectors cannot lie i 11 the same plane, two non-zero orthogonal
vectors cannot be parallel to tlw sam<' line.

Theorem 3.3.4. If S is an o1'lhorJmwl set of non-zero elements of an inner

product space, then S is linearly inrlrpnulent.

Proof. Let v1 , ••• , Vm be vectors t a k<'n from S satisfying a linear relation

Then, for each i we have

r;( I'; it';)= 0

c, = 0.
180

since ( vkiv;) = 0 (when k f:. i) and ( v;iv;) f:. 0. D

For example let us show that the vectors

v1 = (1,0,0), v2 = (0,1,-1), va = (0,1,1)

form an orthogonal basis for R 3 with respect to the standard inner product.
Indeed, we have

Thus { v1 , v2 , v3 } is an orthogonal set consisting of non-zero vectors of R 3 •

By Theorem 3.3.4 it is linearly independent and is a basis since it has three
elements ..

Example 3.3.5. Let us use the orthogonality of sine and cosine functions in
C[-1r, 1r] to evaluate

I= l", (1 + .5cosx- 3sinx + cos3x +sin 8x) 2 dx.

In fact from the values (111) = 21r, (cosnxicosnx) = (sinnxjsinnx) = 1r for

n f:. 0 we obtained in Example 3.3.3, we get

THE GRAM-SCHMIDT ORTHOGONALIZATION PROCEr',.

The following theorem shows that every finite dL.nensional inner product
space has an orthonormal basis and gives us a procedure to construct it.
 181

Theorem 3.3.6. {Gram-Schmidt) !J/ v 1 , v2 , .•• , Vm be linearly independent

vectors of an innr-r product spact \ ·. Tltf·n the vectors defined by

11'1

'll'tn

are mutually orthogonal.

Proof. \Ve prove the theorem hy induction on m. Since the case m = 1

is va.cous we assmne the assertion is true for m.- 1, namely ·wh ... , Wm-l are
mutually orthol!;onal. Then we lia\"P ( U';h) =0 for all i =/:- j taken from the
set { 1.... , m - 1}. Then for any l~ ::::; m - 1 we have

(1l'k'U' 111 ) =

Thus ll' 1 .w 2 • · ··,·wm arc mutually orthogonal. 0

The <:mm-Sr·hmidt m·thogonafi::alirm proress gives us an orthogonal basis

out of any basis for a finite dinu nsional inner product space. If necessary
. dividing ((/(·h wk consfntctcd in thr lhmrnn by its norm we gft an orthonormal
set.
0

Example 3.3.7. Consider the innN product space C[O, 1], find an orthogonal
basis for (1.:L"J 2 ).

Solution. Tl1e Cram-Schmidt ort hogonalization process applied to (1, x, x 2 )

 182

will provide us with the required ortlwgonal basis. We calculate,

Pt(x) = 1 with 11Pdl 2 = 1d~

1
= 1

(xl1) [1 1 2 [1 ( 1) 2 1
p 2 (x) = x - -
1- ·1=x- Jo xdx=x-2 with IIP2II = Jo x-2 d:r= 12

Pa(x) = x2- (x2j1)1- (x2jx- 1/2) (x- ~) = x2- ~- (x- ~)

- . 1 1/12 2 3 2

= x 2 -x+­61

ORTHOGONAL PROJECTIONS

Given a subspace of an inner product space and a vector of the sparE', the
idea of finding a vector in the snhspar<' which '6;6 fits to the givC'n V<'rtor
/

has many useful applications. In this section we discuss the exist<>nre and
uniqueness of such a vector, and give some important applications.

Definition 3.3.8. Let lf be an inrucr prod·uct space, W a subspace of l/ and

·t' a t•ector in V.

(i) A vector p in W is called an orthogonal projection of v in W if ·11- p

is orthogonal to all vectors in TV.

(ii) A vector p in W is called a best approzimation to v by vectors in

lV if ll·v- Pll ~ llv- wii.Jor all wE lV.
-----------------------------------------------------------~!8:?

We prove below that when l'V is finite dimensional the concepts of ort ]Jog­
anal proj0ction and best approximation coincide and for each v E V t lH'r0 is
a unirp1e orthogonal projection in l¥. This fact is an obvious conS0<JI1f'lllf'
of geometric considerations in space but it is not obvious at all for arb it rar.v
inner product spaces.

Theorem 3.3.9. Let V be an inner product space and let lV be a Jinilr

dimensional sttbspace 'lL'ith a basis B = {Vt, ... , v,}. Each v E V has a un iquf'
orthogonal projection p = x 1 v1 + · ··+ x, v, on l'V which is determined by

(vlvt)

~Moreover,

(vlvJ) (vlvn)
(i) P = · I . V1 + · · · + (v,v,)
(v1v 1)
I . '1! 71 when B is orthogonal

(ii) p is the unique best approximation to v by vcctor·s in l1'

(iii) llvll ~ IIPII and equality holds iff v = p.

Proof. Let { v1 , .•. , v,} be any basis for lY and let p = x 1 v1 + · · · + x, 1'" h<'
a vector in Hl. Then, v - p is orthogonal to all vectors of H' iff

(v - PI v;) = 0 for all i = 1, ... , n

equivalently
(plv;) = (vlv;) for all i = 1, ... , n.
Thus p = xlvl +·. ·+XnVn is an orthogonal projection of'/) on vV iff (xh ... ' ;rl/)
is a solution of the system
184

In particular if B is an orthogonal basis for W, (which can be constructed

by Gram-Schmidt process) thE)n the above system takes the form

(11;jv;)x;=(vlv;); i=l, ... ,n

and it has the unique solution

i=1,2, ... ,n.

This yields

Thus it remains to prove that, llvll ~ i!PII and the orthogonal projection p
is the same as the best approximation. In fact, for any w E TV, we have
(v- PIP - w) = 0 since p - w E W and hence

by the pythagorean th<'orem. Therefore

llv- Pll < l!v- wl!

for any w different from pin W, i.e., pis the unique best approximation to v
by vectors in W. Finally for w = 0 the above equality gives
Thus llvll ~ IIPII and equality llolds iff v- p = 0. This completes the proof.
0

In particular if we take only I hf' non-zero vector w the orthogonal projection

of von (lu) (we also say the ortho~onal projection of v along w) becomes
(v!w)
p=--w
(w!w)
since {w} can be considered to I)(' an orthogonal set. Therefore by the last
part of the theorem we obtain

!!vii~ IIPII = j(vjw)111wll = !(v!w)l

(wjw)- llwll
or

which is nothing but the Cauch.v-Schwarz inequality that we obtained before.

Now we shall give Bessel's inrqualily which generalizes Cauchy-Schwarz in­
equality.

Corollary 3.3.10. If { vh . .. , 1'11 } is an orthogonal set of an inner product

space F then fo·r each v E l' U'f haN'

. and equality holds if]

Proof. If we take lV = ( n1 , ••• , r,), then the last part of the theorem says
that llvll 2: IIPII and equa.lity holds iff
(rlr1) (vlun)
v=p=
(/'II I'll
7'1+ .. ·+'(1' lv,) L'n·
71
186

On the other hand by Theorem 3.3.9, IIPII can be expressed as

and the proof is completed. D

Now we illustrate Th('orem 3.3.9.

Example
3.3.11. Find the clos<'st vector to (5,2, -1,3,4) in the subspace generated
by the orthogonal vectors ( 1, 1, 0, 0, -1 ), (1, -1, 1, 0, 0), ( -1, 0, 1, 0, -1 ).

Solu.tion. The closest VC'ctor to v = (5, 2, -1, 3, 4) is the orthogonal projection

and it is obtained as

3 2 10 1
P = -tc 1 + -w.,- -w3 = -(15 -8 0 7)
:3 3-3 :3' ''

by using Theorem 3.3.9.

Example 3.3.12. Find the orthogonal projection of the function ex on the

subspace ofC[-1r,1r] generated by cosx and sinx.

Solu.tion. Since cos x and sin x are orthogonal and

(cos xi cosx) = (sin:rl sin :r) = 1r, the orthogonal projection of ex becomes

p( x) = ~ (l: e"' cos xdx) cos x + ~ (/_1r1r ex sin xdx) sin x.

Now, from

1: ex cos xdx = e"' cos :1' [1r + J: e"' sin xdx =- 2 cosh 1r + e"' sin x [1r-;_: cos xdx
 !S7

we obtain

j_1r" ex cos x dx = -cosh 7r and j_: ex sin x dx = cosh 7r

and thus
cosh 7r •
p( X) =- -(Slll X -
7r
COS X).

APPLICATION 1: THE METHOD OF LEAST SQUARES

Suppose a quantity y is known to depend on the quantities xh x 2, • · ·, .?'"

by a linear relation

where the coefficients bt, b2 , • • • bn are to be determined by means of a sNiPs

of experiments. If we obtain the value of y to be Yk in the k th experinwnt
carried out by taking x 1 ,x 2 ,···,x,. to be the values akbak 2,···,akn• tlwn we
have

(k = 1,2,···).

Thus after m experiments carried out, we should have

Y1 au a12 aln
Y2 an an a2n
=bG + b2 +···+ bn
Yrn aml arn2 a.mn

for some real numbers b1 ,b 2 ,···,bno But because of possible experinH'ntal

188

errors, this system may be inconsistent, i.e.,

Yt
Y2
Y=

Ym
may not be in the subspace spanned by the vectors

In this case it is reasonable to take bt. b2 , ••• , bn in such a way 1l111t b1 Vt +

b2v2 + ··· + bnvn is the best approximation of Y by linear comhinations of
Vt, ... , Vn. Thus by Theorem 3.3.9 b1 , ••• , bn will be obtained hy solving the
system

bn (Yivn)
The solutions b17 b2, · · ·, bn are the values to be able to get the ](last value of
m

IIY- (blvl + ~ · · + bnvn)ll 2 = 2)YA:- (btakl + ·· ·+ bnnkn)f.

A:=l
This is a sum of squares and this is why the method is called the nu !hod of least
squares and (bt. ... , bn) is called the least-square solution of t lte inmnsistent
system under consideration.

Example 3.3.13. A researcher knows that the quantity y is rehlt<'cl to the

quantity x 1 by a linear relation

y = ax 1 + b
 189

where a and b ar0 r0al numbers to be determined. After a series of experiments

one obtains thP following results.

Xt y
1 2
2 1
0 2
1 1
1

Find the valnC's of a and b which best fit th(•se results.

Solution. By taking :r 2 = 1 in the relation y = a:c + bx

1 2 for each exp<>riment

we obtain
2
2 1 1
VJ () v2 = and Y= 2
1 1 1
1 1

So the system which yiPlds "best" values of a, b, cis

that is to say

Ia + Sb G

.sa+ .Sb 7

. .
g iVlllg a
-
= --21 and b = -.
19

10
Thus the relation should be

y = -0.5x 1 + 1.9.
190

Example 3.3.14. Let us find the parabola y = ax 2 + bx + c which best

fits to four points in Fig. 5.

y
7
,.---·-·-·
j 6;

I I
:
!-1
!
!
_ __.:_-.+~--+X

-2! 2
'
f
;

l. -7

The data can be tabulated as follows

y x2 xl xo
6 4 -2 1
-7 1 -1 1
7 0 0 1
6 4 2 1
Denoting the column matrices appearing in the table by v, v 2 , vi> v 0 in the
order of appearance we see that our problem is to determine a, b and c so that
av2 + bv1 + cv0 is the orthogonal projection of von (v2 , v1, v0 }. Vl{e compute

(vlv2) 33, (vdv 2 ) = -1, (volv2) = 9

(vlvl) -7, (v 2 lvi) = -1, (v1lv1 ) = 9, (volvd = -1
(vivo) = 12, (v2lvo) = 9, (vdvo) = -1, (volvo)= 4

and obtain the system

33a - b + 9c = 41
-a+ 9b- c -7
9a- b + 4c 12
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ___,_I.'JL

which has the unique solution (a, b, c)= (1, 1, 1). Thus the required parabola
JS

y = x2 + x + 1.
APPLICATION 2: FOURIER SERIES

fpt 8 = {<P 0 , <1> 11 •.• , <Pm, •..} be an orthogonal subset of the inner product
spac0 C[a, b] and let f be a. function in C[a, b].The orthogonal projection off
on t]Jf' subspace spanned by <P 0 ,<P 11 .•. ,<I>m is

The numbers

k = 0, 1, ... , m, ...
which e~ppea.r as coefficients in these projections, are called the Four·icr coeffi­
r.inlfs off relative to the orthogonal set S and
00

L Ck<I>k(x)
k=O

is ('alhl the Fou·ricr series off relative to S.We write

00

f"" L Ck<l>k(x).
k=O

l\o1P that here

is nothing but the best approximation to f by functions in ( <I> 0 , ••• , <I>n)·

Tn particular by taking the orthogonal set

{1, sinx, cos x, ... , sin nx, cos nx, .. .}

 192

in C[ -1!", 1l"] we obtain the trigonometric Fourier series

ao oo f(
f(:r)"" 2 + L)an cos nx + bn sin x)
n=l

with Fourier coefficients

1
an=-
1l"
f..
-.-
J(.r) cosnx dx, bn = -1
1l"
j . f( x) sin nx dx.
-7r

For example if we take f( .1') = x we obtain

(-1)n-1
a,.= 0, b,. = -'---'-­
n

· and thus
~ - ------------- -~-----
---------------------..

19.")

r
I
194

Bl¥1 EXERCISES :mrm **

1. Considering the standard inner product on 2 X 2 real matrices verify that

{[ -112]
5
' [ -31 02] [48 02] }
is an orthogonal set.. Find a matrix which is orthogonal to every element
of this set

2. Verify that S = {1, :r- t, x2 - x + i} is an orthogonal subset of C[O, 1].

Use this fact to compute
 195

(a) fo 1 [2+~(r-~)+s(x 2 -x+~)r dx,

(b) the orthogonal projP.ction of x 2 on (S).

3. Find the orthogonal projection of f(x) = lxl.

(a) On th<' subspace ofC[-1r,1r] spanned by {sinx,sin2x}

(b) On th<' subspace of C[-1r, 1r] spanned by {1, sin 1, cos x}

(c) On th<• subspace ofC[-1r,1r] spanned by {1,cosx,cos2:r}.

4. Find the orthogonal projection of ( 6, 0, 0)

(a) along t lH' v<>rtor ( 1, 2, 1 )

(b) on tlw plan<' determined by (1, 2, 1) and (1,-1,1)

(c) on the subspace spanned by (1,2,1),(1,-1,1) and (3,0,-3), (ex­

plain t hP rr'snlt ).

5. \Vrite cos 3 ;rasa linear combination of l,cosJ:,cos2x,cos3x and find its

projection on {l,sin:~:,sin2;r,sin3x} over the interval [-1r,1r].

6. Find the orthogonal projection of (1, 1, 1, 1, 1) on

(a) (1,0, 1.0.1)

(b) (-1,-1,-1.-1)

(c) thesnhspar<'ofR 5 spanned by (1,-1,1,-l,l)and (1,0,1,0,1).

7. Apply the Gram-Schmidt orthogonalization process to the following sets

(considering standard innN products) to produce mutually orthogonal
vectors:

(a) (1,-1.1).(2.0,-1) in R 3

(b) (1,1,1.0.0.0),(1,1,1,1,1,1),(0,0,1,1,0,0)in R 6
196

-'-1
1 l in R2x2

(d) l,x,x 2 ,x 3 in C[-1,1]

(e) cos 2 x,sin 2 xinC[0,21r]

8. It is known that a quantity y depends on quantities x 1 and x 2 by a linear

relation. The result of a series of experiments is tabulated as follows

x1 1 1 2 3 1 2
x2 1 2 1 1 2 1
y 221121

Find the linear relation which best fits these results.

9. Find the Fourier expansion of f(x) = 1-lxl over [-1r,1r].

10. Find the Fourier sine expansion of f( x) = cos x over [0, 211"].

11. Find the Fourier cosine expansion of f(x) = sinx + cosx over [0,211"].
 197

DIAGONALIZATION AND ITS

APPLICATIONS
Since diagonal matrices are <'XI r0mely simple to k1 n dle it is advantageous
to develop a procedure of lliagonali::ation to facilitat0 computations. Such a
procedure has many applications.

@II EIGENVALUES, EIGENVECTORS AND DIAGONALIZATION 11111

A square matrix il is said to he diagonali::able ir there exists an invertible

matrix P such that p-l AP is a diagonal matrix.

Suppose now that A is diagnna lizable namely t lwr<> is an invertible matrix

P with linearly independent columns P1 , ••• ,Pn such that

p- 1 AP=D

where D is a diagonal matrix, say D = diag()q, .... .A,). Then we have AP =

198

PD, namf'ly

Thus if A is diagonalizahlc then we can find scalars )q, ... , >., and linearly
indepencknt column matrices P1 , •.. , P, such that

Reversing th<' argumf'nt W<' can say that if we have scalars and column matrices
as we l1a\'0 just d0scrih<'d then

r- 1 AP = diag(>.h ... , >.,)

namely A is diagona1izah1P. Thus we see that the problem of diagonalization
consists of finding scalars .>. and non-zero column vectors X for which AX =
.XX.

Definition 4.1.1. Gi\"f'n a square matrix A, a scalar ). is called a char­

acteristic value (or Fgir·nrrilue) if the system has non-trivial solutions. For a
characteristic value ). a 11 nonzero vectors X satisfying

AX=.\X

are called charactrri.<;! ir· rrdors (or eigenvectors) corresponding (or belonging)
to >..

Now, the abovP dis\ussion can be summarized in

 199

Theorem 4.1.2. An n X 11 matrix is dia.gonalizahle iff it has n linearly

indep<'tHIPttt f'i)!;Pnvectors. 0

At this point we also note that ,\is a characteristic value iff (.\I- A)X = 0
has a nontrivial solution equival<'tllly ,\is a root of the polynomial

~ . tUl = dl't(tf- A).

This polynomial is known as the <"hamderistic polynomial of A, the equation

dPt(Al- A)= 0

in turn is called the chamcterislir £qua/ion of A.

Example 4.1.3. Let

A=r-:; -: -:J

Find an invertible matrix P and a dia~onal matrix D such that p-tA P = D.

We find the characteristic equation and its roots:

.\ + 11 5 3
det(.\I- A)= -12 .\- 7 -2 =(A- 1)(.\- 2)(,\ + 3) = 0.
-12 -!i .\ -·1
Thus the eigenvalues of A are

.\a= -a.
Now, for Pach eigenvalue ,\ WC' dPt('rtniue linearly independent eigenvectors
namely fundamental solutions of hotnogeneous equatious (.\I- A)X::: 0. For
.\ 1 = 1, the coefficient matrix is
12 ,) 3]
.\1!- A= [ -12 -6 -2
-12 -5 -3

200

and is row-equivalent to

[ 120 -15 3]
1 .
0 0 0
So we have y = z and x = -~z which yield the fundamental solution

p1 = [-2/3] 1
1

Similarly the reductions

A2I- A= [ -12 -5 -2
13
5 3] [ -12 -50 -21]
-+ '
1
-+

-12 -5 -2 0 0 0

1 1]
[ 0 -5 10
0 -+
[1
0 1
0 -21]
0 0 0 0 0 0

).. 3 !-A= [ -1~ 5 3] [ 8 5 3]

-10 -2 -+ -4 -5 1 -+

[
-4 -.5
0 -.5
0
-12

10 -10
l[ l
-5

1
5
-7

_,.
-4 -5
0
1
1 -1
0
-12 -5 -7

0 0

~l l
yield

P, =[- =d P, = [ -:
 201

rcspcctivdy. lly taking these <'il!;!'llV<'ctors to lw columns we form the diago­

nalizing matrix

The matrix Dis the diagonal matrix whose diagonal entries are the eigenvalues
corresponding to the eigenvectors P1 , P..J and P3 respectively:

D=[~ ~ ~]·

0 0 -3
These matrices satisfy th<' cqualit;.·

/'- 1 :IP =D.

Example 4.1.4. Fort hc m:~trix

3 - 2 0
ft ;::::
n
-
<;
0

we find the characteristic pol.ntomial

t- 5 0 -·1
(i t-1 =(/-l)(t 2 -1)=(/-1) 3 (t+l).

6 0 t+G
Therefore its cltaracterist ic val uPs arc

Aa = -1.

For A = 1 we find

1-A= [ 6
.\ 1
- I

(j
0
0
()
-·1
1
(j
l[ l 1 0 1
0
()
0
()
l
0
 202

which giv<'s us x + z = 0 and z = O,namely characteristic vectors b<'lon?;ing to

this ch<nacteristic value are of the form

Similarly we find that the characteristic vectors beloning to A3 = -1 ar<' of

the form
-2/3 ]
z [ -:/2 .
Therefore we can find only two linearly independent eigenvcctors;however ;,ve
need thr<'<' lin('arly independent eigenvectors to be able to construct an invert­
ible 3 X 3 matrix P diagonalizing A. This shows that A is not diagonnlizable.

APPLICATION: SYSTEMS OF LINEAR DIFFERENTIAL EQUATIONS

Diagonalization of matrices can be effectively used in the solution of sys­

tems of ('qnations involving unknown funcL,ms and their derivatives. Such
equations are called differential equations. The most simple example of differ­
ential equations are those of the form
dx
-=Ax (1)
dt

· which invoh·cs the dependent variable x, the independent varia hle t and the
d.r
derivative - 1-. To obtain its nonzero solutions we write
d

dx 1 ,
-=Act
X

and int<'grat ing we obtain

ln lxl =At+ C
 203

or

where Cis the constant of int<'~?;ration. Note that when C ranges over all real
numbers, =fee will range over all non-zero reals. Including the zero solution
also we see that all solutions of ( I) are functions of the form

(2)

namely they are linear combinations of { e)..t}.

Suppose we are given a syst<'m

dx 1
ell
dx 2
dt

clxn
(3)
dt

where the A;i are real numbers and .r 1{1), .. . ,xn(t) are functions to be deter­
mined. Letting
!k.l.
dt
~
dX dt
..tl= and
dt
4:!:....
dt

we can write the system in th<' matrix form

1,.
Tt = JL\. (.:t)

Now, if A is a diagonalizable matrix with eigenvalues A1 , ... , An we have

(5)

204

From A= PDP- 1 we get

or
p-1 d;~ = np-1 x.
Now the substitution
y = p-l X

reduces the equation to

dY
-=DY.
dt
The explicit form of this matrix equation is the system
dyl
dt
dy2
dt

By (2) the solution of the system is

namely
J(le>.,t

K2e>.2t
p-lx =Y =
--------------------------------------------------------=2Q2

Finally we obtain the general solution of the system {3) as:

A"1e>.,t

](2e>.,t

X=P (o)

where P is the invo·tible matrix diagonalizing the coefficient matrix A. tllf

numbers )q, .A 2 , ••• , An arc cigerwalues of A and I\. h A" 2 , ••• , K,. are arb it mry
constants.

Example 4.1.5. Find the general solution of the system

dx 1
lit
dx 2
dt

\Ve diagonalize the coefficient matrix

A= [-4 6]
-3 5

by finding eigenvalues and eigenvectors. We have

.A+4 -6 ., )( .A+l)=O.
'
det{.AJ- A)= =.A~-.A-2=().-2
3 .A-5 '

Thus the eigenvalues arc

For ). = 2 the system (.XI- A )X = 0 takes the form

206

namely we get

6x- 6y = 0

3x- 3y 0

wl1ich is equivalent to

X- y 0

0 = 0.

Thus the unique fundamental solution is

Similarly, for>. = -1 reducing the coefficient matrix of the system (>.I- A )X =

0 we get

[: =: l~ [~ -~ ]·

Thus x = 2y giving the fundamental solution

Thus the eigenvectors corresponding to the eigenvalues >. 1 = 2 a.nd >. 2 = -1

are

respectively and a. matrix dia.gona.lizing the coefficient matrix is

p = [ : ~ l
 207

Now, from (4) we obtain the getH'ral solution

In terms of components, we ha\'('

x1(t) l\.te 21 + 2K2 e-t

x2(l) /\" 1e 31 + K 2e- 1•

Example 4.1.6. Find the getwral solution of the system

d:r 1
dt
(L!:2
dt
d:r 3
dt

The coefficient matrix is

A= [ -~ ~
-G 0 -5
_: ]·

It is kft to the reader to find tit<' eig<'nvalues

and the diagonalizing matrix P to he

208

From (4) one obtains

[ :: ]· = [P = [
X3
~ ~ -2/~ ;:~,]·
1 0 1
]· [
/~3e

We can use the met hod <'Xpla.ined above in solving higher order linear dif­
ferential equations. To this <'nd, let
dnx dn-lx dx
-d + a,_~-1--1 + ·· ·+ a1-dt + a 0x =0
t" <tn-

be a differential equation ,.,·i1h constant coefficients a.a,a 1 , • .. ,an-t· \Ve can

produce a system of di ff<'r<'nt ial equations out of this equation by letting

Xt X
dx dxt
x2 dt = dt
d2 x dx 2
:r 3 dt 2 =
dt

and using the equation

dxn d"x
. dt = (/tn = -aoXt - atX2 - • • •- an-tXn•

The resulting system is

ll.t~ 1
dt =
d:r'J
=
dt
 20.9

d.rn-1
x,
dt
d.rn
dl
and it has the coc•ffici<'lll ma.trix

0 0 0
0 0 1 0
A=
0 () 0 1
-ao -(/1 -a2 -a,._ I
Th<' characteristic equation of A IS

). -1 0 0
0 ). -1 0
= ao +alA+···+ a 11 _ 1>."- 1 + ;.n =0
0 0 0 -1
flo a! a2 A +an-I
.i\ot<' 1 hat this equation is obtained from the given difTerential equation by
suhstit tiling ).k for~:~. Tht:' si~nifiranre of this characteristic equation is illns­
t ra1<'d in the following example.

Example 4.1.6. Find t.IH' ~eneral solution of the difTNential equation

lP.r
dt~ - :r = 0.

a: 1 .r
d.r d:t 1
:z·~ dt = --;[[

d:r., lP.r
d(- = d!~ =x=:z·l·
210

Thus, we obtain the system

· dx 1
dt = x2
dx2
dt = X!

with coefficient matrix

with characteristic equation

). -1

= ). 2 - 1 = 0.
-1 ).

So the characteristic values are

For >. 1 = 1 and >. 2 = ...:...1, solving the homogeneous systems

[-~ -~ l [:l ~ l = [ and [ =~ =~ l [:l ~ l

= [
we obtain the fundam<'ntal solutions

Hence the dia.gonalizing matrix is

and the solution is obtainNl as

Consid~·ring x = x 1 only we obtain the general solution of the given equation:

This can be rewritten in tlw form

This shows also that the solutions form a vector space which is spanned hy
eA 1 t = e1 and eA 2 t = e-t. Note that here )q = 1 and .-\ 2 = -1 are roots of the
characteristic equation.

CAYLEY-HAMILTON TI!F:OilE~I

One of the basic theorems of linear algebra is the celebrated Cayley­

Hamilton theorem.

Theorem 4.1.7. (Cayley-IIarnilton) Every square matrix Jt satisfies its

chara.ctPI"istic eqtra.tion, nanwly.

Proof. (Optional) Let,\ lw an~· n X 11 matrix. Note that the entries of the
matrix .AI - A can be viewed I o IH' polynomials of degree ::; 1 in .A, so that
this matrix can be considered as a matrix over a field containing polynomials
(such a field do exist!). The ent riPs of its adjoint are cofactors of entries of
.AI- A and as such they are polynomials of degree ::; n- 1 and therefore we
can write

where fl 0 , B 1 , · • ·, fl,_ 1 are n X n matrices. Now letting

212

and using
(AT- A)adj(Al- A)= det(>.J- A)I
we get

Comparing the coefficient matrices of Ak we get

-ABo
Eo- AB1

Bn-1 = I.

Now multiply the first equality by A 0 =I, the second by A,··· the last one
by An and add them all to get 0

as asserted.

Example 4.1.7. Us(' the Cayley-Hamilton theorem to compute the inverse

of

A=[~~ ~]·
1 -1 0

Solution: We have
A -1 -1 -1
b.A(X) = -1 ). -1 = A3 - A2 - A- 1.
-1 1 ),
 213

The f'ayl0y-Hamilton theorPm says that

So we have

nanwly
 215

~II EXERCISES

1. Find the characteristic polynomial of each of the following matrices:

2.( a.) Compute the charact<'fistic poly nom ia.l of

[~ ~ =:
0 1 -c
l

(b) Find a 3 X~! matrix whose characteristic polynomial is -A 3 - 2X! +A+ 5.

3. Find the eigenvalues of each of the following matrices:

(a)
[ l [110]
2 k
k 2 (b) ~ ~ ~ (c) [ (2) 20. 2]
2 2 0
2

4. Prove that a square matrix A is singular iff 0 is an eigenvalue of A .

l
.) . What must be kif the matrix

[
~ ~1 ~;
0 1 -1

has three distict real eigenvalues.

6. Show that eigenvalues of a triangular matrix are its diagonal entries.

216

7. What condition must be placed on a, b, c, d if

[: :l

has real eigenvalues. Prove that this condition is satisfied by 2 X 2 real sym­
metric matrices.

8. Find the characteristic polynomial, the eigenvalues and their associated

eigenvectors for each of the following matrices:

2 -2 -2 1

~2 ~3]
-1

(a) [ ~2 :l (b) [
-2 -2
2 (c)
-1
1
0
2
-3
0
-2
1
0
2
-1
4

9. Given the matrix

(a) Find its characteristic polynomial.

(b) Use the characteristic polynomial to evaluate the determinant of A
(c) Use the CAYLEY-HAMILTON theorem to find the inverse of A.
(d) Compute the eigenvalues and corresponding eigenvectors of A.
(e )If possible find an invertible matrix P such thatP- 1 AP
is diagonal and write this diagonal matrix.

10. Given

(a) If possible find an invertible matrix P such that p- 1 AP is diagonal. (b)

Use this diagonalization to compute A 13 •
 ·_., ...,.

:J/7

ll.(a) Prove that if). is an eigenva.lue of a matrix A then P(,\) is an <'i~en­

value of P(A) for any polynomial P.
(b) Use Part( a) to prove that for any nilpotent matrix A (i.e. AA' = 0 for sonw
positive integer k) the only ('igenva.lue is 0.

12. Prove that if il is diagonalizable, then

(a) :lT is diagonalizable.
(b) for any polynomial P, the matrix P(il) is diagonalizable.

] :3. Prove that if Ph P2 , • • ·, P,. are <'igpnv('ctors of a matrix belon~ing to

distinct eig('nvalues )q, ,\", · · ·, >.,. l'C'SJl<'Ctiv<'ly, any linC'<H rdation

impli<'s a r<'lation of the form

Use this to prove that cigem;C'ctors lwlonging to distinct t>ig<>nvahws are lin­
('arly independent and that an n xu matrix which has n distinct ('ig<'nva]uC's
is diagonalizahk.
218

t lfJ DIAGONAUZATION OF REAL SYMMETRIC MATRICES~

Symmetric matrices are the most common examples of diagonaliza.ble ma­
trices as they are extremely useful in the investigation of quadratic forms.
Even further they can be diagonalized by matrices whose columns form ,not
only a linearly independent set, fortunately, an orthonormal set.This is due to
the following

Theorem 4.2.1. For a real symmetric matrix

(a) al1 eigenvalues are real.

(b) <'igenvcctors corresponding to distinct eigenvalues arc orthogonal.

Proof. (Optional) Let S be a real symmetric matrix and let At and A2

be two (real or complex) eigenvalues with corresponding eigenvectors Pt and
P2 respectively.Then we have

P[ SP2 = P[(SP2) = P[(A2P2) = A2P[ P2

P[ 5 P2 = P[ 5T P2 = (SP!)T P2 = (AtPl)T P2 = AtP[ P2.

It follows that

that is to say
either A2 - At = 0 or P[ P2 = 0.

Now,when At and A2 are distinct real eigenvalues Pt and P 2 become eigenvec­

tors with real entries such that Pf P2 = 0, namely they are orthogonal.This
proves (b ).To prove (a) by using the same conclusion, we take an eigenvalue
A corresponding to the eigenvector P and note that S P = AP implies
S P = SP = SP = AP

 21.9

that is to say 5\ is an eig('nYalu(' corresponding to the eigenvector P.\Vith

A1 = A, A2 = 5\, P 1 = P and P2 = P our conclusion implies that A = 5\ since

, whNe P1. p2 , • • ·, Pn are entri('s of P at least one of which is nonzero .Thus

any eigenvalue A of the real symmetric matrix Sis real.D

Now, we give a name to matrices whose columns (and rows) form an or­
thonorma.l set.

Definition 4.2.2. A square matrix Q is said to be m·tlwgonalif QTQ =I.

Note that a matrix is orthogonal iff Q-t =QT. Considering the partitioned
form

Q?:
we see that the equality QTQ =I is equivalent to the conditions

Thus Q is orthogonal iff its columns form an orthonormal set. Q- 1 QT

shows that the same is true of its rows. Now, we can state.

Theorem 4.2.3. If S is a real symmetric matrix then there exists an

orthogonal matrix P such that pT S P is diagonal.

Proof: (Optional) We prove the th('orem by induction on the order n of

S. If n = 1 the matrix is already diagonal. Assume the assertion is true for
all (n- 1) x (n- 1) symmetric mat rir('s and use this assumption to prove the
assertion for n x n symmetric mat ric('s.The fundamental theorem of algebra
220

guarantees the existence of a root of the characteristic polynomial.Ac:cording

to the theorem we proved a hove such a root is a real eigenvalue of S .Now,
pick up an eigenvalue A1 of S and find a unit eigenvector belonging to this
eigenvalue, complete this to a b;tsis for n X 1 matrices and use the Gram­
Schmidt orthogonalization process to find an orthonormal basis containing
Q 1 • So taking these to be columns we form the orthogonal matrix

for which SQ 1 = .\ 1Q 1 • Now, the matrix QTSQ is symmetric and the parti­
tioned product

shows that

In particular

(QTSQ)il = Q[SQl

= Q[AtQ1

= .\Q[Ql = At6il.
Therefore
At 0 ... 0
0
QTSQ =
s
0

where Sis a real symmetric matrix since QT SQ is symmetric By induction

hypothesis there is an ortl10gonal (n- 1) X (n- 1) matrix Q and a diagonal
 221

matrix iJ such that {JT S'QT = D. Thus we have

[ ~ ~ ] 7 Q'l' sQ [ ~~ l
which is a diagonal matrix D. Il<'nn' W<' have

pT s p = D

where
I 0 ]
P=Q [ 0 Q .
Here P is an orthogonal matrix sine<'

pTp 0] [1 QT0][10]
(J Q
[1 QT0]
Q
0 0 0

Example 4.2.4. Diagonalize t h<' r<'al symmetric matrix

s=
2 -1
[ -1
-1
1
2 -1
-1 2
l
by means of an orthogonal matrix.

We have

.X-2 1 1
b.s= 1 .X-2
1 A-2

222

Therefore the eigenvalues of S are >. 1 = 0, >. 2 = >. 3 = 3. The eigenvector

belonging to >. 1 = 0 is obtained to be

a.nd eigenvectors belonging to >. 2 = 3 ca.n be ta.ken to be

Our a.im is to find an orthogonal ma.trix Q such that QT SQ is diagonal. To

this end we use the Gra.m-Schmidt orthogonalization process to construct an
orthonormal set out of th(' lilwarly independent set {Ph P2 , P 3 }. This process
gives

[~I l= ~ [ -J

 Thus we have an orthonormal set ofeigenvectors:

Hence t h<' orthogonal matrix

[v3
1
.J:l
l
.J:l
1
-vz
0
1
,f'i
1

-~
J6
1
J6
l
cliagonalizPs S :

APPLICATION: QUADRATIC FORMS

A quadratic form is an expression of the form

where ;1 is a real symmetric n X n matrix and

X=

that is, it is an expression which has an explicit form

n
xr AX= L A;jXiXj.
i,j=l
224

Since A is a real symmetric matrix we can find an orthogonal matrix Q such

that D = QT AQ is dia.grioal. 'l'herefore substituting A = Q DQT t 11<' qnadratic
form can be put to the form

and the change of coordinates given by

or X=QY

l
reducl's tlH' form to

yT DY = [Y1 · · ·Yn] [ A1 ?.·] [ y.··1

Q An Yn
yT DY = A1Y; + A2Y~ + ···+AnY~
. 1:,\, '
where A1 , A2 , • • ·,An a.r<' <'lg<'nva.lues o\.:) j\
Example 4.2.3. ld<'ntiry the conic 2xi + 2v'2x 1 x2 + 3x~ = 16 by finding
its standard equation. Writing the equation in the form

we may bring the left hand side to the form XT AX namely we get

Eigenvalues of

2 v'2]
[ v'2 3
are found to be A1 = 1 and A2 = 4 and thus the substitution Y = QT X gives
us
 225

So the standard C'(jll<tt ion of thC' t?;i\'<'ll conic is

y~ + .Jy~ = 16
or
Y¥ + y~ = 1.
1(j ·1
\Ye identify the conic as an ellipsC' with principal axes a = 4 and b = 2.

This exampl<' shows that to find the standard equation of a conic it is

<'nough to find tiH' <'ippnva.luC's oft he matrix describing the quadratic part, no
need to eigenv<'rtors. Eigcnv0c1nrs n'pr<'sents axes of the conic.

Example 4.2.4. find tlw standard cquation of the quadric surface

2 .1,21 + ·l ,2
,;.I~ + ,;J.a-
•l,2 ;) , .,
,;.l1.1~- 2·. .
.l2::Z3- 2 .1.a:r1-
• • - 2~
1.

Noting that the ldt l1and side is

.Y'~' .. tX

with
2
A= [ -I
-1
we can use Exampl<' 1.2.2 to writ<' th<' standard <'quation as

or

which is a circular rylindN. Its ax<'s ran lw tak<'n to he lines paralkl to the

l
unit vectors

V}
I [ :I l U·>
-
= v2
Ir.:, [ -011 and u, = ~ -~ [ ]
G!fJ EXERCISES

1. For each of the following matric~s find, if possible, an invertible matrix

P such that P- 1 AP is diagonal. In the ca.se A is symmetric find an orthogonal
diagonalizing matrix.

(a) [ ~! n :l [ 0 2
(b) 2 0
2 2
(c) [ ~!
-2
2
-4
3
~2]
1 2 0 0

~J
3
2 1 0 0
(d) [ i6 -2
...:.a
(e)
0 0 1 2
0 0 2 1
 227

2. Write the following equations in the matrix form xr AX = b, where A

is a real symmetric matrix:

(a) x2 + 4xy + 2y 2 = 5, (b)3x 2 - xy- y 2 = 1,

3. Find the standard equation of the following conics:

(a) 5x 2 - 6xy + 5y 2 = 32, (b) :r 2 - 2:ry + y 2 = 2, (c) x2 - 2xy + y 2 =1

4. Find the standard equation of each of the following:

(c) - a: 2 + y 2 + 2VJxy + 3y = 0
5. \Vrite the standard equations of the quadric surfaces

(a) 2x 2 + 2y 2 + 4z 2 + 2:ry- 2y::- 2zx = 7, (b) 4x 2 + 4:: 2 - 10yz =9

6. Find the general solutions oft he follwing systems of homogeneous linear
differential equations:

:r't = -x3 x~ = 4x 1 - 12x2 - 3x3

(a) :r~ = x 1 (c) x~ = x1 - 3x 2 - x3
:r~ = -14x 1 - 8x2 + 7a:3 X~= X 1 - 4x 2
228

7. Giwn the diffNential equation y" + 2y'- 3y 0. Convert this to a

system of equations and find its general solution.

8. Given the matrix

r
L.
-2
1
1
1
-2
1
1
1
-2
l
(a.) Find eigenvalues and eigenvectors of A .
(b) Determine whether A is invertible or not.
(c) Find, if possible, an ivertible matrix P for which p-l AP is diagonal.
\Vrite this diagonal matrix.
(d) Find the genNal solution of the system of differential equations:

x; = -2x 1 + X2 + xa
x; = x 1 - 2x2 + xa
x; = x1+ x2 - 2xa

(e) Find, if possible,an orthogonal matrix diagonalizing A.

(f) ·write an orthogonal change of coordinates converting the equation of
the conic

to the standard form.

 21.9

~Chapter I
5

LINEAR TRANSFORMATIONS
In every branch of mat hemal irs 0110 introduces some objects, investi~?;atf's

their properties and tlwn considPrs mappings preserving essential ingredi<'nt s

of the objects under consideratio11. This pattern in linear algebra appears to
be introduction of vector spaces, inn's1iga tion of various properties of el<'nH'Ilt s
of vector spaces and then consid('ra1 ion of "operators" preserving "add it ion ..
and "scalar multiplication". This sort of operators, such as differentiation and
integration operators, Laplace trans forms etc ... , arise naturally in all scienr<'s.

Jejll DEFINITION AND EXAMPLES

Definition 5.1.1. Let V and H. lw two V<'ctor spaces. A function L : l " ­

H1 is called a linmr transfornwlion (or a linear map) from V into H' if

(a) L(n + v) = L(u) + L(v) . for allu and ·v in V.

230

(b) L( cu) = cL( u) for each scalar c and vector u in V.

In particular, a linear transformation L from ll into itself is also called a linrnr

operator on ll.

Example 5.1.2. Consider the map L: JR 3 -+ JR 2 given by

To test whether it is a. Jirwar transformation or not we check the con <lit ions
(a) and (b) of the definition.

(a) Take two vectors u = = (yby 2,y3) and rompntc

(xbx 2,x 3) and v
L( u + v) and L(u) + L('w) and verify that L( u + v) = L( u) + L( v ). In fact we
have

L(u+v) = L(xt+Yt,X2+Y2,xa+Ya)
((.1't + yi) + (x2 + Y2), (x2 + Y2) + (xa + YJ)
(;z·1 + X2 + Y1 + Y2, x2 + X3 + Y2 + Ya)
L(u) + L(v) L(xh x2, X3) + L(yh Y2, Ya)
(.rt + x2, X2 + x3) + (Yt + Y2, Y2 + Y3)
(:z·t + x2 + Yt + Y2, X2 + X3 + Y2 + Y3).

which shows that L( 1t + 1!) = L( u) + L( t• ).

(b) We take any vector u = (x 1,x 2,x3) and any scalar c and verify that
L(cu) = cL( u). Indeed, we have

L(cu) = L( cx 17 cx2, ex a)
L( c(Xt, x2, x3))

(cx1 + cx2, cx2 + cx3)

 231

c(x1 + x2, X2 + xa)

cL( xh x 2 , x 3 ) = cL(v)

as n'qnired. Therefore L satisfies the requirem('nts (a) and (b) and conse­
qn('nt ly L is a linear transformation.

Example 5.1.3. The mapping L: JR 2 -+ IR 2 giv<'n hy

IS nnf a. linear transformation because for u = (.1· 1• :r 2 ) and v = (Yt. Y2) we

have

wll<.'r('as

L(u) + J:(v) + L(y~-yJ = (.r 1 + l,.1· 2 + 1) + (Yt + 1,y2 + 1)

L(x 1 ,x 2 )
(x1 + Y1 + 2, x2 + Y2 + 2).

Thus
L(u +v) i- L(u) + L(r).
Theorem 5.1.4. Let V and H' be vector spaces and let L : V -+ lV be a
mapping. Then Lis a linear transformation iff

L(au + br) = aL(u) + H(u)

for <lll scalars a, band vectors u, v in V. Further, for a linear transformation
L \\'<' have

(i) J.(Oy) =Ow;

(ii) J:(-v) = -L(v);

232

Proof: If L is a lin<'<H transformation then

(a) it preserves addition, namely L(au + bv) = L(au) + L(bv).

(b) it preserves scalar multiplication, namely

L(au)=aL(u) and L(bv)=bL(v)

and consequently

L(au + bv) = aL(u) + bL(v).

Conversely if
L(au + bv) = aL(u) + bL(v)
is satisfied then

(a) taking a= b = 1 we have L(u + v) = L(u) + L(v)

(b) taking a = c, b =0 we have L( cu) = cL(u) and hence L is a linear
transformation.

The rest is very easy:

(i) L(O,) = L(o.O,) = o.L(O,) =Ow.

(ii) L(-v) = L((-l)t') = ((-l)L(v)) = -L(v).

(iii) This follows from a repeated application of L( au+ bv) = aL( u) + bL( v).
 23.9

This theorem enables us to rPach to a quick conclusion in some ex­

amples. For example the mapping of Example .5.1.3 is not linear, since
L(O,O) = (1, 1) ~ (0,0).

Example 5.1.5. The map L: R 3 _,. R 3 given by

is not linear. Dy Theorem 5.1.'1. 1n SP(' this it is enough to find a specific vector
v for which

[,(- r·) ~ -L(v).

For example we have

L(-(1,1,1)) /,( -1, -1, -1) = (1, 1, -3)

-L( 1, 1, 1) -(1,1,3)= (-1,-1,-3)

and they arc not equal. Note th<-lt ])('r<' J,(O,O,O) = (0,0,0) but Lis not linear.
Examples of linear transformations are abundant. Now, we shall exhibit
some of them.

GEOMETRIC EXAMPLES

Examples 5.1.6. (a) Projection: The mapping P: R 3 _.,.. R 3 defined by

234

X!

is called the projection on the x 1 x 2 -plane and it is a linear tra.nsforma1 ion. In

fact, for
v = (xhx 2,x 3 ) and v = (y1,y 2,y3)

on one hand we have P( u) = (x 1 , x 2 , 0) and P( v) == (y 1 , y2 , 0) and on the other

hand since u + v = (a: + yhx 2 + y2,x3 + y3 ) we have
1

Further for each u == (;1· 1 ,x 2 ,x3 ) in R 3 and for each scalar c we have
cu = (cx 1 ,cx 2 ,cx 3 ) and hence

(b) Reflection: The mapping R defined by

is called the reflection with respect to the x 1 x 2 -plane and it is a linear trans­
formation.
 2.15

In fact for vectors u = (x 1..r:J.a· 3), v = (yhy2,y3) and for any scalar we
have:

= R(u) + R(v)

(c) Rotation: The mapping

given by

is obta.ined by rotating each wc1or ·z· through a fixed angle (J because the angle,
between any v =(xi. x 2 ) and RH( r·) = R 8 (x 1 , x 2 ) is computed from

to be fJ.

NATURAL OPERATORS ON f'F:\'CTIONS

Examples 5.1.7. (a) The' mapping D from the vector space of differen­
tiable functions in an interval (a. b) into the set of all functions defined in (a, b)
236

which maps each function f to its derivative f' is a linear transformation since

D(f +g) (! + g)' = !' + g'

D(cf) = (cf)' = cf'.

(b) The mapping

:T: C[a, b]-+ C[a, b]

sending f to the function j given by

](x) = lx f(x)dx

is obviously linear.

CANONICAL TRANSFORMATION

Examples 5.1.8. Given an m x n matrix A we define a canonical mapping

from the space of n X 1 matrices into m X 1 matrices by

L(X) =AX.

It is straightforward to verify that it is a linear transformation:

L(aX +bY)= A( aX+ bY)= aAX +bAY= aL(X) + bL(Y).

Further any linear transformation between vector spaces of column matrices

are in this form.
 2S7

Indeed, if Lis a linear transformation then image of

0
0

0
Ei = - j - th
1
0

l
will be a column matrix say

.r\ IJ
[ : = Ai
rl~,j

and we get

L(X) !,( [
;l~j
.tn
.:
l
)= L('f:, :ri Ei)

[.-\J"'Ai"'An] [
XJ
.:
:In
l
= :IX.

In the next section we shall try to express all linear transformations of finite
dimensional vector spaces in this canonical form.
238

EXISTENCE OF LINEAR TRANSFORMA'l;'IONS

Theorem 5.1.9. L<'t V be a vector space with a basis

and let W be a vector space. Then any linear transformation L : V --+ W is

completely determin<'d by L( tJ 1), ... , L( vn) and further for arbitrary vectors
w 1, .•• , Wn taken from H' we can find a uniquely determined L : V --+ lV
sending each vi to w;.

Proof: If v is a vector in V, it can be written uniquely in the form

and hence

by Theorem 5.1.4. This means that L(v) is completely determined by

L( vi), ... , L( vn)· Now if wh ... , Wn are arbitrary vectors of lV then for
v = C1V1 + ··· + C 71 Vn lPtting

we obtain a map from ll to liV. Note that since c1 , ••• , Cn are uniquely deter­
mined for v there is only one vector c1 w 1 + · · · + Cn Wn to take as L( v ). This
map is a linear map because for

we have

L(u + v)
 2.3.9

L(u) + L(v)

and for any scalar c we obtain

L(cn) L(ca1v1 + · · · + canvn) = ca1w1 + · · · + canwn

= c(a 1 w 1 +···+a,.w,.)=cL(u).

Example 5.1.10. Determine all linear transformations L R3 --... R2

satisfying
L(1,1,1)=(-2,3) and £(1,0,1)=(1,1).

First we extend {( 1, 1, 1), ( 1, 0, 1)} to a basis for R 3, and express ( x 1 , x 2 , x 3 ) in

terms of these basis vectors:

[:
1 1
0 0 a
0 0 1 0 b
1 0 0 1 ('
l [~ -1
1
-1
1 0 0
1 0
0 -1 0 1 c- a
b~ a]
B {(1,1,1),(L0.1),(1,0,0)}
/-.,
rx2p, 1,1)+ (,7:3- x2)(1,0, 1)+ (xl- x3)(1,0,0)

Now, applying the abo~d theorem to

v1 = (1, 1, 1) , v2 = ( 1, 0, 1) , v 3 = ( 1 , 0, 0)

and
w 1 =(-2,3), w2 =(1,1) and w 3 =(a,b)

we obtain a. linear transformation L sending v; 's to a.rbitra.yr w; 's. Thus

L(.rt,x 2 ,x3) = x2(-2,:3)+(.r3-x2)(1,1)+(xt-x3)(a,b)

= (ax 1 - 3x 2 + (1- a)x 3 , bx 1 + 2x 2 + (1- b)x 3 )
t-111 EXERCISES

1. Which of the following a.r(:' linear transformations?

(b) L: R 2 x 2 --+ R 2 giwn by L([ Xt x2 ]) = (x 1 + x 4, x2 + xa)

xa x4
(c) L: R 2 x 2 --+ R 1 giv(:'n by L(A) = det(A)
(d) L: R 3 --+ R 2 giv(:'n by L(xt,x2,xa) = (lxt + x2l, lx1- x2l)
(e) L: R 4 --+ R 3 given by L(xbx2,xa,x4) = (x1 + x2,x3 + x4, 1).
2. Show that the mapping givf'n by

is a linear transforma1 ion from the space of polynomials of degree S 2

into R 3 .
 241

3. Show that the mapping

L( J>(l)) = tP'(t) + P(O),

where P'(t) stands for the <!Privative of P(t), is a linear operator on the
space of polynomial functions.

4. L<'t A be a fixed n X n matrix over a field F and let T: pnxn _. pnxn

be the mapping given by T( /J) = AB + BA 2 • Prove that Tis a linear
transformation.

5. Prove tltat there is a lin<'ar transformation T: R2 --+ R4 such that

1'(1,1)= (1,-1,1,2),7'(-1,1) = (2,3,-2,-1). Compute T(x,y).

6. Is there a linear transform at ion T : R 3 _. R2 satisfying

(a) (1, -1, 1) = (2, -1), T( I. I ,0) = ( 1, -1), 1'(3, -1, 2) = (5, -3)

(b) 1'(1,-1,1) = (2,-1), 7(1.1,0) = (1,-1), 1'(3,-1,2) = (-5,3)

(c) 1'(1,0,0) = (a,b),T(O, 1.0) = (b,r},T(O,O, 1) = (c,a),

1'(1,1,1)= (a+b+c)(l.l)'?

7. LetT: R2 x 2 _. R 2 x 2 be t.IH• lirwar operator given by

T( [ 1 0
0 0
l[ l [ l [ l[ l
)= 2 -1
1 1
, T( 1 1
0 0
) = T( 1 1
1 0
)= -1
-1
1
-1

and

T( [ : : l) ~ ~ l·
= [

Compute

T( [
(I

r·
b
d
l )
2.f2

8. Determine the real numbers a, b, c and d if there is a linear transformation

from the vector· space of real polynomials of degree :S 2 to tlw vector
space of 2 X 2 matrices such that

T( 1 - x + x2) = [ a a
-b 2
l
T( -1 + x + 3x 2 ) = [ be -22]

T(x 2 ) = 0 1
[ 1 d ]

operator T on R 3 such that

 ,.-J:J MATRIX REPRESENTATIONS OF LINEAR TRANSFORMATIONS a
H1•call I hat ~ivm a fin it<• dinwnsional VPctor space V with a presnihed ba­
sis 8 <'<tch vector 'I' of V can he repn•sented by a column matrix which is tlre
coordinate matrix of u relatiV(' to B. The same is true for liuear transforma­
tions. llnwevi.•r, a linear transformation f.; V--+ H' is ~iven by means of two
\'Pet or span•s V and 11·, thus this tillH' we fix a basis for Pach wctor space and
<tssi~n a matrix to this pair of has<'s. This is <•stablish1•d in

ThE>orem and Definition 5.2.1. L('t \' and 11' IH· fiuit<• dinH•usioual
V<'ct or spaces with bases B = { 1' 1 , •••• 1' 11 } and C = { 11• 1 , ••• , w,,} r<•:,qwct in•ly.
For !'acl, litH'ar trausforlllatiou {,: \ ' - 11.· thPn' <•xists a uuiqu<' 111atrix .-\
such t !tat

fin· all r 111 \ •• This nr;tl rix :I is ral!Pd /}!( nwlriJ· of/, n la/i('( to /ftc pair B.C
of IHisrs. and is d!'Jt011•d h.v [rJt'· It is tl1!' 111 X 11 rnatrix ~i\'Pil by

/_( I'J) .·ltrll't + .-1'21~~''2+ ···+ .-1,,,11' 771

/_( ,.'2)

/_( 1',) .-1,,11'1 + :1'2,11''2 + · · ·+ :1 711 ,11' 771 •

\\'hl'll \ · = 11· and 8 =C W(' sp1•ak oft h(' 1//(/ff'i.r of t/1( /inmr OJ}( mlnr /,
N lulit·r loB.

Proof: f'niqll(l/lss: 1.!'1 .-II><• a 111atrix satisf_ving

(I)

for allt·tn 1·. Sinn• [t·]L, is art 11 X l tllatrix a11d [!_(t·)],· is an 111 x l rnatrix \\'('
first rtoff' that .-ltnnst IH• all 111 X. II IJI<tlrix. l.1•t .-llt<t\'(' r()llltlllls .-1 1.. 1'2 ..... .-1,.
In (1) taking v = v; we get

[L(v;)]c = A[v;]s = [A1 ···An] 1 = A;.

Thus A must be the matrix whose ith column A, is the coordinate matrix of
L( v;) relative to C namely the entries of the ith column are coefficients of WJ.
's when L( v;) is expressed in terms of WJ. 's.

Existence: Now take A to be the matrix whose columns are the coordinate
matrices [L( v1))c. Then for each v in V we have a unique expression

giving

L(v) = c1L(v1)+···+cnL(vn)
[L( v )]c cl[L( vl)]c + · · ·+ Cn[L('Vn)]c

[(L(v.)]c · · ·[L(v.)]c] [ :~ ]

= A[v]s

that is to say (1) is satisfied.

0

Example 5.2.2. Let L: R 4 --+ R 2 be the linear transformation given by

\Ye want to find th(' matrix of/, relativ<' to the pair of bases

[ = {(1,0,0.0).(0.1.0.0),(0,0,1,0),(0,0,0, 1)}
c = {(I. 1). ( 1' -1 )}.
To this <'nd we exprPss T(t'd = (1,0), T(11~) = (0,-1), T(v 3 )
(1,0), T(v.d = (0,-1) in t('rms of u· 1 = (1,1) and w~ = (1,-1):

[
1 1 I
L -1 0
0
-1

l
0 I /'2

1/2

Thus we obtain
I 1
T(t't) (1.0)= -(1,1)+ -(1.-1)
2 2
I 1
T( I'~) (0.-1)= --(1.1)+-(1,-1)
:2 2
I 1
T(z·:d (1.0)= -(1.1)+-(l.-1)
:2 2
I I
T(1•.1 ) = ((l. - I ) = - :{1. 1 ) + 2( I. - 1).
Taking th<' nwfrici<•nts to IH'

;1
<'111

= [
riPs of columns we gd tlH' matrix of Las

:!l
&
- :!l
& &
2l - 2I
~
l.

.!:]

l
1. _1. 1. _1.
[/,(I' )]c = .I[ 1']t = [ :!
1.
".!
:!
1.
2
:!
l
2
:!
1.
2
:r2
;1:3

;r4
 -246

in other words we have

L(xt. x2, xa, x4) = (x1 + xa, -x2- x4) = ~(x1 - x2 + xa- x4)(1, 1)
1
+2(x1 + X2 + Xa + x4)(1, -1)

Example 5.2.3. Let A be a fixed m x n matrix and L : R.nxl ...... R.mxl be

the mapping given by
L(X)= AX.
Then with respect to the usual bases for vector spaces of column matrices the
matrix of Lis the one whose columns are
0 0 0

0 0 0
L( 1 ) = .t1 1 = [A1 ···A;·· ·An] 1 =A;.
0 0 0

0 0 0
Thus this matrix is precisely equal to A. Actually the aim of representing
linear transformations by matrices is to describe the linear transformation in
the form
L(X) =AX
. by means of coordinate matrices of vectors. If L is the transformation from
R.n to R.m given by

L(xb. ·., Xn) = (Auxl + A12X2 + · · · + AlnXn,

A21xl + A22X2 + ... + A2nXn,
tlwn identifying

W(' can write it in the form

namely in the form L(X) = AX and its matrix relative to the standard ha.ses
is A.

For ('Xample the matrix of L : R 4 __.... R 3 defined by

J"('lative to the standard bases is

248

le-J:.J EXERCISES
1. Let L : JR3 -+lR 2 be the linear transformation for which
L(1, 0, 0) = (-2, -3), L(O, 1, 0) = (3, 2), L(O, 0, 1) = (1, -1).
a) Find L(l, -1, 2).

c) Determine the matrix of L relative to the standard bases for JR 3 and

JR2.

d) Determine the matrix of L relative to the standard basis for JR 3 and

the basis {(2,3),(3,2)} oflR2 •
 249

2. Let L be the linear operator on the space of polynomials of degree~ 3

so that

L (1 ) = 1 + t, I(t) = t + t2, L ( t 2) t 2 + t,3 L ( t 3) = 1

a) Find L(2- t + t 2 - n.
b) Compute the matrix of/, r<'lative to the basis B = { 1, t, t 2 , t 3 } (namely

to the pair B, B).

c) Find the matrix of L r<'la1iv<' to the basis C = {1 + t, t + t2, t 2 + t 3 , 1}

d) Find the matrix of L rP]a1ive to the pair B,C.

3. Let

Prove that the mapping ~iv<'n by

L(X) =AX- XA

is a linear operator on R 2 x 2 • Find its matrix relative to the standard

ordered basis for R 2 x 2

4. LC't V be a finite dinwnsional vector space with bases B and B' and let
P, be the B to B' chan~" of basis matrix. The operator I: V __,. V givC'n
by J( v) = v for all v in l" is ra ll<'d the identity operator. Find the matrix
of I

a) relative toC

b) relative to B'

c) relative to B and B'

d) relative to B' and B

250

5. Find the matrix of the projection (of R. 3 ) on the x 1 x 2 -plane relative to

the standard basis

6. Find the matrix of the reflection (ofiR 3 ) with respect to the x 1 xTplane
relative to the standard basis.

7. Find the matrix of the reflection (of R 2) with respect to the x 2-axis
relative to the standard basis.

8. Find the matrix of the rotation (of 1R 2 ) through the angle e relative to
the standard basis.

9. Let V be the vcctor space spanned by the UC-functions B

{1, x, x 2 , cos x, sin x}

a) Is D = dd a. linear operator on V? If so find its matrix relative to B.

b) Is :J = J; a linear operator on V? If so find its matrix relative to B.

10. Let L : JR 3 -+ R. 3 be the linear operator given by

Find scalars .\ and vectors v so that

L(v) = .\v.

Find a basis for R 3 consiting of v's satisfying this condition and deter­
mine the matrix of L relative to this basis.
 2/'il

A BASIS FOR THE SPACE OF UC-FUNCTIONS

The aim of this appendix is to gin' a rigorous proof of the fact that funclions
of the form xneax cos bx and xmeu sin d.1: constitute a basis for UC-functions.
For this purpose we make use of rompl<'x valued fnnrtions. All functions that.
we are going to consider will be dPfilli'd in the intNval (-oo,oo) and as su('h
this domain of definition will not IH' specified.

If u and v are differentiable real valued functions then the derivative oft h<'
complex-valued function <p given hy

<p(:r) = u(:r) + i·v(:r)

is defined by
i.p 1 (X ) = II 1 ( .1') + i 1' 1( .1· ).

It is a straight forward matter to wrify the propNti<'s

(<p+?J')' 'P' + 'lj/

( O'!.p )' n<.p1 for n E C
('P . lj;)' 'P' . 1/' + 'P . t'·'.
Recalling the definition

eu+it• = r" ( ros v + ·i sin I')

they yied at once that
 252

(ii) for cp( x) =e 0 "' we have cP'( a:) = ae 0 "'

(iii) for cp(x) = p(x)e 0 "' we have ~.p'(;l~) = p(x)e 0 "'

where a 0,ab ···,an and a are complex numbers and p(x) = p(x) + ap(x) is
a polynomial which has the same dPgree as p( x) when a -:/= 0.

Now, we are in a position to est<thlish our primary result given in

Theorem: The set consisting of functions of the form

xn ea"' cos bx, n E N, a E R, bER and b 2:: 0

and

xmec"' sin dx, mEN, c E R, dE R d>0

is linearly independent.

Proof: Suppose on the contrary t.l1at the set under consideration is linearly
dependent. Then there exists real numbers Aii and Bkl such that

2:::: AijXn;ea;x cos bj:t' + 2:::: Bk1Xmk ec'"' sin b,x = 0

i,j k,l

where some coefficient is non-zero. Tl1a.t is, a sum of non-zero terms of the
form Axnea"' cos bx and Bxmeex sitl da: will be equal to zero. Collecting terms
involving the same ea"' cos bx or the same ec"' sin dx such an equality can be
brought to the form

2::: Pk(x )eakx cos bkx + 2::: Q,(x )ec,x sin b,x = 0
k I
 25.'3

where Pd .r) and Q1(x) arc non-z0ro polynomials. By making the substitutions

ei8 + e-i&
cosO=---­
'2
deriwd from r.i 8 = cos(} + -i sin B and £ -iB = cos() - i sin(} we obtain a rcla tion
of the form
11

Lfl(.t)("kX =0
A'=l

where the pk( x) are non-zero polynomials with complex coeffici<'nts and
o 1 ,o 2 ,···,on are distinct compi<'X numbers.

Thus, if the set under consid('!'il1 iou is linearly dependent there exists non­
zero polynomials p 1 ( x ), · · ·, p,( :r) with complex coefficients and distinct com­
plex numhei·s oh · · ·, o,. so that

L" (lA-( .r )( "•(x) = 0.

k=l

Among all such relations chaos<.' the one for which n is minimal.

Multiplying this equality through E-o.x and letting f3k = ok - o,. the
relation becomes
n.-1
L pk(.r)rl'k·r + Pn(x) = 0.
k=l

Differentiating l+deg (p,.(.r)) tinH'S and using (iii) above the last relation
yields

n-1
LPk(x)e.6•x = 0, with deg(f3k(x)) = deg(pk(x)).
'k=l

So there is a relation with n-1 non-zeo tNms which contradicts the minimality
of n. This contradiction comes from 1he assumption that the set given in the
theorem is linearly dependent. So 1his set must be linearly independent. 0
 254

LAGRANGE'S INTERPOLATION

A standard basis for the space of polynomials of degree :S n is

Now, we exhibit another basis for this space associated to any n + 1 distinct
sclars a 0 , a 1 , ···,an. Consider the polynomials
(x - ao) · · · ( x - ak_l)( x - ak+r) · · · ( x - an)
Pk(x) = (ak- ) .
ao · · ·(ak- ak_t)(ak- ak+I) · · ·(ak- an)
;k = o, .. ·,n

Note that Pk(ak) = 1 and Pk(aj) = 0 for j :jo k, that is to say

Hen\e, therelation
n
l::.:CkPk(x) =0
k=O
implies that for each j = 0, · · ·, n we have
n n
0 = L ckPk(aj) = L ck8kj = Cj.
k=O k=O
Thus the polynomials P0 (x ), · · ·, Pn(x) are linearly independent polynomials
of degree n. Since the dimension of the space of polynomials of degree :S n
is 11 + 1 these n + 1 linearly independent polynomials form a basis. Namely,
every polynomial P(x) is a linear combination of P0 (x),P1 (x),·· ·,Pn(x):
n

P(x) = I:ckPk(x).
k=O
 255

\Vc compute the scalars CJ.: at one<' as

n n

r(aj) = L,c.. r..(cj) = L:>kOkj = Cj 1

k:O k=O

and thus \VC obtain

P(x) = r(ao)r0 (:r) + r(a 1 )Pdx) + · · · + P(a,)Pn(a:).

This formula is knm,·n as Lagranw 's interpolation formula.

For <'Xample 1 h<' polynomial /'( .r) of d<'grc<' :S 2 so that

P( -1) = 2. r((l) = 2, P( 1) = -1
IS

where

(a: - 0 )( .r ­ 1) :r( x - :1.:)

P0 (.r) = ---:------:-=---=-­
-1-0)(-1-1) 2
(:z:-(-l))(.r-1)
Pt(.r) = (0- 1-1))(0-1) = -(x+ 1 )(:r~ 1 )
(:t:{ -1 ))(.r- 0) (x + l):z:
P._A:r) = ------=--­
(1-(-1))(1-0) 2

So

P( X) = .2 . :r( .1· 2- I) ,
- 2 . ( .!' + I )( ./' - 1) ­
(a: + 1 ).r = - -X
3 2
-
3
-X + 2.
2 2 2
 256

additive inverse 2
Cramer's rule 90
adjoint 88

alternating 79
d0gree 105

augmented matrix 53
df'terminant 76

diagonal 2.5

diagona.liza ble 197

basic variable 55
differential equations 202

. basis 121
dimension 126

Bessel's inequality 185

dot prod11ct 161

best approximation 182

echelon matrix 28

Cauchy-Schwarz inequality 169

eigenvaule 19?~

Cayley-Hamilton Theorem 211

eigenvector 198

change of coordinate matrix 135

elementary matrix 39

characteristic equation 199

equivalent .51

characteristic polynomial 199

Euclidean spare 159

characteristic value 198

characteristic vectors 198

field 2

cof'ffident matrix 53
free variahlf' 1R5

cofactor 88
Fourier roefficif'nts 191

column space 142

fundamental solutions 60

consistent 53

coordinate matrix 134

GAUSSIAN Plimination 55

GAUSS-JORDAN reduction .5.5

linear span 109

general solution 197

linear transform::\~ ion 229

generator 109
linearly depend<'n t 119

Gram-Scdmidt 181
linearly indPrwnt 11 D

lower triangular 1.7

LU-decomposition ()()

homog0neous system 60

identity elenwnt 2

magic sq u arc !)

identity matrix 7

main diagonal (j

identity op<'rator 249

matrix 6

inconsist<'nt 5:l

matrix of a lin<'ar trans-formation 2·1:l

inner product 159

matrix of t!H' liiH'<H operator 234

inner product space 1!)9

inwrtihlc ,12
minor /.1

multiplicativ<' invcrs<' :l

Kron<'ckN <klt.a 7

n-lin<>ar function 7S

nodal incidenr<' matrix 10

quadratic form 1.2:~

norm Hi7

normalized verlor Hi7

Lagran~c's intNpolation for111ula 2!)!)

leading <'ntry 28

orthogonal 1GI

l<'ast-sqnare solution 187

ort ho~onal basis 177

Jpft inwrs<' :12

orthogonal com plf'Tll<'Jit 167

kngt h 1(i7

orthogonal matrix 219

linear combination 10~)

orthogonal projf'rt ion 182

linear map 229

ort ho~onal set 177

linear opNator :no

 258

orthogonal vector 167

spanning set 109

orthonormal set 177

standard basis 124

standard inner product 161

Parallelogram law 175

submatrix 17

partitioned form 18
subspace 108

'
polynomial .function 99
symmetric matrix 28

projection 233
system of linear equations 53

pythagoras theorem 168

transition matrix 135

reflection 234
transpose 8

right inverse 42
triangle inequality 169

rotation 235
trivial subspace 108

row equivalent 3.5

U C-function 125

row operation 33

row rank 135

unit vector 167

row space 135

unity 2

row-reduced echelon matrix 29

upper triangular 26

Sarrus' rule 75
Vandermoncle determinant 98

scalar 3
vector space 100

scalar matrix 26
vector 100

scalar product 161

skew-symmetric 28

zero element 2

solution .53

zero vector 100