Matrices

1 Matrix Algebra
Definition 1.1. Matrix
An m × n matrix is a rectangular array of Complex numbers, with m rows and
n columns. The number in row i and column j of matrix A is denoted [A]ij . If
A is an m × n matrix, we often write it as
h i
A = a1 | a2 | · · · | an

where aj is column j of A, for j = 1, 2, . . . , n.

Example 1.1.
 
1 3 8 2 h i
A = 0 1 7 0 = a1 | a2 | a3 | a4
 

1 0 2 3

where        
1 3 8 2
a1 = 0 , a2 = 1 , a3 = 7 , a4 = 0
       

1 0 2 3

Definition 1.2. Mmn

The set of all m × n matrices with complex entries is called the vector space
Mmn .

1
Definition 1.3. Matrix Equality
If A and B are m × n matrices, A = B means

[A]ij = [B]ij

for all i = 1, 2, . . . m, and j = 1, 2, . . . n.

Example 1.2.    
1 3 8 2 1 3 8 2
0 1 7 0 = 0 1 7 0
   

1 0 2 3 1 0 2 3

   
1 3 8 2 1 3 5 2
6 0 1 7 0
0 1 7 0 =
   

1 0 2 3 1 0 2 3

Definition 1.4. Matrix Addition

If A and B are m × n matrices, the sum of A and B is the m × n matrix A + B,
defined by
[A + B]ij = [A]ij + [B]ij

for all i = 1, 2, . . . m, and j = 1, 2, . . . n.

Example 1.3.
     
−1 3 5 4 0 −2 3 3 3
 0 −2 6 +  2 1 1  =  2 −1 7
     

2 5 3 −3 −2 0 −1 3 3

Definition 1.5. Matrix Scalar Multiplication

If A is an m × n matrix and α ∈ C, the product of α and A is the m × n
matrix αA, defined by
[αA]ij = α[A]ij

for all i = 1, 2, . . . m, and j = 1, 2, . . . n.

2
Example 1.4.    
−1 3 5 −2 6 10
2  0 −2 6 =  0 −4 12
   

2 5 3 4 10 6

Definition 1.6. Zero Matrix

The zero matrix in Mmn is the m × n matrix 0mn defined by

[0]ij = 0

for all i = 1, 2, . . . m, and j = 1, 2, . . . n.

Example 1.5. " #

0 0 0
023 =
0 0 0

Theorem 1.1. Vector Space Properties

If A, B, and C are m × n matrices, and α and β are complex numbers, then

1. (A + B) ∈ Mmn

2. αA ∈ Mmn

3. A + B = B + A

4. (A + B) + C = A + (B + C)

5. A + 0 = A for all A ∈ Mmn

6. For every A ∈ Mmn there is a matrix −A such that A + (−A) = 0

7. α(βA) = (αβ)A

8. α(A + B) = αA + αB

9. (α + β)A = αA + βA

10. 1A = A

3
Definition 1.7. Transpose
The transpose of a matrix A is AT , defined by

[AT ]ij = [A]ji

for all i = 1, 2, . . . m, and j = 1, 2, . . . n.

One way to think of AT is that it is A with its rows and columns swapped.

Example 1.6.  
" # 1 4
1 2 3
A= AT = 2 5
 
4 5 6
3 6

Definition 1.8. Symmetric Matrices

The matrix A is symmetric if A = AT .

Note: only square matrices can be symmetric. You should try to prove this.
Specifically, ”If A is symmetric, then A is square.”

Example 1.7. Here are a few 3 × 3 symmetric matrices. Symmetric matrices

can be any size, as long as they are square, i.e. n × n. Notice the pattern.
       
−1 0 2 π −1 −3 0 0 0 1 0 0
 0 −2 5 −1 15 5 0 0 0 0 1 0
       

2 5 3 −3 5 0 0 0 0 0 0 1

Theorem 1.2. Transpose Properties

If A and B are in Mmn and α ∈ C, then

1. (A + B)T = AT + B T

2. (αA)T = αAT

3. (AT )T = A

4
Definition 1.9. Conjugate
If A ∈ Mmn , the conjugate of A is A, defined by

[A]ij = [A]ij

for all i = 1, 2, . . . m, and j = 1, 2, . . . n.

Example 1.8. Here are some matrices and their conjugates.

   
1 + 2i 5 + 5i 2−i 1 − 2i 5 − 5i 2+i
A= 0 −2 + 7i 5+i A= 0 −2 − 7i 5−i
   

2 − 3i 5−i 3 − 7i 2 + 3i 5+i 3 + 7i
   
0 −2 1 0 −2 1
B = 1 −5 3 B = 1 −5 3
   

1 7 −2 1 7 −2

Note that if the entries of a matrix B are all Real numbers, then B = B.

Theorem 1.3. Conjugate Properties

If A and B are in Mmn and α ∈ C, then

1. A + B = A + B

2. αA = αA

3. (A) = A

4. (AT ) = (A)T

Note: since it doesn’t matter whether we take the conjugate or the transpose of
a matrix first, we will often simplify the notation for the ”conjugate transpose”
as follows

T
(AT ) = (A)T = A

Definition 1.10. Adjoint

If A ∈ Mmn , the adjoint of A is A∗ , defined by

T
A∗ = A

5
Theorem 1.4. Adjoint Properties
If A and B are in Mmn and α ∈ C, then

1. (A + B)∗ = A∗ + B ∗

2. (αA)∗ = αA∗

3. (A∗ )∗ = A

Definition 1.11. Diagonal Matrices

A diagonal matrix is an n × n matrix with 0’s everywhere except possibly on
the main diagonal. We can write this as
(
0 i 6= j
[A]ij =
aii i = j, aii ∈ C

Example 1.9. The matrix A below is the general form of a 3 × 3 diagonal

matrix. B is a specific 3 × 3 diagonal matrix
   
a11 0 0 −5 0 0
A =  0 a22 0  B = 0 2+i 0 
   

0 0 a33 0 0 −π

6
2 Matrix Multiplication
Definition 2.1. Matrix-Vector Product
If A ∈ Mmn and x ∈ Cn , the matrix-vector product Ax is the linear combi-
nation
n
X
Ax = [x]i ai
i=1

where ai is column i of A.

Example 2.1.
        
1 0 2 2 1 0 2
0 3 5 3 = 2 0 + 3 3 1 5
        

3 4 1 1 3 4 1
     
2 0 2
= 0 +  9  + 5
     

6 12 1

4
= 14
 

19

Definition 2.2. Standard Basis Vectors

The set of standard basis vectors in Cn is {e1 , e2 , . . . , en } where
(
0 i 6= j
[ei ]j =
1 i=j

for all i = 1, 2, . . . n, and j = 1, 2, . . . n.

Example 2.2. The standard basis vectors in C3 are:

     
1 0 0
e1 = 0 , e2 = 1 , e3 = 0
     

0 0 1

7
Definition 2.3. Identity Matrix
The n × n identity matrix is In , whose columns are the standard basis vectors
in Cn : h i
In = e1 e2 · · · en

Example 2.3.  
1 0 0
I3 = 0 1 0
 

0 0 1

Theorem 2.1. Three Views of a System

If A is an m × n matrix with columns a1 , a2 , . . . , an , and if b is a vector in Cm ,
then the matrix equation
Ax = b

has the same solution set as the vector equation

n
X
xi ai = b
i=1

which has the same solution set as the system of linear equations LS(A, b).

8
Example 2.4. Let
   
1 0 −3 8
A = 2 2 9  , and b =  7 
   

0 1 5 −2

Then we can write the linear system LS(A, b) in three ways:

System of Equations:

x1 − 3x3 = 8
2x1 + 2x2 + 9x3 = 7
x2 + 5x3 = −2
Matrix Equation:
    
1 0 −3 x1 8
=
2 2 9  x2   7 
    

0 1 5 x3 −2
Vector Equation:
       
1 0 −3 8
x1 2 + x2 2 + x3  9  =  7 
       

0 1 5 −2

Definition 2.4. Matrix Multiplication

If h i
A = a1 a2 · · · an

is an m × n matrix, and
h i
B = b1 b2 · · · bp

is an n × p matrix, then the product of A and B is the m × p matrix

h i
AB = Ab1 | Ab2 | · · · | Abp

9
Example 2.5.  
" # 1 0 −2 −1
3 −2 1 
1 3 −1 0 

0 1 −5
0 1 −1 2

 
       
" # 1 " # 0 " # −2 " # −1
 3 −2 1   3 −2 1   3 −2 1   3 −2 1   
= 1 3 −1  0 
 
 0 1 −5 0 1 −5 0 1 −5 0 1 −5 
0 1 −1 2


" # " # " # " # " # " # " # " # " #
 3 −2 1 3 −2 1 3 −2 1
=
 1 0 + 1 1 + 0 −5 0 +3 +1 −2 −1 −1
0 1 −5 0 1 −5


" # " # " #
3 −2 1
−1 +0 +2 
0 1 −5 

" #
1 −5 −5 −1
=
1 −2 4 −10

Theorem 2.2. Multiplicative Identity

For all A ∈ Mmn
AIn = A, and Im A = A

Proof.
h i
AIn = Ae1 Ae2 · · · Aen

n n n
" #
X X X
= [e1 ]i ai [e2 ]i ai · · · [en ]i ai
i=1 i=1 i=1

h i
= a1 a2 · · · an

=A

10
Theorem 2.3. Matrix Equality
If A, B ∈ Mmn , and Ax = Bx for all x ∈ Cn , then A = B.

Proof. If Ax = Bx for all x ∈ Cn , then in particular, Aei = Bei so we can

write

A = AIn
h i
= Ae1 Ae2 · · · Aen
h i
= Be1 Be2 · · · Ben

= BIn
=B

Theorem 2.4. Row-Column Algorithm

If A is an m × n matrix and B is an n × p matrix, then AB can be computed
one component at a time as
n
X
[AB]ij = [A]ik [B]kj = [A]i1 [B]1j + [A]i2 [B]2j + · · · + [A]in [B]nj
k=1

Example 2.6.  
" # 1 0 −2 −1
3 −2 1 
1 3 −1 0 

0 1 −5
0 1 −1 2
 
3(1) − 2(1) + 1(0) 3(0) − 2(3) + 1(1) 3(−2) − 2(−1) + 1(−1) 3(−1) − 2(0) + 1(2)
=



0(1) + 1(1) − 5(0) 0(0) + 1(3) − 5(1) 0(−2) + 1(−1) − 5(−1) 0(−1) + 1(0) − 5(2)
" #
1 −5 −5 −1
=
1 −2 4 −10

11
Definition 2.5. Dot Product
If u and v are vectors in Rn , the dot product of u and v is

n
X
u•v= [u]i [v]i
i=1

Example 2.7.    
3 1
  1 = 3(1) − 2(1) + 1(0) = 1
−2
 • 

1 0

Note, if u and v are vectors in Rn , then

n
X n
X
hu, vi = [u]i [v]i = [u]i [v]i = u • v
i=1 i=1

Theorem 2.5. Dot Product Form of Row-Column Algorithm

If A and B are the m × n, and n × p matrices
 
a1
 a2 
  h i
A= .  , B = b1 b2 · · · bp
 . 
 . 
am

with Real number entries, then the row-column algorithm for multiplying matri-
ces can be written as:
 
a1 • b1 a1 • b2 ... a1 • bp
 a2 b1 a2 • b2 ... a2 • bp 
 • 
AB = 
 .. .. ..  
 . . . 
am • b1 am • b2 . . . am • bp

or alternately
[AB]ij = ai • bj

where a1 , a2 , . . . , am are the rows of A and b1 , b2 , . . . , bp are the columns

of B.

12
Example 2.8.  
" # 1 0 −2 −1
3 −2 1 
1 3 −1 0 

0 1 −5
0 1 −1 2
               
3 1 3 0 3 −2 3 −1
−2 1 −2 3 −2 −2 −2  0 
  •    •   •    •  
 
 1 0 1 1 1 1 1 2 
 
=
 

               
 0
 1 0 0 0 −2 0 −1 
 1  1  1  3  1  −2  1   0 
  •    •   •    •  

−5 0 −5 1 −5 1 −5 2

 
3(1) − 2(1) + 1(0) 3(0) − 2(3) + 1(1) 3(−2) − 2(−1) + 1(−1) 3(−1) − 2(0) + 1(2)
=



0(1) + 1(1) − 5(0) 0(0) + 1(3) − 5(1) 0(−2) + 1(−1) − 5(−1) 0(−1) + 1(0) − 5(2)

" #
1 −5 −5 −1
=
1 −2 4 −10

Theorem 2.6. Properties of Matrix Multiplication

Let A be an an m × n matrix and B and C have sizes for which the indicated
products are defined, and let α be in C.

1. A(BC) = (AB)C

2. A(B + C) = AB + AC

3. (B + C)A = BA + CA

4. α(AB) = (αA)B = A(αB)

5. 0p×m A = 0p×n

6. A0n×p = 0m×p

7. Im A = A = AIn

8. (AB)T = B T AT

9. (AB) = A B

10. (AB)∗ = B ∗ A∗

13
Many of the properties of addition and multiplication of matrices are the same
as the analogous properties for addition and multiplication of real numbers, but
there are some important exceptions.

There is no Commutative Property of Multiplication for matrices:

If A is m × n and B is n × p, then

AB 6= BA

In fact, if A is m × n and B is n × p, then BA isn’t even defined!

A handy theorem about multiplication of real numbers is not true for multipli-
cation of matrices. Recall that if a and b are real numbers:

If ab = 0, then a = 0 or b = 0.

However if A and B are matrices, AB = 0 does not imply that A = 0 or B = 0.

Example 2.9. " #" # " #

1 0 0 0 0 0
=
0 0 1 0 0 0

Note that neither matrix in the product is the zero matrix.

To see why this theorem does not apply to matrix multiplication, we need to
examine why it does apply to real number multiplication. Here is a little proof:

Theorem 2.7. If a and b are real numbers, and ab = 0, then a = 0 or b = 0.

Proof. Let ab = 0. If a = 0 then the conclusion is satisfied (i.e. it’s true that
0
a = 0 or b = 0). If a 6= 0, then b = = 0 and the conclusion is satisfied. 
a

Why couldn’t we use the same argument if a and b are matrices instead of real
numbers? The proof relies on the operation of division, and division of matrices
is not defined.

14
3 Matrices and Inner Products
Theorem 3.1. Inner Product as Matrix Multiplication
If u and v are in Cm , then we can write the inner product of u and v as a
matrix product:
hu, vi = u∗ v

Theorem 3.2. Adjoint and Inner Product

If If A ∈ Mmn , x ∈ Cn , and y ∈ Cm , then

hAx, yi = hx, A∗ yi

Proof.

hAx, yi = (Ax)∗ y
= x ∗ A∗ y
= x∗ (A∗ y)
= hx, A∗ yi

Definition 3.1. Hermitian Matrices

A square matrix A is Hermitian if A = A∗ .

Hermitian matrices are also sometimes called self-adjoint.

15
Theorem 3.3. Hermitian Matrices and Inner Product
If A is n × n, and x, y ∈ Cn , then A is Hermitian iff

hAx, yi = hx, Ayi

for all x, y ∈ Cn .

Proof. If A is Hermitian, then

hAx, yi = hx, A∗ yi = hx, Ayi

Conversely, if hAx, yi = hx, Ayi, then

hAx − A∗ x, Ax − A∗ xi = hAx − A∗ x, Axi + hAx − A∗ x, −A∗ xi

= hAx − A∗ x, Axi − hAx − A∗ x, A∗ xi

= hAx − A∗ x, Axi − hA(Ax − A∗ x), xi

= hAx − A∗ x, Axi − hAx − A∗ x, Axi

=0

Since the inner product of Ax − A∗ x with itself equals zero, Ax − A∗ x must be

zero. So we have

Ax − A∗ x = 0
Ax = A∗ x

Therefore A = A∗ and A is Hermitian. 

16