MAT188: Linear Algebra

Arnav Patil

University of Toronto

Contents
1 Vectors, Limits, Sets, and Planes 2

2 Systems of Linear Equations 3

3 Linear Transformations 6

4 Composition and Inverse of Linear Transformations 7

5 Determinant 9

6 Subspaces, Span, Linear Independence, and Basis 10

7 Basis and Coordinates 12

8 Orthogonal Projection 13

9 Orthogonal Transformations and Least Square 14

10 Eigenvectors and Eigenvalues 15

11 Diagonalization 17

12 Singular Value Decomposition 17

1
1 Vectors, Limits, Sets, and Planes
Introduction
A vector is a representation of displacement – it gives a set of instructions on how to move from one point
to the next. This is known as a coordinate vector. When the coordinate vector is taken from the origin,
it becomes known as the standard representation of the vector. Each vector has n components, where n
is a positive integer.

The collection of all vectors with n components forms Rn , also known as the n-dimensional Euclidean
vector space.

We use the head to tail rule for vector addition; that is, add up the individual components of each vec-
tor being summed to get the resulting vector.

Definition: Parallel Vectors

⃗ for some k ∈ Rn , or is one is a scalar multiple of the other.
Two vectors are parallel if ⃗v = k w

We can define a matrix as an array of numbers. For example, we say that

 
1 2 3
M=
4 5 6
is a 2 × 3 matrix. A column vector is just a matrix with one column and multiple rows, and a row vector is
just a matrix with multiple columns and one row.

There are eight “axioms” of linear algebra that will guide us through this course, and allows us to ensure
vector arithmetic works the way we want it to:
1. Vector addition is associative, or (⃗v + w)
⃗ + ⃗u = ⃗v + (w
⃗ + ⃗u),
2. Vector addition is commutative, or ⃗v + w
⃗ =w
⃗ + ⃗v ,
3. ⃗v + ⃗0 = ⃗v ,
4. For each ⃗v ∈ Rn , there exists a vector ⃗x such that ⃗v + ⃗x = ⃗0,
5. k(⃗v + w)
⃗ = k⃗v + k w,
⃗
6. (c + k)⃗v = c⃗v + k⃗v ,
7. c(k⃗v ) = (ck)⃗v , and
8. 1⃗v = ⃗v .

Solidify
Definition: Dot Product
Let ⃗v and w⃗ be two vectors with components v1 , v2 , ..., vn and w1 , w2 , ..., wn respectively. The dot
⃗ denoted by ⃗v · w,
product of ⃗v and w, ⃗ is given by: v1 w1 , v2 w2 , ..., vn wn .

Definition: Norm
Let ⃗v be a vector in Rn . The norm of ⃗v is given by:
q
v12 + v22 + ... + vn2

2
For vectors in R2 and R3 , the angle between ⃗v and w
⃗ can be given by:
 
⃗v · w⃗
arccos
||⃗v ||||w||
⃗
Definition: Perpendicular

⃗ are perpendicular or orthogonal if their dot product is 0, or if ⃗v · w

Two vectors ⃗v and w ⃗ = 0.

Now we’ll discuss a little bit about sets.

Definition: Subset
We say a set A is a subset of a set B, and write A ⊆ B, if all the elements of A are also in B. In
other words, A ⊆ B, if for every a ∈ A, a ∈ B.

Definition: Equality of Sets

We say sets A and B are equal if A is a subset of B and B is a subset of A. That is A = B if A ⊆ B

and B ⊆ A.

Expand
We can define lines in set notation. Let’s take an example of y = −2x. As a set, we can describe it as:
l = {(x, y)|y = −2x} = {(t, −2t)|t ∈ R}
We can generalize this to vectors:
l = {⃗ ⃗ ∈ R}
p + td|t
We refer to ⃗v = p⃗ + td⃗ as the vector parametric form of the line l.

Similarly, we can define a plane in Rn as well.

Definition: Plane
A plane in Rn has the form:
⃗ + s⃗n + ⃗b|s, t ∈ R}
P = {tm
where m
⃗ and ⃗n may NOT be parallel.

2 Systems of Linear Equations

Introduce
Definition: Linear Equations

An equation of the form a1 x1 + a2 x2 + ... + an xn = b is known as a linear equation.

A collection of linear equations with the same variables is known as a system of linear equations. Values
for each xn that make all the equations true simultaneously are known as solutions to the system.

There are two matrices associated with a system of linear equations.


2x1 + 4x2 = 8

4x1 + 3x2 = 10

x1 − x2 =2


3
The coefficient matrix:  
2 4
4 3
1 −1
and the augmented matrix:  
2 4 8
4 3 10
1 −1 2
We use the process of Gauss-Jordan elimination to systematically solve systems of linear equations.
There are three fundamental operations that do not change the general solution to the system:
1. Multiplying an equation by a scalar,
2. Adding a multiple of an equation to another, and
3. Switching the order of equations.

Definition: Row Echelon Form and Reduced Row Echelon Form

A matrix is in row echelon form if it satisfies the following properties:
1. All zero rows are at the bottom,
2. The leading entry in each row is to the right of the leading entry of the row above, and

3. All entries below a leading entry are zero.

A matrix is in reduced row echelon form if it is in REF and additionally:
1. All the leading entries are 1 (we call them “leading ones”), and

2. Each leading 1 is the only nonzero entry in its column.

Definition: System of Linear Equations

A particular solution is one single tuple (c1 , c2 , ..., cn ) that makes all the equations true simulta-
neously. The set of all possible solutions to a system of linear equations is known as the general
solution.

Solidify
For each matrix M , there are many REF forms of it, however, there is only one RREF form.

The first nonzero entry is called that row’s leading entry, or leading 1 if the entry is a 1. A pivot
position is a location in a matrix that corresponds to a leading 1 in the RREF in that matrix.

If a variable’s column is a pivot column, it is known as a leading/basic/dependent variable; if not,

it’s known as a free variable.

Here is a step-by-step process to solve systems of linear equations:

1. Write the system in standard form by ordering the variables into an augmented matrix.
2. Reduce the matrix into its RREF.
3. Identify free and basic variables.
4. Transform the RREF matrix into equations. The result is a much simpler system of linear equations.

4
 5. Solve for the basic variables in term of the free variables.
6. Let the free variables run over all scalars.
7. We typically write the general solution in set builder notation.
A system of equations is consistent if there’s at least one solution; otherwise, it is called inconsistent.

Theorem: Solution Type

A linear system is inconsistent only if the RREF matrix has the equation 0 = 1, in other words, if
the augmented column is a pivot column. Otherwise, if a linear system is consistent, then it has:
• Infinitely many solutions (there’s at least one free variable), or

• Exactly one solution (all the variables are leading).

Definition: Rank

The rank of a matrix M is the number of leading 1s in RREF(M).

Expand
Some matrix terminology:

• If A is n × n, then the matrix is a square matrix,

• A square matrix is diagonal if all entries not in the diagonal line are 0, that is, aij = 0 when i ̸= j,
• A square matrix is upper triangular if all its entries below the main diagonal are zero, and lower
triangular inversely,

• A zero matrix has all entries set to 0, and

• A square diagonal matrix is called the identity matrix if all its nonzero entries are set to 1.

Definition: Matrix-Vector Product

Let A be an n × m matrix with row vectors w ⃗ 1, w ⃗ n and ⃗x = x1 , x2 , ..., xn be a vector in Rn .
⃗ 2 , ..., w
We define:    
−− w ⃗ 1 −− x1 w⃗ 1 · ⃗x
−− w ⃗ 2 −−    ⃗ 2 · ⃗x 
  x2   w
A⃗x = 
 
..   ..   .. 
−− . −−  .   . 
−− w
⃗n −− xn ⃗ n · ⃗x
w

Theorem: Matrix Form of a Linear System

We can write the linear system with augmented matrix [A|⃗b] in matrix form as A⃗x = ⃗b.

Definition: Linear Combination

A linear combination of finite vectors ⃗v1 , ⃗v2 , ..., ⃗vn in Rn is an expression of the form:

c1⃗v1 + c2⃗v2 + ... + cn⃗vn

5
Expand
Theorem: Algebraic Rules for A⃗x

If A is an n × m matrix, ⃗x and ⃗y are vectors in Rn , and k is a scalar, then:

• A(⃗x + ⃗y ) = A⃗x + A⃗y , and

• A(k⃗x) = k(A⃗x).

3 Linear Transformations
Introduce
We sometimes use more archaic words to describe linear algebra than calculus, this is due to linear algebra’s
deep connections to geometry. For example, what we call a ‘function’ in calculus, we may call a ‘transfor-
mation’ or ‘mapping’ in linear algebra.

The space from which the input to a transformation comes is called the domain. The codomain is the
space in which the output vectors lie, while the set of vectors that make up the output is called the range.

Definition: Linear Transformation

A linear transformation from Rm to Rn is a mapping T: Rm → Rn that satisfies:

1. T (⃗x + ⃗y ) = T (⃗x) + T (⃗y ) for all vectors ⃗x, ⃗y ∈ Rm , and

2. T (k⃗x) = kT (⃗x) for all vectors ⃗x ∈ Rm and all scalars k ∈ R.

Solidify
Definition: Standard Vectors

The standard vector ⃗ei ∈ Rm is defined as a unit vector with all zeros except for the 1 in the ith
entry.

Theorem: Linear Transformations and Matrices

If T : Rm → Rn is a linear transformation, there exists an n × m matrix A such that T (⃗x) = A⃗x.

The standard matrix representation of the linear transformation T is:

 
| | |
T⃗ (⃗e1 ) T⃗ (⃗e2 ) · · · T⃗ (⃗em )
| | |

Expand
The linear transformation that rotates vector ⃗u by θ radians is:
 
cos θ − sin θ
T (⃗u) =
sin θ cos θ

6
  
u·⃗
We call ⃗x|| = ⃗ x
||u||2 the projection of ⃗x onto ⃗u.

 
⃗u · ⃗x
proj⃗u (⃗x) =
||u||2

Similarly, the reflection is given by:

refL (⃗x) = ⃗x|| − ⃗x⊥
or
refL (⃗x) = 2proj⃗u (⃗x) − ⃗x

Figure 1: Projection and Reflection of a Vector

4 Composition and Inverse of Linear Transformations

Introduce
Not any two linear transformations can be composed. For S ◦ T to be a valid composition, the domain of
S must be the codomain of T . The inverse, T ◦ S, may not necessarily be defined. The composition of two
linear transformations will also be a linear transformation as it is closed under vector addition and scalar
multiplication.

Definition: Product of Matrices

Let T : Rm → Rn , T (⃗x) = A⃗x and S : Rn → Rp , S(⃗y ) = B⃗y be linear transformations with associated
standard matrices A and B respectively. Then BA is defined to be the unique matrix associated to
the composition S ◦ T : Rm → Rp .

7
Solidify
Theorem: The Columns of the Matrix Product
 
| | |
Let A = ⃗a1 ⃗a2 · · · ⃗am  be an n × m matrix and B is a p × n matrix. The BA is a p × m matrix
| | |
given by:  
| | |
BA = B⃗a1 B⃗a2 · · · B⃗am 
| | |

Expand
Injective (or one-to-one) transformation – A transformation T is called injective if no two vectors in
the domain get sent to the same vector in the same codomain.

Surjective (or onto) transformation – A transformation T is called surjective if every vector in the
codomain has a vector that gets mapped to it, by T .

Bijectivity – A transformation is called bijective if it is both injective and surjective.

Theorem: Bijective Maps are Invertible

The mapping T : X → Y is invertible iff it is bijective.

Definition: Invertibility of Matrices

A square matrix A is said to be invertible if the linear transformation T (⃗x) = A⃗x is invertible. In
this case, the matrix of T −1 is denoted by A−1 . If the linear transformation T (⃗x) = A⃗x is invertible,
then its inverse is T −1 (⃗y ) = A−1 ⃗y .

The product of two invertible matrices is also invertible. Furthermore, (AB)−1 = B −1 A−1 , meaning that
the order of the matrix multiplication reverses when we take an inverse.

Theorem: Criterion for Invertibility

If A is an n × n matrix, and the RREF of A = In , then A is invertible.

Theorem: Inverting a Matrix

If A is an invertible n × n matrix, then A−1 can be computed by creating an augmented matrix

[A | In ] (notice that we are augmenting by an n × n matrix, not just a single column), and performing
elementary row operations on A until it reaches its RREF (which is In ). The resulting augmented
matrix will then be of the form [In | A−1 ].

8
5 Determinant
Introduce
Definition: 2 × 2 Determinant
 
a b
Let A = be a 2 × 2 matrix. The determinant of A is the scalar ad − bc.
c d

Theorem: Determinant and Invertibility of a 2 × 2 Matrix

 
a b
1. A 2 × 2 matrix is invertible iff det A = ad − bc ̸= 0
cd
 
d −b
2. If A is invertible then A−1 = ad−bc1
.
−c a

The absolute value of the determinant of A is the expansion factor, or the ratio by which T changes the area
of any subset Ω in R2 .
area of T (Ω)
= | det A|
area of Ω

Solidify
Definition: Cross Product in R3

The cross product ⃗v × w ⃗ in R3 is the vector in R3 with the following three

⃗ of two vectors ⃗v and w
properties:

• ⃗v × w
⃗ is orthogonal to both ⃗v and w.
⃗
• ∥⃗v × w∥
⃗ = ∥⃗v ∥∥w∥ ⃗ with 0 ≤ θ ≤ π.
⃗ sin θ, where θ is the angle between ⃗v and w,
• The direction of ⃗v × w
⃗ follows the right-hand rule.

Definition: 3 × 3 Determinant

If A = [⃗v1 ⃗v2 ⃗v3 ] is a 3 × 3 matrix, then:

det A = ⃗v3 · (⃗v1 × ⃗v2 )

Theorem: Invertibility and Determinant

A square matrix A is invertible if and only if det A ̸= 0.

Theorem: Determinant of the Transpose

If A is a square matrix then det A = det AT .

9
 Theorem: Determinants of Products and Powers
Let A, B be n × n matrices:
1. det AB = det A det B
2. det(Am ) = (det A)m

Theorem: Determinant of an Inverse

If A is an invertible matrix:
1
det(A−1 ) =
det A

Expand
Theorem: Elementary Row Operations and Determinants

Suppose A is an n × n matrix:
1
1. If B is obtained from A by dividing a row of A by a scalar k, then det B = k det A.

2. If B is obtained from A by a row swap, then det B = − det A.

3. If B is obtained from A by adding a multiple of a row of A to another row, then det B = det A.

Definition: An Elementary Matrix

An elementary matrix is a matrix you get by applying an elementary row reduction step to the
identity matrix.

Theorem: Cramer’s Rule

Consider the linear system A⃗x = ⃗b, where A is an invertible n × n matrix. The components xi of the
solution vector ⃗x are
det Ab,i
xi =
det A
where Ab,i is the matrix obtained by replacing the i-th column of A by ⃗b.

6 Subspaces, Span, Linear Independence, and Basis

Introduce
Definition: Subspace

A subspace W of Rn which contains ⃗0 and is closed under vector addition and scalar multiplication.

10
 Definition: Span

Let ⃗v1 , ⃗v2 , . . . , ⃗vm be vectors in Rn . The set of all linear combinations of ⃗v1 , ⃗v2 , . . . , ⃗vm is called their
span. That is,

span{⃗v1 , ⃗v2 , . . . , ⃗vm } = {c1⃗v1 + c2⃗v2 + · · · + cm⃗vm | c1 , c2 , . . . , cm ∈ R}

If span(⃗v1 , ⃗v2 , . . . , ⃗vm ) = V , then {⃗v1 , ⃗v2 , . . . , ⃗vm } is called a spanning set for V , or is said to span V .

Theorem: Span is a Subspace

Given vectors ⃗v1 , ⃗v2 , ..., ⃗vm in Rn , span(⃗v1 , ⃗v2 , ..., ⃗vm ) is a subspace.

Definition: Image

The image of f : X → Y is defined to be the set im(f ) = {f (x) : x ∈ X} = {y ∈ Y | f (x) =

y for some x ∈ X}.

Theorem: Image is a Subspace

Let T : Rm → Rn be a linear transformation given by T (⃗x) = A⃗x.

Then im(T ) = Col(A) is a subspace of Rn .

Solidify
Definition: Kernel
The kernel of linear transformation T : Rm → Rn is the set of all vectors in the domain such that
T (⃗v ) = ⃗0.
ker(T ) = {⃗v ∈ Rm |T (⃗v ) = ⃗0}

Theorem: Kernel is a Subspace

Let T : Rm → Rn be a linear transformation given by T (⃗x) = A⃗x.

Then Im(T ) = Col(A) is a subspace of Rn .

Theorem:

For a linear transformation T : Rm → Rn , T is injective if and only if ker(T ) = {⃗0}, and T is surjective
if and only if im(T ) = Rn .

Definition: Linear Relation

Let ⃗v1 , ⃗v2 , . . . , ⃗vn be vectors in a subspace V of Rm . A linear relation among ⃗v1 , ⃗v2 , . . . , ⃗vn is any
equation of the form
c1⃗v1 + · · · + cn⃗vn = ⃗0
where ci are scalars.

11
 Definition: Linearly Dependent

Let ⃗v1 , ⃗v2 , . . . , ⃗vn be vectors in a subspace V of Rm . We say ⃗v1 , ⃗v2 , . . . , ⃗vn are linearly dependent if
there are scalars c1 , . . . , cn that are not all zero such that c1⃗v1 + · · · + cn⃗vn = ⃗0.

Definition: Linearly Independent

Let ⃗v1 , ⃗v2 , ..., ⃗vn are linearly independent if c1⃗v1 + · · · + cn⃗vn = ⃗0 has only one solution, where all the
scalars are 0.

Expand
Let’s summarize what we know about a set of vectors that’s linearly independent:
1. The set {⃗v1 , ⃗v2 , ..., ⃗vn } contains no redundant vector,

2. None of the ⃗vi s can be written as a linear combination of the others,

3. None of the ⃗vi s is in the span of the rest,
4. The only relation between ⃗v1 , ⃗v2 , ..., ⃗vn is the trivial relation,
5. Null(A) = ⃗0,

6. Rank of A with m column vectors is m,

7. Linear map T (⃗x) = A⃗x is injective, and
8. The kernel of the map T (⃗x) = A⃗x is {⃗0}.

Definition: Basis
A basis of a subspace V of Rn is a linearly independent set of vectors in V that spans V .

Theorem:
Vectors ⃗v1 , ⃗v2 , ..., ⃗vn in V form a basis iff every vector ⃗v in V can be expressed as a linear combination
⃗v = c1⃗v1 + c2⃗v2 + ... + cn⃗vn . The coefficients c are called the coordinates of ⃗v with respect to basis
⃗v1 , ⃗v2 , ..., ⃗vn .

7 Basis and Coordinates

Introduce
Theorem: Every Spanning Set is Larger or Equals Every Linearly Independent Set in V

Given a nonzero subject of Rn , there are infinitely many bases of V . The theorem above guarantees
that ALL bases of a subspace V have the same number of vectors. This is called the dimension of V .

The rank of matrix A is the dimension of Im(A), and the nullity is the dimension of Ker(A).
Let TA : Rn → Rm . The rank-nullity theorem states that:

Rank(A) + Nullity(A) = n

12
Solidify
Theorem: Bases and Unique Representation

Vectors ⃗v1 , ⃗v2 , ..., ⃗vm in V only make up a basis for V if every vector ⃗v can be expressed as a linear
combination of ⃗v1 , ⃗v2 , ..., ⃗vm .

Definition: Coordinates

Suppose B = (⃗v1 , . . . , ⃗vn ) is an ordered basis of a subspace V . The B-coordinates of ⃗v ∈ V are the
unique scalars ai such that

⃗v = a1⃗v1 + · · · + an⃗vn .
The B-coordinates are arranged into a column vector, denoted [⃗v ]B . That is,
 
a1
 .. 
[⃗v ]B =  .  .
an

Definition: Change-of-Coordinates Matrix

If B = (⃗b1 , . . . , ⃗bn ) and C = (⃗c1 , . . . , ⃗cn ) are two ordered bases of a subspace V , the change-of-
coordinates matrix from B to C is the unique matrix S such that S[⃗v ]B = [⃗v ]C for all ⃗v ∈ V .

We usually use a subscript B → C (SB→C ) to denote that S changes B-coordinates into

C-coordinates.

Expand
Definition: Similar Matrices

Two n × n matrices are called similar if there exists an invertible matrix S such that B = S −1 AS.

8 Orthogonal Projection
Introduce
Definition: Orthogonal Set

A set of vectors is orthogonal if each vector’s dot product with every other vector is 0.

Definition: Orthonormal Set

A set of vectors is orthonormal if it is orthogonal and the norm of each vector is 1.

Theorem: Orthonormal Sets are Linearly Independent

An orthonormal set is always linearly independent.

13
Solidify
Definition: Orthogonal Complement

If W is a subspace of Rn , the orthogonal complement of W in Rn is the set

W ⊥ = {⃗v ∈ Rn : ⃗v · w ⃗ ∈ W}
⃗ = 0 for all w

Theorem:

If W is a subspace for Rn , then W ⊥ must also be a subspace of Rn .

dim W + dim W ⊥ = n

Definition: Orthogonal Projection

If W is a subspace of Rn and if ⃗v ∈ Rn , the orthogonal projection of ⃗v onto W is the unique vector

⃗ ∈ W such that ⃗v − w
w ⃗ ∈ W ⊥ . The orthogonal projection of ⃗v onto W is denoted projW (⃗v ).

Expand
Theorem: Orthogonal Decomposition

For a vector ⃗x in subspace V , we can say ⃗x = ⃗x|| +⃗x⊥ where ⃗x|| is in V and ⃗x⊥ is in V ⊥ . Furthermore,
this representation is unique for every ⃗x in V .

Theorem: Gram-Schmidt Method

Let V be a subspace of Rn and B = {⃗b1 , . . . , ⃗bk } be a basis for V . Then we can construct an
orthonormal basis U = {⃗u1 , . . . , ⃗uk } for V where

p⃗i
⃗ui =
∥⃗
pi ∥
and

p⃗1 = ⃗b1 , p⃗i = ⃗bi − ProjVi−1 ⃗bi ,

p1 , p⃗2 , . . . , p⃗i−1 } = Span{⃗b1 , . . . , ⃗bi−1 }.

Vi−1 = Span{⃗u1 , . . . , ⃗ui−1 } = Span{⃗

9 Orthogonal Transformations and Least Square

Introduce
A linear transformation that maps a vector from Rn to Rn is called orthogonal if the magnitude of ⃗x and
T (⃗x) is the same, for all ⃗x. Furthermore, the mapping T (⃗x) = A⃗x is orthogonal when the columns of A are
an orthonormal basis.

Theorem: Orthogonal Linear Transformations and Dot Product

A linear transformation is orthogonal iff T (⃗v ) · T (w)

⃗ = ⃗v · w ⃗ ∈ Rn .
⃗ for all ⃗v , w

14
 Theorem: Orthogonal Matrices

Consider an n × n matrix A. A is orthogonal iff AT A = In or A−1 = AT .

Solidify
Theorem:

If A is an n × m matrix whose columns are linearly independent, then AT A is an invertible m × m

matrix.

Theorem: Standard Matrix of Orthogonal Projection

Consider a subspace V ⊆ Rn with basis B = {⃗v1 , . . . , ⃗vm }. Then if we take

 
| |
A = ⃗v1 · · · ⃗vm  ,
| |

the projection of ⃗b onto V is given by the following formula:

ProjV (⃗b) = A(AT A)−1 AT ⃗b.

10 Eigenvectors and Eigenvalues

Introduce
Working with diagonal matrices is almost always easier because it allows us to work with much more com-
plicated operations.

Definition: Diagonalizable Matrices

A linear transformation T : Rn → Rn is diagonalizable if there is a basis B of Rn such that the

B-matrix [T ]B of T is diagonal. A matrix A is diagonalizable if the linear transformation defined by
left multiplication by A is diagonalizable.

Definition: Eigenvectors & Eigenvalues

An eigenvector of a linear transformation T : Rn → Rn is any non-zero vector ⃗v ∈ Rn such that

T (⃗v ) = λ⃗v for some scalar λ. The scalar λ is called the eigenvalue of T corresponding to the
eigenvector ⃗v .

Definition: Eigenbasis

Let T : Rn → Rn be a linear transformation. An eigenbasis of Rn for T is a basis of Rn consisting of

eigenvectors of T .

Theorem:
A linear transformation T : Rn → Rn is diagonalizable iff it has an eigenbasis.

15
Solidify
Theorem: Eigenvalues of a Matrix

A scalar λ is an eigenvalue of the n × n matrix A iff det(A − λIn ) = 0.

Definition: Characteristic Polynomial of a Matrix

The characteristic polynomial of an n × n matrix A is the degree n polynomial

fA (x) = det(A − xIn ).

Definition: Trace of a Matrix

The sum of the diagonal entries of a square matrix A is called the trace of A, denoted by tr(A).

Theorem: Characteristic Polynomial of a 2 × 2 Matrix

Let A be a 2 × 2 matrix. Then

det(A − λI2 ) = λ2 − (tr(A))λ + det A.

Definition: Algebraic Multiplicity

Let T : Rn → Rn be a linear transformation of a finite-dimensional vector space Rn . The algebraic

multiplicity of an eigenvalue λ is the largest power r such that (x − λ)r divides the characteristic
polynomial of T .

Expand
Definition: Eigenspace

If T : Rn → Rn is a linear transformation of the vector space Rn and if λ is an eigenvalue of T , then

the subset

Vλ = {⃗v ∈ Rn | T (⃗v ) = λ⃗v }

consisting of all λ-eigenvectors (together with ⃗0) is a subspace of V , called the λ-eigenspace, or the
eigenspace corresponding to λ.

Definition: Geometric Multiplicity

The geometric multiplicity of λ is the dimension of the λ-eigenspace or, equivalently, the maximal
size of a linearly independent set of eigenvectors with eigenvalue λ.

16
11 Diagonalization
Introduce
Theorem:
Let T : Rn → Rn be a linear transformation. Eigenvectors of T that correspond to distinct eigenvalues
are linearly independent.

Theorem:
If T : Rn → Rn has n eigenvectors then T is diagonalizable.

Let’s summarize what we know so far. Let T (⃗x) = A⃗x:

1. The linear transformation T is diagonalizable,
2. There exists an eigenbasis for T ,

3. There exists a diagonal matrix D and an invertible matrix S such that A = SDS −1 ,
4. The dimensions of the eigenspaces of T add up to n,
5. Geometric multiplicities of T adds up to n, and

6. Algebraic multiplicities of T add up to n and for every eigenvalue λ, gemu(λ) = almu(λ)

Solidify
Definition: Orthogonally Diagonalizable Map

We say a mapping T is orthogonally diagonalizable if T has an orthonormal eigenbasis.

Definition: Orthogonally Diagonalizable Matrix

A n × n matrix is orthogonally diagonalizable if there is an n × n orthogonal matrix Q such that

QT AQ is diagonal.

Theorem: The Spectral Theorem

The matrix A of a mapping T is orthogonally diagonalizable iff A is symmetric.

Expand
A distribution vector is a vector ⃗x in Rn where all components add up to 1, and all components are
either positive or zero. A square matrix A is a transition or stochastic matrix if all of its columns are
distribution vectors. Furthermore, if all entries are positive, or nonzero, then we call it positive.

12 Singular Value Decomposition

Singular Value Decomposition, or SVD, is a method used to decompose an n × m matrix into three matrices,
A = U ΣV T . U is an n × n is an orthogonal matrix, V is an m × m orthogonal matrix, and Σ is an n × m

17
matrix r = Rank(A) nonzero entries listed in decreasing order.
 
  σ1  T
| | |  ..  | | |
A = ⃗u1 ⃗u2 · · · ⃗un  
 . 0 ⃗v1

⃗v2 ··· ⃗vm 
| | |  σ r
 | | |
0 0

We may also write this as:

A = σ1 ⃗u1⃗v1T + ... + σr ⃗ur ⃗vrT
The values σi are called the singular values of A and they tell us a lot about the matrix.

We can use singular value decomposition to compress data; for example, we construct a single value de-
composition for an RGB image, then set the smaller σi ’s, which are substantially smaller than σ1 , to 0. That
way, we can make the vectors (and necessary operations) significantly less intense, while minimizing loss of
quality.

18