Scribd
Upload
234 views

Singular Value Decomposition Example

The document discusses singular value decomposition (SVD), which expresses any matrix A as the product of three matrices: A = UΣV T , where U and V are orthogonal matrices and Σ is a diagonal matrix containing the singular values of A. It provides an example of computing the SVD of the matrix A and using it to describe A's action on the unit circle. Specifically, it finds the singular values and vectors of A, constructs the matrices U, Σ, and V, and shows that applying V T , then Σ, then U maps the unit circle to itself, representing the action of A.

Uploaded by

dev1712
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
234 views

Singular Value Decomposition Example

The document discusses singular value decomposition (SVD), which expresses any matrix A as the product of three matrices: A = UΣV T , where U and V are orthogonal matrices and Σ is a diagonal matrix containing the singular values of A. It provides an example of computing the SVD of the matrix A and using it to describe A's action on the unit circle. Specifically, it finds the singular values and vectors of A, constructs the matrices U, Σ, and V, and shows that applying V T , then Σ, then U maps the unit circle to itself, representing the action of A.

Uploaded by

dev1712
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 5

Singular Value Decomposition Example

Paul Skoufranis
March 11, 2010
We shall explore a concept known as the singular value decomposition of a matrix.
Denition Let A be an n m matrix. The singular values of A are the square roots of the eigenvalues of
m m matrix A
T
A listed according to their algebraic multiplicity. We should note that the eigenvalues of
A
T
A will always be positive numbers so it makes sense to take the square root. It is customary to denote
the singular values by
1
,
2
, . . .,
m
with
1

2

m
.
Theorem Any n m matrix A can be written in the form A = UV
T
where U is an orthogonal n n
matrix, V is an orthogonal mm matrix, and is an n m matrix whose rst r diagonal entries are the
non-zero singular values
1
,
2
, . . .,
r
of A and whose other entries are all zeros. The expression UV
T
is
known as the Singular Value Decomposition of A.
Now we shall go through an example of computing the Singular Value Decomposition (SVD) of a ma-
trix A, why it works, and how we can use the SVD to determine the action of A on the unit circle x
2
+y
2
= 1.
Example Compute the SVD of the matrix
A =
_
_
1 1
1 1

3 0
_
_
and use this decomposition to describe the action of A on the unit circle.
Solution: The rst step in nding the SVD of A is to nd the singular values of A. To do this, we
simply need to compute the eigenvalues of A
T
A. Thus let us rst compute A
T
A;
A
T
A =
_
1 1

3
1 1 0
_
_
_
1 1
1 1

3 0
_
_
=
_
5 2
2 2
_
Therefore the characteristic polynomial of A
T
A is
f
A
T
A
() = det(A
T
AI) = (5 )(2 ) 4 =
2
7 + 6 = ( 1)( 6)
Therefore the eigenvalues of A
T
A are 1 and 6. Thus the singular values of A are
1
= 1 and
2
=

6 (in
general, at this step you would take the square root of each eigenvalue to get a singular value and include
it am() times). This tells us that the matrix will be the 3 2 matrix
=
_
_

6 0
0 1
0 0
_
_
where we place
1
=

6 in the rst column and


2
= 1 in the second column since

6 > 1 (normally, if

1

2

r
> 0 are your non-zero singular values (including repetitions), we would place these
entries along the diagonal of and place zeros elsewhere).
Our next goal is to determine the matrix V . We know that since A
T
A is a symmetric matrix, A
T
A will
have an orthonormal basis of eigenvectors. It turns out that the matrix V will be 22 matrix whose columns
are an orthonormal eigenbasis for A
T
A. Therefore we compute the eigenspaces of A
T
A. We notice that
E
6
= ker(A
T
A6I) = ker
__
1 2
2 4
__
= ker
__
1 2
0 0
__
= span
_
1

5
_
2
1
__
and
E
1
= ker(A
T
AI) = ker
__
4 2
2 1
__
= ker
__
1
1
2
0 0
__
= span
_
1

5
_
1
2
__
Let
v
1
=
1

5
_
2
1
_
and v
2
=
1

5
_
1
2
_
Then we let V be the 2 2 matrix
V =
_
v
1
v
2

=
_
2

5
1

5
2

5
_
where we place the eigenvector for the eigenvalue 6 in the rst column and the eigenvector for the eigenvalue
1 in the second column since we placed

6 in rst column of and

1 in the second column of .


Now our question is how do we nd the orthogonal 3 3 matrix U? Well, if we could nd an orthogonal
3 3 matrix U such that AV = U, we would be done since V
1
= V
T
as V is an orthogonal matrix. Let
us consider the actions of AV and on the standard basis e
1
and e
2
of R
2
. Well

_
1
0
_
=
1
_
_
1
0
0
_
_

_
0
1
_
=
2
_
_
0
1
0
_
_
and
AV
_
1
0
_
= Av
1
AV
_
0
1
_
= Av
2
Therefore, if U were an orthogonal 3 3 matrix such that AV = U and u
1
, u
2
, and u
3
were the columns
of U, then {u
1
, u
2
, u
3
} would be an orthonormal basis of R
3
and
Av
1
= AV
_
1
0
_
= U
_
1
0
_
=

6U
_
_
1
0
0
_
_
=
1
u
1
and
Av
2
= AV
_
0
1
_
= U
_
0
1
_
=

1U
_
_
0
1
0
_
_
=
2
u
2
Thus we should let
u
1
=
1

1
Av
1
=
1
6
_
_
1 1
1 1

3 0
_
_
_
1

5
_
2
1
__
=
1

30
_
_
3
3
2

3
_
_
and
u
2
=
1

2
Av
2
=
1
1
_
_
1 1
1 1

3 0
_
_
_
1

5
_
1
2
__
=
1

5
_
_
1
1

3
_
_
2
If we check, we see that {u
1
, u
2
} are indeed orthonormal vectors. The question is, why does this work?
Well, just using the facts that u
1
=
1
1
Av
1
, u
2
=
1
2
Av
2
, A
T
Av
1
=
2
1
v
1
(since v
1
is an eigenvector for A
T
A
with eigenvalue
2
1
), A
T
Av
2
=
2
2
v
2
(v
2
is an eigenvector for A
T
A with eigenvalue
2
2
), and {v
1
, v
2
} is an
orthonormal set, we see that
u
1
u
1
=
_
1

1
Av
1
_

_
1

1
Av
1
_
=
_
1

1
Av
1
_
T
_
1

1
Av
1
_
=
1

2
1
v
T
1
A
T
Av
1
=
1

2
1
v
T
1
(A
T
Av
1
)
=
1

2
1
v
T
1
(
2
1
v
1
)
= v
T
1
v
1
= v
1
v
1
= 1
and
u
2
u
2
=
_
1

2
Av
2
_

_
1

2
Av
2
_
=
_
1

2
Av
2
_
T
_
1

2
Av
2
_
=
1

2
2
v
T
2
A
T
Av
2
=
1

2
2
v
T
2
(A
T
Av
2
)
=
1

2
2
v
T
2
(
2
2
v
2
)
= v
T
2
v
2
= v
2
v
2
= 1
and
u
1
u
2
=
_
1

1
Av
1
_

_
1

2
Av
2
_
=
_
1

1
Av
1
_
T
_
1

2
Av
2
_
=
1

2
v
T
1
A
T
Av
2
=
1

2
v
T
1
(A
T
Av
2
)
=
1

2
v
T
1
(
2
2
v
2
)
=

2

1
v
T
1
v
2
=

2

1
v
1
v
2
= 0
3
which is want we wanted.
However, to nd an orthogonal matrix using u
1
and u
2
, we need a third orthonormal vector as we need
U to be a 3 3 matrix and we only have two orthonormal vectors (in some cases, you will not need this
step). To nd u
3
, write
u
3
=
_
_
a
b
c
_
_
Then we want a
2
+ b
2
+ c
2
= 1 so u
3
is unit length, and we want
0 = u
1
u
3
=
1

30
(3a + 3b + 2

3c)
and
0 = u
2
u
3
=
1

5
(a b +

3c)
By solving a system of linear equations, we obtain that

_
_
1
1
0
_
_
where R is the set of all solutions to the equations 0 =
1

30
(3a +3b +2

3c) and 0 =
1

5
(a b +

3c).
Therefore, if we let u
3
=
1

2
_
_
1
1
0
_
_
, u
3
is a normal vector that is orthogonal to u
1
and u
2
. Thus {u
1
, u
2
, u
3
}
is an orthonormal basis for R
3
. Then, if we let
U =
_
u
1
u
2
u
3

=
_

_
3

30

1

5
1

2
3

30

1

5

1

2
2

10

5
0
_

_
then U is an orthogonal matrix and A = UV
T
as desired. If you are not condent in your computations,
you could easily check your answer by multiplying this out.
Now we desire to determine the action of A on the unit circle. To see this, we will rst apply V
T
to the
unit circle, then to that image, and then we will apply U. The following is a diagram representing the
actions of each map:
4
In the diagram, we start with the top left diagram. The vector v
1
and its images are shown in blue and the
vector v
2
and its images are shown in red. First we know that V take e
1
to v
1
and e
2
to v
2
. Since V
T
= V
1
as V is an orthogonal matrix, V
T
takes v
1
to e
1
and v
2
to e
2
. Then adds a z-axis and dilates the x-axis by

6. Then U rotates the whole diagram in an awkward manner (you probably do not need to know exactly
what this rotation is but, it takes the x-axis to the line through u
1
, it takes the y-axis to the line through
u
2
, and it takes the z-axis to the line through u
3
). It is hard to see what the curve in the nal diagram looks
like. The blue line goes to the point

30
(3, 3, 2

3) and the red line goes to


1

5
(1, 1,

3).
5

You might also like