LU AND PLU FACTORIZATION

TERRY A. LORING

1. P RODUCTS

OF

E LEMENTARY

MATRICES , REVISITED

In the posting Expanding an Invertible to a Product of Elementary Matrices there was this matrix

0
1 0
A = 8 8 1
2 2 0

which we expressed as a product of elementary matrices as

1 1 0
1 0 0
2 0 0
1 0 0
0 0 1
A = 0 1 0 0 0 1 0 1 0 0 1 0 0 1 0 .
0 0 1
0 0 1
0 1 0
4 0 1
1 0 0

Suppose we multiply together the first two as

0 0 1
1 0 0
0 1 0
P = 0 1 0 0 0 1 = 0 0 1
1 0 0
0 1 0
1 0 0

and let L be the third matrix

1 0 0
L= 0 1 0
4 0 1

and multiply the last two as

2 2 0
2 0 0
1 1 0
U = 0 1 0 0 1 0 = 0 1 0 .
0 0 1
0 0 1
0 0 1
We have this description of A :

A = P LU
or

1 0 0
2 2 0
0 1 0
0
1 0
8 8 1 = 0 0 1 0 1 0 0 1 0
1 0 0
4 0 1
0 0 1
2 2 0

where
P is a so-called permutation matrix
L is lower triangular
1

TERRY A. LORING

U is upper triangular
This is called an LU or PLU decomposition of A.
Lower and upper triangular matrices are computationally easier than
your typical invertible matrix. The matrix P is easy to deal with as well
since it is mostly full of zeros. It is called a permutation matrix because it
would equal the identity matrix if we could permute its rows.
In computer solutions of systems of equations, and elsewhere, it is easier
to deal with three easier matrices than one hard one. We will deal with
applications of this later, or not at all. For now, I want to show you how to
use row and column ops to create an LU factorization.
2. E LEMENTARY,

BUT

D IFFERENT

Lets rework this example, but using different row operations. We still
will do the operations in this order: type II, type III clearing below, type
II, them type III clearing above.

R1 R2

0
1 0
8 8 1
2 2 0

R3 R3 + 41 R1

R1 81 R1

8 8 1
1 0
0
2 2 0

8 8 1
0 1 0
0 0 14

1 1 18
0
0 1
1
0 0
4

R3 4R3

1 1 18
0 1
0
0 0
1

R1 R1 + R2

1 0 81
0 1 0
0 0 1

R1 R1 + 81 R3

1 0 0
0 1 0
0 0 1

LU AND PLU FACTORIZATION

Now we look at the undo operations:

99K
R1 R2
99K R3 R3 41 R1
99K
R1 8R1
99K
R3 14 R3
99K R1 R1 R2
99K R1 R1 81 R3

and now the elementary matrices:

1 0 0
1 0 0
1 1 0
0 1 0
8 0 0
1 0 18
A = 1 0 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0
0 0 1
0 0 1
0 0 1
0 0 1
0 0 41
14 0 1

Let us now multiply together in three parts:

0 1 0
P1 = 1 0 0 ,
0 0 1

1 0 0
L1 = 0 1 0 ,
41 0 1

1 0 0
1 1 0
8 0 0
1 0 18
U1 = 0 1 0 0 1 0 0 1 0 0 1 0
0 0 1
0 0 1
0 0 1
0 0 14

1
8 0 0
1 1 8
= 0 1 0 0 1
0
1
0 0
1
0 0 4

8 8 1
= 0 1 0 .
0 0 14
That is, we have found

A = P1 L1 U1
or

1
0
1 0
0 1 0
8 8 1 = 1 0 0 0
0 0 1
2 2 0
14

3. O NCE

MORE , USING

0 0
8 8 1
1 0 0 1 0 .
0 1
0 0 41
M ATLAB

In Matlab, the rather odd input

TERRY A. LORING

L =
1.0000
-0.5000
-0.5000
0
U =

0
1.0000
1.0000
0

0
0
1.0000
-0.5000

0
0
0
1.0000

-4.0000
-1.0000
0
0
0.5000
2.0000
0
0
-2.0000
0
0
0
P =
0
0
0
1
0
1
0
0
1
0
0
0
0
0
1
0
A weird thing is that Matlab promises that

-2.0000
2.0000
-2.0000
1.0000

[L,U,P] = lu(A)
will set the three variables on the left in a way to ensure
A = P 1 LU.
In this case, that did not matter since P = P 1 .
4. PLU

WITHOUT

E LEMENTARY M ATRICES .

In practice, one does not this list of elementary matrices. One can get
directly the PLU factorization.
The new idea is that one can take an equation like
A = BC
and do a row operation on C and a balancing column operation on B to get
A = B1 C1 .
This is because
BC = BE 1 EC
whenever E is an elementary matrix. For example



1
1
BC = B
C
0 1
0 1
tells us that if we do the row operation
R1 R1 + R2
on the right factor we can offset this with the column operation
C2 C2 C2.

LU AND PLU FACTORIZATION

The offsets are as follows:

on left factor
1
Cj Cj
Cj Ck
Ck Ck Cj

on right factor
1
Rj Rj
Rj Rk
Rj Rj + Rk

We can start with the silly equality

A = IIA

and proceed to make the last factor upper triangular, the middle lower triangular and the left a permutation matrix. nop stands for no operation.
Each step involves two adjacent factors and leaves the other alone
For example:

1 0 4
0 0 1 =
3 2 13

1 0 4
1 0 0
1 0 0
0 1 0 0 1 0 0 0 1
3 2 13
0 0 1
0 0 1

nop & C3 C3 + 3C1 &

1 0 4
0 0 1 =
3 2 13

R3 R3 3R1

1 0 0
1 0 0
1 0 4
0 1 0 0 1 0 0 0 1
0 0 1
3 0 1
0 2 1
nop & C2 C3 & R2 R3

1 0 4
0 0 1 =
3 2 13

1 0 0
1 0 0
1 0 4
0 1 0 0 0 1 0 2 1
0 0 1
3 1 0
0 0 1
C2 C3 & R2 R3 &

1 0 4
0 0 1 =
3 2 13

nop

1 0 0
1 0 0
1 0 4
0 0 1 3 1 0 0 2 1
0 1 0
0 0 1
0 0 1

TERRY A. LORING

5. A 4- BY-4 PLU

EXAMPLE

Now lets do a 4-by-4 factoring into PLU form.

2
1 0 1
1 0 0 0
1
2

0 1 0 0 0
1
2
3

0
0 0 1 0 0
0 1 2
4 1 0 2
0 0 0 1
0

0
1
0
0

0
0
1
0

0
2
1
2
0
1

0
0
0
1
4 1

nop & C1 C1 + C2 &

2
1
2
1

0
0
4 1

0 1
2 3
=
1 2
0 2

1
0

0
0

0
1
0
0

0
0
1
0

1
0
1
1

0 0
2
0

0
1
0
0

0
0
1
0

0 1
2 3

1 2
0 2

R2 R2 R1
R4 R4 + 2R1

2 1
0
0 0
0

0 0 0
0 1
1

0
2
1
0

1
2

2
0

0
0
1
2

1
0

2
2

0
0
1
2

1
0

2
2

nop & C2 C4 & R2 R4

2
1
2
1

0
0
4 1

0 1
2 3
=
1 2
0 2

1
0

0
0

0
1
0
0

0
0
1
0

1
0
1
0

0 0
2
1

C2 C4 &

2
1
2
1

0
0
4 1

0 1
2 3
=
1 2
0 2

1
0

0
0

0
0
0
1

0
0
1
0

1
0
2
1

0 0
0
1

0
0
0
1

0
0
1
0

2 1
0
0 1
1

0 0 0
0 0
0

R2 R4
0
1
0
0

0
0
1
0

& nop

0
2 1
0 1
0

0 0 0
0 0
1

nop & C3 C3 + 2C4&R4 R4 2R3

2
1
2
1

0
0
4 1

0 1
2 3
=
1 2
0 2

1
0

0
0

0
0
0
1

0
0
1
0

1
0

1 2
0 0
0
1

0
1
0
0

0
0
1
2

0
2 1

0 0 1
0 0 0
0 0
1

The Matlab session

0 1
0 0

1 2
0 2

LU AND PLU FACTORIZATION

0
0
1

1
0
0

0
0
0

0
1
0

>> L
L =
1.0000
0
0
0
-0.5000
1.0000
0
0
-0.5000
1.0000
1.0000
0
0
0
-0.5000
1.0000
>> U
U =
-4.0000
-1.0000
0
-2.0000
0
0.5000
2.0000
2.0000
0
0
-2.0000
-2.0000
0
0
0
1.0000
shows another of the possible PLU factorization of this is

1 0 0 0
2
1 0 1
0 0 1 0

1
1 2 3

= 0 1 0 0 21 1 0 0
0
0 1 2 0 0 0 1 2 1 1 0
4 1 0 2
1 0 0 0
0 0 12 1

I prefer the one we found by hand, as we eschewed fractions.

4 1 0
1
1
0
2
2
2

0
0 2 2
0
0
0
1