MATH2033 (2021–2022)

Introduction to Scientific Computation

Linear Systems: Direct Methods III

School of Mathematical Sciences

Linear Systems: Direct Methods III 2/40

Matrix factorisation
Suppose that A ∈ Rn×n and b ∈ Rn .
Suppose that there exists a unique solution x ∈ Rn to Ax = b and that
we know a lower triangular matrix L ∈ Rn×n with all entries on its main
diagonal being nonzero and an upper triangular matrix U ∈ Rn×n with all
entries on its main diagonal being nonzero which are such that A = LU .
We can let y = U x and use forward substitution to find the solution y to
Ly = b
which requires O(n2 ) arithmetic operations.
We can then use backward substitution to find the solution x to
Ux = y
which requires an additional O(n2 ) arithmetic operations.
The total number of arithmetic operations required is O(n2 ).
Recall that for Gaussian elimination the total number of arithmetic
operations required was O(n3 ).
Linear Systems: Direct Methods III 3/40

Problem 1
Let
1 1
A=[ ].
2 1
Find matrices L and U such that

A = LU

with L being of the form

1 0
[ ]
l21 1
and U being of the form
u11 u12
[ ].
0 u22
Linear Systems: Direct Methods III 4/40

We can compute that

u11 u12
LU = [ ].
l21 u11 l21 u12 + u22
So A = LU when
⎧
⎪ u11 = 1,
⎪
⎪
⎪
⎪ u12 = 1,
⎨
⎪
⎪ l21 u11 = 2,
⎪
⎪
⎪
⎩ l21 u12 + u22 = 1,
from which we can conclude that
u11 = 1,
u12 = 1,
2 2
l21 = = = 2,
u11 1
and
u22 = 1 − l21 u12 = 1 − 2 = −1.
Linear Systems: Direct Methods III 5/40

Hence,
1 0
L=[ ]
2 1
and
1 1
U =[ ].
0 −1
Linear Systems: Direct Methods III 6/40

Problem 2
Let
⎡ 0 ⎤⎥
⎢ 1 1
⎢ ⎥
A=⎢ 2 1 −1 ⎥ .
⎢ ⎥
⎢ 0 −1 −1 ⎥⎦
⎣
Determine the LU factorisation of A where L is of the form
⎡ 1 0 ⎤⎥
⎢ 0
⎢ ⎥
⎢ l21 1 0 ⎥
⎢ ⎥
⎢ l31 l32 1 ⎥
⎣ ⎦
and U is of the form
⎡ u11 u12 u13 ⎤
⎢ ⎥
⎢ ⎥
⎢ 0 u22 u23 ⎥.
⎢ ⎥
⎢ 0 0 u33 ⎥
⎣ ⎦
Linear Systems: Direct Methods III 7/40

We can compute that

⎡ u11 ⎤
⎢ u12 u13 ⎥
⎢ ⎥
LU = ⎢ l21 u11 l21 u12 + u22 l21 u13 + u23 ⎥.
⎢ ⎥
⎢ l31 u11 l31 u12 + l32 u22 l31 u13 + l32 u23 + u33 ⎥
⎣ ⎦
So A = LU when
⎧
⎪ u11 = 1,
⎪
⎪
⎪
⎪
⎪
⎪ u12 = 1,
⎪
⎪
⎪
⎪
⎪ u13 = 0,
⎪
⎪
⎪
⎪
⎪ l21 u11 = 2,
⎪
⎨ l21 u12 + u22 = 1,
⎪
⎪
⎪
⎪
⎪ l21 u13 + u23 = −1,
⎪
⎪
⎪
⎪
⎪ l31 u11 = 0,
⎪
⎪
⎪
⎪
⎪ l31 u12 + l32 u22 = −1,
⎪
⎪
⎪
⎩ l31 u13 + l32 u23 + u33 = −1,

from which we can conclude that

Linear Systems: Direct Methods III 8/40

u11 = 1,
u12 = 1,
u13 = 0,
2 2
l21 = = = 2,
u11 1
u22 = 1 − l21 u12 = 1 − 2 = −1,
u23 = −1 − l21 u13 = −1 − 0 = −1,
0 0
l31 = = = 0,
u11 1
1 1
l32 = (−1 − l31 u12 ) = (−1 − 0) = 1
u22 −1
and
u33 = −1 − l31 u13 − l32 u23 = −1 − 0 − (−1) = 0.
Linear Systems: Direct Methods III 9/40

Hence,
⎡ 1 0 0 ⎤
⎢ ⎥
⎢ ⎥
L=⎢ 2 1 0 ⎥
⎢ ⎥
⎢ 0 1 1 ⎥
⎣ ⎦
and
⎡ 1 0 ⎤⎥
⎢ 1
⎢ ⎥
U = ⎢ 0 −1 −1 ⎥ .
⎢ ⎥
⎢ 0 0 0 ⎥⎦
⎣
Linear Systems: Direct Methods III 10/40

Problem 3
Let
⎡ 0 −1 −1 ⎤
⎢ ⎥
⎢ ⎥
B=⎢ 2 1 −1 ⎥ .
⎢ ⎥
⎢ 1 1 0 ⎥⎦
⎣
Note that A ∼ B (R1 ↔ R3 ) where A is the matrix from Problem 2. Can
we write B = LU where
⎡ 1 0 ⎤⎥
⎢ 0
⎢ ⎥
L = ⎢ l21 1 0 ⎥
⎢ ⎥
⎢ l31 l32 1 ⎥
⎣ ⎦
and
⎡ u11 u12 u13 ⎤
⎢ ⎥
⎢ ⎥
U =⎢ 0 u22 u23 ⎥?
⎢ ⎥
⎢ 0 0 u33 ⎥
⎣ ⎦
Linear Systems: Direct Methods III 11/40

We can compute that

⎡ u11 ⎤
⎢ u12 u13 ⎥
⎢ ⎥
LU = ⎢ l21 l21 u12 + u22 l21 u13 + u23 ⎥.
⎢ ⎥
⎢ l31 u11 l31 u12 + l32 u22 l31 u13 + l32 u23 + u33 ⎥
⎣ ⎦
So B = LU is equivalent to
⎧
⎪ u11 = 0,
⎪
⎪
⎪
⎪
⎪
⎪ u12 = −1,
⎪
⎪
⎪
⎪
⎪ u13 = −1,
⎪
⎪
⎪
⎪
⎪ l = 2,
⎪ 21
⎨ l21 u12 + u22 = 1,
⎪
⎪
⎪
⎪
⎪ l21 u13 + u23 = −1,
⎪
⎪
⎪
⎪
⎪ l31 u11 = 1,
⎪
⎪
⎪
⎪
⎪ l31 u12 + l32 u22 = 1,
⎪
⎪
⎪
⎩ l31 u13 + l32 u23 + u33 = 0.
However, the equations u11 = 0 and l31 u11 = 1 contradict each other.
Consequently, we cannot write B = LU .
Linear Systems: Direct Methods III 12/40

Theorem
Suppose that A ∈ Rn×n . Let A(1) = A. Suppose that forward elimination
can be performed without any row interchanges so that, for
i = 1, 2, . . . , n − 1, the matrix A(i+1) can be obtained by performing the
elementary row operations

Rj − mji Ri → Rj with mji = aji /aii for j = i + 1, i + 2, . . . , n

(i) (i)

on the matrix A(i) . Then A = LU where U = A(n) and L ∈ Rn×n is the

lower triangular matrix with entries
⎧
⎪ m if i < j,
⎪
⎪ ji
lji = ⎨ 1 if i = j,
⎪
⎪
⎪
⎩ 0 if i > j,
for j = 1, 2, . . . , n and i = 1, 2, . . . , n.
Linear Systems: Direct Methods III 13/40

Problem 4
Let
⎡ 0 ⎤⎥
⎢ 1 1
⎢ ⎥
A=⎢ 2 1 −1 ⎥ .
⎢ ⎥
⎢ 0 −1 −1 ⎥⎦
⎣
Find a lower triangular matrix L and an upper triangular matrix U which
are such that A = LU .
Linear Systems: Direct Methods III 14/40

Performing forward elimination without interchanging any rows yields

⎡ 0 ⎤⎥
⎢ 1 1
⎢ ⎥
⎢ 2 1 −1 ⎥
⎢ ⎥
⎢ 0 −1 −1 ⎥⎦
⎣
⎡ 0 ⎤⎥
⎢ 1 1
R − m21 R1 → R2 , m21 = 21 = 2
⎢ ⎥
∼ ⎢ 0 −1 −1 ⎥ ( 2 )
⎢ ⎥ R3 − m31 R1 → R3 , m31 = 01 = 0
⎢ 0 −1 −1 ⎦⎥
⎣
⎡ 0 ⎤⎥
⎢ 1 1
⎢ ⎥
∼ ⎢ 0 −1 −1 ⎥ ( R3 − m32 R2 → R3 , m32 = −1
−1 = 1 .
)
⎢ ⎥
⎢ 0 0 0 ⎦⎥
⎣
So by the theorem on page 12, A = LU with
⎡ 1 0 0 ⎤ ⎡ 1 0 ⎤⎥
⎢ ⎥ ⎢ 1
⎢ ⎥ ⎢ ⎥
L = ⎢ 2 1 0 ⎥ and U = ⎢ 0 −1 −1 ⎥ .
⎢ ⎥ ⎢ ⎥
⎢ 0 1 1 ⎥ ⎢ 0 0 0 ⎥⎦
⎣ ⎦ ⎣
Linear Systems: Direct Methods III 15/40

Permutation matrices
If the permutation matrix P has been obtained by performing row
interchanges on the appropriate identity matrix then P A will be the
matrix obtained by performing the same row interchanges on the matrix
A.
Moreover, P −1 = P T .
Linear Systems: Direct Methods III 16/40

Problem 5
Let
⎡ 1 0 ⎤⎥
⎢ 1
⎢ ⎥
A=⎢ 2 1 −1 ⎥
⎢ ⎥
⎢ 0 −1 −1 ⎥
⎣ ⎦
and
⎡ 0 −1 −1 ⎤
⎢ ⎥
⎢ ⎥
B=⎢ 2 1 −1 ⎥ .
⎢ ⎥
⎢ 1 1 0 ⎥⎦
⎣
Find the permutation matrix P which is such that P B = A and hence
obtain the factorisation B = P −1 LU where L and U are the matrices
from Problem 4.
Linear Systems: Direct Methods III 17/40

We first observe that A can be obtained by performing R1 ↔ R3 on B.

We then perform R1 ↔ R3 on the 3 × 3 identity matrix to obtain
⎡ 0 0 1 ⎤
⎢ ⎥
⎢ ⎥
P = ⎢ 0 1 0 ⎥.
⎢ ⎥
⎢ 1 0 0 ⎥
⎣ ⎦
Now,
⎡ 0 0 1 ⎤
⎢ ⎥
⎢ ⎥
P −1
=P T
= ⎢ 0 1 0 ⎥ = P.
⎢ ⎥
⎢ 1 0 0 ⎥
⎣ ⎦
Since P B = A, B = P −1 A = P A = P LU from the solution to Problem
4. Hence,
⎡ 0 0 1 ⎤⎡ 1 0 0 ⎤⎡ 1 0 ⎤⎥
⎢ ⎥⎢ ⎥⎢ 1
⎢ ⎥⎢ ⎥⎢ ⎥
B = ⎢ 0 1 0 ⎥ ⎢ 2 1 0 ⎥ ⎢ 0 −1 −1 ⎥ .
⎢ ⎥⎢ ⎥⎢ ⎥
⎢ 1 0 0 ⎥⎢ 0 1 1 ⎥⎢ 0 0 0 ⎥⎦
⎣ ⎦⎣ ⎦⎣
Linear Systems: Direct Methods III 18/40

LU (Doolittle, Crout) factorisation

We write A = LU where L is a lower triangular matrix and U is an
upper triangular matrix with:
the entries on the main diagonal of L all being 1 for Doolittle
factorisation.
the entries on the main diagonal of U all being 1 for Crout
factorisation.
Linear Systems: Direct Methods III 19/40

T
LDL factorisation
We write A = LDLT where L is a lower triangular matrix where the
entries on the main diagonal are all 1 and D is a diagonal matrix.
Linear Systems: Direct Methods III 20/40

T
Cholesky (LL ) factorisation
We write A = LLT where L is a lower triangular matrix where the entires
on the main diagonal are all positive.
Linear Systems: Direct Methods III 21/40

Strictly diagonally dominant matrices

A matrix A ∈ Rn×n is a strictly diagonally dominant matrix if and only if

∣aii ∣ > ∑ ∣aij ∣ for all i ∈ N

j∈N i

where N = {i ∈ Z ∶ 1 ≤ i ≤ n} and N i = {j ∈ N ∶ j ≠ i}.

If A ∈ Rn×n is strictly diagonally dominant then forward elimination can
be performed on A without any row interchanges with all pivot entries
nonzero.
Linear Systems: Direct Methods III 22/40

Problem 6
Let
⎡ 7 2 0 ⎤⎥
⎢
⎢ ⎥
A = ⎢ 3 5 −1 ⎥ .
⎢ ⎥
⎢ 0 5 −6 ⎥
⎣ ⎦
Is A strictly diagonally dominant?
Is AT strictly diagonally dominant?
Linear Systems: Direct Methods III 23/40

We have that ∣7∣ = 7 and ∣2∣ + ∣0∣ = 2 and so

∣7∣ > ∣2∣ + ∣0∣,
∣5∣ = 5 and ∣3∣ + ∣ − 1∣ = 4 and so
∣5∣ > ∣3∣ + ∣ − 1∣,
and ∣ − 6∣ = 6 and ∣0∣ + ∣5∣ = 5 and so
∣ − 6∣ > ∣0∣ + ∣5∣.
Hence, A is strictly diagonally dominant.
Now,
⎡ 7 3 0 ⎤⎥
⎢
⎢ ⎥
A =⎢ 2
T
5 5 ⎥.
⎢ ⎥
⎢ 0 −1 6 ⎥
⎣ ⎦
We have that ∣5∣ = 5 and ∣2∣ + ∣5∣ = 7 and so
∣5∣ ≯ ∣2∣ + ∣5∣.
Hence, AT is not strictly diagonally dominant.
Linear Systems: Direct Methods III 24/40

Symmetric positive definite matrices

A square matrix A is a symmetric matrix if and only if A = AT .
A matrix A ∈ Rn×n is a positive definite matrix if and only if

xT Ax > 0 for all nonzero x ∈ Rn .

A symmetric positive definite matrix is a matrix that is both symmetric

and positive definite.
If A ∈ Rn×n is a symmetric matrix then the following statements are
equivalent:
A is positive definite.
forward elimination can be performed on A without any row
interchanges with all pivot entries positive.
A has an LDLT factorisation where L is a lower triangular matrix
where the entries on the main diagonal are all 1 and D is a diagonal
matrix.
A has a Cholesky factorisation.
Linear Systems: Direct Methods III 25/40

Problem 7
Let
2 −1
B=[ ].
−1 2
Is B symmetric positive definite?
Linear Systems: Direct Methods III 26/40

We first observe that

2 −1
BT = [ ]=B
−1 2

and so B is symmetric. Now,

2 −1 x 2x1 − x2
[ x1 x2 ] [ ] [ 1 ] = [ x1 x2 ] [ ]
−1 2 x2 −x1 + 2x2
= 2x21 − x1 x2 − x1 x2 + 2x22
= 2x21 − 2x1 x2 + 2x22
= x21 + x22 + (x1 − x2 )2
> 0
x1
for all nonzero vectors [ ] ∈ R2 and so B is positive definite. Hence,
x2
B is symmetric positive definite.
Linear Systems: Direct Methods III 27/40

Alternatively, after observing that B is symmetric, we can perform

R2 − −1
2 R1 → R2 on B to arrive at

2 −1
[ 3 ]
0 2

from which we can conclude that B is positive definite because forward

elimination can be performed on B without any row interchanges with all
pivot entries positive and hence B is symmetric positive definite.
Linear Systems: Direct Methods III 28/40

Problem 8
Let
2 −1
A=[ ].
−1 2

Determine the LDLT factorisation of A, where

1 0
L=[ ]
l21 1

and
d11 0
D=[ ].
0 d22
Linear Systems: Direct Methods III 29/40

We can compute that

1 0 d 0 1 l21
LDLT = [ ] [ 11 ][ ]
l21 1 0 d22 0 1
d11 0 1 l21
= [ ][ ]
d11 l21 d22 0 1
d11 d11 l21
= [ ].
2
d11 l21 d11 l21 + d22
Linear Systems: Direct Methods III 30/40

So A = LDL when
T
⎧
⎪ d = 2,
⎪
⎪ 11
⎨ d11 l21 = −1,
⎪
⎪
⎪
⎩ d11 l21 + d22 = 2
2

from which we can conclude that

d11 = 2,
−1 −1 1
l21 = = =−
d11 2 2
and
2 3
d22 = 2 − d11 l21
2
=2− = .
4 2
Hence, A = LDLT with

1 0 2 0
L=[ ] and D = [ ].
− 12 1 0 3
2
Linear Systems: Direct Methods III 31/40

Problem 9
Let
2 −1
A=[ ].
−1 2

Determine the LLT factorisation of A, where

l11 0
L=[ ]
l21 l22

with l11 > 0 and l22 > 0.

Linear Systems: Direct Methods III 32/40

We can compute that

l11 0 l l
LLT = [ ] [ 11 21 ]
l21 l22 0 l22
2
l11 l11 l21
= [ 2 ].
2
l11 l21 l21 + l22

So A = LLT when
⎧
⎪ l2 = 2,
⎪
⎪ 11
⎨ l11 l21 = −1,
⎪
⎪
⎪
⎩ l21 + l22 = 2,
2 2

or equivalently,
⎧
⎪ l2 = 2,
⎪
⎪ 11
⎨ l21 = − l11
1
,
⎪
⎪
⎪
⎩ l22 = 2 − l21 .
2 2
Linear Systems: Direct Methods III 33/40

Consequently, since l11 > 0 and l22 > 0,

√
l11 = 2,
1
l21 = − √
2
and √
3
l22 = √ .
2
Hence, A = LLT with
⎡ √2 0 ⎤⎥
⎢
L = ⎢⎢ √ ⎥.
⎥
⎢ − √2 √3
1
⎥
⎣ 2 ⎦
Linear Systems: Direct Methods III 34/40

Tridiagonal matrices
An n × n matrix A is a tridiagonal matrix if and only if

aij = 0 for all i, j ∈ N such that ∣i − j∣ > 1

where N = {i ∈ Z ∶ 1 ≤ i ≤ n}.
If A ∈ Rn×n is tridiagonal and is strictly diagonally dominant then A has
an LU factorisation where L is a tridiagonal lower triangular matrix and
U is a tridiagonal upper triangular matrix.
If A ∈ Rn×n is tridiagonal and is symmetric positive definite then A has
an LU factorisation where L is a tridiagonal lower triangular matrix and
U is a tridiagonal upper triangular matrix.
If A ∈ Rn×n is tridiagonal and det(A) ≠ 0, b ∈ Rn and advantage is taken
of the fact that A is tridiagonal, obtaining the solution x ∈ Rn to Ax = b
requires O(n) arithmetic operations.
Linear Systems: Direct Methods III 35/40

Problem 10
Let
⎡ 2 −1 0 ⎤⎥
⎢
⎢ ⎥
A = ⎢ −1 2 −1 ⎥ .
⎢ ⎥
⎢ 0 −1 2 ⎥⎦
⎣
Determine an LU factorisation of A.
Linear Systems: Direct Methods III 36/40

Since A is a tridiagonal matrix, we seek

⎡ l11 0 ⎤
⎢ 0 ⎥
⎢ ⎥
L = ⎢ l21 l22 0 ⎥
⎢ ⎥
⎢ 0 l32 l33 ⎥
⎣ ⎦
and
⎡ 1 u12 0 ⎤
⎢ ⎥
⎢ ⎥
U =⎢ 0 1 u23 ⎥.
⎢ ⎥
⎢ 0 0 1 ⎥
⎣ ⎦
We can compute that
⎡ l11 ⎤
⎢ l11 u12 0 ⎥
⎢ ⎥
LU = ⎢ l21 l21 u12 + l22 l22 u23 ⎥.
⎢ ⎥
⎢ 0 l32 l32 u23 + l33 ⎥
⎣ ⎦
Linear Systems: Direct Methods III 37/40

So A = LU when
⎧
⎪ l11 = 2,
⎪
⎪
⎪
⎪
⎪
⎪ l11 u12 = −1,
⎪
⎪
⎪
⎪
⎪ l21 = −1,
⎪
⎨ l21 u12 + l22 = 2,
⎪
⎪
⎪
⎪
⎪ l22 u23 = −1,
⎪
⎪
⎪
⎪
⎪ l32 = −1,
⎪
⎪
⎪
⎩ l32 u23 + l33 = 2.
Linear Systems: Direct Methods III 38/40

Consequently,
l11 = 2,
1 1
u12 = − =− ,
l11 2
l21 = −1,
1 3
l22 = 2 − l21 u12 = 2 −= ,
2 2
1 2
u23 = − =− ,
l2 2 3
l32 = −1
and
2 4
l33 = 2 − l32 u23 = 2 − = .
3 3
Linear Systems: Direct Methods III 39/40

Hence, A = LU with
⎡ 2 0 ⎤⎥
⎢ 0
⎢ ⎥
L = ⎢ −1 2
3
0 ⎥
⎢ ⎥
⎢ 0 −1 43 ⎥⎦
⎣
and
⎡ 1 −1 0 ⎤⎥
⎢
⎢ 2 ⎥
U =⎢ 0 1 − 23 ⎥ .
⎢ ⎥
⎢ 0 0 1 ⎥⎦
⎣
Linear Systems: Direct Methods III 40/40

References
Burden, Faires & Burden, Numerical Analysis, 10E
Section 6.5
Section 6.6 (only those parts that are mentioned in these slides)