QR factorisation and pseudoinverse of rank-

deficient matrices
Peter Moylan
Glendale, NSW
[email protected] http://www.pmoylan.org
April 2016

ABSTRACT: Standard algorithms for QR decomposition assume

that the matrix in question has full rank. We present an
alternative that can, without pivoting, handle the rank-deficient
case, and that produces an matrix that has linearly independent
rows. One application of this approach is the efficient calculation
of pseudoinverses.

1. Introduction
The QR factorisation of a (real or complex) matrix is defined to be the
decomposition
=
where has the property = , and is a (not necessarily square) upper triangular
matrix. Here, the superscript star indicates the adjoint (complex conjugate transpose) of a
matrix. The size of the matrices is not specified, except to the extent that the matrix product
must make sense. In practice there are three important ways of defining a QR factorisation.
In the traditional definition is an unitary matrix ( = = ), and is an
upper triangular matrix. This is called the full QR decomposition. It turns out, however,
that some of the most useful applications of QR factorisation are when . In such cases
the traditional definition leads to unreasonably large matrices.
If > , the traditional factorisation has the form

= [1 2 ] [1 ] = 1 1
0

where 1 still has the property 1 1 = , and 1 is still upper triangular. Now 1 is
and 1 is . In this case we choose to say that the pair (1,1 ) is still a QR factorisation
of , one that is computationally more convenient because the matrices are smaller. Because
1 is not square it is technically not unitary, but for our purposes that does not matter. In the
literature this is called the thin QR factorisation or the reduced QR factorisation. It is unique
if has full column rank and we restrict attention to those solutions where the diagonal
elements of 1 are positive. Obviously 2 is not unique, but that does not matter because we
have no interest in computing 2 .

page 1 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

The goal of the present paper is to present an algorithm that can also handle rank-deficient
matrices. Specifically, we will find a factorisation where the number of rows of is equal to
the rank of . Clearly, no smaller decomposition is possible, so we will call this the minimal
or fully reduced QR factorisation.
Our approach is not the same as in rank-revealing QR factorisations, see e.g. [Ming96].
Those use a modified full or thin QR factorisation, and then look at the matrix to look for a
small or zero subblock. The goal in the present paper is not to generate the negligible part of
at all. This give computational savings, and also ensures that the rows of are linearly
independent.
One motivation for using a QR factorisation is that triangular matrices are easy to invert. This
does not appear to help in the rank-deficient case, but for some applications it turns out that a
pseudoinverse can be used instead of an inverse. For this reason, we show at the end of the
paper how to computer a pseudoinverse, cheaply, using the minimal QR factorisation as an
intermediate step.

2. Existence and uniqueness of a minimal QR decomposition

In this section we prove the existence and uniqueness of a specified kind of fully reduced QR
factorisation. In general a QR factorisation is not unique, but it becomes unique once we add
some constraints.
Definition: An upper triangular matrix is in fully reduced row echelon form if it has no all-
zero rows, if the first nonzero element in each row is real and positive, and if that first
nonzero element lies strictly to the right of the first nonzero element of the row above it.
Clearly, such a matrix has linearly independent rows. In other words, the number of rows is
equal to its rank. Note that the no all-zero rows condition implies that it also has at least as
many columns as rows.
In what follows we will sometimes need the matrix to be nonzero, so we should first
dispose of the case = 0. In that case the decomposition is non-unique, but one solution is
1
0
=[ ] = [0 0 0]

0
This is, strictly speaking, not a minimal decomposition (since has rank 0 and has one
row), but it is the best we can do.
For any nonzero , we have the following result.
Theorem. Any nonzero real or complex matrix has a unique decomposition = with
the properties that is in fully reduced row echelon form, and = .
Proof. We proceed by induction on the number of columns of . If is nonzero and has only
one column, the result is obvious by inspection. ( is then a 1 1 matrix.) Otherwise, let 1
be the first column of , so that
= [1 2 ]

page 2 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

If 1 = 0, then by assumption 2 0, and 2 has fewer columns than . By the inductive

hypotheses, there exists a unique factorisation 2 = 2 2 with 2 and 2 in the required
form. Then we have
= [0 2 2 ] = 2 [0 2 ]
This satisfies the requirements of the theorem statement.
If 1 0, any factorisation, if it exists, necessarily has the form
1
= [1 2 ] = [1 2 ] [ 0 ] = [1 1 1 + 2 2 ]
2

where 1 is the first column of and 1 is a scalar. The fact that = means that 1 1 =
1 and 1 2 = 0 and 2 2 = . This implies
1
1 = 1 1 1 = 1
1

and these values are unique because of the constraint that 1 be real and positive. There are
now two subcases to consider. For reasons that will become clear below, define
= 1 2 2 = 2 1
If 2 = 0 (which, it is easy to show, occurs iff has rank 1), we have
= [1 2 ] = [1 1 1 ] = 1 [1 ]
Observe that in this case the matrix has a single row, which is trivially in row echelon
form. This solution is unique, because any other factorisation would have more rows in
than the rank of .
Finally, consider the general case where 1 0 and 2 0. By the inductive hypothesis,
and the fact that 2 has fewer columns than , there exist unique matrices 2 and 2 such
that 2 = 2 2 , 2 2 = , and 2 is in fully reduced row echelon form. Define
1
= [1 2 ] = [0 ]
2

Then

2 ] [ 01 2 ] = [1 1
= [1 1 + 2 ] = [1 2 ] =

Clearly has the desired properties, so it only remains to be shown that = . By

construction, 1 1 = 1 and 2 2 = , so only the cross product is in question. Observe that
1 2 2 = 1 2 1 1 = 1 2 = 0
and 1 2 = 0 follows because the rows of 2 are linearly independent.

3. A practical implementation
The proof in the last section is a constructive one, so to turn it into software we need only
copy the steps of the proof. The result appears to be a recursive algorithm, but in practice no
recursion is involved: we are simply stepping through the columns of , at the same time
filling in columns of and rows and columns of , and modifying the remaining columns of
. The calculation is complete when we have finished processing either the last row or the
last column of the modified .

page 3 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

Pivoting can be added to the algorithm, if desired. The only change this requires is to find, at
each step, the largest of the columns not yet processed, and if necessary to swap that with the
current column. This, however, does mean that the factorisation now has the form =
or equivalently = , where is a permutation matrix. Depending on the application,
this might not be desirable, because will not be a triangular matrix.
Note that the special case 2 = 0 does not require any special handling, other than
detecting that that matrix is zero. At that point in the calculation 1 and have already had
their values assigned, so all that remains to be done is to exit from the calculation loop.
Let be an matrix. The fully reduced decomposition is most beneficial when ,
but it is also valid for the case . We start the calculation by implicitly assuming that
has min(, ) rows, but we omit one or more rows each time we meet a special case (1 = 0
or 2 = 0). This does imply discarding partial rows of that we have set in previous
steps, but no information is lost, because the discarded entries are merely some of the zero
entries below the main diagonal.
An in-place calculation is possible, where each column of that we calculate replaces a
column of . If we are doing an in-place calculation, we need to watch out for one detail. In
the case 1 = 0, where we step to the next column of without adding a column to , we
might have to be prepared to shift the remainder of one column left. Alternatively, and
perhaps more efficiently, we just need to keep track of the fact that the current column of is
not necessarily the same as the current column of .

4. QR decomposition using Householder reflections

It is commonly stated that a QR decomposition via Householder reflections has better
numerical stability than the modified Gram-Schmidt process already described. We are
therefore motivated to see whether the Householder approach can be modified to produce a
minimal decomposition.
As shown in Appendix 2, a Householder transformation can act on a vector to produce

0
= [ ]

0
where is a scalar. If is the first column of an matrix , then we have

= [ 0 ]
2
which is the first step towards producing an upper triangular matrix. Repeating this operation
on smaller and smaller submatrices, we get
1 2 1 =
and therefore
=
where
= 1 2

page 4 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

Observe that is an matrix. That means that, regardless of the rank of , we have a
full QR decomposition, which is not really acceptable if . In this case, the last
rows of are zero, so we can get a thin QR decomposition by deleting the last
columns of .
This, unfortunately, does not translate into deleting rows or columns of the Householder
matrices . We have no alternative but to retain an matrix until the end of the
calculation; only at the end can we discard the redundant columns. For a long thin matrix ,
this can create serious storage problems.
But can we modify the Householder approach to get a minimal factorisation? Luckily, we
can. Partway through the calculation, we have
1
= = [ ]
0 22
Uniqueness of the solution for means that 1 and are precisely the matrices obtained by
our Section 2 calculations. That means that we can use the Section 2 insights to work out how
to modify the Householder approach to produce a minimal decomposition.
Let the upper left corner of 22 be at row and column (0 , 0 ). In the full-rank case we have
0 = 0 , but for our algorithm this is no longer true. From Section 2 we know that we have to
deal with two special cases:
(a) If the first column of 22 is zero, then we should increment 0 without incrementing
0 for the next step.
(b) Otherwise, if the updated 22 is zero, then we have finished the calculation.
It turns out that we do not need to check for case (b). If the case (b) condition is satisfied,
then on the following iterations we will immediately run into case (a), so the calculations will
be terminated in any case. Whether we do this check is purely a matter of software efficiency,
and does not affect the final answer.
So does the Householder approach give a more accurate answer? If running a test suite, we
should check (which should be zero), and (which should be zero). Our tests
on random matrices shows that the Householder approach gives better values of , but
worse values of . In other words, the Householder approach is better at producing
orthogonal columns of the matrix, but is not as good at producing a QR factorisation. If our
goal is to produce a good matrix, then the Householder approach is less desirable.

5. LQ decomposition
An LQ decomposition of a matrix has the form
=
where is lower triangular and has the property that = . As in the QR case, we can
have full, thin, or fully reduced version. The details are exactly as in the QR case, except that
we swap the labelling of rows and columns.

page 5 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

6. Pseudoinverses
A pseudoinverse calculation is typically done via a singular value decomposition [SVD], but
this is computationally expensive. In this section we show that the minimal
decomposition can be used as a computationally cheap way to compute the result. The
pseudoinverse (Moore-Penrose inverse) # of a real or complex matrix is defined
uniquely by the properties [Pen55]:
# and # are both Hermitian
# =
# # = #
The pseudoinverse arises in connection with solving equations like
=
The solution = # has the properties
1. If one or more exact solutions exist, then the given solution is the one of least norm.
2. If no exact solution exists, then the pseudoinverse solution is the one that comes
closest to being a solution in that it minimises .
3. If is invertible then # = 1.
The following properties can be confirmed by substitution into the above conditions for a
pseudoinverse.
The operation of taking the Hermitian (complex conjugate transpose) of a matrix
commutes with the pseudoinverse operation. That is, ( )# = (# ) .
If has full column rank, then # = ( )1 .
If has full row rank, then # = ( )1 .

7. Calculating the pseudoinverse

As always, we need to assume that we are dealing with nonzero matrices. The pseudoinverse
of an all-zero matrix is equal to its transpose, but we have to separate this out as a special
case.
Let = , where = . Then it is easily shown, directly from the definition of a
pseudoinverse, that # = # . (There is no similar result for an arbitrary matrix product.)
That leaves us with the problem of computing # . Because of the way we do the
decomposition, has full row rank and is upper triangular.
If that is square, it is nonsingular, so the pseudoinverse is equal to the inverse. Calculating
the inverse of an upper triangular matrix is a well-known low-cost algorithm.
More generally, if is and has rank , then is . If < then the upper
triangular property turns out to be of no help, because the pseudoinverse of a non-square
upper triangular matrix is usually not a triangular matrix. However, the full rank property
means that
# = ( )1
At first sight, this appears to complicate matters by requiring us to calculate a full matrix
inverse. The matrix is Hermitian, and the usual recommended way of inverting an

page 6 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

Hermitian matrix is via the Cholesky decomposition [Cho]. A Cholesky decomposition works
by factoring the matrix in a certain way. Below, we show a cheaper method.
Suppose we do a QR factorisation of the matrix . The result is
= 1 1
where 1 is now a square upper triangular matrix, and 1 is , i.e. it has more
rows than columns. Then
( )1 = (1 1 1 1 )1 = (1 1 )1 = 11 (11 )
# = ( )1 = 1 1 11 (11 ) = 1 (11 )
This is a significant gain, because now the only inversion we have to deal with is the
inversion of a triangular matrix. The cost is that we have had to do a new QR factorisation,
but this is still cheaper than doing a full matrix inversion.
The conclusion is that we can calculate the pseudoinverse of any arbitrary matrix using at
most two QR factorisations and inverting a triangular matrix (which is easy). This appears to
be computationally more efficient than, for example, using a singular value decomposition.
Computational accuracy can be improved by using pivoting in the course of the
calculation. This is a minor change to the calculation, and results in finding a permutation
matrix such that = . With this change, we have = and # = # . That
is, it is the same calculation plus a final permutation.

8. Relationship to singular value decomposition

It is not hard to see that the method just described is effectively a QR decomposition followed
by an LQ decomposition, and in fact we can save one adjoint calculation by doing it that way.
If we have a factorisation = , where = and = and is lower triangular,
then it is easy to show that # = 1 .
If our goal is to compute a pseudoinverse then we need go no further, because the triangular
matrix is easy to invert. If, however, we choose to do a similar decomposition of , we get
a sequence of factorisations
=
where each is the result of doing a PLQ decomposition of 1 . It turns out that at each
step is more diagonal than its predecessor, in the sense that the off-diagonal elements are
becoming smaller, so that eventually we converge towards a diagonal .
This is, in fact, a known algorithm for finding a singular value decomposition, although not
the most efficient [SVD]. We do, however, depart from the usual definition in the following
way. If is and has rank , then according to the conventional approach it has
min(, ) singular values, of which are nonzero. Because we are using minimal QR and
LQ decompositions, instead of the more usual thin decompositions, our are nonsingular at
every step. As a result, we calculate only the nonzero singular values. Of course, no
information is lost, because we know that the missing ones are all zero. We can restore them,
if this is really necessary, by adding zero rows and/or columns to , and padding the and
matrices with extra blocks that, because they are multiplied by zero, have no effect on the
product. For most applications this will not be necessary, because the positive singular values
are the interesting ones.

page 7 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

9. Conclusions
It is possible, with a fairly simple change to the conventional Gram-Schmidt approach, to do
a QR decomposition where the matrix is guaranteed to have full row rank. One
consequence of this is that we can calculate pseudoinverses far more efficiently than by using
a singular value decomposition.
The software for the algorithms described here are available for download [Moy16].

AUTHORS NOTE: As a retired academic I have no access to on-line research journals, so

cannot find out which parts of this paper, if any, are original. I gather that some universities
let retired staff retain their library privileges, but that is not the case for my university. I
apologise for the poor referencing.

References

[Cho] Cholesky Decomposition, https://en.wikipedia.org/wiki/Cholesky_decomposition

[Ming96] Ming Gu and Stanley C. Eisenstat, Efficient algorithms for computing a strong
rank-revealing QR factorisation, SIAM J Sci. Comput. 17 (4), 848-868, July 1996.
[Moy16] P.J. Moylan, Numerical Analysis Software,
http://pmoylan.org/pages/m2/NumAnal.html
[Pen55] R. Penrose, A generalized inverse for matrices, Mathematical Proceedings of the
Cambridge Philosophical Society, 51, pp 406-413, 1955.
doi:10.1017/S0305004100030401.
[SVD] Singular Value Decomposition,
https://en.wikipedia.org/wiki/Singular_value_decomposition

page 8 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

Appendix 1: Inversion of triangular matrices

The following results are well-known, but are collected here for convenience.

For a partitioned upper triangular (and nonsingular) matrix, the equation [ ][ ]=
0
0 + + 0
[ ] expands to [ ]=[ ], which implies = 1 and = 0.
0 0
From this observation, there is an obvious inductive argument that shows that the inverse (if
it exists) of an upper triangular matrix is itself upper triangular, and the diagonal elements of
the inverse are the reciprocals of the corresponding diagonal elements of the original matrix.
Let be an upper triangular matrix, and let be its inverse. The property = expands out
to

() = =
=1
and because we know that some of the terms are zero, this reduces to

=
=

For row , we already know that = 1 and that = 0 for all < . For > ,
1

+ = 0
=

or
1
1
=

=
If we move from left to right through the array, the right side of this equation involves only
quantities we have already calculated.
An alternative approach, starting from the equation = , leads to the formula

1
=

=+1

for > , which is appropriate if we want to move backwards through the rows. These two
methods are of equal complexity and apparently equal accuracy, so either method should
work equally well.
The results for a lower triangular matrix are similar. If is a nonsingular lower triangular
matrix and is its inverse, then is also lower triangular, with
1
=

and

page 9 of 10
 QR factorisation and pseudoinverse of rank-deficient matrices

1
1
=

=

for > . This is in a form suitable for working through the rows from first to last. There is a
second formula which also works, but is less convenient because it requires working
backwards through the columns.

Appendix 2: Householder transformations

This is again known material, but it is included here because most of the available references
cover only the case of real matrices, and the extension to the complex case is not entirely
obvious.
A Householder transformation on a matrix is multiplication on the left by a matrix
=
where is a real or complex scalar and is a unit-norm vector ( = 1). It is easy to show
that = = iff = 1 + , where is a real or complex number such that || = 1.
In the real case the only possible choices are = 0 (meaning no change) or = 2. In the
complex case the choice of is a little trickier.
The property = implies that = for any vector . This says that by suitable
choice of we can rotate or reflect the vector, but we cannot change its size. In the context
of QR calculations, we are interested in producing the result = , where is a unit
vector. In matrix notation, that means a vector with 1 in its first position and zero entries
elsewhere. The norm-preserving property implies that || = , which fixes the magnitude
of the scalar but not its phase. For our application there is no freedom of choice in the
phase either. If we are using the Householder transformation to do a QR factorisation, then
must be real and nonnegative. This forces the choice = .
Geometrical considerations suggest that we should choose = , where
=
Then
+
= = =
2 2
The result that we want is = = , which requires that
+ = 2
and a short calculation reduces this to

=

This is all we need to complete the calculation.

page 10 of 10