A Contour Integral-Based Algorithm for Computing

Generalized Singular Values

Yuqi Liu1 , Xinyu Shan2 , and Meiyue Shao2,3
1
School of Mathematical Sciences, Fudan University, Shanghai 200433, China
2
School of Data Science, Fudan University, Shanghai 200433, China
arXiv:2401.00121v1 [math.NA] 30 Dec 2023

3
MOE Key Laboratory for Computational Physical Sciences, Fudan University,
Shanghai 200433, China

January 2, 2024

Abstract
We propose a contour integral-based algorithm for computing a few singular values of a
matrix or a few generalized singular values of a matrix pencil. Mathematically, the general-
ized singular values of a matrix pencil are the eigenvalues of an equivalent Hermitian–definite
matrix pencil, known as the Jordan–Wielandt matrix pencil. However, direct application
of the FEAST solver does not fully exploit the structure of this problem. We analyze sev-
eral projection strategies on the Jordan–Wielandt matrix pencil, and propose an effective
and robust scheme tailored to GSVD. Both theoretical analysis and numerical experiments
demonstrate that our algorithm achieves rapid convergence and satisfactory accuracy.

Keywords: Generalized singular value decomposition, contour integration, FEAST, Jordan–

Wielandt matrix pencil, subspace iteration
AMS subject classifications (2020). 15A18, 65F15, 65F50

1 Introduction
The generalized singular value decomposition (GSVD) of a matrix pencil proposed by Van Loan
in [43] is a generalization of the singular value decomposition (SVD) of a single matrix. It was
further developed by Paige and Saunders [31] and many others [5, 33, 44]. GSVD is closely
related to many numerical linear algebra and data science problems, such as the solution of
eigenvalue problems [10, 23, 26], several variants of least squares problems [16, 29, 43], the
general Gauss–Markov linear model [10, 29], the linear discriminant analysis [32], information
retrieval [19], real-time signal processing [27, 39], etc. GSVD has been applied to solve several
real world problems, e.g., the comparative analysis of the DNA microarrays [1] and ionospheric
tomography [6].
There exist several numerical algorithms for computing the GSVD for dense matrices [4, 30,
43]. Recent development on stable computation of the CS decomposition [40] provides another
alternative for computing the GSVD. As for large and sparse problems, there are also several
approaches. Zha’s algorithm [48], which is based on the CS decomposition and the Lanczos
bidiagonalization process, can be used to compute a few extreme generalized singular values.

1
Jacobi–Davidson (JD) type algorithms [18, 21] are capable for computing generalized singular
values at arbitrary location. In practice JD type algorithms are mostly used to find interior
generalized singular values. Other popular symmetric eigensolvers, such as the LOBPCG al-
gorithm [25] and the ChASE algorithm [45], can also be adopted for GSVD because GSVD is
essentially a symmetric eigenvalue problem.
Contour integration is a recently developed technique for solving eigenvalue problems, espe-
cially, for finding interior eigenvalues. The Sakurai–Sugumura (SS) method [37] solves a sym-
metric eigenvalue problem by constructing a small moment matrix. This method sometimes
suffers from numerical instability since the moment matrix, which is a Hankel matrix, is often
ill-conditioned [38]. FEAST [35, 41] and CIRR [38] belong to another type of contour integral-
based eigensolvers, which are essentially subspace iteration applied to spectral projectors. The
technique of contour integration has been extended to solve other eigenvalue problems, including
non-Hermitian eigenvalue problems [46], polynomial eigenvalue problems [2], and more general
nonlinear eigenvalue problems [47]. Contour integral-based eigensolvers require solving a num-
ber of shifted linear systems with multiple right-hand-sides. Efficient methods for solving these
shifted systems can be found in, e.g., [11, 28, 36].
In this work we discuss how to use contour integration to compute a few singular values
of a matrix A or a few generalized singular values of a matrix pencil (A, B) within a given
interval. We solve this problem by applying the FEAST solver to the Jordan–Wielandt matrix
(pencil). Although such an approach may look straightforward since SVD and GSVD are special
cases of the symmetric eigenvalue problem, additional care is required in order to make use of
the symmetry of the spectrum of the Jordan–Wielandt matrix (pencil). We shall focus on the
projection strategy of the FEAST solver, and discuss in detail how to choose an effective and
robust projection scheme tailored to SVD/GSVD. Without lose of generality, we assume that
both A and B have full column rank. This assumption is usually plausible in practice as long as
the null space is properly deflated, possibly by another extremal eigensolver.
The rest of this paper is organized as follows. We first briefly review some basic knowledge
about GSVD and the FEAST algorithm in Section 2. In Section 3 we propose a contour integral-
based algorithm for computing partial SVD. Several projection schemes are discussed in detail.
The algorithm naturally carries over to GSVD, which is discussed in Section 4. In Section 5
we present experimental results to demonstrate the effectiveness of our algorithm. The paper is
concluded in Section 6.

2 Preliminaries
2.1 Generalized singular value decomposition
We first briefly review the generalized singular value decomposition of two matrices A ∈ Cm×n
and B ∈ Cp×n . For simplicity, we assume that both A and B have full column rank, i.e.,
rank(A) = rank(B) = n. There exist two matrices U ∈ Cm×n , V ∈ Cp×n with orthonormal
columns (i.e., U ∗ U = V ∗ V = In ), and a nonsingular matrix X ∈ Cn×n such that

A = U CX −1 , B = V SX −1 , (1)

where C = diag{α1 , α2 , . . . , αn } and S = diag{β1 , β2 , . . . , βn } are nonnegative diagonal matrices

satisfying C 2 + S 2 = In . The decomposition (1) is known as the generalized singular value
decomposition (GSVD) of (A, B) [43]. The formulation used here is a compressed form of the
GSVD; see, e.g., [16]. When B = In , the GSVD reduces to the usual SVD of A.

2
 By partitioning U = [u1 , u2 , . . . , un ], V = [v1 , v2 , . . . , vn ], and X = [x1 , x2 , . . . , xn ], we can
reformulate (1) as
Axi = ui αi , Bxi = vi βi , (1 ≤ i ≤ n).
According to [20, 21], the five-tuple (αi , βi , ui , vi , xi ) is called a GSVD component of (A, B). The
number σi = αi /βi is called a generalized singular value; ui and vi are the corresponding left
generalized singular vectors; xi is the corresponding right generalized singular vector.
The GSVD is mathematically equivalent to either

A∗ Axi = B ∗ Bxi σi2 (2)

or      
A ui I ui
= σ, (3)
A∗ wi B∗B wi i
where wi = xi /βi . Numerically, (3) is often superior to (2) since the cross-product A∗ A, which
can largely increase the condition number, is avoided. In the case that B ∗ B is ill-conditioned,
     
B vi I vi
= σ −1 (4)
B∗ xi /αi A∗ A xi /αi i

is another alternative. Finally, we remark that in (3) and (4) the eigenvalues appear in pairs.
For instance, (3) can be reformulated as
      
A ui ui I ui ui σi
= .
A∗ wi −wi B ∗ B wi −wi −σi

2.2 The FEAST algorithm

FEAST [35, 41] is a popular contour integral-based eigensolver for computing a few eigenvalues
in a given domain on the complex plane. In the following we briefly review the FEAST algorithm
for symmetric eigenvalue problems.
Let H be an n × n Hermitian matrix with spectral decomposition
n
X
H= λi qi qi∗ ,
i=1

where λi ’s are the eigenvalues of H, and qi ’s are the corresponding normalized eigenvectors. Let
us consider a domain Ω ⊂ C that encloses the eigenvalues λs+1 ≤ λs+2 ≤ · · · ≤ λs+k . Then the
corresponding spectral projector is given by
k Z
X
∗ 1
PΩ (H) = qs+i qs+i = (ξIn − H)−1 dξ.
i=1
2πi ∂Ω

By applying one step of subspace iteration on PΩ (H), a basis of the invariant subspace of H
with respect to the eigenvalues λs+1 , . . ., λs+k can be extracted from the columns of
Z
1
PΩ (H) · Z = (ξIn − H)−1 Z dξ (5)
2πi ∂Ω

for almost any Z ∈ Cn×k . Then these eigenvalues can be easily calculated by the Rayleigh–Ritz
projection scheme.

3
 FEAST is an algorithm that makes use of (5) to compute the eigenvalues within Ω. In
practice, an approximate spectral projector P̃Ω (H) is applied because a numerical quadrature rule
is performed to evaluate the right-hand-side of (5). For instance, if Ω = {ξ ∈ C : |ξ − µ0 | < ρ},
the trapezoidal rule applied to ∂Ω yields
N −1
1 X
P̃Ω (H)Z = ωj (ξj In − H)−1 Z,
N j=0

where ξj ’s are equally-spaced quadrature nodes on ∂Ω, and ωj ’s are the corresponding quadrature
weights. Then a number of shifted linear systems with multiple right-hand-sides need to be
solved; see [11, 28, 36] for discussions on how to solve these linear systems efficiently. Since
PΩ (H) · Z is computed only approximately, a few steps of subspace iteration may be needed to
refine the solution. In practice, an ellipse instead of a circle can be used as the contour to achieve
more rapid convergence, and the trapezoidal rule often has satisfactory accuracy here since the
integrand is a sufficiently smooth periodic function; see, e.g., [15, 42] for detailed discussions.
The Zolotarev quadrature is sometimes a good alternative to the trapezoidal rule, especially
when some unwanted eigenvalues are close to the contour [15].

In reality we need to estimate the number of eigenvalues in Ω, which is equal to trace PΩ (H) ,
as this number is often not known in advance. For small-to-medium size problems, the number
of desired eigenvalues can be determined by computing the positive index of inertia through
the LDL∗ decomposition of two shifted matrices [34]. For large-scale problems, stochastic trace
estimators [3, 9, 12, 13, 14] are often used.
The FEAST algorithm also carries over to generalized eigenvalue problems. For instance, for
a Hermitian–definite matrix pencil (H, M ), the corresponding spectral projector becomes
Z Z
1 1
PΩ (H, M ) = (ξIn − M −1 H)−1 dξ = (ξM − H)−1 M dξ,
2πi ∂Ω 2πi ∂Ω
which is a self-adjoint positive semidefinite operator in the M -inner product. Hence there is no
essential difference compared to the usual symmetric eigenvalue problem.

3 A FEAST-SVD Algorithm
We first discuss how to compute the (partial) singular value decomposition using contour inte-
gration, as this is an important special case of GSVD. This is essentially a symmetric eigenvalue
problem because the singular values of A can be extracted by computing eigenvalues of either
the Gram matrix A∗ A or the Jordan–Wielandt matrix
 
0 A
Ǎ =
A∗ 0

according to (2) and (3).

The FEAST algorithm applied to the Gram matrix A∗ A is relatively straightforward—if
singular values in the interval (α, β) are of interest, a contour that encloses (α2 , β 2 ) can be chosen,
assuming β > α ≥ 0. However, the accuracy of this approach can be low when small singular
values of A are desired. More accurate results can be expected if we compute the eigenvalues
of the Jordan–Wielandt matrix Ǎ instead. Notice that the spectrum of Ǎ is symmetric with
respect to the origin—nonzero eigenvalues of Ǎ appear in positive and negative pairs. In the
following we mainly discuss how to compute the eigenvalues of Ǎ using FEAST by taking care
of the symmetry of the spectrum.

4
3.1 Structured Galerkin condition and Rayleigh–Ritz projection
Suppose that [Ũ ∗ , W̃ ∗ ]∗ contains approximate eigenvectors of Ǎ with Ũ ∗ Ũ = W̃ ∗ W̃ = I, and Σ̃ is
a diagonal matrix whose diagonal entries are the corresponding approximate eigenvalues.1 Then
the standard Galerkin condition reads
 
Ũ
R ⊥ span , (6)
W̃
where    
Ũ Ũ
R = Ǎ − Σ̃
W̃ W̃
is the (block) residual. Since the nonzero eigenvalues of Ǎ appear in pairs ±σ, assuming Σ̃ is
positive definite, a structured Galerkin condition is
 
Ũ Ũ
Ř ⊥ span , (7)
W̃ −W̃
where     
Ũ Ũ Ũ Ũ Σ̃
Ř = Ǎ −
W̃ −W̃ W̃ −W̃ −Σ̃
is the structured residual. Clearly (7) is stronger than (6), under the assumption that Σ̃ is
positive definite.
The condition (7) simplifies to Ũ ∗ AW̃ = Σ̃, or equivalently,
(
(AW̃ − Ũ Σ̃) ⊥ span(Ũ ),
(A∗ Ũ − W̃ Σ̃) ⊥ span(W̃ ).

A Rayleigh–Ritz projection scheme can then be derived. Suppose that we are given Ũ0 ∈ Cm×k
and W̃0 ∈ Cn×k with Ũ0∗ Ũ0 = W̃0∗ W̃0 = Ik . Let the (full) singular value decomposition of
Ap = Ũ0∗ AW̃0 be
Ap = Ũp Σ̃W̃p∗ .
∗ ∗
By setting ŨRR ← Ũ0 Ũp , W̃RR ← W̃0 W̃p , we obtain ŨRR ŨRR = W̃RR W̃RR = Ik and
∗
ŨRR AW̃RR = Σ̃,
which satisfies the structured Galerkin condition (7). This is essentially equivalent to the usual
Petrov–Galerkin method for SVD. We remark that
 
1 ŨRR ŨRR
√
2 W̃RR −W̃RR
contains orthonormalized approximate eigenvectors of Ǎ. In this manner we are able to extract
eigenvalues of Ǎ in pairs.

3.2 Choice of the spectral projector

Suppose that we are interested in computing the singular values of A that lie in the interval
(α, β) with α ≥ 0. These singular values correspond to Ǎ’s eigenvalues in (−β, −α) ∪ (α, β). In
the following we discuss several possible choices of the spectral projector.
1 Matrices with tilde symbols are used to denote approximations to certain exact values.

5
 ξ2 ξ1
ξ3 ξ0
b
a
α µ0 β
ξ4 ξ7
ξ5 ξ6

Figure 1: An ellipse contour with eight quadrature nodes (i.e., N = 8).

3.2.1 A simple spectral projector

A simple spectral projector is based on an ellipse Γ+ that encloses (α, β). For instance, by
choosing a ≥ (β − α)/2 and b > 0, the ellipse
 
α+β
Γ+ = + a cos θ + ib sin θ : θ ∈ [0, 2π]
2

of semi-axes a and b encloses the interval (α, β); see Figure 1.

The corresponding spectral projector is
Z
+ 1
P (Ǎ) = (ξIm+n − Ǎ)−1 dξ.
2πi Γ+

If Ũ and W̃ , respectively, contain approximate left and right singular vectors of Ǎ, we expect
   
Ũnew Ũ
= P + (Ǎ) (8)
W̃new W̃

to produce a better approximation than [Ũ ∗ , W̃ ∗ ]∗ , as this can be viewed as one step of subspace
iteration.
∗ ∗
However, we remark that (8) is not always a good choice in practice, as [Ũnew , W̃new ]∗ can be
rank deficient if the initial guess is poorly chosen. Let us split the full SVD of A into
∗ ∗
A = Uin Σin Win + Uout Σout Wout ,

where the diagonal elements of Σin are exactly the desired singular values. Then we can express
[Ũ ∗ , W̃ ∗ ]∗ of the form
         
Ũ Uin + Uout + Uin − Uout
= C + C + C + C− .
W̃ Win in Wout out −Win in −Wout out

Applying P + (Ǎ) yields      

Ũnew Ũ Uin
+
= P (Ǎ) = C +,
W̃new W̃ Win in
+ −
so that severe cancellation can occur when ∥Cin ∥ is much smaller than ∥Cin ∥. An extreme case
+ −
is that ∥Cin ∥ = 0 while ∥Cin ∥ is at modest scale, which leads to
   
Ũnew + Ũ
= P (Ǎ) =0
W̃new W̃

6
because [Ũ ∗ , W̃ ∗ ]∗ belongs to an invariant subspace of Ǎ that is orthogonal to the desired one.
This causes the algorithm to break down in exact arithmetic (or converge slowly in practice by
taking into account quadrature and rounding errors), even through useful information of the
desired solution is already encoded in the initial guess. This issue needs to be tackled since we
often do not have much control over the initial guess in practice.

3.2.2 A simple spectral projector with Rayleigh–Ritz projection

One way to resolve the potential issue of rank deficiency is to perform a Rayleigh–Ritz projection
scheme in the first iteration of the FEAST algorithm before applying the spectral projector.
Suppose that we update the orthogonal bases through ŨRR ← Ũ Ũp and W̃RR ← W̃ W̃p , where
Ũp Σ̃W̃p∗ is the singular value decomposition of Ũ ∗ AW̃ . Let us represent [ŨRR
∗ ∗ ∗
, W̃RR ] as
     
ŨRR U + U −
= CRR + CRR .
W̃RR W −W

It can be verified that

 ∗  
1 ŨRR 1 ŨRR + ∗ + − ∗ −
Σ̃ = √ · Ǎ · √ = (CRR ) ΣCRR − (CRR ) ΣCRR .
2 W̃RR 2 W̃RR
Then we have
+ −
∥CRR ∥Σ ≥ ∥CRR ∥Σ ,
+ +
indicating that ∥CRR ∥ is reasonably large. Moreover, CRR is in general not rank deficient when Σ̃
+ ∗ +
is positive definite, since the smallest eigenvalue of (CRR ) ΣCRR is at least as large as ∥Σ̃−1 ∥−1
2 .
Thus, the issue of rank deficient by applying (8) can be resolved by performing a Rayleigh–Ritz
projection scheme.
However, there exist situations that a Rayleigh–Ritz projection at beginning can be harmful.
For example, let us consider an unbalanced initial guess of the form
   
Ũ U
= in ,
W̃ E

where ∥E∥2 = O(ϵ) so that E does not deliver useful information on right singular vectors. We
can rewrite it as
         
Ũ Uin I + Ein Uin I − Ein Uout Eout Uout Eout
= + + − , (9)
W̃ Win 2 −Win 2 Wout 2 −Wout 2
assuming E = Win Ein + Wout Eout . However, after the Rayleigh–Ritz projection we obtain
   
ŨRR Uin Q
= ,
W̃RR W̆

where Q is a unitary matrix, and columns of W̆ form an orthonormal basis of span(E). By

expressing W̆ as W̆ = Win Cin + Wout Cout , we obtain
         
ŨRR Uin Q + Cin Uin Q − Cin Uout Cout Uout Cout
= + + − . (10)
W̃RR Win 2 −Win 2 Wout 2 −Wout 2
Notice that    
Ein Cin
= O(ϵ) ≪ 1 = .
Eout 2
Cout 2

7
In this case (10) can be worse than (9) for two reasons: 1) ∥Q + Cin ∥2 can be small when
cancellation is present, while ∥I + Ein ∥2 is always as large as O(1); 2) the unwanted components
from E are greatly enlarged in (10).

3.2.3 A pair of contours

Another way to resolve the potential difficulty of rank deficiency when applying (8) is to introduce
a contour Γ− that is symmetric to Γ+ with respect to the origin, i.e.,
Z    
− 1 −1 Im + Im
P (Ǎ) = (ξIm+n − Ǎ) dξ = P (Ǎ) .
2πi Γ− −In −In

Then we expect Z
1
P + (Ǎ) + P − (Ǎ) = (ξIm+n − Ǎ)−1 dξ
2πi Γ+ ∪Γ−

to be a safer projector compared to either P + (Ǎ) or P − (Ǎ), since P + (Ǎ) + P − (Ǎ) can properly
preserve the useful information in [Ũ ∗ , W̃ ∗ ], regardless of the sign of the Ritz values.
Though we introduce P + (Ǎ) + P − (Ǎ) in order to make the filter robust, and expect that it
is better than P + (Ǎ), we remark that P + (Ǎ) + P − (Ǎ) can sometimes be worse than P + (Ǎ) in
terms of convergence. We use a simple example to illustrate this. Suppose that we are computing
the first eigenvector, [u∗1 , w1∗ ]∗ , from the initial guess
  X n    +
ũ ui ui αi
=
w̃ wi −wi αi−
i=1

using an approximate filter function P̃ + (·) satisfying

(
Θ(1), if µ ≈ σ1 ,
P̃ + (µ) =
O(ϵ), otherwise,

where ϵ is a tiny positive number. On the one hand, we have

n
u1 (α1+ + α1− ) αi+
 ±      X   
ũnew + −

ũ ui ui
± = P̃ ( Ǎ) + P̃ (Ǎ) = Θ(1) · + − + O(ϵ) · − .
w̃new w̃ w1 (α1 − α1 ) wi −wi αi
i=2

If cancellation occurs in either α1+ + α1− or α1+ − α1− , which is not uncommon in practice, then it
requires additional effort to identify u1 and w1 since the remaining O(ϵ) terms have nonnegligible
impact. On the other hand,
 +        n    +
ũnew + ũ u1 + u1 −
X ui ui αi
= P̃ (Ǎ) = Θ(1) · α + O(ϵ) · α + O(ϵ) ·
+
w̃new w̃ w1 1 −w1 1 wi −wi αi−
i=2

is a reasonably good approximation to a multiple of [u∗1 , w1∗ ]∗ as long as |α1+ | is not too small.
Since we are only interested in one singular value in this example, it is possible to compute
the Rayleigh quotient explicitly. We compute the following Rayleigh quotients:
±
Pn 2 2 2 + − 2
(Aw̃new )∗ (Aw̃new
±
) 2 i=2 O(ϵ )(σ1 − σi )(αi − αi )
± ± = σ 1 − n ,
)∗ w̃new Θ(1)(α1+ − α1− )2 + i=2 O(ϵ2 )(αi+ − αi− )2
P
(w̃new
Pn 2 2 2 + − 2
(11)
(A∗ ũ± ∗ ∗ ±
new ) (A ũnew ) 2 i=2 O(ϵ )(σ1 − σi )(αi + αi )
= σ1 − Pn ,
(ũ± ∗ ±
new ) ũnew Θ(1)(α1+ + α1− )2 + i=2 O(ϵ2 )(αi+ + αi− )2

8
and Pn
+ 2 2 2 + − 2
(Aw̃new )∗ (Aw̃new
+
) 2 i=2 O(ϵ )(σ1 − σi )(αi − αi )
+ + = σ 1 − 2 ,
(w̃new )∗ w̃new n
Θ(1)α1+ − O(ϵ)α1− + i=2 O(ϵ2 )(αi+ − αi− )2

P
Pn 2 2 2 + − 2
(12)
(A∗ ũ+ ∗ ∗ +
new ) (A ũnew ) 2 i=2 O(ϵ )(σ1 − σi )(αi + αi )
+ + = σ 1 − 2 .
(ũnew )∗ ũnew n
Θ(1)α1+ + O(ϵ)α1− + i=2 O(ϵ2 )(αi+ + αi− )2

P

Using the monotonicity of linear fractional functions, we easily see that (12) is better when
cancellation occurs in either α1+ + α1− or α1+ − α1− , while (11) is better when |α1+ | is tiny. Since
cancellation in α1+ + α1− or α1+ − α1− is not uncommon, especially for real matrices, the use of (11)
can potentially be problematic.

3.2.4 An augmented scheme with a pair of contours

We have seen that using P + (Ǎ) alone can be problematic since [Ũ ∗ , W̃ ∗ ]∗ can contain eigenvectors
of Ǎ with respect to negative eigenvalues so that P + (Ǎ)[Ũ ∗ , W̃ ∗ ]∗ is rank deficient. Using
P + (Ǎ) + P − (Ǎ) is not fully satisfactory for our purpose as it can sometimes be even worse than
P + (Ǎ). Another possibility is to compute
      
Ũnew Ũ Ũ
= P + (Ǎ) , P − (Ǎ) (13)
W̃new W̃ W̃

and then use Ũnew and W̃new for the Rayleigh–Ritz projection. Note that
     
Ũ I Ũ
P − (Ǎ) = m P + (Ǎ) .
W̃ −In −W̃

We can use    
Ũnew Ũ Ũ
= P + (Ǎ) (14)
W̃new W̃ −W̃
instead of (13). In this manner all useful information from P + (Ǎ) and P − (Ǎ) is kept, while the
price to pay is that size of the projected problem is doubled.
Fortunately, we do not have to always double the problem size. The issue of using P + (Ǎ)
alone is avoided by applying (14) since P − (Ǎ) is also implicitly involved. Starting from the
second iteration, we can ensure that components with respect to positive eigenvalues of Ǎ are
already encoded in [Ũ ∗ , W̃ ∗ ]∗ . Therefore, we only need to apply (14) in the first iteration.
Then in subsequent iterations (8) already suffices. In fact, such a combination of (14) and (8)
can dramatically accelerate the convergence. We shall illustrate this phenomenon by numerical
experiments in Section 5, and provide a brief explanation in the Appendix. Finally, we summarize
this strategy as Algorithm 1.
It is worth mentioning that in Step 14 of Algorithm 1, there are 2ℓ SVD triplets after the
Rayleigh–Ritz projection, and we need to choose ℓ of them. The singular values are usually
chosen to be the closest to the desired interval. If there are more than ℓ Ritz values lie in the
interval, which is not uncommon in practice, the SVD triplets with smaller residuals will be
chosen.
Once a few singular triplets have attained satisfactory accuracy, it is sensible to deflate
these singular triplets to reduce the cost in subsequent calculations. The following soft locking
strategy [24] can be adopted: the Rayleigh–Ritz projection involves all approximate singular
vectors, while spectral projectors are only applied to undeflated singular vectors.

9
Algorithm 1 A FEAST-SVD algorithm.
Input: A matrix A ∈ Cm×n ; location of desired singular values (α, β); initial guess U0 ∈ Cm×ℓ
and W0 ∈ Cn×ℓ satisfying U0∗ U0 = W0∗ W0 = Iℓ ; quadrature nodes with weights (ξj , ωj )’s for
j = 1, 2, . . ., N . The number of desired singular values k cannot exceed ℓ.
Output: Approximate singular triples (Σ, U, W ).
1: U ← U0 , W ← W0 .
2: for iter = 1, 2, . . . do
3: if iter = 1 then 
U U
4: Z← .
W −W
5: else  
U
6: Z← .
W
7: end if PN
8: [U ∗ , W ∗ ]∗ ← j=1 ωj (ξj I − Ǎ)−1 Z.
9: Orthogonalize U and W so that U ∗ U = W ∗ W = I.
10: Solve SVD of Ap = U ∗ AW to obtain the singular triplet (Σ, Up , Wp ).
11: U ← U Up , W ← W Wp .
12: Check convergence.
13: if iter = 1 then
14: Choose ℓ best components from (Σ, U, W ) as the new (Σ, U, W ).
15: end if
16: end for

3.3 Trace estimation

In practice, we need to estimate the number of singular values k that lie in the desired interval
(α, β) before applying Algorithm 1. In Section 2.2, we already mentioned that k is equal to the
trace of the spectral projector. Once an estimate of k is available, we can choose an appropriate
initial guess with sufficiently many columns.
In order to estimate k we adopt the Monte Carlo trace estimator [14]. Notice that
Z
1
P + (Ǎ) = (ξIm+n − Ǎ)−1 dξ
2πi Γ+

is Hermitian and positive semidefinite. Let y ∈ Rm+n be a random vector satisfying E[yy ∗ ] =
Im+n . Then we have
trace P + (Ǎ) = E y ∗ P + (Ǎ)y .

 

PN
Thus the problem reduces to calculate ŷ = P + (Ǎ)y ≈ j=1 ωj (ξj I − Ǎ)−1 y. In practice we can
draw several independent samples y1 , y2 , . . ., yK of y to obtain a reasonably accurate estimate
of trace P + (Ǎ) through


K
1 X ∗
trace P + (Ǎ) ≈


y ŷi .
K i=1 i
Finally, we remark that when computing ŷ, the number of quadrature nodes N does not need
to be the same as that in Algorithm 1. Sometimes a smaller number of quadrature nodes suffices
in the trace estimator.

10
4 A FEAST-GSVD Algorithm
Similar to (partial) singular value decomposition, the computation of generalized singular value
decomposition can be reduced to a generalized symmetric eigenvalue problem through (2), (3)
or (4). We assume that B is moderately well-conditioned so that (3) is an appropriate choice.
In the following we discuss how to derive a FEAST algorithm for the Jordan–Wielandt matrix
pencil    
A I
(Ǎ, B̌) = , .
A∗ B∗B

4.1 Rayleigh–Ritz projection

Let [Ũ ∗ , W̃ ∗ ]∗ consists of approximate eigenvectors of (Ǎ, B̌), and Σ̃ be a positive definite diagonal
matrix whose diagonal entries are the corresponding approximate eigenvalues. The standard
Galerkin condition becomes
       
Ũ Ũ Ũ
Ǎ − B̌ Σ̃ ⊥ span . (15)
W̃ W̃ W̃

Then a structured Galerkin condition is

       
Ũ Ũ Ũ Ũ Σ̃ Ũ Ũ
Ǎ − B̌ ⊥ span , (16)
W̃ −W̃ W̃ −W̃ −Σ̃ W̃ −W̃

which simplifies to (
(AW̃ − Ũ Σ̃) ⊥ span(Ũ ),
(17)
(A∗ Ũ − B ∗ B W̃ Σ̃) ⊥ span(W̃ ).

To derive a Rayleigh–Ritz projection scheme based on (17), we assume that Ũ0 ∈ Cm×k and
W̃0 ∈ Cn×k contain orthonormalized columns in the sense that

Ũ0∗ Ũ0 = Ik , W̃0∗ B ∗ B W̃0 = Ik .

Let the singular value decomposition of Ap = Ũ0∗ AW̃0 be

Ap = Ũp Σ̃W̃p∗ .

By setting ŨRR ← Ũ0 Ũp , W̃RR ← W̃0 W̃p , we obtain

∗ ∗ ∗
ŨRR AW̃RR = Σ̃, ŨRR ŨRR = Ik , W̃RR B ∗ B W̃RR = Ik .

The other approximate left generalized singular vectors of (A, B) can be obtained by

Ṽ = B W̃RR .

Computing Ṽ from W̃RR is reasonably accurately here, since we assume that B is moderately
well-conditioned. If, in addition, X, C, and S are needed, they can be computed through

S̃ = (I + Σ̃2 )−1/2 , C̃ = Σ̃S̃, X̃ = W̃RR S̃.

11
4.2 A FEAST algorithm for GSVD
Suppose that we are interested in computing the generalized singular values in the interval (α, β)
with α ≥ 0. Let Γ+ be a contour that encloses (α, β). The corresponding spectral projector is
Z
+ 1
P (Ǎ, B̌) = (ξ B̌ − Ǎ)−1 B̌ dξ.
2πi Γ+

For approximate generalized singular vectors Ũ and W̃ , the filtering scheme

   
Ũnew + Ũ
= P (Ǎ, B̌) (18)
W̃new W̃

is expected to produce a better approximation. Similar to what we have discussed in Section 3

for SVD, applying (18) is not robust in general. A safer filtering scheme
          
Ũnew Ũ Ũ Ũ I Ũ
= P + (Ǎ, B̌) = P + (Ǎ, B̌) , m P − (Ǎ, B̌) (19)
W̃new W̃ −W̃ W̃ −In W̃

needs to be applied in the first iteration, where

Z    
1 I Im
−
P (Ǎ, B̌) = (ξ B̌ − Ǎ) B̌ dξ = m
−1 +
P (Ǎ, B̌)
2πi Γ− −In −In

is the spectral projector associated with the contour Γ− that is symmetric to Γ+ with respective
to the origin. Algorithm 2 summarizes a FEAST algorithm for GSVD based on (18) and (19).
We remark that additional care is required in order to estimate the trace of the spectral
projector for GSVD. Because
Z
+ 1
P (Ǎ, B̌) = (ξ B̌ − Ǎ)−1 B̌ dξ
2πi Γ+

is not Hermitian, the classical Monte Carlo trace estimator can be very inaccurate. To avoid this
difficulty, we observe that P + (Ǎ, B̌) is similar to
Z
1 ∗
C (ξ B̌ − Ǎ)−1 dξ · C (20)
2πi Γ+

where C = diag{In , B ∗ }. The matrix in (20) is Hermitian and positive semidefinite so that
 Z   Z 
1 ∗ −1 1 ∗ ∗ −1
trace C (ξ B̌ − Ǎ) dξ · C = E y C (ξ B̌ − Ǎ) Cy dξ
2πi Γ+ 2πi Γ+

can be accurately estimated through several independent samples of y with E[yy ∗ ] = I2n .

5 Numerical Experiments
In this section we report experimental results of Algorithms 1 and 2. All numerical experi-
ments were performed using MATLAB R2022b on a Linux server with two 16-core Intel Xeon
Gold 6226R 2.90 GHz CPUs and 1024 GB main memory.

12
Algorithm 2 A FEAST-GSVD algorithm.
Input: Two matrices A ∈ Cm×n , B ∈ Cp×n ; location of desired generalized singular values
(α, β); initial guess U0 ∈ Cm×ℓ and W0 ∈ Cn×ℓ satisfying U0∗ U0 = W0∗ B ∗ BW0 = Iℓ ; quadra-
ture nodes with weights (ξj , ωj )’s for j = 1, 2, . . ., N . The number of desired generalized
singular values k cannot exceed ℓ.
Output: Approximate generalized singular values Σ and the corresponding left and right gen-
eralized singular vectors (U, W ).
1: U ← U0 , W ← W0 .
2: for iter = 1, 2, . . . until convergence do
3: if iter = 1 then 
U U
4: Z← .
W −W
5: else  
U
6: Z← .
W
7: end if PN
8: [U ∗ , W ∗ ]∗ ← j=1 ωj (ξj B̌ − Ǎ)−1 B̌Z.
9: Orthogonalize U and W so that U ∗ U = W ∗ B ∗ BW = I.
10: Solve SVD of Ap = U ∗ AW to obtain the singular triplet (Σ, Up , Wp ).
11: U ← U Up , W ← W Wp .
12: Check convergence.
13: if iter = 1 then
14: Choose ℓ best components from (Σ, U, W ) as the new (Σ, U, W ).
15: end if
16: end for

5.1 Experiment settings

We choose twelve test matrices from the SuiteSparse Matrix Collection (formally, the University
of Florida Sparse Matrix Collection) [8], as listed in Table 1. For GSVD, the matrix B is taken
as the scaled transpose of the discretized first order derivative, i.e.,
 ∗
1 −1
 1 −1 
B=  ∈ C(n+1)×n .
 
.. ..
 . . 
1 −1
Table 1 also contains other information regarding the experiments, such as the desired interval
(α, β), the number of (generalized) singular values in the interval kSVD (or kGSVD ), reported by
eig() in MATLAB, as well as the estimated number k̃SVD (or k̃GSVD ) through stochastic trace
estimator. These test matrices are chosen from a broad range of applications, and have been
used by researchers for testing SVD/GSVD algorithms [22].
For computed approximate generalized singular triplets (σ̂i , ûi , ŵi )’s, the convergence criteria
are


∥Aŵi − ûi σ̂i ∥2 ≤ tol · ∥A∥2 ∥ŵi ∥2 + |σ̂i | , (21)
∗ ∗ 2


∥A ûi − B B ŵi σ̂i ∥2 ≤ tol · ∥A∥2 + |σ̂i |∥B∥2 ∥ŵi ∥2 , (22)
∥Û ∗ Û − I∥2 = O(u), (23)
∗ ∗
∥Ŵ B B Ŵ − I∥2 = O(u), (24)

13
 Table 1: List of test matrices.
ID Matrix A m n nnz(A) ∥A∥2 (α, β) kSVD k̃SVD kGSVD k̃GSVD
1 plat1919 1,919 1,919 32,399 2.93 (2.1, 2.5) 8 8.42 5 4.62
2 rosen10⊤ 6,152 2,056 68,302 20,200 (9.9, 10.1) 12 11.32 17 16.47
3 GL7d12 8,899 1,019 37,519 14.4 (11, 12) 17 17.91 19 18.23
4 3elt dual 9,000 9,000 26,556 3 (1.5, 1.6) 368 346.31 251 261.46
5 fv1 9,604 9,604 85,264 4.52 (3.1, 3.15) 89 84.18 56 56.21
6 shuttle eddy 10,429 10,429 103,599 16.2 (7, 7.01) 6 7.24 2 2.07
7 nopoly 10,774 10,774 70,842 23.3 (12, 12.5) 340 352.44 159 160.45
8 flower 5 4⊤ 14,721 5,226 43,942 5.53 (4.1, 4.3) 137 136.75 51 53.09
9 barth5 15,606 15,606 61,484 4.23 (1.5, 1.6) 384 389.89 317 326.99
10 L-9 17,983 17,983 71,192 4 (1.2, 1.3) 477 489.40 607 619.06
11 crack dual 20,141 20,141 60,086 3 (1, 1.1) 330 329.34 602 601.41
12 rel8 345,688 12,347 821,839 18.3 (13, 14) 13 12.69 185 186.77

where u is the machine precision, tol is a user specified threshold, and Û = [û1 , û2 , . . . ], Ŵ =
[ŵ1 , ŵ2 , . . . ]. Because (23) and (24) are automatically ensured by our algorithms, we only need to
compute the residuals when checking the convergence. However, we remark that as an interior
eigensolver, the number of Ritz values within the desired interval (α, β) can be larger than
the actual number of generalized singular values in (α, β). Therefore, we impose an additional
stopping criterion—the algorithm terminates either all Ritz values within (α, β) have converged
according to (21) and (22), or the number of converged Ritz values remains unchanged in two
consecutive iterations.
In all runs, the initial guess is randomly generated unless otherwise specified, with ℓ =
⌈3/2 · k̃SVD ⌉ + 5 (or ℓ = ⌈3/2 · k̃GSVD ⌉ + 5) columns, where k̃SVD (or k̃GSVD ) is estimated using
12 quadrature nodes and 30 random trial vectors. The contour Γ+ is chosen as the ellipse with
+
major axis [α, β] and aspect ratio ρ = a/b = 5. The number √ of quadrature nodes on Γ is set to
−14
N = 12. The convergence threshold is set to tol = 10 m.

5.2 Convergence history

Figure 2 shows the convergence history for computing the singular values in the desired interval
(α, β). We plot the maximum relative residual

∥A∗ ûi − B ∗ B ŵi σ̂i ∥2

 
∥Aŵi − ûi σ̂i ∥2
max max ,
σi ∈(α,β) ∥A∥2 ∥ŵi ∥2 + |σ̂i | ∥A∥2 + |σ̂i |∥B∥22 ∥ŵi ∥2

for singular values inside (α, β) in these convergence plots. Algorithm 1 typically finds all singular
values in the desired interval within three iterations for most matrices, and, moreover, due to
our additional convergence criterion the third iteration is merely for confirming the convergence.
Even for matrices with relatively slow convergence, most singular values are found in the first two
or three iterations. Subsequent iterations only make marginal contribution on the convergence
of one or two singular values.
For GSVD, the behavior of Algorithm 2 is similar. Figure 3 shows the convergence history.
Algorithm 2 successfully finds all generalized singular values in the desired interval within three
or four iterations for our examples.

14
 100 plat1919 100 rosen10T 100 GL7d12
Alg 1 Alg 1 Alg 1
tol tol tol
10 4 10 4 10 4
Residual

Residual
Residual
10 8 10 8 10 8

10 12 10 12 10 12

10 16 0 1 2 10 16 0 1 2 3 10 16 0 1 2 3
Iterations Iterations Iterations
100 3elt_dual 100 fv1 100 shuttle_eddy
Alg 1 Alg 1 Alg 1
tol tol tol
10 4 10 4 10 4
Residual

Residual

Residual
10 8 10 8 10 8

10 12 10 12 10 12

10 16 0 1 2 3 10 16 0 1 2 3 10 16 0 1
Iterations Iterations Iterations
100 nopoly flower_5_4T 100 barth5
Alg 1
100 Alg 1
Alg 1
tol tol tol
10 4 10 4 10 4
Residual

Residual
Residual

10 8 10 8 10 8

10 12 10 12 10 12

10 16 0 1 2 3 10 16 0 1 2 3 10 16 0 1 2 3 4 5
Iterations Iterations Iterations
100 L-9 100 crack_dual 100 rel8
Alg 1 Alg 1 Alg 1
tol tol tol
10 4 10 4 10 4
Residual

Residual

Residual

10 8 10 8 10 8

10 12 10 12 10 12

10 16 0 1 2 3 10 16 0 1 2 3 4 5 6 7 8 10 16 0 1 2
Iterations Iterations Iterations

Figure 2: Convergence history for SVD experiments.

5.3 Comparison with the Jacobi–Davidson algorithm

To illustrate the effectiveness and efficiency of our algorithm, we test our examples with another
frequently used general-purpose interior eigensolver—the Jacobi–Davidson (JD) algorithm [18].
In Table 2, we list the number of converged (generalized) singular values by the Jacobi–
Davidson algorithm, as well as the execution time. Execution time of Algorithm 1 or 2 is also
listed for reference. To avoid abuse computational resources, we have set the time limit to half
an hour for each test. Within this time limit, the Jacobi–Davidson algorithm can only solve five
relatively small problems. For all test cases, Algorithm 1 or 2 is much faster compared to the
Jacobi–Davidson algorithm, mainly because Algorithm 1 or 2 converges very rapidly. Moreover,
Algorithms 1 and 2 can effectively make use of sparse direct solvers for solving shifted linear
systems, making these algorithms practically attractive for medium-scale problems.

15
 100 plat1919 100 rosen10T 100 GL7d12
Alg. 2 Alg. 2 Alg. 2
tol tol tol
10 4 10 4 10 4
Residual

Residual

Residual
10 8 10 8 10 8

10 12 10 12 10 12

10 16 0 1 2 10 16 0 1 2 3 4 10 16 0 1 2
Iterations Iterations Iterations
100 3elt_dual 100 fv1 100 shuttle_eddy
Alg. 2 Alg. 2 Alg. 2
tol tol tol
10 4 10 4 10 4
Residual

Residual

Residual
10 8 10 8 10 8

10 12 10 12 10 12

10 16 0 1 2 3 10 16 0 1 2 3 10 16 0 1
Iterations Iterations Iterations
100 nopoly 100 flower_5_4T 100 barth5
Alg. 2 Alg. 2 Alg. 2
tol tol tol
10 4 10 4 10 4
Residual

Residual

Residual
10 8 10 8 10 8

10 12 10 12 10 12

10 16 0 1 2 3 10 16 0 1 2 3 10 16 0 1 2 3
Iterations Iterations Iterations
100 L-9 100 crack_dual 100 rel8
Alg. 2 Alg. 2 Alg. 2
tol tol tol
10 4 10 4 10 4
Residual

Residual

Residual

10 8 10 8 10 8

10 12 10 12 10 12

10 16 0 1 2 3 10 16 0 1 2 3 4 10 16 0 1 2 3
Iterations Iterations Iterations

Figure 3: Convergence history for GSVD experiments.

5.4 Comparison on spectral projectors

In the following we compare several choices of spectral projectors discussed in Section 3.2. We
+
apply four different filters, P + , PRR , P + + P − and P + &P − , as explained in Table 3, to compute
the GSVD of GL7d12. In order to make the difference easily visible, we replace the ellipse with a
circle and reduce the number of quadrature nodes to N = 8 in this test to slow down convergence.
With a randomly generated initial guess, P + &P − demonstrates the best convergence rate,
and the other three behave similarly; see Figure 4(a). This is consistent with our discussions in
Section 3.2.
We also construct two artificial initial guesses
√
  
U
, Q(m+n)×(ℓ−k) · Qℓ×ℓ + 10−12 m · Q(m+n)×ℓ , (25)
−W

16
Table 2: The number of converged (generalized) singular values computed by the Jacobi–
Davidson algorithm, together with the corresponding execution time (sec). Numbers in boldface
means that all (generalized) singular values have converged within the time limit—half an hour.
ID 1 2 3 4 5 6
desired 8 12 17 368 89 6
converged 8 12 17 83 89 6
SVD
iter. (JD) 47 57 74 2836 853 22
time (JD) 15.74 14.46 62.73 1800+ 1127 23.92
time (Alg. 1) 2.31 1.76 7.47 11.42 10.66 4.58
desired 5 17 19 251 56 2
converged 5 17 19 56 56 2
GSVD
iter. (JD) 31 98 113 840 451 13
time (JD) 13.04 26.49 110.7 1800+ 605.2 17.82
time (Alg. 2) 1.67 2.56 5.42 17.22 10.47 4.86
ID 7 8 9 10 11 12
desired 340 137 384 477 330 13
converged 141 17 149 140 125 0
SVD
iter. (JD) 16981 130 1512 1325 1174 9
time (JD) 1800+ 1800+ 1800+ 1800+ 1800+ 1800+
time (Alg. 1) 16.65 52.64 28.12 31.55 46.13 850.1
desired 159 51 317 607 602 185
converged 86 20 58 54 41 0
GSVD
iter. (JD) 748 114 453 783 362 3
time (JD) 1800+ 1800+ 1800+ 1800+ 1800+ 1800+
time (Alg. 2) 23.56 51.44 38.80 54.86 72.20 1474.5

Table 3: Four choices of spectral projectors.

Symbol Meaning
P+ +
the simple projector on Γ (see Section 3.2.1)
+
PRR P + with a Rayleigh–Ritz projection (see Section 3.2.2)
P + P−
+
the projector combining Γ+ and Γ− (see Section 3.2.3)
P + &P − the augmented projector combining Γ+ and Γ− (see Section 3.2.4)

and
√
    
U −10 U
+ (1 − 10 ) , Q(m+n)×(ℓ−k) · Qℓ×ℓ + 10−12 m · Q(m+n)×ℓ , (26)
W −W
where [U ∗ , W ∗ ]∗ is the desired solution, and the Q’s are randomly generated matrices with or-
thonormal columns. These initial guesses already contain useful information of the true solution.
+
Figure 4(b) illustrates that P + behaves poorly if (25) is used; Figure 4(c) illustrates that PRR
+ −
and P + P can also converge slowly on (26). These examples support our preference on
P + &P − .
In Section 3.2.4, we have mentioned that P + &P − can preserve a higher convergence rate for
several steps even if the trial subspace is only augmented in the first iteration. Such an observa-
tion is also illustrated in Figure 4(a)—the quick convergence of its first iteration is inherited by
a few subsequent iterations. Extra cost by augmenting the trial subspace in the first iteration is
compensated by rapid convergence.

17
 100 GL7d12 100 GL7d12 100 GL7d12
P+ P+ P+
P+RR P+RR P+RR
10 4
P+ + P 10 4
P+ + P 10 4
P+ + P
P+&P P+&P P+&P
Residual

Residual

Residual
10 8 tol 10 8 tol 10 8 tol

10 12 10 12 10 12

10 16 0 1 2 3 4 5 6 7 8 9 10 10 16 0 1 2 3 4 5 6 7 10 16 0 1 2 3 4 5 6 7
Iterations Iterations Iterations
(a) (b) (c)

Figure 4: Comparison on four spectral projectors applied to compute the GSVD of GL7d12:
(a) using random initial guess; (b) using artificial initial guess (25). (c) using artificial initial
guess (26).

5.5 Iterative refinement

We have seen that Algorithms 1 and 2 typically converge very rapidly. These algorithms can nat-
urally be adopted to refine low-precision solutions produced by other eigensolvers. To illustrate
this, we generate a few artificial initial guesses of the form
√
  
U
, Q(m+n)×(ℓ−k) · Qℓ×ℓ + 10−q m · Q(m+n)×ℓ , (27)
W

where [U ∗ , W ∗ ]∗ is the desired solution, and the Q’s are randomly generated matrices with
orthonormal columns. The parameter q is chosen from {2, 4, 6, 8, 10, 12} to control the quality of
the initial guess.
We choose the matrix plat1919 and set the desired interval to (10−4 , 10−3 ), which contains 52
generalized singular values. This problem becomes relatively ill-conditioned as the desired gen-
eralized singular values are small. From Figure 5(a), we see that Algorithm 2 can refine the
solution in two iterations when q ≥ 6. In fact, it is possible to skip the second iteration because
in this setting the number of desired generalized singular values, kGSVD , is already known, so
that the additional stopping criterion in Section 5.1 can be avoided. It is also possible to simply
use the simple spectral projector P + and skip trace estimation, as the quality of the initial guess
is known to be good.
A more practical setting is to use the solution of MATLAB’s eigs(A∗ A, B ∗ B) as the initial
guess. We see from Figure 5(b) that the accuracy is improved from about 10−7 to 10−13 in a
single iteration.

6 Conclusions
In this work we propose a contour integral-based algorithm for computing partial SVD/GSVD
through the Jordan–Wielandt matrix (pencil). This is a special case of the symmetric eigenvalue
problem, targeting interior eigenvalues. We analyze four choices of spectral projectors tailored
to this problem and identify one spectral projector that is both robust and effective. Numerical
experiments illustrates that our proposed algorithm can compute partial SVD/GSVD efficiently
and accurately.
There are several potential usages of our algorithm. When a large number of (generalized)
singular values are of interest, our algorithm can be incorporated into a spectral slicing frame-
work. Our algorithm can also be adopted to improve low-precision solutions produced by other

18
 100 plat1919 100 plat1919
10 2 Alg. 2
10 4 tol
10 4
10 6 10 4
10 8
Residual

Residual
10 8 10 10
10 8
10 12
tol
10 12 10 12

10 16 0 1 2 3 4 5 10 16 0 1
Iterations Iterations
(a) (b)

Figure 5: Compute the generalized singular values of plat1919 in the interval (10−4 , 10−3 ) with
low-precision initial guesses by Algorithm 2: (a) using artificial initial guesses (27); (b) using
MATLAB’s eigs(A∗ A, B ∗ B) as the initial guess.

algorithms. This is a promising feature that can possibly be exploited in modern mixed precision
algorithms. Development in these directions are planned as our future work.

A Further discussions
In Section 3.2.4, we mentioned that augmenting the trial subspace with a pair of contours can
often accelerate the convergence of the FEAST-SVD solver (in fact, also for FEAST-GSVD),
and, in addition, such an acceleration can be inherited by subsequent iterations even if the trial
subspace is only augmented in the first iteration. In the following, we provide a brief explanation
for such a fast convergence.
Let us assume that in the generic case
 +
    C+in
Ũ Uin Uout Uin Uout  Cout

= − ,

W̃ W in W out −W in −W out
 C in
−
Cout

and  
    C1
Ũ Uin Uout Uin Uout 
C2  ,
P̃ + (Ǎ)

=
W̃ Win Wout −Win −Wout C3 
C4
+
where C1 ≈ Cin , ∥C2 ∥2 = O(ϵ1 ), ∥C3 ∥2 = O(ϵ2 ), ∥C4 ∥2 = O(ϵ3 ), with max{ϵ2 , ϵ3 } ≪ ϵ1 because
− −
the eigenvalues corresponding to Cin and Cout are much farther from the interval. Note that
     
Ũ I Ũ
P̃ − (A) = m P̃ + (A)
W̃ −Im −W̃
 
   D1
Im Uin Uout Uin Uout 
D2  ,

=
−Im Win Wout −Win −Wout D3 

D4

19
 −
where D1 ≈ Cin , ∥D2 ∥2 = O(ϵ1 ), ∥D3 ∥2 = O(ϵ2 ), ∥D4 ∥2 = O(ϵ3 ). Then after one step of
Algorithm 1 we obtain the coefficient matrix
   + −

C1 D1 Cin Cin
C2 D2  O(ϵ1 ) O(ϵ1 ) 2n×2ℓ
C3 D3  ≈ O(ϵ2 ) O(ϵ2 ) ∈ C . (28)
   

C4 D4 O(ϵ3 ) O(ϵ3 )
If, instead, we apply the simple spectral projector to a generic initial guess with 2ℓ columns, the
corresponding coefficient matrix becomes
   + 
C1 Cin
C2  O(ϵ1 ) 2n×2ℓ
 ≈
C3  O(ϵ2 ) ∈ C ,


C4 O(ϵ3 )

which is about equally good compared to (28). Since the FEAST algorithm is essentially subspace
iteration on the spectral projector, it remains explaining why one step of subspace iteration with
an augmented initial guess can accelerate subsequent iterations.
In the rest of this section, we consider an n × n Hermitian matrix A whose eigenvalues λ1 ,
. . ., λn are ordered such that
|λ1 | ≥ |λ2 | ≥ · · · ≥ |λk | > |λk+1 | ≥ · · · ≥ |λℓ | > |λℓ+1 | ≥ · · · ≥ |λn |.
Let U represent the matrix containing eigenvectors u1 , . . ., un . The eigenvalues of interest are
λ1 , . . ., λk . The initial guess contains ℓ columns. We assume that
|λ1 | = Θ(1), |λk | = Θ(1), |λℓ+1 | = O(ϵ). (29)
The spectral projector arising in the FEAST algorithm usually satisfies (29).
Convergence analysis of subspace iteration can be found in many text books; see, e.g., [34].
Classical results focus on the asymptotic convergence rate, while we are interested in convergence
at early stage. To this end we establish the following technical results.
Lemma 1. Let M ∈ Cm×m be Hermitian and positive semidefinite. Then
1
∥(Im + M )−1/2 − Im ∥2 ≤ ∥M ∥2 .
2
Proof. Let µ1 , µ2 , . . ., µm be the eigenvalues of M . Then
∥(Im + M )−1/2 − Im ∥2 = max (1 + µi )−1/2 − 1
1≤i≤m
µi
= max
1≤i≤m (1 + µi )1/2 + (1 + µi )
1
≤ max µi
2 1≤i≤m
1
= ∥M ∥2 .
2
Theorem 2. Let A ∈ Cn×n be a nonsingular Hermitian matrix with normalized eigenpairs
(λ1 , u1 ), (λ2 , u2 ), . . ., (λn , un ). Assume that |λ1 | ≥ |λ2 | ≥ · · · ≥ |λn | > 0. A matrix X ∈ Cn×ℓ
is of the form  
⊥ X1
X = [Uℓ , Uℓ ] ,
X2

20
where Uℓ = [u1 , . . . , uℓ ], Uℓ⊥ = [uℓ+1 , . . . , un ], and X1 ∈ Cℓ×ℓ is nonsingular. Let X3 = X2 X1−1
be partitioned into X3 = [X3,1 , X3,2 ], where X3,1 ∈ C(n−ℓ)×k . Then there exists Y ∈ Cn×ℓ such
that span(Y ) = span(AX), Y ∗ Y = Iℓ , and

 k ℓ − k
E1,1 E1,2 k
Y = Uℓ + [Uk , Uℓ\k , Uℓ⊥ ]E2,1 E2,2  ℓ − k ,
F1 F2 n−ℓ

for 1 ≤ k ≤ ℓ, where Uk = [u1 , . . . , uk ] and Uℓ\k = [uk+1 , . . . , uℓ ]. Define

−1 −1
η̃ = Λ⊥
ℓ X3,1 Λk 2
, η̂ = Λ⊥
ℓ X3,2 Λℓ\k 2
,

where Λk = diag{λ1 , . . . , λk }, Λℓ\k = diag{λk+1 , . . . , λℓ } and Λ⊥

ℓ = diag{λℓ+1 , . . . , λn }. Then
we have
1 2
∥E1,1 ∥2 ≤ η̃ , ∥E2,1 ∥2 = 0, ∥F1 ∥2 ≤ η̃,
2
1 2
∥E1,2 ∥2 ≤ η̃ η̂, ∥E2,2 ∥2 ≤ η̂ , ∥F2 ∥2 ≤ η̂.
2
Proof. Let Λ = diag{Λk , Λℓ\k , Λ⊥
ℓ }. We express AX as
   
X1 Iℓ
AX = A[Uℓ , Uℓ⊥ ] = [Uℓ , Uℓ⊥ ]Λ X1 = [Uℓ , Uℓ⊥ ]BΛℓ X1 ,
X2 X2 X1−1

where  
Iℓ
B= −1 . (30)
Λ⊥
ℓ X3 Λℓ

Partition B into [Bk , Bℓ\k̃ ], where Bk is n × k. Let Mk = Λ−1 ∗ ⊥ 2 −1

k X3,1 (Λk ) X3,1 Λk , E1,1 =
(Bk∗ Bk )−1/2 − Ik . The Bk can be orthonormalized through
   
Ik Ik
Qk = Bk (Bk∗ Bk )−1/2 =   Ik + Mk −1/2 ,


0  (Ik + E1,1 ) =  0
−1 −1
Λ⊥ℓ X3,1 Λk Λ⊥
ℓ X3,1 Λk

where ∥E1,1 ∥2 can be bounded by Lemma 1 as

1 1
∥E1,1 ∥2 ≤ ∥Mk ∥2 ≤ η̃ 2 .
2 2
Let
−1
F1 = Λ⊥
ℓ X3,1 Λk (Ik + E1,1 ).
∗
Notice that [Ik + E1,1 , 0, F1∗ ]∗ has orthonormal columns. By CS decomposition we obtain

1/2
∥F1 ∥2 = 1 − σmin (Ik + E1,1 )2


1/2
= 1 − σmin (Ik + Mk )−1

1/2 (31)
= 1 − ∥Ik + Mk ∥−1
2
1/2
= ∥Mk ∥2 (1 + ∥Mk ∥2 )−1/2
≤ η̃.

21
We then orthonormalize Bℓ\k against Qk through
−1 
−(Ik + E1,1 )F1∗ Λ⊥
ℓ X3,2 Λℓ\k


Cℓ\k = (In − Qk Q∗k )Bℓ\k = Iℓ−k 

∗ ⊥ −1
(In−ℓ − F1 F1 )Λℓ X3,2 Λℓ\k

and
−1/2
∗
Qℓ\k = Cℓ\k (Cℓ\k Cℓ\k )−1/2 = Cℓ\k Iℓ−k + Mℓ\k = Cℓ\k (Iℓ−k + E2,2 ),
where
∗
Mℓ\k = Cℓ\k Cℓ\k − Iℓ−k = Λ−1 ∗ ⊥ ∗ ⊥ −1
ℓ\k X3,2 Λℓ (In−ℓ − F1 F1 )Λℓ X3,2 Λℓ\k
∗
and E2,2 = (Cℓ\k Cℓ\k )−1/2 − Iℓ−k . By Lemma 1 we have

1 1 −1 2 1 2
∥E2,2 ∥2 ≤ ∥Mℓ\k ∥2 ≤ ∥In−ℓ − F1 F1∗ ∥2 ∥Λ⊥
ℓ X3,2 Λℓ\k ∥2 ≤ η̂ .
2 2 2
Let Q = [Qk , Qℓ\k ] and
 
E1,1 E1,2
Y = [Uk , Uℓ\k , Uℓ⊥ ]Q = Uℓ + [Uk , Uℓ\k , Uℓ⊥ ]  0 E2,2  ,
F1 F2

where E1,2 and F2 satisfy    

E1,2 0
E2,2  = Qℓ\k − Iℓ−k  .
F2 0
Then Y ∗ Y = Iℓ and span(Y ) = span(AX). Notice that [E1,2
∗ ∗
, Iℓ−k + E2,2 , F2∗ ]∗ has orthonormal
columns. Similar to the proof of (31), we obtain
 
E1,2 1/2
∥F2 ∥2 ≤ = ∥Mℓ\k ∥2 (1 + ∥Mℓ\k ∥2 )−1/2 ≤ η̂
F2 2

and
−1 −1
∥E1,2 ∥2 ≤ ∥Ik + E1,1 ∥2 ∥F1 ∥2 ∥Λ⊥ ⊥
ℓ X3,2 Λℓ\k ∥2 ∥Iℓ−k + E2,2 ∥2 ≤ ∥F1 ∥2 ∥Λℓ X3,2 Λℓ\k ∥2 ≤ η̃ η̂.

Theorem 3. Let H = Λ + ∆H ∈ Cℓ×ℓ be a Hermitian matrix with spectral decomposition

H = QΘQ∗ , where Λ and Θ are real diagonal matrices, and Q is unitary. Partition Λ, Θ
and ∆H into Λ = diag{Λk , Λℓ\k }, Θ = diag{Θk , Θℓ\k }, and ∆H = [∆H1 , ∆H2 ], where Λk ,
Θk ∈ Rk×k and ∆H1 ∈ Cℓ×k . Suppose

spec(Λk ) ⊂ [α, β], spec(Θℓ\k ) ⊂ R\(α − δ, β + δ),

where δ > 0. Then there exist unitary matrices Q1 ∈ Ck×k and Q2 ∈ C(ℓ−k)×(ℓ−k) satisfying
(
ϵ2 , i = j,
   
Q1 ∆1,1 ∆1,2
Q= + , ∥∆ij ∥2 ≤
Q2 ∆2,1 ∆2,2 ϵ, i ̸= j,

in which ϵ = ∥∆H1 ∥2 /δ.

22
Proof. Partition Q into  
Q1,1 Q1,2
Q= ,
Q2,1 Q2,2
where Q1,1 ∈ Ck×k . According to the Davis–Kahan sin θ theorem [7], it can be verified that

∥∆H1 ∥2
∥Q1,2 ∥2 ≤ = ϵ.
δ
Notice that ∥Q1,2 ∥2 = ∥Q2,1 ∥2 . Using [17, Lemma 5.1], we know that there is a unitary matrix
Q1 ∈ Ck×k satisfying

∥Q1,1 − Q1 ∥2 ≤ ∥Ik − Q∗1,1 Q1,1 ∥2 = ∥Q∗2,1 Q2,1 ∥2 ≤ ϵ2 .

Similarly, there exists a unitary matrix Q2 ∈ C(ℓ−k)×(ℓ−k) such that

∥Q2,2 − Q2 ∥2 ≤ ∥Iℓ−k − Q∗2,2 Q2,2 ∥2 = ∥Q∗1,2 Q1,2 ∥2 ≤ ϵ2 .

Setting    
∆1,1 ∆1,2 Q1
=Q−
∆2,1 ∆2,2 Q2
yields the conclusion.

Theorem 4. Let A be Hermitian with spectral decomposition A = U ΛU ∗ . Suppose

   
Xk Xk
X = U Xℓ\k  = [Uk , Uℓ\k , Uℓ⊥ ] Xℓ\k  ∈ Cn×k
Xt⊥ Xt⊥

has full column rank, where Xk ∈ Ck×k , Xℓ\k ∈ C(ℓ−k)×k . Then

|λℓ+1 | Λℓ\k Xℓ\k Xk−1 2

tan ∠(Uk , AX) ≤ tan ∠(Uk , X) + . (32)
|λk | |λk |

Proof. We have
Xℓ\k Xk−1
   
Xℓ\k
tan ∠(Uk , X) = = Xk−1
Xℓ⊥ Xk−1 2
Xℓ⊥ 2

because    
Xk Ik
X = U Xℓ\k  = U Xℓ\k Xk−1  Xk .
Xℓ⊥ Xℓ⊥ Xk−1
Applying A to X yields
   
Xk Ik
AX = U Λ Xℓ\k  = U Λℓ\k Xℓ\k Xk−1 Λ−1
k
 Λk Xk
Xℓ⊥ ⊥ −1 −1
Λ⊥
ℓ X ℓ X k Λ k

and
Λℓ\k Xℓ\k Xk−1 Λ−1
   
1 Λℓ\k Xℓ\k
tan ∠(Uk , AX) = ⊥ −1 −1
k ≤ Xk−1 .
Λ⊥
ℓ Xℓ Xk Λk 2
|λk | Λ⊥
ℓ Xℓ
⊥
2

23
Notice that
 
Λℓ\k Xℓ\k 2 1/2
 
2
Xk−1 ≤ Λℓ\k Xℓ\k Xk−1 + Λ⊥ ⊥ −1
ℓ Xℓ Xk
Λ⊥
ℓ Xℓ
⊥
2
2 2

2 1/2
  !
2 Xℓ\k
≤ Λℓ\k Xℓ\k Xk−1 2 + λ2ℓ+1 −1
Xk
Xℓ⊥ 2
 
Xℓ\k
≤ Λℓ\k Xℓ\k Xk−1 + |λℓ+1 | Xk−1 .
2 Xℓ⊥ 2

Therefore, we have

|λℓ+1 | Λℓ\k Xℓ\k Xk−1 2

tan ∠(Uk , AX) ≤ tan ∠(Uk , X) + .
|λk | |λk |

We aim at computing k leading eigenvalues of A. First, we act A on the initial matrix

X ∈ Cn×ℓ , where ℓ > k. According to Theorem 2, we know that there exists Y = Uℓ (I + E) +
Uℓ⊥ F ∈ Cn×ℓ such that Y ∗ Y = Iℓ and span(Y ) = span(AX), where
 
E1,1 E1,2  
E= , F = F1 F2 .
E2,1 E2,2

The norm of each block of E and F can also be estimated by Theorem 2.

The projected matrix Y ∗ AY can be written as
 ∗  
∗ ∗ E E
Y AY = Λℓ + E Λℓ + Λℓ E + Λ = Λℓ + ∆H,
F F

i.e., Y ∗ AY can be regarded as a diagonal matrix Λℓ with a Hermitian perturbation ∆H. The
leading k columns of ∆H, denoted as ∆H1 , is bounded through
 ∗   ∗ ∗
Λℓ\k E2,1 + F1∗ Λ⊥
  
E1,1 Λk Λk E1,1 E1,1 Λk E1,1 + E2,1 ℓ F1
∥∆H1 ∥2 = ∗ + + ∗ ∗
E1,2 Λk Λk E2,1 E1,2 Λk E1,1 + E2,2 Λℓ\k E2,1 + F2∗ Λ⊥
ℓ F1 2
≤ |λ1 |(2∥E1,1 ∥2 + ∥E1,2 ∥2 + ∥E1,1 ∥22 + ∥E1,1 ∥2 ∥E1,2 ∥2 ) + |λℓ+1 |(∥F1 ∥22 + ∥F1 ∥2 ∥F2 ∥2 )


= O |λ1 | · ∥E1,2 ∥2 + |λℓ+1 | · ∥F1 ∥2 ∥F2 ∥2 .

Suppose that the conditions of Theorem 3 hold, and the upper bound of value ϵ is

∥∆H1 ∥2 |λ1 |
ϵ= ≤ · O(η̂ η̃).
δ δ
After the Rayleigh–Ritz projection Y ∗ AY = QΘQ∗ , the approximate eigenvectors X become

X = Y Q = Uℓ (Iℓ + E)Q + Uℓ⊥ F Q. (33)

Then we have  
(Ik + E1,1 )(Q1 + ∆1,1 ) + E1,2 ∆2,1 ∗
(Iℓ + E)Q = ,
E2,1 (Q1 + ∆1,1 ) + (Iℓ−k + E2,2 )∆2,1 ∗
and  
F Q = F1 (Q1 + ∆1,1 ) + F2 ∆2,1 ∗ ,

24
where

∥(Ik + E1,1 )(Q1 + ∆1,1 ) + E1,2 ∆2,1 − Q1 ∥2 ≤ ∥E1,1 ∥2 + ∥E1,2 ∥2 ∥∆2,1 ∥2 = O(η̃ 2 ), (34)
 
|λ1 |
∥E2,1 (Q1 + ∆1,1 ) + (Iℓ−k + E2,2 )∆2,1 ∥2 = ∥(Iℓ−k + E2,2 )∆2,1 ∥2 ≤ ∥∆2,1 ∥2 = O η̂ η̃ ,
δ
(35)
and

∥F1 (Q1 + ∆1,1 ) + F2 ∆2,1 ∥2 ≤ ∥F1 ∥2 + ∥F2 ∥2 ∥∆21 ∥2 = O(η̃). (36)

Since the diagonal entries of Λℓ\k decay, η̂ is usually much smaller than the most pessimistic
upper bound |λℓ+1 |/|λℓ | · ∥X32 ∥2 . Then (36) is typically larger than (35) when |λ1 |/δ = O(1).
We remark that |λ1 |/δ = O(1) is not unusual in our original setting of contour integral-based
solvers, as the gap δ depends on the distance from the contour to the closest unwanted eigenvalue.
Next, we update X by keeping its leading k columns and dropping the trailing ones. Then
 
Xk
X = [Uk , Uℓ\k , Uℓ⊥ ] Xℓ\k  ,
Xℓ⊥

where Xℓ\k and Xℓ⊥ are bounded by (35) and (36), respectively. By (32) in Theorem 4, we have

tan ∠(Uk , AX) |λℓ+1 | ∥Λℓ\k Xℓ\k ∥2

≤ + κ(Xk ) , (37)
tan ∠(Uk , X) |λk | |λk |∥Xℓ⊥ ∥2

We conclude that κ(Xk ) = Θ(1) according to (34), and ∥Xℓ⊥ ∥2 is in general larger than ∥Xℓ\k ∥2 .
In practice, |λk |∥Xℓ⊥ ∥2 is even much larger than ∥Λℓ\k Xℓ\k ∥2 , because the diagonal entries of Λℓ\k
decay rapidly. In fact, from numerical experiments we observe that |λℓ+1 |/|λk | dominates the
right-hand-side of (37). Therefore, the local convergence rate is roughly equal to |λℓ+1 |/|λk | =
O(ϵ). After a few iterations, the local convergence rate gradually deteriorates, and eventually
returns to the asymptotic convergence rate |λk+1 |/|λk |.

Acknowledgments
The authors thank Zhaojun Bai, Weiguo Gao, Zhongxiao Jia, Daniel Kressner, Yingzhou Li, and
Jose Roman for helpful discussions.

References
[1] Orly Alter, Patrick O. Brown, and David Botstein. Generalized singular value decomposition
for comparative analysis of genome-scale expression data sets of two different organisms.
Proc. Natl. Acad. Sci., 100(6):3351–3356, 2003. doi:10.1073/pnas.0530258100.
[2] Junko Asakura, Hiroto Sakurai, Tetsuya Tadano, Tsutomu Ikegami, and Kinji Kimura.
A numerical method for polynomial eigenvalue problems using contour integral. Japan J.
Indust. Appl. Math., 27(1):73–90, 2010. doi:10.1007/s13160-010-0005-x.
[3] Haim Avron and Sivan Toledo. Randomized algorithms for estimating the trace of an
implicit symmetric positive semi-definite matrix. J. ACM, 58(2):1–34, 2011. doi:10.1145/
1944345.1944349.

25
 [4] Zhaojun Bai and James W. Demmel. Computing the generalized singular decomposition.
SIAM J. Sci. Comput., 14(6):1464–1486, 1993. doi:10.1137/0914085.
[5] Zhaojun Bai and Hongyuan Zha. A new preprocessing algorithm for the computation of the
generalized singular value decomposition. SIAM J. Sci. Comput., 14(4):1007–1012, 1993.
doi:10.1137/0914060.

[6] K. Bhuyan, S. B. Singh, and P. K. Bhuyan. Application of generalized singular value

decomposition to ionospheric tomography. Ann. Geophys., 22(10):3437–3444, 2004. doi:
10.5194/angeo-22-3437-2004.
[7] Chandler Davis and W. M. Kahan. The rotation of eigenvectors by a perturbation. III.
SIAM J. Numer. Anal., 7(1):1–46, 1970. doi:10.1137/0707001.
[8] Timothy A. Davis and Yifan Hu. The university of Florida sparse matrix collection. ACM
Trans. Math. Software, 38(1):1:1–1:25, 2011. doi:10.1145/2049662.2049663.
[9] Edoardo Di Napoli, Eric Polizzi, and Yousef Saad. Efficient estimation of eigenvalue counts
in an interval. Numer. Linear Algebra Appl., 23(4):674–692, 2016. doi:10.1002/nla.2048.

[10] L. Magnus Ewerbring and Franklin T. Luk. Canonical correlations and generalized SVD:
applications and new algorithms. J. Comput. Appl. Math., 27(1):37–52, 1989. doi:10.
1016/0377-0427(89)90360-9.
[11] Yasunori Futamura, Tetsuya Sakurai, Shinnosuke Furuya, and Jun-Ichi Iwata. Effi-
cient algorithm for linear systems arising in solutions of eigenproblems and its applica-
tion to electronic-structure calculations. In Michel Daydé, Osni Marques, and Kengo
Nakajima, editors, High Performance Computing for Computational Science — VEC-
PAR 2012, volume 7851 of Lect. Notes in Comput. Sci., pages 226–235, 2013. doi:
10.1007/978-3-642-38718-0_23.

[12] Yasunori Futamura, Hiroto Tadano, and Tetsuya Sakurai. Parallel stochastic estimation
method of eigenvalue distribution. JSIAM Lett., 2:127–130, 2010. doi:10.14495/jsiaml.
2.127.
[13] Shuai Gao, Zhengchun Du, and Yujun Li. An improved contour-integral algorithm for
calculating critical eigenvalues of power systems based on accurate number counting. IEEE
Trans. Power Syst., 38(1):549–558, 2022. doi:10.1109/TPWRS.2022.3159494.
[14] A. Girard. A fast ‘Monte-Carlo cross-validation’ procedure for large least squares problems
with noisy data. Numer. Math., 56(1):1–23, 1989. doi:10.1007/BF01395775.
[15] Stefan Güttel, Eric Polizzi, Ping Tak Peter Tang, and Gautier Viaud. Zolotarev quadrature
rules and load balancing for the FEAST eigensolver. SIAM J. Sci. Comput., 37(4):A2100–
A2122, 2015. doi:10.1137/140980090.
[16] Per Christian Hansen. Regularization, GSVD and truncated GSVD. BIT, 29:491–504, 1989.
doi:10.1007/BF02219234.
[17] Nicholas J. Higham. The matrix sign decomposition and its relation to the polar decompo-
sition. Linear Algebra Appl., 212/213:3–20, 1994. doi:10.1016/0024-3795(94)90393-X.
[18] M. E. Hochstenbach. A Jacobi–Davidson type method for the generalized singular value
problem. Linear Algebra Appl., 431(3–4):471–487, 2009. doi:10.1016/j.laa.2009.03.003.

26
[19] Peg Howland, Moongu Jeon, and Haesun Park. Structure preserving dimension reduction
for clustered text data based on the generalized singular value decomposition. SIAM J.
Matrix Anal. Appl., 25(1):165–179, 2003. doi:10.1137/S0895479801393666.
[20] Jinzhi Huang and Zhongxiao Jia. On choices of formulations of computing the generalized
singular value decomposition of a large matrix pair. Numer. Algorithms, 87:689–718, 2021.
doi:10.1007/s11075-020-00984-9.
[21] Jinzhi Huang and Zhongxiao Jia. A cross-product free Jacobi–Davidson type method for
computing a partial generalized singular value decomposition of a large matrix pair. J. Sci.
Comput., 94:3, 2023. doi:10.1007/s10915-022-02053-w.

[22] Zhongxiao Jia and Kailiang Zhang. A FEAST SVDsolver based on Chebyshev–Jackson
series for computing partial singular triplets of large matrices. J. Sci. Comput., 97(1):21:1–
21:36, 2023. doi:10.1007/s10915-023-02342-y.
[23] Bo Kågström. The generalized singular value decomposition and the general (A − λB)-
problem. BIT, 24:568–583, 1984. doi:10.1007/BF01934915.

[24] A. V. Knyazev, M. E. Argentati, I. Lashuk, and E. E. Ovtchinnikov. Block locally optimal

preconditioned eigenvalue xolvers (BLOPEX) in hypre and PETSc. SIAM J. Sci. Comput.,
29(5):2224–2239, 2007. doi:10.1137/060661624.
[25] Andrew V. Knyazev. Toward the optimal preconditioned eigensolver: Locally optimal block
preconditioned conjugate gradient method. SIAM J. Sci. Comput., 23(2):517–541, 2001.
doi:10.1137/S1064827500366124.
[26] S. R. Kuo, W. Yeih, and Y. C. Wu. Applications of the generalized singular-value decom-
position method on the eigenproblem using the incomplete boundary element formulation.
J. Sound Vib., 235(5):813–845, 2000. doi:10.1006/jsvi.2000.2946.

[27] Keisuke Nakamura, Kazuhiro Nakadai, and Gökhan Ince. Real-time super-resolution sound
source localization for robots. In Proceedings of the IEEE/RSJ International Conference on
Intelligent Robots and Systems, pages 694–699, 2012. doi:10.1109/IROS.2012.6385494.
[28] Hiroshi Ohno, Yoshinobu Kuramashi, Tetsuya Sakurai, and Hiroto Tadano. A quadrature-
based eigensolver with a Krylov subspace method for shifted linear systems for Hermitian
eigenproblems in lattice QCD. JSIAM Lett., 2:115–118, 2010. doi:10.14495/jsiaml.2.
115.
[29] C. C. Paige. The general linear model and the generalized singular value decomposition.
Linear Algebra Appl., 70:269–284, 1985. doi:10.1016/0024-3795(85)90059-X.
[30] C. C. Paige. Computing the generalized singular value decomposition. SIAM J. Sci. Com-
put., 7(4):1126–1146, 1986. doi:10.1137/0907077.
[31] C. C. Paige and M. A. Saunders. Towards a generalized singular value decomposition. SIAM
J. Numer. Anal., 18(3):398–405, 1981. doi:10.1137/0718026.
[32] Cheong Hee Park and Haesun Park. A relationship between linear discriminant analysis and
the generalized minimum squared error solution. SIAM J. Matrix Anal. Appl., 27(2):474–
492, 2005. doi:10.1137/040607599.

27
[33] Haesun Park and L. Magnus Ewerbring. An algorithm for the generalized singular value
decomposition on massively parallel computers. J. Parallel Distrib. Comput., 17(4):267–276,
1993. doi:10.1006/jpdc.1993.1026.
[34] Beresford N. Parlett. The Symmetric Eigenvalue Problem. SIAM, Philadelphia, PA, USA,
1998. doi:10.1137/1.9781611971163.
[35] Eric Polizzi. Density-matrix-based algorithms for solving eigenvalue problems. Phys. Rev. B,
79:115112, 2009. doi:10.1103/physrevb.79.115112.
[36] Tetsuya Sakurai, Yasunori Futamura, and Hiroto Tadano. Efficient parameter estima-
tion and implementation of a contour integral-based eigensolver, 2013. doi:10.1260/
1748-3018.7.3.249.
[37] Tetsuya Sakurai and Hiroshi Sugiura. A projection method for generalized eigenvalue
problems using numerical integration. J. Comput. Appl. Math., 159(1):119–128, 2003.
doi:10.1016/S0377-0427(03)00565-X.
[38] Tetsuya Sakurai and Hiroshi Sugiura. CIRR: a Rayleigh–Ritz type method with contour
integral for generalized eigenvalue problems. Hokkaido Math. J., 36(4):745–757, 2007. doi:
10.14492/hokmj/1272848031.
[39] J. M. Speiser and C. F. Van Loan. Signal processing computations using the generalized
singular value decomposition. In Proceedings of the SPIE, volume 495, pages 47–55, 1984.
[40] Brian D. Sutton. Stable computation of the CS decomposition: Simultaneous bidiagonal-
ization. SIAM J. Matrix Anal. Appl., 33(1):1–21, 2012. doi:10.1137/100813002.
[41] Ping Tak Peter Tang and Eric Polizzi. FEAST as a subspace iteration eigensolver accelerated
by approximate spectral projection. SIAM J. Matrix Anal. Appl., 35(2):354–390, 2014.
doi:10.1137/13090866X.
[42] Lloyd N. Trefethen and J. A. C. Weideman. The exponentially convergent trapezoidal rule.
SIAM Rev., 56(3):385–458, 2014. doi:10.1137/130932132.
[43] Charles F. Van Loan. Generalizing the singular value decomposition. SIAM J. Numer.
Anal., 13(1):76–83, 1976. doi:10.1137/0713009.
[44] Charles F. Van Loan. Computing the CS and the generalized singular value decompositions.
Numer. Math., 46(4):479–491, 1985. doi:10.1007/BF01389653.
[45] Jan Winkelmann, Paul Springer, and Edoardo Di Napoli. ChASE: Chebyshev accelerated
subspace iteration eigensolver for sequences of Hermitian eigenvalue problems. ACM Trans.
Math. Software, 45(2):21:1–21:34, 2019. doi:10.1145/3313828.
[46] Xin Ye, Jianlin Xia, Raymond H. Chan, Stephen Cauley, and Venkataramanan Balakrish-
nan. A fast contour-integral eigensolver for non-Hermitian matrices. SIAM J. Matrix Anal.
Appl., 38(4):1268–1297, 2017. doi:10.1137/16m1086601.
[47] Shinnosuke Yokota and Tetsuya Sakurai. A projection method for nonlinear eigenvalue
problems using contour integrals. JSIAM Lett., 5:41–44, 2013. doi:10.14495/jsiaml.5.41.
[48] Hongyuan Zha. Computing the generalized singular values/vectors of large sparse or struc-
tured matrix pairs. Numer. Math., 72:391–417, 1996. doi:10.1007/s002110050175.

28