FAST ADAPTIVE EIGENVALUE DECOMPOSITION :

A MAXIMUM LIKELIHOOD APPROACH

Christian Riou Thierry Chonavel

ENSTB, S.C. Dpt., B.P. 832, 29285, Brest cedex, FRANCE

first name. surname@enst-bre t agne.fr

ABSTRACT its eigenvalue decomposition (e.g. [4, 31). In ([4]), some

simplifying assumptions (the space is splitted into only 2
A new adaptive subspace estimation algorithm is presented,
eigensubspaces, in view to obtain basis of the so-called noise
based on the maximisation of the likelihood functional. It
and signal subspaces) permit, to get an algorithm with com-
requires little computational cost and the particular struc-
putat,ional cost. in O(NIC). This algorithm makes use of
ture cif the algorithm ensures the orthonormality of the esti-
Householder reflections matrices. By means of a Givens
mated basis of eigenvectors. Application to moving sources
rotations-based algorithm, and other simplifying hypoth-
localization shows the very good behavior of the algorithm
esis, [3] presents an O(NI<) algorithm that ensures com-
when applied to problems of practical interest. plete eigenvalue decomposition, and perfect orthonormality
of the estimated eigenvectors.
1. INTRODUCTION Alternatively, another approach consists in using some
constrained or unconstrained criterion, the optimum of which
Let X = (X,),,, be a vector, complex valued, stochastic corresponds to the EVD of Rx,,, and optimize it by means
process ( X , E U?). In signal processing, the computation of some stochastic gradient-like algorithm [9,13, lo]. These
(or estimation) of the Eigenvalue Decomposition (EVD) of stochastic algorithms have various computational cost :
the hermitian covariance matrix Rx,, = E {X,HXn} has O ( N 2IC) for Regalia’s Givens rotations-based algorithm [9],
been intensively used in different applications. In partic- with exact, orthonormality of the estimated EVD, O ( N K 2 )
ular, Direction Of Arrival (DOA) estimation (MUSIC al- in [lo], where only approximate orthonormality is ensured,
gorithm, e.g. [ll]), frequencies estimation, blind channel as well as for Yang’s O ( N K ) RLS algorithm [13]. But for
impulse response estimation [SI or vector quantization [13], this last algorithm, an additional orthonormalieation proce-
are dlassical applications in which this problem arises. dure is often necessary to get satisfactory results in practice
If the process X is stationary, R x , =~ Rx can be esti- (at the expense of O ( N K * )additional operations).
mated easily, and there exist in the literature several itera- In this paper, a fast adaptive eigenvalue decomposition
tive methods to perform its EVD. Among these methods the is presented. It relies on the optimization of a log-likelihood
orthcgonal iteration, that requires a QR algorithm at each functional by means of a stochastic gradient algorithm, that
iteration is currently used [7]. Unfortunately, in many ap- incorporates the orthonormality constraint upon the eigen-
plications (moving sources, time varying transmission chan- vectors. Its coniputational cost is in the order of O ( N K ) .
nels,...), the process X under study is not stationary, and The paper is organized as follows. In section two, the
the a.bove approach cannot be applied directly. maximum likelihood approach is presented, and in section
To overcome this problem, several studies have been three, the algorithm is derived and its convergence proper-
carried out during the last decade. First, straightforward ties and asymptotic variance are studied. The last section
extensions of the stationary case to the non stationary case is devoted to simulations and comparisons with other meth-
have been proposed (e.g. [5]). They rely on adaptive es- ods.
timation of the instantaneous covariance matrix Rx,., by
means of a smoothing technique, yielding estimates such as
2. PROBLEM STATEMENT AND ADAPTIVE
MAXIMUM LIKELIHOOD
In Inany problems, the observed data model a t time n is in
where cr is a forgetting factor, 0 < cr < 1, and zn is the
the form
observed data vector at time n. Then, one iteration of the
classical orthogonal iteration EVD method is performed,
that makes use of the EVD performed at the previous step,
Xn = HnSn + Bn (2)
and (of the covariance matrix &,,. The computational cost where X, is a random N-vector, and S, a random p-vector
of suich an approach is in general O(NzII(),where I( is the (p < N), with full rank covariance matrix. B, is an additive
number of estimated eigen-vectors. noise N-vector uncorrelatecl with S,, and with covariance
T o reach lower computational cost, several authors pro- matrix is QXIN (IK is the K x IC identity matrix). H n is a
posed to replace in (1) the estimated covariance matrix by full rank, time-varying N x p matrix.

0-818#6-7919-0/97$10.00 0 1997 IEEE 3565

Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on September 06,2024 at 06:04:48 UTC from IEEE Xplore. Restrictions apply.
 The problem is to estimate H,,or some of its parame- Under t,he unitary constraint- U H U = I N , we have dUH =
ters of interest, out of the knowledge of the space spanned - U H d U U H , and first, order differential yields :
by its columns. Clearly, getting this vector space amounts
to estimate the space spanned by the eigenvectors asoci- +
drr@ = z ~ J & ? J A - ' U ~ U A - ' d U H ) x n
(7)
ated to the largest eigenvalues of Rx,,, = E[XnXf]. Also, = -yn (A-'UHdU - U H A - ' d U H )yn
the complete knowledge of the EVD of Rx,,, is often inter-
esting, for instance to estimate the possibly unknown range From this expression, the matrix , with general
of Hn (see e.g. [6, 121). term [8*(~;&~*~13 = &'*(Zn;A,U) is given by
&'U,,
Let R x , ~= UnAnU," denote the EVD of Rx,,. As- '83

suming that X , is a zero mean complex valued gaussian

random vector, the probabilty density function (p.d.f.) of
X , is parametrized by U and A, that represent the matrix
of eigenvecors and the diagonal matrix of the eigenvalues, This yields the following stochast,ic algorithm :
denoted ( X ~ ) ~ = I , N respectively.
, This p.d.f. is given by
H
yn = Un-lzn (9a)
An = An-1 -,U (Ai:' - 12;?1diag(Yny,X)) (9b)
(3) un = un-l(~+ (AiLlynYF -Y n Y F C 1 ) ) (9c)
Up to additive and multiplicative constants that, have been
dropped, the opposite of the corresponding log-likelihood is where cliag(ynyF) is a diagonal matrix with diagonal terms
given by matching those of y,~:. This algorithm has been bap-
tized MALASE (MAxirnuni Likelihood Adaptive Subspace
N Estimation). The coniputational cost of this algorithm is
@ ( x n ; AIU ) = C l o g (A,) + XFUA-' UHEnl (4) O(NK).
k=l

The minimization of the "distance" criterion Kn (A, U ) 3.2. Convergence properties

= E[@ (xn;A,U)] can be searched adaptively by means of To st.uc1y the st.at.ionary points (A, U ) of the algorithm, let
the following stochastic gradient algorithm : us recall t,liat. they satisfy the equat,ions :
H
Yn = Un-1x" ( 5a) hl (A) = A-' - A-'E[diag(YYH)] (10)
An = An-1 - / A (A,-L1 - A,-Z1diag(VnyF)) (5b) = A-' - A-2diag(UHRxU)
= o
un = Un-1-
-1 H
/AAn-lXnyn. (5c)
h z ( U ) = E {U (A-'YYH - YYwA-')} (11)
where diag(yny:) is a diagonal matrix with diagonal terms
matching those of yny:. Unfortunately the convergence = U ( A - ~ U ~ R-~PUR ~ U A - ~ )
properties of this algorithm are not satisfactory. Indeed, let = o
us consider for instance the equation of actualization of Un.
A stationary point ( A , U ) should satisfy E[A-'XXHU] = One can easily check that a necessary and sufficient condi-
A-'RxU = 0. Obviously, the EVD of R x does not match tion for A to be a stationary point, is that [A]i,i = UTRxU;,
this constraint. and that, the matrix UHRxU is a block diagonal matrix
where any block has all equal diagonal terms.
3. C O N S T R A I N E D A D A P T I V E M A X I M U M To st,udy the stability, we consider perturbations of the
LIKELIHOOD stationary point (A, U ) . More particularly, we consider mul-
tiplicat,ive perturbat,ions in the form of Givens rotations
To overcome the above mentionned problem, we will modify (U+dU = UG, where G is a Given's rotation), and this per-
the previous algorithm by accounting for the orthonormal- mits to show simply that, a necessary condition for stability
ity constraint upon the eigenvectors, in the computation of is that, UHRxU is a diagonal matrix. This shows that only
the gradient of @ ( x n ; A, U ) . We will show the good proper- eigenvalue decompositions may represent stable points.
ties of the corresponding algorithm. In particular, we will Furthermore, we account, for the important fact, proved
show that it produces exactly unitary estimates a t each it- in the following, that if U, is unitary, the matrix Un+l
eration. Moreover, we prove its convergence towards the is also unitary (up to a second order term in p ) . Thus,
EVD of RxPnand study the asymptotic variance of its es- since the stability of the the eigenvalue decomposition is
timate. ensured for Given's rotations multiplicative perturbations,
and since such matrices parametrize all matrix transfor-
3.1. Derivation of t h e M A L A S E algorithm mations on the sphere of unitary matrices, the stability of
eigenvalue decompositions is guaranteed. Thus, we have :
Let us consider the following differential of @ :
Proposition 1 the point (A, U ) is a stable point of the ol-
du@( x n ;A, U )= CP ( z n ; A, U +dU) - ( x n ; A>U ) (6) yoritlirri (9) iff it represents the E V D :R x = U A U H .

3566
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on September 06,2024 at 06:04:48 UTC from IEEE Xplore. Restrictions apply.
 3.3. Orthonormality properties 4. APPLICATIONS AND COMPARISONS
To show the unitarity conservation property of MALASE
Let us remark that, the algorithm MALASE has a structure
algorithm, let us recall first a straightforward property of
close to t,liat,of the algorit,lim PR.OTEUS-1 [3]. The latter
skew-symmetric matrices (i.e. matrices A such that AH =
differs fxoni MALASE by the fact that, An is obtained from a
-A) : for any such matrix A,
minixnuxu mean square enox criterion (11 A - UHRx,,,U II),
exp(A) x exp(AH) = I N , (12) and that. U, = Un-1 x exp(On), where the skew matrix
i.e. exp(A) is unitary. Now, let us remark that, the act,uali-
On satisfies [@n]i,j = pyn,iV:,,/(An-l,i - An-i,j), instead
of [On];,, = pYn,iy:,j(l/An-l,i - l/An-~,j) for MALASE.
sation of U in (9) can be rewritten as
Furthermore, the PROTEUS-1 algorithm requires
Un = Un-1 x exp(p[A:ilyny,X - YnYnH An-11)
-1
+ O(P') O ( N Z I < multiplications.
) It can be approximated to get the
(13) O(NIC) PROTEUS-2 algorithm. Together with MALASE
these are the only algorithms that ensure O ( N K )computa-
and that the matrix tional cost and perfect. orthonormality of the eigenvectors.
CiYnY,H - YnYnHA-^
n-1 (14) However, since PROTEUS-2 is obtained thanks to simplify-
ing asumptions, we checked that, this results in worse results
is skew-symetric. Thus, if U0 is a unitary matrix, up to the than for MALASE in pract,ical non stationary situations, as
second order in ,U, the sequence of estimated matrices Un shown figure 1.
produced by the algorithm MALASE will be unitary.
We checked the efficiency of the proposed algorithm
Let us point out that the previous remark suggests a to locate and track moving sources, and compared it to
new algorithm for which exact ortonormality will occur at Proteus-1 and S. Affes' met,liod [l].At each instant n, we
each step, provided U0 is unitary. This algorithm is ob- update the sources posit,ions as in [IO] and deal with
tainedl by replacing equation (9c) in MALASE by : crossing sources by means of a kinematic model (see [l]).
C'n = Un-1 exp (,U[AzL1ynY,H - Y ~ Y , H A E ~. ~ ] ) (15) The step is pi = 0.01 for our algorithm and Proteus-1, and
p~ = 0.005 for S. Affes method. These values have been
Although it might seem to be quite difficult to compute chosen in order t o guarantee optimum tracking capability
the exponential matrix that appears in this algorithm, the in the context. of our sinlulations. Two moving crossing
following straightforward result sources s l and s2 are tracked over 10,000 iterations using
an m a y of N = 16 sensors. We assume the presence of
euUN= Z +-
UVH
(e('"") - 1) Vu,U E p, (16) a white noise with SNR = 5dB. The time averaged bias
b = E { E } and standard deviation U = [E { IE - b12}]1'2
VHU

yields in (15): of the est,imated location E = cp - @ are computed. We

U, = un-l x check in table 1 that our algorithm stands the comparison
to PROTEUS-1. Affes' method appears to show better re-
sults. But. it. requires the use of a kinematic model, the ad-
justment of which is not, straightforward. Furthermore, for
problems where non Toeplitz hermitian matrices are con-
sidered (e.g. blind deconvolution of mutli-channel digital
communications signals), it cannot be used any longer.
where only a scalar exponential computation is involved. Finally, a zoom over 1000 iterations is displayed on
This shows that this algorithm has a computational cost, of figures 2 and 3, showing the similarity of performances
0 ( N K ) . Since this algorithm only differs from MALASE between MALASE ( O ( N K ) )and PROTEUS-1 ( O ( N K 2 ) )
by a factor O(p2),they both yield similar results. me tho ds.
3.4. A s y m p t o t i c performances b U

From ([2], p. 103), we know that the asymptotic covariance I I

matrix of the vector parameters estimated by a stochas- Proteus-1 I 0.24 I 0.30 I 0.36 I 0.41
tic grdient algorithm can be estimat,ed by solving a Lya- MALASE I 0.22 I 0.27 I 0.33 I 0.38
pouncv equation. Let us consider the ( N 2 N)-vector + S. Affes I 0.08 I 0.10 I 0.18 I 0.19
H = [ U T . . .U$ A 1 . ..A,] T where uk denotes the kth col-
umn of U. Then, for MALASE algorithm, the asymptotic Table 1: Bias and St,andard Deviation of
covariance matrix of the error of estimation of H is the t,he error of estimation, in degree, over
+
( N 2 $- N) x ( N 2 N) matrix given by 10000 iterations.
1
P = ,UZINZ+N (18)
This is a very interesting result since the asymptotic vaxui-
ance depends only on p , and not on the eigenvalues of R x , 5. REFERENCES
as in other algorithms (e.g. [lo]). Furthermore, let, us re-
mark that any of the N 2 + N partameters are asymptotically 113 S. Affes, S. Gazor, and Y. Grenier. An algorithm for
uncorrelated, and have the same variancep f . niulti-source beainforming and multi-target, tracking.

3567
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on September 06,2024 at 06:04:48 UTC from IEEE Xplore. Restrictions apply.
 Proleus-1 ys Proleus-2 [I11 R.O. Schmidt. Multiple emitter location and signal
40
r parameter estimation. IEEE Trans. on Antenna Prop-
,’ agation, AP-34:276-280, March 1986.
[12] Q. Wu and D.R. Fuhrmann. A parametic method for
determining the number of signals in narrow-band di-
rection finding. IEEE Trans. on Signal Processing,
39(8):1848-1857, August 1991.
[13] B. Yang. Projection approximation subspace tracking.
IEEE Trans. on Signal Processing, 43( 1):95-107, Jan-
uary 1995.

MALASE - O(N)O

-401 ’ ’ 1
0 100 200 300 400 5M) 800 700 800 9oolooD
iterations

Figure 1: Proteus-1 versus Proteus-2 : tracking capa-

bility; +
: true position cp - Dashed line : estimated
position @, Proteus-1 - Solid line : estimated position
@, Proteus-2 - S N R = 10dB.

IEEE Trans. on Signal Processing, 44(6):1512-1522,

June 1996.
A. Benveniste, M. Metivier, and P. Priouret. Al-
gorithmes Adaptatifs et Approximations Stochas-
0 100 200 300 400 500 600 700 800 900 lo00
tiques, Techniques stochastiques. Masson, 1957. Ilerallons
B. Champagne and Q.G. Liu. A new family of
EVD tracking algorithms using Givens rotations. In Figure 2: Tracking of two moving sources - Dashed line :
ICASSP’96, pages 1-4, 1996. true position cp - Solid line : estimated position @ - S N R =
R.D. DeGroat. Noniterative subspace tracking. IEEE 5dB.
Trans. on Signal Processing, 40(3):571-577, Mar&
1992.
S. Erlich and R. Yao. Convergences of adaptative block
simultaneous iteration method for eigenstructure de-
composition. Signal Processing, 37:l-13, January 1994.
J.J. Fuchs. Estimation of the number of signals in
the presence of h o w correlated sensor noise. IEEE
Trans. on Signal Processing, 40(5):1053-1061, May
1992.
G. H. Golub and C . F. van Loan. Matris Computation
- Second Edition. John Hopkins Series in Mathematical
Sciences. John Hopkins University Press, 1990.
E. Moulines, P. Duhamel, J.F. Cardoso, and
S. Mayrargue. Subspace methods for the blind iden-
tification of multichannel FIR filters. IEEE Trans. on
Signal Processing., 43(2):516-525, February 1995.
P.A. Regalia. An adaptative unit norm filter with ap- 0 100 m 300 400 500 6w 700 800 900 lo00
lleraliom
plication to signal analvsis and Karhunen-Loeve trans-
formations. I z E E Trans. on CAS, 37(5):646-649, May
Figure 3: Tracking of two moving sources - Dashed line :
1990.
true position cp - Solid line : estimated position @ - S N R =
[lo] C . Riou, T. Chonavel, and P.Y. Cocliet.. Atlap- 5 d ~ .
tive subspace estimation : Aplication to moving
sources localization and blind channel identification.
In ICASSP’96, pages 1649-1652, 1996.

3568
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on September 06,2024 at 06:04:48 UTC from IEEE Xplore. Restrictions apply.