1.

FIR Filter Design via Spectral Factorization and Convex Optimization 1

FIR Filter Design via Spectral

Factorization and Convex
Optimization
Shao-Po Wu
Stephen Boyd
Lieven Vandenberghe

ABSTRACT We consider the design of nite impulse response (FIR) lters

subject to upper and lower bounds on the frequency response magnitude.
The associated optimization problems, with the lter coecients as the
variables and the frequency response bounds as constraints, are in general
nonconvex. Using a change of variables and spectral factorization, we can
pose such problems as linear or nonlinear convex optimization problems. As
a result we can solve them eciently (and globally) by recently developed
interior-point methods. We describe applications to lter and equalizer de-
sign, and the related problem of antenna array weight design.

 To appear as Chapter 1 of Applied Computational Control, Signal and Commu-

nications, Biswa Datta editor, Birkhauser, 1997.
 The paper is also available at URL http://www-isl.stanford.edu/people/boyd
and from anonymous FTP to isl.stanford.edu in pub/boyd/reports.
2 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
1 Introduction
A nite impulse response (FIR) lter is a linear system described by a
convolution input-output relation
n,1
X
y(t) = h(i)u(t , i); (1.1)
i=0
where u : Z ! R is the input signal and
, y : Z ! R is the output
signal.
We say n is the lter order, and h = h(0); h(1); : : : ; h(n , 1) 2 Rn are
the lter coecients. The lter frequency response H : R ! C is de ned
as
H (!) = h(0) + h(1)e,j! +


+ h(n , 1)e,j(n,1)! (1.2)
p
where j = ,1. Since H is 2 periodic and satis es H (,!) = H (!), it is
sucient to specify it over the interval ! 2 [0; ].
A (frequency response) magnitude speci cation has the form
L(!)  jH (!)j  U (!) for all ! 2 [0; ]: (1.3)
We refer to L and U as the (lower and upper) frequency response bound
functions.
We can assume that 0  L(!)  U (!) for all !, since L(!) can be
replaced by maxfL(!); 0g without changing the constraint, and if L(!) >
U (!) for some ! then the magnitude speci cation is evidently infeasible.
We allow U (!) to take on the value +1 (which means there is no upper
bound on jH (!)j). Similarly, L(!) = 0 means there is no lower bound
on jH (!)j. For any ! with L(!) = U (!), the magnitude speci cation (1.3)
implies the (magnitude) equality constraint jH (!)j = L(!). The magnitude
speci cation (1.3) is sometimes called a semi-in nite constraint since it
consists of an in nite number of inequalities, i.e., one for each ! 2 [0; ].
In this paper we consider the design of FIR lters (i.e., the choice of
h 2 Rn ) that satisfy magnitude speci cations. We will refer to such design
problems as magnitude lter design problems. A classic example is the de-
sign of a lowpass lter that maximizes stopband attenuation subject to a
given maximum passband ripple. This design problem can be expressed as
the optimization problem
minimize 
subject to 1=  jH (!)j  ; ! 2 [0; !p] (1.4)
jH (!)j  ; ! 2 [!s ; ]:
The optimization variables are the lter coecients h 2 Rn and the stop-
band attenuation (bound)  2 R. The problem parameters are the lter
order n, the passband frequency !p, the stopband frequency !s , and the
1. FIR Filter Design via Spectral Factorization and Convex Optimization 3
maximum allowed passband ripple . Note that the passband ripple speci -
cation is logarithmic or fractional; the upper and lower bounds are given by
a constant factor. Passband ripple is commonly expressed in decibels, i.e.,
as 20 log10 dB. Similarly, the stopband attenuation is also often expressed
in decibels (as 20 log10  dB).
The magnitude speci cations in (1.4), which are given over separate in-
tervals, can be converted to a single magnitude speci cation of the form (1.3).
In this example, there are no speci cations over the transition band (!p; !s ),
so L(!) = 0 and U (!) = 1 for !p < ! < !s ; similarly we have L(!) = 0
for !  !s .
Magnitude lter design problems (such as the lowpass lter design prob-
lem (1.4)) are not, in general, convex optimization problems, except when
certain symmetry (sometimes called linear phase) constraints are imposed
(e.g., the condition that the lter coecients are symmetric around the
middle index); see, e.g., [40]. Therefore numerical methods that directly
solve magnitude lter design problems are not guaranteed to nd the glob-
ally optimal design; such methods can get \stuck" at a locally optimal, but
globally suboptimal, design. For some applications this doesn't matter, as
long as a good design is found. For cases where nding the global minimum
is valuable, there are several heuristic methods for minimizing the likelihood
of not nding the global solution, e.g., running the algorithm repeatedly
from di erent starting points. There are also methods for global optimiza-
tion, e.g., branch-and-bound (see, e.g., [33, 9]) that are guaranteed to nd
the globally optimal design. These methods, however, are often orders of
magnitude less ecient than the standard (local optimization) methods.
The purpose of this paper is to show that by a change of variables, a
wide variety of magnitude lter design problems can be posed as convex
optimization problems, i.e., problems in which the objective and constraint
functions are convex. As a consequence we can (globally) solve such mag-
nitude lter design problems with great eciency using recently developed
interior-point methods for convex optimization (see, e.g., [39, 58, 61, 57]).
The new variables are the autocorrelation coecients of the lter; the
lter coecients are then recovered by spectral factorization (see x2). The
idea of designing FIR lters via spectral factorization was rst used by
Herrmann and Schussler in the 1970 paper [25]. Since then many authors
have studied variations on and extensions of this idea, including di erent
methods for performing the spectral factorization (see, e.g., [12, 38]), and
di erent methods for solving the transformed problem e.g., by exchange-
type algorithms or linear or quadratic programming (see, e.g., [12, 29, 18,
51, 21]).
In this paper we extend the idea and apply nonlinear convex optimiza-
tion techniques to a variety of magnitude lter design problems. Linear
and quadratic programming are well-developed elds; extremely ecient
software is widely available. It is less well known that in the last decade
or so the powerful interior-point methods that have been used for linear
4 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
and quadratic programming have been extended to handle a huge variety of
nonlinear convex optimization problems (see, e.g., [39, 56, 36]). Making use
of convex optimization methods more general than linear or quadratic pro-
gramming preserves the solution eciency, and allows us to handle a wider
class of problems, e.g., problems with logarithmic (decibel) objectives.
We close this introduction by comparing magnitude lter design, the
topic of this paper, with complex Chebychev design, in which the speci ca-
tions have the general form
jH (!) , D(!)j  U (!) for all ! 2 [0; ];
where D : [0; ] ! C is a desired or target frequency response, and
U : [0; ] ! R (with U (!)  0) is a frequency-dependent weighting func-
tion. Complex Chebychev design problems, unlike the magnitude design
problems addressed in this paper, are generally convex, directly in the
variables h; see, e.g., [17, 45].
1. FIR Filter Design via Spectral Factorization and Convex Optimization 5
2 Spectral factorization
The autocorrelation coecients associated with the lter (1.1) are de ned
as
n,1
X
r(t) = h(i)h(i + t); t 2 Z; (1.5)
i=,n+1
where we interpret h(t) as zero for t < 0 or t > n , 1. Since r(t) =
r(,t) and r(t) = 0 for t  n, it suces to specify the autocorrelation
coecients for t = 0; : : : ; n,1. With some abuse, of notation, we will

write
the autocorrelation coecients as a vector r = r(0); : : : ; r(n , 1) 2 Rn .
The Fourier transform of the autocorrelation coecients is
X n,1
X
R(!) = , j!t
r(t)e = r(0) + 2r(t) cos !t = jH (!)j2 ; (1.6)
t2Z t=1
i.e., the squared magnitude of the lter frequency response. We will use
the autocorrelation coecients r 2 Rn as the optimization variable in
place of the lter coecients h 2 Rn . This change of variables has to
be handled carefully, since the transformation from lter coecients into
autocorrelation coecients is not one-to-one, and not all vectors r 2 Rn
are the autocorrelation coecients of some lter.
From (1.6) we immediately have a necessary condition for r to be the
autocorrelation coecients of some lter: R must satisfy R(!)  0 for all !.
It turns out that this condition is also sucient: The spectral factorization
theorem (see, e.g., [4, ch9]) states that there exists an h 2 Rn such that
r 2 Rn is the autocorrelation coecients of h if and only if
R(!)  0 for all ! 2 [0; ]: (1.7)
The process of determining lter coecients h whose autocorrelation
coecients are r, given an r 2 Rn that satis es the spectral factorization
condition (1.7), is called spectral factorization. Some widely used spectral
factorization methods are summarized in the appendix.
Let us now return to magnitude lter design problems. The magnitude
speci cation (1.3) can be expressed in terms of the autocorrelation coe-
cients r as
L(!)2  R(!)  U (!)2 for all ! 2 [0; ]:
Of course, we must only consider r that are the autocorrelation coecients
of some h, so we must append the spectral factorization condition to obtain:
L(!)2  R(!)  U (!)2 ; R(!)  0 for all ! 2 [0; ]: (1.8)
6 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
These conditions are equivalent to the original magnitude speci cations
in the following sense: there exists an h that satis es (1.3) if and only if
there exists an r that satis es (1.8). Note that the spectral factorization
constraint R(!)  0 is redundant: it is implied by L(!)2  R(!).
For each !, the constraints in (1.8) are a pair of linear inequalities in the
vector r; hence the overall constraint (1.8) is convex in r. In many cases, this
change of variable converts the original, nonconvex optimization problem
in the variable h into an equivalent, convex optimization problem in the
variable r.
As a simple example, consider the classical lowpass lter design prob-
lem (1.4) described above. We can pose it in terms of r as
minimize ~
subject to 1= 2  R(!)  2 ; ! 2 [0; !p] (1.9)
R(!)  ~; ! 2 [!s ; ]
R(!)  0; ! 2 [0; ]:
(~ here corresponds to 2 in the original problem (1.4).)
In contrast to the original lowpass lter design problem (1.4), the prob-
lem (1.9) is a convex optimization problem in the variables r 2 Rn and
~ 2 R. In fact it is a semi-in nite linear program, since the objective is
linear and the constraints are semi-in nite, i.e., a pair of linear inequalities
for each !.

3 Convex semi-in nite optimization

In this paper we encounter convex optimization problems of the general
form
minimize f0 (x)
subject to Ax = b; (1.10)
fi (x)  0; i = 1; : : : ; m;
gi (x; !)  0; ! 2 [0; ]; i = 1; : : : ; p;
where x 2 Rk is the optimization variable, f0 ; : : : ; fm : Rk ! R are
convex functions, and for each ! 2 [0; ], gi (x; !) are convex functions of
x. Note the three types of constraints: Ax = b are the equality constraints;
fi (x)  0 are the (ordinary) inequality constraints; and gi (x; !)  0 for
all ! are the semi-in nite inequality constraints. If the objective function
f0 is identically zero, the problem (1.10) reduces to verifying whether the
constraints are feasible or not, i.e., to a feasibility problem.
In the remainder of this section we give a brief discussion of various
methods for handling the semi-in nite constraints. General semi-in nite
convex optimization is a well-developed eld; see for example [8, 44, 7,
1. FIR Filter Design via Spectral Factorization and Convex Optimization 7
27, 26, 41, 47]. While the theory and methods for general semi-in nite
convex optimization can be technical and complex, our feeling is that the
very special form of the semi-in nite constraints appearing in lter design
problems allows them to be solved with no great theoretical or practical
diculty.
Let us start by pointing out that the semi-in nite inequality constraint
gi (x; !)  0 for all ! 2 [0; ]
can be handled by expressing it as the ordinary inequality constraint
hi (x) = sup gi (x; !)  0:
!2[0;]
It is easily veri ed that hi is a convex function of x, since for each !, gi (x; !)
is convex in x. (On the other hand, hi is often nondi erentiable, even
if the functions gi are di erentiable.) Thus, the semi-in nite constraints
in (1.10) can be handled by several methods for general (nondi erentiable)
convex optimization, e.g., bundle methods [28], ellipsoid methods [11], or
cutting plane methods [30, 23, 46]. What is required in these methods is an
ecient method for evaluating hi and a subgradient at any x. This involves
computing a frequency  for which gi (x;  ) = hi (x). In some problems this
can be done analytically; in any case it can be found approximately by a
(one-dimensional) search over frequency. (See, e.g., [15, 13].)
It is also possible to solve some magnitude lter design problems exactly,
by transforming the semi-in nite constraints into ( nite-dimensional) con-
straints that involve linear matrix inequalities [14, 55, 63], but we will not
pursue this idea here.
The semi-in nite constraints can also be approximated in a very straight-
forward way by sampling or discretizing frequency. We choose a set of fre-
quencies
0  !1  !2 


 !N  ;
often uniformly or logarithmically spaced, and replace the semi-in nite
inequality constraint
gi (x; !)  0 for all ! 2 [0; ]
with the set of N ordinary inequality constraints
gi (x; !k )  0 k = 1; : : : ; N:
Note that sampling preserves convexity. When N is suciently large, dis-
cretization yields a good approximation of the SIP (see [19]). A standard
rule of thumb is to choose N  15n (assuming linear spacing).
8 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
As an example consider the lowpass lter design problem given by (1.9).
The discretized approximation has the form
minimize ~
subject to 1= 2  R(!k )  2 ; !k 2 [0; !p] (1.11)
R(!k )  ~; !k 2 [!s ; ]
R(!k )  0; !k 2 [0; ]:
This is in fact a linear program (LP) with n + 1 variables (r; ~), and 2N
linear inequality constraints. One immediate advantage is that existing,
ecient LP software can be used to solve this (discretized) lowpass lter
design problem.
The reader should note the di erences between the three variations on the
lowpass lter design problem. The original problem (1.4) is a semi-in nite,
nonconvex problem. The transformed problem (1.9) is a semi-in nite but
convex problem, which is equivalent (by change of variables) to the original
problem (1.4). The transformed and discretized problem (1.11) is an ap-
proximation of the semi-in nite problem (1.9). It is also a linear program,
and so can be solved using standard linear programming codes (or, for even
greater eciency, by an interior-point LP solver that exploits the special
problem structure [16]).
Evidently for the discretized problem we can no longer guarantee that the
semi-in nite constraints are satis ed between frequency samples. Indeed
the sampled version of the problem is an outer approximation of the original
problem; its feasible set includes the feasible set of the original problem. In
many cases this causes no particular harm, especially if N is large. But if
the spectral factorization condition does not hold, i.e., R( ) < 0 for some
 between samples, then spectral factorization breaks down; we cannot
nd a set of lter coecients h that have r as its autocorrelation. Several
methods can be used to avoid this pitfall. The simplest is to add a small
safety margin to the sampled version of the spectral factorization condition,
i.e., replace it by
R(!k )  ; k = 1; : : : ; N (1.12)
where  is small and positive. This can be done in an ad hoc way, by
increasing  (and re-solving the problem) until the spectral factorization of
R is successful. If N is large this will occur when  is small.
We can also analyze the approximation error induced by discretization
by bounding the variation of the functions gi (x; !) for ! between samples.
To give a very simple example, assume we use uniform frequency sampling,
i.e., !k = (k , 1=2)=N , k = 1; : : : ; N . We assume we have (or impose)
some bound on the size of h, say,
p
khk = h(0)2 +


+ h(n , 1)2  M:
1. FIR Filter Design via Spectral Factorization and Convex Optimization 9
Thus, r(0) = khk2  M 2 , and a standard result shows that jr(t)j  M 2 for
all t.
Now let ! be any frequency in [0; ] and let !k denote the nearest sam-
pling frequency, so that j! , !k j  =(2N ). We have
n,1
X
jR(!) , R(!k )j = 2r(t)(cos !t , cos !k t)
t=1

n,1
X n,1
X
 4M 2 sin j! , !k jt  4M 2 ! , !k t  M 2 n(n , 1)=N
t=1 t=1

Thus, the intersample error cannot exceed M 2 n(n,1)=N (which evidently

converges to zero as N ! 1). For example, if we take  = M 2 n(n,1)=N
in (1.12), then it is guaranteed that the (semi-in nite) spectral factorization
condition (1.7) will be met. The bound developed here is very simple, and
only meant to give the general idea; far more sophisticated bounds can be
derived; see, e.g., [19, x3.7].
Let us mention several reasons why simple discretization works very well
in practice and is widely used. First of all, its computational cost is not
prohibitive. Even when special structure (see below) is not exploited, the
complexity of solving the discretized problem grows approximately as a
linear function of N , so making N large (to get a good discretized approx-
imation) incurs only a linear increase in computation required.
When uniform frequency sampling is used, the fast Fourier transform
can be used to great advantage to speed up many of the computations in
interior-point methods. In [16] the authors demonstrate that FIR design
problems with 1000s of variables and 10000s of constraints can be solved in
tens of seconds on a small workstation, by exploiting the special structure
of the discretized problems.
Finally, we should point out that frequency sampling plays a key role
in the more sophisticated methods for handling semi-in nite constraints.
Indeed, most of the more sophisticated algorithms for SIPs solve a sequence
of discretized problems with carefully chosen frequency samples (see, e.g.,
[26, 47].
In the remainder of this paper, we will pose problems rst as convex,
semi-in nite problems. We then either give the sampled version, or point
out if it has a special form such as LP. The sampled versions can be thought
of as approximations (which are probably more than adequate for practical
designs) or as subproblems that arise in sophisticated algorithms that han-
dle the semi-in nite constraints exactly. The numerical results given in this
paper were obtained from ( ne) discretizations, using the nonlinear convex
programming solver SDPSOL [62].
10 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe

1=

jH (!)j

!p !s 
!
FIGURE 1. Lowpass lter design speci cations.

4 Lowpass lter design

In this section we consider the lowpass lter design problem described above
as an example. Of course the same techniques are readily applied to other
lters such as highpass, bandpass, notch, bandstop, or complex lter types
with multiple stop and pass bands.
The constraints consist of a passband ripple speci cation,
1=  jH (!)j  for ! 2 [0; !p];
where !p is the passband frequency, and > 1 gives the passband ripple.
The stopband attenuation speci cation is given by
jH (!)j   for ! 2 [!s ; ];
where !p > !p is the stopband frequency, and  gives the maximum stop-
band gain. These speci cations are illustrated in gure 1.
We have already seen that the problem of maximizing stopband atten-
uation (i.e., minimizing ) can be formulated as the convex optimization
problem
minimize ~
subject to 1= 2  R(!)  2 ; ! 2 [0; !p]
R(!)  ~; ! 2 [!s ; ]
R(!)  0; ! 2 [0; ];
which, when discretized in frequency, yields an LP.
An example design is shown in gure 2. The lter order is n = 30, the
passband ripple is = 1:1 (about 1dB), the passband frequency is !p =
1. FIR Filter Design via Spectral Factorization and Convex Optimization 11

0
10

−1
10
jH (!)j

−2
10

−3
10
:12 :24 
!
FIGURE 2. Maximum stopband attenuation design. The lter order is n = 30,
maximum passband ripple is = 1:1 (about 1dB), passband frequency is
!p = 0:12 , and stopband frequency is !s = 0:24 . The stopband attenuation
achieved is ,56dB.

0:12, and the stopband frequency is one octave higher, i.e., !s = 0:24.
The maximum attenuation achieved is opt = 0:0016, i.e., around ,56dB.
We remind the reader that this design is the globally optimal solution: it
is not merely the best design achieved by our method, but in fact the best
design that can be achieved by any method.
We can consider several variations on this problem. Suppose we x the
stopband attenuation and wish to minimize the passband ripple. This can
be expressed as the optimization problem
minimize ~
subject to 1= ~  R(!)  ~; ! 2 [0; !p]
R(!)  2 ; ! 2 [!s ; ]
R(!)  0; ! 2 [0; ];
where the optimization variables are r and ~ . The problem parameters are
, !p, !s (and the lter order n). (The optimization variable ~ corresponds
to 2 in the ripple speci cation.)
This problem is in fact a convex optimization problem. To see this we
consider the speci cations at a xed frequency !. The constraints
R(!)  ~; R(!)  2 ; R(!)  0
are linear inequalities on the variables (r; ~). The remaining, nonlinear
constraint is
1= ~ , R(!)  0:
12 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
The function 1= ~ , R(!) can be veri ed to be convex in the variables (r; ~)
(since ~ > 0). Indeed, when sampled this problem can be very eciently
solved as a second-order cone program (SOCP); see [36, x3.3] and [35].
Note that passband ripple minimization (in dB) cannot be solved (directly)
by linear programming; it can, however, be solved by nonlinear convex
optimization.
We can formulate other variations on the lowpass lter design problem
by xing the passband ripple and stopband attenuation, and optimizing
over one of the remaining parameters. For example we can minimize the
stopband frequency !s (with the other parameters xed). This problem is
quasiconvex ; it is readily and eciently solved using bisection on !s and
solving the resulting convex feasibility problems (which, when sampled,
become LP feasibility problems). Similarly, !p (which is quasiconcave) can
be maximized, or the lter order (which is quasiconvex) can be minimized.
It is also possible to include several types of constraints on the slope of
the magnitude of the frequency response. We start by considering upper
and lower bounds on the absolute slope, i.e.,
a  djH (!)j  b:
d!
This can be expressed as
1=2
a  dR(d!
!) = dR=d!
p  b;
2 R(!)
which (since R(!) is constrained to be positive) we can rewrite as
p p
2 R(!)a  dR=d!  2 R(!)b:
Now we introduce the assumption that a  0 and b  0. The inequalities
can be written
p p
2a R(!) , dR=d!  0; dR=d! , b R(!)  0: (1.13)
p
Since R(!) is a linear function of pr (and positive),
p
R(!) is a concave
function of r. Hence the functions a R(!) and ,b R(!) are convex (since
a  0 and b  0). Thus, the inequalities (1.13) are convex (since dR=d! is a
linear function of r). When discretized, this constraint can be handled via
SOCP. As a simple application we can design a lowpass lter subject to the
additional constraint that the frequency response magnitude is monotonic
decreasing.
Let us now turn to the more classical speci cation of frequency response
slope, which involves the logarithmic slope:
jH j !  b:
a  dd! jH (!)j
1. FIR Filter Design via Spectral Factorization and Convex Optimization 13
(The logarithmic slope is commonly given in units of dB/octave or dB/decade.)
This can be expressed as
a  (1=2) dR !
d! R(!)  b;
which in turn can be expressed as a pair of linear inequalities on r:
2R(!)a=!  dR
d!  2R(!)b=!:
Note that we can incorporate arbitrary upper and lower bounds on the
logarithmic slope, including equality constraints (when a = b). Thus, con-
straints on logarithmic slope (in dB/octave) are readily handled.

5 Log-Chebychev approximation
Consider the problem of designing an FIR lter so that its frequency re-
sponse magnitude best approximates a target or desired function, in the
sense of minimizing the maximum approximation error in decibels (dB).
We can formulate this problem as
minimize sup log jH (!)j , log D(!) (1.14)
!2[0;]
where D : [0; ] ! R is the desired frequency response magnitude (with
D(!) > 0 for all !). We call (1.14) a logarithmic Chebychev approximation
problem, since it is a minimax (Chebychev) problem on a logarithmic scale.
We can express the log-Chebychev problem (1.14) as
minimize
subject to 1=  R(!)=D(!)2  ; ! 2 [0; ];
R(!)  0; ! 2 [0; ]
where the variables are r 2 Rn and 2 R. This is a convex optimization
problem (as described above), eciently solved, for example, as an SOCP.
Simple variations on this problem include the addition of other constraints,
or a frequency-weighted log-Chebychev objective.
As an example we consider the design of a 1=f spectrum-shaping lter,
which is used to generate 1=f noise by ltering white noise through an
FIR lter. The goal is to approximate the magnitude D(!) = !,1=2 over
a frequency band [!a ; !b ]. If white noise (i.e., a process with spectrum
Su (!) = 1) is passed through such a lter, the output spectrum Sy will
satisfy Sy (!)  1=! over [!a ; !b ]. This signal can be used to simulate 1=f
noise (e.g., in a circuit or system simulation) or as a test signal in audio
applications (where it is called pink noise [54]).
14 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
1
10

jH (!)j
0
10

−1
10
:01 :1 
!
FIGURE 3. Magnitude frequency response of a 50th order 1=f spectrum
p shaping
lter for frequency range [0:01; ]. Dashed line shows ideal 1= f magnitude,
which falls at 10dB/octave. The maximum approximation error is 0:5dB.

w
v H (!) T (! ) y

FIGURE 4. Magnitude equalization. The frequency response T of the system is

known; the goal is to design the equalizer H so the product T H has approximately
constant magnitude.

Using a minimax logarithmic (dB) criterion, and frequency sampling

yields the problem
minimize
subject to 1=  (!i = )R(!i )  ; !i 2 [!a ; !b] (1.15)
R(!i )  0; i = 0; : : : ; N;
which is readily solved as an SOCP. Figure 3 shows a design example, with
lter order n = 50, !a = 0:01, !b = . The t obtained is opt = 1:12,
which is around 0:5dB.

6 Magnitude equalizer design

In the simplest magnitude equalizer problem, we are given a function T :
[0; ] ! C (not necessarily the frequency response of an FIR lter), and
1. FIR Filter Design via Spectral Factorization and Convex Optimization 15
need to design an FIR lter (equalizer) H so that the product TH has
approximately constant magnitude, e.g., one:
jT (!)H (!)j  1 for all ! 2 [0; ]:
This is illustrated in gure 4, where the equalizer processes the signal before
the given function; but the problem is the same if the order of the equalizer
and given function is reversed. Note also that we only need to know the
magnitude jT (!)j, and not T (!), for ! 2 [0; ].
The equalizer problem can be posed as the log-Chebychev approximation
problem
minimize sup log jT (!)H (!)j ;
!2[0;]
which is readily formulated as a convex problem using the autocorrelation
coecients r as the design variables:
minimize
subject to 1=  R(!)jT (!)j2  ; ! 2 [0; ]
R(!)  0; ! 2 [0; ]:
In many applications we must add regularization constraints on the
equalizer frequency response, to keep the magnitude or its slope from being
too large. These constraints are readily handled. For example, we can im-
pose (frequency-dependent) bounds on jH j and its absolute or logarithmic
derivative, as described above. A very simple way to bound the size of h is
to impose the constraint
r(0) = h(0)2 +


+ h(n , 1)2  M 2
(which is a single linear inequality on r).
So far we have assumed that the target equalized gain, i.e., the desired
value of jTH j, is one. We can also allow some freedom in the target value of
the equalized gain. This type of problem can be handled using an absolute
(or more accurately, squared) criterion
minimize sup jT (!)H (!)j2 , ;
!2[0;]
where is subject to some bounds such as l   h, and h is subject to
some regularization constraints. Note that the optimization variables here
are h and .
We can cast this as the convex problem
minimize
subject to R(!)jT (!)j2 ,  ; ! 2 [0; ]
R(!)  0; ! 2 [0; ]
16 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
2
10

1
10

jT (!)H (!)j
10
0

−1
10

−2
10

−3
10
0 0.5 1 1.5 2 2.5 3
!
FIGURE 5. Log-Chebychev magnitude equalization. Magnitude of function T (!)
(dashed) and equalized frequency response T (!)H (!) (solid). The maximum er-
ror is 4:8dB.

where in addition we have limits on and some constraints on H (i.e., R).

If this problem is discretized, it becomes an LP.
An example of log-Chebychev magnitude approximation is illustrated in
gure 5. The 50th order equalizer achieves a maximum equalization error
of 4:8dB.
We now turn to an interesting and useful extension of the magnitude
equalizer problem: simultaneous equalization of several (or many) func-
tions. We are given functions Tk : [0; ] ! C, k = 1; : : : ; K , and need to
design H so that jTk H j are all approximately constant. In other words,
we need to design a single equalizer for multiple functions. This situation
arises in several contexts. As an example, suppose that Tk is the frequency
response from the (electrical) input to a public address or sound reinforce-
ment system to the (acoustic) response at a location k in the theater. The
equalizer H is meant to give approximately constant magnitude response at
any of K locations in the theater. The multi-system equalization problem
setup is illustrated in gure 6.
We can formulate the multi-system magnitude equalization problem as
a minimax log-Chebychev approximation problem:
minimize k=1max;::: ;K
sup log jTk (!)H (!)j : (1.16)
!2[0;]
In this formulation we have xed the target value for each jTk H j as one;
it is of course possible to have di erent target values for di erent k. We
should point out that this minimax formulation already builds in a form of
regularization: H (!) will become large only if all of the Tk (!) are small.
1. FIR Filter Design via Spectral Factorization and Convex Optimization 17

y1
2
T1 (!)
3 y2
w ..
v H (!) 6
4 .
7
5
TK (!)
yK

FIGURE 6. Multisystem magnitude equalization. The goal is to choose H so that

the magnitude response at y1 ; : : : ; yK are all approximately constant over !.

The idea of allowing the equalized gain to ` oat' is quite useful in the con-
text of multi-system equalization. In the sound reinforcement application
mentioned above, the gain at di erent locations could vary by, say, 10dB,
as long as each equalized frequency response is at within, say, 4dB. To
allow the equalized gains to di er, we can formulate a minimax squared
magnitude problem such as
minimize k=1max
;::: ;K
sup jTk (!)H (!)j2 , k ;
!2[0;]
to which we might add constraints on k such as a lower and upper bound.
Note that the variables here are h and 1 ; : : : ; K . This problem can be
cast as the convex problem
minimize
subject to jTk (!)j2 R(!) , k  ; k = 1; : : : ; K; ! 2 [0; ]
R(!)  0; ! 2 [0; ]:
This becomes an LP when discretized. An example with K = 2 and lter
order n = 25 is shown in gure 7.
Our last topic is equalization over frequency bands, which is the most
common method used in audio applications. We de ne a set of K frequency
intervals
[ 1 ; 2 ]; [ 2 ; 3 ]; : : : [ K ; K +1 ];
where 0 < 1 < : : : < K +1  . A common choice of frequencies di er by
one-third octave, i.e., k = 2(k,1)=3 1 for k = 1; : : : ; K . The average gain
of a function G : [0; ] ! C over the kth band [ k ; k+1 ] is de ned by
Z !1=2
1 k+1
jG(!)j2 d! :
k+1 , k k

In frequency band equalization, we choose the equalizer H so that the

average gain of the equalized frequency response TH in each frequency
18 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe

2.5

2
jTi (!)j2 (dB)

1.5

0.5

0
0 0.5 1 1.5 2 2.5 3

2.5

2
jTi (!)H (!)j2

1.5

0.5

0
0 0.5 1 1.5 2 2.5 3
!
FIGURE 7. Example of multi-system magnitude equalization. The two frequency
response squared magnitudes jT1 j2 (solid) and jT2 j2 (dashed) are shown in the
upper plot. The equalized frequency response magnitudes jT1 H j2 (solid) and
jT2 H j2 (dashed) are shown in the lower plot, along with the ranges opt  opt
(dotted).
1. FIR Filter Design via Spectral Factorization and Convex Optimization 19
band is approximately the same. Using a log-Chebychev (minimax dB)
criterion for the gains and r as the variable, we can express this equalization
problem as
minimize R
subject to 1=  k+11, k kk+1 R(!)jT (!)j2 d!  ; k = 1; : : : ; K;
R(!)  0; ! 2 [0; ]:
This is a convex problem in r and . To solve it numerically, we can approx-
imate the integral by frequency sampling. (Indeed, jT j is likely to be given
by its values at a ne sampling of frequencies, and not in some analytical
form.) We can also, of course, add constraints on H .
An example is shown in gure 8. The equalizer has order n = 20, and we
consider 15 third-octave bands from 1 = 0:031 to 16 = . Note that
the function jT j has several deep `dips' and `notches'. This implies that
ordinary Chebychev equalization would require very large values of jH j to
achieve good equalization.

7 Linear antenna array weight design

Our last application example treats a problem closely related to FIR lter
design: antenna array weight design. Consider a linear array of n isotropic
antennas spaced uniformly a distance d apart in a plane, as shown in g-
ure 9. A plane harmonic wave of wavelength  is incident on the array
from angle . The antennas sample the incident wave, and the resulting
signals are demodulated and then linearly combined with antenna weights
w1 ; : : : ; wn 2 C (which are our design variables) to form the combined out-
put of the antenna array, which is a complex number G. The array output
G depends on the incidence angle  of the incoming wave (and also the
weights). As a function of the incidence angle, G : [0; ] ! C is called the
pattern function, and is given by
n,1
X
G() = wk e,jk ; (1.17)
k=0
where depends on the incidence angle as
= , 2d cos :
 (1.18)
More details, and a derivation of this formula for G, can be found in any
text or survey on antennas or antenna arrays, e.g., [37, 22, 20].
Note that G has the same form as the frequency response of an FIR lter
(given in (1.2)), with two di erences: the lter coecients are replaced with
20 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe

1
10
jT (!)j and jH (!)T (!)j
0
10

−1
10

−2
10

−3
10 −1 0
10 10

1
10

0
10
Third-octave gain

−1
10

−2
10

−3
10 −1 0
10 10
!
FIGURE 8. Example of third-octave equalization, with constraint that equalizer
frequency response magnitude not exceed 10. Upper plot shows jT j (dashed) and
the equalized frequency response jT H j (solid). Lower plots shows third-octave
gains for T (dashed) and T H (solid). The third-octave gains have been equalized
within 1:1dB. Since jT j is small at several frequencies, ordinary log-Chebychev
equalization to 1:1dB would require very large values of jH j.
1. FIR Filter Design via Spectral Factorization and Convex Optimization 21

2d
FIGURE 9. Linear antenna array of antennas (shown as dots), with spacing d,
in the plane. A plane wave with wavelength  is incident from angle .

the antenna array weights (which can be complex), and the \frequency"
variable is related to the incidence angle  by (1.18).
If we de ne H : [,; ] ! C as
n,1
X
H( ) = wk e,jk ;
k=0
then we have G() = H ( ). H is then the frequency response of an
FIR lter with (complex) coecients w1 ; : : : ; wn . Since H does not sat-
isfy H (, ) = H ( ) (as the frequency response of an FIR lter with real
coecients does), we must specify H over 2 [,; ].
For  2 [0; ], is monotonically increasing function of , which we will
denote , i.e., () = ,2d= cos . As the incidence angle  varies from 0
to , the variable = () varies over the range 2d=. To simplify the
discussion below, we make the (common) assumption that d < =2, i.e.,
the element spacing is less than one half wavelength. This implies that for
 2 [0; ], () = is restricted to an interval inside [,; ]. An interval
min    max ;
where min; max 2 [0; ], transforms under (1.18) to the corresponding
interval
min   max ;
where min = (min) and max = (max), which lie in [,; ]. (The
importance of this will soon become clear.)
By analogy with the FIR lter design problem, we can de ne an antenna
pattern magnitude speci cation as
L()  jG()j  U () for all  2 [0; ]: (1.19)
22 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
An antenna array weight design problem involves such speci cations. As a
simple example, suppose we want the array to have approximately uniform
sensitivity for  2 [0; b], and sensitivity as small as possible in the interval
[s ; ]. This problem can be posed as
minimize 
subject to 1=  jG()j  ;  2 [0; b] (1.20)
jG()j  ;  2 [s ; ]:
This problem is the analog of the lowpass lter design problem (1.4). Here,
b denotes the (half) beamwidth, s denotes the beginning of the sidelobe,
and  is called the sidelobe attenuation level.
We can recast this problem as
minimize 
subject to 1=  jH ( )j  ; 2 [ (0); (b)] (1.21)
jH ( )j  ; 2 [ (s ); ()]:
(Here we use the interval mapping property mentioned above).
Now (1.21) is a lowpass lter design problem, but with complex coef-
cients wi , and speci cations over the interval [,; ]. It can be handled
like an FIR lter magnitude design problem, by an extension of spectral
factorization to the complex case.
We de ne the (now complex) autocorrelation coecients r(k), associated
with w, as
n,1
X
r (k ) = wi wi+k ; k = 0; : : : ; n , 1: (1.22)
i=,n+1
The Fourier transform of r is
n,1
X
R() = r(k)e,j k = jG()j2 ;
k=,(n,1)
is the squared magnitude of the antenna pattern function (where  and
are related as in (1.17)). We can use r 2 Cn as the design variables,
provided we add the spectral factorization condition R()  0 for all  2
[0; ].
The magnitude constraint can be expressed in terms of R as
L()2  R()  U ()2 for all  2 [0; ];
i.e., as an (in nite) set of linear inequalities on r.
An example of an antenna array weight design problem is shown in g-
ure 10. The problem is (1.20), with the following parameters: antenna el-
ement spacing d = 0:45; n = 12 (antenna elements); (half) beamwidth
b = 30; allowed ripple = 1:58 (which corresponds to 2dB); and side-
lobe angle s = 45. The sidelobe attenuation level achieved is around 0:11
(,19dB).
1. FIR Filter Design via Spectral Factorization and Convex Optimization 23

45
30

0dB ,20dB
10dB ,10dB ,30dB

FIGURE 10. Example of antenna array weight design for 12-element array with
spacing d = 0:45. The constraints require 2dB coverage over a beam of 30
and the objective is to minimize the maximum pattern response outside a 45
sector. The sidelobe level achieved is about ,19dB.
24 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
8 Conclusions
We have shown that a variety of magnitude FIR lter design problems
can be formulated, in terms of the autocorrelation coecients, as (pos-
sibly nonlinear) convex semi-in nite optimization problems. As a result,
the globally optimal solution can be eciently computed. By considering
nonlinear convex optimization problems, we can solve a number of prob-
lems of practical interest, e.g., minimax decibel problems, with an eciency
not much less than standard methods that rely on, for example, linear or
quadratic programming.

Acknowledgments: This research was supported in part by AFOSR (un-

der F49620-95-1-0318), NSF (under ECS-9222391 and EEC-9420565), and
MURI (under F49620-95-1-0525). The authors would like to thank Babak
Hassibi, Laurent El Ghaoui, and Herve Lebret for very helpful discussions
and comments. They also thank Paul Van Dooren for bringing reference [59]
to their attention.

9 References
[1] F. A. Aliyev, B. A. Bordyug, and V. B. Larin. Factorization of poly-
nomial matrices and separation of rational matrices. Soviet Journal
of Computer and Systems Sciences, 28(6):47{58, 1990.
[2] F. A. Aliyev, B. A. Bordyug, and V. B. Larin. Discrete generalized
algebraic Riccati equations and polynomial matrix factorization. Syst.
Control Letters, 18:49{59, 1992.
[3] B. Anderson. An algebraic solution to the spectral factorization prob-
lem. IEEE Trans. Aut. Control, AC-12(4):410{414, Aug. 1967.
[4] B. Anderson and J. B. Moore. Optimal Filtering. Prentice-Hall, 1979.
[5] B. Anderson and S. Vongpanitlerd. Network analysis and synthesis: a
modern systems theory approach. Prentice-Hall, 1973.
[6] B. D. O. Anderson, K. L. Hitz, and N. D. Diem. Recursive algo-
rithm for spectral factorization. IEEE Transactions on Circuits and
Systems, 21:742{750, 1974.
[7] E. J. Anderson and P. Nash. Linear Programming in In nite-
Dimensional Spaces: Theory and Applications. John Wiley & Sons,
1987.
[8] E. J. Anderson and A. B. Philpott, editors. In nite Programming.
Springer-Verlag Lecture Notes in Economics and Mathematical Sys-
tems, Sept. 1984.
1. FIR Filter Design via Spectral Factorization and Convex Optimization 25
[9] V. Balakrishnan and S. Boyd. Global optimization in control system
analysis and design. In C. T. Leondes, editor, Control and Dynamic
Systems: Advances in Theory and Applications, volume 53. Academic
Press, New York, New York, 1992.
[10] F. L. Bauer. Ein direktes Iterationsverfahren zur Hurwitz-Zerlegung

eines Polynoms. Arch. Elek. Ubertr. , 9:844{847, 1955.
[11] R. G. Bland, D. Goldfarb, and M. J. Todd. The ellipsoid method: A
survey. Operations Research, 29(6):1039{1091, 1981.
[12] R. Boite and H. Leich. A new procedure for the design of high order
minimum phase FIR digital or CCD lters. Signal Processing, 3:101{
108, 1981.
[13] S. Boyd and C. Barratt. Linear Controller Design: Limits of Perfor-
mance. Prentice-Hall, 1991.
[14] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrishnan. Linear Matrix
Inequalities in System and Control Theory, volume 15 of Studies in
Applied Mathematics. SIAM, Philadelphia, PA, June 1994.
[15] S. Boyd and L. Vandenberghe. Introduction to convex op-
timization with engineering applications. Course Notes, 1997.
http://www-leland.stanford.edu/class/ee364/.
[16] S. Boyd, L. Vandenberghe, and M. Grant. Ecient convex optimiza-
tion for engineering design. In Proceedings IFAC Symposium on Ro-
bust Control Design, pages 14{23, Sept. 1994.
[17] D. Burnside and T. W. Parks. Optimal design of r lters with the
complex chebyshev error criteria. IEEE Transactions on Signal Pro-
cessing, 43(3):605{616, March 1995.
[18] X. Chen and T. W. Parks. Design of optimal minimum phase FIR
lters by direct factorization. Signal Processing, 10:369{383, 1986.
[19] E. W. Cheney. Introduction to Approximation Theory. Chelsea Pub-
lishing Company, New York, second edition, 1982.
[20] D. K. Cheng. Optimization techniques for antenna arrays. Proceedings
of the IEEE, 59(12):1664{1674, Dec. 1971.
[21] J. O. Coleman. The Use of the FF Design Language for the Linear
Programming Design of Finite Impulse Response Digital Filters for
Digital Communication and Other Applications. PhD thesis, Unversity
of Washington, 1991.
[22] R. S. Elliott. Antenna Theory and Design. Prentice-Hall, 1981.
26 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
[23] J. Elzinga and T. Moore. A central cutting plane algorithm for the
convex programming problem. Math. Program. Studies, 8:134{145,
1975.
[24] P. A. Fuhrmann. Elements of factorization theory from a polynomial
point of view. In H. Nijmeijer and J. M. Schumacher, editors, Three
Decades of Mathematical System Theory, volume 135 of Lecture Notes
in Control and Information Sciences, pages 148{178. Springer Verlag,
1989.
[25] O. Herrmann and H. W. Schussler. Design of nonrecursive digital
lters with minimum-phase. Electronic Letter, 6:329{330, 1970.
[26] R. Hettich. A review of numerical methods for semi-in nite program-
ming and applications. In A. V. Fiacco and K. O. Kortanek, editors,
Semi-In nite Programming and Applications, pages 158{178. Springer,
Berlin, 1983.
[27] R. Hettich and K. O. Kortanek. Semi-in nite programming: theory,
methods and applications. SIAM Review, 35:380{429, 1993.
[28] J.-B. Hiriart-Urruty and C. Lemarechal. Convex Analysis and Mini-
mization Algorithms II: Advanced Theory and Bundle Methods, volume
306 of Grundlehren der mathematischen Wissenschaften. Springer-
Verlag, New York, 1993.
[29] Y. Kamp and C. J. Wellekens. Optimal design of minimum-phase FIR
lters. IEEE Trans. Acoust., Speech, Signal Processing, 31(4):922{926,
1983.
[30] J. E. Kelley. The cutting-plane method for solving convex programs.
J. Soc. Indust. Appl. Math, 8(4):703{712, Dec. 1960.
[31] P. Lancaster and L. Rodman. Solutions of the continuous and discrete
time algebraic Riccati equations: a review. In S. Bittanti, A. J. Laub,
and J. C. Willems, editors, The Riccati equation, pages 11{51. Springer
Verlag, Berlin, Germany, 1991.
[32] V. B. Larin. An algorithm for factoring a matrix polynomial relative to
a unit-radius circle. Journal of Automation and Information Sciences,
26(1):1{6, 1993.
[33] E. L. Lawler and D. E. Wood. Branch-and-bound methods: A survey.
Operations Research, 14:699{719, 1966.
[34] H. Lebret and S. Boyd. Antenna array pattern synthesis via con-
vex optimization. IEEE Trans. on Signal Processing, 45(3):526{532,
March 1997.
1. FIR Filter Design via Spectral Factorization and Convex Optimization 27
[35] M. S. Lobo, L. Vandenberghe, and S. Boyd. socp: Software for
Second-Order Cone Programming. Information Systems Laboratory,
Stanford University, 1997.
[36] M. S. Lobo, L. Vandenberghe, S. Boyd, and H. Lebret. Second-order
cone programming: interior-point methods and engineering applica-
tions. Linear Algebra and Appl., 1997. Submitted.
[37] M. T. Ma. Theory and Application of Antenna Arrays. John Wiley
and Sons, 1974.
[38] G. A. Mian and A. P. Nainer. A fast procedure to design equiripple
minimum-phase FIR lters. IEEE Trans. Circuits Syst., 29(5):327{
331, 1982.
[39] Y. Nesterov and A. Nemirovsky. Interior-point polynomial methods in
convex programming, volume 13 of Studies in Applied Mathematics.
SIAM, Philadelphia, PA, 1994.
[40] A. V. Oppenheim and R. W. Schafer. Digital Signal Processing.
Prentice-Hall, Englewood Cli s, N. J., 1970.
[41] E. Panier and A. Tits. A globally convergent algorithm with
adaptively re ned discretization for semi-in nite optimization prob-
lems arising in engineering design. IEEE Trans. Aut. Control, AC-
34(8):903{908, 1989.
[42] A. Papoulis. Signal Analysis. McGraw-Hill, New York, 1977.
[43] T. W. Parks and C. S. Burrus. Digital Filter Design. Topics in Digital
Signal Processing. John Wiley & Sons, New York, 1987.
[44] E. Polak. Semi-in nite optimization in engineering design. In A. Fi-
acco and K. Kortanek, editors, Semi-in nite Programming and Appli-
cations. Springer-Verlag, 1983.
[45] A. W. Potchinkov and R. M. Reemtsen. The design of FIR lters in the
complex plane by convex optimization. Signal Processing, 46(2):127{
146, Oct. 1995.
[46] R. M. Reemtsen. A cutting-plane method for solving minimax prob-
lems in the complex plane. Numerical Algorithms, 2:409{436, 1992.
[47] R. M. Reemtsen. Some outer approximation methods for semi-in nite
optimization problems. Journal of Computational and Applied Math-
ematics, 53:87{108, 1994.
28 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
[48] R. M. Reemtsen and A. W. Potchinkov. FIR lter design in regard to
frequency response, magnitude, and phase by semi-in nite program-
ming. In J. Guddat, H. T. Jongen, F. Nozicka, G. Still, and F. Twilt,
editors, Parametric Optimization and Related Topics IV. Verlag Peter
Lang, Frankfurt, 1996.
[49] J. Rissanen. Algorithms for triangular decomposition of block Hankel
and Toeplitz matrices with application to factoring positive matrix
polynomials. Mathematics of Computation, 27:147{154, 1973.
[50] J. Rissanen and L. Barbosa. Properties of in nite covariance matrices
and stability of optimum predictors. Information Sciences, 1:221{236,
1969.
[51] H. Samueli. Linear programming design of digital data transmis-
sion lters with arbitrary magnitude speci cations. In Conference
Record, International Conference on Communications, pages 30.6.1{
30.6.5. IEEE, June 1988.
[52] L. L. Scharf. Statistical Signal Processing. Addison-Wesley, 1991.
[53] K. Steiglitz, T. W. Parks, and J. F. Kaiser. METEOR: A constraint-
based r lter design program. IEEE Trans. Acoust., Speech, Signal
Processing, 40(8):1901{1909, Aug. 1992.
[54] M. Talbot-Smith, editor. Audio Engineer's Reference Handbook. Focal
Press, Oxford, 1994.
[55] L. Vandenberghe and S. Boyd. Connections between semi-in nite and
semide nite programming. In R. Reemtsen and J.-J. Rueckmann, edi-
tors, Proceedings of the International Workshop on Semi-In nite Pro-
gramming. 1996. To appear.
[56] L. Vandenberghe and S. Boyd. Semide nite programming. SIAM
Review, 38(1):49{95, Mar. 1996.
[57] R. J. Vanderbei. LOQO User's Manual. Technical Report SOL 92{
05, Dept. of Civil Engineering and Operations Research, Princeton
University, Princeton, NJ 08544, USA, 1992.
[58] R. J. Vanderbei. Linear Programming: Foundations and Extensions.
Kluwer, Boston, 1996.
[59] Z. Vostry. New algorithm for polynomial spectral factorization with
quadratic convergence. Part I. Kybernetika, 11:415{422, 1975.
[60] G. Wilson. Factorization of the covariance generating function of a
pure moving average process. SIAM J. on Numerical Analysis, 6:1{7,
1969.
1. FIR Filter Design via Spectral Factorization and Convex Optimization 29
[61] S. J. Wright. Primal-Dual Interior-Point Methods. SIAM, Philadel-
phia, 1997.
[62] S.-P. Wu and S. Boyd. sdpsol: A Parser/Solver for Semide nite
Programming and Determinant Maximization Problems with Matrix
Structure. User's Guide, Version Beta. Stanford University, June
1996.
[63] S.-P. Wu, S. Boyd, and L. Vandenberghe. FIR lter design via semidef-
inite programming and spectral factorization. In Proc. IEEE Conf. on
Decision and Control, pages 271{276, 1996.
[64] D. C. Youla. On the factorization of rational matrices. IRE Trans.
Information Theory, IT-7(3):172{189, July 1961.

Appendix: Spectral factorization

In this section we give a brief overview of methods for spectral factor-
ization. This problem has been studied extensively; general references in-
clude [64, 3, 5, 42, 4]. We consider spectral factorization for real-valued
h; the extensions to h complex (which arises in the antenna weight design
problem, for example) can be found in the references. We assume (without
loss of generality) that r(n , 1) 6= 0 (since otherwise we can de ne n as the
smallest k such that r(t) = 0 for t  k).
In this appendix we assume that R satis es the following strengthened
version of the spectral factorization condition:
R(!) > 0 for all ! 2 R:
All of the methods described below can be extended to handle the case
where R(!) is nonnegative for all !, but zero for some !. The strengthened
condition greatly simpli es the discussion; full details in the general case
can be found in the references.
We rst describe the classical method based on factorizing a polynomial.
We de ne the rational complex function
T (z ) = r(n , 1)z n,1 +


+ r(1)z + r(0) + r(1)z ,1 +


+ r(n , 1)z ,(n,1);
so that R(!) = T (ej! ). We will show how to construct S , a polynomial in
z ,1,
S (z ) = h(0) + h(1)z ,1 +


+ h(n , 1)z ,(n,1); (1.23)
so that T (z ) = S (z )S (z ,1). Expanding this product and equating coef-
cients of z k shows that r are the autocorrelation coecients of h. In
30 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
other words, the coecients of S give the desired spectral factorization:
H (!) = S (ej! ).
Let P (z ) = z n,1T (z ). Then P is a polynomial of degree 2(n , 1) (since
r(n , 1) 6= 0), with real coecients. Now suppose  2 C is a zero of P , i.e.,
P () = 0. Since P (0) = r(n , 1) 6= 0, we have  6= 0. Since the coecients
of P are real, we have that P () = 0, i.e.,  is also a zero of P . From the
fact that T (z ,1) = T (z ), we also see that
P (,1 ) = ,(n,1) T (,1 ) = 0;
i.e., ,1 is also a zero of P . In other words, the zeros of P (z ) are symmetric
with respect to the unit circle and also the real axis. For every zero of P
that is inside the unit circle, there is a corresponding zero outside the unit
circle, and vice versa. Moreover, our strengthened spectral factorization
condition implies that none of the zeros of P can be on the unit circle.
Now let 1 ; : : : ; n,1 be the n , 1 roots of P that are inside the unit
circle. These roots come in pairs if they are complex: if  is a root inside the
unit circle and is complex, then so is , hence 1 ; : : : ; n,1 has conjugate
symmetry. Note that the 2(n , 1) roots of P are precisely
1 ; : : : ; n,1 ; 1=1 ; : : : ; 1=n,1 :
It follows that we can factor P in the form
n,1
Y
P (z ) = c (z , i )(i z , 1);
i=1
where c is a constant. Thus we have
n,1
Y
T (z ) = z ,(n,1) P (z ) = c (1 , i z ,1)(i z , 1):
i=1
By our strengthened assumption, R(0) > 0, so
n,1
Y
R(0) = T (1) = c j1 , i j2 > 0;
i=1
so that c > 0.
Finally we can form a spectral factor. De ne
p n,1
Y
S (z ) = c (1 , i z ,1 );
i=1
so that T (z ) = S (z )S (z ,1). Since 1 ; : : : ; n,1 have conjugate symmetry,
the coecients of S are real, and provide the required FIR impulse response
coecients from (1.23).
1. FIR Filter Design via Spectral Factorization and Convex Optimization 31
The construction outlined here yields the so-called minimum-phase spec-
tral factor. (Other spectral factors can be obtained by di erent choice of
n , 1 of the 2(n , 1) roots of P ).
Polynomial factorization is the spectral factorization method used by
Herrmann and Schussler in the early paper [25]. Root nding methods that
take advantage of the special structure of P improve the method; see [18]
and the references therein. Root nding methods are generally used only
when n is small, say, several 10s.
Several other spectral factorization methods compute the spectral fac-
tor without computing the roots of the polynomial P . One group of meth-
ods [10, 50, 49] is based on the Cholesky factorization of the in nite banded
Toeplitz matrix
2 3
r(0) r(1) r(2)


r(n , 1) 0



6
6 r(1) r(0) r(1)


r(n , 2) r(n , 1)


7
7
6
6 r(2) r(1) r(0)


r(n , 3) r(n , 2)


7
7
6 .. .. .. ... 7
6
6 . . . 7
7 :
6
6 r(n , 1) r(n , 2) r(n , 3) 7
7
6
4 0 r(n , 1) r(n , 2) 7
5
.. .. ..
. . .
This matrix is positive de nite (i.e., all its principal minors are positive def-
inite) if R(!) > 0 for all !, and it was shown in [50] that the elements of the
Cholesky factors converge to the coecients of the minimum-phase spectral
factor of R. As a consequence, fast recursive algorithms for Cholesky fac-
torization of positive de nite Toeplitz matrices also yield iterative methods
for spectral factorization.
Wilson [60] and Vostry [59] have developed a method for spectral factor-
ization based on directly applying Newton's method to the set of nonlinear
equations (1.5). Their method has quadratic convergence, so very high ac-
curacy can be obtained rapidly once an approximate solution is found.
(Indeed, Newton's method can be used to re ne an approximate solution
obtained by any other method.)
Anderson et al. [3, 6] show that the minimum-phase spectral factor can
also be obtained from the solution of a discrete-time algebraic Riccati equa-
tion. They present an iterative method, based on iterating the correspond-
ing Riccati di erence equation, and retrieve from this some of the earlier
spectral factorization methods (e.g., [10]) as special cases.
We conclude by outlining a fourth method, which is based on the fast
Fourier transform (FFT) (see, e.g., [42, x7.2] or [52, x10.1]). The idea be-
hind the method is usually credited to Kolmogorov.
The method is based on the following explicit construction of the minimum-
phase spectral factor Smp (as described above) from r. It can be shown that
log Smp is de ned and analytic in the exterior of the unit disk, i.e., it can
32 Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe
be expressed as a power series in z ,1,
1
X
log Smp(z ) = ak z ,k
k=0
for jz j > 1, where ak 2 R. Now consider the real part of log Smp on the
unit circle:
1
X
< log Smp(ej! ) = log jSmp(ej! )j = (1=2) log R(!) = ak cos k!:
k=0
Therefore we can nd the coecients ak as the Fourier coecients of the
function (1=2) log R(!) (where we use the strengthened spectral factoriza-
tion condition), i.e.,
Z 2
ak = 21 (1=2) log R(!) e,jk! d!; k = 0; 1; : : : : (1.24)
0
Once the coecients ak are known, we can reconstruct Smp(ej! ) as
1
X
Smp(ej! ) = exp ak e,jk! :
k=0
The Fourier coecients of Smp give us the required impulse response coef-
cients:
1 Z 2 X1
h(t) = 2 ej!t exp ak e,jk! d!; t = 0; : : : ; n , 1: (1.25)
0 k=0
Taken together, equations (1.24) and (1.25) give an explicit construction
of the impulse response coecients of the minimum-phase spectral factor,
starting from R.
In the language of signal processing, this construction would be described
as follows. We know the log-magnitude of Smp since it is half the log-
magnitude of R, which is given (or found from r via a Fourier transform).
We apply a Hilbert transform to nd the phase of Smp, and then by expo-
nentiating get Smp (in the frequency domain). Its Fourier coecients are
the desired impulse response.
The method is applied in practice as follows. We pick n~ as a power of
two with n~  n (say, n~  15n). We use an FFT of order n~ to obtain
(from r) R(!) at n~ points uniformly spaced on the unit circle. Assuming
the result is nonnegative, we compute half its log. Another FFT yields the
coecients
P1 a0 ; : : : ; an~,1 (using (1.24)). Another FFT yields the function
a e , jk! for ! at the n~ points around the unit circle. A nal FFT
k=0 k
yields h(0); : : : ; h(n , 1) using (1.25). (In a practical implementation of this
1. FIR Filter Design via Spectral Factorization and Convex Optimization 33
algorithm symmetry and realness can be exploited at several stages, and
the last FFT can be of order n.)
FFT-based spectral factorization is ecient and numerically stable, and
is readily applied even when n, the order of the lter, is large, say several
hundred. This method has been independently rediscovered several times;
see, e.g., [12, 38].