A draft of INRIA lectures, May 1991

Wavelets, Multiresolution Analysis and Fast Numerical

Algorithms
1
G. Beylkin

I Introduction
These lectures are devoted to fast numerical algorithms and their relation to a number of
important results of Harmonic Analysis. The representation of wide classes of operators
in wavelet bases, for example, Calderón-Zygmund or pseudo-differential operators, may
be viewed as a method for their “compression”, i.e., conversion to a sparse form. The
sparsity of these representations is a consequence of the localization of wavelets in both,
space and wave number domains. In addition, the multiresolution structure of the wavelet
expansions brings about an efficient organization of transformations on a given scale and
interactions between different neighbouring scales. Such an organization of both linear
and non-linear transformations has been a powerful tool in Harmonic Analysis (usually
referred to as Littlewood-Paley and Calderón-Zygmund theories, see e.g. [33]) and now
appears to be an equally powerful tool in Numerical Analysis.
In applications, for example in image processing [12] and in seismics [22], the mul-
tiresolution methods were developed in a search for a substitute for the signal processing
algorithms based on the Fourier transform. A technique of subband coding with the ex-
act quadrature mirror filters (QMF) was introduced in [37]. It is clear that the stumbling
block on the road to both, the simpler analysis and the fast algorithms, was a limited
variety of the orthonormal bases of functional spaces. In fact, there were (with some
qualifications) only two major choices, the Fourier basis and the Haar basis. These two
bases are almost the antipodes in terms of their space–wave number (or time-frequency)
localization. Therefore, it is a remarkable discovery that besides the Fourier and Haar
bases, there is an infinite number of various orthonormal bases with a controllable lo-
calization in the space–wave number domain. The efforts in mathematics and various
applied fields culminated in the development of orthonormal bases of wavelets [39], [30],
and the notion of Multiresolution Analysis [31], [27]. There are many new constructions
of orthonormal bases with a controllable localization in the space–wave number domain,
notably [19], [16], [18], [17].
In Numerical Analysis many ingredients of Calderón-Zygmund theory were used
in the Fast Multipole algorithm for computing potential interactions [35], [24], [13]. The
1
Program in Applied Mathematics, University of Colorado at Boulder, Boulder, CO 80309-0526; Yale
University, P.O.Box 2155 Yale Station, New Haven, CT 06520

1
Fast Multipole algorithm requires order N operation to compute all the sums
qi qj
, where xi ∈ R3 i, j = 1, . . . , N.
X
pj = (1.1)
i6=j |xi − xj |

This algorithm is order N independently of the configuration of the charges, therefore

it is providing a substantial improvement over F F T , even though the problem may be
reduced to an evaluation of a convolution. The reduction of the complexity of computing
the sums in (1.1) from order N 2 to (− log )N , where  is the desired accuracy, is achieved
by approximating the far field effect of a cloud of charges located in a box by the effect of
a single multipole at the center of the box. All boxes are organized in a dyadic hierarchy
enabling an efficient O(N ) algorithm.
The fast algorithms of [7] provide a systematic generalization of the Fast Mul-
tipole method and its descendents (e.g. [34], [3], [23]) to all Calderón-Zygmund and
pseudo-differential operators. Both, the subdivision of the space into “boxes” and the
approximation of the “far field effect”, are provided adaptively (and automatically) by a
wavelet basis. The subdivision of the space and its organization in a dyadic hierarchy are
a consequence of the multiresolution properties of the wavelet bases, while the vanishing
moments of the basis functions make them useful tools for approximation.
A novel aspect of representing operators in wavelet bases is the so-called non-
standard form [7]. The remarkable feature of the non-standard form is the uncoupling
achieved among the scales. A straightforward realization, or the standard form, by
contrast, contains matrix entries reflecting “interactions” between all pairs of scales. The
non-standard form leads to an order N algorithm for evaluating operators on functions,
whereas the standard form yields, in general, only an order N log(N ) algorithm. It is
also quite remarkable that the error estimates for the non-standard form lead to a proof
of the selebrated “T(1)” theorem of David and Journé.
The direct links between practical algorithms and results of Harmonic Analy-
sis established in [7], are extended in these lectures. By generalizing the theorems of
J.M. Bony [10], [11], [9], [15] on propagation of singularities for linear and non-linear
equations in Section IX, we obtain a novel method for computing F (u) (directly in the
wavelet basis), where F is a smooth function and u is represented in a wavelet basis. The
complexity of this algorithm is automatically adaptable to the complexity of the wavelet
representation of u. This algorithm opens a number of new possibililies for solving linear
and non-linear equations and capturing singularities.
The non-standard forms of many basic operators, such as fractional derivatives,
the Hilbert and Riesz transforms, etc., may be computed explicitly [5]. The derivative
operators and, in general, the operators with homogeneous symbols have an explicit di-
agonal preconditioner in the wavelet bases. Since the condition number κp controls the
rate of convergence of a number of iterative algorithms (for example, the number of it-
√
erations of the conjugate gradient method is proportional to κp ), this simple remark
implies a completely new outlook on finite difference- and finite element- type methods.

2
These methods may be viewed as devices for reducing a partial differential equation to a
sparse linear system for the cost of an inherently high condition number of the resulting
matrices. If instead of finite difference or finite element representations, we use the rep-
resentation of the derivatives in wavelet bases, then a simple modification by a diagonal
preconditioner produces a well-conditioned system. This, in turn, “automatically” yields
fast solvers for P DE’s and removes one of the original reasons for constructing such
methods as multigrid.
Another novel and important element of computing in the wavelet bases is that the
“compressed” operators in standard and non-standard forms may be multiplied rapidly.
The product of two operators in the standard form requires C(− log )N (or maybe
C(− log )N log2 N ) operations, where  is the desired accuracy. In Section VII, we de-
scribe a new algorithm for the multiplication of operators in the non-standard form in
order (− log )N operations. Since the performance of many algorithms requiring multi-
plication of dense matrices has been limited by O(N 3 ) operations, these fast algorithms
address a critical numerical issue. Among the algorithms requiring multiplication of
matrices is an iterative algorithm for constructing the generalized inverse [36], [4], [38],
the scaling and squaring method for computing the exponential of an operator (see, for
example, [40]), and similar algorithms for sine and cosine of an operator, to mention
a few. By replacing the ordinary matrix multiplication in these algorithms by the fast
multiplication in the wavelet bases, the number of operations is reduced to, essentially,
an order N operations. For example, if both, the operator and its generalized inverse,
admit sparse representations in the wavelet basis, then the iterative algorithm [36] for
computing the generalized inverse requires only O(N log κ) operations, where κ is the
condition number of the matrix. Various numerical examples and applications may be
found in [6], [2] and [8].
As a final remark, we note, that computing in the wavelet bases has the following
three distinct attributes:
1. The operators and functions are represented in an orthonormal basis
2. The basis functions have vanishing moments leading to the sparsity of representa-
tions
3. The algorithms are recursive due to the multiresolution properties of the basis
One of the goals of these notes is to explain why (1-3) are of importance for fast
numerical algorithms. The algorithms of these notes are useful practical tools and at the
same time, directly related to several important results of Harmonic Analysis. I hope,
that the explicit interrelation between pure and numerical analysis in the algorithms of
these notes will be perceived as an accomplishment of the program of developing fast,
wavelet based algorithms for Numerical Analysis. This program was initiated jointly with
R. R. Coifman and V. Rokhlin at Yale University, and consequently, this work would not
have been possible without the collaboration with them.

3
II Preliminary Remarks

II.1 Multiresolution analysis.

We start with the definition of the multiresolution analysis. This notion, introduced
by Meyer [31], and Mallat [27], captures the essential features of all multiresolution
approaches developed so far.

Definition II.1 Multiresolution analysis is a decomposition of the Hilbert space L 2 (Rd ),

d ≥ 1, into a chain of closed subspaces

. . . ⊂ V2 ⊂ V1 ⊂ V0 ⊂ V−1 ⊂ V−2 ⊂ . . . (2.1)

such that
1. Vj = {0} and Vj is dense in L2 (Rd )
T S
j∈Z j∈Z

2. For any f ∈ L2 (Rd ) and any j ∈ Z, f (x) ∈ Vj if and only if f (2x) ∈ Vj−1
3. For any f ∈ L2 (Rd ) and any k ∈ Zd , f (x) ∈ V0 if and only if f (x − k) ∈ V0
4. There exists a scaling function ϕ ∈ V0 such that {ϕ(x − k)}k∈Zd is a Riesz basis of
V0 .

Since all constructions in these lecture notes only use orthonormal bases, we will require
the basis of Condition 4 to be an orthonormal rather than just a Riesz basis.

4’. There exists a scaling function ϕ ∈ V0 such that {ϕ(x−k)}k∈Zd is an orthonormal

basis of V0 .

By defining Wj as an orthogonal complement of Vj in Vj−1 ,

Vj−1 = Vj ⊕ Wj , (2.2)

the space L2 (Rd ) is represented as a direct sum

L2 (Rd ) =
M
Wj . (2.3)
j∈Z

If there is the coarsest scale n, then the chain of the subspaces (2.1) is replaced
by
Vn ⊂ . . . ⊂ V2 ⊂ V1 ⊂ V0 ⊂ V−1 ⊂ V−2 ⊂ . . . , (2.4)
and
L2 (Rd ) = Vn
M
Wj . (2.5)
j≤n

4
 If there is a finite number of scales, then without loss of generality, we set j = 0
to be the finest scale. Instead of (2.4), we then have

Vn ⊂ . . . ⊂ V 2 ⊂ V 1 ⊂ V 0 , V0 ⊂ L2 (Rd ). (2.6)

In numerical realizations the subspace V0 is finite dimensional.

II.2 The Haar basis

For a long time the only example of the multiresolution analysis satisfying Definition II.1
with Condition 4’ was the Haar basis [25]. Computing in the Haar basis offers a glimpse
of the algorithms that we will describe in these lectures and provides a useful prototype
for numerical experimentation. If d = 1, then the Haar basis hj,k (x) = 2−j/2 h(2−j x − k)
j, k ∈ Z, is formed by the dilation and translation of a single function

1
for 0 < x < 1/2
h(x) = −1 for 1/2 ≤ x < 1 (2.7)
0 elsewhere.



In this case ϕ(x) = χ(x), where χ(x) is the characteristic function of the interval (0, 1).
For each j, χj,k (x) = 2−j/2 χ(2−j x − k), k ∈ Z, is the basis of Vj and hj,k (x) =
2−j/2 h(2−j x − k), k ∈ Z, is the basis of Wj .
The decomposition of a function into the Haar basis is an order N procedure.
Given N = 2n “samples” of a function, which may for simplicity be thought of as values
of scaled averages of f on intervals of length 2−n ,
Z 2−n (k+1)
s0k = 2n/2 f (x)dx, (2.8)
2−n k

we obtain the Haar coefficients

1
dj+1
k = √ (sj2k−1 − sj2k ) (2.9)
2
and averages
1
sj+1
k = √ (sj2k−1 + sj2k ) (2.10)
2
for j = 0, . . . , n − 1 and k = 0, . . . , 2n−j−1 − 1. It is easy to see that evaluating the whole
set of coefficients djk , sjk in (2.9), (2.10) requires 2(N −1) additions and 2N multiplications.
In two dimensions, there are two natural ways to construct Haar systems. The first
is simply the tensor product hj,j 0 ,k,k0 (x, y) = hj,k (x)hj 0 ,k0 (y), so that each basis function
hj,j 0 ,k,k0 (x, y) is supported on a rectangle. Representing an operator in this basis leads

5
to the standard form. The second basis is defined by the set of three kinds of basis
functions supported on squares: hj,k (x)hj,k0 (y), hj,k (x)χj,k0 (y), and χj,k (x)hj,k0 (y), where
χ(x) is the characteristic function of the interval (0, 1) and χj,k (x) = 2−j/2 χ(2−j x − k).
Representing an operator in this basis leads to the non-standard form (the terminology
will become clear later).
By considering an integral operator
Z
T (f )(x) = K(x, y)f (y)dy, (2.11)

and expanding its kernel in a two-dimensional Haar basis, we find that for Calderón-
Zygmund and pseudo-differential operators the decay of entries as a function of the
distance from the diagonal is faster in these representations than that in the original
kernel. These classes of operators are given by integral or distributional kernels that
are smooth away from the diagonal. For example, kernels K(x, y) of Calderón-Zygmund
operators satisfy the estimates
1
|K(x, y)| ≤ ,
|x − y|
CM
|∂xM K(x, y)| + |∂yM K(x, y)| ≤ , (2.12)
|x − y|1+M

for some M ≥ 1. Let M = 1 in (2.12) and consider

Z Z
j
βkk 0 = K(x, y) hj,k (x) χj,k0 (y) dxdy, (2.13)

where we assume that the distance between |k − k 0 | ≥ 1. Since

Z
hj,k (x) dx = 0, (2.14)

we have (2.12)
Z Z h i
j
|βkk 0| ≤ | K(x, y) − K(x0j,k , y) hj,k (x) χj,k0 (y) dxdy|
C
≤ , (2.15)
|k − k 0 |2
j
where x0j,k = 2j (k + 21 ) denotes the center of the support of hj,k . Thus, entries βkk 0 decay

quadratically as functions of the distance between k and k 0 . Given a finite precision of

j 0
calculations, entries βkk 0 may be discarded as the distance between k and k becomes

large, leaving only a band around the diagonal.

The rate of decay depends on the number of vanishing moments of the functions of
the basis. The Haar functions have only one vanishing moment, h(x)dx = 0, and for this
R

6
reason the gain in the decay is insufficient to make computing in the Haar basis practical.
To have a faster decay, it is necessary to use basis functions with several vanishing
moments. The vanishing moments are responsible for attaining practical algorithms, i.e.
controlling the constants in the complexity estimates of the fast algorithms.

II.3 Orthonormal bases of compactly supported wavelets

The question of the existence of a multiresolution analysis with a smooth function ϕ of
Condition 4’ was not resolved until the construction of the orthonormal wavelet bases
generalizing the Haar functions by Stromberg [39] and Meyer [30] 2 . We will consider
only compactly supported wavelets with vanishing moments constructed by I. Daubechies
[19], following the work of Y. Meyer [32] and S. Mallat [28], though most of the results
will remain valid for other choices of wavelet bases as well. We will further restrict our
choice of wavelet bases by only considering, for dimensions d ≥ 2, the bases constructed
from those for d = 1.
Let us consider the multiresolution analysis for L2 (R) and let {ψ(x−k)}k∈Z be an
orthonormal basis of W0 . We will require that the function ψ has M vanishing moments,
Z +∞
ψ(x)xm dx = 0, m = 0, . . . , M − 1. (2.16)
−∞

We outline here the properties of compactly supported wavelets and refer for details to
[19], [20] and [33].
There are two immediate consequences of Definition II.1 with Condition 4’. First,
the function ϕ may be expressed as a linear combination of the basis functions of V−1 .
Since the functions {ϕj,k (x) = 2−j/2 ϕ(2−j x − k)}k∈Z form an orthonormal basis of Vj ,
we have
√ L−1
X
ϕ(x) = 2 hk ϕ(2x − k), (2.17)
k=0

which we may rewrite as

ϕ̂(ξ) = m0 (ξ/2)ϕ̂(ξ/2), (2.18)
where
1 Z +∞
ϕ̂(ξ) = √ ϕ(x) eixξ dx, (2.19)
2π −∞
and the 2π-periodic function m0 is defined as
L−1
−1/2
hk eikξ .
X
m0 (ξ) = 2 (2.20)
k=0

2
We note that the notion of multiresolution analysis is more recent than the constructions of [39],
[30] and, of course, [25].

7
 Second, the orthogonality of {ϕ(x − k)}k∈Z implies that
Z +∞ Z +∞
δk0 = ϕ(x − k)ϕ(x) dx = |ϕ̂(ξ)|2 e−ikξ dξ, (2.21)
−∞ −∞

and therefore, Z 2π
|ϕ̂(ξ + 2πl)|2 e−ikξ dξ,
X
δk0 = (2.22)
0 l∈Z

and
|ϕ̂(ξ + 2πl)|2 = 1.
X
(2.23)
l∈Z

Using (2.18), we obtain

|m0 (ξ/2 + πl)|2 |ϕ̂(ξ/2 + πl)|2 = 1.

X
(2.24)
l∈Z

By taking the sum in (2.24) separately over odd and even indeces, we have

|m0 (ξ/2+2πl)|2 |ϕ̂(ξ/2+2πl)|2 + |m0 (ξ/2+2πl+π)|2 |ϕ̂(ξ/2+2πl+π)|2 = 1. (2.25)

X X

l∈Z l∈Z

Using the 2π-periodicity of the function m0 and (2.23), we obtain, after replacing ξ/2 by
ξ, a necessary condition
|m0 (ξ)|2 + |m0 (ξ + π)|2 = 1. (2.26)
Thus, the coefficients hk in (2.20) are such that the 2π-periodic function m0 satisfies
equation (2.26).
Equation (2.26) defines a pair of the quadrature mirror filters (QMF). On denot-
ing
m1 (ξ) = e−iξ m0 (ξ + π), (2.27)
and defining the function ψ,
√ X
ψ(x) = 2 gk ϕ(2x − k), (2.28)
k

where
gk = (−1)k hL−k−1 , k = 0, . . . , L − 1, (2.29)
or, the Fourier transform of ψ,

ψ̂(ξ) = m1 (ξ/2)ϕ̂(ξ/2), (2.30)

where
k=L−1
m1 (ξ) = 2−1/2 gk eikξ ,
X
(2.31)
k=0

8
it is not difficult to show (see e.g., [33], [19], [20]), that on each fixed scale j ∈ Z, the
wavelets {ψj,k (x) = 2−j/2 ψ(2−j x − k)}k∈Z form an orthonormal basis of Wj .
The following lemma (I. Daubechies [19]) characterizes trigonometric polynomial
solutions of (2.26) which correspond to the orthonormal bases of compactly supported
wavelets with vanishing moments.

Lemma II.1 Any trigonometric polynomial solution m0 (ξ) of (2.26) is of the form
h iM
m0 (ξ) = 1
2
(1 + eiξ ) Q(eiξ ) (2.32)

where M ≥ 1 is the number of vanishing moments, and where Q is a polynomial, such

that
|Q(eiξ )|2 = P (sin2 12 ξ) + sin2M ( 21 ξ) R( 21 cos ξ), (2.33)
where
k=M
X−1
!
M −1+k
P (y) = yk, (2.34)
k=0
k
and R is an odd polynomial, such that

0 ≤ P (y) + y M R( 12 − y) for 0 ≤ y ≤ 1, (2.35)

and h i
sup P (y) + y M R( 21 − y) < 22(M −1) . (2.36)
0≤y≤1

The proof of this Lemma contains an algorithm for generating the coefficients
of the quadrature mirror filters H and G. The number L of the filter coefficients in
(2.20) and (2.31) is related to the number of vanishing moments M . For the wavelets
constructed in [19], L = 2M . If additional conditions are imposed (see [7] for an example),
then the relation might be different, but L is always even.
The decomposition of a function into the wavelet basis is an order N procedure.
Given the coefficients s0k , k = 0, 1, . . . , N as “samples” of the function f , the coefficients
sjk and djk on scales j ≥ 1 are computed at a cost proportional to N via
n=L−1
sjk = j−1
X
hn sn+2k+1 , (2.37)
n=0

and
n=L−1
djk j−1
X
= gn sn+2k+1 , (2.38)
n=0

9
where sjk and djk may be viewed as periodic sequences with the period 2n−j . Computing
via (2.37) and (2.38) is illustrated by the pyramid scheme

{s0k } −→ {s1k } −→ {s2k } −→ {s3k } · · ·

& & & (2.39)
1 2
{dk } {dk } {d3k } · · · .

II.4 Multi-wavelet bases with vanishing moments

We note that the Haar system is a degenerate case of Daubechies’s wavelets with M = 1.
There is, however, a different construction of orthonormal bases for L2 ([0, 1]) or L2 (R),
which generalizes the Haar system and yields basis functions with several vanishing
moments [2], [1].
Let us construct M functions, f1 , . . . , fM : R → R, supported on the interval
[−1, 1] and such that (i) on the interval (0, 1) the function fi is a polynomial of the
degree M − 1. Let us (ii) extend fi to the interval (−1, 0) as an even or odd function
according to the parity of i + M − 1. We require these functions to be orthogonal (iii)
Z 1
fi (x) fl (x) dx = δil , i, l = 1, . . . , M. (2.40)
−1

and have vanishing moments (iv)

Z 1
fi (x) xm dx = 0, m = 0, 1, . . . , i + M − 2. (2.41)
−1

The properties (i) and (ii) imply that there are M 2 polynomial coefficients that determine
the functions f1 , . . . , fM , while the properties (iii) and (iv) provide M 2 constraints. It
turns out that the equations uncouple to give M nonsingular linear systems which may
be solved to obtain the coefficients and, thus, yielding the functions f1 , . . . , fM up to a
sign.
The functions f1 , . . . , fM may be obtained constructively as follows. Let us start
with
m−1

 x
 , x ∈ (0, 1),
1
fm (x) =  −xm−1 , x ∈ (−1, 0), (2.42)

0, otherwise,
and note that the 2M functions 1, x, . . . , xM −1 , f11 , f21 , . . . , fM
1
are linearly independent.
Then, by the Gram-Schmidt process, we orthogonalize fm with respect to 1, x, . . . , xM −1 ,
1
2
to obtain fm , for m = 1, . . . , M . This orthogonality is preserved by the remaining or-
2
thogonalizations, which only produce linear combinations of the fm . First, if at least
2
one of fm is not orthogonal to x , we reorder the functions, so that hf12 , xM i 6= 0.
M

10
 3 2
We then define fm = fm − am · f02 , where am is chosen so that hfm 3
, xM i = 0 for
m = 2, . . . , M , achieving the desired orthogonality to xM . Similarly, we continue to
M +1
orthogonalize with respect to xM +1 , . . . , x2M −2 to obtain f12 , f23 , f34 , . . . , fM , such that
m+1
hfm , xi i = 0 for i ≤ m + M − 2. Finally, we perform the Gram-Schmidt orthogonal-
M +1 M 2
ization on fM , fM −1 , . . . , f1 , in that order, and normalize to obtain fM , fM −1 , . . . , f1 .
m=M
By construction, the functions {fm }m=1 satisfy properties (i)-(iv). On denoting
hm (x) = 21/2 fm (2x − 1), m = 1, . . . , M, (2.43)
we define the space WjM , j = 0, −1, −2, . . ., as a linear span of functions

hnm,j (x) = 2−j/2 hm (2−j x − n), m = 1, . . . , M ; n = 0, . . . , 2−j − 1. (2.44)

We also define V0M to be the space of polynomials of degree less than M . By defining
recursively
M
= VjM WjM ,
M
Vj−1 (2.45)
starting with j = 0, we obtain multiresolution analysis
V0M ⊂ V−1
M
⊂ · · · ⊂ VjM ⊂ · · · . (2.46)
It is not difficult to check that
L2 ([0, 1]) = V0M WjM .
M
(2.47)
j≤0

Also, it is easy to see that the orthonormal set

hnm,j (x) = 2−j/2 hm (2−j x − n), m = 1, . . . , M ; j, n ∈ Z. (2.48)
is an orthonormal basis of L2 (R).
We refer to bases constructed in this manner as multi-wavelet bases of order
M . Algorithms, various numerical examples, and applications utilizing these bases are
described in [2] and [1]. It is interesting to note that the multi-wavelet bases have been
constructed only after “ordinary” wavelets had been used in algorithms of [7], though
their construction addresses the problem of vanishing moments directly and does not
require any prior knowledge of the wavelet theory. If the number of vanishing moments
M ≥ 2, then, on a given scale j, the multi-wavelet basis functions with different labels n
do not have an overlaping support unlike the “ordinary” compactly supported wavelets.
On the other hand, the non-standard form of operators in the multi-wavelet bases is
more complicated than that in the bases of the “ordinary” wavelets. This becomes clear
once we outline the construction of bases for L2 [0, 1]d and L2 (Rd ), for any dimension d.
We describe this extension by giving the basis for L2 ([0, 1]2 ), which is illustrative of the
construction for any finite-dimensional space. Let us define the space VjM,2 as

VjM,2 = VjM × VjM , j = 0, −1, −2, . . . , (2.49)

11
where VjM is given in (2.45), and the space WjM,2 as the orthogonal complement of VjM,2
M,2
in Vj−1 ,
M,2
= VjM,2 WjM,2 .
M
Vj−1 (2.50)
The space W0M,2 is spanned by the orthonormal basis

{ui (x)hl (y), hi (x)ul (y), hi (x)hl (y) : i, l = 1, . . . , M }, (2.51)

m=M
where {um }m=1 is an orthonormal basis for V0M . Each element among these 3M 2 basis
elements has vanishing moments, i.e., it is orthogonal to the polynomials xi y l , i, l =
0, 1, . . . , M − 1.
The space WjM,2 is spanned by dilations and translations of the basis functions of
W0M,2 and the basis of L2 ([0, 1]2 ) consists of these functions and the low-order polynomials
xi y l , i, l = 0, 1, . . . , M − 1.
We note that the two-dimensional multi-wavelet bases require 3M 2 different com-
binations of one-dimensional basis functions, where M is the number of vanishing mo-
ments. On the other hand, the two-dimensional bases, obtained by using compactly
supported wavelets [19], require only three such combinations which simplifies the con-
struction of the non-standard form (see Section III).

II.5 A remark on computing in the wavelet bases

Finally, we note that once the filter H has been chosen, it completely determines the
functions ϕ and ψ and therefore, the multiresolution analysis. It is an interesting observa-
tion, that in properly constructed algorithms, the functions ϕ and ψ are never computed.
Due to the recursive definition of the wavelet bases, all the manipulations are performed
with the quadrature mirror filters H and G, even if they involve quantaties associated
with ϕ and ψ. As an example, let us compute the moments of the scaling function φ.
The expressions for the moments,
Z
Mm
∞ = xm ϕ(x) dx, m = 0, . . . , M − 1, (2.52)

k=L
in terms of the filter coefficients {hk }k=1 , may be found using a formula for ϕ̂ ,
∞
−1/2
m0 (2−j ξ),
Y
ϕ̂(ξ) = (2π) (2.53)
j=1

where
k=L−1
−1/2
hk eikξ .
X
m0 (ξ) = 2 (2.54)
k=0

12
 The moments Mm ∞ are obtained numerically (within the desired accuracy) by
m=M −1
recursively generating a sequence of vectors, {Mm
r }m=0 for r = 1, 2, . . . ,
j=m !
m
Mm 2−jr Mrm−j Mj1 ,
X
r+1 = (2.55)
j=0
j

starting with
k=L−1
1
Mm −m− 2
hk k m ,
X
1 = 2 m = 0, . . . , M − 1. (2.56)
k=0
m=M −1
Each vector {Mm
r }m=0 represents M moments of the product in (2.53) with r terms,
and this iteration converges rapidly. Notice, that we never computed the function ϕ
itself.

13
III The non-standard and standard forms

III.1 The Non-Standard Form

Let T be an operator
T : L2 (R) → L2 (R), (3.1)
with the kernel K(x, y). Defining projection operators on the subspace Vj , j ∈ Z,

Pj : L2 (R) → Vj , (3.2)

as X
(Pj f ) (x) = hf, ϕj,k iϕj,k (x), (3.3)
k

and expanding T in a “telescopic” series, we obtain

X
T = (Qj T Qj + Qj T Pj + Pj T Qj ), (3.4)
j∈Z

where
Qj = Pj−1 − Pj (3.5)
is the projection operator on the subspace Wj . If there is the coarsest scale n, then
instead of (3.4) we have
n
X
T = (Qj T Qj + Qj T Pj + Pj T Qj ) + Pn T Pn , (3.6)
j=−∞

and if the scale j = 0 is the finest scale, then

n
X
T0 = (Qj T Qj + Qj T Pj + Pj T Qj ) + Pn T Pn , (3.7)
j=1

where T ∼ T0 = P0 T P0 is a discretization of the operator T on the finest scale. Expan-

sions (3.4), (3.6) and (3.7) decompose the operator T into a sum of contributions from
different scales.
The non-standard form is a representation (see [7]) of the operator T as a chain
of triplets
T = {Aj , Bj , Γj }j∈Z (3.8)
acting on the subspaces Vj and Wj ,

A j : Wj → W j , (3.9)

B j : Vj → W j , (3.10)

14
 Γj : Wj → V j , (3.11)
where the operators {Aj , Bj , Γj }j∈Z are defined as Aj = Qj T Qj , Bj = Qj T Pj and
Γj = P j T Q j .
The operators {Aj , Bj , Γj }j∈Z admit a recursive definition via the relation
!
Aj+1 Bj+1
Tj = , (3.12)
Γj+1 Tj+1

where operators Tj = Pj T Pj ,
Tj : V j → V j , (3.13)
and the operator represented by the 2 × 2 matrix in (3.12) is a mapping
!
Aj+1 Bj+1
: Wj+1 ⊕ Vj+1 → Wj+1 ⊕ Vj+1 (3.14)
Γj+1 Tj+1
.
If there is a coarsest scale n, then

T = {{Aj , Bj , Γj }j∈Z:j≤n, Tn }, (3.15)

where Tn = Pn T Pn . If the number of scales is finite, then j = 1, 2, . . . , n in (3.15) and

the operators are organized as blocks of the matrix (see Figures 1 and 2).
Let us make the following observations:

1. The operator Aj describes the interaction on the scale j only, since the subspace
Wj in (3.9) is an element of the direct sum in (2.3).

2. The operators Bj , Γj in (3.10) and (3.11) describe the interaction between the scale
j and all coarser scales. Indeed, the subspace Vj contains all the subspaces Vj 0
with j 0 > j (see (2.1)).

3. The operator Tj is an “averaged” version of the operator Tj−1 .

The operators Aj , Bj and Γj are represented by the matrices αj , β j and γ j ,

Z Z
j
αk,k 0 = K(x, y) ψj,k (x) ψj,k0 (y) dxdy, (3.16)
Z Z
j
βk,k 0 = K(x, y) ψj,k (x) ϕj,k0 (y) dxdy, (3.17)
and Z Z
j
γk,k 0 = K(x, y) ϕj,k (x) ψj,k0 (y) dxdy. (3.18)

15
 Α1 Β1 d1 d^ 1

Γ1 s1 ^s 1

=
^
Α2 Β2 d2 d2

s2 ^s 2
Γ2

Α 3 Β3 d3 ^3
d
Γ 3Τ 3 s3 ^s 3

Figure 1: Organization of the non-standard form of a matrix. The submatrices Aj , Bj ,

and Γj , j = 1, 2, 3, and T3 are the only non-zero submatrices.

16
Figure 2: An example of a matrix in the non-standard form (see Example 1)

17
The operator Tj is represented by the matrix sj ,
Z Z
sjk,k0 = K(x, y) ϕj,k (x) ϕj,k0 (y) dxdy. (3.19)

Given a set of coefficients s0k,k0 with k, k 0 = 0, 1, . . . , N − 1, a repeated application of the

formulas (2.37), (2.38) produces
L−1
j j−1
X
αi,l = gk gm sk+2i+1,m+2l+1 , (3.20)
k,m=0

L−1
j j−1
X
βi,l = gk hm sk+2i+1,m+2l+1 , (3.21)
k,m=0
L−1
j j−1
X
γi,l = hk gm sk+2i+1,m+2l+1 , (3.22)
k,m=0
L−1
sji,l = j−1
X
hk hm sk+2i+1,m+2l+1 , (3.23)
k,m=0

with i, l = 0, 1, . . . , 2n−j − 1, j = 1, 2, . . . , n. Clearly, formulas (3.20) - (3.23) provide

an order N 2 scheme for the evaluation of the elements of all matrices αj , β j , γ j with
j = 1, 2 . . . , n.
A fast scheme for the evaluation of the elements of matrices α j , β j , γ j with j =
1, 2 . . . , n, requires that the locations of the singularities were known in advance. Given
the locations of the singularities, only the significant coefficients above a threshold of ac-
curacy are computed, and the only issue is how to compute averages sji,l in a neighborhood
of the singularities without (3.23). We refer to [7] for the details of such computations
using quadrature formulas.

Remark. It follows from (3.7) that after applying the non-standard form to a vector,
we arrive at the representation
j=n j=n
dˆjk ψkj (x) + ŝjk φjk (x),
X X X X
(T0 u0 )(x) = (3.24)
j=1 k∈Z j=1 k∈Z

where u0 = P0 u (see 1). We note, that in order to reconstruct a vector from the first and
the second sums in (3.24), we may apply the same algorithm. The reconstruction from
the first sum is just a reconstruction from the wavelet expansion and is accomplished by
using the quadrature mirror filters H and G. The reconstruction from the second sum
may be accomplished by using the same algorithm but with the pair of filters H and H.
Both results are then added. The number of operations required for this computation is
proportional to N .

18
III.2 The Standard Form
The standard form is obtained by representing
M
Vj = Wj 0 , (3.25)
j 0 >j

0 0
and considering for each scale j the operators {Bjj , Γjj }j 0 >j ,
0
Bjj : Wj 0 → Wj , (3.26)
0
Γjj : Wj → Wj 0 . (3.27)
If there is the coarsest scale n, then instead of (3.25), we have
0 =n
jM
Vj = V n Wj 0 . (3.28)
j 0 =j+1

0 0
In this case, the operators {Bjj , Γjj } for j 0 = j + 1, . . . , n are the same as in (3.26) and
(3.27) and, in addition, for each scale j, there are operators {Bjn+1 } and {Γn+1 j },

Bjn+1 : Vn → Wj , (3.29)

Γn+1
j : Wj → V n . (3.30)
(In this notation, Γn+1
n = Γn and Bnn+1 = Bn ). If the number of scales is finite and V0 is
finite dimensional, then the standard form is a representation of T0 = P0 T P0 as
0 0 0 0
T0 = {Aj , {Bjj }jj 0 =n j j =n n+1
=j+1 , {Γj }j 0 =j+1 , Bj , Γn+1
j , Tn }j=1,...,n . (3.31)

The operators (3.31) are organized as blocks of the matrix (see Figure 3 and Figure 4).
If the operator T is a Calderón-Zygmund or a pseudo-differential operator then,
for a fixed accuracy, all the operators in (3.31), except Tn , are banded. As a result, the
standard form has several “finger” bands which correspond to the interaction between
different scales. For a large class of operators (pseudo-differential, for example), the
interaction between different scales characterized by the size of the coefficients of “finger”
bands decays as the distance j 0 − j between the scales increases. Therefore, if the scales
0 0
j and j 0 are well separated, then for a given accuracy, the operators Bjj , Γjj can be
neglected.
There are two ways of computing the standard form of a matrix. First consists in
applying the one-dimensional transform (see (2.37) and (2.38)) to each column (row) of
the matrix and then to each row (column) of the result. Alternatively, one can compute
the non-standard form and then apply the one-dimensional transform to each row of all
operators Bj and each column of all operators Γj . We refer to [7] the for details.

19
 Α1 Β 21 Β 31 Β 41 d1 d^ 1

=
^
Γ 21 Α 2 Β 32 Β 42 d2 d2

Γ 31 Γ 32 Α 3 Β 43 d3 ^3
d
Γ 41 Γ 42 Γ 43 Τ 3 s3 ^s 3

Figure 3: Organization of a matrix in the standard form

Figure 4: An example of a matrix in the standard form (see Example 1)

20
IV The compression of operators
The compression of operators or, in other words, the construction of their sparse repre-
sentations in orthonormal bases, directly affects the speed of computational algorithms.
While the compression of data (of images, for example) achieved by methods other than
finding a sparse representation in some basis may be adequate for some applications, the
compression of operators calls for a representation in a basis in order to effectively com-
pute in the sparse form. The standard and non-standard forms of operators in the wavelet
bases may be viewed as compression schemes for a wide class of operators. We restrict
our attention to two specific classes frequently encountered in analysis and applications,
Calderón-Zygmund and pseudo-differential operators.
The non-standard form of an operator T with a kernel K(x, y) is obtained by
evaluating the expressions
Z Z
αII 0 = K(x, y) ψI (x) ψI 0 (y) dxdy, (4.1)
Z Z
βII 0 = K(x, y) ψI (x) ϕI 0 (y) dxdy, (4.2)
and Z Z
γII 0 = K(x, y) ϕI (x) ψI 0 (y) dxdy, (4.3)

where the coefficients are labeled by the intervals I = Ikj and I 0 = Ikj0 denoting the
supports of ϕjk and ϕjk0 .
If K is smooth on the square I × I 0 , then expanding K into a Taylor series around
the center of the square and combining (2.16) with (4.1) - (4.3) and remembering that
the functions ψI and ψI 0 are supported on the intervals I and I 0 , we obtain the estimate
 
|αII 0 | + |βII 0 | + |γII 0 | ≤ C|I|M +1 sup |∂xM K(x, y)| + |∂yM K(x, y)| . (4.4)
(x,y)∈I×I 0

Obviously, the right-hand side of (4.4) is small whenever either |I| or the derivatives
involved are small, and we use this fact to “compress” matrices of integral operators
by converting them to the non-standard form, and discarding the coefficients that are
smaller than a chosen threshold.
For most numerical applications, the following Proposition IV.1 is quite adequate,
as long as the singularity of K is integrable across each row and each column.

Proposition IV.1 If the wavelet basis has M vanishing moments, then for any kernel
K satisfying the conditions
1
|K(x, y)| ≤ , (4.5)
|x − y|
C0
|∂xM K(x, y)| + |∂yM K(x, y)| ≤ (4.6)
|x − y|1+M

21
the matrices αj , β j , γ j (3.16) - (3.18) of the non-standard form satisfy the estimate

j j j CM
|αi,l | + |βi,l | + |γi,l |≤ , (4.7)
1 + |i − l|M +1

for all |i − l| ≥ 2M .

Similar considerations apply in the case of pseudo-differential operators. Let T

be a pseudo-differential operator with symbol σ(x, ξ) defined by the formula
Z Z
T (f )(x) = eixξ σ(x, ξ)fˆ(ξ) dξ = K(x, y)f (y) dy, (4.8)

where K is the distributional kernel of T .

Proposition IV.2 If the wavelet basis has M vanishing moments, then for any pseudo-
differential operator with symbol σ of T and σ ∗ of T ∗ satisfying the standard conditions

|∂ξα ∂xβ σ(x, ξ)| ≤ Cα,β (1 + |ξ|)λ−α+β (4.9)

|∂ξα ∂xβ σ ∗ (x, ξ)| ≤ Cα,β (1 + |ξ|)λ−α+β , (4.10)

the matrices αj , β j , γ j (3.16) - (3.18) of the non-standard form satisfy the estimate

j j j 2 λ j CM
|αi,l | + |βi,l | + |γi,l |≤ , (4.11)
(1 + |i − l|)M +1

for all integer i, l.

If we approximate the operator T0N by the operator T0N,B obtained from T0N by
j j j
setting to zero all coefficients of matrices αi,l , βi,l , and γi,l outside of bands of width
B ≥ 2M around their diagonals, then it is easy to see that
C
||T0N,B − T0N || ≤ log2 N, (4.12)
BM
where C is a constant determined by the kernel K. In most numerical applications, the
accuracy  of calculations is fixed, and the parameters of the algorithm (in our case,
the band width B and order M ) have to be chosen in such a manner that the desired
precision of calculations is achieved. If M is fixed, then B has to be such that
C
||T0N,B − T0N || ≤ log2 N ≤ , (4.13)
BM

22
or, equivalently,
1/M
C
 
B≥ log2 N . (4.14)
ε
The estimate (4.12) is sufficient for practical purposes. It is possible, however, to
obtain
C
||T0N,B − T0N || ≤ (4.15)
BM
instead of (4.12). In order to obtain this tighter estimate and avoid the factor log 2 N , we
use some of the ideas arising in the proof of the “T(1)” theorem of David and Journé.
j j j
The estimates (4.5), (4.6) are not sufficient to conclude that αi,l , βi,l , γi,l are
1
bounded for |i − l| ≤ 2M (for example, consider K(x, y) = |x−y| ). Therefore, we need to
assume that T defines a bounded operator on L2 or, a substantially weaker condition,
Z
| K(x, y) dxdy| ≤ C|I|, (4.16)
I×I

for all dyadic intervals I. This is the so-called “weak cancellation condition” (see [32]).
Under this condition and the conditions (4.5), (4.6) Proposition IV.1 may be extended
so that the estimate (4.7) holds for all integer i, l (see [32]).
Let us evaluate an operator T in the non-standard form on a function f ,
X X X
T (f )(x) = ψI (x) αII 0 dI 0 + ψI (x) βII 0 sI 0 + ϕI (x) γII 0 dI 0 (4.17)
I,I 0 I,I 0 I,I 0

and write (4.17) as a sum of three terms,

T = A + Lβ + L∗γ , (4.18)
X X X
A(f )(x) = ψI (x) αI,I 0 dI 0 + ψI (x) βII 0 (sI 0 − sI ) + (ϕI (x) − ϕI 0 (x)) γII 0 dI 0 (4.19)
I,I 0 I,I 0 I,I 0
X
Lβ (f )(x) = ψI (x) sI βI , (4.20)
I

L∗γ (f )(x) =
X
ϕI 0 (x) dI 0 γI 0 (4.21)
I0

where
1 1
X Z Z
βI = βII 0 = ψI (x) K(x, y) dxdy = hψI , β(x)i (4.22)
I0 |I|1/2 |I|1/2
1 1
X Z Z
γ =
I0 γ II 0 = 0 1/2 ψI 0 (y) K(x, y) dxdy = hψI 0 , γ(y)i (4.23)
I |I | |I 0|1/2
and
β(x) = T (1)(x), (4.24)

23
 γ(y) = T ∗ (1)(y) (4.25)
It is a remarkable fact that by analysing the functions (4.24) and (4.25) (and, therefore,
the operators Lβ and L∗γ ), it is possible to decide if a Calderón-Zygmund operator is
bounded.

Theorem IV.1 (G. David, J.L. Journé) Suppose that the operator (3.1) satisfies the
conditions (4.5), (4.6), and (4.16). Then a necessary and sufficient condition for T to
be bounded on L2 is that β(x) in (4.24) and γ(y) in (4.25) belong to dyadic B.M.O., i.e.
satisfy condition
1
Z
sup |β(x) − mJ (β)| 2 dx ≤ C, (4.26)
J |J| J

where J is a dyadic interval and

1 Z
mJ (β) = β(x)dx. (4.27)
|J| J

Splitting the operator T into the sum of three terms (4.18) and estimating them
separately leads to the estimate (4.15). We note that the functions T (1) and T ∗ (1) are
easily computed in the process of constructing the non-standard form and and may be
used to provide a useful estimate of the norm of the operator.

24
V The differential operators in wavelet bases.

V.1 The operator d/dx in wavelet bases

The non-standard forms of several commonly used operators may be computed explicitly.
First we construct the non-standard form of the operator d/dx. The matrix elements αilj ,
βilj , γilj of Aj , Bj , Γj , and rilj of Tj = Pj T Pj , i, l, j ∈ Z, for the operator d/dx are easily
computed as
Z ∞
αilj =2 −j
ψ(2−j x − i) ψ 0 (2−j x − l) 2−j dx = 2−j αi−l , (5.1)
−∞
Z ∞
βilj =2 −j
ψ(2−j x − i) ϕ0 (2−j x − l) 2−j dx = 2−j βi−l , (5.2)
−∞
Z ∞
γilj = 2−j ϕ(2−j x − i) ψ 0 (2−j x − l) 2−j dx = 2−j γi−l , (5.3)
−∞

and Z ∞
rilj = 2−j ϕ(2−j x − i) ϕ0 (2−j x − l) 2−j dx = 2−j ri−l , (5.4)
−∞

where
+∞ d
Z
αl = ψ(x − l)
ψ(x) dx, (5.5)
−∞ dx
d
Z +∞
βl = ψ(x − l) ϕ(x) dx, (5.6)
−∞ dx
d
Z +∞
γl = ϕ(x − l) ψ(x) dx. (5.7)
−∞ dx
and
+∞ d
Z
rl = ϕ(x − l) ϕ(x) dx. (5.8)
−∞ dx
Moreover, using (2.17) and (2.28) we have
L−1
X L−1
X
αi = 2 gk gk0 r2i+k−k0 , (5.9)
k=0 k 0 =0

L−1
X L−1
X
βi = 2 gk hk0 r2i+k−k0 , (5.10)
k=0 k 0 =0

and
L−1
X L−1
X
γi = 2 hk gk0 r2i+k−k0 , (5.11)
k=0 k 0 =0

25
and, therefore, the representation of d/dx is completely determined by rl in (5.8) or in
other words, by the representation of d/dx on the subspace V0 .
Rewriting (5.8) in terms of ϕ̂(ξ), where
1 +∞
Z
ϕ̂(ξ) = √ ϕ(x) eixξ dx, (5.12)
2π −∞

we obtain Z +∞
rl = (−iξ) |ϕ̂(ξ)|2 e−ilξ dξ. (5.13)
−∞

We prove the following

Proposition V.1 1. If the integrals in (5.8) or (5.13) exist, then the coefficients r l ,
l ∈ Z in (5.8) satisfy the following system of linear algebraic equations
 
L/2
1
X
rl = 2 r2l + 2
a2k−1 (r2l−2k+1 + r2l+2k−1 ) , (5.14)
k=1

and X
l rl = −1, (5.15)
l
where
L−2k
X
a2k−1 = 2 hi hi+2k−1 , k = 1, . . . , L/2. (5.16)
i=0

2. If M ≥ 2, then equations (5.14) and (5.15) have a unique solution with a finite
number of non-zero rl , namely, rl 6= 0 for −L + 2 ≤ l ≤ L − 2 and

rl = −r−l , (5.17)

Remark 1. If M = 1, then equations (5.14) and (5.15) have a unique solution but
the integrals in (5.8) or (5.13) may not be absolutely convergent. For the Haar basis
(h1 = h2 = 2−1/2 ) a1 = 1 and r1 = −1/2 and we obtain the simplest finite difference
operator (1/2, 0, −1/2). In this case the function ϕ is not continuous and
1 sin 21 ξ i 1 ξ
ϕ̂(ξ) = √ e2 .
2π 12 ξ

Any trigonometric polynomial m0 (ξ) satisfying (2.26) may be written as

L/2
|m0 (ξ)|2 = 1 1
X
2
+ 2
a2k−1 cos(2k − 1)ξ, (5.18)
k=1

26
where an are the autocorrelation coefficients of the filter H = {hk }k=L−1
k=0 ,
L−1−n
X
an = 2 hi hi+n , n = 1, . . . , L − 1. (5.19)
i=0

It is easy to see that the autocorrelation coefficients an with even indices are zero,

a2k = 0, k = 1, . . . , L/2 − 1. (5.20)

This may be verifyed by using (2.20) to compute |m0 (ξ)|2 and |m0 (ξ + π)|2 ,
L−1
2 1 1
X
|m0 (ξ)| = 2
+ 2
an cos nξ, (5.21)
n=1

L/2 L/2−1
|m0 (ξ + π)|2 = 1 1 1
X X
2
− 2
a2k−1 cos(2k − 1)ξ + 2
a2k cos 2kξ, (5.22)
k=1 k=1

where an are given in (5.19). Combining (5.21) and (5.22) to satisfy (2.26), we obtain
L/2−1
X
a2k cos 2kξ = 0, (5.23)
k=1

and hence, (5.20) and (5.18).

The even moments of the coefficients a2k−1 from (5.19) vanish, namely
k=L/2
a2k−1 (2k − 1)2m = 0 for 1 ≤ m ≤ M − 1,
X
(5.24)
k=1

since m
1
 
2
∂ξ |m0 (ξ)| = 0, for 1 ≤ m ≤ 2M − 1, (5.25)
i ξ=0

which follows from the explicit representation in (2.32).

Proof.
d
Using (2.17) for both ϕ(x − l) and dx
ϕ(x) in (5.8) we obtain
L−1
X L−1 Z +∞
ϕ(2x − 2i − k) ϕ0 (2x − l) 2 dx
X
ri = 2 hk hl (5.26)
k=0 l=0 −∞

and hence,
L−1
X L−1
X
ri = 2 hk hl r2i+k−l . (5.27)
k=0 l=0

27
Substituting l = k − m, we rewrite (5.27) as
L−1
X k−L+1
X
ri = 2 hk hk−m r2i+m . (5.28)
k=0 m=k
PL−1
Changing the order of summation in (5.28) and using the fact that k=0 h2k = 1, we
arrive at
L−1
X
rl = 2r2l + an (r2l−n + r2l+n ), l ∈ Z, (5.29)
n=1

where an are given in (5.19). Using (5.20) we obtain (5.14) from (5.29).
In order to obtain (5.15) we use the following relation
l=+∞ l=m
!
m
m m l
Mlϕ xm−l ,
X X
l ϕ(x − l) = x + (−1) (5.30)
l=−∞ l=1
l

where Z +∞
Mlϕ = ϕ(x) xl dx, l = 1, . . . , m, (5.31)
−∞

are the moments of the function ϕ(x). Relation (5.31) follows simply on taking Fourier
transforms and using Leibniz’ rule. Using (5.8) and (5.30) with m = 1 we obtain (5.15).
If M ≥ 2, then
|ϕ̂(ξ)|2 |ξ| ≤ C(1 + |ξ|)−1−, (5.32)
where  > 0, and hence, the integral in (5.13) is absolutely convergent. This assertion
follows from Lemma 3.2 of [19], where it is shown that
|ϕ̂(ξ)| ≤ C(1 + |ξ|)−M +log2 B , (5.33)
where
B = sup |Q(eiξ )|.
ξ∈R

Due to the condition (2.36), we have log2 B = M − 1 −  with some  > 0.

The existence of a solution of the system of equations (5.14) and (5.15) follows
from the existence of the integral in (5.13). Since the scaling function ϕ has a compact
support there are only finite number of non-zero coefficients rl . The specific interval
−L + 2 ≤ l ≤ L − 2 is obtained by the direct examination of (5.14).
Let us show now that X
rl = 0. (5.34)
l
ilξ
Multiplying (5.14) by e and summing over l, we obtain
 
L/2  
r̂(ξ) = 2  r̂even (ξ/2) + 21 r̂odd (ξ/2) a2k−1 e−i(2k−1)ξ/2 + ei(2k−1)ξ/2  ,
X
(5.35)
k=1

28
where
rl eilξ ,
X
r̂(ξ) = (5.36)
l

r2l eilξ ,
X
r̂even (ξ/2) = (5.37)
l

and
r2l+1 ei(2l+1) ξ/2 .
X
r̂odd (ξ/2) = (5.38)
l

Noticing that
2 r̂even (ξ/2) = r̂(ξ/2) + r̂(ξ/2 + π) (5.39)
and
2 r̂odd (ξ/2) = r̂(ξ/2) − r̂(ξ/2 + π), (5.40)
and using (5.18), we obtain from (5.35)
h   i
r̂(ξ) = r̂(ξ/2) + r̂(ξ/2 + π) + 2|m0 (ξ/2)|2 − 1 (r̂(ξ/2) − r̂(ξ/2 + π) ) . (5.41)

Finally, using (2.26) we arrive at

 
r̂(ξ) = 2 |m0 (ξ/2)|2 r̂(ξ/2) + |m0 (ξ/2 + π)|2 r̂(ξ/2 + π) . (5.42)

Setting ξ = 0 in (5.42) we obtain r̂(0) = 2r̂(0) and, thus, (5.34).

Uniqueness of the solution of (5.14) and (5.15) follows from the uniqueness of
the representation of d/dx. Given the solution rl of (5.14) and (5.15) we consider the
operator Tj defined by these coefficients on the subspace Vj and apply it to a sufficiently
smooth function f . Since rlj = 2−j rl (5.4), we have
!
−j
X X
(Tj f )(x) = 2 rl fj,k−l ϕj,k (x), (5.43)
k∈Z l

where Z +∞
−j/2
fj,k−l = 2 f (x) ϕ(2−j x − k + l) dx. (5.44)
−∞

Rewriting (5.44) as
Z +∞
−j/2
fj,k−l = 2 f (x − 2j l) ϕ(2−j x − k) dx, (5.45)
−∞

and expanding f (x − 2j l) in the Taylor series at the point x, we have

+∞ +∞ l2 +∞
Z Z Z
j 0 2j
fj,k−l = f (x) ϕj,k (x) dx − 2 l f (x) ϕj,k (x) dx + 2 f 00 (x̃) ϕj,k (x) dx,
−∞ −∞ 2 −∞
(5.46)

29
where x̃ = x̃(x, x − 2j l) and |x̃ − x| ≤ 2j l. Substituting (5.46) in (5.43) and using (5.34)
and (5.15), we obtain
X Z +∞ 
0
(Tj f )(x) = f (x) ϕj,k (x) dx ϕj,k (x) +
k∈Z −∞
!
Z +∞
j 1 2 00
X X
2 2
rl l f (x̃) ϕj,k (x) dx ϕj,k (x). (5.47)
k∈Z l −∞

It is clear that as j → −∞, operators Tj and d/dx coincide on smooth functions. Using
standard arguments it is easy to prove that T−∞ = d/dx and hence, the solution to (5.14)
and (5.15) is unique. The relation (5.17) follows now from (5.13).

Remark 2. We note that expressions (5.9) and (5.10) for αl and βl (γl = −β−l ) may
be simplified by changing the order of summation in (5.9) and (5.10) and introducing
PL−1−n PL−1−n
the correlation coefficients 2 i=0 gi hi+n , 2 i=0 hi gi+n and 2 L−1−n gi gi+n . The
P
i=0
expression for αl is especially simple, αl = 4r2l − rl .

Examples. For the examples we will use Daubechies’ wavelets constructed in [19]. First,
let us compute the coefficients a2k−1 , k = 1, . . . , M , where M is the number of vanishing
moments and L = 2M . Using relation (4.22) of [19],

(2M − 1)! ξ
Z
2
|m0 (ξ)| = 1 − sin2M −1 ξ dξ, (5.48)
[(M − 1)!]2 22M −1 0

Rξ
we find, by computing 0 sin2M −1 ξ dξ, that
M
(−1)m−1 cos(2m − 1)ξ
|m0 (ξ)|2 = 1
+ 12 CM
X
2
, (5.49)
m=1 (M − m)! (M + m − 1)! (2m − 1)

where " #2
(2M − 1)!
CM = . (5.50)
(M − 1)! 4M −1
Thus, by comparing (5.49) and (5.18), we have

(−1)m−1 CM
a2m−1 = , where m = 1, . . . , M. (5.51)
(M − m)! (M + m − 1)! (2m − 1)
We note that by virtue of being solutions of a linear system with rational co-
efficients (a2m−1 in (5.51) are rational by construction), the coefficients rl are rational
numbers. The coefficients rl are the same for all Daubechies’ wavelets with a fixed number
of vanishing moments M , while there are several wavelet bases for a given M depending
on the choice of the roots of polynomials in the construction described in [19].

30
 Solving equations of Proposition 1, we present the results for Daubechies’ wavelets
with M = 2, 3, 4, 5, 6.
1. M = 2
9 1
a1 = , a3 = − ,
8 8
and
2 1
r1 = − , r2 = ,
3 12
The coefficients (−1/12, 2/3, 0, −2/3, 1/12) of this example can be found in many books
on numerical analysis as a choice of coefficients for numerical differentiation.

2. M = 3
75 25 3
a1 = , a3 = − , a5 = ,
64 128 128
and
272 53 16 1
r1 = − , r2 = , r3 = − , r4 = − .
365 365 1095 2920

3. M = 4
1225 245 49 5
a1 = , a3 = − , a5 = , a7 = − ,
1024 1024 1024 1024
and
39296 76113 1664
r1 = − , r2 = , r3 = − ,
49553 396424 49553
2645 128 1
r4 = , r5 = , r6 = − .
1189272 743295 1189272

4. M = 5
19845 2205 567 405 35
a1 = , a3 = − , a5 = , a7 = − , a9 = ,
16384 8192 8192 32768 32768
and
957310976 265226398 735232
r1 = − , r2 = , r3 = − ,
1159104017 1159104017 13780629
17297069 1386496 563818
r4 = , r5 = − , r6 = − ,
2318208034 5795520085 10431936153
2048 5
r7 = − , r8 = − .
8113728119 18545664272

31
5. M = 6
160083 38115 22869
a1 = , a3 = − , a5 = ,
131072 131072 262144
5445 847 63
a7 = − , a9 = , a11 = − ,
262144 262144 262144
and
3986930636128256 4850197389074509 1019185340268544
r1 = , r2 = , r3 = ,
4689752620280145 18759010481120580 14069257860840435
136429697045009 7449960660992 483632604097
r4 = , r5 = , r6 = ,
9379505240560290 4689752620280145 112554062886723480
78962327552 31567002859 2719744
r7 = , r8 = , r9 = ,
6565653668392203 75036041924482320 937950524056029
1743
r10 = .
2501201397482744

Coefficients for M = 5 and M = 6 can be compared with the corresponding

output from the following iterative algorithm.

Iterative algorithm for computing the coefficients rl .

As a way of solving equations (5.14) and (5.15) we may also use an iterative
algorithm. We start with r−1 = 0.5 and r1 = −0.5 and iterate using (5.14) to recompute
rl . It is easy to verify using (5.42) that (5.15) and (5.17) are satisfied due to the choice
of initialization. The following 1 for Daubechies’ wavelets with M = 5, 6, 7, 8, 9 was
L−2
computed using this algorithm. It displays the coefficients {rl }l=1 . We note, that r−l =
−rl and r0 = 0.

V.2 The operators dn /dxn in the wavelet bases

Similar to the operator d/dx, the non-standard form of the operator dn /dxn is completely
determined by its representation on the subspace V0 , i.e., by the coefficients
(n)
Z +∞ dn
rl = ϕ(x − l) ϕ(x) dx, l ∈ Z, (5.52)
−∞ dxn
or, alternatively, Z +∞
(n)
rl = (−iξ)n |ϕ̂(ξ)|2 e−ilξ dξ. (5.53)
−∞
if the integrals in (5.52) or (5.53) exist (see also Remark 3 below).

32
 Coefficients Coefficients
l rl l rl

M =5 1 -0.82590601185015 M =8 1 -0.88344604609097
2 0.22882018706694 2 0.30325935147672
3 -5.3352571932672E-02 3 -0.10636406828947
4 7.4613963657755E-03 4 3.1290147839488E-02
5 -2.3923582002393E-04 5 -6.9583791164537E-03
6 -5.4047301644748E-05 6 1.0315302133757E-03
7 -2.5241171135682E-07 7 -7.6677069083796E-05
8 -2.6960479423517E-10 8 -2.4519921109537E-07
9 -3.9938104563894E-08
10 7.2079482385949E-08
M =6 1 -0.85013666155592 11 9.6971849256415E-10
2 0.25855294414146 12 7.2522069166503E-13
3 -7.2440589997659E-02 13 -1.2400785360984E-14
4 1.4545511041994E-02 14 1.5854647516841E-19
5 -1.5885615434757E-03
6 4.2968915709948E-06
7 1.2026575195723E-05 M =9 1 -0.89531640583699
8 4.2069120451167E-07 2 0.32031206224855
9 -2.8996668057051E-09 3 -0.12095364936000
10 6.9686511520083E-13 4 3.9952721886694E-02
5 -1.0616930669821E-02
6 2.1034028106558E-03
M =7 1 -0.86874391452377 7 -2.7812077649932E-04
2 0.28296509452594 8 1.9620437763642E-05
3 -9.0189066217795E-02 9 -4.8782468879634E-07
4 2.2687411014648E-02 10 1.0361220591478E-07
5 -3.8814546576295E-03 11 -1.5966864798639E-08
6 3.3734404776409E-04 12 -8.1374108294110E-10
7 4.2363946800701E-06 13 -5.4025197533630E-13
8 -1.6501679210868E-06 14 -4.7814005916812E-14
9 -2.1871130331900E-07 15 -1.6187880013009E-18
10 4.1830548203747E-10 16 -4.8507474310747E-24
11 -1.2035273999989E-11
12 -6.6283900594600E-16

l=L−2
Table 1: The coefficients {rl }l=1 for Daubechies’ wavelets, where L = 2M and M =
5, . . . , 9.

33
Proposition V.2 1. If the integrals in (5.52) or (5.53) exist, then the coefficients
(n)
rl , l ∈ Z satisfy the following system of linear algebraic equations
 
L/2
(n) (n) (n)
= 2n r2l + 1
X
rl 2
a2k−1 (r2l−2k+1 + r2l+2k−1 ) , (5.54)
k=1

and
(n)
l n rl = (−1)n n!,
X
(5.55)
l

where a2k−1 are given in (5.19).

2. Let M ≥ (n + 1)/2, where M is the number of vanishing moments in (2.16). If
the integrals in (5.52) or (5.53) exist, then the equations (5.54) and (5.55) have a
(n) (n)
unique solution with a finite number of non-zero coefficients rl , namely, rl 6= 0
for −L + 2 ≤ l ≤ L − 2. Also, for even n
(n) (n)
rl = r−l , (5.56)
(n)
l2ñ rl
X
= 0, ñ = 1, . . . , n/2 − 1, (5.57)
l

and X (n)
rl = 0, (5.58)
l

and for odd n

(n) (n)
rl = −r−l , (5.59)
(n)
l2ñ−1 rl
X
= 0, ñ = 1, . . . , (n − 1)/2. (5.60)
l

The proof of Proposition 2 is analogous to that of Proposition 1.

Remark 3. If M ≥ (n + 1)/2, then the solution of the linear system in Proposition 2

may exist while integrals (5.52) and (5.53) are not absolutely convergent.
A case in point is the Daubechies’ wavelet with M = 2. The representation of the
first derivative in this basis is described in the previous Section. The equations (5.54)
and (5.55) do not have a solution for the second derivative n = 2. However, the system
of equations (5.54) and (5.55) has a solution for the third derivative n = 3. We have
9 1
a1 = , a3 = − ,
8 8
and
1 1
r−2 = − , r−1 = 1, r0 = 0, r1 = −1, r2 = .
2 2

34
 We note that among the wavelets with L = 4, the wavelets with two vanishing
moments M = 2 do not have the best Hölder exponent (see [21]), but the representation
of the third derivative exists only if the number of vanishing moments M = 2.
(n)
The equations for computing the coefficients rl may be viewed as an eigenvalue
problem. Let us derive the equation corresponding to (5.42) for dn /dxn directly from
(5.53). We rewrite (5.53) as
Z 2π
(n)
|ϕ̂(ξ + 2πk)|2 in (ξ + 2πk)n e−ilξ dξ.
X
rl = (5.61)
0 k∈Z

Therefore,
|ϕ̂(ξ + 2πk)|2 in (ξ + 2πk)n ,
X
r̂(ξ) = (5.62)
k∈Z

where
(n)
rl eilξ .
X
r̂(ξ) = (5.63)
l

Substituting the relation

ϕ̂(ξ) = m0 (ξ/2)ϕ̂(ξ/2), (5.64)
into the right hand side of (5.62), and summing over even and odd indices in (5.62)
separately, we arrive at
 
r̂(ξ) = 2n |m0 (ξ/2)|2 r̂(ξ/2) + |m0 (ξ/2 + π)|2 r̂(ξ/2 + π) . (5.65)

Let us consider the operator M0 on 2π-periodic functions

(M0 f )(ξ) = |m0 (ξ/2)|2 f (ξ/2) + |m0 (ξ/2 + π)|2 f (ξ/2 + π). (5.66)

We rewrite (5.65) as
M0 r̂ = 2−nr̂, (5.67)
so that r̂ is an eigenvector of the operator M0 corresponding to the eigenvalue 2−n . Thus,
finding the representation of the derivatives in the wavelet basis is equivalent to finding
trigonometric polynomial solutions of (5.67) and vice versa. (The operator M0 is also
introduced in [14] and [26], where the problem (5.67) with the eigenvalue 1 is considered).

V.3 The explicit preconditioning of the derivative operators

.
While theoretically it is well understood that the derivative operators (or, more
generally, operators with homogeneous symbols) have an explicit diagonal preconditioner

35
in wavelet bases, the numerical evidence illustrating this fact is of interest, since it rep-
resents one of the advantages of computing in the wavelet bases.
For operators with a homogeneous symbol the bound on the condition number
depends on the particular choice of the wavelet basis (by the condition number we un-
derstand the ratio of the largest singular value to the smallest singular value above the
threshold of accuracy). After applying such a preconditioner, the condition number κp of
the operator is uniformly bounded with respect to the size of the matrix. We recall that
the condition number controls the rate of convergence of a number of iterative algorithms;
√
for example the number of iterations of the conjugate gradient method is O( κp ).
We present here two tables, 2 and 3, illustrating such preconditioning applied to
the standard form of the second derivative. In the following examples the standard form
of the periodized second derivative D2 of size N × N , where N = 2n , is preconditioned
by the diagonal matrix P ,
D2p = P D2 P.
where Pil = δil 2j , 1 ≤ j ≤ n, and where j is chosen depending on i, l so that N −
N/2j−1 + 1 ≤ i, l ≤ N − N/2j , and PN N = 2n .
In the tables we compare the original condition number κ of D2 and κp of D2p .

A remark the boundary value problems.

The wavelet bases are not well adapted for the boundary value problems (unless
the problem is reduced to solving equations on the boundary). In one dimension, there is
a simple construction of the wavelet basis for an interval due to Y. Meyer. Unfortunately,
it does not generalize to higher dimensions.
There is, however, a simple method for setting the boundary conditions. This
method is based on an observation, that the value of a function at a given point may
be obtained by considering a linear combination of the basis functions which contain
this point inside their supports. For a fixed point, the number of such functions is
proportional to the number of scales, log2 N . In two-dimensional problems, the number of
basis functions involved in the determination of the boundary conditions is proportional
to the length of the boundary times the number of scales. Thus, the boundary conditions
may be imposed as sparse linear constraints.

36
 N κ κp

64 0.14545E+04 0.10792E+02

128 0.58181E+04 0.11511E+02

256 0.23272E+05 0.12091E+02

512 0.93089E+05 0.12604E+02

1024 0.37236E+06 0.13045E+02

Table 2: Condition numbers of the matrix of periodized second derivative (with and with-
out preconditioning) in the basis of Daubechies’ wavelets with three vanishing moments
M = 3.

N κ κp

64 0.10472E+04 0.43542E+01

128 0.41886E+04 0.43595E+01

256 0.16754E+05 0.43620E+01

512 0.67018E+05 0.43633E+01

1024 0.26807E+06 0.43640E+01

Table 3: Condition numbers of the matrix of periodized second derivative (with and
without preconditioning) in the basis of Daubechies’ wavelets with six vanishing moments
M = 6.

37
VI The convolution operators in wavelet bases
In this Section we consider the computation of the non-standard form of convolution
operators. For convolution operators the quadrature formulas for representing the kernel
on V0 are of the simplest form due to the fact that the moments of the autocorrelation
of the scaling function ϕ vanish. Moreover, by combining the asymptotics of the wavelet
coefficients of the operator with the system of linear algebraic equations (similar to those
in Section V) we arrive at an effective method for computing these coefficients [5]. This
method is especially simple if the symbol of the operator is homogeneous of some degree.
(j−1)
Let us assume that the matrix ti−l (i, l ∈ Z) represents the operator Pj−1 T Pj−1
on the subspace Vj−1 . To compute the representation of Pj T Pj , we have the following
formula (3.26) of [7]
L−1
X L−1
(j) X (j−1)
tl = hk hm t2l+k−m . (6.1)
k=0 m=0

It easily reduces to
L/2
(j) (j−1) 1
X (j−1) (j−1)
tl = t2l + 2
a2k−1 (t2l−2k+1 + t2l+2k−1 ). (6.2)
k=0

where the coefficients a2k−1 are given in (5.19).

We also have
Z +∞ Z +∞
(j)
tl = K(x − y) ϕj,0(y) ϕj,l (x) dxdy, (6.3)
−∞ −∞

and by changing the order of integration obtain

Z +∞
(j) j
tl =2 K(2j (l − y)) Φ(y) dy, (6.4)
−∞

where Φ is the autocorrelation function of the scaling function ϕ,

Z +∞
Φ(y) = ϕ(x) ϕ(x − y) dx. (6.5)
−∞

Let us verify that Z +∞

Φ(y)dy = 1 (6.6)
−∞

and Z +∞
Mm
Φ = y m Φ(y)dy = 0, for 1 ≤ m ≤ 2M − 1. (6.7)
−∞

Clearly, we have m
1
 
Mm
Φ = ∂ξ |ϕ̂(ξ)| 2
. (6.8)
i ξ=0

38
Using (6.8) and the identity ϕ̂(ξ) = ϕ̂(ξ/2)m0 (ξ/2) (see [19]), it follows from (5.25) that
(6.7) holds.
Since the moments of the function Φ vanish, (6.7), equation (6.4) leads to a one-
point quadrature formula for computing the representation of convolution operators on
the finest scale for all compactly supported wavelets. This formula is obtained in exactly
the same manner as for the special choice of the wavelet basis described in [7] (eqns.
3.8-3.12), where the shifted moments of the function ϕ vanish; we refer to this paper for
the details.
Here we introduce a different approach, which consists in solving the system of
linear algebraic equations (6.2) subject to asymptotic conditions. This method is espe-
cially simple if the symbol of the operator is homogeneous of some degree since in this
case the operator is completely defined by its representation on V0 .
Let us consider two examples of such operators, the Hilbert transform and the
operator of fractional differentiation (or anti-differentiation).

VI.1 The Hilbert Transform

We apply our method to the computation of the non-standard form of the Hilbert trans-
form
1 Z ∞
f (s)
g(x) = (Hf )(y) = p.v. ds, (6.9)
π −∞ s − x

where p.v. denotes a principal value at s = x.

The representation of H on V0 is defined by the coefficients
Z ∞
rl = ϕ(x − l) (Hϕ)(x) dx, l ∈ Z, (6.10)
−∞

which, in turn, completely define all other coefficients of the non-standard form. Namely,
H = {Aj , Bj , Γj }j∈Z , Aj = A0 , Bj = B0 , and Γj = Γ0 , where matrix elements αi−l , βi−l ,
and γi−l of A0 , B0 , and Γ0 are computed from the coefficients rl ,
L−1
X L−1
X
αi = gk gk0 r2i+k−k0 , (6.11)
k=0 k 0 =0

L−1
X L−1
X
βi = gk hk0 r2i+k−k0 , (6.12)
k=0 k 0 =0

and
L−1
X L−1
X
γi = hk gk0 r2i+k−k0 . (6.13)
k=0 k 0 =0

39
 Coefficients Coefficients
l rl l rl

M =6 1 -0.588303698 9 -0.035367761
2 -0.077576414 10 -0.031830988
3 -0.128743695 11 -0.028937262
4 -0.075063628 12 -0.026525823
5 -0.064168018 13 -0.024485376
6 -0.053041366 14 -0.022736420
7 -0.045470650 15 -0.021220659
8 -0.039788641 16 -0.019894368

Table 4: The coefficients rl , l = 1, . . . , 16 of the Hilbert transform for Daubechies’ wavelet

with six vanishing moments.

The coefficients rl , l ∈ Z in (6.10) satisfy the following system of linear algebraic

equations
L/2
1
X
rl = r2l + 2
a2k−1 (r2l−2k+1 + r2l+2k−1 ), (6.14)
k=1

where the coefficients a2k−1 are given in (5.19). Using (6.4), (6.6) and (6.7) we obtain
the asymptotics of rl for large l,
1 1
rl = − + O( 2M ). (6.15)
πl l
By rewriting (6.10) in terms of ϕ̂(ξ),
Z ∞
rl = −2 |ϕ̂(ξ)|2 sin(lξ) dξ. (6.16)
0

we obtain rl = −r−l and set r0 = 0. We note that the coefficient r0 cannot be determined
from equations (6.14) and (6.15).
Solving (6.14) with the asymptotic condition (6.15), we compute the coefficients
rl , l 6= 0 with any prescribed accuracy.
Example.
We compute the coefficients rl of the Hilbert transform for Daubechies’ wavelets
with six vanishing moments with accuracy 10−7 . The coefficients for l > 16 are obtained
using asymptotics (6.15). (We note that r−l = −rl and r0 = 0).

40
VI.2 The fractional derivatives
We use the following definition of fractional derivatives
Z +∞ (x − y)−α−1
+
(∂xα f ) (x) = f (y)dy, (6.17)
−∞ Γ(−α)

where we consider α 6= 1, 2 . . .. If α < 0, then (6.17) defines fractional anti-derivatives.

The representation of ∂xα on V0 is determined by the coefficients
Z +∞
rl = ϕ(x − l) (∂xα ϕ) (x) dx, l ∈ Z, (6.18)
−∞

provided that this integral exists.

The non-standard form ∂xα = {Aj , Bj , Γj }j∈Z is computed via Aj = 2−αj A0 , Bj =
2−αj B0 , and Γj = 2−αj Γ0 , where matrix elements αi−l , βi−l , and γi−l of A0 , B0 , and Γ0
are obtained from the coefficients rl ,
L−1
X L−1
αi = 2 α
X
gk gk0 r2i+k−k0 , (6.19)
k=0 k 0 =0

L−1
X L−1
βi = 2 α
X
gk hk0 r2i+k−k0 , (6.20)
k=0 k 0 =0

and
L−1
X L−1
γi = 2 α
X
hk gk0 r2i+k−k0 . (6.21)
k=0 k 0 =0

It easy to verify that the coefficients rl satisfy the following system of linear
algebraic equations
 
L/2
α 1
X
rl = 2 r2l + 2
a2k−1 (r2l−2k+1 + r2l+2k−1 ) , (6.22)
k=1

where the coefficients a2k−1 are given in (5.19). Using (6.4), (6.6) and (6.7) we obtain
the asymptotics of rl for large l,
1 1 1
rl = 1+α
+ O( 1+α+2M ) for l > 0, (6.23)
Γ(−α) l l
rl = 0 for l < 0. (6.24)

Example.

41
 Coefficients Coefficients
l rl l rl

M =6 -7 -2.82831017E-06 4 -2.77955293E-02
-6 -1.68623867E-06 5 -2.61324170E-02
-5 4.45847796E-04 6 -1.91718816E-02
-4 -4.34633415E-03 7 -1.52272841E-02
-3 2.28821728E-02 8 -1.24667403E-02
-2 -8.49883759E-02 9 -1.04479500E-02
-1 0.27799963 10 -8.92061945E-03
0 0.84681966 11 -7.73225246E-03
1 -0.69847577 12 -6.78614593E-03
2 2.36400139E-02 13 -6.01838599E-03
3 -8.97463780E-02 14 -5.38521459E-03

Table 5: The coefficients {rl }l , l = −7, . . . , 14 of the fractional derivative α = 0.5 for
Daubechies’ wavelet with six vanishing moments.

We compute the coefficients rl of the fractional derivative with α = 0.5 for

Daubechies’ wavelets with six vanishing moments with accuracy 10−7 . The coefficients
for rl , l > 14 or l < −7 are obtained using asymptotics
1 1 1
rl = − √ 1+ 1 + O( 13+ 1 ) for l > 0, (6.25)
2 πl 2 l 2
rl = 0 for l < 0. (6.26)

42
VII Multiplication of operators in wavelet bases

VII.1 Multiplication of matrices in the standard form

The multiplication of matrices of Calderón-Zygmund and pseudo-differential operators
in the standard form requires at most O(N log2 N ) operations. In addition, it is possible
to control the width of the “finger” bands by setting to zero all the entries in the product
below a threshold of accuracy.
Let us compute T = T̃ T̂ , where
0 0 0 0
T̃ = {Ãj , {B̃jj }jj 0 =n j j =n n+1
=j+1 , {Γ̃j }j 0 =j+1 , B̃j , Γ̃n+1
j , T̃n }j=1,...,n (7.1)
and 0 0 0 0
T̂ = {Âj , {B̂jj }jj 0 =n j j =n n+1
=j+1 , {Γ̂j }j 0 =j+1 , B̂j , Γ̂n+1
j , T̂n }j=1,...,n . (7.2)
Since the standard form is a representation in an orthonomal basis (which is
the tensor product of one-dimensional bases), the result of the multiplication of two
standard forms is also a standard form in the same basis. Thus, the product T must
have a representation
0 0 0 0
T = {Aj , {Bjj }jj 0 =n j j =n n+1
=j+1 , {Γj }j 0 =j+1 , Bj , Γn+1
j , Tn }j=1,...,n . (7.3)
Due to the block structure of the corresponding matrix, each element of (7.3) is
obtained as a sum of products of the corresponding blocks of T̃ and T̂ . For example,
j 0 =n+1
0 0
Γ21 Γ̃21 Â1 Ã2 Γ̂21 B̃2j Γ̂j1 .
X
= + + (7.4)
j 0 =3

If the operators T̃ and T̂ are Calderón-Zygmund or pseudo-differential operators, then

all the blocks of (7.1) and (7.2) (except for T̃n and T̂n ) are banded and it is clear that
Γ21 is banded. This example is generic for all operators in (7.3) except for Bjn+1 , Γn+1 j ,
(j = 1, . . . , n) and Tn . The latter are dense due to the terms involving T̃n and T̂n . It
is easy now to estimate the number of operations necessary to compute T . It takes no
more than O(N log2 N ) operations to obtain T , where N = 2n .
0 0
If, in addition, when the scales j and j 0 are well separated, the operators Bjj , Γjj
can be neglected for a given accuracy (as in the case of pseudo-differential operators),
then the number of operations reduces asymptotically to O(N ).
We note, that we may set to zero all the entries of T below the threshold of
accuracy and, thus, prevent the widening of the bands in the product. On denoting T̃
and T̂ the approximations to T̃ and T̂ obtained by setting all entries that are less than
 to zero, and assuming (without a loss of generality) ||T̃ || = ||T̂ || = 1, we obtain using
the result of [7]
||T̃ − T̃ || ≤ , ||T̂ − T̂ || ≤ , (7.5)

43
and, therefore,
||T̃ T̂ − (T̃ T̂ ) || ≤  + (1 + ) + (1 + )2 . (7.6)
The right hand side of (7.6) is dominated by 3. For example, if we compute T 4 then we
might lose one significant digit.

VII.2 Multiplication of matrices in the non-standard form

We will now outline an algorithm for the multiplication of the operators in the non-
standard form. This new algorithm is remarkable in the way it decouples the scales in
the process of multiplication. Let T̂ and T̃ be two operators

T̂ , T̃ : L2 (R) → L2 (R). (7.7)

Given the non-standard forms of T̂ and T̃ , {Âj , B̂j , Γ̂j }j∈Z and {Ãj , B̃j , Γ̃j }j∈Z , we com-
pute the non-standard form {Aj , Bj , Γj }j∈Z of T = T̂ T̃ .
We recall that the operators of the non-standard form are defined using the pro-
jection operators Pj (on the subspace Vj ), and Qj = Pj−1 − Pj (on the subspace Wj ),
j ∈ Z, as follows Âj = Qj T̂ Qj , B̂j = Qj T̂ Pj , Γ̂j = Pj T̂ Qj , Ãj = Qj T̃ Qj , B̃j = Qj T̃ Pj
and Γ̃j = Pj T̃ Qj .
We will also need the operators T̂j = Pj T̂ Pj and T̃j = Pj T̃ Pj , j ∈ Z. These
operators may not be sparse but the algorithm requires only a band around the diagonal
of their matrices.
In order to decouple the scales, we write a “telescopic” series,
j=n
Xh i
T̂0 T̃0 − T̂n T̃n = (Pj−1 T̂ Pj−1 )(Pj−1 T̃ Pj−1 ) − (Pj T̂ Pj )(Pj T̃ Pj ) (7.8)
j=1

Since Pj−1 = Pj + Qj , we obtain from (7.8)

j=n
X
T̂0 T̃0 − T̂n T̃n = [ (Pj T̂ Pj )(Pj T̃ Qj ) + (Qj T̂ Pj )(Pj T̃ Pj ) +
j=1

(Qj T̂ Pj )(Pj T̃ Qj ) + (Pj T̂ Qj )(Qj T̃ Pj ) + (7.9)

(Pj T̂ Qj )(Qj T̃ Qj ) + (Qj T̂ Qj )(Qj T̃ Pj ) +
(Qj T̂ Qj )(Qj T̃ Qj ) ] ,
or
j=n
X
T̂0 T̃0 − T̂n T̃n = [ (Âj Ãj + B̂j Γ̃j ) + (B̂j T̃j + Âj B̃j ) + (7.10)
j=1

(T̂j Γ̃j + Γ̂j Ãj ) + Γ̂j B̃j ]

44
Finally, we rewrite (7.10) as a sum of two terms,

T̂0 T̃0 = F + R, (7.11)

where
j=n
Xh i
F = (Âj Ãj + B̂j Γ̃j ) + (B̂j T̃j + Âj B̃j ) + (T̂j Γ̃j + Γ̂j Ãj ) (7.12)
j=1

and
j=n
X
R = T̂n T̃n + Pj Γ̂j B̃j Pj (7.13)
j=1

The operators in the sum (7.12) are acting on following the subspaces,

Âj Ãj + B̂j Γ̃j : Wj → Wj , (7.14)

B̂j T̃j + Âj B̃j : Vj → Wj , (7.15)

T̂j Γ̃j + Γ̂j Ãj : Wj → Vj , (7.16)
and the operators in the sum (7.13),

Γ̂j B̃j : Vj → Vj . (7.17)

where j = 1, . . . , n, and
T̂n T̃n : Vn → Vn . (7.18)
We now describe a fast O(N ) algorithm for the multiplication of the non-standard
forms of Calderón-Zygmund and pseudo-differential operators.

1. We compute the operators (7.14)-(7.18). This involves the multiplication of banded

matrices. There is no problem in obtaining (7.14) or (7.17), since all matrices in the
products are banded. We are yet to explain that in computing (7.15) and (7.16)
we need to use only the band around the diagonal of T̂j and T̃j , j = 1, . . . , n (see
below).
Since the number of operations to compute (7.14)-(7.18) on a given scale j is half
of that on the previous finer scale j − 1, the total number of operations is twice
that on the finest scale j = 1. Thus, the total number of operations at this step is
proportional to N .

2. We compute the non-standard form of all the operators in (7.17). First, we observe
that Γ̂j B̃j , j = 1, . . . , n are banded. Starting from the finest scale j = 1, we
expand Γ̂j B̃j to obtain Āj+1 , B̄j+1 , Γ̄j+1 and T̄j+1 . We then add T̄j+1 to Γ̂j+1 B̃j+1
and expand the sum of the two, etc. As a result, we obtain Āj , B̄j , Γ̄j , j = 2, . . . , n
and T̄n . Since at all scales we are expanding a banded matrix, and the number

45
 of operations is halved each time we go to the sparser scale, the total number of
operations at this step is proportional to N .
The resulting operators Āj , B̄j , Γ̄j , j = 2, . . . , n, and T̄n are acting on the sub-
spaces,
Āj : Wj → Wj , (7.19)
B̄j : Vj → Wj , (7.20)
Γ̄j : Wj → Vj , (7.21)
and
T̄n : Vn → Vn , (7.22)

3. At this step we add the corresponding operators computed at step 1 and step 2, to
obtain the non-standard form of the operator T = T̂ T̃ ,

Aj = Āj + Âj Ãj + B̂j Γ̃j , (7.23)

Bj = B̄j + B̂j T̃j + Âj B̃j , (7.24)

Γj = Γ̄j + T̂j Γ̃j + Γ̂j Ãj , (7.25)
and
Tn = T̄n + T̂n T̃n . (7.26)
This step obviously requires O(N ) operations.

If the product of two operators satisfies the estimates of Section IV, then the
operators {Aj , Bj , Γj }j∈Z of the non-standard form of T = T̂ T̃ are banded (for a given
precision). Examining (7.23)-(7.25), we find that in (7.24) and (7.25) there is only one
term that may potentially be dense. However, if the multiplicants and the product are
banded then all terms are banded, and we conclude that B̂j T̃j and T̂j Γ̃j must be banded.
Thus, we need only a banded version of the operators T̂j and T̃j . The banded versions
of T̂j and T̃j are computed in the process of constructing the non-standard form and,
therefore, we only need to store the results of these computations.
These is an alternative (direct) argument to show that B̂j T̃j and T̂j Γ̃j are banded.
It requires a proof that the first several moments of rows of B̂j and columns of Γ̃j are
negligible, which may be found in [29] for pseudo-differential operators.

46
VIII Fast iterative algorithms in wavelet bases
The fast multiplication algorithms of Section VII give a second life to a number of iterative
algorithms.

VIII.1 An iterative algorithm for computing the generalized

inverse
In order to construct the generalized inverse A† of the matrix A, we use the following
result [36]:

Proposition VIII.1 Consider the sequence of matrices Xk

Xk+1 = 2Xk − Xk AXk (8.1)

with
X0 = αA∗ , (8.2)
where A∗ is the adjoint matrix and α is chosen so that the largest eigenvalue of αA ∗ A is
less than two. Then the sequence Xk converges to the generalized inverse A† .

When this result is combined with the fast multiplication algorithms of Sec-
tion VII, we obtain an algorithm for constructing the generalized inverse in the stan-
dard form in at most O(N log2 N log R) operations and in the non-standard form in
O(N log R), where R is the condition number of the matrix. By the condition number
we understand the ratio of the largest singular value to the smallest singular value above
the threshold of accuracy.
The details of this algorithm in the context of computing in wavelet bases will
appear in [6]. Throughout the iteration (8.1)-(8.2), it is necessary to maintain the “fin-
ger” band structure of the standard form or the banded structure of the submatrices
of the non-standard form of the matrices Xk . Hence, the standard and non-standard
forms of both the operator and its generalized inverse must admit such structures. Since
pseudo-differential operators of order zero form an algebra, the operators of this class
satisfy this condition. Also, since any pseudo-differential operator may be multiplied by
a compressible operator so that the product is a pseudo-differential operator of order
zero, it is easy to see that the iteration (8.1)-(8.2) is applicable to all pseudo-differential
operators.

Example.
The following table contains timings and accuracy comparison of the construction
of the generalized inverse via the singular value decomposition (SVD), which is O(N 3 )

47
procedure, and via the iteration (8.1)-(8.2) in the wavelet basis using Fast Wavelet Trans-
form (FWT). The computations were performed on Sun Sparc workstation and we used
a routine from LINPACK for computing the singular value decomposition. For tests we
used the following full rank matrix
1


 i−j
i 6= j
Aij = ,

1 i=j


where i, j = 1, . . . , N . The accuracy threshold was set to 10−4 , i.e., entries of Xk below
10−4 were systematically removed after each iteration.

Size N × N SVD FWT Generalized Inverse L2 -Error

128 × 128 20.27 sec. 25.89 sec. 3.1 · 10−4

256 × 256 144.43 sec. 77.98 sec. 3.42 · 10−4

512 × 512 1,155 sec. (est.) 242.84 sec. 6.0 · 10−4

1024 × 1024 9,244 sec. (est.) 657.09 sec. 7.7 · 10−4

... ... ... ...

215 × 215 9.6 years (est.) 1 day (est.)

Let us describe several iterative algorithms indicating that numerical functional

calculus with operators can be implemented efficiently (at least for pseudo-differential op-
erators). Numerical results and relative performance of these algorithms will be reported
separately.

VIII.2 An iterative algorithm for computing the projection op-

erator on the null space.
Let us consider the following iteration

Xk+1 = 2Xk − Xk2 (8.3)

with
X0 = αA∗ A, (8.4)
where A∗ is the adjoint matrix and α is chosen so that the largest eigenvalue of αA∗ A is
less than two.

48
 Then I −Xk converges to Pnull . This can be shown either directly or by combining
an invariant representation for Pnull = I − A∗ (AA∗ )† A with the iteration (8.1)-(8.2) to
compute the generalized inverse (AA∗ )† . The fast multiplication algorithm makes the
iteration (8.3)-(8.4) fast for a wide class of operators (with the same complexity as the
algorithm for the generalized inverse). The important difference is, however, that (8.3)-
(8.4) does not require compressibility of the inverse operator but only of the powers of
the operator.

VIII.3 An iterative algorithm for computing a square root of

an operator.
Let us describe an iteration to construct both A1/2 and A−1/2 , where A is, for simplicity,
a self-adjoint and non-negative definite operator. We consider the following iteration
Yl+1 = 2Yl − Yl Xl Yl , (8.5)
1
Xl+1 = (Xl + Yl A), (8.6)
2
with
α
Y0 = (A + I),
2
α
X0 = (A + I), (8.7)
2
√
where α is chosen so that the largest eigenvalue of α2 (A + I) is less than 2.
The sequence Xl converges to A1/2 and Yl to A−1/2 . By writing A = V ∗ DV , where
D is a diagonal and V is a unitary, it is easy to verify that both Xl and Yl can be written
as Xl = V ∗ Pl V and Yl = V ∗ Ql V , where Pl and Ql are diagonal and
Ql+1 = 2Ql − Ql Pl Ql ,
1
Pl+1 = (Pl + Ql D), (8.8)
2
with
α
Q0 = (D + I),
2
α
P0 = (D + I). (8.9)
2
Thus, the convergence need to be checked only for the scalar case, which we leave to the
reader. If the operator A is a pseudo-differential operator, then the iteration (8.5) - (8.6)
leads to a fast algorithm due to the same considerations as in the case of the generalized
inverse in Section VIII.

49
VIII.4 Fast algorithms for computing the exponential, sine and
cosine of a matrix
The exponential of a matrix (or an operator), as well as sine and cosine functions are
among the first to be considered in any calculus of operators. As in the case of the
generalized inverse, we use previously known algorithms (see e.g. [40]), which obtain
a completely different complexity estimates when we use them in conjunction with the
wavelet representations.
An algorithm for computing the exponential of a matrix is based on the identity
h i2L
exp(A) = exp(2−L A) . (8.10)

First, exp(2−L A) is computed by, for example, using the Taylor series. The number L is
chosen so that the largest singular value of 2−L A is less than one. At the second stage
of the algorithm the matrix 2−L A is squared L times to obtain the result.
Similarly, sine and cosine of a matrix may be computed using the elementary
double-angle formulas. On denoting

Yl = cos(2l−L A) (8.11)
Xl = sin(2l−L A), (8.12)

we have for l = 0, . . . , L − 1

Yl+1 = 2Yl2 − I (8.13)

Xl+1 = 2Yl Xl , , (8.14)

where I is the identity. Again, we choose L so that the largest singular value of 2−L A is
less than one, compute the sine and cosine of 2−L A using the Taylor series, and then use
(8.13) and (8.14).
Ordinarily such algorithms require at least O(N 3 ) operations, since a number of
multiplications of dense matrices has to be performed [40]. The fast algorithm for the
multiplication of matrices in the standard form reduces the complexity to no more than
O(N log2 N ) operations, and the fast algorithm for the multiplication of matrices in the
non-standard form to O(N ) operations (see Section VII).
To achieve such performance, it is necessary to maintain the “finger” band struc-
ture of the standard form or the banded structure of the submatrices of the non-standard
form throughout the iteration. Whether it is possible to do depends on the particular
operator and, usually, may be verified analytically.
Unlike the algorithm for the generalized inverse, the algorithms of this Remark
are not self-correcting. Thus, it is necessary to maintain sufficient accuracy initially so
as to obtain the desired accuracy after all the multiplications have been performed.

50
IX Computing F (u) in the wavelet bases
In this Section we describe a fast, adaptive algorithm for computing F (u), where F is an
infinitely differentiable function and u is represented in a wavelet basis. An important
example is F (u) = u2 . Our analytic results generalize theorems of J. M. Bony [10], [11],
[9], [15] on the propagation of singularities of solutions of non-linear equations. Our
numerical approach, however, is novel. We expect a wide range of applications of this
algorithm.

IX.1 The algorithm for evaluating u2

We start with an algorithm to compute F (u) = u2 . Let us project u ∈ L2 (R) on
subspaces Vj , j ∈ Z, so that
uj = Pj u, uj ∈ Vj . (9.1)
In order to decouple the scales, we write a “telescopic” series,
j=n
Xh i j=n
u20 u2n 2 2
X
− = (Pj−1 u) − (Pj u) = (Pj−1 u + Pj u)(Pj−1 u − Pj u) (9.2)
j=1 j=1

Using Pj−1 = Pj + Qj , we obtain

j=n
u20 − u2n =
X
(2Pj u + Qj u)(Qj u), (9.3)
j=1

or
j=n j=n
u20 = 2 (Qj u)(Qj u) + u2n .
X X
(Pj u)(Qj u) + (9.4)
j=1 j=1

In (9.4) there is no interaction between different scales j and j 0 , j 6= j 0 .

For the numerical purposes, we need formulas (9.3) or (9.4) with a finite number
of scales, though it is clear that by taking limits j → ∞ and j → −∞ we have

u2 =
X
(2Pj u + Qj u)(Qj u), (9.5)
j∈Z

which is essentially the para-product of J.M. Bony. In what follows, we will consider
each term of (9.5) as a bilinear mapping

MVj W : Vj−1 × Wj → L2 (R) = Vj

M
Wj 0 , (9.6)
j 0 ≤j

and show that, for a given precision, very few coefficients are needed to describe mapping
(9.6) for all j.

51
 Before proceeding further, let us consider an example of (9.4) in the Haar basis.
We have the following explicit relations,

(χjk (x))2 = 2−j/2 χjk (x),

(hjk (x))2 = 2−j/2 χjk (x), (9.7)
χjk (x)hjk (x) = 2−j/2 hjk (x).

All other products on the same scale are zero.

Expanding u0 explicitly into the Haar basis,
j=n
djk hjk (x) + snk χnk (x),
X X X
u0 (x) = (9.8)
j=1 k∈Z k∈Z

and using (9.7), we obtain from (9.4)

j=n j=n
u20 (x) = 2 2−j/2 djk sjk hjk (x) + 2−j/2 (djk )2 χjk (x) + 2−n/2 (snk )2 χnk (x). (9.9)
X X X X X

j=1 k∈Z j=1 k∈Z k∈Z

On denoting

dˆjk = 2−j/2+1 djk sjk ,

ŝjk = 2−j/2 (djk )2 , (9.10)
ŝnk = 2−n/2 (snk )2 ,

we rewrite (9.9) as
j=n j=n
u20 (x) = dˆjk hjk (x) + ŝjk χjk (x) + ŝnk χnk (x).
X X X X X
(9.11)
j=1 k∈Z j=1 k∈Z k∈Z

We note that if the coefficient djk is zero then there is no need to keep the corresponding
average sjk . In other words, we need to keep averages only near the singularities, i.e.,
where the wavelet coefficients djk ( or products sjk djk ) are significant for a given accuracy.
Finally, to compute the coefficients of the wavelet expansion of the function u20 ,
we need to expand the second sum in (9.11) into the wavelet basis. Starting from the
scale j = 1, we compute the differences and averages d¯j+1 k and s̄j+1
k from sjk . We then
add s̄j+1
k to ŝj+1
k before expanding it further. As a result, we compute d¯jk , j = 2, . . . , n,
(we set d¯1k = 0) and ŝnk and obtain
j=n
u20 (x) = (dˆjk + d¯jk ) hjk (x) + (ŝnk + ŝnk ) χnk (x).
X X X
(9.12)
j=1 k∈Z k∈Z

52
It is clear, that the number of operations for computing the Haar expansion of u20 is
proportional to the number of significant coefficients djk in the wavelet expansion of u0 .
If the original function is represented by a vector of the length N , then, in the worst
case, the number of operations is proportional to N . If the original function is smooth
with a finite number of singularities, then the number of operations is proportional to
log2 N .
We now return to the general case and derive an algorithm to expand (9.4) into
the wavelet basis. For each scale j, and for each term in (9.4), we consider the bilinear
mappings,
MVj W : Vj × Wj → L2 (R) = Vj
M
Wj 0 , (9.13)
j 0 ≤j
j 2
M
MW W : Wj × Wj → L (R) = Vj Wj 0 , (9.14)
j 0 ≤j

and
MVn V : Vn × Vn → L2 (R) = Vn
M
Wj 0 , (9.15)
j 0 ≤n

which map a product of two functions into its expansion in the wavelet basis. The choice
of the representation of L2 (R), which depends on the scale j, turns out to be important
for the speed of the algorithm. 3
The mappings (9.13)-(9.15) are tabulated by computing the coefficients
0
Z +∞ 0
MVj,jW W (k, k 0 , l) = φjk (x) ψkj 0 (x) ψlj (x) dx, (9.16)
−∞

0
Z +∞ 0
j,j
MW 0
W W (k, k , l) = ψkj (x) ψkj 0 (x) ψlj (x) dx, (9.17)
−∞
0
Z +∞ 0
MVj,jV W (k, k 0 , l) = φjk (x) φjk0 (x) ψlj (x) dx, (9.18)
−∞

and Z +∞
MVj W V (k, k 0 , l) = φjk (x) ψkj 0 (x) φjl (x) dx, (9.19)
−∞
Z +∞
j
MW 0
W V (k, k , l) = ψkj (x) ψkj 0 (x) φjl (x) dx, (9.20)
−∞
Z +∞
MVj V V (k, k 0 , l) = φjk (x) φjk0 (x) φjl (x) dx, (9.21)
−∞

where j 0 ≤ j. It is clear, that the coefficients are identically zero for |k − k 0 | > k0 ,
where k0 depends on the overlap of the supports of the basis functions. The number
3
There might be fewer significant coefficients if we write, for example, L2 (R) = Vj−j0
L
j 0 ≤j−j0 Wj 0
in (9.13)-(9.15) with some j0 ≥ 1.

53
of coefficients which need to be stored may be reduced further by observing that, for
example,
Z +∞
j,j 0 0 0
MW 0
W W (k, k , l) =2 −j 0 /2
ψ0j−j (x) ψk−k
j−j 0
0 (x) ψ j−j 0
2 k−l
(x) dx, (9.22)
−∞

so that
j,j 0 j−j 0 0 0
0 −j /2 0 j−j
MW W W (k, k , l) = 2 M̃W W W (k − k , 2 k − l). (9.23)
However, the most significant reduction in the number of coefficients is a conse-
quence of the fact that the coefficients in (9.16)-(9.18) decay as the distance r = j − j 0
between the scales increases. For example, rewriting (9.23) as
Z +∞
r 0 r −r
M̃W W W (k − k , 2 k − l) = 2 ψ(2−r x) ψ(2−r x − k + k 0 ) ψ(x − 2r k + l) dx, (9.24)
−∞

and remembering that the regularity of the product ψ(2−r x) ψ(2−r x − k + k 0 ) increases
linearly with the number of vanishing moments of the function ψ, we obtain
r 0 r −rλM
|M̃W W W (k − k , 2 k − l)| ≤ C2 (9.25)

with some λ (see [19], [21]). It might be possible to estimate λ in (9.25), but since the
coefficients in (9.16)-(9.18) are computed numerically, the rate of decay and the number
of significant coefficients may be measured directly. To summarize, for a given precision,
only the coefficients with labels r = j − j 0 , such that 0 ≤ r ≤ j0 , where j0 is a small
constant, need to be stored.
Evaluating the mappings (9.13)-(9.15), we arrive at the following representation
2
of u0 ,
j=n j=n
u20 (x) dˆjk ψkj (x) + ŝjk φjk (x),
X X X X
= (9.26)
j=1 k∈Z j=1 k∈Z

where dˆjk , ŝjk are obtained by adding contributions (on the appropriate scales) of the
results of evaluating (9.13)-(9.15) for all scales j in (9.4).
The next stage of the algorithm is the expansion of the second term in (9.26) into
the wavelet basis and is similar to that for the Haar basis.
Let us evaluate the complexity of computing u20 in the worst case. We assume that
the original function u0 is represented by a vector of length N = 2n and its expansion
into the wavelet basis also requires order N significant entries of the vector djk . In the
process of evaluating mappings (9.13)-(9.15), each coefficient djk (j and k are fixed) will
combine with the coefficients djk0 (or sjk0 ), where |k − k 0 | ≤ k0 , to produce a non-zero
contribution on scales j 0 , 0 ≤ j − j 0 ≤ j0 . Since j0 and k0 are (small) constants and are
independent of the size N , the total number of operations is proportional to the number
of significant coefficients djk , which is O(N ) in this case.

54
 If the number of significant coefficients djk is proportional to the number of scales,
log2 N , so will be the number of operations required to evaluate the mappings (9.13)-
(9.15). It is also clear, that it is necessary to store only those averages s jk0 , which combine
with the significant coefficients djk to produce a non-zero contribution. Therefore, it
is sufficient to store only those sjk0 , for which there exist the coefficient djk , such that
|k − k 0 | ≤ k0 and the product sjk0 djk is above the threshold of accuracy. It means that we
need to store averages only in the neighborhood of singularities.
The number of operation for expanding of the second term in (9.26) into the
wavelet basis is proportional to the number of significant entries, and the estimate is
completely similar to that for the Haar basis.
Remark. The algorithm for evaluation F (u) = u2 in the wavelet basis allows us to
evaluate the product of two functions, since uv = 41 [(u + v)2 − (u − v)2 ].

IX.2 The algorithm for evaluating F (u)

Let F be an infinitely differentiable function. In order to decouple the scales, we again
use a “telescopic” series,
j=n
X
F (u0 ) − F (un ) = [F (Pj−1 u) − F (Pj u)] . (9.27)
j=1

Expanding the function F in the Taylor series at the point (x + y)/2, we have
∞
1 x+y
F (2k−1) ( )(x − y)2k−1 .
X
F (x) − F (y) = 2k−2
(9.28)
k=1 (2k − 1)! 2 2

Using (9.28) and Pj−1 = Pj + Qj , we obtain from (9.27)

j=n ∞
1
F (2k−1) (Pj u + 12 Qj u) (Qj u)2k−1 .
X X
F (u0 ) − F (un ) = 2k−2
(9.29)
j=1 k=1 (2k − 1)! 2

Let us write out several terms of this formula explicitly. If we keep only the first term,
we obtain
j=n
F (1) (Pj u + 21 Qj u) (Qj u).
X
F (u0 ) − F (un ) ≈ (9.30)
j=1

If F (u) = u2 , then formula (9.30) is exact and we obtain (9.3). If we keep two terms, we
obtain
j=n
1
 
(1) 1
(Qj u) + F (3) (Pj u + 21 Qj u) (Qj u)3
X
F (u0 ) − F (un ) ≈ F (Pj u + Q u)
2 j
(9.31)
j=1 24

55
We note that there is no term with the second derivative of F in (9.31) (there are no even
derivatives in (9.29)). Using (9.30) and considering the remainder of the series in (9.29)
as an error term, we obtain the results of J.M. Bony. The error term, however, will be
slightly smoother than in Bony’s results, since the remainder has a factor (Qj u)3 instead
of (Qj u)2 . We may also keep more terms to make the remainder arbitrarily smooth.
It is not clear at this point, if there is an advantage in computing F (u) via (9.29),
or a repeated application of the algorithm for F (u) = u2 is sufficient to compute various
functions F . However, there are several analytic advantages in considering (9.29). In
particular, by using theorems characterizing the functional spaces, for example the Hölder
spaces C s (R), in terms of their wavelet coefficients (see [33]), the series (9.29) may be
truncated for given precision.
Finally, we remark that the adaptive numerical algorithms based on the expansion
in (9.29) are novel and we expect to develop new methods for solving P.D.E.’s using these
algorithms.

56
References
[1] B. Alpert. Sparse representation of smooth linear operators. Ph.D. thesis, Yale
University, 1990.

[2] B. Alpert, G. Beylkin, R. R. Coifman, and V. Rokhlin. Wavelets for the fast solu-
tion of second-kind integral equations. Technical report, Department of Computer
Science, Yale University, New Haven, CT, 1990.

[3] B. Alpert and V. Rokhlin. A fast algorithm for the evaluation of legendre expansions.
SIAM J. on Sci. Stat. Comput., 12(1):158–179, 1991. Yale University Technical
Report, YALEU/DCS/RR-671 (1989).

[4] A. Ben-Israel and D. Cohen. On iterative computation of generalized inverses and

associate projections. SIAM J. Numer. Anal., 3(3):410–419, 1966.

[5] G. Beylkin. On the representation of operators in bases of compactly supported

wavelets. SIAM J. Numer. Anal., 1991. submitted.

[6] G. Beylkin, R. R. Coifman, and V. Rokhlin. Fast wavelet transforms and numerical
algorithms II. in progress.

[7] G. Beylkin, R. R. Coifman, and V. Rokhlin. Fast wavelet transforms and numerical
algorithms I. Comm. Pure and Appl. Math., 44:141–183, 1991. Yale University
Technical Report YALEU/DCS/RR-696, August 1989.

[8] G. Beylkin, R. R. Coifman, and V. Rokhlin. Wavelets in Numerical Analysis. In

Wavelets and Their Applications, pages 181–210. Jones and Bartlett, 1992.

[9] J. M. Bony. Interaction des singularités pour les équations aux dérivées partielles
non-linéaires. Sém. e.d.p., 1979/80, 22, 1981/82, 2 et 1983/84, 10, Centre de
Mathématique, Ecole Polytechnique, 91128-Palaisau, France.

[10] J. M. Bony. Calcul symbolique et propagation des singularités pour les équations
aux dérivées partielles non-linéaires. Ann. Scient. E.N.S., 14:209–246, 1981.

[11] J. M. Bony. Propagation et interaction des singularités pour les solutions des
équations aux dérivées partielles non-linéaires. In Proceedings of the International
Congress of Mathematicians, Warszawa, pages 1133–1147, 1983.

[12] P. J. Burt and E. H. Adelson. The Laplacian pyramid as a compact image code.
IEEE Trans. Communications, 31(4):532–540, Apr. 1983.

57
[13] J. Carrier, L. Greengard, and V. Rokhlin. A fast adaptive multipole algorithm for
particle simulations. SIAM Journal of Scientific and Statistical Computing, 9(4),
1988. Yale University Technical Report, YALEU/DCS/RR-496 (1986).

[14] A. Cohen, I. Daubechies, and J.-C. Feauveau. Biorthogonal bases of compactly

supported wavelets. preprint, 1990.

[15] R. R. Coifman and Y. Meyer. Au delà des opérateurs pseudo-différentiels. In

Astérisque, 57, (seconde édition revue et augmentée). Société Mathématique de
France.

[16] R. R. Coifman and Y. Meyer. Nouvelles bases orthonormée de l 2 (r) ayant la structure
du sysème de walsh. 1989. preprint.

[17] R. R. Coifman and Y. Meyer. Nouvelles bases orthogonales. C.R. Acad. Sci., Paris,
1990.

[18] R. R. Coifman and V. Wickerhauser. Best-adapted wave packet bases. 1990.

[19] I. Daubechies. Orthonormal bases of compactly supported wavelets. Comm. Pure

and Appl. Math., 41:909–996, 1988.

[20] I. Daubechies. Ten Lectures on Wavelets. CBMS-NSF Series in Applied Mathemat-

ics. SIAM, 1991. in press.

[21] I. Daubechies and J. Lagarius. Two scale difference equations: I-II. SIAM J. Math.
Anal., 1990.

[22] P. Goupillaud, A. Grossmann, and J. Morlet. Cycle-Octave and related transforms

in seismic signal analysis. Geoexploration, 23:85–102, 1984.

[23] L. Greengard. Potential flow in channels. SIAM J. Sci. Stat. Comput., 11(4):603–
620, 1990.

[24] L. Greengard and V. Rokhlin. A fast algorithm for particle simulations. J. Comp.
Phys., 73(1):325–348, 1987.

[25] A. Haar. Zur theorie der orthogonalen funktionensysteme. Mathematische Annalen,

pages 331–371, 1910.

[26] W. M. Lawton. Necessary and sufficient conditions for constructing orthonormal

wavelet bases. J. Math. Phys., 32(1), 1991.

[27] S. Mallat. Multiresolution approximation and wavelets. Technical report, GRASP

Lab, Dept. of Computer and Information Science, University of Pennsylvania.

58
[28] S. Mallat. Review of multifrequency channel decomposition of images and wavelet
models. Technical Report 412, Courant Institute of Mathematical Sciences, New
York University, 1988.

[29] Y. Meyer. Le calcul scientifique, les ondelettes et filtres miroirs en quadrature.

CEREMADE, Université Paris-Dauphine.

[30] Y. Meyer. Principe d’incertitude, bases hilbertiennes et algébres d’opérateurs.

In Séminaire Bourbaki,, page 662. Société Mathématique de France, 1985-86.
Astérisque.

[31] Y. Meyer. Ondelettes et functions splines. Technical report, séminaire edp, Ecole
Polytechnique, Paris, France, 1986.

[32] Y. Meyer. Wavelets and operators. In N.T. Peck E. Berkson and J. Uhl, editors,
Analysis at Urbana. London Math. Society, Lecture Notes Series 137, 1989. v.1.

[33] Yves Meyer. Ondelettes et Opérateurs. Hermann, Paris, 1990.

[34] S.T. O’Donnel and V. Rokhlin. A fast algorithm for the numerical evaluation of con-
formal mappings. SIAM J. Sci. Stat. Comput., 10(3):475–487, 1989. Yale University
Technical Report, YALEU/DCS/RR-554 (1987).

[35] V. Rokhlin. Rapid solution of integral equations of classical potential theory. J.

Comp. Phys., 60(2), 1985.

[36] G. Schulz. Iterative berechnung der reziproken matrix. Z. Angew. Math. Mech.,
13:57–59, 1933.

[37] M. J. Smith and T. P. Barnwell. Exact reconstruction techniques for tree-structured

subband coders. IEEE Transactions on ASSP, 34:434–441, 1986.

[38] T. Söderström and G.W.Stewart. On the numerical properties of an iterative method

for computing the Moore-Penrose generalized inverse. SIAM J. Numer. Anal.,
11(1):61–74, 1974.

[39] J. O. Stromberg. A Modified Franklin System and Higher-Order Spline Systems on

Rn as Unconditional Bases for Hardy Spaces. In Conference in harmonic analysis
in honor of Antoni Zygmund, Wadworth math. series, pages 475–493, 1983.

[40] R. C. Ward. Numerical computation of the matrix exponential with accuracy esti-
mates. SIAM. J. Numer. Anal., 14(4):600–610, 1977.

59