INTRODUCTION TO SAMPLING

LIA PETRACOVICI AND JOSEPH M. ROSENBLATT

1. Introduction
These notes are meant to introduce Shannon's sampling theorem
and some of its various aspects and generalizations. We will see how
this theorem can be viewed as having its origin in earlier types of interpolation series. We will consider some extensions of sampling to
non-uniform sampling and to general integral transforms. We will try
to summarize the conditions on functions needed to obtain a sampling
theorem, and also describe the error in various sampling expansions.
Sampling is of great practical importance. It has many applications
in engineering and physics; for example, it has applications in signal
processing, data transmission, optics, cryptography, time-varying systems, and boundary value problems. Many people have discovered,
or rediscovered, this sampling theorem during this century; but these
notes will not try to unravel the claims to priority. See Higgins [10]
and Jerri [12] for additional information and references concerning the
history, generalizations, and applications of spectral sampling.
2. The Shannon Sampling Theorem = SST
The Shannon Sampling Theorem was apparently discovered by Shannon and described in a manuscript by 1940, but it was not published
until 1949, after World War II had ended. The principal impact of the
Shannon sampling theorem on information theory is that it allows the
replacement of a continuous band-limited signal by a discrete sequence
of its samples without the loss of any information. Also it species the
lowest rate (the Nyquist rate of such sample values) that is possible to
use to reproduce the original signal. Higher rates of sampling do have
advantages for establishing bounds, but would not be necessary for a
general signal reconstruction.
There are many variants on how the Fourier transform is normalized.
Here we will take these denitions for the Fourier Transform (FT) and
its inverse transform (INVFT):
L. Petracovici is partially supported by Rosenblatt's NSF Grant DMS-970522.
J. Rosenblatt is partially supported by NSF Grant DMS-9705228.
1

L. PETRACOVICI AND J. ROSENBLATT

(

1 F (x)e,ixt dx; the FT.

f (t) = Fb(t) = 21R ,1
1 f (t)eixt dt;
F (x) = fb(x) = ,1
the INVFT,
Although the notation is the same for these two dierent formulas,
the convention of using lower case and upper case for the functions will
be kept throughout these remarks, and so there should be no confusion
between the two transforms. A band-limited signal f (i.e. band-limited
function) is one whose inverse Fourier transform has compact support.
We will x the notation that such a signal has no frequencies outside
[,2w; 2w] when its inverse Fourier transform F has support contained in [,2w; 2w].
Shannon's original proof of his sampling theorem used the following interpolation formula for a band-limited signal f with frequencies
conned to [,2w; 2w]:
1


X
(2wt , n) :
(2.1)
f (t) =
f 2nw sin(2
wt , n)
n=,1
Such formulas are also sometimes called cardinal series. Suitable conditions will need to be imposed on f for this formula to be valid. Many
varieties of interpolation formulas appear in other contexts too. See for
example Remark 3.3 and the proof of Theorem 3.8 in Zygmund [30].
Remark 2.1. The SST and most of its extensions are stated primarily
for band-limited functions here, instead of for random processes which
would be more relevant to the information theorist.
By CPS, we will mean the number of cycles per second.
Theorem 2.2 (Shannon's original statement of SST). If a function
contains no frequencies higher than w CPS, then it is completely de-

termined by giving its ordinates at a series of points spaced 21w apart.

Remark 2.3. The limitation on frequencies in this statement translates
with our choice of notation to saying that the INVFT of f has support
in [,2w; 2w].
Proof. We are going to assume at least that F is Lebesgue integrable
on [,2w; 2w], and we have denoted the FT of F by
Z 1
Z 2w
1
1
,
iut
f (t) = 2
F (u)e du = 2
F (u)e,iut :du
,1
,2w
Since F has compact support, we can also use appropriate Fourier
series expansions for it. The Fourier series expansion of F (u) on

INTRODUCTION TO SAMPLING

[,2w; 2w] is given as follows. Let cn be the Fourier coecient

Z 2w
1
cn = 4w
F (u)e,2iun=4w du
,2w
Z 2w
1
= 4w
F (u)e,iun=2w du
,2w
= 1 f ( n ):
2w 2w
Also, then the Fourier series
expansion of F that corresponds to these
1


P
coecients is F (u) =
cneiun=2w : Here the functions e,iun=2w
n=,1
form a complete orthonormal basis ,in L
2[,2w; 2w]
Note that cn is proportional to f 2nw , the sample of the signal f (t).
Also fcng uniquely determines F (u), and hence they also uniquely determine f (t). This in itself is the uniqueness part of SST. But we can
also derive the formula the Shannon used too. Let us proceed formally
rst.
Z 2w
1
f (t) = 2
F (u)e,iutdu
,2w
!
Z 2w X
1


1 f n eiun=2w e,iutdu
= 21
,2w ,1 2w 2w
1   Z 2w
X
n
1
eiu( 2w ,t)du
= 4w f 2nw
,2w
,1
1  
h
X
n ,t) i
n ,t)
1
n
1
,
2
iw
2
iw
(
(
2
w
2
w
,


= 4w f 2w
,e
n ,t e
i
2w
,1
1


X
(2wt , n) :
=
f 2nw sin(2
wt , n)
n=,1
Hence, when the interchange of summation and integration is valid,
1


X
(2wt , n) :
f (t) =
f 2nw sin(2
wt , n)
n=,1
For example, a sucient condition for this is that F 2 L2 [,2w; 2w].
Remark 2.4. a) The sampling rate in this theorem of 2w samples/sec is
called the Nyquist sampling rate. It is the lowest rate of sampling which
can recover the whole signal. Indeed, for example for h; 0 < h < 2,
let Fh(x) = h1 [,;,+h](x). Also, for  > 0, let G (x) = [0;] , [;2].

L. PETRACOVICI AND J. ROSENBLATT

c (0) = 0. But it is also not hard to

Then G is mean zero and so G
see by direct calculation that Fch(t) = 0 for all samples t = 2h n where
n 2 Znf0g. So if F is the convolution Fh ? G , then Fb(t) = 0 for all t
in the arithmetic progression 2h Z, where Z denotes the integers. Also,
if we choose , + h + 2  , then the support of F is in [,; ].
But F 6= 0 since Fb(1) 6= 0 for small . Since h can be as close to 2
as we like, for suitably small choice of , this shows that a non-zero
function supported in [,; ] can sample as if it were the identically
0 function, if one samples at a rate any slower than the appropriate
Nyquist sampling rate (which means sampling on Z since w = 12 in this
case).
b) It is clear that some condition on the nature of F is necessary
here. It is not so clear to us what is the right condition to make
SST hold. For example, consider the discrete measure (distribution)
f,2wg , f2wg which is band-limited to [,2w; 2w]. Its FT f (t) is
1 2iwt , e,2iwt ) = i sin(2wt). So f ( n ) = 0 for all integers n, and
2 (e

2w
the cardinal series does not represent f at all.
c) Due to the similarity of INVFT and FT, the sampling theorem
is also valid for time-limited functions, i.e. f (t) which is zero outside
[,T; T ]. For the INVFT of f we have

F (x) =

1
X


 sin(Tx , n )
F n
:
T
(
Tx
,
n
)
n=,1

3. Cardinal Series, Sampling Bounds, and Other Ways to

Sample

As is well-known, E. T. Whittaker anticipated SST just from the

viewpoint of wanting to solve a certain type of interpolation problem.
In 1915, E. T. Whittaker [27] posed the interpolation problem of nding
a function f (t) when the values of the function are known for equidistant values a  nw where n = 0; 1; 2; : : : , and such that this expression
is free of periodic components with a period less than 2w. He showed
that the sum of what we now call the cardinal series picks out a special
member of the class of all such functions, the \cardinal" function. His
new idea was that this function is special because it is entire (analytic
in all of the complex plane C ) and free from \violent oscillations" i.e.
has no constituent frequencies whose period is larger than 2w, i.e. it is
band-limited. The formula that he wrote was this one, a very similar

INTRODUCTION TO SAMPLING

formula to Shannon's formula:

1
X

sin w (t , a , nw)
f (t) = f (a + nw)  (t , a , nw) :
(3.1)
w
,1
Actually, he did not call this a cardinal series, the name appears rst
in the work of his son J. M. Whittaker in about 1920, see J. M. Whittaker [28].
Perhaps the most obvious question to ask about SST is what happens
if one samples slower, or faster than the Nyquist sampling rate? From
the viewpoint of simply sampling a known analytic function, it is really
obvious that one can get into trouble if one samples inappropriately.
For example, if we were sampling the function cos(2t), and took the
samples to be at values of t which were whole numbers, we might think
the function is constant. See b) in Remark 2.4. The general question
with such functions would be if there is any rate, or way, of sampling
that could give information about the function, or even perhaps be
used to recreate it elsewhere.
3.1. Sampling Bounds. Consider rst the following sampling problem. We want to bound the general values of a band-limited signal by
its behavior along a doubly-innite arithmetic progression of values n
where n 2 Z. That is, we take a band-limited signal f and ask for a
bound of the form
sup jf (t)j  C sup jf (n)j:
t2R

n2Z

We allow the constant to depend on  and the band-limit of f , but we

do not want it to depend on any particularities of the distribution of f .
It turns out that this is possible, even possible along jittered samples
that are not nicely arithmetic, but the sampling must be strictly faster
than the Nyquist sampling rate for this to be achieved.
First, assume that F 2 L1 (R ) and that F is supported in the interval [,; ], with  < . We want to estimate the FT f (t) =

1 R F (x)e,ixt dx: Let the function h be dened by cases as follows:
x
2
,

itx
if ,  x  ;
ht (x) = e
at(x) if ,  x  , or   x  .
Here the functions at(x) need to be chosen to be continuous and to make
ht(x) into a continuous 2-periodic function. We can also arrange for
the function ht to be innitely dierentiable, so that1its Fourier series is
P
absolutely summable. That is, we write ht (x) =
cneinx where the
n=,1

L. PETRACOVICI AND J. ROSENBLATT

coecients cn = cn(ht ) are the usually Fourier coecients of ht, and

hence are dependent
at least on the value of t. Hence, for ,  x  ,
1
P
,
itx
we have e =
cne,inx. So
n=,1

f (t) = 21
=

1
P

n=,1

,

1
X
,1

Hence, jf (t)j  sup jf (n)j

n2Z

Z

F (x)

1
X
n=,1

cne,inxdx

cnFb(n):

1
P
n=,1

jcnj. Therefore, if we can arrange for

jcnj to be bounded independently of the choice of t and at, this

would give us the desired inequality.

To achieve this, for a xed t, choose an integer m such that m 
t < m + 1. Then consider the function dt (x) = ei(t,m)x . This has
derivatives for x 2 [,; ] of all orders. Also, if one is considering only
derivatives up to a xed level, then these derivatives would be bounded
independently of t and m since 0  t , m < 1. We can extend this
function for each t to being a 2-periodic function, with the extension
Ht(x) being given by
(

i(t,m)x if ,  x  ;
Ht (x) = e
bt (x) if ,  x  , or   x  .

for suitable choice of bt (x). We can also arrange for Ht (x) to be an

innitely dierentiable function, and for there to be a bound on the
second derivative of Ht over [,; ] which is independent of t, because
of the existence of bounds on the derivatives of dt(x). This was the
point of using t , m in place of1just t. But then there is a constant C
P
such that for all t, we have
jcn(Ht)j  C . Indeed, by Theorem
n=,1
4.12 in Zygmund [30], there is a constant V which is independent of
t, depending actually only on the bound for Ht00 , such that for all n,
we have jcn(Ht)j  jnjV2+1 . So now let ht(x) = eimx Ht(x), implicitly
dening at from
bt . This only shifts the Fourier series of Ht, and so we
1
P
also have
jcn(Ht)j  C .
n=,1

INTRODUCTION TO SAMPLING

Remark 3.1 (Carleson's inequality). Let us point out that the core issue here
of getting a bound C which is independent of t for the se1
P
ries
jcn(Ht)j can be resolved easily also by using Carleson's inn=,1
equality. This inequality
is the following. Suppose
one has a Fourier
1
1
P
P
series H (x) =
cneinx. Let kH k`1 =
jcnj. Let kH k2 =
 R

,

jH (t)j2

1=2

n=,1

n=,1

. Then


kH k`1  6 kH k2kH 0k2

1=2

+ jHb (0)j:
This can be used above since actually our derivatives Ht0(x) are being
kept uniformly bounded independently of t. How is Carleson's inequality proved? It is easy to see that it is enough to prove the inequality
in the case that H 6= 0, but Hb (0) = co = 0. Now, split H (x) into two
N
P
P
sums, 1 =
cneinx and 2 =
cneinx. By the Cauchy-Schwartz
n=,N

jnj>N

inequality, k1 k`1   2N + 1kH k2. But

also by the Cauchy-Schwartz
1=2  P 1 1=2
P
inequality, k2 k`1 
jcnj2n2
. Hence,
n2
jnj>N

jnj>N

p
kH k`1  2 N kH k2 + p2 kH 0k2:

N
0
Take N to be the smallest integer greater than or equal to kkHH kk22 . Notice
0
0
that since co = 0, we have kH k2  kH 0k2. Hence, kkHH kk22  N  2kkHHkk22 .
Thus, with this value of N , one gets
1=2

1=2
p
kH k`1  2 2 kH k2kH 0k2 + 2 kH k2kH 0k2 :
The same analysis as the above can be made in the standardized
band [,2w; 2w]. The above was for the case that w = 12 . The
conclusion can be stated as the following proposition.
Proposition 3.2. For any  < 2w, there exists a constant C depending only on  and w, such that for any F 2 L1 [,; ], one has the
bound
sup jFb(t)j  C sup jFb( 2nw )j:
t2R

n2 Z

Remark 3.3. This theorem has been noticed by many others over the
years. A nice reference for it is the article by Cartwright [6]. Actually,
she derives the existence of such a sampling bound for a more general

L. PETRACOVICI AND J. ROSENBLATT

class of functions, entire functions of nite type. Moreover, she obtains

the result by working with an interpolation formula which is closely
related to the cardinal series. Explicitly, she assumes that f (z) is an
entire function and for some   0, one has jf (z)j = O(ejzj+) for all
 > 0. Then she shows that if  < , there is a constant C , depending
on  , , such that sup jf (x)j  C sup jf (n)j. To do this, she examines
x2R
n2Z
the behavior of an interpolation formula of Tschakalo's for functions
of this type, namely:
h
n
oi
X
(3.2) f (z) = sin(z f 0(0) + f (0) +
(,1)nf (n) 1 + 1 :

z n2Znf0g
z,n n

See Tschakalo [25] for a proof that will work in this context, or see
Zygmund [30], (7.22), Vol. II. See also Timan [23], pp. 183-184, where
her result is discussed.
One can think of the sampling bound above in terms of convolutions
too. Let F 2 L1 (R ). If G 2 L2 (R ), then the convolution F ? G
is in L2 (R ). Suppose we want the best constant C such that for all
G 2 L2 (R ), one has
kF ? Gk2  C kGk2:
It is well-known that this constant C is sup 2jFb(t)j. The same fact
t2R
holds if one is carrying out the analogous operations on the circle T,
i.e. for 2-periodic functions. But then the constant C is sup 2jFb(l)j.
l2Z
The sampling bound above is thus saying saying this. If a function F
is integrable over [,; ] where  < , then there is a constant C (),
depending only on , such that if one has
kF ? GkL2[,;]  C kGkL2[,;]
for all 2-periodic functions G, then one has
kF ? GkL2(R)  C ()C kGkL2(R)
for all G 2 L2 (R ). This is the content of Proposition 3.2 when w = 21 .
Another interesting aspect of sampling functions of small support
comes out in the above. Indeed, x  <  and let F 2 L1 [,; ].
Dene E to be the mapping which sends the sequence sF = (Fb(n) :
n 2 Z) to the value Fb(t). The Riemann-Lebesgue Lemma says that
lim Fb(n) = 0. Hence, E is a well-dened, continuous linear funcjnj!1
tional on co(Z) with jE (sF )j  C ()ksF k1. Note: at rst E is only
being dened on a proper subspace of co(Z), but the sampling bound

INTRODUCTION TO SAMPLING

not only makes the mapping continuous there, but allows its continuous extension to the closure of this subspace in co (Z), which is actually all of co (Z) in this case. But since the dual space of co (Z) is
l1(Z), there is a uniquely determined sequence ( t(n) : n 2 Z) 2 l1(Z)
1
P
such that E (sF ) =
t(n)sF (n) for all sequences sF . Moreover,
n=,1
k tkl1(Z)  C (). That is, we have an absolutely summable sequence
1
P
( t(n)) such that
j t(n)j  C (), and such that we have the abn=,1
stract interpolation formula

Fb(t)

1
X
n=,1

t(n)Fb(n)

for all F 2 L1 [,; ]. Using the denition of FT, one sees easily that
1
P
this means that for all x 2 [,; ], we have eixt =
t (n)eint. By
n=,1
examining the proof of Proposition 3.2, one can see that this type of
expansion was also being used there, and as a result the coecients
t(n) can be chosen to be continuous in t. But one reason to include
this alternative derivation is that it will always work whenever there
is a sampling bound as in Proposition 3.2, even when the samples are
some irregular or non-arithmetic sequence.
Compare this abstract interpolation formula with Shannon's formula
Equation 2.1 or with the cardinal series of Whittaker Equation 3.1.
The coecients t are not uniquely determined as in those formulas,
but they are absolutely summable. It would be perhaps worthwhile
to write down a nice, explicit choice of these coecients; there may
even already be ones available in the existing literature. But the point
we would like to observe is that the Shannon's formula or the cardinal
series of Whittaker could only be better than these expansions if the
coecients in the those interpolation formulas were also absolutely
summable. This would mean there could be a sampling bound as above
even with  = 2w. However, without the restriction on the support
of F to being well within the band [,2w; 2w], there actually cannot
be such sampling bounds. That is, the constant C () grows to 1 as 
tends to 0. These means that while SST and the cardinal series in it
give a reconstruction of the signal f , they cannot give sampling bounds
of the kind in Proposition 3.2.
First, we prove that bounds do not always exist.

10

L. PETRACOVICI AND J. ROSENBLATT

Proposition 3.4. There is no C such that for all F 2 L1 [,; ],

sup jFb(t)j  C sup jFb(n)j:
t2R

n2Z

Proof. Consider the function Fh = h1 [0;h]. Its FT is denoted by fh ; so

h
1 R e,ixt dx = e,iht ,1 . Hence, jf (t)j  min(1; j 1 ). Take the
fh(t) = 2h
h
,2ith
jthj
0
dierence Dh(x) = Fh(x , ) , Fh(x +  , h) of two shifts of Fh. The
FT of Dh is then fh(t)(eit , eit(h,) ). This has a value Dch( 12 ) which
is about 21 (ei=2 , e,i=2 ) = ipfor small values of h. Now jDch(t)j  1
everywhere. Also, jDch(t)j  2 h for all integers t with jtj  h,3=2 by
the basic estimate jfh(t)j  jth1 j , which is useful like this for large t. But
p
for integers t with jtj  h,1=2 , jDch(t)j  2 h. Indeed, these integers t,
jDch(t)j  jeitx , eit(h,) j
= j1 , eith j
 2jthj
p
 2 h:
So choose a set fhk g of some K values of h which are rapidly decreasing
K
P
to zero, and let F (x) = K1 Dhk (x). If we arrange for h3k  hk+1 for
k=1

all k, then any integer is in the interval [h,k 1=2 ; h,k 3=2] for at most one
value of k. Thus, for large K , we have jFb( 12 )j is approximately 1 and
jFb(n)j is uniformly small on the integers.
Remark 3.5. a) See the construction for Proposition 3.6 for another
example of a class of functions which show that there is no such inequality.
b) Sometimes inequalities like this can be shown to fail by constructing a function which has the right-hand side begin 0 while the left-hand
side is not. But in this case, SST shows that this is not possible.
Let us now see how the constant in the sampling bound grows as the
support of F gets closer to being all of [,2w; 2w]. With no loss in
generality, we can take w = 12 and work just on the interval [,; ]. We
claim that the constant C in Proposition 3.2 is on the order of log( ,1  ).
First, to see that this is the right order to bound a suitable choice of
C , we follow the proof of Proposition 3.2 a little more carefully. We
can choose the extension of ei(x,m)t so that Ht (x) is 2-periodic and
continuously dierentiable. We can arrange also that the derivative

INTRODUCTION TO SAMPLING

11

of Ht (x) is of bounded variation, with variation no more than C ,1  ,

ct (n)j  C for n 2 Znf0g:
for some absolute constant C . But then jH
jnj
c
Also, by integrating by parts, one sees also that jHt(n)j  C, n12 for
n 2 Znf0g: Hence,


X
jHct(n)j  C log  ,1  ;
1jnj(,),1
and for some absolute constants C1 and C2 , we have
 1 1
X
X
jHct(n)j  C1
 ,  n2
(,),1 jnj
(,),1 jnj
 C2:
Thus, for a constant C which is independent of t, we have
1
X

jHct(n)j  C log  ,1  :
n=,1
The proof of Proposition 3.2 is thus showing that the constant in the
sampling bound is O(log( ,1  ).
But also, the constant has to be at least on this order. To see this,
for N  10, we construct a function fN 2 L1 [, + N1 ;  , N1 ] so that
jfcN (n)j  C for all n 2 Z, but such that jfcN ( 21 )j  C log N . Indeed,
start with
8 N
<P

m imx if jxj   ;
GN (x) = :m=1(,1) e
0
if jxj  .

Then for any t which is not a whole number,

d
GN (t)

=
=

N
X
m=1
N
X

(,1)m

1 Z ei(m,t)x dx
2
,

(,1)m 1 2 sin(m , t)

2
m,t
m=1

N
X
1
sin(t) :
=
m=1 t , m

12

L. PETRACOVICI AND J. ROSENBLATT

So jd
GN ( 12 )j  C log N . However, it is also clear that for any integer k,
we have jd
GN (k)j  1. Now let
(

1
FN (x) = GN (x) if jxj 1 , N ;
0
if  , N < jxj  .
R

jFN (x) , GN (x)jdx  2 for all N . It follows that for some

constants C3 and C4, we have jFcN (k)j  C3 for all k 2 Z, while
jFcN ( 21 )j  C4 log N . The result of the argument above is inherently

Hence,

,

another proof of Proposition 3.4, since the constants C (; w) in Proposition 3.2 do not remain bounded as  increases to 2w. The statement
summarizing the above in general is this one.

Proposition 3.6. There are absolute constants A and B , such that

for any  < 2w, if C (; w) is the smallest constant such that Proposition 3.2 holds, then A log( 2w1, )  C (; w)  B log( 2w1, ).
Remark 3.7. The functions used to prove the related results Proposition 3.4 and Proposition 3.6 are similar in that they both have large
variance, enabling them to have Fourier transforms small in appropriate places and not small somewhere else.

3.2. Jittered Sampling. Another important issue in sampling signals is that it may not always be feasible to sample regularly as in the
classical case. The sample points may be inherently irregular for reasons beyond the control of the system, e.g. noise creates irregularity.
Such a sample is called a jittered sample. However, if one has some
control of the rate of sampling, and some control on the bandwidth
of the signal, a jittered sample can be as eective in reconstruction as
an arithmetic sample. The critical theorem that gives such results is a
theorem of Bernstein about entire functions of controlled growth.
Let f (z) be a function analytic in the entire complex plane. We
say that f is of nite type if there exist constants A and  such that
for all z 2 C , we have jf (z)j  Aejzj: The type of f is the smallest
constant  such that for all  > 0 and all suciently large jzj, we have
jf (z)j  e(+)jzj: For example, if F 2 L1 [,; ], then its FT f can be
extended to all of C , by using the same integral formula, and it is of
nite type. The type of f is , if no smaller interval would support the
function F .

INTRODUCTION TO SAMPLING

13

Theorem 3.8 (Bernstein's Theorem). Let f be an entire function of

nite type  . Then

sup jf 0(x)j   sup jf (x)j:

x2R

x2R

Proof. See Zygmund [30], Theorem 7.24, Vol. II, for a complete proof.
We just want to observe that the argument proceeds from a careful inspection of the same Tschakalo's interpolation formula used by
Cartwright.
Now assume that we have a sampling set S  R with the property
that for some , to be specied later, one has for any x 2 R , there is
some s 2 S such that jx , sj  . We take a band-limited signal f ,
whose INVFT is supported on a band [,; ]. We want to estimate
f (to). For s 2 S , let I (to; s) denote the closed interval between to and
s. There is no Mean Value Theorem for analytic functions which is
quite as simple as the one for real-valued functions of a real-variable,
but we can estimate, using Bernstein's Theorem, that
jf (to)j  jf (to) , f (s)j + jf (s)j
 sup jf 0(t)jjx , sj + jf (s)j
t2I (to ;s)

  sup jf (t)j + sup jf (s)j:

t2R
s2S
Assuming then that  < 1, and varying to 2 R , this gives
sup jf (t)j  1 ,1  sup jf (s)j:
t2R

s2S

t2R

s2S

Thus, depending on , if  is suciently small, then any jittered sample

with gap constant no more than  satises this sampling bound: there
exists a constant C (; ) such that for all F 2 L1 [,; ], we have
sup jFb(t)j  C (; ) sup jFb(s)j:
This jittered sampling theorem does not give as good a result as the
regular sampling theorem would in case the sample is an arithmetic
progression. However, it does have a wider range of application in
that the sample does not need to be arithmetic to apply this principle.
Also, by the same argument as the one above for arithmetic sampling,
if there is a sampling bound, then one can obtain at least abstract
reconstruction formulas that allow one to evaluate Fb(t) if one knows
the values of Fb(s) for all s 2 S . See the articles by Benedetto and
by Feichtinger and Grochenig in Benedetto and Frazier [1] for information concerning these types of representations. See also Clark, Palmer

14

L. PETRACOVICI AND J. ROSENBLATT

and Lawrence [7]. More fundamentally perhaps, this inequality gives

criteria which show when we would know that F = 0, given that we
have Fb(s) = 0 for all s 2 S . This implicitly is related to the behavior
and existence of non-harmonic series; see Levinson [17] for additional
material on this topic.
4. The Generalized Sampling Theorem = GST
The material in this section is a brief summary taken mostly from
Jerri [12]. There are many more interesting details and possible topics
that we have not covered which are connected with GST.
R
4.1. GST. Let I be an interval and L2(I ) = f' : I j'(x)j2 dx < 1g.
Suppose that g 2 L2(I ), and let
Z

f (t) = K (x; t)g(x)dx

I
for any t 2 R . For this to be well-dened, we need to suppose that for
each t, K (x; t) 2 L2(I ). Now suppose also that there is a countable set
E = ftng such that fK (x; tn )g is a complete orthogonal set in L2 (I ).
Then
f (t) =
where

1
X

n=,1

Sn(t) = S (t; tn) =

f (tn)Sn(t):

I RK (x; t)K (x; tn ) dx :

2
I jK (x; tn )j dx

We call Sn(t) the sampling functions. This expansion is the abstract

analogue of Shannon's interpolation formula in Equation 2.1.
To see that this is a valid expansion for g 2 L2(I ), write g(x) =
1
P
cnK (x; tn ), the expansion of g with respect to fK (x; tn )g. Here
n=1

cn =
So

g(x)K (x; tn) dx R f (tn)

2 dx = jK (x; tn )j2 dx :
j
K
(
x;
t
)
j
n
I
I

RI

K (x; t)g(x) dx = f (t) =

=

1
X
n=1

1 f (t )
X
n
n=1

K (x; t) K (x; tn) dx

2
I jK (x; tn )j dx

RI

f (tn)Sn(t):

Here are some examples of kernels K (x; t) that one might use:

INTRODUCTION TO SAMPLING

K (x; t) = Jm (xt);
K (x; t) = Pt(x);
K (x; t) = Lt (x);

15

Bessel function of the rst kind of order m;

Legendre function, see Campbell [5],
Laguerre function on (0; 1); see Jerri [13]:

We illustrate the calculation with the Bessel transform that comes

from the rst of these examples of kernels. Recall that 1
the Bessel funcP
k 2k
tion of the rst kind of order m is given by Jm (x) = xm 22k+(,m1)k!(xk+m)! :
k=0
R
We take f (t) = 01 xJm (xt)g(x) dx: In this case the sampling function
Sn(t) is derived as:

Sn(t) = S (t; tm;n) =

=

R1

m (xt) Jm (xtm;n) dx
0 xJ
R1
2
0  [Jm (xt)] dx
2tm;n Jm (t)
;
2
(tm;n , t2)Jm+1 (tm ; n)

where ftm;n : n = 1; 2; 3; : : : g are the zeros of Jm .

The nal sampling interpolation becomes:

f (t) =

1
X

Jn(t)
:
f (tm;n) (t2 2,tm;n
2
m;n t )Jm+1 (tm;n )
n=1

There is the possibility of applying SST to functions that can be

sampled by the GST. If a function can be expanded by the GST, then
it can also be expanded by the use of SST, when considering integral
transforms with kernels such as
Ptm(x) = Legendre function,
Ctv (x) = Gegenbauer function,
Utv (x) = Chebyshev function of the second kind, see Jerri [14].
There is no advantage in using GST when the function is represented
by a double inverse FT with nite limits. This is the case only when we
assume that in the application, there will only be interest in working
with the Fourier transform, and no other transform. The advantage of
GST is that it can be used when considering other integral transforms,
and it is useful when dealing with time-varying systems.

16

L. PETRACOVICI AND J. ROSENBLATT

4.2. Time-Varying Systems. The interpretation of GST for the special case K (w; t) = e,iwt (i.e. the SST) is that f (t) is the output of
an ideal low-pass lter with impulse response K (t; tn ) = 2wS (t; tn) =
sin 2w(t,n(2w)) , with the input taken to be f (t ) = f (n=2w). The inn
(t,n(2w))
terpretation of the GST is that f (t) is the output of a band-limited,
low-pass lter in the sense of some general integral transforms, with
a time-varying impulse response that is related directly to the sampling function S (t; tn), and with input ff (tn)g. This can be done for a
transform-limited function
Z

fI (t) = (w)K (w; t)F (w) dw

I

with the Fourier-type inverse

F (w) =

(t)K (w; t) f (t) dt

where  is a weighting function. The limits of the integration are not

necessarily nite and will depend on the particular transform being
considered.
4.3. Error Analysis in Sampling Representation. We give a brief
summary of the various errors that may arise in the practical implementation of the sampling theorems. See Papoulis [20] for more information
about sampling errors.
Here generally the types of errors that arise in sampling:
Truncation Error { This results when only a nite number of samples
are used, instead of the innite samples needed for a complete interpolation,
Aliasing Error { This is caused by violating the Nyquist sampling rate
that is appropriate for the particular band-limited signal being analyzed,
Jitter Error { This is caused by sampling at times dierent from the
appropriate sampling points,
Round-o Error { The usual problem of round-o error can occur in
calculations,
Amplitude Error { This is the result of uncertainty in measuring the
amplitude of the sample values.
We do not want to try to discuss all of these types of errors that can
occur. But let us consider some the truncation error and the possible

INTRODUCTION TO SAMPLING

17

bounds for it. For the band-limited signal

Z 2w
1
f (t) = 2
e,ixtF (x) dx;
,2w
the SST representation Equation 2.1 has a truncation error "T which
is the result of considering the partial sum fN (t) using only the central
2N + 1 terms of the SST interpolation formula. So
X  n  sin  (2wt , n)
"T = f (t) , fN (t) =
f 2w (2wt , n) :
jnj>N

A result of Tsybakov and Iakovlev [26] gives the estimate

s
p
2
j"T (t)j   E jsin(2wt)j 2w(TT2 , t2)
whereR ,T  t  T . Here E denotes total nite energy, given by
2
E = ,2w
2w jF (x)j dx:
Another type of bound for the truncation error is given by Helms
and Thomas [9]. They take
KX
+N  
wt , k) 0 < N < 1
fN (t) =
f 2nw sin(2
2wt , n
n=K ,N
with a = 2w, K being the integer valued function given by 2wt ,
(1=2)  K (t)  2wt + (1=2), and N being some xed integer. So
"T = f (t) , fN (t) ! 0 as N ! 1 provided f is band-limited to
a = 2w. They consider also the additional assumption that there
is actually a band-limit 2rw, where 0 < r < 1, and get an explicit
estimate on the truncation error:
j"T (t)j  2 N4(1M, r) ;
where M = sup jf (t)j.
t2R

4.4. Conditions for a Stable Sampling Expansion. As in the case

of the solutions of many problems in applied mathematics, we do not
just seek a solution, but we seek a stable solution, so that a small error
in the input produces a corresponding small error in the output. Yao
and Thomas [29] considered the stability of the sampling expansion in
the sense that a small error in reading the sample values produces only
a corresponding small error in the energy of the recovered signal. They
gave this condition for what we will call here energy stable sampling:
for the class of functions f (t) band-limited to [,a; a] to possess an

18

L. PETRACOVICI AND J. ROSENBLATT

energy stable sampling with respect to a sequence (tn), there must

exist a positive nite constant C (independent of f ) such that
Z 1

,1

jf (t)j2 dt  C

jf (tn)j2:

One should assume that the summation is over the natural numbers N
or the integers Z.
This condition is one half of the frame condition. Let Ba denote all
functions f (t) which are in L2 (R ) and band-limited to [,a; a]. We say
that (eitn x : n) is a frame for Ba if there exists constants c and C such
that for all f 2 Ba ,

)j2

jf (tn 

Z 1

,1

jf (t)j2 dt  C

jf (tn)j2:

The SST gives the best known simple example of a frame and hence
an energy stable sampling sequence. Let f 2 B2w and let an = f ( 2nw ).
Then the Parseval identity from classical analysis is saying that we
have
1
2 Z 1
X 
n
1 X
1

2
2
2w n=,1 janj = 2w f ( 2w ) = ,1 jf (t)j dt:
R1
So SST is showing that every signal f of nite energy ,1
jf (t)j2 dt
and bandwidth 4w may be completely recovered at Nyquist rate of
2w/second, but also, the recovery is energy stable in the sense that
a small error in reading sample values produces only a corresponding
small error in the energy of the recovered signal.
It is implicit in the energy stability condition that (tn ) is a set of
uniqueness: that is, if f 2 L2 [,a; a] and f (tn) = 0 for all n = 1; 2; 3; : : :
then f = 0. Also, the characters (eitn x : n = 1; 2; 3; : : : ) must span
L2[,a; a] because of the bandwidth in question. What the maximum
such bandwidth will be in terms of (tn) is the focus of the article by
Beurling and Malliavin [3], which depends also on the results in Beurling and Malliavin [4]. Of course, this stability estimate would follow
from (eitn x : n = 1; 2; 3; : : : ) being a frame for L2 [,a; a], but energy
stability is not as strong as the frame requirement. It is interesting
to ask what is needed for a set of characters to be a frame of a class
of band-limited signals. The results in Dun and Schaeer [8] give
good conditions sucient for a set of characters to be a frame. They
consider the two conditions: 1) there is d > 0 and L < 1, such that
for all n, jtn , nd j  L, and 2) there is  > 0, such that for any m 6= n,
jtm , tnj  . Under conditions 1) and 2), they show that (eitn x) is a

INTRODUCTION TO SAMPLING

19

frame for any Ba for which a < d. See also Jaard [11] for a characterization of when one has the frame condition, extending the work
of Dun and Schaeer [8]; and see also the article by Benedetto in
Benedetto and Frazier [1] for a discussion of the frame condition and
reconstruction in this context. We will need to use the result of Dun
and Schaeer [8], giving sucient conditions for a set of characters to
have the frame condition.
It is probably worth making a distinction here between energy stability and what we will call proper energy stability, this being energy
stability in which the data series is always nite. The energy stability
criterion P
on the face of it does not require that for all f 2 Ba , the
value of jf (tn)j2 is nite. If this series is innite, then the bound
n
is not a useful one because no conclusions about
P perturbations can be
made. It is not hard to see actually that jf (tn)j2 is nite for all
n
f 2 Ba if and only if the left-hand sidePof the frame condition
holds i.e.
R1
2
there is a constant A < 1 such that jf (tn)j  A ,1 jf (t)j2 for all
n
f 2 Ba . Indeed, clearly this inequality is sucient for the niteness.
Conversely, if the series is always nite, then we can consider the linear
mapping T given by T (F ) = (Fb(tn) : n). By assumption, this mapping
is well-dened from L2 [,a; a] to l2 (I ) where I = N of I = Z. It is
easy to show that this mapping T has a closed graph. Therefore, T is
continuous, i.e. there exists a nite constant A such that
X

)j2

jf (tn  A
=A

Z 1

,1

Z 1

,1

jF (x)j2 dx
jf (t)j2dt:

for all F 2 L2 [,a; a]. This is to say, a properly energy stable sampling
sequence (tn) is nothing more than onepfor which (eitn x) forms a frame
for Ba . For example, the sequence ( jnj : n 2 Z) is energy stable
because it contains a smaller sampling set that gives a frame for Ba by
the theorem of Dun and Schaeer. Here a is actually not restricted.
In particular, it is a set of uniqueness of all band-limited functions
simultaneously. However, it does not give a frame for any Ba , with
a > 0 and it is not a properly energy stable sampling sequence for any
Ba with a > 0. See Jaard [11].
We have actually been considering here another type of stability
for a sampling set, which could be called uniform stability. This is
the stability that comes from having a constant C such that for all

20

L. PETRACOVICI AND J. ROSENBLATT

functions f (t) which are band-limited to [,a; a], one has

sup jf (t)j  C sup jf (tn)j:
t2R

This means that a small error in reading the sample values produces
only a corresponding small uniform error in the recovered signal. It is
implicit in this type of stability as with energy stability that the (tn)
form a set of uniqueness for the corresponding functions and that the
characters corresponding to them span L2[,a; a].
Uniqueness is an interesting property in itself. It is generally a much
less restrictive condition than either stability condition. For example,
it is well-known that the complex zeros of a band-limited signal f have
certain distributional properties. These properties were rst observed
in Titchmarsh [24], and then later are discussed in Levinson [17], Chapter III. See also Rosenblatt
[21]. These results say
p for instance that a
p
sequence like tn = n; n = 1; 2; 3; : : : , or tn = jnj; n 2 Z, is a set
of uniqueness for all bandwidths simultaneously because the density of
the sequence gets bigger without bound. Moreover, such a sequence can
always be modied to have p
larger and larger gaps and still be a set of
uniqueness. In particular, ( n : n = 1; 2; 3; : : : ) is a set if uniqueness
for all bandwidth class, but is not
p uniformly stable or energy stable
for any bandwidth class. Also, ( jnj : n 2 Z) is energy stable, uniformly stable, but not properly energy stable for any bandwidth. See
Jaard [11].
Another interesting issue is to understand the relationship between
the two types of stability, energy stability and uniform stability. First,
the Shannon sampling theorem and Proposition 3.4 show that one can
have a properly energy stable sampling method which is not uniformly
stable for the bandwidth in question. Perhaps, the reverse is true too,
but we do not have an example of this at this time.
However, it is interesting to note that except for a loss of some bandwidth, uniform stability does imply energy stability, and at least proper
energy stability also implies uniform stability. This can be seen as follows. Assume that (tn) is a uniformly stable sampling method for bandwidth [,a; a], with constant C . Take the function F (x) = eibx1[,a;a]
and sample f = Fb. Since jf (t)j  jt1,bj , while f (b) = a , the largest
possible distance G from b to an element of (tn) satises G  Ca . Hence,
the largest possible closed interval in R which contains no elements of
(tn) is of length at most 2G  2aC . Now take the expanded lattice
( md : m 2 Z). Assume that 1d  3aC . Then let L = 31d . It follows that
each interval [ md , L; md + L] contains some sample point tnm since actually 2L  2aC . For each m, choose such a tnm 2 [ md , L; md + L]. Since

INTRODUCTION TO SAMPLING

21

3L = d1 , there is a gap size at least L = 31d between the sample points

tnm . We now use Theorem I in Dun and Schaeer [8] to conclude
that the characters (eitnm x : m 2 Z) form a frame for the bandwidth
[, ; ] as soon as 0 < < d. Therefore, sampling by (tnm ) is properly
energy stable, and certainly therefore over sampling by (tn) is energy
stable for any bandwidth less than 3aC . Notice that the new bandwidth
is proportional to the original one, and not much bandwidth is lost if
C is not too large.
Conversely, this argument can sometimes show in a similar way that
certain energy stable sampling methods cannot have too much gap
between terms. It is not clear what the general situation is, but if the
sampling set (tn) is energy stable, with constant C , and there exists at
least some gap  between terms, then there is a larger gap G, which
is a function only of C and , so that any closed interval of length
2G must contain some element of the sampling set. In particular, by
passing to a smaller sampling set, one can have this sampling method
provide a frame of characters too. This argument can be used together
with the results characterizing frames, to show that a properly energy
stable sampling sequence is uniformly stable for some possibly smaller
bandwidth. Also, this is giving at least some general restriction on a
set of uniqueness with some separation between terms if it is to have
any type of stability: it cannot have too large a gap. In particular,
if the set of uniqueness is one-sided and has some separation between
terms, then it certainly cannot be stable in either sense. This can be
extended without too much trouble to cases where the gaps between
terms get small but not too quickly. However, if the sequence is onesided like this so it cannot be uniformly stable for any bandwidth, but
is unbounded with gaps between terms getting small too fast, then it
can be energy stable in an improper sense. An example is the sequence
tn = log n; n = 1; 2; 3; : : : . Results of Titchmarsh [24] show that bandlimited functions cannot vanish too rapidly. From this1one can show
P
that for any non-zero band-limited signal f , the series jf (log n)j2 =
n=3
1.
Finally, it is worthwhile saying something about Landau [16]; he considered there the possibility of improving the Nyquist rate. Emphasizing that only stable sampling is meaningful in practice, he proved the
following very sharp and useful results:
(a) Stable sampling cannot be performed at a lower rate than the
Nyquist rate;

22

L. PETRACOVICI AND J. ROSENBLATT

(b) Data cannot be transmitted as samples at a rate higher than the

Nyquist rate, regardless of the location of the sampling instants, the
nature of the set of frequencies which the signal occupies, or the method
of construction.
4.5. Derivative Sampling. Derivative sampling is useful when samples of both a band-limited function f with frequencies in [,2w; 2w]
and its derivative f 0 are available. The interpolation formula becomes
1 h   
  n i  sin  (wt , 2n)=2 2
X
n
n
f (t) =
f w + t , w f0 w
:

(
wt
,
2
n
)
=
2
n=,1

This is valid if f and f 0 are bounded, continuous, and integrable over

R.

4.6. Band-pass Functions. Up to now the band region was an interval centered at the origin. What happens if the band region is
[a , 2w; a + 2w]? This is actually just a simple translation, so SST
interpolation becomes
1
 

X
sin

(2
wt
,
n
)
n
n
exp ia t , w :
f (t) =
f( )
n=,1 2w  (2w + n)
But it becomes much less trivial in knowing how to sampling eciently
if the band region is of the form [,a , 2w; ,a + 2w] [ [a , 2w; a +
2w].
Acknowledgment: We would like to thank R. Kaufman who helped
quite a bit in understanding the behavior of sampling bounds, as presented in Section 3.
References
[1] J. Benedetto and M. Frazier, eds., Wavelets: mathematics and applications,
CRC Press, Boca Raton, Ann Arbor, London, Tokyo, 1994
[2] J. Benedetto, Irregular sampling and frames, in Wavelets, a Tutorial in Theory
and Applications, C. K. Chui, ed., 1992, Academic Press, Boston, pp. 445-507.
[3] A. Beurling and P. Malliavin, On the closure of characters and the zeros of
entire functions, Acta Math, 118 (1967) 79-93.
[4] A. Beurling and P. Malliavin, On Fourier transforms of measures with compact
support, Acta Math, 107 (1962) 291-309.
[5] L. L. Campbell, A comparison of the sampling theorems of Kramer and Whittaker, SIAM J. 12 (1964) 117-130.
[6] M. Cartwright, On certain integral functions of order one, Oxford Quarterly
Journal of Math, 7 (1936) 46-55.

INTRODUCTION TO SAMPLING

23

[7] J. J. Clark, M. R. Palmer, and P. D. Lawrence, A transformation method

for the reconstruction of functions from non-uniformly spaced samples, IEEE
Trans. ASSP, 33, no. 4 (1985) 1151-1165.
[8] R. J. Dun and A. C. Schaeer, A class of nonharmonic Fourier series, Transactions AMS, 72, no. 2 (1952) 341-366.
[9] H. D. Helms and J. B. Thomas, Truncation error of sampling theorem expansion, Proc. IEEE, 50 (1962) 179-184.
[10] J. R. Higgins, Five short stories in cardinal series, Bulletin AMS, 12, no. 1
(1985) 45-89.
[11] S. Jaard, A density criterion for frames of complex exponentials, Mich. Math.
J., 38 (1991) 339-348.
[12] A. J. Jerri, The Shannon sampling theorem{its various extensions and applications: a tutorial review, Proc. IEEE, 65, no. 11 (1977) 1565-1596.
[13] A. J. Jerri, Sampling expansion for the L -Laguerre integral transform, J.
Research of Nat. Bur. Standards, B. Math Sciences, 80B (1976) 415-418.
[14] A. J. Jerri, On the application of some interpolating functions in physics, J.
Research Nat. Bur. Standards, B. Math Sciences, 73B, no. 3 (1969) 241-245.
[15] H. J. Landau, Necessary density conditions for sampling and interpolation of
certain entire functions, Acta Math, 117 (1967) 37-52.
[16] H. J. Landau, Sampling data transmission and the Nyquist rate, Proc. IEEE,
55, no. 10 (1967) 1701-1706.
[17] N. Levinson, Gap and Density Theorems, AMS Colloquium Publications, 26,
1940, New York.
[18] R. Meeks, Advanced Topics in Shannon Sampling and Interpolation Theory,
Springer-Verlag, 1993, New York.
[19] R. Meeks, Introduction to Shannon Sampling and Interpolation Theory,
Springer-Verlag, 1991, New York.
[20] A. Papoulis, Error analysis in sampling theory, Proc. IEEE, 54, no. 7 (1966)
947-955.
[21] J. Rosenblatt, Determining a distribution from the modulus of its Fourier
transform, Complex Variables and Applications, 10 (1988) 319-326.
[22] J. B. Thomas and B. Liu, Error problems in sampling representation, IEEE
Int. Conv. Rec. (USA), 12, part 5 (1964) 269-277.
[23] A. F. Timan, Theory of Approximation of Functions of a Real Variable, Hindustan Publishing Corp., Delhi, 1966.
[24] E. C. Titchmarsh, The zeros of certain integral functions, Proc. London Math
Soc. (2) 25 (1926) 283-302
[25] L. Tschakalo, Zweite Losung der Aufgabe, Jahresber. der Deutschen Math.
Ver. 43 (1933) 11-12.
[26] B. S. Tsybakov and V. P. Iakovlev, On the accuracy of restoring a function
with a nite number of terms of Kotel'nikov series, Radio Engineering and
Electronics 4, no. 3 (1959) 274-275.
[27] E. T. Whittaker, On the functions which are represented by the expansion of
interpolating theory, Proc. Roy. Soc. Edinburgh, 35 (1915) 181-194.
[28] J. M. Whittaker, The Fourier theory of cardinal functions, Proc. Math. Soc.
Edinburgh, 1 (1929) 169-176.
[29] K. Yao and J. B. Thomas, On some stability and interpolating properties of
nonuniform sampling expansions, IEEE Trans. Circuit Theory, 14 (1967) 404408.

24

L. PETRACOVICI AND J. ROSENBLATT

[30] A. Zygmund, Trigonometric Series, Vols. I and II, Cambridge University Press,
Cambridge, 1979.
(L. Petracovici) Department of Mathematics, University of Illinois at
Urbana, Urbana, IL 61801

E-mail address : [email protected]

(J. Rosenblatt) Department of Mathematics, University of Illinois at

Urbana, Urbana, IL 61801

E-mail address :

[email protected]