THEORY AND PRACTICE OF IRREGULAR SAMPLING

Hans G. Feichtinger

Mathematisches Institut
Universit at Wien
Strudlhofg 4
A-1090 Wien, Austria
Karlheinz Grochenig

Department of Mathematics
The University of Connecticut
Storrs, CT 06269 USA
Abstract
This chapter presents methods for the reconstruction of band-
limited functions from irregular samples. The rst part discusses algo-
rithms for a constructive solution of the irregular sampling problem.
An important aspect of these algorithms is the explicit knowledge of
the constants involved and ecient error estimates. The second part
discusses the numerical implementation of these algorithms and com-
pares the performance of various reconstruction methods.
Although most of the material has already appeared in print, sev-
eral new results are included: (a) a new method to estimate frame
bounds, (b) a reconstruction of band-limited functions from partial
information, which is not just the samples; (c) a new result on the
complete reconstruction of band-limited functions from local averages;
(d) a systematic exposition of recent experimental results.
Contents
1 Introduction: The Irregular Sampling Problem 3
Part I: Quantative Theory 6
2 Iterative Reconstruction Algorithms 6

The rst author was partially supported by the Austrian National Bank through the
project Experimental Signal Analysis and the Kammer der Gewerblichen Wirtschaft
von Wien through the Hochschuljubilaumspreis.

The second author acknowledges the partial support by a DARPA Grant AFOSR-90-
0311.
1
3 Qualitative Sampling Theory 10
3.1 Theorems following Paley-Wiener . . . . . . . . . . . . . . . . 11
3.2 Theorems following Dun-Schaeer . . . . . . . . . . . . . . . 12
3.3 The POCS Method . . . . . . . . . . . . . . . . . . . . . . . . 14
4 The Adaptive Weights Method 15
5 Variations 21
5.1 A Discrete Model for Irregular Sampling . . . . . . . . . . . . 22
5.2 Irregular Sampling in Higher Dimensions . . . . . . . . . . . . 24
5.3 The Uncertainty Principle and Recovery of Missing Segments . 27
6 Irregular Sampling of the Wavelet- and the Short Time Fourier
Transform 30
Part II: The Practice of Irregular Sampling 34
7 Setup for Numerical Calculations 34
8 Hard and Software Environment: PC386/486 and SUN, IR-
SATOL/MATLAB 36
9 Results and Observations from the Experiments 36
9.1 Comparison of Dierent Methods . . . . . . . . . . . . . . . . 36
9.2 Special Features of the POCS Method . . . . . . . . . . . . . 39
9.3 Measuring the Error . . . . . . . . . . . . . . . . . . . . . . . 41
9.4 Sampling Geometry and Spectral Size . . . . . . . . . . . . . . 42
9.5 Actual Spectrum and Known Spectrum . . . . . . . . . . . . . 44
9.6 Choice of Weight Sequence . . . . . . . . . . . . . . . . . . . . 45
9.7 Relevance and Choice of Relaxation Parameters . . . . . . . . 46
9.8 Theory and Practice . . . . . . . . . . . . . . . . . . . . . . . 48
10 Additional Results and Observations 48
10.1 Dierent Implementations . . . . . . . . . . . . . . . . . . . . 48
10.2 2D Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
10.3 Performance, Repeated Reconstruction Tasks . . . . . . . . . 49
10.4 Signal Reconstruction from Local Averages . . . . . . . . . . . 50
10.5 Further Improvements, Recent Numerical Experiments . . . . 51
11 Conclusion 51
2
1 Introduction: The Irregular Sampling Problem
The sampling problem is one of the standard problems in signal analysis.
Since a signal f(x) cannot be recorded in its entirety, it is sampled at a
sequence x
n
, n ZZ. Then the question arises how f can be reconstructed
or at least approximated from the samples f(x
n
).
In practice only a nite number of samples can be measured and stored.
In this case the irregular sampling problem becomes a nite-dimensional
problem and can be approached with methods of linear algebra, see Part II
of this chapter.
For the theoretical model we will assume that an innite sequence of
samples is given. Then the reconstruction of arbitrary signals is clearly an
ill-dened problem. In most applications, however, it is reasonable to assume
that the signal is band-limited, i.e. that it does not contain high frequencies.
If the samples are equally spaced, then the famous sampling theorem by
Shannon, Whittaker, Kotelnikov and others provides an explicit reconstruc-
tion. Assume that f L
2
(IR) and that supp

f [, ], then for any

the signal f has the representation

f(x) =

n=
f(n)
sin (x n)
(x n)
(1)
where the series converges in L
2
(IR) and uniformly on compact sets. The
case of regular sampling is well understood and has led to countless variants.
We refer to the reviews [8, 34, 37].
The problem for irregular samples x
n
, n ZZ, is more dicult. Some
diculties stem from the fact that Poissons formula is no longer applicable;
also, for very irregular sampling sets much more attention has to be paid to
the convergence of the innite series involved.
The irregular sampling problem has inspired many dicult and deep the-
orems which culminate in the work of A. Beurling, P. Malliavin, and H.
Landau [3, 4, 40]. We apologize in advance that in this chapter many theo-
retical developments will not receive the treatment they deserve. Instead we
refer to J. Benedettos excellent surveys [1].
The objective of this article is quite dierent. Our goal is the develop-
ment of ecient reconstruction algorithms and ultimately their numerical
implementation. This requires a quantitative theory of irregular sampling.
Merely qualitative statements of the form There exists a constant, such
that ... may be insucient in concrete applications. Even so-called explicit
reconstruction formulas may be of little use, when their ingredients are very
complicated.
Instead we would like to nd algorithms that are simple enough to be used
in numerical computations. Furthermore, in order to make a step beyond the
3
trial-and-error method in numerical computations, all numerical constants
should be explicit and should yield reasonable error estimates. For iterative
algorithms, which are our main concern, this should also include estimates
for the number of iterations required to achieve a desired accuracy of the
reconstruction.
In view of this program the chapter is divided into two parts. In the rst
part, written by the second-named author, a class of iterative algorithms is
developed which satises the requirements listed above.
Section 2 starts with an abstract description of iterative reconstruction
algorithms and discusses the concept of frames. We introduce a new method
for the estimation of frame bounds. In Section 3 we briey outline results in
the literature which have been or could be used in numerical work with ir-
regular sampling. In Section 4 we derive the basic reconstruction algorithm,
the so-called adaptive weights method. Since this algorithm has proved
to be the most ecient method in the numerical experiments, we give a de-
tailed proof for this theorem. In Section 5 we present several variations of
this method. We discuss (a) a discrete model for irregular sampling. The
adaptive weights method in this situation can be implemented directly. (b)
We touch briey on the problem of irregular sampling in higher dimensions,
which is a very important problem in image processing. (c) We improve a
recent result of Donoho and Stark [15] on the uncertainty principle and the
problem of missing segments. In Section 6 we give a recent application of
sampling results for band-limited functions to wavelet theory. We state ir-
regular sampling results for the wavelet transform and the short time Fourier
transform.
The second part, written by the rst-named author, is devoted to the
numerical implementations of these algorithms and provides empirical results
which either support or complement the theoretical results given in Part I.
Section 7 gives a short description of the discrete model underlying the
numerical experiments. It is followed by a summary of the hard- and soft-
ware environment (MATLAB on PC and SUN workstations), on which the
collection IRSATOL of MATLAB M-les is built (cf. [23])). The main
section of this part is Section 9. It gives a general comparison of the ve
most important methods studied so far in terms of iterations and time. We
also discuss the sensitivity of dierent methods with respect to irregularities
in the sampling set (cf. subsection 4), and the role of relaxation parameters
(subsection 7). More practical questions are treated in subsection 3 (how to
measure errors). In the last subsection theoretical and practical convergence
rates are compared. Section 10 deals among others with questions of speed.
In subsection 3 we indicate how to modify the algorithms for repeated recon-
struction tasks with xed spectrum and sampling set. Subsection 4 shows
that the reconstruction from averages works surprisingly well. The last sec-
tion summarizes the arguments for the use of ADPW, the Adaptive Weights
4
Method, as proposed by the authors.
5
Part I
A Quantitative Theory of Irregular Sampling
2 Iterative Reconstruction Algorithms
In this section we discuss the generic form of many reconstruction methods
that are currently used in signal analysis as well as in other areas of math-
ematics. This class of algorithms is distinguished (a) by their linearity, (b)
by their being iterative, and (c) by the geometric convergence of successive
approximations. From the functional analytic point of view they are just the
inversion of a linear operator by a Neumann series.
A particular case of these methods are the algorithms based on frames.
We outline a new method for constructing frames which will be very useful
to obtain explicit estimates in the following section.
Proposition 1 Let A be a bounded operator on a Banach space (B, | |
B
)
that satises for some positive constant < 1
|f Af|
B
|f|
B
for all f B. (2)
Then A is invertible on B and f can be recovered from Af by the following
iteration algorithm. Setting f
0
= Af and
f
n+1
= f
n
+ A(f f
n
) (3)
for n 0, we have
lim
n
f
n
= f (4)
with the error estimate after n iterations
|f f
n
|
B

n+1
|f|
B
. (5)
Proof: By inequality (2) the operator norm of Id A is less than .
This implies that A is invertible and that the inverse can be represented as
a Neumann series:
A
1
=

n=0
(Id A)
n
(6)
and any f B is determined by Af and the norm-convergent series
f = A
1
Af =

n=0
(Id A)
n
Af.
The reconstruction (4) and the error estimate (5) follow easily after we
have shown that the n-th approximation f
n
as dened in (3) coincides with
the n-th partial sum

n
k=0
(IdA)
k
Af. This is clear for n = 0, since f
0
= Af
by denition. Next assume that we know already that
f
n
=
n

k=0
(Id A)
k
Af. (7)
6
Then we obtain for n + 1
n+1

k=0
(Id A)
k
Af = Af +
n+1

k=1
(Id A)Af
= Af + (Id A)
n

k=0
(Id A)
k
Af (by induction)
= Af + (Id A)f
n
= f
n
+ A(f f
n
).
Now clearly f = lim
n
f
n
and since

k=n+1
(Id A)
k
= (Id A)
n+1
A
1
,
we obtain
|f f
n
|
B
= |

k=n+1
(Id A)
k
Af|
B
= |(Id A)
n+1
A
1
Af|
B

n+1
|f|
B
.
It is a noticeable strength of this reconstruction algorithm that it allows
to determine a priori the number of iterations required to achieve a given
precision of the approximation f
n
.
To prove a sampling theorem combined with an ecient reconstruction
algorithm, it is thus necessary to assemble a linear approximation operator
A which contains only the sampled values of f and then prove a norm esti-
mate of the form (2). This strategy has been applied successfully to prove
irregular sampling theorems in spaces of analytic functions [11], for short
time Fourier transforms and wavelet transforms [26], and for a general class
of spaces of band-limited functions [18, 19, 20].
An important instance of this algorithm occurs in the presence of frames
in a Hilbert space. Frames were introduced by R. Dun and A. Schaeer
[17] as a generalization of Riesz bases in their fundamental work on irregular
sampling of band-limited functions. The concept was revived in wavelet
theory through the work of I. Daubechies and A. Grossmann [12, 13, 14].
Denition. A sequence e
n

nIN
in a (separable) Hilbert space H with inner
product , ) is called a frame, if there exist constants 0 < A B, such that
A|f|
2

n=1
[f, e
n
)[
2
B|f|
2
for all f H. (8)
The left-hand inequality implies not only the uniqueness of f, but also
that f depends continously and stably on the frame coecients f, e
n
). With
Proposition 1 and (8) we can eectively reconstruct f from the sequence
f, e
n
). For this we introduce the so-called frame-operator
Sf =
2
A + B

n=1
f, e
n
)e
n
. (9)
7
Then S is a positive operator, and (8) implies the inequalities

B A
B + A
|f|
2
|f|
2

2
A + B

n
[f, e
n
)[
2
(10)
= (Id S)f, f)
B A
B + A
|f|
2
.
Since Id S is self-adjoint, (10) entails that the operator norm of Id S
on H is smaller than
BA
B+A
< 1. Now Proposition 1 applies, and yields an
iterative reconstruction of f from Sf with a rate of convergence determined
by =
BA
B+A
. Since the input for Sf consists of the frame coecients f, e
n
)
only, this is indeed the desired reconstruction.
After setting g
n
= S
1
e
n
, we may write the reconstruction of f from
f, e
n
) in closed form
f = S
1
Sf =

n
f, e
n
)g
n
. (11)
We get a non-orthogonal expansion of f with respect to the basis functions
e
n
for free:
f = SS
1
f =

n
S
1
f, e
n
)e
n
=

n
f, g
n
)e
n
. (12)
Note that the two frame bounds A and B play a vital role in the algorithm
and determine the speed of convergence of the algorithm. It is therefore of
practical importance to estimate A, B as sharp as possible.
The upper estimate in (8) is rarely a problem. It expresses the continuity
of f f, e
n
)
nIN
from H into l
2
and it is not dicult to derive reasonable
estimates for B. On the other hand, the lower estimate is usually the dicult
and deep part of the argument, and good estimates for the lower frame bound
A are mostly beyond reach.
This problem is often dealt with in the following way: one is content with
the mere existence of frame bounds A and B and considers the family of
operators
S

f =

n
f, e
n
)e
n
where is the so-called relaxation parameter. With an estimate similar to
(10) one obtains
|f S

f| ()|f|
where () = max[1 A[, [1 B[ and () < 1 for small values of
. As long as numerical ecacy is not an issue, this trick is good enough.
Let us mention that the choice
0
= 2/(A + B) gives the optimal value
(
0
) = (B A)/(B + A), and that the actual rate of convergence is the
better ( closer to this optimal value (
0
)) the closer is to
0
.
8
Here we present a new method to obtain frame bounds. In the applica-
tions to irregular sampling of band-limited functions this alternative strategy
leads to better and explicit estimates of the frame bounds.
This new method is based on the observation that in most cases it is more
natural to work with an approximation operator of the form
Af =

n
f, e
n
)h
n
(13)
for two sequences e
n
and h
n
in H.
Proposition 2 Suppose that the sequences e
n

nIN
and h
n

nIN
have the
following properties: there exist constants C
1
, C
2
> 0, 0 < 1, so that

n
[f, e
n
)[
2
C
1
|f|
2
(14)
|

n
h
n
|
2
C
2

n
[
n
[
2
(15)
and
|f

n
f, e
n
)h
n
| |f| (16)
for all f H and (
n
)
nIN
l
2
. Then e
n
is a frame with frame bounds
(1 )
2
/C
2
and C
1
, and h
n
is a frame with bounds (1 )
2
/C
1
and C
2
.
Proof: Let A be dened as in (13), then A is a bounded operator on
H by (14) and (15) and A is invertible by (16) with A
1
=

n=0
(Id A)
n
and the operator norm of A
1
is less than (1 )
1
. Using (14) and (15) we
obtain the following inequality
|f|
2
= |A
1
Af|
2
(1 )
2
|

n
f, e
n
)h
n
|
2
(17)
(1 )
2
C
2

n
[f, e
n
)[
2
(1 )
2
C
1
C
2
|f|
2
.
Next we verify two inequalities which are dual to (14) and (15):
|

n
e
n
|
2
C
1

n
[
n
[
2
(18)

n
[f, h
n
)[
2
C
2
|f|
2
(19)
for all (
n
) l
2
and f H. Since |

n

n
e
n
| = sup
f:f=1
[

n

n
e
n
, f)[,
(18) follows from
[

n
e
n
, f)[
2
= [

n
e
n
, f)[
2
(

n
[
n
[
2
)(

n
[f, e
n
)[
2
) C
1
|f|
2

n
[
n
[
2
.
9
(19) follows similarly. To see that h
n

nIN
is a frame, we can apply the same
argument as in (17) to the adjoint operator
A

f =

n
f, h
n
)e
n
.
It follows that f can be reconstructed from the frame coecients f, e
n
)
in two ways: (a) we use the approximation operator A, which requires the
use of both frames e
n
and h
n
, and the iteration dened in Proposition 1,
or (b) we use the frame operator Sf =

n
f, e
n
)e
n
for a suitable relaxation
parameter , e.g. = 2C
2
((1 )
2
+ C
1
C
2
)
1
.
Clearly, in its abstract form, Proposition 2 does not have much substance,
but in the following sections it will be a valuable guide to prove the existence
of frames, irregular sampling theorems and explicit numerical constants.
3 Qualitative Sampling Theory
This section provides an overview of constructive, but qualitative solutions
of the irregular sampling problem. A detailed discussion of these results and
their applicability and numerical eciency will be carried out in Part II.
To x notation, let B

be the space of band-limited functions

B

=
_
f : |f| = (
_
IR
[f(x)[
2
dx)
1
2
< and supp

f [, ]
_
. (20)
Here the L
2
-norm corresponds to the energy of the signal f, 2 > 0 is the
bandwidth of f. The Fourier transform f Tf =

f is normalized by
Tf() =

f() =
_
IR
f(x)e
ix
dx (21)
its inverse is f(x) =
1
2
_
IR

f()e
ix
d, so that we have

f, g) = 2f, g). (22)

B

is a closed subspace of L
2
(IR) with some remarkable properties.
1. A multiple of the Fourier transform,
1

2
T is a unitary transformation
from B

onto L
2
(, ).
If
A
is the characteristic function of a set A and L
x
f(t) = f(t x) is
translation by x, then for x IR
T
1
_
e
ix

[, ]
()
_
(t) =
sin (t x)
(t x)
=

L
x
sinc

t (23)
10
where sinc

x =
sinx
x
for x ,= 0 and sinc

0 = 1. From (23) one concludes

immediately the following.
2. Since
1

2
e
in/
, n ZZ is an orthonormal basis for L
2
(, ) the
collection
_

2
L
n

sinc

, n ZZ is an orthonormal basis of B

.
3. B

is a reproduction kernel Hilbert space, i.e. point evaluations f

f(x) are continous on B

. More precisely, for f B

f(x) =
1
2
_
IR

f()e
ix

[, ]
()d (24)
=
_
f(t)
sin (t x)
(t x)
dx =

f, L
x
sinc

) =

f sinc

(x).
The regular sampling theorem (1) for the Nyquist rate =

follows
immediately from 2. and 3.
4. Every f B

extends to an entire function f(x + iy) of exponential

growth . By a famous theorem of Paley-Wiener, the converse is also true
[50].
Each of these properties of B

can be taken as the foundation of a sam-

pling theory.
The most profound results are based on the identication of band-limited
functions with entire functions, see [50, 3, 4, 42]. The work of Nashed and
Walter [48] and a part of our own work [19, 20] are based on the reproducing
kernel property (24). Finally, any statement about a nonharmonic Fourier
series () =

n
C
n
e
ix
n

in L
2
(, ) is equivalent to a sampling theorem
by (23) and (24). Indeed, many well-known irregular sampling theorems are
just reformulations about earlier results on nonharmonic Fourier series, cf.
[59] for an introduction.
3.1 Theorems following Paley-Wiener
Paley and Wiener [50] were interested in the question which perturbations
e
ixn
, n ZZ of the orthonormal basis e
in/
, n ZZ are still a Riesz
basis for L
2
(, ). Recall that a Riesz basis of a Hilbert space is the image
of an orthonormal basis under an invertible operator.
In the context of irregular sampling the sharp form of their result can be
stated as follows:
Theorem 1 Suppose that the irregular sampling set x
n
, n ZZ satises
sup
nZZ

x
n

L <

4
. (25)
Then there exists a unique sequence g
n
B

, so that every f B

has
the representations
f(x) =

nZZ
f(x
n
)g
n
(x) =

nZZ
f, g
n
) L
x
n
sinc

x. (26)
11
The series converges in L
2
(IR) and uniformly on compact sets. The col-
lections g
n
, n ZZ and L
xn
sinc

, n ZZ are Riesz bases of B

and

L
xn
sinc

, g
n
) =
kn
for k, n ZZ.
Remarks
1. A proof of a slightly weaker statement can be based on Propositions
1 and 2. For this one considers the approximation
Af =

f(
n

) L
xn
sinc

(27)
on B

. Since f =

f(
n

) L
n

sinc

, it is not dicult to show that

|f Af| |f| for all f B

and some < 1,

provided that [x
n

n

[ <
ln 2

. See [16, 60] for details. Using the adjoint

operator A

f =

f(x
n
)L
n

sinc

, Proposition 1 provides an iterative recon-

struction of f from A

f, in other words, a reconstruction from the irregular

samples f(x
n
).
2. It is more dicult to nd the optimal constant in (25). The precise
bound

4
in (25) is due to Kadec [38].
3. Levinson [42] has found explicit formulas for the g
n
s in terms of
Lagrange interpolation functions. Set g(x) = (xx
0
)

n=1
(1
x
xn
)(1
x
x
n
),
then g
n
(x) =
g(x)
(xx
n
)g

(x
n
)
. In this way one obtains an explicit reconstruction
of f B

from its samples f(x

n
).
4. The theorem of Paley-Wiener and Kadec has been reformulated and
interpreted many times, see for example [5, 6, 56, 34].
5. Finally, the sampling density is the same that is required for regular
sampling. If n(r) denotes the minimum number of samples found in an
interval of length r, then lim
r
n(r)/r =

is exactly the Nyquist density.

3.2 Theorems following Dun-Schaeer
Theorem 1 permits only a slight perturbation of the regular sampling ge-
ometry. Weakening the notion of basis, Dun and Schaeer in their very
inuential paper [17] found the following Theorem.
Theorem 2 Suppose that there exist constants 0 < < 1, , L > 0, so that
the sampling set x
n
, n ZZ satises [x
n
x
m
[ > 0 for n ,= m and
sup
nZZ

x
n

L < . (28)
Then one can nd constants 0 < A B, depending only on , , L, so that
for any f B

A|f|
2

nZZ
[f(x
n
)[
2
B|f|
2
. (29)
12
Remarks
1. Since f(x
n
) =

f, L
xn
sinc

), (29) is equivalent the statement that

the collection L
xn
sinc

, n ZZ, is a frame for B

. Proposition 1 now
provides an iterative reconstruction of f from f(x
n
) by means of the frame
operator
Sf =

f(x
n
) L
x
n
sinc

.
2. The theorem of Dun-Schaeer is very powerful and its proof is still
considered dicult. Many applications and several early numerical treat-
ments of irregular sampling are just reformulations of this theorem, see e.g.
[46, 55, 56, 2].
From a practical and numerical point of view the range of this theorem is
limited, because the constants A and B are only implicit. Whereas it is not
dicult to see that the upper bound B in (29) behaves like
L

, of the lower
bound A only its existence is known.
As long as there is no concern about the numerial ecacy or the required
number of iterations, one can work with an appropriate relaxation parameter
determined by trial-and error (!), see [45], and achieve a reasonable recon-
struction from irregular samples. Since the frame reconstruction has been
used implicitely by Marvasti, and already before in a very similar form by
Wiley, we well refer to this method in Part II as the Wiley-Marvasti (for
short WIL/MAR), or frame method.
3. Let n(r) be the minimal number of samples in an interval of length r.
Then we obtain from (28)
lim
r
n(r)
r
=

>

for the sampling density. Compared to the regular sampling theorem (1),
Theorem 2 always requires oversampling.
In 1967 H. Landau has proved a converse [40].
Theorem 3 If an irregular sampling set x
n
, n ZZ generates a frame
L
x
n
sinc

, n ZZ for B

, then necessarily
liminf
r
n(r)
r
>

.
Recently, S. Jaard has completely characterized all sampling sequences
that generate frames [36]. The proof requires both theorems of Dun-
Schaeer and of Landau. For the formulation we recall that a set
x
n
, n ZZ is called relatively separated [19] if it can be divided into a nite
number of subsets so that [x
n
x
m
[ > 0 for a xed > 0, n ,= m, and
x
n
, x
m
in the same subset.
13
Theorem 4 A sampling sequence x
n
, n ZZ generates a frame
L
xn
sinc

, n ZZ for B

, if and only if x
n
, n ZZ is relatively sepa-
rated and
liminf
r
n(r)
r
>

.
This is sometimes regarded as the most general sampling theorem. As
explained above, its applicability is marred by the absence of estimates for
the frame bounds. The relaxation parameter in the frame reconstruction
has to be determined by trial-and-error, for very irregular sampling sets the
convergence of the frame algorithm can be very slow.
3.3 The POCS Method
The POCS-method (= projection onto convex sets) has been used in [57] to
interpolate band-limited functions from a nite number of irregular samples.
More precisely, given N samples (x
n
, y
n
), n = 1, 2, . . . , N, the task is to
nd a band-limited f B

, so that f(x
n
) = y
n
. An easy argument shows
that this problem is always solvable and that the set of solutions is a closed
convex subset of B

contained in a subspace of codimension N.

Although this is a very weak form of the irregular sampling problem, we
discuss its treatment briey because the POCS-method has brought a new
idea into the eld of irregular sampling.
The abstract background of the POCS-method is described as follows [58]
(it goes back to classical papers by John von Neumann [49], Kaczmarcz [41],
cf. [53] and [35] for more recent references): Let C
n
, n = 1, . . . , N, be closed
convex subsets of a Hilbert space H and let P
n
be the (nonlinear) projection
operators from H onto C
n
, uniquely dened by |f Pf| = min
gCn
|f g|.
The problem is to nd C :=

N
n=1
C
n
and the corresponding projection in
terms of the P
n
.
The theory of the POCS-method [58] guarantees that the projection Pf
of a given f onto C can be computed iteratively by f
0
= f,
f
k+1
= P
N
P
N1
. . . P
2
P
1
f
k
k 0 (30)
and
f = lim
k
f
k
in H.
More generally, with appropriate relaxation parameters
n
, 0 <
n
<
2, n = 1, . . . , N and the operators T
n
= f +
n
(P
n
f f), the projection onto

N
n=1
C
n
= C can be approximated by g
0
= f,
g
k+1
= T
N
T
N1
. . . T
1
g
k
k 0
and f = lim
k
g
k
.
14
In the application of the POCS-method to the interpolation of a band-
limited function f from the samples (x
n
, y
n
), n = 1, . . . , N one looks at
the closed convex sets of B

C
n
=
_
f B

: f(x
n
) = y
n
_
and has to nd an f

N
n=1
C
n
by means of the iteration (30). In this
problem the projection P
n
onto C
n
is calculated explicitly in [57] as
P
n
f =

f sinc

_
y
n

f sinc

(x
n
)
_
L
xn
sinc

. (31)
The iteration (30) then produces a band-limited function f B

, f(x
n
) =
y
n
, n = 1, . . . , N. This algorithm extends immediately to higher dimensions.
It works reasonable well as long as the number of samples is small, < 100,
say. However, when the number of sampling points becomes larger, both the
POCS method described above, as well as the so-called single step method,
which requires the calculation of a (pseudo-) inverse matrix, whose size is
the number of sampling points, becomes very slow or not feasable at all. We
will report about this, but also about ways to enhance its performance in a
simple way, in section 9.2, cf. also [24].
4 The Adaptive Weights Method
In this section we discuss a new method for the reconstruction of band-limited
functions from irregular samples. It is practical and has produced the best
numerical results so far. Whereas Dun-Schaeers theorem is too hard to
be proved in this review, the adaptive weights method is easy enough to be
proved completely. The additional benet of this method are explicit frame
bounds and estimates for the rate of convergence.
We assume that the sampling points x
n
, n ZZ, are ordered by mag-
nitude, . . . < x
n1
< x
n
< x
n+1
< . . . and that lim
n
x
n
= . The
sampling density is measured by the maximal length of the gaps between the
samples, i.e.
= sup
nZZ
_
x
n+1
x
n
_
. (32)
We denote the midpoints between the samples by y
n
, y
n
=
xn+x
n+1
2
and the
distance between the midpoints by w
n
=
x
n+1
x
n1
2
. If the maximal gap
length is , then we have obviously y
n
x
n


2
and x
n
y
n1


2
.
According to Proposition 1 we have to design an approximation of the
identity operator on B

that requires only the samples f(x

n
) as input, in
order to obtain an iterative reconstruction of f.
15
An obvious and simple approximation procedure is to interpolate f by a
step function rst, followed by the projection onto B

.
Denoting the characteristic function of the interval [y
n1
, y
n
) by
n
and
the orthogonal projection from L
2
(IR) onto B

by P, where P is given by
(Pf)

=

f
[, ]
, f L
2
(IR), then our rst guess for an approximation of
f by its samples is the operator
Af = P
_

nZZ
f(x
n
)
n
(33)
It is easy to verify that A maps B

into B

, see also below, so the question

is how well A approximates the identity on B

.
For this we need two simple inequalities.
Lemma 1 (Bernsteins inequality) If f B

, then f

and
|f

| |f|. (34)
Proof: Using (22) we have,
|f

|
2
=
1
2
|(f

|
2
=
1
2
|i

f()|
2

2
2
|

f|
2
=
2
|f|
2
.
Lemma 2 (Wirtingers inequality) If f, f

L
2
(a, b) and either f(a) = 0 or
f(b) = 0, then
_
b
a
[f(x)[
2
dx
4

2
(b a)
2
_
b
a
[f

(x)[
2
dx. (35)
Lemma 2 follows from [32], p. 184, by a change of variables. It is inter-
esting to note that (35) has also been used to obtain uniqueness results for
band-limited functions [54].
Proposition 3 If = sup
nZZ
(x
n+1
x
n
) <

, then for all f B

|f Af| <

|f|. (36)
Consequently, A is bounded and invertible on B

with bounds
|Af|
_
1 +

_
|f| (37)
and
|A
1
f|
_
1

_
1
|f|. (38)
16
Proof: Since f = Pf = P(

nZZ
f
n
) for f B

, and since the

characteristic functions
n
have mutually disjoint support, we may write
|f Af|
2
= |Pf P
_

nZZ
f(x
n
)
n
_
|
2
(39)
|

nZZ
_
f f(x
n
)
_

n
|
2
=

nZZ
_
y
n
y
n1
[f(x) f(x
n
)[
2
dx.
Now split each integral into two parts,
_
y
n
y
n1
. . . =
_
x
n
y
n1
+
_
y
n
xn
and apply
Wirtingers inequality:
_
y
n
y
n1
[f(x) f(x
n
)[
2
dx

2
(x
n
y
n1
)
2
_
x
n
y
n1
[f

(x)[
2
dx +
4

2
(y
n
x
n
)
2
_
y
n
x
n
[f

(x)[
2
dx

2
_
y
n
y
n1
[f

(x)[
2
dx.
In the last step we sum over n and use Bernsteins inequality to obtain

nZZ
_
yn
y
n1
[f(x) f(x
n
)[
2
dx

2

nZZ
_
yn
y
n1
[f

(x)[
2
dx (40)
=

2

2
|f

|
2

2
|f|
2
.
A combination of these estimates yields (36). (37) and (38) are obvious:
|Af| < |f[[ +|Af f|
_
1 +

_
|f|
and
|A
1
f| = |

k=0
(Id A)
k
f|

k=0
_

_
k
|f|.
To obtain a quantitative sampling theory, we apply Propositions 1-3 in
various ways.
Theorem 5 (Iterative Reconstruction from Irregular Samples)
If = sup
nZZ
(x
n+1
x
n
) <

, then f B

is uniquely determined by its

samples f(x
n
) and can be reconstructed iteratively as follows:
f
0
= Af = P
_

nZZ
f(x
n
)
n
_
(41)
f
k+1
= f
k
+ A(f f
k
) fork 0.
17
Then lim
k
f
k
= f in L
2
(IR) and
|f f
k
|
_

_
k+1
|f|. (42)
Proof: Combine Propositions 1 and 3.
To deduce further conclusions, we look once more at the approximation
operator A. With (17) and w
n
=
_

n
(x)dx =
x
n+1
x
n1
2
, A becomes
Af =

nZZ
f,

w
n
L
x
n
sinc

)
1

w
n
P
n
. (43)
This normalization is natural, because |
1

w
n
P
n
| 1. It is now easy to
verify the assumptions of Proposition 2. Therefore the collections

w
n
L
x
n
sinc

, n ZZ and
1

w
n
P
n
, n ZZ are frames for B

.
Theorem 6 Let x
n
, n ZZ be a sampling set with
= sup
nZZ
(x
n+1
x
n
) <

. Then:
(A) Weighted Frames. For any f B

_
1

_
2
|f|
2

nZZ
w
n
[f(x
n
)[
2

_
1 +

_
2
|f|
2
. (44)
Consequently, the collection

w
n

L
xn
sinc

, n ZZ is a frame with
frame bounds
A =
_
1

_
2
and B =
_
1 +

_
2
. (45)
(B) The Adaptive Weights Method. Any f B

can be recon-
structed from its samples f(x
n
) by the adapted frame algorithm:
f
0
=

2

2
+
2

nZZ
f(x
n
)w
n

L
x
n
sinc

:= Sf (46)
f
k+1
= f
k
+ S(f f
k
) k 0
with f = lim
k
f
k
in L
2
(IR) and
|f f
k
|
k+1
|f| (47)
where =
2

2
+
2

2
.
18
Proof of (A). For the left hand inequality of (44) we use Proposition 3 and
obtain
|f|
2
= |A
1
Af|
2

_
1

_
2
|P
_

nZZ
f(x
n
)
n
_
|
2
(48)

_
1

_
2
|

nZZ
f(x
n
)
n
|
2
.
Since the supports of
n
are mutually disjoint, we have
|

nZZ
f(x
n
)
n
|
2
=
_
[

n
f(x
n
)
n
(x)[
2
dx
=

n
_
y
n
y
n1
[f(x
n
)[
2

n
(x)dx =

nZZ
[f(x
n
)[
2
w
n
and (44) follows. For the right side we use

n
[f(x
n
)[
2
w
n
= |

f(x
n
)
n
|
2

_
|f| +|

n
f(x
n
)
n
f|
_
2
(49)
and the estimates (39) and (40). Since f(x
n
) =

f, L
x
n
sinc

), the second
statement is clear.
(B) now follows from Proposition 1, (9) and (10), when we use the relaxation
parameter
=
2
A + B
=
1
1 +
2

2
/
2
which gives as rate of convergence
=
B A
B + A
=
2

2
+
2

2
< 1.
The dual theorem is new and comes as a big surprise. Since for
f B

the inner product

f,
1

w
n
P
n
) = Pf,
1

w
n

n
) =

w
n
1
w
n
_
y
n
y
n1
f(x)dx (50)
is just a multiple of the average of f over the interval [y
n1
, y
n
], the fact that
the collection
1

w
n
P
n
, n ZZ is a frame for B

implies that any band-

limited function is completely determined by its local averages. Since it is
impossible to measure a sample of f at a precise instant x
n
as a consequence
of the uncertainty principle and technical limitations, the reconstruction from
averages is a more realistic approach to the sampling problem. In this sense
the reconstruction from averages can be considered as an extreme form of
stability of sampling theorems and an important addition to the common
error analysis [51, 21].
The precise statement is the content of the next theorem:
19
Theorem 7 (Reconstruction from local averages.)
Let x
n
, n ZZ be a sampling sequence with = sup
nZZ
(x
n+1
x
n
) <

, y
n
the midpoints y
n
=
xn+x
n+1
2
, w
n
= y
n
y
n1
, and let
n
be the local average

n
=
1
w
n
_
y
n
y
n1
f(x)dx for a function f.
Then every f B

is uniquely determined by its averages

n
and can be
reconstructed by the following algorithm:
f
0
=

nZZ

n
w
n

L
xn
sinc

(51)
f
k+1
= f
k
+ f
0

nZZ
__
yn
y
n1
f
k
(x)dx
_

L
x
n
sinc

k 0 .
Then f = lim
k
f
k
in L
2
(IR) and
|f f
k
|
_

_
k+1
|f|.
Proof: We rst compute the adjoint of the approximation operator A
from (33): for f, g B

we nd
A

f, g) = f, Ag) = f,

n
g(x
n
)P
n
)
=

n
Pf,
n
)

L
x
n
sinc

, g)
or in brief
A

f =

nZZ
_
1
w
n
_
y
n
y
n1
f(x)dx
_

w
n
L
x
n
sinc

. (52)
Under the hypothesis stated, Proposition 3 implies |f A

f|

|f| for
all f B

. (51) is just a concrete formulation of the iteration of Proposition

1. Since the input for this algorithm is f
0
= A

f and requires only the local

averages, this is the desired reconstruction.
According to the remarks after Proposition 2, a reconstruction can also
be executed by means of the corresponding frame operator
Sf =

nZZ
_
1
w
n
_
yn
y
n1
f(x)dx
_
P
n
(53)
and an appropriate relaxation parameter.
Remarks
1. The theorems of this section require a slightly more restrictive concept
of density than [40] and [17] used. This seems the price to be paid for explicit
constants and a numerically ecient theory.
20
2. The proof of Theorem 6 contains a new idea to compute frame bounds
explicitly. Compared with the diculty of Dun-Schaeers theorem [17] it
is almost disappointing that with a new method this should be so simple.
On the other hand, it is exactly this simplicity which allows to apply this
method to other situations which are intractable by entire function theory.
Some variations are discussed in the next section, we refer to [27] for further
sampling theorems.
3. It is notable that all estimates are independent of the distribution of
the sampling points. This is relevant when the local sampling density varies
and when the sampling points are not separated by a minimal distance. By
contrast the frame bounds in the unweighted case of [17] seem to be more
sensitive to irregularities of the sampling set.
This is seen dramatically in examples when Jaards frame criterion fails,
but the adaptive weights method still applies: choose = as the bandwidth
and let the sampling set consist of the points
k
2
, [k[ 20

n=1
j +
k
10
n
for 10
n
j < 10
n+1
, 0 k < 10
n
. The maximal gap for this set is 1/2, so
that Theorem 6 gives a weighted frame with estimates for the frame bounds
A =
1
4
, B =
9
4
and a rate of convergence =
4
5
. On the other hand, the
unweighted sequence L
xn
sinc

fails to be a frame!
Thus one obtains the following qualitative picture: adding more points,
in other words, more information to the sampling set makes the ordinary
frame reconstruction behave worse. The ratio B/A of the frame bounds
will increase, and the reconstruction from the samples will converge slower
with more information a rather counter-intuitive behaviour! On the other
hand, the adaptive weights methods is not aected, since the frame bounds
are uniform. An increase of information is likely to improve the performance
of this algorithm. See Part II for a numerical demonstration of this eect.
4. The algorithms of Theorems 5 and 6 are very special cases of the sampling
theorems in [19, 20] adapted to the L
2
-norm. Their quantitative analysis was
carried out in [27].
The rst form of Theorem 7 also appeared in [27]. There a higher sam-
pling rate is required, but very general averaging procedures are allowed.
5. The adaptive weights method is also used in wavelet theory. It serves
the construction of irregular wavelet and Gabor frames and allows to prove
irregular sampling theorems for wavelet and short time Fourier transforms
[26], Thm.U.
5 Variations
In this section we present several variations of the adaptive weights method.
Since the proofs are similar to those of the previous section, we give only
brief indications and refer to the original publications for details.
21
5.1 A Discrete Model for Irregular Sampling
The irregular sampling problem rarely arises in concrete applications and
in numerical implementations , if ever, in the form discussed so far. In
digital signal processing a signal s consists of a nite sequence of data
s(n), n = 0, 1, . . . , N 1, where N is usually large. The signal is sam-
pled at a subsequence 0 n
1
< n
2
< . . . < n
r
< N. Thus the problem of
irregular sampling is under which conditions and how s could be completely
reconstructed from its samples s(n
i
), i = 1, . . . , r.
This problem can be treated in complete analogy with band-limited func-
tions on IR. For this we extend the nite signals periodically, i.e.
s(j + lN) = s(j) for j = 0, 1, . . . , N 1 and l ZZ, and interpret s as
a function on the nite cyclic group ZZ
N
. The discrete Fourier transform on
ZZ
n
is dened as
s(k) = N

1
2
N1

n=0
s(n)e
2ikn/N
(54)
s s is a unitary operator on l
2
(ZZ
N
) = I C
N
and preserves the energy-norm
|s| = (

N1
n=0
[s(n)[
2
)
1
2
= | s|.
Using the periodicity of s and s and negative indices, the space of discrete
band-limited signals of bandwidth M N/2 is
B
M
=
_
s I C
N
[ s(k) = 0 for [k[ > M
_
.
The orthogonal projection from I C
N
onto B
M
is given by (Ps)

(k) = s(k)
for [k[ M and (Ps)

(k) = 0 for M < [k[ N/2. With proper care and the
discrete versions of Bernsteins and Wirtingers inequality [39], the previous
reconstruction Theorems 5 7 can all be translated into this discrete model
of irregular sampling.
We use the notation [q] for the largest integer q IR, l
i
= [
n
i
+n
i+1
2
] for
the midpoints between the sampling points,
i
= l
i+1
l
i
for the weights. To
be consistent with periodicity, we have to dene n
r+1
= n
1
+N, n
0
= n
r
N,
and l
r+1
= l
1
.
If
l
I C
N
is the sequence
l
(l) = 1,
l
(k) = 0 for k ,= l, then the
substitute of sinc

is
P
l
(n) =
sin(2M + 1)(n l)/N
N sin (n l)/N
. (55)
We can now state the discrete version of the adaptive weights method in
the exact form in which it is implemented.
Theorem 8 [28] : Let 0 n
1
< n
2
< . . . < n
r
< N be a subset of ZZ
N
and
d = max
i=1,...,r
(n
i+1
n
i
). If
2M
_
2[
d
2
] 1
_
< N (56)
22
and > 0 is a suciently small relaxation parameter, then every s B
M
can be reconstructed iteratively from its samples s(n
i
), i = 1, . . . , r, by
s
0
=
r

i=1
s(n
i
)
i
P
n
i
(57)
s
k+1
= s
k
+
r

i=1
_
s(n
i
) s
k
(n
i
)
_

i
P
n
i
.
If = sin
M
N
/ sin

4[d/2]2
, the algorithm converges to s for any , 0 < <
2
(1+)
2
. The rate of convergence is
|s s
k
| ()
k+1
|s| (58)
when () = max[1 (1 + )
2
[, [1 (1 )
2
[.
The proof follows the same steps as the proofs of Theorems 5 and 6.
Again all constants are explicit and depend only on the bandwidth M and
the maximal gap length d.
A critical evaluation of this theorem has to consider two aspects.
1. The irregular sampling problem for discrete band-limited signals is
nite-dimensional and amounts to solving the following system of linear equa-
tions for s(k), [k[ M.
N

1
2
M

k=M
s(k)e
2in
j
k/N
= s(n
j
) j = 1, . . . , r. (59)
Here both M and r are usually large, typically of order 100-1000, and the
system is overdetermined.
Since there is an extensive literature on the (iterative) solution of large
linear systems, e.g. [31], the question is rather to determine which solution
method is appropriate for the particular system (59). The iteration (57) can
be written in terms of matrix multiplication and is known as Richardsons
method for the solution of linear systems. The merit of Theorem 8 is the
explicit range for the relaxation parameter and the rate of convergence. From
the point of view of linear algebra we have proved that the condition number
of Richardsons method is small and that this algorithm is well adapted to
the irregular sampling problem.
2. If we identify a band-limited signal s B
M
with the trigonometric
polynomial p(x) =

M
k=M
s(k)e
2ikx
, then s(n
i
) = p(
n
i
N
), and the discrete
problem seems to be entirely trivial. The polynomial p is of order M and
therefore uniquely determined by its values at any 2M + 1 distinct points.
The well-known Lagrange interpolation formula then provides an explicit
reconstruction of p from the p(n
i
/N). However, Lagrange interpolation tends
23
to be fairly ill-conditioned and high-order Lagrange polynomials are unhandy
to use. In this sense Theorem 8 provides a practical solution, i.e. computable,
with error estimates and a good condition number.
For more on the discrete theory of irregular sampling see [29].
5.2 Irregular Sampling in Higher Dimensions
Whereas irregular sampling in one dimension is well-developed both in theory
and in its numerical exploitation, we are only at the beginning of a coherent
and explicit theory in higher dimensions. This is an active eld of current
research which oers many dicult and interesting problems.
We present rst some easy extensions of the one-dimensional theory which
are certainly well-known, but apparently not in print. To keep notation
simple, we restrict to dimension 2.
Given a compact set IR
2
let
B

(IR
2
) =
_
f L
2
(IR
2
) : supp

f
_
be the subspace of band-limited functions f with spectrum in and with
nite energy.
If is a rectangle, = [
1
,
1
] [
2
,
2
], then B

is the tensor
product of the corresponding subspaces of L
2
(IR):
B

(IR
2
) = B

2
. (60)
This means that B

(IR
2
) is the closed subspace of L
2
(IR
2
) generated by
all products f(x) g(y) with f B

1
, g B

2
.
The following abstract lemma allows to extend one-dimensional results
to higher dimensions.
Lemma 3 Let H = H
1

H
2
and suppose that e
m
, m ZZ H
1
is a
frame (a Riesz basis) for H
1
, and f
n
, n ZZ H
2
a frame (a Riesz basis
resp.) for H
2
. Then the collection e
m
f
n
, m, n ZZ is a frame (a Riesz
basis) for H
1
H
2
.
Proof: Let S
i
be the frame operators for H
i
, i.e., S
1
g =

m
g, e
m
)e
m
,
S
2
h =

n
h, f
n
)f
n
and S the frame operator on H
1
H
2
. We test S on an
element of the form g h H
1
H
2
and nd
S(g h) =

n
g h, e
m
f
n
)e
m
f
n
=

n
g, e
m
)h, f
n
)e
m
f
n
=
_

m
g, e
m
)e
m
_

n
h, f
n
)f
n
_
= S
1
g S
2
h.
24
In other words, S = S
1
S
2
, therefore its spectrum is the product of the
spectra of S
1
and S
2
. In particular, S is invertible and thus e
m
f
n
is a
frame. Its frame bounds are A
1
A
2
and B
1
B
2
, if A
i
, B
i
are the frame bounds
of e
m
and f
n
.
The case of a Riesz basis is treated similarly.
Theorem 9 Assume that = [
1
,
1
] [
2
,
2
] and that the sampling
set is of the form (x
m
, y
n
), m, n ZZ. Then any f B

is uniquely and
stably determined under any of the following conditions:
(a) (Paley-Wiener): [x
m

1
[ L
1
<

4
1
, and [y
n

2
[ L
2
<

4
2
for
all m, n ZZ.
(b) (Dun-Schaeer): [x
m
x
n
[ > 0, [y
m
y
n
[ > 0 for m ,= n
and [x
m


1
m

1
[ L
1
< , [y
n

2
n

2
[ L
2
< for some constants
> 0, L
i
> 0, 0 <
1
,
2
< 1.
(c) (Adaptive Weights Method): Both sequences are increasing and

1
= sup
mZZ
_
x
m+1
x
m
_
<

1
,
2
= sup
nZZ
_
y
n+1
y
n
_
<

2
.
In the rst case the collection
_
sin
1
(x x
m
)
(x x
m
)

sin
2
(y y
n
)
(y y
n
)
, n, m ZZ
_
is a Riesz basis for B

, in the second case it is a frame, in the third case is

a frame after multiplying with the weights
1
4
(x
m+1
x
m1
)(y
n+1
y
n1
). Its
frame bounds are A = (1

1

)
2
(1

2

)
2
and B = (1 +

1

)
2
(1 +

2

)
2
.
The proof follows from the one-dimensional theorems and Lemma 3.
The theorem of Paley-Wiener-Levinson (case a) has been extended to
sampling sets of the form (x
mn
, y
n
), m, n ZZ in [7].
It has been shown that appropriate versions of the reconstruction algo-
rithms of Theorems 5-7 converge for arbitrary spectrum and arbitrary
sampling sets x
n
, n ZZ IR
2
, provided that the maximal distance to the
next neighbors is small enough [19, 20]. A two-dimensional POCS-method
was proposed in [52].
To summarize, the known reconstruction methods for two-dimensional
irregular sampling require either a very restrictive product structure of the
sampling set or the strong faith that the used algorithm will converge some-
how, despite its qualitative nature.
The following theorem is a rst and moderate step towards a quantitative
sampling theory in IR
2
. Its proof is similar to the proof of Theorem 5 and
can be found in [27].
25
For the following theorem we measure the density of a sampling set
x
i
, i I by the size of rectangles centered at x
i
that is required to cover
IR
2
. For x = (x
1
, x
2
) IR
2
and = (
1
,
2
) IR
2
let R(x, ) denote the
rectangle
R(x, ) =
_
x
1

1
2
, x
1
+

1
2
_

_
x
2

2
2
, x
2
+

2
2
_
.
We say that a set x
i
, i I IR
2
is -dense, if

iI
R(x
i
, ) = IR
2
.
For a -dense set x
i
we choose a partition of unity
i
with the properties
0
i
1, supp
i
R(x
i
, ) and

i

i
1. Finally P

is the orthogonal
projection from L
2
(IR
2
) B

.
Theorem 10 Assume that = [
1
,
1
] [
2
,
2
] and that x
i
, i I
is any -dense subset of IR
2
satisfying

1
+
2

2
< ln 2. (61)
Then every f B

(IR
2
) is uniquely determined by its samples f(x
i
), i I
and it can be reconstructed iteratively by
f
0
= P

iI
f(x
i
)
i
_
(62)
f
k+1
= f
k
+ P

iI
(f(x
i
) f
k
(x
i
))
i
_
k 0.
Then f = lim
k
f
k
in L
2
(IR
2
) and
|f f
k
|
_
e

1
+
2

2
1
_
k+1
|f|. (63)
The proof consists of the fact that the operator Af = P

iI
f(x
i
)
i
)
satises
|Af f| (e

1
+
2

2
1)|f| (64)
for all f B

, see [27].
If the partition of unity consists of characteristic functions
i
=
V
i
with
V
i
R(x
i
, ), then one obtains by Theorem 10 a two-dimensional adaptive
weights method.
Corollary 1 Set w
i
=
_

V
i
(x) dx, then any f B

satises
_
2 e

1
+
2

2
_
2
|f|
2

iI
w
i
[f(x
i
)[
2
e
2
1

1
+2
2

2
|f|
2
. (65)
26
Remarks: 1. The corresponding reconstruction algorithm diers from
the adaptive weights method in Theorem 6B only in the details of notation.
We leave its formulation to the reader.
2. Once the estimate (64) is accepted, the corollary is proved in exactly
the same way as Theorem 6.
This corollary is of theoretical interest because it is the rst genuine
generalization of Dun and Schaeers theorem to higher dimensions. It is
of practical interest because it is the basis for the ecient implementations
of Part II.
5.3 The Uncertainty Principle and Recovery of Miss-
ing Segments
In [15] D. Donoho and P. Stark consider versions of the uncertainty principle
of the following form: if f L
2
(IR) is essentially concentrated on a mea-
surable set T IR and

f essentially concentrated on a measurable set W,
then
[T[ [W[ 2
where [T[ is the Lebesgue measure of T and where > 0 is small and depends
on the precise denition of essentially concentrated.
They motivate one of their applications in the following way: Often
the uncertainty principle is used to show that certain things are impossible
. . . We present . . . examples where the generalized uncertainty principle shows
something unexpected is possible; specically, the recovery of a signal or
image despite signicant amounts of missing information. [15], p. 912.
This is the content of the following theorem.
Theorem 11 [15]: Let W and T IR be arbitrary measurable sets with
[W[ [T[ < 2. Suppose that supp

f W and that f is known on the comple-
ment of T. Then f is uniquely determined and can be stably reconstructed
from f (1
T
) by the algorithm
f
0
= (1
T
)f
f
k+1
=
_
1
T
_
f +
T
P
W
f
k
k 0
and
f = lim
k
f
k
in L
2
(IR)
where P
W
from L
2
(IR) onto f L
2
(IR) : supp

f W is the orthogonal
projection.
27
The point of the theorem is that the spectrum can be an arbitrary mea-
surable set.
If the spectrum W is an interval or contained in an interval W = [, ],
then the theorem of Donoho and Stark is not so surprising. Information on
f B

is missing only on a subset T of measure [T[ <

2
|W|
=

. If we
cover T by disjoint intervals I
l
, so that [T[

l
[I
l
[ <

, the gaps of missing

information are at most of length [I
l
[ [T[ <

. Therefore we may select a

discrete set of samples f(x
n
) of f, so that x
n+1
x
n
[T[ <

for all n and

so that x
n
, T. Theorems 5 and 6 now provide a complete reconstruction
of f. For this argument the size of T is completely irrelevant; what matters
are the gaps between the given information on f. Thus, at least in the case
W = [, ] one can do innitely better. The missing segment T can
have innite measure or even be the complement of a set of measure zero!
In order to apply one of the Theorems 5 or 6, one has to discard even
more information and preserve only samples of f on a discrete set. In most
applications it is quite unreasonable to throw away any given information on
f. In the following we give the modications of the algorithms in order to
use the full information f (1
T
).
Since Lebesgue measure is regular, we can cover T for any > 0 by an
at most countable union of open intervals, i.e. T

nZZ
(
n
,
n
), so that

n
<
n

n+1
and [

nZZ
(
n
,
n
) T[ < .
In order to obtain an approximation of f in B

, interpolate f linearly
between the gaps and then project onto B

, i.e. dene
If(x) =
_
f(x) for x , T
f(
n
) +
f(
n
)f(
n
)

n
(x
n
) for x (
n
,
n
)
(66)
and
A
1
f = PIf. (67)
Finally we need another version of Wirtingers inequality [39].
Lemma 4 If f(a) = f(b) = 0 and f, f

L
2
(a, b), then
_
b
a
[f(x)[
2
dx
_
b a

_
4
_
b
a
[f

(x)[
2
dx. (68)
The next theorem contains a considerable improvement of Theorem 11
and uses the full information f(1
T
).
Theorem 12 Assume that T
nZZ
(
n
,
n
) and that
:= sup
nZZ
(
n

n
) <

. (69)
28
Then f B

is uniquely determined by f(1

T
) and can be reconstructed
as follows:
f
0
= A
1
f , f
k+1
= f
k
+ A
1
(f f
k
) for k 0
and f = lim
k
f
k
with the error estimate
|f f
k
|
_

_
2(k+1)
|f|.
Proof: We have to show that |f A
1
f| (

)
2
|f| and then apply
Proposition 1. For f B

we obtain by the same argument as in (39)

|f A
1
f|
2
= |P(f If)|
2
|f If|
2
=

nZZ
_

n

n
[f(x) f(
n
)
f(
n
) f(
n
)

n
(x
n
)[
2
dx.
Since the integrand vanishes at both end points
n
and
n
, Lemma 4 implies
|f A
1
f|
2

nZZ
_

_
4
_
n
n
[f

(x)[
2
dx
(

)
4
|f

|
2
(

)
4
|f|
2
.
In the last two inequalities we have used the hypothesis (69) and Bernsteins
inequality
The following corollary which specializes the theorem to sampling sets
IRT = x
n
, n ZZ has already been obtained in [27]. The point is an
improved rate of convergence of the reconstruction algorithm.
Corollary 2 Let x
n
, n ZZ be a sampling set so that := sup(x
n+1

x
n
) <

. Then every f B

can be reconstructed iteratively from its samples

f(x
n
) by f
0
= A
1
f, f
k+1
= f
k
+ A
1
(f f
k
) for k 0 and f = lim
k
f
k
with the error estimate
|f f
k
|
_

_
2(k+1)
|f|.
Here A
1
f = PIf, where If(x) = f(x
n
) +
f(x
n+1
)f(xn)
x
n+1
x
n
(x x
n
) for x
[x
n
, x
n+1
].
29
6 Irregular Sampling of the Wavelet- and the Short
Time Fourier Transform
In this section we discuss how the well-known results of I. Daubechies, A.
Grossmann and Y. Meyer [14] and [12] on the construction of wavelet frames
can be extended to irregularly spaced wavelet frames.
Let g L
2
(IR) be a xed functions, a wavelet, then the continuous
wavelet transform of a function f on IR is dened as
W
g
f(x, s) =
1
s
_
IR
g
_
t x
s
_
f(t)dt for x, s IR, s > 0 (70)
and the short time Fourier transform S
g
f of a function f is
S
g
f(x, y) =
_
IR
e
iyt
g(t x)f(t)dt for x, y IR. (71)
Given g, f is uniquely determined by both W
g
f and S
g
f and can be recov-
ered by explicit inversion formulas, see e.g. [30]. However, a representation
of f by one of these transforms is highly redundant, and one seeks to reduce
the redundancy by sampling.
The case of regular sampling is well-understood through the work of
I. Daubechies [12]. She has found nearly optimal estimates on the den-
sity (, ), so that any f can be completely and stably reconstructed from
W
g
f(
j
k,
j
), j, k ZZ, or from S
g
f(j, k).
For irregular sampling only qualitative results are known. W
g
f and S
g
f
are completely determined by their samples on a suciently dense set in
the plane [25, 26].
In the following we consider band-limited wavelets g B

and sam-
pling sets of the form (x
jk
, s
j
), j, k ZZ with the usual assumptions that
lim
k
x
jk
= and lim
j
ln s
j
= , and that all sequences
x
jk
, k ZZ and s
j
are increasing. This amounts to sampling the wavelet
transform rst irregularly in the scaling parameter s, and then on each scale
s
j
at the points x
jk
.
Theorem 13 [28] Let g B

L
1
(IR) and g() = 0 for [[ < < for
some (small) > 0. Suppose that the sequence s
j
, j ZZ IR
+
satises
for two constants a, b > 0,
a

jZZ

g(s
j
)

2
b for a.a. . (72)
Let x
jk
, j, k ZZ IR be a double sequence satisfying
:= sup
j,kZZ
x
jk+1
x
jk
s
j
<

. (73)
30
Then we have for all f L
2
(IR)
a
_
1

_
2
|f|
2

j,kZZ
1
2
_
x
j,k+1
x
j,k1
_

W
g
f(x
jk
, s
j
)

2
(74)
b
_
1 +

_
2
|f|
2
.
A Reformulation of (74).
The collection of functions
_
1
s
j
g
_
x x
jk
s
j
_
, j, k ZZ
_
(75)
is a frame for L
2
(IR) with frame bounds
A = a
_
1

_
2
and B = b
_
1 +

_
2
. (76)
The reconstruction of f from the irregular samples of W
g
f can be carried
out explicitly as follows. Let =
2
A+B
and =
BA
B+A
=
b(+)
2
a()
2
b(+)
2
+a()
2
and
S be the wavelet frame operator
Sf(x) =

j,kZZ
1
2
(x
j,k+1
x
j,k1
)W
g
f(x
jk
, s
j
)
1
s
j
g(
x x
jk
s
j
). (77)
The sequence f
k
, where
f
0
= Sf, f
k+1
= f
k
+ S(f f
k
) for k 0, (78)
converges to f in L
2
(IR) and
|f f
k
|
k+1
|f|.
By duality, every f L
2
(IR) has a wavelet expansion of the form
f(x) =

j,kZZ
c
jk
1
2
(x
j,k+1
x
j,k1
)
1
s
j
g(
x x
jk
s
j
) (79)
with coecients satisfying
_
b(1 +

)
_
2
|f|
2

j,kZZ
[c
jk
[
2

_
a(1

)
_
2
|f|
2
.
Remarks
1. The conditions on the wavelet g are identical to those in [14], and (72)
is just a paraphrase of the known condition for regular sampling.
31
2. The left side of (72) implies that
s
j+1
s
j

for all j otherwise the

dilates of supp g would not cover IR and (72) is violated. The right side of
(72) requires
s
j+1
s
j
> 0 for all j.
The condition that the s
j
have to be separated becomes unnecessary if
appropriate weights for the scaling parameter s are used [28].
3. Since g is band-limited, the frequency resolution of W
g
f is the best
possible. In order to obtain good spacial resolution as well, the wavelet g
must have rapid decay. This is achieved by a choice g B

, g C

.
4. A similar theorem without explicit frame bounds is reviewed in the
article of Benedetto and Heller [2].
5. A dierent reconstruction method that uses a deconvolution argument
is indicated in [28].
Proof of Theorem 13. In the light of Section 2 and Theorem 6 the proof
is much easier than the task of formulating the theorem.
We observe rst that for xed scale s > 0,
W
g
f(x, s) =
_
f(t)
1
s
g(
x t
s
)dt = f g

s
(x) (80)
where g

(t) = g(t) and g

s
(t) =
1
s
g(
t
s
). Taking Fourier transform of f g

s
,
we nd
supp
_
f g

s
_

= supp

fg

s
= supp

f() g(s)
_

s
,

s
_
.
Thus at each scale s > 0 W
g
f(, s) is a band-limited function with band-
width

s
.
Step 1. Reconstruction of W
g
f(x, s
j
) from W
g
f(x
jk
, s
j
).
Since W
g
(, s
j
) = f g

s
j
B

s
j
and by assumption for each j ZZ
sup
kZZ
(x
j,k+1
x
j,k
) s
j
and (s
j
) (

s
j
) = , Theorem 6, (44) yields for
each j
_
1

_
2
_
_
_
_
f g

s
j
_
_
_
_
2

k
1
2
_
x
j,k+1
x
j,k1
_

W
g
f(x
jk
, s
j
)

_
1 +

_
2
_
_
_
_
f g

s
j
_
_
_
_
2
. (81)
Step 2. Reconstruction of W
g
f from W
g
f(, s
j
).
For this we use Plancherels theorem (15) and write |f g

s
j
|
2
=
1
2
|

fg

s
j
|
2
=
1
2
_
IR
[

f()[
2
[ g(s
j
)[
2
d. Now we take the sum over j and obtain with (72)

jZZ
|f g

s
j
|
2
=
1
2
_
[

f()[
2
_

jZZ
[ g(s
j
)[
2
_
d
b
1
2
|

f|
2
= b|f|
2
and a|f|
2
.
32
Combined with (81) this proves the theorem. Since W
g
f(x, s) is the inner
product of f with the function
1
s
g(
tx
s
), the other statements are obvious.
A parallel statement for the short time Fourier transform reads as follows.
Theorem 14 Suppose g B

L
1
(IR) and that for two constants a, b > 0
the sequence y
j
, j ZZ satises
0 < a

jZZ
[ g( y
j
)[
2
b. (82)
If x
jk
, j, k ZZ is a double sequence satisfying
:= sup
j,k
_
x
j,k+1
x
j,k
_
<

then for any f L

2
(IR)
a
_
1

_
2
|f|
2

j,kZZ
1
2
(x
j,k+1
x
j,k1
)[S
g
f(x
j,k
, y
j
)[
2
b
_
1 +

_
2
|f|
2
. (83)
Thus the collection e
iy
j
t
g(t x
j,k
), j, k ZZ is a frame for L
2
(IR) with
frame bounds A = a(1

)
2
and B = b(1 +

)
2
.
The proof of Theorem 14 is almost identical to the previous proof, and
the necessary modications are left to the reader.
It is worth noting the (83) contains implicitly a sampling theorem for
the short time Fourier transform with a compactly support wavelet g. This
follows from the symmetry relation
2S
g
f(x, y) = e
ixy
S
g

f(y, x).
In particular, the collection
1
2
(x
j,k+1
x
j,k1
)e
ix
j,k
t
g(t y
j
), j, k ZZ is
now a frame for L
2
(IR) based on the compactly supported wavelet g.
33
Part II
The Practice of Irregular Sampling
The aim of this part is to support some of the general statements made
in Part I through numerical evidence. In particular, we aim at a verication
of the fact that the Adaptive Weights Method (ADPW) is in many respects
the method of choice due to several advantages over other methods, such
as good guaranteed rates of convergence, and low computational complexity.
Furthermore we will provide some evidence that there are additional features
which do not follow from the theory but which have to be taken into account
if one is to judge the usefulness of dierent methods. Such observations have
more to do with statistical features and are thus not directly related to the
theoretical results which have to be considered - at the present state - as
worst case analysis.
7 Setup for Numerical Calculations
Restricting ourselves to the discussion of one-dimensional (= 1D) signals
we consider synthetic signals which are obtained through discrete Fourier
transform, with known spectrum. Thus, depending on our point of view we
are dealing with functions on the cyclic group of order n (typically we take
n = 512, or some other power of two, in order to take advantage of the ex-
istence of the FFT = fast Fourier transform in this case), or trigonometric
polynomials. From the rst point of view we assume that the spectrum (the
set of coordinates for which we allow non-zero Fourier coecients) is given,
and for most examples small compared to the full signal length. Using the
second, equivalent description, we are assuming that the frequencies which
constitute the trigonometric polynomial form a nite subset of an appropri-
ate lattice in the frequency domain. Practically, we dene the signal as a
superposition of those admissible frequencies with random amplitudes. We
understand the irregular sampling problem as the question for constructive
and ecient methods to recover signals from suciently many sampling val-
ues. Theory guarantees that several iterative methods will converge with a
geometric rate provided that the maximal gap between successive samples is
not larger than the Nyquist distance, i.e. the the maximal gap size admissible
for regular sampling sets according to Shannons theorem..
Let us give next a commented version of the MATLAB-code containing
the essential parts of an experiment, which establishes a random signal with
a given spectrum from the sampled coordinates and then applies the ADPW
method to solve it. At the beginning the lter sequence filt of a length n
is given (taking the value 1 over the spectrum yp, and 0 elsewhere), and a
subsequence xp of the integer sequence 1, . . . , n, representing the sampling
positions. The variable iter gives the number of iterations.
34
(14,8.33)sit51251
Figure 1:
(14,8.33)snc51251
Figure 2:
% APDW.M H.G. Feichtinger, 9/91, ADAPTIVE WEIGHTS METHOD
n = length(filt); % n = signal length
yp = (filt == 1); % yp : spectrum
sp = length(yp); % sp = spectral dimension
y = randc(1,n) % generating complex random spectrum
y = y.*filt; % setting high frequencies to zero
x=ifft(y);x=x/norm(x); % generating normalized signal
xs = x(xp); % determining sampling values
lxp = length(xp); % lxp: number of sampling points
xp1=[xp(2:lxp),n+xp(1)]; xp2=[(xp(lxp)-n), xp(1:(lxp-1))];
w = 0.5*(xp1-xp2); % calculating "adaptive" weigths
xa = zeros(x); xd = zeros(x); % initialisation...
for j = 1 : iter % ITERATIONS
xd(xp) = (xs - xa(xp)).*w; % establishing auxiliary signal
xa=xa+ifft((filt.*fft(xd))); % filtering auxilary signal
end;
Fig. 1 shows the signal (real and imaginary part) and the sampling values
and coordinates (marked by crosses), which are the data from which the
reconstruction process has to start. The number of spectral values (non-
vanishing DFT-coecients) is 51, the signal length 512. We have a total of
93 sampling points, and there are three gaps of length 10 (which is about the
Nyquist rate for this spectrum) in the sampling set.
Fig. 2 shows the corresponding SINC function, i.e. the signal obtained
by means of IFFT from the lter function. It is the discrete analogue of the
classical (cardinal) SINC-function. It has the property of reproducing any
band-limited signal with appropriate spectrum through convolution.
35
8 Hard and Software Environment: PC386/486 and
SUN, IRSATOL/MATLAB
Let us only give a brief description of the practical setup on which the nu-
merical experiments described below have been carried out by a group of
mathematicians at the Math.Dept. of the University of Vienna under the
guidance of the rst named author.
1
The most systematic contributions
over the time came from Thomas Strohmer, without whose constant engage-
ment and cooperation the package would not exist in the present form. We
also gratefully acknowledge his help in producing the plots, with technical as-
sistance from B. vanThannen, who also took care of setting up an appropriate
environment on the SUN workstations. H.Steier deserves to be mentioned
separately, because he has designed an independent collection of M-les.
We have chosen MATLAB as the basis for our experiments, because it is
a widespread mathematical software package by MathWorks, running on
a large variety of platforms. Most of our experiments were done on PC386
and PC486 (timing given below was taken on a 33MHz PC486), and to
some extent on a SUN workstation. Extending the basic collection of M-les
(i.e. modular program pieces, such as function, auxiliary utilities, and script
les, containing a sequence of commands) by a collection of several hundred
additional M-les we have produced the program package IRSATOL (for
IRregular SAmpling TOoLbox). A more detailed report about some general
features, more from the project management and the programmers viewpoint
is given in the report [23].
Some of the MATLAB M-les which are of general interest also to other
users can be obtained, at least for some limited period, through ANONY-
MOUS FTP from the internet address TYCHE.MAT.UNIVIE.AC.AT
(=131.130.22.36). It is assumed that colleagues making use of those M-les
or nd improvements will notify us (FEI@TYCHE.MAT.UNIVIE.AC.AT).
More details can be obtained from the same address.
9 Results and Observations from the Experiments
9.1 Comparison of Dierent Methods
(Eciency of methods in terms of iterations or time)
We are mainly comparing the following ve methods in this article:
1) The Wiley/Marvasti or (ordinary/unweighted) frame method, for short
WIL/MAR-method. It is based on the fact that the collection of shifted
1
Among the colleagues that have helped with the project were C.Cenker, M.Herrmann,
M.Mayer, and F.Stettinger.
36
(14,8.33)s551tim2
Figure 3:
SINC-functions forms a frame (cf. Theorem 4).
2) The VOR(onoi) method as described in Theorem 5 which requires
step functions, constant in the nearest neighborhood of any of the sampling
points, followed by convolution with a SINC-function as approximation step.
3) The PLN (piecewise-linear) method, which make use of piecewise linear
interpolations, given the sampling values, again followed by low pass ltering
for the approximation operator A. (cf. Cor.2).
4) The (single step) POCS method, as described in section 3.3, where at
each step the given approximate signal xa is updated by adding a shifted
SINC-function, to correct the sampling value of the approximate signal at
the sampling point under consideration.
5) The mail role will be played by the ADPW method discovered by
the authors, cf. Theorems 6 and 8. This method is also labelled as dps,
following the older notation for the relevant approximation operator.
Those methods have already been described at several occasions and
proofs for their guaranteed rate of convergence are known, at least for the
VOR and ADPW methods, if the maximal gap does not exceed the Nyquist
rate. (cf. [9], [10], [27], [22] for details).
Fig. 3 shows the convergence behavior, described through the natural log-
arithm of the
2
norm of the error versus the number of iterations (we have
taken 32 iterations for each method in this example). Note that 35 corre-
sponds to a numerical value of the error of about 10
16
which is of the order
of machine precision for MATLAB. The methods are ordered by eciency.
Thus, in this particular example the piecewise linear method (labelled pln)
is slightly more ecient in terms of iterations than the ADPW-method (la-
belled dps). We have chosen this example despite the fact that we have
found that on the average ADPW did perform better even in terms of this
form of eciency (for timing cf. below).
One iteration is one step of any of the iterative methods discussed. We
call the gain at each step (which actually comes close to its limit value after
view steps ), i.e. |xx xa
n+1
|
2
/|xx xa
n
|
2
, the rate of convergence.
With the exception of the POCS method (labelled as pcs) for all methods
under discussion one step consists of establishing an auxiliary function and
subsequent ltering. Thus each such step makes use of all the sampling values
at once. In contrast, for the POCS method a single step makes use of only
one sampling value. We have therefore decided to count one cycle, made
up by a sequence of single steps and using all the sampling values once, as
one iteration in our comparison. This way of counting clearly disguises the
much higher eort to run through a single cycle, as can be seen from Fig. 4
37
(14,8.33)s551prc1
Figure 4:
which shows the convergence behavior in terms of the natural logarithm of
approximation error versus time
The time-plot Fig. 4 describes the time needed for the same reconstruction
task, in order to achieve machine precision. It shows that the ADPW method
and the frame method are fast compared to VOR and PLN (because the
auxiliary signals are easier to build). Since one step of the frame method takes
about the same time as one step of ADPW the fact that WIL/MAR takes
about twice the time of ADPW is only due to the fact that about twice as
many iterations are needed to obtain the prescribed degree of approximation.
Fig. 4 also shows that in this particular example the POCS method is by far
the slowest, despite an ecient implementation, where a matrix of shifted
SINC functions is stored at rst. Of course, it takes some time to do this, and
in this particular case about the same time as to run the complete ADPW
algorithm (to full precision). Without this trick the rate of convergence of
POCS would be even slower.
In general the Voronoi-method as well as the adaptive weights method
show very good performance in terms of the degree of approximation within
a given number of iterations. On the other hand the computational costs for
each iteration of the frame-method or the ADPW-method are low (essentially
a double FFT) compared to that of the Voronoi- or PLN-method, because
those methods require the construction of an auxiliary function (step-function
or piecewise linear). Actually, the computation of the auxiliary functions gets
more expensive as more sampling points are available. Thus there is a trade-
o between the gain in performance through improved rate of convergence
and increasing computational eort in order to perform a single iteration
for the last two methods. In contrast, the computational costs of a single
step of the WIL/MAR or ADPW method are independent of the number of
sampling points.
We have to point out here that usually there are several dierent ways to
describe the each of the algorithms and to implement it. Typically a linear
algorithm involving some Fourier transform may be described as a linear
mapping on a given vector space (using a basis and matrix representations) or
by direct use of the FFT. A discussion about the choice of the implementation
for a given method and situation will be given elsewhere. For the present
comparison we haven chosen implementations for the Voronoi iteration (or
PLN) which are based on the following observation:
Since step-functions or piecewise linear interpolants over a xed geometry
can always be written as a linear combination of box or triangular functions
respectively one may store those primitive auxiliary function rst and then
38
built only linear combinations with the given coecients at the moment
when the new step function (PLN function) is needed. In most cases it is
even better lter those functions and store the low-pass versions of those
functions for later use during the iterations.
This method is much faster than the naive one, where at each iteration a
new step (or PLN) function has to be built from a sequence of coecients.
SUMMARY
Among the discussed methods the Voronoi and the Adaptive Weights
Method are the most ecient ones. But VOR is rather slow, while PLN is
both slow and often not very ecient, in contrast to theoretical predictions
(more about this point in section 9.8). Similar observations apply to methods
which use higher order spline interpolations for the iterative process. ADPW
and the WIL/MAR method are both fast, but WIL/MAR tends to be very
sensitive towards irregularities of the sampling geometry (cf. below). There-
fore ADPW can be seen as the best approach, both in terms of speed and
robustness.
9.2 Special Features of the POCS Method
There are several positive features of the POCS method. First of all, it is
a relatively simple method which allows to include dierent kinds of con-
vex conditions, including positivity (e.g. of the Fourier coecients), or the
knowledge of several values of derivatives or averages, because all these con-
ditions can be formulated as membership in convex sets with easy to describe
projection operators. In most cases a single POCS condition is just the mem-
bership of the signal in a hyperplane of the L
2
-space under consideration, or
rather B

. Nevertheless the practical value of the POCS method for re-

construction of band-limited signals from irregular sampling appears to be
limited compared to the other methods available.
As a main reason, why POCS is slow for our example, we see the fact
that the number of sampling points is not very small. Increasing the number
of sampling points makes each cycle computationally more expensive, while
the rate of convergence (per cycle) does not necessarily improve strongly
enough to compensate this eect. For a small number of sampling points
the situation might be dierent, but for such a case many dierent ecient
methods are available. Actually, it may be the best solution just to solve
an appropriate and comparatively small linear system of equations. Since
for small problems such direct methods are most ecient anyway the lack of
speed is one of the major drawbacks of the POCS method.
Nevertheless, there are several strategies of improving the straightforward
method for POCS (which consists in going through the sampling set from
left to right) by an appropriate choice of the order or points. Some of those
39
(14,8.33)pcsperm1
Figure 5:
strategies will be discussed in M.Mayers thesis (cf. [47]), cf. also [24] for a
very recent account of this approach. Despite the fact that it appears at rst
sight a natural order, e.g. from left to right, this turns out to be a fairly
inecient way of using the given information. In fact, a second look tells us
that the improvement from a pair of adjacent points (the second being very
close to the rst one) is almost the same as of just one of those points, since
after applying the correction based on the rst sampling value the value at
the second sampling point will be almost correct. Therefore a correction at
the second point will only give minimal further improvement. On the other
hand, if two points are far from each other this phenomenon does not arise.
It turns out that one may either randomly choose the next point at each
single step, or what is easier from a practical point of view to x a
random enumeration of the given sampling set, and to follow that xed se-
quence within each cycle. In both cases the outcome is similar. Fig. 5
exactly demonstrates this. Whereas the rate of convergence for the direct
POCS method is quite poor, 6 out of 7 random orders have achieved ma-
chine precision within 22 iterations. Of course the eciency of this trick
depends on the density of sampling points. Usually a similar eect can be
obtained by applying the so-called red/black method. It consists in using
the odd indices rst, and the even afterwards, i.e. in the following order:
x
1
, x
3
, x
5
, . . . , x
2l+1
, x
2l
, x
2l2
, x
2l4
, . . . , x
4
, x
2
.
Another idea is to choose the order of points which are used within the
series of single POCS steps according to the size of the errors at the cor-
responding sampling points (use those points rst where the maximal error
occurs). Unfortunately, determining those points becomes a computational
intensive extra task if the number of sampling points is large.
Since from those arguments it appears as an advantage to use only the
minimal number of sampling points one might be tempted to try to reduce
the number of points by ignoring some of the sampling values in order to
perform the reconstruction. This will do as long as exactness prevails (e.g.
if we are dealing with synthetic signals) but becomes a serious problem in a
more practical situation, where data are contaminated by noise (which even
might vary from point to point, and might have been the reason for sampling
the signal at a variable rate). Thus, in fact, ignoring available information
does not improve the algorithm.
The following example indicates that the POCS method shows more sen-
sitivity towards noise than most the other methods: Assume that there is a
single disturbed sampling value. It is obvious that this error is fully trans-
mitted to the reconstructed signal if the POCS-method happens to use this
40
(14,8.33)s551errm
Figure 6:
point last. Since one usually does not know, which one of a sequence of given
data points is corrupted most, and also subsequent correct values at other
points do not help to correct the error, this is a serious problem when dealing
with noisy data, using the POCS method.
9.3 Measuring the Error
Plots such as Fig. 3 or Fig. 4 also allow to read o easily how many iterations
or how many units of time are needed in order to reach a certain precision
with a given method. For both plots the choice of the logarithmic scale turns
convergence curves into almost straight lines, for all of the methods under
discussion. Typically, after an initial phase of few iterations with steeper
decay a certain xed quotient is obtained at each of the additional iterations.
Since we have worked with synthetic signals we are able to compute the
error (band-limited signal minus approximation) in an exact way. Fig. 6
describes the error behavior measured with respect to dierent norms. Note
that we have used normalized signals xx of
2
-norm = 1, so that absolute and
relative error coincide in that case. The
1
-error (absolute sum of coordinate
deviations) represents the top line of that graph, and the lowest line describes
the maximal error (sup-norm, or

-norm. However, in practical situations

we cannot measure the true deviation, because only the sampling values are
given. If we just try to avoid the use of the limit element (in case it is
unknown) we might use the size of the relative error as a criterion. In the
example this value appears in line two.
On the other hand we can measure the error between the given sampling
values and sampling values of the reconstructed signal. The second line in
the graph from the bottom represents the
2
-sum (sometimes this is called
the sampling energy), devided by the sampling energy of the given signal.
We also look at the relative change of the sampling energy (third line from
the top).
It is the frame estimate (cf. (29) ) which tells us that it is sucient to
know that the deviation of sampling values (in the
2
-norm) tends to zero
at the same speed as the global deviation. This general equivalence of two
norms implies that the curve for the sampling error has to stay within a
strip of xed height, parallel to the true error curve. However, it does
not directly imply that the curve obtained by plotting the natural log of the

2
-sampling error is itself almost a straight line, as can indeed be seen from
Fig. 6.
This is very useful for practical applications, as it allows to give a fairly
41
(14,8.33)dpswilap
Figure 7:
precise estimate of the number of required steps (hence time needed) in order
to reach a given precision (e.g. the machine precision, which is 10
16
for
MATLAB). This is particularly useful for those implementations, where the
approximative signal is not fully calculated at each step, but only certain
coecients or parameters, sucient to continue the iteration (cf. [10] for
explanations in this direction).
9.4 Sampling Geometry and Spectral Size
It is of great practical importance to know to which extent the sampling
geometry or the spectral size have an inuence on the performance of a given
method (cf. Fig. 7). A detailed discussion of this topic has to be left for a
separate note, because it depends too much on the implementation of the
method under consideration.
It is not surprising that the size of the maximal gap in a sampling set has
considerable inuence on the speed of convergence for any of the methods
under consideration. If the maximal gap is too large (the critical size depends
also on the size of the spectrum in use) then practically all methods under
consideration will diverge, at least for the natural relaxation parameters (cf.
section 7 for details). Very small relaxation parameters will in principle
enforce convergence, but often enough at a rate which is so poor that it is
not useful from a practical point of view.
Of course there are many other geometric features that a sampling set
may have. For example, the degree of irregularity plays an important role.
We cannot give a formal denition here. A sampling family may have slowly
varying sampling density (e.g. many sampling points on the left side, fewer
sampling points at the right side), or may be a set with almost uniform
density, but spoiled by few clusters of points, i.e. regions of higher density.
In any case the quality of an algorithm should include statements about its
sensitivity with respect to those dierent forms of irregularity.
Among all methods under discussion the Wiley/Marvasti method (the
only one which compares with ADPW in terms of computational eort per
step of the iteration) has shown the highest sensitivity in this respect. This is
no surprise if we understand the choice of a relaxation parameter as the choice
of a constant weight. Therefore it cannot compensate the irregularities of the
sampling set as ADPW does. Only for sampling sets with uniform density,
such as sampling sets which are generated by a uniform random number
generator, the performance of the simple frame method is comparable to
that of the ADPW method. This can be explained by the fact that on
42
the average the weights can be assumed to be equal in this case. Since
many authors in the eld restrict their attention too much on such sampling
sets we have to insist here on making the dierence between arbitrary and
random sampling sets. In the rst case the sampling geometry is given and
may have any kind of irregularity. In the second case randomness might have
played a role in the process of generating the sampling set.
We also found that not only the ADPW method, but also the PLN and
VOR methods are not very sensitive towards irregularities. One possible ex-
planation for this is the fact that the local mass bilance of the auxiliary
signal built up before ltering (which is a piecewise linear or step function
respectively, and a discrete measure for ADPW) is not far from that of the
full signal for those methods, whereas it will substantially deviate for Wi-
ley/Marvasti.
Fig. 7 shows the (negative) eect of adding points to a given sampling
set. We start with a quasiregular sampling sequence, i.e. an arithmetic
progression with step-width 17, consisting of 15 points. Since we work on
the cyclic group of order 256 there is one gap of size 18, so the sampling
set is NOT a subgroup of the full group. Then we add successively single
points at any of the 17 possible positions of that largest gap and compare
the (natural) ADPW method with the frame method with optimal relaxation
parameter (although it appears as unrealistic that one can obtain that for
practical applications), to make things fair.
The plot shows the actual rate of convergence (factor by which the error
decreases per iteration), in dependence of the position of the point added.
At the far left end we have the situation where NO point is added. For
both methods the error decreases by a factor of about 0.05 per iteration.
One can explain this by observing that the original sampling set is very
close to a regular one. Adding now extra (single) points results in a weaker
performance. For the WIL/MAR method the eect is very strong, in any
case the rate of convergence is less than 0.25. Even for the ADPW method
(with two exceptions) performance degrades, but only to a rate of about 0.15
in the worst case. However, in any of the cases the actual rate of convergence
of ADPW is much better than that of WIL/MAR. Moreover, the worst case
for the frame method is that of double points (positions 1 and 17), whereas
those situations are the best ones for the ADPW method. As explanation we
oer the fact that those positions are the ones where adding a point causes
the strongest irregularity.
For other, less dramatic situations, we have found, that usually the per-
formance of the simple, unweighted frame method decreases quickly if one
increases the irregularity of a sampling set by adding further points. On the
contrary, the added information (through knowledge of additional sampling
values) will result in improved rates of convergence (at constant costs in terms
of computational expenses) in case of the ADPW method. Furthermore (cf.
43
(14,8.33)ltenl2
Figure 8:
section 9.7 below) the knowledge of the optimal relaxation parameter for the
frame method cannot be used to make a guess for the new optimal relax-
ation parameter, even if we add only few sampling points (so in a practical
situation, where the optimal relaxation parameter is not known the discrep-
ancy between WIL/MAR and ADPW will be even larger than in Fig. 7. In
contrast, the weights for the ADPW method adjust automatically and no
relaxation parameter has to be calculated. Furthermore, if only few points
are added the old weight can be used, and only those weight factors have to
be changed which correspond to points not far from the new points.
9.5 Actual Spectrum and Known Spectrum
In this section we deal with the question of lacking knowledge of the actual
spectrum of the signal that should be reconstructed. Even if the sampled
signal is a low frequency signal, i.e. if it has a small spectrum, there might
be not enough knowledge about its spectrum, so that for the reconstructive
procedure a lter larger than necessary has to be used. It then shows that
the rate of convergence depends more on the size of the used lter than on
the actual maximal frequency of the signal. Putting this into other words we
can say that higher smoothness of a signal does not speed up the reconstruc-
tion process if this information is not incorporated as a priori knowledge. To
illustrate this think of a pure frequency (e.g., with the 8th Fourier coecient
dierent from zero), and assume that we know that we have to look for fre-
quencies up to order 8 only. Then the rate of convergence will be - for all
the methods under comparison - much better than for the reconstruction of
a pure frequency 3, if we assume to know only that it is smaller than 11. In
fact, the auxiliary signals (such as the VOR step function) will always con-
tain signicant contributions within the full spectrum under discussion which
spoil the eciency of the iterations in a signicant way. Of course, taking
the lter too small prevents us from complete reconstruction, and therefore
this case has to be avoided under all circumstances. Fig. 8 shows the actual
(logarithmic) decay for a xed low-pass signal, ADPW method (left) and the
frame method with natural relaxation (right). The best approximation is
obtained using the true spectrum, and the gure shows the eect of adding
more and more points to the spectrum. For the two methods the degrada-
tion eect due to the growing spectrum is comparable, although in each case
ADPW was performing much better than the frame method.
44
(14,8.33)donorlx1
Figure 9:
9.6 Choice of Weight Sequence
The description of the ADPW method includes the choice of the weights,
which in higher dimensions are to be taken as the areas/volumes of the
Voronoi domains (sets of nearest neighbors of a given sampling point). Here
we cannot address the problem of actual calculation of those Voronoi-weights
for more than one dimensions. The 1D case is trivial, we just take half the
distance between the left and the right neighbor as the length of the Voronoi
interval. From our numerical experiments we can say the following concerning
the dependence of the rate of convergence on the choice of the weight:
1) In most cases the speed of convergence does not depend very much on
the specic weights in use, as long as they compensate the irregularities of
the sampling geometry. In particular, it is important to have smaller weight
factors for areas of higher sampling density. On the other hand, for sampling
sets with gaps larger than that for which theory gives a guaranteed rate of
convergence (maximal Nyquist gaps) we have found that one should not take
the formula for weights too literal. In fact, points which are very isolated
would gain too much weight, and the eect would be negative. Instead
of combining such a weight sequence with a global relaxation parameter
< 1 (which helps to guarantee convergence in this case) it is better to
determine some threshold. Numerical experiments indicate that the weight
which corresponds to a Nyquist gap appears as a good recommendation for
the threshold.
2) As rule the total sum of weights should be equal to the signal length.
For the 1D case this comes up in a natural way if one follows the Voronoi
rule (neighborhoods are taken in the cyclic sense). For alternative methods
it makes sense to normalize the weights (which might roughly represent the
variable density of the sampling set) to make their total sum equal to the
signal length. If we apply this rule to a constant weight we end up with the
recommendation to choose the weight factor (actually it is just a relaxation
parameter for the unweighted frame reconstruction) n/lxp (number of sam-
pling points divided by signal length). For sampling sets which roughly show
a uniform distribution of sampling points this is a good choice. In fact,
the Wiley/Marvasti reconstruction method with this relaxation parameter is
more or less the same as the ADPW method!
The choice of weights, i.e. the ADPW method, can also be applied in
a situation as described by Donoho/Stark in [15], where a function with
multiband spectrum (the support of its Fourier transform is contained in a
collection of intervals) is given over a sequence of intervals, with relatively
45
(14,8.33)s551rxw1
Figure 10:
small gaps between those intervals. Fig. 9 compares the rate of convergence
for the frame method with dierent relaxation parameters (this also shows
that the relaxation parameter 1.25 is more ecient than the natural one
(using n/lxp as reference value). Since we had 90 sampling points for n=128
the direct application of the Donoho/Stark algorithm (which simply replaces
the given data at each step over the intervals by the given ones) corresponds
to the frame method with a relaxation parameter of 1.422 in our terminology.
It only ranks third after 32 iterations in our example. Evidently, the ADPW
method performs much better than any of the relaxed frame methods (about
twice the gain of the frame method per iteration) even for such a sampling
geometry.
9.7 Relevance and Choice of Relaxation Parameters
For dierent methods the choice of relaxation parameters plays a completely
dierent role. In our experiments we have found that the eect of choosing
relaxation parameters dierent from 1 does not bring any signicant im-
provement to the Voronoi- or PLN-method on the average, although there
are few exceptional cases. In constrast, for the WIL/MAR method a good
choice of the good relaxation parameter has very often signicant inuence
on the actual speed of convergence. Fig. 10 displays the rate of convergence
for the signal shown in Fig.1 for the frame method, using dierent relaxation
parameters.
From the frame-theory point of view one should just take = 2/(A+B),
given the frame constants A and B. In practice, however, A and B cannot
obtained numerically from the sampling geometry and the lter itself, and
even a good guess appears to be dicult by simple methods. For small
problems it is possible to calculate the two values A and B as the largest and
smallest eigenvalues of the matrix describing the positive and invertible frame
operator. Of course, the computational complexity for this analysis is higher
than that for solving the irregular sampling problem directly. Therefore
this approach helps us with the analysis, but does not provide an ecient
practical method. In general we can only say that B tends to be large if
there are accumulations of sampling points, and that for this reason has
to be a small value, which in turn is responsible for slow convergence in this
case.
On the other hand, in contrast to statements in the literature (cf. [45]) we
can give a simple recipe, derived from the ADPW approach, to give a fair rule
of thumb for sampling sets with mild irregularities, by setting the relaxation
46
(14,8.33)relax7
Figure 11:
parameter to n/lxp. Although this is a reasonably good rst choice as a
relaxation parameter for the frame method there are two arguments why one
should better take a sligthly smaller value (we suggest to take 0.8n/lxp).
Argument a) In general it is true that a small relaxation parameter im-
proves stability of the method, at the cost of speed. For the frame method this
means that the additional factor 0.8 prevents divergence for sampling geome-
tries with more than just slight irregularities, while not sacricing speed of
convergence for sampling situations which are fairly regular. A smaller value
would slow down for most cases, and help to overcome situations which are
somewhat more irregular.
Argument b) is an immediate consequence of the two histograms ( Fig.
11) of optimal relaxation parameters for a large number of random geometries
having a xed maximal gap. We have carried out a number of similar exper-
iments and have found that one can make the following general observations:
whereas the variation of optimal relaxation parameters is relatively large for
the ordinary frame method (Wiley/Marvasti method) optimal relaxation pa-
rameters are very well concentrated near 1 in the case of the ADPW method.
The rst is due to the fact that it is not unlikely that an accumulation of
points occurs, which makes the upper frame constant large, and therefore
requires a small relaxation parameter for optimal relaxation. In the case of
ADPW the small weights for areas of high sampling density compensate this
automatically.
The optimal relaxation parameter for ADPW is usually very close to one,
often within 2 decimals. Therefore optimal adjustment of the relaxation
parameter is already part of the method and there is no further need to
obtain optimal relaxation through trial and error or other more sophisticated
methods.
The histogram for unweighted frames shows another reason to make use
of the additional factor 0.8 as relaxation parameter (thus = 0.8n/lxp
altogether): the expectation of obtaining a suboptimal relaxation parameter
is highest in that case.
WARNING: Improved frame estimates do not guarantee better conver-
gence!
From a practical point of view we have to mention that a given recipe (how to
calculate the optimal relaxation parameter from the frame bounds) only gives
an optimal rate of convergence if the frame bounds are close to the optimal
ones. It is NOT true in general that knowledge of better frame bounds per
se gives a better relaxation parameter. As an example we consider a frame
47
(14,8.33)thvorpln
Figure 12:
which is simply an ONS, hence has frame bounds A = 1 = B. Assume now
we would only know that B 1.5, and A = 1. The general recipe would
suggest to use the relaxation parameter 2/(A + B) = 4/5, which results in
a convergence rate of 0.2. On the other hand, if we would only have the
estimate A 0.5 we would have been lead to the choice 2/(A + B) = 1,
hence perfect reconstruction after one step. It is true, however, that an
improvement of an estimate that is close to optimal will also result in an
improved guess of the optimal relaxation parameter.
9.8 Theory and Practice
One of the surprising observations made over and over again is the stun-
ning eciency of the Voronoi method in terms of iterations, in comparison
to the PLN method. Whereas in theory, not unexpected (if we take sophis-
tication as a measure for the expected performance of a method), the PLN
should show better decay per iteration, we found that in practice the Voronoi
method is astonishing ecient, comparable to ADPW in this respect. Simple
comparisons between theoretical and actual rates of convergence show that
for PLN the theory comes quite close to the true decay. Since the present
estimates just give the rate of decay (i.e. the steepness of the error curve in
the logarithmic scale) one cannot expect to have sharp absolute values. On
the other hand those results can be used to calculate the number of iterations
which will be sucient to obtain a given precision. For the PLN method such
a prediction will not be too far from the number of steps actually required
in a given situation. For the VOR-method the situation is quite dierent,
and numerical error curves are substantially better than the predicted ones.
Whether this is rather a statistical phenomen (for a high percentage of cases
this is true, few exceptional cases show the predicted rate), or rather a topic
where theory has to be adjusted to actual observations is a question that has
to be analyzed in the future.
10 Additional Results and Observations
10.1 Dierent Implementations
For the comparisons presented we have used the standard implementations,
using FFT to carry out convolution for all methods except the POCS method
(which does not require the smoothing step). However, we have to mention
48
that there are several dierent implementations for most of the methods, and
it diers from case to case which one (for the same mathematical algorithm)
is the best. We have developed certain heuristical rules for a good choice
(depending on the signal length, spectral dimension, number of sampling
points, . . . ). A detailed discussion of these questions requires a separate
report. Typically in alternative implementations one replaces a single step
of the iteration by a matrix multiplication. Depending on the size of that
matrix (either number of spectral components or sampling points) and the
speed of the FFT for the given signal length one or the other implementation
is better. Note that the signal may be determined by the problem (in the
worst case it is a prime number) therefore not allowing the use of an ecient
FFT.
10.2 2D Situations
Although we have already some experience with most of the 2D-versions
of the methods described we do not have enough space here to describe the
special tricks required in order to implement those 2Dalgorithms eciently.
For example, it is a non-trivial task to establish the Voronoi step functions,
and even an ecient calculation of the area of the Voronoi domains of each of
the sampling points of an irregular sampling set in the plane is not an obvious
task. We will give a comprehensive discussion of the 2D-case elsewhere. We
note that for 2D-signal reconstruction the advantages of the ADPW-method
over other methods is even bigger compared to the 1D-situation. In fact,
sampling sets in the plane tend to be quite irregular. Furthermore, the
number of variables and spectral points becomes quite large, so that it is a
computational intensive task, if not impossible, to build auxiliary functions
(such as piecewise linear ones over triangulations) or to describe the iterations
using matrix representations of the underlying linear mapping. On the other
hand it was no problem to run the ADPW method in one of our 2D-examples
(which is a 256 256 image) with about 25000 variables and 40000 sampling
values. In fact, it is possible to carry out one iterative step within less than
1 minute on a SUN Sparc-station.
10.3 Performance, Repeated Reconstruction Tasks
For questions of speed one has to take dierent situations into account. For
real-time applications it might be necessary to obtain a fair partial recon-
struction within a given time rather than a better one much later. In other
practical situations a method will be considered good if it is good on the
average, i.e. if the expected waiting time for a series of reconstructions is
minimal among the available methods. The experimental evidence is over-
whelming that among the considered methods the ADPW-method is by far
49
(14,8.33)vorvorav
Figure 13:
the most ecient reconstruction method.
There are also situations where dierent signals with the same spectrum
are reconstructed from samples taken at a xed family of sampling points.
In that case preprocessing the information about the spectrum and the sam-
pling geometry can be used to considerably shorten the reconstruction time
needed for a series of signals. In principle this is possible by means of ma-
trix methods. The basic idea is to make use of the fact that the operators
involved are linear or at least ane mappings (for POCS)). Therefore it su-
cies to run the algorithm rst using appropriate unit vectors. If the problem
under discussion are not too large direct matrix methods might be the best
solution.
A typical application is the reconstruction of a 2D band-limited signal
with rectangular spectrum from a sampling set which arises as the product of
sampling sets in each of the coordinates. Thus the sampling coordinates are
of the form (k
j
, n
l
)
1jj
0
,1ll
0
. Since the restriction of such a band-limited
image to a single row (or column) is again band-limited (the spectrum is
just the projected rectangle on the corresponding axis), we can reconstruct
the rows (k
j
)
1jj
0
, from the given sampling values. Having done this we
have to reconstruct (!many) columns (= vertical sections of the image) from
their values at the coordinates (k
j
)
1jj
0
. Obviously all those sections have
the same spectrum and have to be reconstructed from their sampling values
over a xed geometry. Since the number of column vectors us usually large
compared to the number j
0
of sampling points in each of those columns the
application of the above mentioned algorithms will be much more ecient
than the application of the standard versions of any of those algorithms (not
taking into account the special situation). We have successfully used such a
scheme for the reconstruction of band-limited images of size 256256, where
a number of horizontal and vertical lines has been removed.
10.4 Signal Reconstruction from Local Averages
We have tested the reconstruction of signals from averages as described in
Theorem 7 for the case that averages are taken over a sequence of intervals
which are not longer than the Nyquist rate (see Fig. 13). It has been observed
that the rate of convergence (per iteration) is approximately the same as that
for the Voronoi method (using the same intervals). This is surprising since
one might expect that the knowledge of average values of a function contains
less information than that of exact sampling values. However, one has to
admit that at least for the natural implementation of the reconstruction
50
from averages method the computational eort is increased, because at each
step the local averages for the new approximate signal have to be calculated
(instead of sampling the signal).
10.5 Further Improvements, Recent Numerical Exper-
iments
We only mention that there are of course more advanced methods to improve
the rate of convergence, e.g., by an adaptive choice of the value of a correction
term at each step (variable over dierent iterations, contrary to the xed rate
used by the idea of the relaxation method). A typical representative of this
approach is the steepest descent method. Combining the ADPW-approach
with the steepest descent method gives sometimes a signicant improvement
over unweighted steepest descent and ordinary ADPW iterations. Further-
more the idea of Chebychev acceleration and in particular the application of
Conjugate Gradient methods has turned out to be extremely ecient. Again
it is often possible to reconstruct the signal from sampling values success-
fully, even if the Nyquist criterion is not satised. Also in this sense practical
evidence shows that the performance of the ADPW-method is even better
than by the theory.
11 Conclusion
Among the ve methods discussed in this note only the ADPW method is
both simple (hence low computational eort per step) and ecient, and at
the same time fairly stable with respect to irregularities of the sampling set.
The Voronoi method comes next, it is also stable, but at much higher compu-
tational expenses. Similarly, POCS is ecient for small numbers of sampling
points, but as for VOR and PLN the computational load per step increases
with the number of sampling points. This is not the case for the ordinary
frame method and the ADPW-method. However, unweighted frames show
strong sensitivity with respect to irregularities in the sampling geometry, such
as accumulations of sampling points. Furthermore, the WIL/MAR method
often performs slowly if one does not choose the optimal relaxation parame-
ter, which in turn cannot be obtained from the given data (sampling set and
spectrum) in a simple way. In contrast, the optimal relaxation parameter for
ADPW is in most cases very close to 1.
51
References
[1] J. J. Benedetto. Irregular sampling and frames. In Wavelets: A Tutorial
in Theory and Applications, C. K. Chui, ed., pp. 445508, Academic
Press, 1992.
[2] J. Benedetto, W. Heller. Frames and irregular sampling. Math. Note 10,
Suppl. n.1 (1990), pp. 181194.
[3] A. Beurling. Local harmonic analysis with some applications to dier-
ential operators. In Some Recent Advances in the Basic Sciences, pp.
109125, Belfer Grad. School of Science, Annual Science Conference
Proc., A. Gelbart, ed. Vol. I, 196264.
[4] A. Beurling, P. Malliavin. On the closure of characters and the zeros of
entire functions. Acta Math. 118 (1967), pp. 7995.
[5] F. Beutler. Sampling theorems and bases in Hilbert space. Inform. Contr.
4 (1961), pp. 97117.
[6] F.J.Beutler. Error-free recovery of signals from irregularly spaced sam-
ples. SIAM Review 8 (1966), pp. 328335.
[7] P.L. Butzer, L. Hinsen. Two-dimensional nonuniform sampling expan-
sions I, II. Appl. Anal. 32 (1989), pp. 5368 and 6985.
[8] P.L.Butzer, W. Splettstoer, R.Stens. The sampling theorem and linear
prediction in signal analysis. Jber.d. Dt. Math.-Verein. 90 (1987), pp.
170.
[9] C. Cenker, H. G. Feichtinger, M. Herrmann. Iterative methods in irreg-
ular sampling: a rst comparison of methods. Proceedings of the 10th
IPCCC, March 91, Scottsdale, Arizona, IEEE Computer Soc. Press,
pp. 483489.
[10] C. Cenker, H. G. Feichtinger, H. Steier. Fast iterative and non-iterative
reconstruction of band-limited functions from irregular sampling values.
Proc. ICASSP 1991, Toronto, IEEE Signal Processing Soc., 1991, Vol.
3, pp. 17731776.
[11] R. R. Coifman and R. Rochberg. Representation theorems for holomor-
phic and harmonic functions in L
p
. Asterisque 77 (1980), pp. 1166.
[12] I. Daubechies. The wavelet transform, time-frequency localization and
signal analysis. IEEE Trans. Inf. Theory, Vol. 36 (1990), pp. 9611005.
52
[13] I. Daubechies, A. Grossmann. Frames in the Bargmann space of entire
functions. Comm. Pure Appl. Math. 41 (1985), pp. 151164.
[14] I. Daubechies, A Grossmann, and Y. Meyer. Painless nonorthogonal
expansions. J. Math. Phys. 27 (1986), pp. 12711283.
[15] D. L. Donoho, P. B. Stark. Uncertainty principles and signal recovery.
SIAM J. Appl. Math. 49 (1989), pp. 906931.
[16] R. J. Dun, J. J. Eachus. Some notes on an expansion theorem of Paley
and Wiener. Bull. Amer. Math. Soc. 48 (1942), pp. 850855.
[17] R. Dun, A. Schaeer. A class of nonharmonic Fourier series. Trans.
Amer. Math. Soc. 72 (1952), pp. 341366.
[18] H.G.Feichtinger, K.Grochenig. Multidimensional irregular sampling of
band-limited functions in L
p
-spaces. Proc. Conf. Oberwolfach, Feb. 1989,
ISNM 90, Birkhauser, 1989, pp. 135-142.
[19] H.G.Feichtinger, K.Grochenig. Iterative Reconstruction of Multivariate
Band-Limited Functions from Irregular Sampling Values. SIAM J. Math.
Anal. 23 (1992), pp. 244261.
[20] H.G.Feichtinger, K.Grochenig. Irregular sampling theorems and series
expansions of band-limited functions. J. Math. Anal. Appl. 167 (1992),
pp. 530556.
[21] H.G.Feichtinger, K.Grochenig. Error analysis in regular and irregular
sampling theory. Applicable Anal, (1992), to appear.
[22] H.G.Feichtinger, K.Grochenig, M. Hermann. Iterative methods in irreg-
ular sampling theory. Numerical Results. 7. Aachener Symposium f ur
Signaltheorie. ASST 1990, Aachen, Informatik Fachber. 253, pp. 160
166, Springer, 1990.
[23] H.G. Feichtinger and T. Strohmer. IRSATOL Irregular Sampling of
Band-limited Signals TOOLBOX, CIP-92, Conference on Computers
for Teaching, Berlin, Sept.1992 (available through ANONYMOUS FTP
from 131.130.22.36 = tyche.mat.univie.ac.at).
[24] H.G.Feichtinger, C.Cenker, M.Mayer, H.Steier and T.Strohmer. New
variants of the POCS method using ane subspaces of nite codimen-
sion, with applications to irregular sampling, SPIE92 Conf., Visual Com-
munications and Image Processing, Boston, Nov. 1992. pp. 299 310.
[25] M. Frazier, B. Jawerth. A discrete transform and decomposition of dis-
tribution spaces. J. Functional Anal. 93 (1990), pp. 34170.
53
[26] K. Grochenig. Describing functions: atomic decompositions versus
frames. Monatsh. Math. 112 (1991), pp. 142.
[27] K. Grochenig. Reconstruction algorithms in irregular sampling. Math.
Comp. 59 (1992), pp. 103125.
[28] K. Grochenig. Irregular sampling of wavelet and short time Fourier
transforms. Constr. Appr.Th. 9 (1993), pp. 283 297 (special issue on
wavelet theory).
[29] K. Grochenig. A discrete theory of irregular sampling. Lin.Alg.Appl., to
appear (1993).
[30] A. Grossmann, J. Morlet, and T. Paul. Transforms associated to square
integrable group representations I: general results. J. Math. Phys. 26
(1985), pp. 24732479.
[31] L. A. Hageman, D. M. Young. Applied Iterative Methods. Academic
Press, 1981.
[32] G. Hardy, J. E. Littlewood, G. Polya. Inequalities. 2nd Ed., Cambridge
Univ. Press. 1952.
[33] J.R. Higgins. A sampling theorem for irregularly sampled points. IEEE
Trans. Information Theory Sept. 1976, pp. 621622.
[34] J.R. Higgins. Five short stories about the cardinal series. Bull. Amer.
Math. Soc. 12 (1985), pp. 4589.
[35] N.E.Hurt. Phase Retrieval and Zero Crossings. Mathematical methods
in image reconstruction. Kluwer Acad.Publ.. Dordrecht - Boston - Lon-
don, 1989.
[36] S. Jaard. A density criterion for frames of complex exponentials. Michi-
gan Math. J. 38 (1991), pp. 339348.
[37] A.J. Jerri. The Shannon sampling theorem its various extensions and
applications. A tutorial review. Proc. IEEE 65 (1977), pp. 15651596.
[38] M.I. Kadec. The exact value of the Paley-Wiener constant. Soviet Math.
Dokl. 5 (1964), pp. 559561.
[39] Ky Fan, O. Taussky, J. Todd. Discrete analogs of inequalities of
Wirtinger. Monatsh. Math. 59 (1955), pp. 7390.
[40] H. Landau. Necessary density conditions for sampling and interpolation
of certain entire functions. Acta Math. 117 (1967), pp. 3752.
54
[41] S.Kaczmarcz. Angenaherte Auosung von Systemen von linearen Glei-
chungen, Bull.Acad.Polon.Sci.Lett. 40 (1972), pp. 596599.
[42] N. Levinson. Gap and Density Theorems. Coll. Publ. 26, Amer. Math.
Soc., New York, 1940.
[43] F. A. Marvasti. Spectral Analysis of Random Sampling and Error Free
Recovery by an Iterative Method, IECE, Japan, vol. 69/2, 1986.
[44] F.A. Marvasti. A unied approach to zero-crossing and nonuniform
sampling of single and multidimensional systems. Nonuniform, P.O.Box
1505, Oak Park, IL 60304, 1987.
[45] F.Marvasti and M.Analoui. Recovery of signals from nonuniform sam-
ples using iterative methods. in Proc.Int.Symp. Circuits Syst., Portland,
OR, May 1989.
[46] F. A. Marvasti. An iterative method to compensation for interpolation
distortion, IEEE Trans. ASSP 37/1 (1989), pp. 16171621.
[47] M.Mayer. Variants of the POCS method applied to irregular sampling,
Thesis, Univ. Vienna, 1993.
[48] M. Z. Nashed, G.G. Walter, General sampling theorems for functions in
reproducing kernel Hilbert spaces. Math. Control Signals Syst., 4 (1991),
pp. 373412.
[49] John von Neumann. Functional Operators, Vol. II, Annals of Math.
Studies 22, Theorem 13.7., p. 55), Princeton, N.J., 1950.
[50] R.E.A.C. Paley, N. Wiener. Fourier transforms in the complex domain.
Amer. Math. Soc. Coll. Publ. Vol. 19. 1934.
[51] A. Papoulis. Error analysis in sampling theory. Proc. IEEE 54/7 (1966),
pp. 947955.
[52] K.D.Sauer, J.P.Allebach. Iterative reconstruction of band-limited images
from nonuniformly spaced samples. IEEE Trans. CAS-34/2 (1987), pp.
14971506.
[53] H.Stark. Image Recovery: Theory and Application. Acad.Press, San
Diego - London - Tokyo. 1987.
[54] W.J. Walker. The separation of zeros for entire functions of exponential
type. J. Math. Anal. Appl. 122 (1987), pp. 257259.
[55] R. G. Wiley. Recovery of band-limited signals from unequally spaced
samples. IEEE Trans. Communications COMM-26 (1978), pp. 135137.
55
[56] K. Yao, J.O. Thomas. On some stability and interpolatory properties
of nonunifrom sampling expansions. IEEE Trans. Circuit Theory 14
(1967), pp. 404408.
[57] S. Yeh, H. Stark. Iterative and one-step reconstruction from nonuniform
samples by convex projections. J. Opt. Soc. Am. A, Vol. 7 (1990), pp.
491499.
[58] D. C. Youla, H. Webb. Image restoration by the method of convex pro-
jections: Part I. IEEE Trans. Med. Imag. MI-1 (1982), pp. 8194.
[59] R. Young. An Introduction to Nonharmonic Fourier Series. Academic
Press, New York. 1980.
[60] R. Young. On the stability of exponential bases in L
2
(, ). Proc.
Amer. Math. Soc. 100 (1987), pp. 117122.
56