Smoothing and Convolution

Why smooth? Images have noise. The intensity measured at a pixel is the ”true” intensity plus
noise. If nearby pixels have similar ”true”intensities, then we can use smoothing to reduce the
noise. We can also think of smoothing as a simple example of how information can be passed
between neighboring pixels.
Example: Suppose the true intensities in a region = 100 100 100 100 100 100. P Measured
ni
intensities = 100 + ni , where
P nin i is some noise. If we average M pixels we get: 100 + M . If ni
are zero mean i.i.d. then M is a random variable with the same mean as ni (i.e., 0) and much
lower variance, which drops how?

1 Convolution
If we average pixels in a neighborhood, for the continuous, 1-D case we get:
Z t+δ
1
h(t) = f (t0 )dt0 .
2δ t−δ
1
Equivalently, we can define a function g such that g(t) = 2δ for −δ <= t <= δ, and 0 otherwise.
Then:
Z ∞
h(t) = f (t − u)g(u)du
−∞

This is convolution, and can be done with an arbitrary function g. We write h = g ∗ f .

Try to visualize this as taking g and sliding it across the function f . For each position of g,
the value of h(t) is computed at the center of g’s position, by multiplying g and f together, and
integrating.
As another example, we might want to take a weighted average of t’s neighborhood. For
example, if we weight according to a Gaussian, we just have:

1 −t2
g(t) = √ e 2σ2 .
σ 2π
Then we convolve with g.
Z ∞
h(t) = f (t − u)g(u)du
−∞

Convolution is linear and shift invariant. Linear, because each element of h is a linear com-
bination of elements of f (we multiply and integrate). So we have: g ∗ kf = k(g ∗ f ) and
g ∗ (f 1 + f 2) = g ∗ f 1 + g ∗ f 2. Because of this linearity, convolution has some important properties,
such as commutivity and associativity.
Shift invariant means g is applied to every point in f in exactly the same way. Or to put it
another way, suppose we have functions f and f 0 such that f is a shifted version of f 0 . That is,
f 0 (t) = f (t + k) for some constant k. Then we can show that if h = g ∗ f and h0 = g ∗ f 0 then
h0 (t) = h(t + k).

1
2 Convolution Theorem
Now we get to the reason why the fourier transform is important in understanding convolution. This
is because of the convolution theorem. Let F, G, H be the fourier transform of f, g, and h. The
convolution theorem states that convolution in the spatial domain is equivalent to multiplication in
the transform domain, ie, f ∗ g = h ⇐⇒ F G = R H. R
Proof: Suppose h(t) = g ∗ f (t). H(w) = e−itw ( f (t − u)g(u)du)dt We change variables, so
that v = t − u.

Z Z
H(w) = e−i(u+v)w f (v)g(u)dudv
Z Z
−ivw
= ( e f (v)dv)( e−iuw g(u)du) = G(w)F (w)

This is a remarkable theorem. It means that if we convolve a sine wave with any kernel, we
produce a sine wave of the same frequency. One way to say this is that sines and cosines of
varying frequencies are the eigenvectors of convolution. It means that convolution attenuates
every frequency by different amounts, and we can understand the effects of convolution by taking
the fourier transform of the convolution kernel.
Simplest example: The Fourier transform of a delta function is a constant function. We can
prove this by just noting that for t = 0 cos(kt) = 1 for any k, and sin(kt) = 0 for any k. This means
that convolving with a delta function affects all frequencies the same (since it doesn’t change
them). (Why doesn’t this violate Parsevaal’s theorem?)
As another simple example, we note that if we convolve a function with a single, pure harmonic
(sine or cosine of a specific frequency) we get out the same harmonic, with a possibly different
amplitude.
As another example, we point out that the Fourier transform of a Gaussian is a Gaussian.
If g(t) is gaussian, the broader g(t) is the narrower G(w) is (and vice versa). This means that
smoothing kills high frequency components of a signal. The more we smooth, the more high
frequency components we kill. In fact, attenuating high frequency components of a signal can
be taken to be the definition of smoothing. To some extent, this is why we learned the Fourier
transform. So we can understand what smoothing really does.
We can also see why simple averaging isn’t as good a way of smoothing. Let f (t) be constant
in the range −T to T and 0 elsewhere. This is sometimes called a box function. Then it’s fourier
transform is:
Z T
F (w) = eiwt dt = 2 sin(T w)/w
−T

the sinc function. This does not decay as fast as a Gaussian. Also it has oscillations that
produce odd effects.
The perfect band-pass filter looks like this function in frequency space. Because the inverse
fourier transform is taken in the same way as the fourier transform, in the inverse transform of a
perfect band-pass is a sinc. It does not decay quickly in space.
Another interesting example is found by taking the derivative of a function. Suppose we have:

2
 X X
f (t) = a0 + ak cos(kt) + bk sin(kt).
Then:
X X
f 0 (t) = −kak sin(kt) + kbk cos(kt)
and

F 0 (w) = iwF (w).

This means that taking a derivative magnifies the high frequency components in a signal rel-
ative to the low frequency components. (It also eliminates the constant component (called the
DC component) of the signal). One important implication of this arises when we have a smooth
signal that has had noise added to it. If a signal is smooth, this means, almost by definition, that
it is dominated by low frequencies. Noise, on the other hand, often contains considerable high
frequency components. White noise is called that because it has equal energy at all frequencies
(just like white light). So, in a smooth signal with noise added, the high frequency components
can be due mostly to noise. Taking a derivative amplifies the high frequency components, and
so amplifies the noise. This is why it is a good idea to smooth a signal, removing high frequency
components, before taking a derivative.
The Uncertainty Principle: The Heisenberg uncertainty principle essentially says that no func-
tion can be arbitrarily well-localized in both space and time. This is a proof that the situation with
Gaussians is in some sense canonical.

3 Aliasing and the Sampling Theorem

We’ll just consider these issues for the Fourier series, but qualitatively similar results hold for the
Fourier transform.
Sampling means that we know the value of f at discrete intervals, f ( nπ/T
) for n = 0, +−1, +−T .
That is, we have 2T + 1 uniform samples of f (t), assuming f is a periodic function. Suppose now
that f is band-limited to T . That means,
T
X T
X
f (t) = a0 + ak cos(kt) + bk sin(kt).
k=1 k=1

We know the value of f (t) at 2T + 1 positions, giving us 2T + 1 equations. The ai and bi give
us 2T + 1 unknowns. These are linear equations. So we get a unique solution. This means that
we can reconstruct f from its samples. However, if we have fewer samples, reconstruction is not
possible. If there is a higher frequency component, then this can play havoc with our reconstruction
of all low frequency components, and be interpreted as odd low frequency components.
This means that if we want to sample a signal, the best thing we can do is low-pass filter it
and then sample it. Then, we get a signal that matches our original signal perfectly in the low-
frequency components, and this is the best we can do. To shrink images, in fact, we low-pass filter
them and then sample them.