. .

Subject 10: Fourier Filtering

Discrete Fourier Transform (DFT)

The 2D DFT maps an M N scalar picture f into a complex-valued Fourier transform F. (We use notation as common in the area of Fourier transforms.) This is also called a mapping from the spatial domain (of pictures) into the frequency domain (of Fourier transforms). The benet is that changes in the frequency domain can be directed on particular components in the picture, dened by combinations of sines and cosines. After those changes, called lter operations in the frequency domain, an inverse 2D DFT maps the modied Fourier transform back into a modied picture. The whole process is called Fourier ltering, and allows, for example, contrast enhancement, noise removal, or smoothing of the picture. 1D Fourier ltering is commonly used in signal theory (e.g., for voice processing or recognition), and 2D Fourier ltering of pictures follows the same principles (just in 2D instead of 1D). Formally, the 2D DFT is dened as follows: 1 F(u, v) = MN
M 1 N 1

f (x, y) exp i2
x=0 y=0

xu yv + M N

for frequencies u = 0, 1, . . . , M 1 and v = 0, 1, . . . , N 1. The letter i = 1 denotes the imaginary unit of complex numbers. The Eulerian number e = 2.71828 . . . is also denoted by exp, and satises the Eulerian formula exp(i) = ei = cos() + i sin() for any real .
Page 1 August 2006

. .

Subject 10: Fourier Filtering

The Complex Plane

Let z1 = a1 + i b1 and z2 = a2 + i b2 , with i = 1, real parts a1 and a2 , and imaginary parts b1 and b2 . We have z1 + z2 = (a1 + a2 ) + i (b1 + b2 ) and z1 z2 = (a1 a2 b1 b2 ) + i (a1 b2 + a2 b1 ) The sum or the product of two complex numbers is again a complex number, and both are invertible. R. Descartes (15961650) still called negative solutions of quadric equations a x2 + b x + c = 0 false and other solutions (that is, complex numbers) imaginary. L. Euler (1707-1783) realized that ei = cos + i sin
1 for e = limn (1 + n )n = 2.71828 . . .. This contributed to the acceptance of complex numbers at the end of the 18th century.

Complex numbers combine real parts and imaginary parts, and the latter ones cannot be identied in reality; they are a creation of mathematics. These new entities simplied mathematics. For example, they allowed to proof the Fundamental Theorem of Algebra, and uniform solution formulas for quadric or cubic (i.e., x3 + b x2 + c x + d = 0) equations. It is common practice to visualize complex numbers a + i b as points (a, b) in the plane, called the complex plane.
Page 2 August 2006

. .

Subject 10: Fourier Filtering

Complex Number Calculations

The norm of a complex number z = a + i b coincides with the length of the vector p = (a, b)T (starting at o = (0, 0)T ); we have ||z|| = a2 + b2

The conjugate z of a complex number z = a + i b is the complex number a i b. We have (z ) = z and (z1 z2 ) = z1 z2 . Let z = 0. Then we have z 1 = ||z||2 z . There are many interesting links between the calculus of complex numbers and geometry in the Euclidean plane. For example, a complex number z can also be written in the form z = r ei , with r = ||z|| and (the complex argument of z) is uniquely dened modulo 2. This maps complex numbers into polar coordinates (r, ). The following theorem is of basic geometric importance: THEOREM: A rotation of a vector (m, n)T [i.e., starting at the origin (0, 0)] about an angle is the vector (a, b)T , with a + i b = ei (m + i n) In particular, the complex number z = exp[i2/M ] denes an M th root of unity; we have z M = z 2M = z 3M = . . . = 1. Assume that M is a multiple of 4. Then we have z 0 = 1 + i 0, z M/4 = 0 + i 1, z M/2 = 1 + i 0, and z 3M/4 = 0 + i (1). These properties allowed to design the algorithm of the Fast Fourier Transform (FFT).
Page 3 August 2006

. .

Subject 10: Fourier Filtering

Assume we apply the 2D Fourier transform to a function f , dened in the spatial domain. For the resulting 2D Fourier transform, we obtain at the origin in the Fourier domain 1 F(0, 0) = MN
M 1 N 1

f (x, y)
x=0 y=0

what is the mean of f . Because f has only real values, it follows that the imaginary part of F(0, 0) equals zero. We can also calculate F(1, 0), F(0, 1), F(1, 1), and so forth. It follows that the values of the Fourier transform are in general complex valued (with non-zero imaginary parts). Values for small frequencies u or v (close to zero) represent long wavelengths of sine or cosine components; values for large frequencies u or v (away from zero) represent short wavelengths.

Left: sin . Right: sin 2, with half of the wave length of sin . Low frequencies represent long wavelengths, and thus smooth components of the input picture f . High frequencies represent short wavelength, and thus contrast or point noise in f . Visible directions in f , for example lines into direction (which is equivalent to + ), create value patterns in the F in orthogonal direction (i.e., in direction + /2 in the assumed line example).
Page 4 August 2006

. .

Subject 10: Fourier Filtering

Representation of Matrix F
In pictures we will not have the origin at the center. Values in matrix F are periodic, with periods M and N . Having an innite number of copies of this matrix, we could tessellate the plane (in the form of a regular rectangular grid):

The shaded area is the M N area of matrix F, and it is surrounded by eight more copies of F. The origins are always at the upper left corner. Due to the periodicity, the gure shows that low frequencies are represented in the four corners of matrix F, and the highest frequencies at the center of the matrix. It is possible to shift F into a centered position within the shaded area by multiplying all values f (x, y) rst by (1)x+y , before performing the 2D DFT.
Page 5 August 2006

. .

Subject 10: Fourier Filtering

Spectrum
The norm (or magnitude) ||z|| = r = a2 + b2 and the complex argument (or phase) = arc tan(b/a) dene complex numbers z = a + i b in polar coordinates (r, ). The norm received much attention, because it provides a convenient way of representing the complex-valued matrix F in form of the spectrum ||F||, with ||F||(u, v) = ||F(u, v)|| for all M N frequencies (u, v). For better visibility, the spectrum is normally log-transformed into log10 (1 + ||F(u, v)||):

Page 6

August 2006

. .

Subject 10: Fourier Filtering

Left, top to bottom: Original (low-quality jpg-image), after histogram equalization (showing jpg-artifacts), and inverted color values. Right: Spectra.
Page 7 August 2006

. .

Subject 10: Fourier Filtering

Simple geometric shapes illustrating that main orientations in a picture are (also) rotated by 90 (e.g., vertical bar also generates a horizontal stripe in its spectrum).
Page 8 August 2006

. .

Subject 10: Fourier Filtering

The ideal wave pattern on top generates non-negative values only at v = 0. The diagonal wave pattern is inuenced by nite-array effects. For the overlaid diagonal wave pattern, compare with DFT spectrum shown two pages earlier.
Page 9 August 2006

. .

Subject 10: Fourier Filtering

The Fourier transform F consists of two M N arrays representing the real (i.e., the as) and the imaginary part (i.e., the bs) of the obtained complex numbers a + i b. Thus, the M N real data of the input picture f are now doubled. However, we have the symmetry property that F(u, v) = F(u, v) (recall: the number on the right is the conjugate complex number). Thus, actually half of the data in both arrays of F can be obtained directly from the others. The FFT algorithm typically performs in place: the original picture is used for initializing the M N matrix of the real part, and the matrix of the imaginary part is initialized by zero at all positions. Then the 2D FFT replaces all values in both matrices by 2D DFT results. There is an inverse 2D DFT which allows to transform F back into f :
M 1 N 1

f (x, y) =
u=0 v=0

F(u, v) exp i2

xu yv + M N

Note that the powers of the root of unity are here used in opposite direction (i.e., the minus sign has been replaced by +), and the scaling factor 1/M N of the 2D DFT changed into the scaling factor 1. Important is that the product of both scaling factors equals 1/M N , and we could have split 1/M N also into two scaling factors, both equal to 1/ M N . The inverse 2D DFT will lead to a real-valued function f as long as F satises the symmetry property. Thus, any change in the frequency domain is constrained by this.
Page 10 August 2006

. .

Subject 10: Fourier Filtering

The inverse 2D DFT can also be read as follows: the complex numbers F(u, v) are the Fourier coefcients of f , dened for different frequencies u and v. Each Fourier coefcient is multiplied with a certain combination of a sine and a cosine function, and the sum of all those combinations forms the picture f . That means, if we modify one of the Fourier coefcients (and its symmetric coefcient!) before applying the inverse 2D DFT, then we obtain a modied function f . (i) Option 1: We modify picture data by linear convolution
M 1 N 1

h (x, y) = (f g) (x, y) =
i=0 j=0

f (i, j) g (x i, y j)

in the spatial domain, where g is the lter kernel (also called convolution function). Function h is the ltered picture. (i) Option 2: We modify the 2D DFT F of f by multiplying values in F (symmetry constraint!), position by position, with corresponding complex numbers in G [i.e., F(u, v) G(u, v)]. Let us denote this by F G (not to be confused with matrix multiplication). The resulting complex array is transformed by the inverse 2D DFT into a modied picture. Interestingly, both options lead to identical results assuming that G is the 2D DFT of g, due to the convolution theorem: f g equals the inverse 2D DFT of F G Thus, either a convolution in the spatial domain, or a position-by-position multiplication in the frequency domain, allow to obtain identical ltered pictures.
Page 11 August 2006

. .

Subject 10: Fourier Filtering

Fourier Pairs
Function f its Fourier transform F f (x, y) F(u, v) f g(x, y) F G(u, v) M N f (x, y) (1)x+y F(u + ,v + ) 2 2 a f (x, y) + b g(x, y) a F(u, v) + b G(u, v)

Steps of Fourier Filtering

1. Preprocess the given picture (e.g., histogram equalization, or map picture into a larger 2n 2n array for applying a Fast Fourier Transform algorithm if this uses this restriction) 2. Transform picture f into frequency domain, into complex-valued F by using a Fast Fourier Transform 3. Multiply complex-valued F, element by element, with symmetric complex-valued lter function G (which is the Fourier transform of lter kernel g) 4. Transform F G back into spatial domain, into real-valued f g by using an inverse Fast Fourier Transform 5. Postprocess if necessary (e.g., map into picture of desired size if size had been modied in the ltering process)
Page 12 August 2006

. .

Subject 10: Fourier Filtering

Design of Filter Functions

Example: The local mean operator is a linear convolution in the spatial domain. Its lter function is dened by g(i, j) = 1/a for (i, j) in a neighborhood of (0, 0), where a is the size of the neighborhood, and g(i, j) = 0, otherwise. If we calculate the 2D DFT of g, then we obtain a Fourier transform G which has absolute values close to one for low frequencies, with a steep decrease in amplitudes towards zero for higher frequencies. This is a typical low pass lter: low frequencies are allowed to pass (because multiplied with values of amplitude close to one), but higher frequencies are drastically reduced (because multiplied with values of amplitude close to zero). The impact of a low-pass lter is a reduction of noise and/or of contrast, also known as smoothing. The design of lter functions is simplied in the frequency domain. We may decide for a high-pass lter (e.g., for edge detection, or for visualizing details and for suppressing low frequencies), high-emphasis lter (e.g., for enhancing contrast), band-pass lter (for allowing only a certain range of frequencies to pass), or a lter which eliminates or enhances selected frequencies (under proper consideration of the symmetry constraint!). Besides the exibility in designing lter functions, the availability of time-efcient 2D FFT algorithms is also an important argument for using a DFT-based lter instead of a global convolution. (Local convolutions are normally efciently performed in the spatial domain by a local operator.)
Page 13 August 2006

. .

Subject 10: Fourier Filtering

Top: a linear high-pass lter. Bottom: a linear low-pass lter (both shown by amplitudes in the frequency domain). In the example above, linear stands for the decrease of values between the upper and the lower plateau. Further options are, for example, ideal lters (with step edges), or exponential lters with exponentially decreasing (or increasing) values.

A wave pattern [here shown as being generated by F(2, 3)] is possibly apparent in transmitted picture data. It can be removed by decreasing values at a very narrow spot [in the given example, at (u, v) = (2, 3) and at (u, v) = (2, 3)].
Page 14 August 2006

. .

Subject 10: Fourier Filtering

Upper row, left: the famous Lena image, used since 1976 in image analysis for comparing results. Upper row, right: its spectrum, here with lower frequencies at the four corners, and without log-transform, but with a general increase in brightness (otherwise it would appear as black). Lower row, left: a low pass, showing a typical smoothing effect. Lower row, right: a high pass, totally removing lower frequencies.

Page 15

August 2006

. .

Subject 10: Fourier Filtering

Coursework
10.1. [possible lab project] For this assignment we assume that you already have programs for the 2D DFT and inverse 2D DFT (from an available image processing software system, or a free download from the internet). In this project you are expected to study the problem of evaluating information contained in amplitude and phase of the Fourier transforms: (i) select scalar images showing some type of homogeneous textures; transform these into the frequency domain and modify either (a) the amplitude or (b) the phase of the Fourier transform in a uniform way (for all frequencies), before transforming back into the spatial domain; (ii) do the same for a set of images showing faces of human beings. Discuss your ndings. How do uniform changes (of different degrees), either in amplitude or in phase, alter the information in the given image? 10.2. Who was Euler, and who was Fourier? 10.3. Assume an N N picture. Proof that a multiplication with (1)x+y in the spatial domain causes a shift by N/2 (both in row and column direction) in the frequency domain. 10.4. Sketch (as on Page 10) lter curves in the frequency domain which might be called exponential high-emphasis lter and ideal band-pass lter.
Page 16 August 2006