Chapter 2 Supplymentary Materials

Fourier transform
Fourier Transform for solving heat equation
Consider
u
2u
(x, t) = 2 (x, t),
t
x
I.C.: u(x, 0) = (x)

> 0,

PDE:

< x < ,

x>0

Taking Fourier transform of both PDE and I.C. (w.r.t. x)

u
(k, t) + k 2 u
(k, t) = 0 and
t

(1)

u
(k, 0) = (k)

(2)

Here, u
and are Fourier transform of u and . Also, we used the fact:
ut (k, t)

u
d
xx (k, t)

u
(k, t) and
t
(ik)2 u
(k, t)
2

Multiply both side of (1) by the integrating factor ek x , we get:


 k2 t
e
u
(k, t) = 0
t

(3)

Integrate (3), we get:

2

ek t u
(k, t)
u
(k, t)

= f (k)
= f (k)ek

Now,
k2 t

u
(k, 0) = (k)
u
(k, t) = (k)e

In order to find u(x, t), we need to do inverse Fourier transform of u

(k, t).
k2 t
Now, inverse Fourier transform of e
:

2
2
1
1
1
ek t eikx dk =
e 4t x
S(x, t) =
2
4t
Now, recall that:
If u = S , then u
= S
Now,
2
t) = S(k,
t)
t)(k,
u
(k, t) = ek t (k,

Hence,
u(x, t)

= S(x, t) (x, t)

=
S(x y, t)(y, t)dy

(xy)2
1
=
e 4t (y)dy
4t
1

Fourier transform to solve the wave equation

Consider:
(
2
2
PDE : t2u (x, t) = c2 xu2 (x, t), < x < , t > 0, c 6= 0
I.C. : u(x, 0) = H(x); ut (x, 0) = 0
Taking the Fourier transform of the PDE (w.r.t. x)
2
u
(k, t) = c2 k 2 u
(k, t)
t2
From Homework 1, we know that:
ikct

u
(k, t) = F (k)eikct + G(k)e

Recakk that for u(t) + c2 k 2 u(t) = 0, we assume that u(t) = ert . Then


u(t) + c2 k 2 u(t) = r2 + c2 k 2 ert = 0
if and only if r = ick.

u(t) = d1 eickt + d2 eickt

Now,
u(x, t)

=
=
=
=

1
u
(k, t)eikx dk
2
h
i
1
ikct

F (k)eikct + G(k)e
eikx dk
2


1
1
ik(xct)
ik(x+ct)

F (k)e
dk +
G(k)e
dk
2
2
F (x ct) + G(x + ct)

This is called the DAlembert formula for the solution of the wave equation.
Now,
0 = ut (x, 0)

= cF 0 (x) + cG0 (x)

F 0 (x)

= G0 (x)

F (x)

= G(x) + D

(4)

And
u(x, 0) = H(x)
F (x) + G(x) = H(x)
Combining (4) and (5), we get:
2G(x) + D

= H(x)
H(x) + D
H(x) D
; F (x) =
G(x) =
2
2

Hence:
u(x, t)

=
=

H(x ct) + D H(x + ct) D

+
2
2
1
[H(x ct) + H(x + ct)]
2
2

(5)

Discrete Fourier transform

Goal: Approximate Fourier coefficients in complex form.
Algorithm for Fourier Series computation:
Input : f (x) for x [0, 2] (infinite data)
Output : Fourier coefficients : c0 , c1 , c2 , ..., ck , ... (infinite data)
Discrete case : approximation :
Input : Given n input data : f0 = f (x0 ), f1 = f (x1 ), ..., fn1 = f (xn1 )
Output : c0 , c1 , c2 , ..., cn1 (discrete Fourier coefficients)
In this case, we want to find a transformation T such that

f0
f1

T . =
..
fn1

c0
c1
..
.

cn1

Simple example ( 4 input data points )

Input : f0 = f (0) = 2, f1 = f (x1 = 2 ) = 4, f2 = f (x2 = ) = 6, f3 = f (x3 =
Goal : Approximate f by F4 (x) = c0 + c1 eix + c2 e2ix + c3 e3ix such that

3
2 )

= 8.

F4 (xi ) = fi = f (xi )
Then we have

F4 (x0 ) = f0 = 2 c0 + c1 + c2 + c3 = 2

3i
F (x ) = f = 4 c + c e i
i
4 1
1
0
1 2 + c2 e + c3 e 2 = 4
F4 (x2 ) = f2 = 6 c0 + c1 ei + c2 e2i + c3 e3i = 6

3i
9i
F4 (x3 ) = f3 = 8 c0 + c1 e 2 + c2 e3i + c3 e 2 = 8
(6) can be written as :

c0
f0
c1 f1

A
c2 = f2
c3
f3

2
4
=

6
8

where

1
1
A=
1
1
Recall

e2i
ei
3i
e 2

1
ei
e2i
e3i

1
3i

e 2
e3i
9i
e 2

f0
c0
f1 c1

T
f2 = c2
f3
c3

1
1
=
1
1

1
i
i2
i3

T = A1

1
i2
i4
i6

1
i3

i6
i9

(6)

Consider

1
1
1
i

A=
1 (i)2
1 (i)3

1
(i)3

(i)6
(i)9

1
(i)2
(i)4
(i)6

We can check that

= 4I
AA = AA
Then we have
T = A1 =
Thus

c0
c1 A

c2 = 4
c3

f0

f1
= 1

f2
4
f3

1
1
1
1

1
i
1
i

A
4

1
1
1
1

2
1
4
i

1 6
8
i

5
1 + i
=

1
1 i

Remark: c0 , c1 , c2 , c3 is an approximation of Fourier coefficients f(k).

Discrete Fourier transform for n inputs
Let f (x) defined on [0, 2]. Approximate f (x) by
Fn (x) =

n1
X

ck eikx ,

x [0, 2]

k=0
2
n

such that Fn (xj ) = f (xj ) = fj and xj = j

j = 0, 1, 2, ..., n 1

Then we have:

Let = e

2i
n

Fn (x0 ) = c0 + c1 + + cn1 = f0
Fn (x1 ) = c0 + c1 eix1 + + cn1 e(n1)ix1 = f1
Fn (x2 ) = c0 + c1 eix2 + + cn1 e(n1)ix2 = f2
..
.
Fn (xn1 ) = c0 + c1 eixn1 + + cn1 e(n1)ixn1 = fn1

(7)

= eix1 , then
2 = e

4i
n

= eix2 ;

3 = eix3 ; etc.

Thus, (7) is reduced to:

1
1
1
..
.

2
..
.

1
2
4
..
.

..
.

n1
2(n1)
..
.

n1

2(n1)

(n1)

Now,
1 + + 2 + + n1 =

1 n
=0
1

c0
c1
c2
..
.
cn1


f0
f1
f2
..
.
fn1


2
n = en n i = 1

and
1 1 + j
k + 2j
2k + + (n1)j
(n1)k
2i(jk)

4i(jk)

=
1+e n
+e n
+ e

n
if j = k
=
1e2i(jk)
=
0
for j 6= k

2i(jk)
n
1e


= AA jk

2i(n1)(jk)
n

Therefore we have
= nI
AA = AA
and so

c0
c1
c2
..
.

cn1

=T

f0
f1
f2
..
.

f0
f1
f2
..
.

1
=A

fn1

fn1

f0
f1
f2
..
.

fn1

then
ck

=
=


1
f
f0 +
k f1 +
2k f2 + + (n1)k
n1
n

2i(n1)k
2i
4i
1
n
f0 + e n k f1 + e n k f2 + + e
fn1
n

That is
n1
2jk
1X
ck =
fj ei( n ) ,
n j=0

k = 0, 1, 2, ..., n 1

Fast Fourier transform

Consider the discrete Fourier transform matrix:

1
1
1
2
1

4
Fn = 1
..
..
..
.
.
.
1 n1 2(n1)

..
.

n1
2(n1)
..
.

(n1)

, where n = ei n

The computation of the

inverse
Fourier transform

(hence discrete Fourier transform)

x1
x1
x2
x2

= computation of Fn . for a given . .

.
.
.
.
x4
x4
Computational cost of Fn ~x = n2 multiplication.
Goal: Improve computational cost from n2 to

n
2

log2 n. (FFT)

Remark: Say n = 210. Then,

faster !!

y0
y1

Let ~y = Fn x. Say ~y = .
..

n log2 n = 210 10 < 214 . But n2 = 220 = 26 214 . So FFT is 26 times

x0
x1
..
.

~x =

. Let n = 2m. Then we have

xn1

yn1

n1
X

yj =

nkj xk =

2m1
X

k=0

nkj xk

k=0

Divide k = 0, 1, 2, , 2m 1 into 2 parts:

Part 1: 0, 2, 4, , 2(m 1);
Part 2: 1, 3, 5, , 2m 1;
Then
yj

m1
X

n2kj x2k +

m1
X

k=0

n(2k+1)j x2k+1

k=0

Part 1

Part 2

where

m = ei m = ei n = n2
Thus,
yj

(Fm ~x0 )j = nj (Fm ~x)j ,

(~y 0 )j + nj (~y )j ,

j = 0, 1, 2, m 1

j = 0, 1, 2, m 1

How about yj for j = m, m + 1, 2m 1??

Replace j by j + m, j = 0, 1, 2, m 1, we have:
yj+m

m1
X

n2k(j+m) x2k +

kj+km 0
m
xk + nj+m

k=0

But

n(2k+1)(j+m) x2k+1

k=0

k=0
m1
X

m1
X

m1
X

k(j+m)
m
xk

k=0

 2 km
km
m
= ei m
= ei2k = 1
 2 m
nm = ei n
= ei = 1

Therefore
yj+m

m1
X

kj 0
m
~xk

k=0

nj

m1
X

kj
m
~xk

k=0

(Fm ~x0 )j nj (Fm ~x)j

= ~yj0 nj ~yj
6

To summarize, ~y = (y1 , y2 , y2m1 ) can be computed :

Step 1: Split ~x into
~x0

x0 , x2 , , x2(m1)

~x

(x1 , x3 , , x2m1 )

T

Step 2: Compute ~y 0 = Fm ~x0 and ~y = Fm ~x, where Fm is an

n
2

matrix.

Step 3: Compute the components of ~y = Fn ~x by:

yj

= ~yj0 + nj ~yj

yj+m

= ~yj0 nj ~yj ,

(j = 0, 1, 2, , m 1)

Also, in Step 2, Fm ~x0 can be reduced to multiplication of Fm/2 and eventually to F1 (assuming n = 2l ).
This process is called the Fast Fourier transform (FFT).
Computation cost of FFT:
Let Cm =computations of Fm . Clearly, C1 = 1.
Claim: C2m = 2Cm + 3m.
Proof:
Recall: ~y 0 = Fm ~x; ~y = Fm ~x in Step 2.
Recall Step 3,
(

yj = ~yj0 + nj ~yj
yj+m = ~yj0 nj ~yj

1 multiplcation and 2 additions = T otal 3m

Therefore
C2m = 2Cm + 3m
l

Let n = 2 . Then:
C2l

2l C2l

=
=
=
=
=

Therefore

2C2l1 + 3 2l1
3
2l+1 C2l1 +
2 
3
l+2
2
C2l2 + 2
2

 
3
20 C20 + l
2
3
1+ l
2

3
3
n
C2l = 2l + l 2l = n + n log2 n = O( log2 n)
2
2
2

Exanple of FFT (F4 )

Goal: Find

y0
y1

y2 = F4
y3

x0
x1

x2
x3

Step 1: Split ~x into:

~x0 =

x0
x2

x1
x3

y0
y1

; ~x =

Step 2: Compute
~y 0 =
y00
y10

y00
y10


= F2

x0
x2


;

~y =


= F2



x0
= F2
, we do one more sub-step :
x2


x0
Sub-step 1: Split
into x0 ; x2 .
x2
Sub-step 2: Compute:
For ~y 0 =

~ys0

= F1 x 0 = x 0

~ys

= F1 x 2 = x 2

y00

= ys0 + 20 ys

Sub-step 3:

= x0 + x2

y10

= ys0 20 ys
= x0 x2


Similarly, for ~y =

y0
y1


= F2

x1
x3


, we have
y0

= x1 + x3

y1

= x1 x3

Step 3:
y0

= y00 + 40 y0 = (x0 + x2 ) + (x1 + x3 )

y1

= y10 + 40 y1 = (x0 x2 ) + i(x1 x3 )

y2

= y00 40 y0 = (x0 + x2 ) (x1 + x3 )

y3

= y10 40 y1 = (x0 x2 ) i(x1 x3 )

x1
x3