Lecture 8 PDF

Fast Fourier Transform:
Theory and Algorithms
Lecture 8
Vladimir Stojanovi
6.973 Communication System Design Spring 2006
Massachusetts Institute of Technology
Cite as: Vladimir Stojanovic, course materials for 6.973 Communication System Design, Spring 2006.
MIT OpenCourseWare (http://ocw.mit.edu/), Massachusetts Institute of Technology.
Downloaded on [DD Month YYYY].
Discrete Fourier Transform A review
Definition
{Xk} is periodic
Since {Xk} is sampled, {xn} must also be periodic
From a physical point of view, both are

repeated with period N
Requires O(N2) operations
6.973 Communication System Design
Fast Fourier Transform History
Twiddle factor FFTs (non-coprime sub-lengths)
1805 Gauss
Predates even Fouriers work on transforms!
1903 Runge
1965 Cooley-Tukey
1984 Duhamel-Vetterli (split-radix FFT)
FFTs w/o twiddle factors (coprime sub-lengths)
1960 Goods mapping
application of Chinese Remainder Theorem ~100 A.D.
1976 Rader prime length FFT

1976 Winograds Fourier Transform (WFTA)
1977 Kolba and Parks (Prime Factor Algorithm PFA)
Divide and conquer
Divide and conquer always has less computations

Suppose all Il sets have same number
of elements N1 so, N=N1*N2, r=N2
Each inner-most sum takes N12 multiplications

The outer sum will need N2 multiplications per output point
N2*N for the whole sum (for all output points)
Hence, total number of multiplications

Generalizations
The inner-most sum has to represent a DFT
Only possible if the subset (possibly permuted)
All main classes of FFTs can be cast in the above form
Both sums have same periodicity (Goods mapping)
No permutations (i.e. twiddle factors)

All the subsets have same number of elements m=N/r
Has the same periodicity as the initial sequence
(m,r)=1 i.e. m and r are coprime
If not, then inner sum is one stap of radix-r FFT

If r=3, subsets with N/2, N/4 and N/4 elements
Split-radix algorithm

The cost of mapping
The goal for divide and conquer
Different types balance mapping with subproblem

cost
E.g. in radix-2
subproblems are trivial (only sum and differences)

Mapping requires twiddle factors (large number of
multiplies)
E.g. in prime-factor algorithm
Subproblems are DFTs with coprime lengths (costly)

Mapping trivial (no arithmetic operations)
FFTs with twiddle factors
Reintroduced by Cooley-Tukey 65
Start from general divide and conquer

Keep periodicity compatible with
periodicity of the input sequence
Use decimation
almost N1 DFTs of size N2

Cooley-Tukey FFT contd.
can be taken mod N2
Yn1 ,k = Yn1 ,k2
1. N1 DFTs of length N2
2. N multplications with twiddle factors
3. N2 DFTs of length N1

Step 1: Evaluate N1 DFTs of length N2

Step 2: N multiplications with twiddle factors
Step 3: Evaluate N2 DFTs of length N1
Vector xi mapped to matrix xn1,n2 (N1xN2)
Compute N1 DFTs of length N2 on each row
Point-to-point multiply with twiddle factors
Compute N2 DFTs of length N1 on the columns

2-D view of Cooley-Tukey mapping
N=15 (N1=3, N2=5)
x0 x1...............
x13 x14
x0 x3 x6 x9 x12
x1 x4 x7 x10 x13
x2 x5 x8 x11 x14
x12
x12
x9
x3
x0
x7
x4
x1
x9
x13
x10
x14
x11
x8
x3
x0
DFT-5
DFT-3
DFT-3
DFT-5
x1
x14
x0
x13
x5
x5
x2
x9
DFT-3
x4
x5
x2
DFT-3
x6
jm
x6
x4
DFT-3
x12
x11
DFT-5
x10
Figure by MIT OpenCourseWare.
Cannot exchange the order of DFTs
Because of twiddle multiply

Different mapping for N1=5, N2=3
Not just transpose
x0
x1
x2
x3
x4
x5 x10
x6 x11
x7 x12
x8 x13
x9 x14

10
Radix 2 and radix 4 algorithms
Lengths as powers of 2 or 4 are most popular

Assume N=2n
N1=2, N2=2n-1 (divides input sequence into even and

odd samples decimation in time DIT)
Butterfly
(sum or difference followed or
preceeded by a twiddle factor multiply)
Xm and XN/2+m outputs of N/2 2-pt DFTs on outputs of

2, N/2-pt DFTs weighted with twiddle factors
11
DIT radix-2 implementations
Several different ways
Reorder the input data
Input samples for inner

DFTs in subsequent
locations
Results in bit-reversed
input, in-order output DIT
Selectively compute
DFTs on evens and
odds
Perform in-place
computation
Output in bit-reversed
order (X3 in position six
(011->110))
X0
DFT
DFT
X1
N=4
X2
{x2i}
N=2
DFT
N=2
X3
X4
DFT
DFT
X5
N=4
w18
X6
{x2i+1}
w2
X7
Division
DFT of
into even and N / 2
odd numbered
sequences
N=2
DFT
w38
N=2
Multiplication
by twiddle
factors
X0
X4
X1
X5
X2
X6
X3
X7
DFT of
2
Which type is this implementation?

12
Decimation in frequency (DIF) radix-2 implementation
If reverse the role of N1 and N2, get DIF
X2k1 =
N1=N/2, N2=2
N / 2-1
n1=0
X2k1+1 =
n k
W N1/ 12(xn +xN / 2+n ),

1
1
N / 2-1
n k
W N1/ 12 Wn1(xn -xN / 2+n ).

1
1
N
n1=0
X0
DFT
DFT
X0
X1
N=2
N=4
X2
X2
DFT
{x2k}
X4
X3
N=2
X4
DFT
X5
N=2
X6
DFT
X7
N=2
w18
w2
X6
DFT
X1
N=4
X3
{x2k+1}
X5
w3
DFT of Multiplication
2
by twiddle
factors
X7
DFT of
N/2
13
Duality DIT<->DIF
X0
DFT
DFT
X1
N=4
X2
{x2i}
N=2
DFT
N=2
X3
X4
DFT
DFT
X5
N=4
w18
X6
{x2i+1}
w2
X7
w38
Division
by twiddle
into even and N / 2
factors
odd numbered
sequences
N=2
DFT
N=2
DFT of
2
X0
X0
X4
X1
X1
X2
X0
DFT
DFT
N=2
DFT
N=2
X5
X3
X2
X4
DFT
X6
X5
N=2
X3
X6
DFT
X7
X7
N=2
N=4
X2
{ }
X4
x2k
w1
X6
DFT
w2
X1
N=4
X3
{ }
X5
x2k+1
w3
X7
2
by twiddle
factors
DFT of
N/2
. are.
Figure by MIT OpenCourseW
Which one is DIT (DIF)?

How can we get one from another?
14
Complexity of radix-2 FFTs
DFT of length N replaced by two length-N/2
At the cost of N complex multiplications (twiddle)
Iterate the scheme log2N-1 times
And N complex additions (2pt DFTs)
Obtain trivial transforms (length 2) of the length-N/2 DFTs
Twiddle multiplies (WNi)
Complex multiply 3 real mult + 3 real add

If i is multiple of N/4, no arithmetic operation required (why?)
4 butterflies (one general, 3 special cases)

15
Radix-4
N=4n, N1=4, N2=N/4
Cost of length-4 DFT
4 DFTs of length N/4

3N/4 twiddle multiplies
N/4 DFTs of length 4
No multiplication
Only 16 real additions
x0
X0
DFT
8
DFT
2
X14
X1
x8
DFT
8
x15
X15
x0
x4
x8
DFT
4
DFT
4
x12
x15
DFT
4
DFT
4
DFT
4
Reduces the number of stages to log4N
X0
X12
X1
X13
X2
X14
X3
X15
Radix-8 can reduce number of operations even more

16
Mixed-radix and Split-radix
Mixed-radix
Diferent radices in different

stages
Split-radix
X0
DFT
8
DFT
2
R2
X14
X1
x8
DFT
8
Different radices in the same

stage
x0
Simultaneously on different
parts of the transform
Can achieve lowest number

of adds and multiplies for
length 2n inputs
x15
X15
x0
x4
x8
X0
DFT
4
DFT
4
X12
X1
X13
X2
X14
X3
DFT
4
DFT
4
x12
DFT
4
x15
X15
X0
x0
x4
x8
x12
R4
DFT
2/4
DFT
8
DFT
4
DFT
4
X14
X1
X13
X3
S-R
X15

17
Split-radix (DIF SRFFT)
Look at DIF radix-2
X2k1 dont have twiddles

X2k1 =
N / 2-1
n1=0
X2k +1 =
1
nk
W N1 / 12(xn +xN / 2+n ),
1
1
N / 2-1
n1=0
nk
W N1/ 12 Wn1(xn -xN / 2+n ).
1
1
N
X0
DFT
DFT
X0
X1
N=2
N=4
X2
X2
DFT
{x2k}
X4
X3
N=2
X4
DFT
X5
N=2
X6
DFT
X7
N=2
w1
w2
Even samples X2k in DIF should

be computed separately from
other samples
With same algorithm (recursively) as

the original sequence
No general rule for odd samples
DFT
X1
N=4
X3
{x2k+1}
X5
w83
2
by twiddle
factors
X6
Radix-4 is more efficient than radix-2

Higher radices are inefficient
X2k =
1
N / 2-1
n1=0
X4k +1 =
1
N / 4-1
n1=0
nk
W N1/ 14Wn1
N
X[(xn -xN / 2+n )

1
1
+j(xn1+N / 4-xn1+3N / 4)],
X7
DFT of
N/2
nk
W N1/ 12(xn +xN / 2+n ),

1
1
X4k +3 =
1
N / 4-1
n1=0
n1k1 3n
1
N / 4WN

X[(xn +xN / 2+n )
1
1
-j(xn1+N / 4-xn1+3N / 4)].
X0
X0
DFT
8
X4
X8
X12
DFT
2/4
DFT
4
DFT
4
X14
X1
X13
X3
X15
Split-radix (DIT SRFFT)
Dual to DIF SRFFT
Considers separately subsets {x2i}, {x4i+1} and {x4i+3}
Redundancy in Xk, Xk+N/4, Xk+N/2, Xk+3N/4 computation

19
FFTs without twiddle factors
Divide and conquer requirements
N-long DFT computed from DFTs with lengths that are

factors of N (allows the inner sum to be a DFT)
Provided that subsets Il guarantee periodic xi
When N factors into co-prime factors N=N1*N2
Starting from any xi form subset with compatible periodicity (the

periodicity of the subset divides the periodicity of the set)
{xi + N1n2 | n2 = 1,..., N 2 1} or {

xi + N2 n1 | n1 = 1,..., N1 1}
Both subsets have only one common point xi
Allows a rearrangement of the input (periodic) vector into a
matrix with a periodicity in both dimensions (rows and
columns), both periodicities being comatible with the initial one
20
Goods mapping
FFTs without twiddle factors all based on the same

mapping
Turns original transform into a set of small DFTs with

coprime lengths
{xi + N1n2 | n2 = 1,..., N 2 1} or {xi+ N2 n1 | n1 = 1,..., N1 1}
equivalent to
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Good's mapping
9 12
8 11 14 2
10 13 1
. are.
This mapping is one-to-one if N1 and N2 are coprime

All congruences modulo N1 obtained
For a given congruence modulo N2 and vice versa

21
Just another arrangement of CRT
Chinese Remainder Theorem (CRT)
If we know the residue of some number k modulo

two coprime numbers N1 and N2
k N
k N
It is possible to reconstruct k N N
Let k N = k1 k N = k2
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Then k N N = N1t1k2 + N 2t2 k1
N
1
N2
t1 N1
= 1 and t2 N 2
N1
= 1
t1 multiplicative inverse of N1 mod N2
t2 multiplicative inverse of N2 mod N1
t1, t2 always exist since N1, N2 coprime (N1,N2)=1
Good's mapping
What are t1, t2 for N1=3, N2=5?
CRT mapping
Reversing N1 and N2
8 11 14 2
10 13 1
9 12
4
6 12 3
10 1 7 13 4
5 11 2 8 14
. are.
Results in transposed mapping
6.973
22
Impact on DFT
Formulating the true multi-dimensional transform

k
N1 N 2
= N1t1k2 + N 2t2 k1
N-1
Xk = xi WNik, k = 0,..., N - 1,
i=0
N1 - 1 N2 - 1
XN1t1k2 + N2r2k1 =
(n1N2 + N1n2)(N1t1k2+N2t2k1)
N
xn1N2 + n2N1 W
n1 = 0 n2 = 0
x3
x6
x9
x12
X9
DFT
5
X4
x0
N2
N
W = WN1
N2t2
N1
(N2t2)N
1
N1
=W
N1 - 1 N2 - 1
XN1t1k2 + N2t2k1 =
= WN1
n k
x5
n k
xn1N2 + n2N1 WN11 2 WN22 2 ,
DFT
3
X2
X8
X14
X11
x10
X5
n1 = 0 n 2 = 0
X'k1k2 = XN1t1k2 + N2t2k1

N1 - 1 N2 - 1
X'k1k2 =
x'n1.n2 = xn1N2 + n2N1

n k
n k
x'n1n2 WN11 1 WN22 2
n1 = 0 n2 = 0
Figure by MIT OCW.
True bidimensional transform!
(no extra twiddle factors)
23
Using convolution to compute DFTs
All sub DFTs are prime length
Rader showed that prime-length DFTs can be
computed as a result of cyclic convolution
E.g. length 5 DFT
Permute last two rows and columns
Cyclic correlation
(a convolution with a reversed sequence)
This is a general result

24
Example
Results in smallest number of multiplies

25
Prime Factor Algorithm
X = F1 x F2T.
Good's mapping
x9
x3
CRT mapping
x12
x6
X9
X4
DFT
5
X14
x0
x5
x10
DFT
3
X2
X5
X8
X11
Efficient small DFTs are a key to the

feasibility of this algorithm

26
Winograds Fourier Transform Algorithm
x10
x0
X5
x5
X11
X2
x3
X8
x6
x9
x12
X9
input additions
N=3
X4
X14
input additions point wise output additions output additions

multiplication
N=5
N=5
N=3
B1xB2 only involves additions

D diagonal (so point multiply)
Winograd transform has many more additions than
twiddle FFTs
27

Lecture 8 PDF

Hochgeladen von

Dokumentinformationen

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Lecture 8 PDF

Hochgeladen von

Copyright:

Verfügbare Formate

Fast Fourier Transform:

Theory and Algorithms

6.973 Communication System Design Spring 2006

Massachusetts Institute of Technology

Discrete Fourier Transform A review

Since {Xk} is sampled, {xn} must also be periodic

From a physical point of view, both are

Fast Fourier Transform History

Twiddle factor FFTs (non-coprime sub-lengths)

Predates even Fouriers work on transforms!

1984 Duhamel-Vetterli (split-radix FFT)

FFTs w/o twiddle factors (coprime sub-lengths)

1960 Goods mapping

application of Chinese Remainder Theorem ~100 A.D.

1976 Rader prime length FFT

Divide and conquer

Divide and conquer always has less computations

Each inner-most sum takes N12 multiplications

Hence, total number of multiplications

6.973 Communication System Design

The inner-most sum has to represent a DFT

Only possible if the subset (possibly permuted)

All main classes of FFTs can be cast in the above form

Both sums have same periodicity (Goods mapping)

No permutations (i.e. twiddle factors)

Has the same periodicity as the initial sequence

(m,r)=1 i.e. m and r are coprime

If not, then inner sum is one stap of radix-r FFT

6.973 Communication System Design

The cost of mapping

The goal for divide and conquer

Different types balance mapping with subproblem

subproblems are trivial (only sum and differences)

E.g. in prime-factor algorithm

Subproblems are DFTs with coprime lengths (costly)

FFTs with twiddle factors

Start from general divide and conquer

almost N1 DFTs of size N2

6.973 Communication System Design

Cooley-Tukey FFT contd.

can be taken mod N2

Yn1 ,k = Yn1 ,k2

2. N multplications with twiddle factors

6.973 Communication System Design

Step 1: Evaluate N1 DFTs of length N2

6.973 Communication System Design

2-D view of Cooley-Tukey mapping

N=15 (N1=3, N2=5)

Figure by MIT OpenCourseWare.

Cannot exchange the order of DFTs

Because of twiddle multiply

Not just transpose

Figure by MIT OpenCourseWare.

6.973 Communication System Design

Radix 2 and radix 4 algorithms

Lengths as powers of 2 or 4 are most popular

N1=2, N2=2n-1 (divides input sequence into even and

(sum or difference followed or

preceeded by a twiddle factor multiply)

Xm and XN/2+m outputs of N/2 2-pt DFTs on outputs of

DIT radix-2 implementations

Several different ways

Reorder the input data