Sie sind auf Seite 1von 37

# The Fast Fourier Transform

and
Applications to
Multiplication
Analysis of Algorithms

Prepared by

## John Reif, Ph.D.

Reading Selection:
CLR,
Chapter
30
Topics
and
Readings:

## The Fast Fourier Transform

Advanced Material :

## Nth Roots of Unity

Assume Commutative Ring (R,+,, 0,1)
is principal nth root of unity if
k 1 for k = 1, , n-1
n = 1, and
n-1

jp

0 for 1 p n
j=0

Example:

2 i/n

## Example of nth Root of Unity

for Complex Numbers

2 i/8

## is the 8th root of unity

Fourier Matrix
1

1 1

M n ( ) = 1 2 2( n1)

n1
( n1)( n1)

a0

given a =
a
n-1

n1

## Discrete Fourier Transform

Input a column n-vector a = (a0, , an-1)T
Output an n-vector which is the product of
the Fourier matrix times the input vector

DFTn (a) = M( ) x a
f0

= where
f
n-1
n-1

fi =

ik
a

k
k=0

## Inverse Fourier Transform

DFT (a) = M( ) x a
-1
n

-1

1 -ij
Theorem M( ) =
n
proof We must show M( ) M( )-1 = I
-1
ij

1
n

n-1

ik -kj

k=0

1
=
n

n-1

k(i-j)

k=0

0 if i-j 0
=
1 if i-j = 0
n-1

## using identity kp 0, for 1 p < n

k=0

Fourier Transform is
Polynomial Evaluation at the
Roots
of Unity
Input a column
n-vector a = (a0, , an-1)T
Output an n-vector (f0, , fn-1)T which are the
values polynomial f(x)at the n roots of unity

f0

f
n-1
f i = f( ) and
i

n-1

f(x) = a j x j
j=0

## Fast Fourier Transform

Viewed as Evaluation Problem:
nave algorithm takes n2 ops
Divide and Conquer gives FFT
with O(n log n) ops for n a power of 2
Key Idea:
If is nth root of unity
then 2 is n/2th root of unity
So can reduce the problem to two
subproblems of size n/2

Algorithm FFTn

## [2] f ,..., f FFTn (( a0 , a2 ,..., an 2 )T )

2

'
0

'
n
1
2

T
f ,..., f FFTn (( a1 , a3 ,..., an 1 ) )
2

n
[3] For i=0, ..., 1 do f i f i' i fi"
2
f n f i' i f i"
"
0

"
n
1
2

i+

## [4] Output (f 0 , f1 , ..., f n-1 )

FFT Circuit
(also known as Butterfly
Network)

## Total Recursion depth = log n

Communication Distance 2d at depth d

## f i = a 0 + a1 i + a 2 2i +...+a n-1 (n-1)i

f i = f i' + i f i" where
i(n-2)
2
2

i(n-2)
2
2

## f i" = a1 + a 3 ( 2 )i +...+ a n-1 ( )

a0
f

a
M M ( 2 ) 2 DFT ((a , a ,..., a )T )
n
n
0
2
n2

M
2
2
'
fn

1
a
2
n 2
"
a1
f0
a
M M ( 2 ) 3 DFT ((a , a ,..., a )T )
n
n
1
3
n 1

M
2
2
f n"

1
a
2
n 1
'
0

Note: f

'
n
i
2

'
i

=f , f

"
n
i
2

n
f , i=0, ..., 1
2
"
i

n
1
2 2

But n =1, so ( )

= n ( 2 )i 2i

n
for i=0, ..., 1
2
Thus, f i = f + f
'
i

and f

i+

'
=
f
n
i +

i+

n
2

f i"

=f - f
'
i

n
for i=0, ..., 1
2

"
i

n
2 2

"
i

n
for i=0, ..., 1
2
n
2

since ( ) n 1, so = -1

## Operation Counts for FFT

Algorithm
Assume n = 2k
# additions
Add(n) = 2 Add(n/2) + n
= n log n
# multiplications
Mult(n) = 2 Mult(n/2) + n/2
= n log n
Total Time
O(n log n)
Note in complex FFT,
# real ops is 5 n log n

Multipoint Polynomial
Evaluation
n1
Input polynomial f ( x)

i
a
x
i
i 0

## Problem evaluate f(x) at x0, x1, , xn-1

Easy Cases: eval at roots of unity
FFT Case xk = k for k=0,,n

Multipoint Polynomial
Evaluation (contd)

Summary of FFT:
method f ( x ) f '( y ) x f "( y )
where y x 2
f '( x), f "( x) both degree halved
needed to only evaluate at half as many points

## Other Polynomial Evaluation

Problems Solved by FFT
Each costs O(n log n) time

Reduced to FFT

## Single point evaluation of all derivatives of a polynomial

Solve by reduction to above Chirp Transform of case 2)

## Evaluate at points Xi = b(ai)2+ cai + d for i=0,, n-1

Solve by divide and conquer similar to FFT

## Single Point Evaluation of all

Derivatives of Polynomial
Input
n1

f ( x) ai xi

and point x0

i 0

output
k
d
f ( x)
k
f ( x)
x x0 for k 0,..., n 1
dx

## Single Point Evaluation of all

Derivatives of Polynomial
(contd)
Taylor Series Representation of
n1

f ( x) ci ( x x0 )i
i 0

Then

f ( k) ( x0 ) k!ck

## xi abi for i 0,..., n 1

Solve this Chirp Transform problem by
reduction FFT

Advanced Material:
Further Applications of FFT
1) Convolution: Products and Powers of
Polynomials
Used for for Integer Multiplication
Algorithms
Also used for Filtering on infinite
input streams
2) Division and Inverse of Polynomials
3) Multipoint Evaluation and
Interpolation

## Advanced Material: Products

and Powers of Polynomials
Input vectors
a = (a0, a1, , an-1)T
b = (b0, b1, , bn-1)T
Definition of Convolution c = a

n-1

Where

ci = a jbi-j

## for i=0, , 2n-1

j=0

define ak = bk = 0

if k< 0 or k>n

## Products and Powers of

Polynomials (contd)
Convolution Theorem

a b = FFT

-1
2n

FFT

2n

n-1

p(x) = a i x i
i=0

n-1

q(x) = b j x j
j=0

2n-2

n-1

i=0

j=0

## p(x) q(x) = ci x i where ci = a jbi-j

Products of m Polynomials
n-1

## for k=1, ..., m let Pk (x) = a k,i x i

i=0

m(n-1)

k=1

i=0

Pk (x) =

ci x i , where ci = a k,jk
k=1

jk =1

## Generalized Convolution Theorem

a1 a 2 ... a m =
FFTn-1m FFTn m (a1 ) FFTn m (a 2 ) ... FFTn m (a m )

Wrapped Convolutions
a = (a0, a1, , an-1)T , b = (b0, b1, , bn-1)T
Positive wrapped convolution is
c = (c0, c1, , cn-1)T
i

n-1

j=0

j=i+1

ci = a jbi-j + a jb n+i-j
Negative wrapped convolution is
d = (d0, d1, , dn-1)T
i

n-1

j=0

j=i+1

di = a jbi-j - a jb n+i-j

Application of Wrapped
Convolution to Modular
n-1
Polynomial
Products
i
p(x) = a i x
i=0

n-1

q(x) = b j x

j=0

n

n-1

i=0

## Computing Positive Wrapped

Convolution
Let = principal nth root of unity
Assume n has multiplicative inverse,
Theorem

-1
n

c = FFT

## is the positive wrapped convolution of

n-vectors a and b.

## Computing Negative Wrapped

Convolution
Also
d = FFT -1 FFT (a)

FFT
(b)
n
n
n

## is the negatively wrapped convolution of

n-vectors a and b
where

b = b , b , ...,

a = a 0 , a1 , ..., a n-1
n-1

n-1

b n-1

## and 2 = = principal nth root of unity

Integer Multiplication by
Polynomial Product (solved via
FFT)
Input n bit integers a,b
define polynomials degree k = n/L
k-1

a(x) = a i x ,
i

0 ai 2

i=0

k-1

b(x) = bi x i ,

0 bi 2L

i=0

so a = a(2 ), b = b(2 )

Integer Multiplication by
Polynomial Product (contd)
Idea
1) Compute c(x) = a(x) b(x)
by convolution
2) Evaluate c(2L) = a b

## Integer Multiplication Algorithms

using Reduction to Polynomial
Product
Pollard Mult Algorithm
2

## O(n(logn) )(loglogn) use L = logn

Karp Mult Algorithm

O(n(logn) 2 ) use L = n
Schnage-Strassen Mult Algorithm

O(n(logn)(loglogn)) use L = n
and wrapped convolution

Pollard Multiplication
Algorithm
n = kL, L = 1 + log k
1) Choose primes P1, P2, P3 where

P1 P2 P3 4 k

and Pi = i 2 + 1, i = O(1)
L

## 2) Compute C(x) by convolution over

finite field Zpi for i =1,2,3
(requires k mults on 2L bit integers)

Pollard Multiplication
Algorithm (contd)
3) Evaluate C(2L)
Time Bounds
recursive mults

FFT

2

## = 3kT (2(1 + logk)) + O(k(log k) )

O(n(log n) 2 (log log n) ) for any > 0

n = 2s = kL

k =

s
2
(s-1)
2

if s even
else

## 1) Compute C(x) modulo k by convolution

2) Compute C(x) modulo (22L+1) by
convolution
3) Compute C(x) coefficients from C(x)
mod k, C(x) mod (22L+1) by Chinese
remaindering

(contd)
4) Compute C(2L)
Time
recursive mults

FFT

## T(n) = 2kT(2L) + O(k logk)O(L)

= 2 nT (2 n ) + O(n log n)
O(n(log n) 2 )

Schnage-Strassen
Multiplication Algorithm
(2) Compute C(x) mod (xk+1) modulo
(22L+1) by wrapped convolution
Requires only k recursive mults on 2L bit
numbers
Time

recursive mults

FFT

## T(n) = kT(2L) + O(k logk)O(L)

= nT (2 n ) + O(n log n)
O(n log n)(log log n)

## Still Open Problem: How Fast

Can You Multiply Integers?
Can you mult n bit integers in
O(n log n) time?

## The Fast Fourier Transform

and
Applications to
Multiplication
Analysis of Algorithms

Prepared by