Function approximation: Fourier, Chebyshev, Lagrange

Orthogonal functions
Fourier Series
Discrete Fourier Series
Fourier Transform: properties
Chebyshev polynomials
Convolution
DFT and FFT
Scope: Understanding where the Fourier Transform comes
from. Moving from the continuous to the discrete world. The
concepts are the basis for pseudospectral methods and the
spectral element approach.

Orthogonal functions

Fourier Series: one way to derive them

The Problem
we are trying to approximate a function f(x) by another function g n(x)
which consists of a sum over N orthogonal functions (x) weighted by
some coefficients an.
N

f ( x) g N ( x) ai i ( x)
i 0

Orthogonal functions

The Problem
... and we are looking for optimal functions in a least squares (l 2) sense ...

f ( x) g

f ( x) g N ( x)
2

1/ 2

( x) dx

Min !

... a good choice for the basis functions (x) are orthogonal functions.
What are orthogonal functions? Two functions f and g are said to be
orthogonal in the interval [a,b] if
b

f ( x) g ( x)dx 0
a

How is this related to the more conceivable concept of orthogonal

vectors? Let us look at the original definition of integrals:
Orthogonal functions

Orthogonal Functions

f ( x) g ( x)dx lim f ( x) g ( x)x

N

i 1

... where x0=a and xN=b, and xi-xi-1=x ...

If we interpret f(xi) and g(xi) as the ith components of an N component
vector, then this sum corresponds directly to a scalar product of vectors.
The vanishing of the scalar product is the condition for orthogonality of
vectors (or functions).

fi
Orthogonal functions

gi

fi gi fi gi 0
i

Periodic functions
Let us assume we have a piecewise continuous function of the form

f ( x 2 ) f ( x)
40

f ( x 2 ) f ( x) x 2

30
20
10
0
-15

-10

-5

10

15

20

... we want to approximate this function with a linear combination of 2

periodic functions:

1, cos( x ), sin( x ), cos( 2 x ), sin( 2 x ),..., cos(nx ), sin( nx )

N
1
f ( x) g N ( x) a0 ak cos(kx) bk sin( kx)
2
k 1
Orthogonal functions

Orthogonality
... are these functions orthogonal ?

jk

cos(
jx
)
cos(
kx
)
dx

2 j k 0

jk 0

0 j k , j, k 0

sin(
jx
)
sin(
kx
)
dx

jk 0

cos( jx ) sin(kx)dx 0

j 0, k 0

... YES, and these relations are valid for any interval of length 2.
Now we know that this is an orthogonal basis, but how can we obtain the
coefficients for the basis functions?
from minimising f(x)-g(x)
Orthogonal functions

Fourier coefficients
optimal functions g(x) are given if

g n ( x) f ( x)

Min!

or

a k

( x) f ( x)

... with the definition of g(x) we get ...

g n ( x) f ( x)
ak

2
2

ak

N
1

a
cos(
kx
)

b
sin(
kx
)

f
(
x
)

0
k
k

2
k 1

leading to
N
1
g N ( x) a0 ak cos(kx) bk sin( kx)
2
k 1

1
ak

1
bk

Orthogonal functions

f ( x) cos(kx)dx,

with

k 0,1,..., N

f ( x) sin(kx)dx,

k 1,2,..., N

dx

Fourier approximation of |x|

... Example ...

f ( x) x ,

leads to the Fourier Serie

g ( x)

1
4 cos( x ) cos(3 x ) cos(5 x )

...

2

12
32
52

.. and for n<4 g(x) looks like

4
3
2
1
0
-20
Orthogonal functions

-15

-10

-5

10

15

20

Fourier approximation of x2
... another Example ...

0 x 2

f ( x) x 2 ,

leads to the Fourier Serie

N
4 2
4
4

g N ( x)
2 cos( kx )
sin( kx)
3
k

k 1 k

.. and for N<11, g(x) looks like

40
30
20
10
0
-10
-10

Orthogonal functions

-5

10

15

Fourier - discrete functions

... what happens if we know our function f(x) only at the points

2
xi
i
N
it turns out that in this particular case the coefficients are given by
2
ak
N
*

2
bk
N
*

f (x
j 1

) cos( kx j ),

k 0,1,2,...

) sin( kx j ),

k 1,2,3,...

f (x
j 1

.. the so-defined Fourier polynomial is the unique interpolating function to

the function f(xj) with N=2m

1 * m 1 *
1
g ( x ) a 0 a k cos(kx) b*k sin( kx) am* cos(kx)
2
2
k 1
*
m

Orthogonal functions

Fourier - collocation points

... with the important property that ...

g m* ( xi ) f ( xi )
... in our previous examples ...
3.5
3
2.5
2
1.5
1
0.5
0
-10

-5

f(x)=|x| => f(x) - blue ; g(x) - red; xi - +

Orthogonal functions

10

Fourier series - convergence

f(x)=x2 => f(x) - blue ; g(x) - red; xi - +

Orthogonal functions

Fourier series - convergence

f(x)=x2 => f(x) - blue ; g(x) - red; xi - +

Orthogonal functions

Gibbs phenomenon
f(x)=x2 => f(x) - blue ; g(x) - red; xi - +
N = 16

N = 64

N = 32

-2

-2

-2

-4

-4

-4

-6

0.5
1
N = 128

1.5

-6

-2

-2

-4

-4

-6

0.5

Orthogonal functions

1.5

-6

0.5
1
N = 256

1.5

-6

0.5

1.5

The overshoot for equispaced Fourier

interpolations is 14% of
the step height.
0

0.5

1.5

Chebyshev polynomials
We have seen that Fourier series are excellent for interpolating
(and differentiating) periodic functions defined on a regularly
spaced grid. In many circumstances physical phenomena which
are not periodic (in space) and occur in a limited area. This quest
leads to the use of Chebyshev polynomials.
We depart by observing that cos(n) can be expressed by a
polynomial in cos():

cos(2 ) 2 cos 2 1
cos(3 ) 4 cos 3 3 cos
cos(4 ) 8 cos 4 8 cos 2 1
... which leads us to the definition:

Orthogonal functions

Chebyshev polynomials - definition

cos(n ) Tn (cos( )) Tn ( x),

x cos( ),

x [1,1],

n N

... for the Chebyshev polynomials Tn(x). Note that because of

x=cos() they are defined in the interval [-1,1] (which - however can be extended to ). The first polynomials are

T0 ( x) 1
T1 ( x) x
T2 ( x) 2 x 2 1
T3 ( x) 4 x 3 3 x
T4 ( x) 8 x 4 8 x 2 1
Tn ( x) 1
Orthogonal functions

for

where

x [1,1]

and n N 0

Chebyshev polynomials - Graphical

The first ten polynomials look like [0, -1]
1

T _ n (x)

0.5

-0.5

-1

0.2

0.4

0.6

0.8

The n-th polynomial has extrema with values 1 or -1 at

x
Orthogonal functions

( ext )
k

k
cos(
),
n

k 0,1,2,3,..., n

Chebyshev collocation points

These extrema are not equidistant (like the Fourier extrema)

x(k)

x
Orthogonal functions

( ext )
k

k
cos(
),
n

k 0,1,2,3,..., n

Chebyshev polynomials - orthogonality

... are the Chebyshev polynomials orthogonal?
Chebyshev polynomials are an orthogonal set of functions in the
interval [-1,1] with respect to the weight function 1 / 1 x 2
such that
1

Tk ( x)T j ( x)

/2
1 x2

dx

for

k j

for k j 0 ,
for k j 0

... this can be easily verified noting that

x cos ,
dx sin d
Tk ( x) cos(k ), T j ( x) cos( j )
Orthogonal functions

k, j N0

Chebyshev polynomials - interpolation

... we are now faced with the same problem as with the Fourier
series. We want to approximate a function f(x), this time not a
periodical function but a function which is defined between [-1,1].
We are looking for gn(x)
n
1
f ( x) g n ( x) c0T0 ( x) ck Tk ( x)
2
k 1

... and we are faced with the problem, how we can determine the
coefficients ck. Again we obtain this by finding the extremum
(minimum)

ck

Orthogonal functions

( x) f ( x)

dx

1 x
2

Chebyshev polynomials - interpolation

... to obtain ...

2
ck

f ( x)Tk ( x)

dx
1 x

k 0,1,2,..., n

... surprisingly these coefficients can be calculated with FFT

techniques, noting that

2
ck f (cos ) cos kd ,
0

k 0,1,2,..., n

... and the fact that f(cos) is a 2-periodic function ...

1
ck f (cos ) cos kd ,

k 0,1,2,..., n

... which means that the coefficients ck are the Fourier coefficients
ak of the periodic function F()=f(cos )!
Orthogonal functions

Chebyshev - discrete functions

... what happens if we know our function f(x) only at the points

xi cos i
N
in this particular case the coefficients are given by
2
ck
N
*

f (cos
j 1

) cos( k j ),

k 0,1,2,... N / 2

... leading to the polynomial ...

m
1 *
g ( x ) c0T 0 ck*Tk ( x )
2
k 1
*
m

... with the property

g m* ( x ) f ( x)

Orthogonal functions

at

x j cos(j/N)

j 0,1,2,..., N

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; gn(x) - red; xi - +
N=8

1
0.8

8 points

0.6
0.4
0.2
0
-1

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.6

0.8

N = 16

1
0.8
0.6

16 points

0.4
0.2
0
-1

Orthogonal functions

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.6

0.8

Chebyshev - collocation points - |x|

f(x)=|x| => f(x) - blue ; gn(x) - red; xi - +
N = 32

1
0.8

32 points

0.6
0.4
0.2
0
-1

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.2

0.4

0.6

0.8

N = 128

1
0.8

128 points

0.6
0.4
0.2
0
-1

Orthogonal functions

-0.8

-0.6

-0.4

-0.2

0.6

0.8

Chebyshev - collocation points - x2

f(x)=x2 => f(x) - blue ; gn(x) - red; xi - +
N=8

1.2
1
0.8

8 points

0.6
0.4
0.2
0
-1

-0.8

-0.6

-0.4

-0.2

0
N = 64

0.2

0.4

0.6

0.8

-0.8

-0.6

-0.4

-0.2

0.2

0.4

0.6

0.8

1.2
1
0.8

64 points

0.6
0.4
0.2
0
-1

Orthogonal functions

The interpolating
function gn(x) was
shifted by a small
amount to be
visible at all!

Chebyshev vs. Fourier - numerical

Chebyshev

Fourier

N = 16

N = 16
35

0.8

30
25

0.6

20
15

0.4

10
5

0.2

0
0

0.2

0.4

0.6

0.8

-5

f(x)=x2 => f(x) - blue ; gN(x) - red; xi - +

This graph speaks for itself ! Gibbs phenomenon with Chebyshev?
Orthogonal functions

Chebyshev vs. Fourier - Gibbs

Chebyshev

Fourier

N = 16

1.5

1.5

0.5

0.5

-0.5

-0.5

-1

-1

-1.5
-1

-0.5

N = 16

0.5

-1.5

f(x)=sign(x-) => f(x) - blue ; gN(x) - red; xi - +

Gibbs phenomenon with Chebyshev? YES!
Orthogonal functions

Chebyshev vs. Fourier - Gibbs

Chebyshev

Fourier

N = 64

N = 64

1.5

1.5

0.5

0.5

-0.5

-0.5

-1

-1

-1.5
-1

-0.5

0.5

-1.5

f(x)=sign(x-) => f(x) - blue ; gN(x) - red; xi - +

Orthogonal functions

Fourier vs. Chebyshev

Chebyshev

Fourier
2
xi
i
N

collocation points

periodic functions

domain

cos( nx ), sin( nx )

basis functions

g m* ( x)

1 *
a0
2
m 1

a *k cos( kx) b*k sin( kx)

k 1

1 *
am cos( kx)
2

Orthogonal functions

xi cos

interpolating
function

i
N

limited area [-1,1]

Tn ( x ) cos( n ),
x cos

m
1 *
g ( x ) c0T 0 ck*Tk ( x )
2
k 1
*
m

Fourier vs. Chebyshev (contd)

Chebyshev

Fourier
2
ak
N
*

2
bk
N
*

f (x
j 1

) cos(kx j )
coefficients

f (x
j 1

2
ck
N
*

f (cos
j 1

) cos( k j )

) sin( kx j )

Gibbs phenomenon for

discontinuous functions
Efficient calculation via FFT
infinite domain through
periodicity

limited area calculations

some properties

grid densification at boundaries

coefficients via FFT
excellent convergence at
boundaries
Gibbs phenomenon

Orthogonal functions

The Fourier Transform Pair

1
F ( )
2
f (t )

f (t )e

i t

dt

Forward transform

it
F
(

)
e
d

Inverse transform

Note the conventions concerning the sign of the exponents and the factor.

Orthogonal functions

Some properties of the Fourier Transform

Defining as the FT:

Linearity
Symmetry
Time shifting
Time differentiation

Orthogonal functions

f (t ) F ( )

af1 (t ) bf 2 (t ) aF1 ( ) bF2 ( )

f (t ) 2F ( )
f (t t ) eit F ( )
n f (t )
n

)
F ( )
n
t

Differentiation theorem

Time differentiation

Orthogonal functions

n f (t )
n

)
F ( )
n
t

Convolution
The convolution operation is at the heart of linear systems.
Definition:

f (t ) g (t )

Properties:

f (t ' ) g (t t ' )dt ' f (t t ' ) g (t ' )dt '

f (t ) g (t ) g (t ) f (t )
f (t ) (t ) f (t )
f (t ) H (t ) f (t )dt
H(t) is the Heaviside function:

Orthogonal functions

The convolution theorem

A convolution in the time domain corresponds to a
multiplication in the frequency domain.
and vice versa
a convolution in the frequency domain corresponds to a
multiplication in the time domain

f (t ) g (t ) F ( )G ( )
f (t ) g (t ) F ( ) G ( )
The first relation is of tremendous practical implication!

Orthogonal functions

Summary

The Fourier Transform can be derived from the problem of

approximating an arbitrary function.
A regular set of points allows exact interpolation (or derivation) of
arbitrary functions
There are other basis functions (e.g., Chebyshev polynomials,
Legendre polynomials) with similar properties
These properties are the basis for the success of the spectral
element method

Orthogonal functions