Digital Signal Processing:

A copy-and-paste introduction

University of Tokyo
61-0811 Computational Hearing Systems
Jean-Julien Aucouturier
Disclaimer:
This course material is a shameful hacking together of a variety of
teaching material available online.
Credits should go primarily to:
- Prof. Juan Bello (NYU) http://homepages.nyu.edu/~jb2843/Teaching.html
- Prof. Dan Ellis (Columbia) http://www.ee.columbia.edu/~dpwe/e6820/
- Prof. Robi Polikar (Rowan Univ.) http://users.rowan.edu/~polikar/classes/ece351/
WHAT IS SOUND ?
 SAMPLING

The continuous analog signal is then converted to digital, i.e.

sampled (in time)
quantized (in amplitude)

Sampling: Quantization:
- sampling period Ts - quantization noise
- sampling frequency Fs=1 / Ts
- value
 ANGULAR FREQUENCIES

As we will deal a lot with sinusoids, we define angular frequencies:

continuous-time frequency f = 1 / T y(t) = sin(2.f.t)
in Hertz

continuous-time angular frequency = 2 / y(t) = sin(t)

in radian/sec.

discrete-time angular frequency = .Ts y(n.Ts) = sin(.Ts.n) = sin(n)

in radian/sample
 ALIAS

If sampling frequency is too small, then there is ambiguity

An infinity of continuous-time signals (with frequency +k.2) which, when
sampled, lead to the same discrete signal
e.g. one can hear another, lower frequency signal which share samples with the
original signal (an alias)
 ALIAS

Related to the wagon-wheel effect

http://www.michaelbach.de/ot/mot_strob/index.html
FOURIER
 FOURIER SERIES

Fourier series: for periodic continuous-time signals

A periodic function (*) can be represented has a finite weighted sum of
sinusoids whose frequencies are integer multiples of the fundamental
frequency of the signal
These frequencies are harmonically related, hence harmonics of the signal

(*) Note there are conditions (Dirichlet, notably )

 FOURIER SERIES

Fourier series: for periodic continuous-time signals

By the way, the ck are complex valued, with a modulus and phase
 FOURIER SERIES

Fourier series: for periodic continuous-time signals

What does this mean for sound ?

http://homepages.gac.edu/~huber/fourier/
 FOURIER TRANSFORM

Fourier transform (FT): extension of FS for non-periodic continuous-time signals

no longer discrete, finite sum of sinusoids
now, integrals over a continuum of frequencies, no longer harmonics
 DISCRETE-TIME FOURIER TRANSFORM

Discrete-time Fourier transform (DTFT): extension of FT for non-periodic discrete-

time signals
so we can use a sampled time-signal
DTFT is still continuous in frequency

We use discrete-time
angular frequency (i.e.
radian/sample)
 DISCRETE-TIME FOURIER TRANSFORM

Discrete-time Fourier transform (DTFT): extension of FT for non-periodic discrete-

time signals
DTFT is 2-periodic

|X()|

The discrete-time angular frequency 2 radian/sample corresponds

to the continuous-time sampling frequency s used to sample x[n]
 DISCRETE FOURIER TRANSFORM

Discrete Fourier transform (DFT): extension of DTFT for non-periodic discrete-

time signals with discrete frequencies
so we can have a sampled (digital) representation of the spectrum
simply sample the discrete-time frequency (which is continuous !) on N points in
the [0,2] interval

sampled
 DISCRETE FOURIER TRANSFORM

The spectrum is a N-point sequence in the frequency domain (sampling [0,2])

N points N points N points t

Here we see that the input
signal must also be a N-point
sequence (we can only
reconstruct N points from the f f f
N points N points N points
N frequency components) 0 s 0 s 0 s
 DFT: TIME-FREQUENCY PROBLEM
N points N points N points t

f f f
N points N points N points
0 s 0 s 0 s

To increase the frequency resolution, increase N

But to increase N is to reduce temporal resolution
 DFT: TIME-FREQUENCY PROBLEM
N points N points N points t

f f f
N points N points N points
0 s 0 s 0 s

To increase the frequency resolution, increase N

But to increase N is to reduce temporal resolution

Solution: compute DFT on

overlapping short-time windows
usually N = 2048 points,
overlapping by 1024 points
at Fs=44100, thats a new
spectrum of 2048 every 23 ms
 BACK TO SAMPLING
c.f. DTFT & DFT spectrum are 2 periodic (in frequency)
This is a consequence of sampling: in frequency domain, a convolution with a comb
of Dirac functions
 BACK TO SAMPLING
If s-M > M, i.e. s > 2*M,
Then the spectrum of the original signal (hence the original signal) can be
recovered exactly from its sampled version
 BACK TO SAMPLING
! BUT ! If s-M < M, i.e. s < 2*M,
The 2 replicated spectrum is folded onto itself: the original spectrum (and
signal) cannot be exactly recovered (distorted at the boundary areas)
 BACK TO SAMPLING
! BUT ! If s-M < M, i.e. s < 2*M,
The 2 replicated spectrum is folded onto itself: the original spectrum (and
signal) cannot be exactly recovered (distorted at the boundary areas)

c.f. aliasing: if sampling frequency is too

small (s < 2*M), then the continous
signal cannot be uniquely/exactly
reconstructed from the sampled signal.
 SAMPLING THEOREM

Nyquist frequency (or folding frequency): s / 2

Nyquist-Shannon sampling theorem:

aliasing can be avoided if the Nyquist frequency is greater than the maximum
frequency (or bandwidth) of the signal being sampled, i.e. s > 2*M
 DFT IS N-PERIODIC IN TIME !

As window-size (N) is not related to the

signals frequency(-cies), this introduces
discontinuities at the boundaries !
 LEAKING PROBLEM

As window-size (N) is not related to the

signals frequency(-cies), this introduces
discontinuities at the boundaries !

In theory, the DFT of a sinusoidal signal is a single spectral component at f0

In practice, the signal is smeared around f0

 LEAKING PROBLEM

One solution is to use a window shape which is not rectangular, but

which smoothly reduces the signal to 0 at its boundaries:
Blacking
Hamming
 A QUICK NOTE ON FFT

Discrete Fourier transform (DFT): extension of DTFT for non-periodic discrete-

time signals with discrete frequencies

all fine, but very costly to compute

Quadratic (4N2 multiplications and additions)

Fast-Fourier Transform (FFT): identical to DFT except much quicker (N.log(N))

based on decimation (J. W. Cooley and J. W. Tukey in 1965)
if N = 2p, N-point DFT can be computed as 2 N/2-point DFT, then 4 N/4-point DFT,
etc.
at the end of the chain, 2-point DFT requires no multiplication, only 2 additions
 SUMMARY

Fourier series: Fourier transform: DTFT: DFT:

time: periodic continuous time: non-periodic, continuous time: discrete time: periodic, discrete

frequency: discrete frequency: continuous frequency: periodic, continuous frequency: periodic, discrete
 SUMMARY

Fourier series: Fourier transform: DTFT: DFT:

time: periodic continuous time: non-periodic, continuous time: discrete time: periodic, discrete

frequency: discrete frequency: continuous frequency: periodic, continuous frequency: periodic, discrete

FFT: same but faster

 SUMMARY

Fourier series: Fourier transform: DTFT: DFT:

time: periodic continuous time: non-periodic, continuous time: discrete time: periodic, discrete

frequency: discrete frequency: continuous frequency: periodic, continuous frequency: periodic, discrete

leaking problem FFT: same but faster

 SUMMARY

Fourier series: Fourier transform: DTFT: DFT:

time: periodic continuous time: non-periodic, continuous time: discrete time: periodic, discrete

frequency: discrete frequency: continuous frequency: periodic, continuous frequency: periodic, discrete

risk of aliasing (folding spectrum) leaking problem FFT: same but faster
sampling theorem s>2.M
FOURIER AND THE EAR
 FOURIER AND THE EAR

Cochlea: travelling wave moving down basilar membrane

Varying stiffness and mass: continuous variation of resonating frequency

So inner ear does some kind of

frequency analysis, not unlike Fourier
 FOURIER AND THE EAR

On BM, ~3500 hair cell convert vibration into berve firing

Spike density is
proportional to vibration
at resonating frequency,
not unlike Fourier
coefficient
FOURIER AND THE EAR

N.B. Constant-Q: bandwidth-to-center-frequency ratio is constant)

 FOURIER AND THE EAR

Then of course, beyond auditory nerve, it gets messy:

brainstem, then auditory cortex
ascending AND descending
integrating modalities
memory, etc.

Still poorly understood