Sie sind auf Seite 1von 15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

SoundProcessingwithShortTime
FourierTransform
This numerical tour explores local Fourier analysis of sounds, and its application to
sourcedenoising.

Contents
Installingtoolboxesandsettingupthepath.
LocalFourieranalysisofsound.
ShorttimeFouriertransform.
AudioDenoising
AudioBlockThresholding

Installingtoolboxesandsettingupthepath.
Youneedtodownloadthefollowingfiles:signaltoolboxandgeneraltoolbox.
You need to unzip these toolboxes in your working directory, so that you have
toolbox_signalandtoolbox_generalinyourdirectory.
ForScilabuser:youmustreplacetheMatlabcomment'%'byitsScilabcounterpart
'//'.
Recommandation: Youshouldcreate a text file named forinstance numericaltour.sce
(in Scilab) or numericaltour.m (in Matlab) to write all the Scilab/Matlab command you
wanttoexecute.Then,simplyrun exec('numericaltour.sce');(inScilab)or numericaltour;
(inMatlab)torunthecommands.
ExecutethislineonlyifyouareusingMatlab.
getd=@(p)path(p,path);%scilabusersmust*not*executethis

Thenyoucanaddthetoolboxestothepath.
getd('toolbox_signal/');
getd('toolbox_general/');

Warning:Nameisnonexistentornotadirectory:toolbox_signal
Warning:Nameisnonexistentornotadirectory:toolbox_general

LocalFourieranalysisofsound.
http://www.numericaltours.com/matlab/audio_1_processing/

1/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

Asoundisa1Dsignalthatislocallyhighlyoscillatingandstationary.AlocalFourier
analysisisthususefulltostudythepropertyofthesoundsuchasitslocalamplitude
andfrequency.
Firstweloadasound,withaslightsubsampling
n=1024*16;
options.n=n;
[x,fs]=load_sound('bird',n);

Warning:WAVREADwillberemovedinafuturerelease.UseAUDIOREADinstead.

You can actually play a sound. In case this does not work, you need to run the
commandwavwrite(x(:)','tmp.wav')andclickonthesavedfile'tmp.wav'toreadit.
sound(x(:)',fs);

Wecandisplaythesound.
clf;
plot(1:n,x);
axis('tight');
set_graphic_sizes([],20);
title('Signal');

Localzoomsonthesoundshowthatitishighlyoscilating.

http://www.numericaltours.com/matlab/audio_1_processing/

2/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

p=512;
t=1:n;
clf;
sel=n/4+(0:p1);
subplot(2,1,1);
plot(t(sel),x(sel));axistight;
sel=n/2+(0:p1);
subplot(2,1,2);
plot(t(sel),x(sel));axistight;

Exercice1:(checkthesolution)ComputethelocalFouriertransformaroundapointt0
of x, which is the FFT (use the function fft) of the windowed signal x.*h where h is
smooth windowing function located around t0. For instance you can use for h a
Gaussianbumpcenteredatt0.TocentertheFFTfordisplay,usefftshift.
exo1;

http://www.numericaltours.com/matlab/audio_1_processing/

3/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

A good windowing function should balance both time localization and frequency
localization.
t=linspace(10,10,2048);
eta=1e5;
vmin=2;

Theblockwindowhasasharptransitionandthusapoorfrequencylocalization.
h=double(abs(t)<1);
hf=fftshift(abs(fft(h)));
hf=log10(eta+hf);hf=hf/max(hf);
clf;
subplot(2,1,1);
title('Blockwindow');
plot(t,h);axis([22,.1,1.1]);
subplot(2,1,2);
plot(t,hf);axis([22,vmin,1.1]);
title('Fouriertransform');

http://www.numericaltours.com/matlab/audio_1_processing/

4/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

AHammingwindowissmoother.
h=cos(t*pi()/2).*double(abs(t)<1);
hf=fftshift(abs(fft(h)));
hf=log10(eta+hf);hf=hf/max(hf);
clf;
subplot(2,1,1);
title('Hammingwindow');
plot(t,h);axis([22,.1,1.1]);
subplot(2,1,2);
plot(t,hf);axis([22,vmin,1.1]);
title('Fouriertransform');

http://www.numericaltours.com/matlab/audio_1_processing/

5/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

AHaningwindowhascontinuousderivatives.
h=(cos(t*pi())+1)/2.*double(abs(t)<1);
hf=fftshift(abs(fft(h)));
hf=log10(eta+hf);hf=hf/max(hf);
clf;
subplot(2,1,1);
title('Haningwindow');
plot(t,h);axis([22,.1,1.1]);
subplot(2,1,2);
plot(t,hf);axis([22,vmin,1.1]);
title('Fouriertransform');

http://www.numericaltours.com/matlab/audio_1_processing/

6/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

A normalized Haning window has a sharper transition. It has the advantage of


generatingatightframeSTFT,andisusedinthefollowing.
h=sqrt(2)/2*(1+cos(t*pi()))./sqrt(1+cos(t*pi()).^2).*double(abs(t)<1);
hf=fftshift(abs(fft(h)));
hf=log10(eta+hf);hf=hf/max(hf);
clf;
subplot(2,1,1);
title('NormalizedHaningwindow');
plot(t,h);axis([22,.1,1.1]);
subplot(2,1,2);
plot(t,hf);axis([22,vmin,1.1]);
title('Fouriertransform');

http://www.numericaltours.com/matlab/audio_1_processing/

7/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

ShorttimeFouriertransform.
Gathering a local Fourier transform at equispaced point create a local Fourier
transform,alsocalledspectrogram.Bycarefullychosingthewindow,thistransform
corresponds to the decomposition of the signal in a redundant tight frame. The
redundancy corresponds to the overlap of the windows, and the tight frame
corresponds to the fact that the pseudoinverse is simply the transposed of the
transform (it means that the same window can be used for synthesis with a simple
summationofthereconstructedsignalovereachwindow).
Theonlyparametersofthetransformarethesizeofthewindowandtheoverlap.
%sizeofthewindow
w=64*2;
%overlapofthewindow
q=w/2;

Gabor atoms are computed using a Haning window. The atoms are obtained by
translatingintimeandinfrequency(modulation)thewindow.
t=0:3*w1;
t1=t2*w;
f=w/8;
%Position0,frequency0.
g1=sin(pi*t/w).^2.*double(t<w);
%Position2*w,frequency0.
g2=sin(pi*t1/w).^2.*double(t1<w&t1>=0);
%Position0,frequencyw/8
g3=g1.*sin(t*2*pi/w*f);
http://www.numericaltours.com/matlab/audio_1_processing/

8/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

%Position2*w,frequencyw/8
g4=g2.*sin(t*2*pi/w*f);
%display
clf;
subplot(2,2,1);
plot(g1);axis('tight');
title('Position0,frequency0');
subplot(2,2,2);
plot(g2);axis('tight');
title('Position2*w,frequency0');
subplot(2,2,3);
plot(g3);axis('tight');
title('Position0,frequencyw/8');
subplot(2,2,4);
plot(g4);axis('tight');
title('Position2*w,frequencyw/8');

We can compute a spectrogram of the sound to see its local Fourier content. The
numberofwindowedusedis(nnoverlap)/(wnoverlap)
S=perform_stft(x,w,q,options);

Toseemoreclearlytheevolutionoftheharmonics,wecandisplaythespectrogramin
logcoordinates.Thetopofthespectrogramcorrespondstolowfrequencies.
%displaythespectrogram
clf;imageplot(abs(S));axis('on');
%displaylogspectrogram
plot_spectrogram(S,x);

http://www.numericaltours.com/matlab/audio_1_processing/

9/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

TheSTFTtransformisdecomposingthesignalinaredundanttightframe.Thiscanbe
checkedbymeasuringtheenergyconservation.
%energyofthesignal
e=norm(x,'fro').^2;
%energyofthecoefficients
eS=norm(abs(S),'fro').^2;
disp(strcat(['Energyconservation(shouldbe1)='num2str(e/eS)]));

Energyconservation(shouldbe1)=1

Onecanalsocheckthattheinversetransform(whichisjustthetransposedoperator
itimplementsexactlythepseudoinverse)isworkingfine.
%onemustgivethesignalsizeforthereconstruction
x1=perform_stft(S,w,q,options);
disp(strcat(['Reconstructionerror(shouldbe0)='num2str(norm(xx1,'fro')./norm(x,'fro'))]));

Reconstructionerror(shouldbe0)=2.2401e16

AudioDenoising
OnecanperformdenosingbyanonlinearthresholdingoverthetransfomedeFourier
domain.
Firstwecreateanoisysignal
http://www.numericaltours.com/matlab/audio_1_processing/

10/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

sigma=.2;
xn=x+randn(size(x))*sigma;

Playthenoisysound.
sound(xn,fs);

DisplaytheSounds.
clf;
subplot(2,1,1);
plot(x);axis([1n1.21.2]);
set_graphic_sizes([],20);
title('Originalsignal');
subplot(2,1,2);
plot(xn);axis([1n1.21.2]);
set_graphic_sizes([],20);
title('Noisysignal');

Onecanthresholdthespectrogram.
%performthresholding
Sn=perform_stft(xn,w,q,options);
SnT=perform_thresholding(Sn,2*sigma,'hard');
%displaytheresults
subplot(2,1,1);
plot_spectrogram(Sn);
subplot(2,1,2);
plot_spectrogram(SnT);
http://www.numericaltours.com/matlab/audio_1_processing/

11/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

Exercice2:(checkthesolution)Adenoisingisperformedbyhardorsoftthresholding
the STFT of the noisy signal. Compute the denosing SNR with both soft and hard
thresholding, and compute the threshold that minimize the SNR. Remember that a
soft thresholding should be approximately twice smaller than a hard thresholding.
Check the result by listening. What can you conclude about the quality of the
denoisedsignal?
exo2;

http://www.numericaltours.com/matlab/audio_1_processing/

12/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

Exercice3:(checkthesolution)Displayandheartheresults.Whatdoyounotice?
exo3;

http://www.numericaltours.com/matlab/audio_1_processing/

13/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

AudioBlockThresholding
ItispossibletoremovemusicalnoisebythresholdingblocksofSTFTcoefficients.
Denoisingisperformedbyblocksoftthresholding.
%performthresholding
Sn=perform_stft(xn,w,q,options);
SnT=perform_thresholding(Sn,sigma,'block');
%displaytheresults
subplot(2,1,1);
plot_spectrogram(Sn);
subplot(2,1,2);
plot_spectrogram(SnT);

Exercice 4: (check the solution) Trie for various block sizes and report the best
results.
exo4;

http://www.numericaltours.com/matlab/audio_1_processing/

14/15

2/5/2016

SoundProcessingwithShortTimeFourierTransform

Copyright(c)2010GabrielPeyre

http://www.numericaltours.com/matlab/audio_1_processing/

15/15