Time series analysis

Matlab tutorial
Joachim Gross
 Outline
Terminology
Sampling theorem
Plotting
Baseline correction
Detrending
Smoothing
Filtering
Decimation
 Remarks
Focus on practical aspects, exercises, getting
experience (not on equations, theory)
Focus on How to do
Learn some basic skills for TS analysis

Note: Usually there is not a single perfectly correct way

of doing a TS operation!
=> learn the limitations!
 What is a time series?
A sequence of measurements over time

1.5

0.5

-0.5

-1
0 2 4 6 8 10
 Terminology
Continuous TS: continuous observations
Discrete TS: observations at specific times usually equally spaced

Deterministic TS: future values can be exactly predicted from past

values
Stochastic TS: exact prediction not possible
 Objectives of TS analysis
Description
Explanation
Prediction
Control
 Simple descriptive analysis
Summary statistics (mean, std) is not always meaningful for
TS

12

10

0
0 50 100 150 200 250 300 350 400 450 500
 Sampling
Converting a continuous signal into a discrete time series
Reconstruction is possible if sampling frequency is greater than twice
the signal bandwidth

1 1

0.5 0.5

0 0

-0.5 -0.5

-1 -1
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
Time (s)
75 Hz sampling
 Sampling
Nyquist frequency: half of sampling frequency

1 3

2
0.5
1

0 0

-1
-0.5
-2

-1 -3
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1

10 Hz sampling 10 Hz reconstruction
 Sampling
Aliasing: Frequencies above Nyquist frequency are
reconstructed below Nyquist frequency

0.5

-0.5

-1
0 0.2 0.4 0.6 0.8 1

8 Hz sampling
 Sampling
Aliasing: Frequencies above Nyquist frequency are
reconstructed below Nyquist frequency

20 20
Power Spectrum Magnitude (dB)

0 0

-20 -20

Power Spectrum Magnitude (dB)

-40 -40

-60 -60

-80 -80

-100 -100

-120 -120

-140 -140
0 5 10 15 20 0 1 2 3 4
Frequency Frequency

40 Hz sampling 8 Hz sampling
 Simple operations on TS
Plotting
Removing a baseline
Removing a trend
Smoothing
Filtering
Decimation
 Plotting in Matlab
For visual inspection of TS
For publications/talks

plot
sptool
 Data preprocessing I
Removing offset
ts=ts-mean(ts);

1
10

8 0.5

6
0

-0.5
2

0 -1
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
 Data preprocessing I
Removing a baseline
basel=find(t<=0);
ts=ts-mean(ts(basel));

5 4

4 3

3 2

2 1

1 0

0
-1

-1
-2

-2
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 -3
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6
 Data preprocessing II
Removing a trend
ts=detrend(ts);
subtracts best fitting line
detrend can be used to subtract mean: detrend(ts,constant)

5
5
data 1
4
4 linear
data 2
3 3

2 2

1 1

0 0

-1
-1
-2
-2
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6
-3
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6
 Data preprocessing III
Smoothing
ts=filter(ones(1,30)/30,1,ts); %mean filter, moving average
uses zeros at beginning!
=> baseline correction or do not use first 30 samples
7

0
-0.2 0 0.2 0.4 0.6
 Data preprocessing III
introduces a shift! => either correct for it or
ts=filtfilt(ones(1,15)/15,1,ts); %mean filter, forward and reverse
no shift!
filter can take any smoothing kernel (gaussian, etc)

shifted by 15 samples filtfilt

7 4

6 3

5 2

4 1

3 0

2 -1

1 -2

0 -3
-0.2 0 0.2 0.4 0.6 -0.2 0 0.2 0.4 0.6
 Data preprocessing III
Smoothing
ts=medfilt1(ts,30); %median filter, takes into account the shift
uses 0 at beginning and end !

7
4
6

3.5
5

3
4

3 2.5

2 2

1
1.5
0
-0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6
1
-0.2 -0.18 -0.16 -0.14 -0.12 -0.1
 Data preprocessing III
Smoothing
ts=sgolayfilt(ts,3,41); %Savitzky-Golay filter
fits 3rd order polynomial to frames of size 41
good at preserving high frequencies in the data

0
-0.2 0 0.2 0.4 0.6
 Data preprocessing III
Smoothing
compare unsmoothed and smoothed data
check for shift
check beginning (and end) of the smoothed time series
Exercise 1
 Data preprocessing IV
Filtering
FIR-Filter (finite impulse IIR-Filter (infinite impulse
response) response)
stable potentially unstable
high filter order low filter order
usually have linear phase non-linear phase distortion
(phase change is proportional to computationally efficient
frequency)
 Data preprocessing IV
IIR-Filter:
Butterworth
Elliptic
Chebychev Typ 1
Chebychev Typ 2
Bessel

FIR-Filter:
fir1
 Data preprocessing IV
Magnitude Response (dB)

lowpass 0

highpass -50

bandpass -100

bandstop -150
Magnitude (dB)

-200

-250

-300

dB is logarithmic unit -350

0dB = factor of 1 -400

3dB = factor of 2 0 5 10 15 20 25
Frequency (Hz)
30 35 40 45

10dB= factor of 10
5 Hz lowpass
 Data preprocessing IV
Magnitude Response (dB)
lowpass
0

highpass -10

bandpass -20

-30

bandstop
Magnitude (dB)

-40

-50

-60

-70

-80

-90
dB is logarithmic unit
-100
0dB = factor of 1
0 5 10 15 20 25 30 35 40 45
3dB = factor of 2 Frequency (Hz)

10dB= factor of 10
30 Hz highpass
 Data preprocessing IV
Magnitude Response (dB)

lowpass 0

-5
highpass
-10

bandpass -15

Magnitude (dB)
bandstop -20

-25

-30

-35

dB is logarithmic unit -40

0 5 10 15 20 25 30 35 40 45

0dB = factor of 1 Frequency (Hz)

3dB = factor of 2
10dB= factor of 10
2-30 Hz bandpass
 Data preprocessing IV
Magnitude Response (dB)

lowpass
0

highpass
bandpass -5

Magnitude (dB)

bandstop -10

-15

-20

dB is logarithmic unit -25

0dB = factor of 1 0 5 10 15 20 25
Frequency (Hz)
30 35 40 45

3dB = factor of 2
10dB= factor of 10
30-40 Hz bandstop
 Simple design: FIR
[b]=fir1(4,2*4/sf); %4 Hz lowpass
[b]=fir1(4,2*4/sf,high); %4 Hz highpass
[b]=fir1(4,2*[4 10]/sf); %4-10 Hz bandpass
[b]=fir1(4,2*[4 10]/sf,stop); %4-10 Hz bandstop

tsf=filter(b,1,ts);
tsf=filtfilt(b,1,ts); %forward and reverse
 Simple design: IIR
[b,a]=butter(4,2*4/sf); %4 Hz lowpass
[b,a]=butter(4,2*4/sf,high); %4 Hz highpass
[b,a]=butter(4,2*[4 10]/sf); %4-10 Hz bandpass
[b,a]=butter(4,2*[4 10]/sf,stop); %4-10 Hz bandstop

tsf=filter(b,a,ts);
tsf=filtfilt(b,a,ts); %forward and reverse
 Simple Inspection
0
freqz(b,a,100,100);

Magnitude (dB)
sf -100

number of frequencies
-200

-300
0 5 10 15 20 25 30 35 40 45 50
Frequency (Hz)

0
Phase (degrees)

-100

-200

-300

-400
0 5 10 15 20 25 30 35 40 45 50
Frequency (Hz)
 Complex design
fdatool
magnitude response
phase response
impulse response
compare filters
effect of changing filter order
 Filter artifacts
onset transients

filter filtfilt
x 10
-3 1 Hz lowpass (4th order Butterworth) x 10
-3

2 2

1 1

0 0

-1 -1

-2 -2

-3 -3

-4 -4

-5 -5

-6 -6
0 2 4 6 8 0 2 4 6 8
 Filter artifacts
ringing

with artifact
without artifact
-3 -3
x 10 x 10
20 20

15 15

10 10

5 5

0 0

-5 -5
3.5 4 4.5 3.5 4 4.5
 Filter artifacts
ringing

filter, 20 Hz lowpass (12th order Butterworth) filtfilt

-3
x 10 x 10
-3
20
4

15 3

2
10
1
5
0

0 -1

-2
-5
3.5 4 4.5 3.5 4 4.5
 Filter artifacts
beginning and end of filtered ts is distorted
filtering artifacts is dangerous
filtering may change the latency of effects!
filtering may change the phase
 Suggestions
be careful with low frequencies
use low order butterworth forward and reverse (to avoid
phase distortions)
carefully check beginning and end of filtered ts
make sure you dont have artifacts in the data
use surrogate data (filtered noise)
 Data preprocessing V
Decimation
ts=decimate(ts,4);
decimate uses a lowpass filter to avoid aliasing artifacts
Exercises 2-4