Change detection in monitoring time series

Formose - Changes Workshop

Thom Bogaard
Delft University of Technology

Challenge the future 1

 Change detection in monitoring series

Content

What is a time series?

What are we monitoring?
What is a change?
How can we detect the time series changes?

Example for streams/rivers

Example for landslides

Thom Bogaard
Delft University of Technology

Challenge the future 2

 What is a time series?

time series is a sequence of data points, measured

typically at successive points in time spaced at uniform
time intervals

Time series analysis comprises methods for analyzing

time series data in order to extract meaningful statistics
and other characteristics of the data

Challenge the future 3

 Characteristics of time series
Plot of hydrograph
68

67
Grou ndwater level (m)

66

65

64

63

62
0 72 144 216 288 360 432 504 576 648 720 792
Time interval (5 days)

Observation interval = 5 days

Observation frequency = 6 times / month
Challenge the future 4
 Characteristics of time series
Mean: central tendency Variance: variation around mean
1 n 1 n
h = hi 2
s= ( hi - h )
2

n i= 1 n - 1 i= 1
45

44
Groundwater level (m)

43

42

41

40

39

38
0 12 24 36 48 60
Time interval (15 days)

Challenge the future 5

Time scale effect on time series

Challenge the future 6

Time and spatial scale effect on discharge
time series

Challenge the future 7

 Scale effect on rainfall time series
Example of 9 rain gauges Luxembourg

0.9

0.8

0.7

0.6
r(d)

0.5 Daily Rainfall Monthly rainfall

0.4

0.3

0.2

0.1

0
0 10 20 30 40 50

Dis tance (k m)

Challenge the future 8

Decompose a time series

Time series

Trend

Periodicity

Catastrophic event

Noise (random)

Challenge the future 9

How to test periodicity?
Serial correlation

Random

Autoregressive, Markov process

Periodicity

Challenge the future 10

 How to test periodicity?
Serial correlation and confidence limits

1.0

0.8

0.6
Correlogram

0.4

1.0
0.2
0.8
0.0
0.6
Correlogram

-0.2
0.4
-0.4
0 2 4 6 8 10 12 14 16 18 0.2
20 22 24 26 28 30 32 34 36 38 40 42

Time 0.0
lag (15 days)

-0.2

-0.4
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42

Time lag (15 days)

Lower limit Correlation coefficient Upper limit

Challenge the future 11

Correlogram of seasonal time series
1.5

1.0

Seasonal variations (m)

0.5

0.0

-0.5

-1.0

-1.5

-2.0

-2.5
0 72 144 216 288 360 432 504 576 648 720 792

Time (5 days)

1.2

1.0

0.8
Correlogram

0.6

0.4

0.2

0.0

-0.2

-0.4
0 36 72 108 144 180

Time la g (5 dyas)

Lower limit Correlogram Upper limit

Challenge the future 12

What are we monitoring in natural hazards?

Floods: discharge, water level height, bed topography,

z
v

Challenge the future 13

What are we monitoring in natural hazards?

Challenge the future 14

What are we monitoring in natural hazards?

Landslide: displacement, groundwater level, precipitation

Challenge the future 15

What is the difference between a cause
and a trigger?

Challenge the future 16

What is the difference between a cause
and a trigger?

Challenge the future 17

 How to detect a change?

Methods

Visual inspection
Double mass (residual mass)
Statistics
Physical modelling

Challenge the future 18

 How to detect a change?
data quality and extremes

Challenge the future 19

 How to detect a change?
data quality and extremes

Challenge the future 20

 How to detect a change?
data quality and extremes

Challenge the future 21

 How to detect a change?
data quality and extremes

Challenge the future 22

How to detect a change?
Visual inspection

Patterns of movement in reactivated

landslides
Massey ey al, 2013 Engineering Geology

Challenge the future 23

How to detect a change?
Visual inspection

Patterns of movement in reactivated

landslides
Massey ey al, 2013 Engineering Geology

Challenge the future 24

 How to detect a change?

Double mass plot

Plot cumulative observation

time series against another
(averaged) cumulative time Double mass analysis
series 100000

90000

Cumulative discharge ( 1 station)

80000

Double Mass curve 70000

2500
60000

2000 50000

40000
Station 2

1500
30000
1000 20000

500 10000

0
0 0 20000 40000 60000 80000
0 2000 4000 6000 8000
Station 1 Cumulative Average Discharge (n-1 stations)

Challenge the future 25

 How to detect a change?

Double mass plot

Challenge the future 26

 How to detect a change?
Characteristics of Time Series
Stationarity

Stationary: probability distribution doesnt change with

time
First-order stationary: mean is a constant
Second-order stationary: mean is a constant and
covariance is only a function of time lag, not actual time
Non-stationary in the mean: presence of a trend or
periodicity

Challenge the future 27

 Characteristics of Time Series

A stationary time series

-200

-220
Groundwater level (cm)

-240

-260

-280

-300
0 12 24 36 48 60 72 84 96 108 120

Time interval (15 days)

Challenge the future 28

 Characteristics of Time Series

Non-stationary time series with a trend

7
Groundwater level (m)

3
Groundwater level Linear trend
2
0 12 24 36 48 60 72 84 96 108 120

Time interval (15 days)

Challenge the future 29

 Characteristics of Time Series

Non-stationary time series with periodic changes

2
Groundwater level (m)

-1

-2
Groundwater level Seasonal trend
-3
0 12 24 36 48 60 72 84 96 108 120

Time interval (15 days)

Challenge the future 30

 Detection of a trend
Step trend

ht = 1 + ( 2 - 1 )t >n1 + t
45
44
Groundwater level (m)

43
42 1
41 2
40
39
38
37
36
0 12 24 36 48 60 72 84 96 108 120
Time interval (15 days)
Groundwa ter level Step trend

Challenge the future 31

 Detection of a step trend

Hypothesis test
H0 1 = 2
Ha 1 2

t statistic | x1 - x 2 |
t= t(n - 2)
2 sp / n
t test result
Given (5%), find t/2(n-2) from Student table
If t > t/2(n-2) accept Ha, step trend is significant
If t t/2(n-2) accept H0, step trend is not significant

Challenge the future 32

Detection of a step trend
1 60 1 120
h1 = h i 42 .0 h 2 = h i 40 .0
60 i= 1 60 i= 61
1 60 120

hi hi ] 1.1
2 2 2
sp= [ ( - 42.0 ) + ( - 40. 0 )
120 - 2 i=1 i = 61

| 42.0 - 40.0 | t0.025(118) = 1.96

t= = 9.89
2 * 1.1/ 120
45
44
Gro undwater level (m)

43
42
41
40
39
38
37
36
0 12 24 36 48 60 72 84 96 108 120
Time interval (15 days)
Groundwa ter level Step trend

Challenge the future 33

Detection of a linear trend

ht = 0 + 1 t + t
h t = b0 + b1 t
34
Groundwa ter level (m)

32

30

28

26

24

22
0 12 24 36 48 60 72 84 96 108 120
Time i nterval (15 days)
Groundwater level Linear trend

Challenge the future 34

Detection of a linear trend

Hypothesis test
H0 1 = 0
Ha 1 0

t statistic t=
| b1 |
t(n - 2)
12 sl / n(n + 1)(n - 1)

t test result
Given (5%), find t/2(n-2) from Student table
If t > t/2(n-2) accept Ha, linear trend is significant
If t t/2(n-2) accept H0, linear trend is not significant

Challenge the future 35

Detection of a linear trend

Example of detecting a linear trend

( ht - h )(t - t )
b1 = t= 1
n 0.05

2
(t - t )
t =1
34

b0 = h - b1 t 32.0
Groundwater level (m)
32

30

28

26

24

22
0 12 24 36 48 60 72 84 96 108 120
Time interval (15 days)
Groundwater leve l Linear trend

Challenge the future 36

Detection of a linear trend

Example of detecting a linear trend

1 n
ht b0 b1 1.0
2 2
sl= ( - - t )
n-2 t = 1

| -0.05 |
t= = 18.97 t0.025(118) = 1.96
12 * 1.0/ 120 * 121* 119
34
Groundwater level (m)
32

30

28

26

24

22
0 12 24 36 48 60 72 84 96 108 120
Time interval (15 days)
Groundwater level Linear trend

Challenge the future 37

Detection of a periodic trend
Characteristics of a harmonic function

ht = A0 + A s in(2 f t 0 )
A=2 amplitude Hramonic series
3

2
Harmo nic function

A0=0 base level

1

-1

-2
0= 0 initial phase T=1/f =12 period
-3
0 2 4 6 8 10 12 14 16 18 20 22 24

T ime (month)

Challenge the future 38

 Detection of a periodic trend

Fit of harmonic series with 2 harmonics

2t 2t 2t 2t
ht = 49.97 + 2.23 cos( ) + 1.68 sin( ) + 0.86 cos( ) + 0.53 sin( )
24 24 12 12

58
Measur ements Har monic se ries
56
Groundwater level (m)

54

52

50

48

46

44
0 12 24 36 48 60 72 84 96 108 120
Time inte rval (15 days)

Challenge the future 39

 Shifts in the Mean

1. Students t-test 9. Regression-based approach

2. Bayesian analysis 10. CUSUM test
3. MannWhitney U-test 11. Oerlemans method
4. Wilcoxon rank sum 12. Signal-to-noise ratio
5. Pettitt test 13. Intervention analysis
6. Mann-Kendall test 14. Markov chain Monte Carlo
7. Lepage test 15. Lanzante method
8. Standard normal homogeneity test

Challenge the future 40

Shifts in the Variance
1. Downton-Katz test

Shifts in the Spectrum

1. Nikiforov method

Shifts in the System

1. Principal component analysis
2. Average standard deviates
3. Fisher information
4. Vector autoregressive method

Challenge the future 41

Example: Min Tu Assessment of the effects of
climate variability and land use change on
the hydrology of the Meuse river basin
(2006)

Challenge the future 42

Example: Min Tu Assessment of the effects of
climate variability and land use change on
the hydrology of the Meuse river basin
(2006)

Challenge the future 43

Example: Min Tu Assessment of the effects of
climate variability and land use change on
the hydrology of the Meuse river basin
(2006)

Challenge the future 44

Example: Min Tu Assessment of the effects of
climate variability and land use change on
the hydrology of the Meuse river basin
(2006)

Challenge the future 45

Example: Min Tu Assessment of the effects of
climate variability and land use change on
the hydrology of the Meuse river basin
(2006)

Challenge the future 46

Change detection in monitoring time series

Summary

What is a time series?

What are we monitoring?
What is a change?
Step, linear, periodicity, etc

How can we detect the time series changes?

Visual, double mass, statistical, .

Thom Bogaard
Delft University of Technology

Challenge the future 47