JFF 07 08 06 PC 10/26/07 1:21 PM Page 1

Journal of Financial Forecasting (1–22) Volume 1/ Number 2, Fall 2007

1
2
Direction-of-change forecasts based on 3
conditional variance, skewness and kurtosis 4
dynamics: international evidence 5
6
Peter F. Christoffersen 7
Desautels Faculty of Management, McGill University, 1001 Sherbrooke Street West, 8
Montreal, Quebec H3A1G5, Canada; email: peter.christoffersen@mcgill.ca 9
and 10
CREATES, School of Economics and Management, University of Aarhus, 11
Building 1322, DK-8000 AarhusC, Denmark 12
13
14
Francis X. Diebold
15
Department of Economics, University of Pennsylvania, 3718 Locust Walk, Philadelphia,
16
PA1914-6297, USA; email: fdiebold@wharton.upenn.edu
17
and
18
NBER, 1050 Massachusetts Avenue, Cambridge, MA 02138-5398, USA
19
20
Roberto S. Mariano 21
School of Economics and Social Sciences, Singapore Management University,
22
90 Stanford Road, Singapore, 178903; email: rsmariano@smu.edu.sg
23
24
Anthony S. Tay 25
School of Economics and Social Sciences, Singapore Management University, 26
90 Stanford Road, Singapore, 178903; email: anthonytay@smu.edu.sg 27
28
Yiu Kuen Tse 29
School of Economics and Social Sciences, Singapore Management University, 30
90 Stanford Road, Singapore, 178903; email: yktse@smu.edu.sg 31
32
33
Recent theoretical work has revealed a direct connection between asset 34
return volatility forecastability and asset return sign forecastability. This 35
suggests that the pervasive volatility forecastability in equity returns 36
could, through induced sign forecastability, be used to produce direction-of- 37
change forecasts useful for market timing. We attempt to do so in an 38
39
40
This work was supported by FQRSC, IFM2, SSHRC (Canada), the National Science Founda- 41
tion, the Guggenheim Foundation, the Wharton Financial Institutions Center (US) and 42
Wharton – Singapore Management University Research Centre. We thank participants of the 43
9th World Congress of the Econometric Society and numerous seminars for comments, but we
emphasize that any errors remain ours alone.
44N

1
 JFF 07 08 06 PC 10/26/07 1:21 PM Page 2

2 P. F. Christoffersen et al

1 international sample of developed equity markets, with some success, as

2 assessed by formal probability forecast scoring rules such as the Brier
score. An important ingredient is our conditioning not only on conditional
3
mean and variance information, but also on conditional skewness and
4 kurtosis information, when forming direction-of-change forecasts.
5
6
7
8 1 INTRODUCTION
9
10 Recent work by Christoffersen and Diebold (2006) has revealed a direct connec-
11 tion between asset return volatility dependence and asset return sign dependence
12 (and hence sign forecastability). This suggests that the pervasive volatility
13 dependence in equity returns could, through induced sign dependence, be used to
14 produce direction-of-change forecasts useful for market timing.
15 To see this, let Rt be a series of returns and t be the information set available at
16 time t. Pr[Rt  0] is the probability of a positive return at time t. The conditional
17 mean and variance are denoted, respectively, as t1|t  E[Rt1t] and  2t1t 
18 Var[Rt1t]. The return series is said to display conditional mean predictability
19 if t1t varies with t; conditional variance predictability is defined similarly.
20 If Pr[Rt  0] exhibits conditional dependence, ie, Pr[Rt1  0t] varies with t,
21 then we say the return series is sign predictable (or the price series is direction-of-
22 change predictable).
23 For clearer exposition, and to emphasize the role of volatility in return sign pre-
24 dictability, suppose that there is no conditional mean predictability in returns, so
25 t1t   for all t. In contrast, suppose that  2t1t varies with t in a predictable
26 manner, in keeping with the huge literature on volatility predictability reviewed in
27 Andersen et al (2006). Denoting D(, 2) as a generic distribution dependent only
28 on its mean  and variance  2, assume
29
30 Rt1t ~ D(,t1
2
t )
31
32
33 Then the conditional probability of positive return is
34
35 Pr(Rt1  0  t )  1  Pr(Rt1 0  t )

 
36
37 Rt1   
 1  Pr
38 t1 t t1 t

 
39

40 1F (1)
41 t1 t
42
43 where F is the distribution function of the “standardized” return (Rt1t 
44N )/t1t). If the conditional volatility is predictable, then the sign of the return is

Journal of Financial Forecasting Volume 1/ Number 2, Fall 2007

JFF 07 08 06 PC 10/26/07 1:21 PM Page 3

Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics 3

predictable even if the conditional mean is unpredictable, provided 

0. Note 1
also that if the distribution is asymmetric, then the sign can be predictable even 2
if the mean is zero: time-varying skewness can be driving sign prediction in 3
this case. 4
In practice, interaction between mean and volatility can weaken or strengthen 5
the link between conditional volatility predictability and return sign predictability. 6
For instance, time variation in conditional means of the sort documented in recent 7
work by Brandt and Kang (2004) and Lettau and Ludvigson (2005) would 8
strengthen our results. Interaction between volatility and higher-ordered condi- 9
tional moments can similarly affect the potency of conditional volatility as a 10
predictor of return signs. 11
In this paper, we use 12
13

 
14
t1 t
Pr(Rt1  0  t )  1  F (2) 15
t1 t 16
17
to explore the sign predictability of one-, two- and three-month returns in three 18
stock markets,1 in which we examine out-of-sample predictive performance. We 19
also use an extended version of Equation (2) that explicitly considers the interac- 20
tion between volatility and higher-ordered conditional moments. We estimate the 21
parameters of the models recursively and we evaluate the performance of sign 22
probability forecasts. We proceed as follows: in Section 2, we discuss our data and 23
its use for the construction of volatility forecasts; in Section 3, we discuss our 24
direction-of-change forecasting models and evaluation methods; in Section 4, we 25
present our empirical results; and in Section 5, we conclude. 26
27
2 DATA AND VOLATILITY FORECASTS 28
29
Estimates and forecasts of realized volatility are central to our analysis; for back- 30
ground, see Andersen et al (2003, 2006). Daily values for the period 1980:01 to 31
2004:06 of the MSCI index for Hong Kong, UK and US were collected from 32
Datastream. From these, we constructed one-, two- and three-month returns and 33
realized volatility. The latter is computed as the sum of squared daily returns 34
within each one-, two- and three-month period. We use data from 1980:01 to 35
1993:12 as the starting estimation sample, which will be recursively expanded as 36
more data becomes available. We reserve the period 1994:01 to 2004:06 for our 37
forecasting application. 38
Tables 1 and 2 summarize some descriptive statistics of the return and the log 39
of the square root of realized volatility (hereafter “log realized volatility”) for the 40
three markets. All markets have low positive mean returns for the period (see 41
42
1
The same analysis extended to 20 stock markets is available from the authors upon request. 43
The results for the three markets shown are representative for the 20 markets. 44N

Research Paper www.journaloffinancialforecasting.com

 JFF 07 08 06 PC 10/26/07 1:21 PM Page 4

4 P. F. Christoffersen et al

1 Table 1). Returns have negative skewness and are leptokurtic at all three frequen-
2 cies. The p-values of the Jarque–Bera statistics indicate non-normality of all
3 returns series. Log realized volatility is positively skewed and slightly leptokurtic
4 (see Table 2). As with the returns series, the p-values of the Jarque–Bera statistics
5 indicate non-normality of all volatility series.
6 Figure 1 presents the plots of the log realized volatilities. There appears to be
7 some clustering of return volatility. Predictability of volatility is indicated by the
8
9
10 TABLE 1 Summary statistics of the full sample of returns, 1980:01–2004:06.
11
12 Mean Standard Skewness Kurtosis Jarque–Bera
13 deviation p-value
14 Hong Kong
15 1 month 0.008 0.091 –1.029 8.902 0.000
16 2 months 0.016 0.125 –0.422 5.046 0.000
17 3 months 0.025 0.164 –0.684 3.712 0.011
UK
18 1 month 0.008 0.049 –1.228 8.236 0.000
19 2 months 0.015 0.064 –0.611 4.186 0.000
20 3 months 0.023 0.085 –1.047 5.254 0.000
21 US
22 1 month 0.008 0.045 –0.841 6.124 0.000
2 months 0.016 0.058 –0.884 5.875 0.000
23
3 months 0.024 0.082 –0.799 4.237 0.000
24
25 Returns are in percent per time interval (one month, two months or one quarter, not annualized).
26
27
28 TABLE 2 Summary statistics of the full sample of realized volatility, 1980:01–
29 2004:06.
30
Mean Standard Skewness Kurtosis Jarque–Bera
31 deviation p-value
32
33 Hong Kong
1 month –2.737 0.451 0.682 3.835 0.000
34
2 months –2.357 0.427 0.719 3.659 0.001
35 3 months –2.126 0.405 0.727 3.437 0.011
36 UK
37 1 month –3.213 0.362 0.821 4.677 0.000
38 2 months –2.843 0.341 0.909 4.496 0.000
39 3 months –2.626 0.323 0.936 4.799 0.000
40 US
41 1 month –3.226 0.400 0.566 4.619 0.000
2 months –2.851 0.377 0.658 4.476 0.000
42
3 months –2.637 0.367 0.679 4.523 0.000
43
44N “Volatility” refers to log of the square root of realized volatility computed from daily returns.

Journal of Financial Forecasting Volume 1/ Number 2, Fall 2007

JFF 07 08 06 PC 10/26/07 1:21 PM Page 5

Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics 5

FIGURE 1 Realized volatility. 1

2
3
One month Two months Three months
0 0 0
4
Volatility
(Hong Kong) −2 −2 −2 5
−4 −4 −4 6
1980:01 1993:12 2004:06 1980:01 1993:06 2004:03 1980:01 1993:04 2004:02
7
1 1 1
Volatility ACF 0
8
0 0
(Hong Kong) 9
−1 −1 −1
0 10 20 0 10 20 0 10 20 10
One month Two months Three months
0 0 0 11
Volatility −2 −2 −2 12
(UK) −4
−4 −4 13
1980:01 1993:12 2004:06 1980:01 1993:06 2004:03 1980:01 1993:04 2004:02
14
1 1 1
Volatility ACF 0 0 0
15
(UK)
−1 −1 −1 16
0 10 20 0 10 20 0 10 20
One month Two months Three months
17
0 0 0 18
Volatility −2 −2 −2
(US) −4
19
−4 −4
1980:01 1993:12 2004:06 1980:01 1993:06 2004:03 1980:01 1993:04 2004:02 20
1 1 1 21
Volatility ACF 0 0 0 22
(US) −1
0 10 20
−1
0 10 20
−1
0 10 20
23
24
“Volatility” refers to the log of the square root of realized volatility constructed from daily returns.
25
26
27
corresponding correlograms. As we move from the monthly frequency to the
28
quarterly frequency, the autocorrelations diminish but still indicate predictability.
29
The correlograms (none of which are reported here) of the return series of the
30
indexes show that they are all serially uncorrelated.
31
Our method for forecasting the probability of positive returns will require fore-
32
casts of volatility, which we discuss here. We use the data from 1980:01 to
33
1993:12 as the base estimation sample. Out-of-sample one-step-ahead forecasts
34
are generated for the period 1994:01 to 2004:06, with recursive updating of
35
parameter estimates, ie, a volatility forecast for period t  1 made at time t will
36
use a model estimated over the period 1980:1 to t. In addition, we also choose our
37
models recursively: at each period, we select ARMA models for log-volatility by
38
minimizing the Akaike information criterion (AIC).2
39
40
2
41
We repeated the analysis using the SIC criterion, but because the subsequent probability fore-
casts generated by the Schwarz information criterion (SIC), and the corresponding evaluation
42
results, are very similar to the AIC results, we report results only for the models selected by the 43
AIC. The SIC results are available from the authors upon request. 44N

Research Paper www.journaloffinancialforecasting.com

 JFF 07 08 06 PC 10/26/07 1:21 PM Page 6

6 P. F. Christoffersen et al

1 Broadly speaking, the AIC favors ARMA(1,1) models, particular at the

2 monthly frequency. In Figure 2, we display the volatility forecasts (with actual log
3 realized volatilities included for comparison) for the three markets. The forecasts
4 generated by the AIC track actual log realized volatility fairly reasonably. The
5 ratios of the mean square prediction errors (MSPEs) to the sample variance of log
6 realized volatility are given in Table 3. The forecasts capture a substantial amount
7 of the variation in actual log realized volatility.
8
9 FIGURE 2 Realized volatility and recursive realized volatility forecasts.
10
11
12 Hong Kong (monthly) Hong Kong (two months) Hong Kong (quarterly)
−1 −1 −1
13
Log realized volatility

−1.5
−1.5
14 −2
−2
15 −2
−2.5
−3
16 −2.5
−3
17
−4 −3.5 −3
18 1994:01 1999:01 2004:06 1994:01 1999:01 2004:03 1994:01 1999:01 2004:02

19 UK (monthly) UK (two months) UK (quarterly)

−1 −1.5 −1.5
20
Log realized volatility

21 −2 −2 −2

22 −3 −2.5 −2.5

23 −4 −3 −3
24
−5 −3.5 −3.5
25 1994:01 1999:01 2004:06 1994:01 1999:01 2004:03 1994:01 1999:01 2004:02

26 US (monthly) US (two months) US (quarterly)

−2 −1.5 −1.5
27
Log realized volatility

−2.5 −2
28 −2
−3 −2.5
29 −2.5
−3.5 −3
30
−3
−4 −3.5
31
−4.5 −4 −3.5
32 1994:01 1999:01 2004:06 1994:01 1999:01 2004:03 1994:01 1999:01 2004:02
33 Actual Fcst

34 “Volatility” refers to log of the square root of realized volatility constructed from daily returns. “Fcst” is the
35 one-step-ahead forecasts of volatility generated from recursively estimated ARMA models chosen recur-
36 sively using the AIC criterion.

37
TABLE 3 Ratio of mean square prediction error (MSPE) of forecasts to sample
38
variance, realized volatility.
39
40 Hong Kong UK US
41
42 1 month 0.479 0.648 0.537
2 months 0.632 0.756 0.592
43
3 months 0.575 0.855 0.563
44N

Journal of Financial Forecasting Volume 1/ Number 2, Fall 2007

JFF 07 08 06 PC 10/26/07 1:21 PM Page 7

Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics 7

A comment on our notation: throughout this paper, we use ˆ t to represent the 1

square root of realized volatility. The symbol ˆ t1t will represent the period t 2
forecast of the square root of period t  1 realized volatility. Note also that our 3
volatility forecasting models use (and forecast) the log of these objects, so that 4
ˆ t1t actually represents the exponent of the forecasts of (log) realized volatil- 5
ity. Finally, for simplicity of notation, we will also write Pr[Rt1t  0] for 6
Pr[Rt1  0t]. 7
8
3 FORECASTING MODELS AND EVALUATION METHODS 9
10
3.1 Forecasting models 11
We will evaluate the forecasting performance of two sets of forecasts and compare 12
their performance against forecasts from a baseline model. Our baseline forecasts 13
are generated using the empirical cumulative distribution function (cdf) of the Rt 14
using data from the beginning of our sample period right up to the time the fore- 15
cast is made, ie, at period k, we compute 16
17
k
ˆ (R   0)  1 18
Pr k1 k
k t1
 I(Rt  0) (3)
19
20
where I(·) is the indicator function. 21
Our first forecasting model makes direct use of Equation (2). Using all avail- 22
able data at time k, we first regress Rt on a constant, log(ˆ t ), and [log(ˆ t )]2, and 23
compute 24
25
ˆ t  ˆ 0  ˆ 1log (ˆ t )  ˆ 2 [log(ˆ t ) ] 2, t  1,...,k (4) 26
27
where ˆ t is the square root of (actual, not forecasted) realized volatility. The 28
period k1 forecast is then generated by 29
30
ˆ
Pr(Rk1 k  0)  1  F̂   ˆ k1 k
ˆ k1 k  31
32

 
33
1 k Rt  ˆ t ˆ k1 k
1
k 
t1
I
ˆ t

ˆ k1 k
(5) 34
35
36
ie, F̂ is the empirical cdf of (Rt  ˆ t)/ˆ t . The one-step-ahead volatility forecast
37
ˆ t1t is generated from a recursively estimated model selected, at each period, by
38
minimizing the AIC, as described in the previous section. The one-step-ahead
39
mean forecast ˆ t1t is computed as
40
41
ˆ t1 t  ˆ 0  ˆ 1log (ˆ t1 t )  ˆ 2 [log(ˆ t1 t ) ] 2 (6) 42
43
The coefficients ˆ0, ˆ1 and ˆ2 are recursively estimated using Equation (4). 44N

Research Paper www.journaloffinancialforecasting.com

 JFF 07 08 06 PC 10/26/07 1:21 PM Page 8

8 P. F. Christoffersen et al

1 A linear relationship between the return mean and the time-varying log return
2 volatility was first found in the seminal ARCH-in-Mean work of Engle et al
3 (1987). A quadratic return mean specification is used here as the quadratic term is
4 significant for almost all series in the starting estimation sample. Although the
5 coefficients are recursively estimated, at each recursion no attempt is made to
6 refine the model. We refer to forecasts from Equation (5) as non-parametric fore-
7 casts (even though the realized volatility forecasts are generated using fully para-
8 metric models) to differentiate it from forecasts from our next model.
9 The second model is an extension of Equation (1) and explicitly considers the
10 interaction between volatility, skewness and kurtosis. This is done by using the
11 Gram–Charlier expansion:
12
13
14 1F  t1 t
t1 t1  
t1 t
t1 t

     
15
16 t1t 3,t1t t1
2
t 4,t1t 3t1 t 3t1 t
 1  
17 t1t 3! t1
2
t 4! t1
3
t t1 t
18
19 where (·) is the distribution function of a standard normal, and 3 and 4 are,
20 respectively, the skewness and excess kurtosis, with the usual notation for condi-
21 tioning on t. This equation can be rewritten as
22
23 1  F(t1t xt1 )  1  (t1t xt1 )
24
(0t  1t xt1  2 t xt1
2
 3t x3t1 )
25
26
27 with 0t  1  3,t1t /6, 1t  4,t1t t1t /8, 2t  3,t1t 2t1t /6 and 3t 
28 4,t1t 3t1t /24, where for notational convenience, we denote xt1  1/t1t.
29 Several points should be noted. The sign of returns is predictable for non-zero
30 t1t even when there is no volatility clustering, as long as the skewness and kur-
31 tosis are time varying. On the other hand, even if t1t is zero, sign predictability
32 arises as long as conditional skewness dynamics is present, regardless of whether
33 volatility dynamics is present. If there is no conditional skewness and excess kur-
34 tosis, the above equation is reduced to
35
36 1  F(t1t xt1 )  1  (t1t xt1 )
37
38 so that normal approximation applies. If returns are conditionally symmetric but
39 leptokurtic (ie, 3,t1t  0 and 4,t1t  0), then 0t  1 and 2t  0, and we have
40
41 1  F(t1t xt1 )  1  (t1t xt1 )(1  1t xt1  3t x3t1 )
42
43 Furthermore, if t1t  0, we have 1t  0 and 1t  0, and the converse is true
44N for t1t  0. Finally, if t1t is small, as in the case of short investment horizons,

Journal of Financial Forecasting Volume 1/ Number 2, Fall 2007

JFF 07 08 06 PC 10/26/07 1:21 PM Page 9

Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics 9

then 2t and 3t can safely be ignored, resulting in 1

2
1  F(t1t xt1 )  1  (t1t xt1 )(0t  1t xt1 ) 3
4
5
Thus, conditional skewness affects sign predictability through 0t, and conditional
6
kurtosis affects sign predictability through 1t. When there is no conditional
7
dynamics in skewness and kurtosis, the above equation is reduced to
8
9
1  F(t1t xt1 )
1  (t1t xt1 )(0  1 xt1 ) (7)
10
11
for some time-invariant quantities 0 and 1. 12
We use Equation (7) as our second model for sign prediction, ie, we generate 13
forecasts of the probability of positive returns as 14
15
ˆ
t1t  0)  1  (t1 t x̂t1 )(0  1 x̂t1 )
Pr(R ˆ ˆ ˆ (8) 16
17
where x̂ t1t  1/ˆ t1t , and where ˆ t1t and ˆ tt1t are as defined earlier. We refer 18
to these as forecasts from the “extended” model. The parameters 0 and 1 19
are estimated by regressing 1  I(Rt  0) on (ˆ t x̂ t) and (ˆ t x̂ t)x̂ t for 20
t  1,…,k. Although we have not explicitly placed any constraints on this model 21
to require (ˆ t x̂ t)(ˆ0 ˆ1x̂ t) to lie between 0 and 1, this was inconsequential as 22
all our predicted probabilities turn out to lie between 0 and 1. 23
24
3.2 Forecast evaluation 25
26
We perform post-sample comparison of the forecast performance of Equations (5) 27
and (8) for the sign of return. Both are compared against baseline forecasts 28
[Equation (3)]. This is done for one-, two- and three-month returns. We assess the 29
performance of the forecasting models using Brier scores; for background, see 30
Diebold and Lopez (1996). 31
Two Brier scores are used: 32
33
T
1
Brier(Abs) 
Tk 
tk
 Pr(R
ˆ
t1 t  0)  zt1
34
35
T 36
1
Brier(Sq) 
Tk  ˆ
2 ( Pr(Rt1 t  0)  zt1 )
2 37
tk 38
39
where zt1  I(Rt1  0). The latter is the traditional Brier score for evaluating the 40
performance of probability forecasts and is analogous to the usual MSPE. A score 41
of 0 for Brier(Sq) occurs when perfect forecasts are made: where at each period, 42
correct probability forecasts of 0 or 1 are made. The worst score is 2 and occurs if 43
at each period probability forecasts of 0 or 1 are made but turn out to be wrong 44N

Research Paper www.journaloffinancialforecasting.com

 JFF 07 08 06 PC 10/26/07 1:21 PM Page 10

10 P. F. Christoffersen et al

1 each time. Note that if we follow the usual convention where a correct probability
2 forecast of I(Rt1  0) is 1 that is greater than 0.5, then correct forecasts will have
3 an individual Brier(Sq) score between 0 and 0.5, whereas incorrect forecasts have
4 individual scores between 0.5 and 2. A few incorrect forecasts can therefore dom-
5 inate a majority of correct forecasts.
6 For this reason, we also consider a modified version of the Brier score, which
7 we call Brier(Abs). Like Brier(Sq), the best possible score for Brier(Abs) is 0. The
8 worst score is 1. Here correct forecasts have individual scores between 0 and 0.5,
9 whereas incorrect forecasts carry scores between 0.5 and 1.
10
11 4 EMPIRICAL RESULTS
12
Figures 3–5 show, for the Hong Kong, UK and US markets, respectively, the pre-
13
dicted probabilities generated by the baseline model, the non-parametric model
14
15
16 FIGURE 3 Predicted probabilities (Hong Kong).
17
18
19 Baseline (monthly) Non-parametric (monthly) Extended (monthly)
Predicted probability

1 1 1
20
21
22 0.5 0.5 0.5
23
24
0 0 0
25 1994:01 1999:01 2004:06 1994:01 1999:01 2004:06 1994:01 1999:01 2004:06
26 Baseline (two months) Non-parametric (two months) Extended (two months)
27
Predicted probability

1 1 1
28
29
0.5 0.5 0.5
30
31
32 0 0 0
33 1994:01 1999:01 2004:03 1994:01 1999:01 2004:03 1994:01 1999:01 2004:03
34 Baseline (quarterly) Non-parametric (quarterly) Extended (quarterly)
Predicted probability

1 1 1
35
36
37 0.5 0.5 0.5
38
39
40 0 0 0
1994:01 1999:01 2004:02 1994:01 1999:01 2004:02 1994:01 1999:01 2004:02
41
42 ˆ(R
“Nonparametric” forecasts (second column) refer to forecasts generated using Pr t1  0) 
43 ˆ ˆ
1  F( t1t),where F̂ is the empirical cdf of (Rt  ˆ t )/ ˆ t. “Extended” (third column) refers to
t1t/ˆ
44N forecasts generated from the extended model Pr ˆ(R  0) 1(ˆ x̂ )(ˆ  ˆ x̂ ).
t1 t t1 0 1 t1

Journal of Financial Forecasting Volume 1/ Number 2, Fall 2007

JFF 07 08 06 PC 10/26/07 1:21 PM Page 11

Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics 11

FIGURE 4 Predicted probabilities (UK). 1

2
3
Baseline (monthly) Non-parametric (monthly) Extended (monthly) 4
Predicted probability

1 1 1
5
6
0.5 0.5 0.5 7
8
9
0 0 0
1994:01 1999:01 2004:06 1994:01 1999:01 2004:06 1994:01 1999:01 2004:06 10
Baseline (two months) Non-parametric (two months) Extended (two months)
11
12
Predicted probability

1 1 1
13
14
0.5 0.5 0.5
15
16
0 0 0 17
1994:01 1999:01 2004:03 1994:01 1999:01 2004:03 1994:01 1999:01 2004:03 18
Baseline (quarterly) Non-parametric (quarterly) Extended (quarterly) 19
Predicted probability

1 1 1 20
21
0.5 0.5 0.5 22
23
24
0 0 0 25
1994:01 1999:01 2004:02 1994:01 1999:01 2004:02 1994:01 1999:01 2004:02
26
See Figure 3 footnote. 27
28
and the extended model (columns 1, 2 and 3, respectively) for the monthly, two- 29
month and quarterly frequencies (rows 1, 2 and 3, respectively). For the non- 30
parametric and extended models, forecasts based on the AIC volatility forecasts 31
are plotted. For all three markets, the baseline forecasts are very flat, at values 32
slightly above 0.5. The non-parametric and extended forecasts show more vari- 33
ability, especially in the later periods. 34
Before reporting our main results, we highlight some interesting regularities in 35
the Brier scores. In Table 4, we report the mean and standard deviation of the 36
Brier(Abs) scores from the AIC-based probability forecasts for the Hong Kong, 37
UK and US markets. Results are reported for three “subperiods”. The ‘low’ 38
volatility subperiod comprises all dates for which realized volatility falls in the 1st 39
to 33rd percentile range. The ‘medium’ and ‘high’ volatility subperiods comprise 40
all dates for which realized volatility falls in the 34–66th and 67–100th percentile 41
ranges, respectively. In all three markets, the Brier score for the low volatility sub- 42
period is lower than the corresponding Brier score for the high volatility subpe- 43
riod. In contrast, the standard deviations of the Brier scores for the non-parametric 44N

Research Paper www.journaloffinancialforecasting.com

 JFF 07 08 06 PC 10/26/07 1:21 PM Page 12

12 P. F. Christoffersen et al

1 FIGURE 5 Predicted probabilities (US).

2
3
4 Baseline (monthly) Non-parametric (monthly) Extended (monthly)
Predicted probability

1 1 1
5
6
7 0.5 0.5 0.5
8
9
0 0 0
10 1994:01 1999:01 2004:06 1994:01 1999:01 2004:06 1994:01 1999:01 2004:06
11
Baseline (two months) Non-parametric (two months) Extended (two months)
12
Predicted probability

1 1 1
13
14
0.5 0.5 0.5
15
16
17 0 0 0
18 1994:01 1999:01 2004:03 1994:01 1999:01 2004:03 1994:01 1999:01 2004:03

19 Baseline (quarterly) Non-parametric (quarterly) Extended (quarterly)

1 1 1
Predicted probability

20
21
22 0.5 0.5 0.5
23
24
25 0 0 0
1994:01 1999:01 2004:02 1994:01 1999:01 2004:02 1994:01 1999:01 2004:02
26
27 See Figure 3 footnote.
28
29 and extended models are higher in the low volatility subperiods than in the high
30 volatility subperiods. For instance, the mean Brier score for the extended model in
31 the US market at the monthly frequency is 0.378 in the low volatility subperiod
32 and 0.532 in the high volatility subperiod. The standard deviation of the same
33 Brier scores falls from 0.143 in the low volatility subperiod to 0.088 in the high
34 volatility subperiod. These findings seem perfectly reasonable: we should expect
35 our models to have more to say in subperiods of low volatility and little to say in
36 subperiods of high volatility. In high volatility subperiods, the models tend to gen-
37 erate probability forecasts that are close to 0.5. The corresponding Brier scores in
38 turn tend to be close to 0.5, resulting in the lower standard deviation of Brier
39 scores in high volatility subperiods.
40 Our main results are reported in Tables 5–8. Table 5 contains our results for
41 the full forecast period. Tables 6–8 contain the results for the low, medium
42 and high volatility subperiods, respectively. In all four tables, both Brier(Abs)
43 and Brier(Sq) are given for the baseline model. The Brier scores for the non-
44N parametric and extended models are expressed relative to the baseline Brier

Journal of Financial Forecasting Volume 1/ Number 2, Fall 2007

 JFF 07 08 06 PC

TABLE 4 Forecast performance, Brier(Abs), three markets.

10/26/07

Baseline Nonparametric Extended

Research Paper
Mean Standard Mean Standard Mean Standard
deviation deviation deviation
1:21 PM

Hong Kong
Low volatility 0.507 0.075 0.510 0.208 0.504 0.239
1 month Medium volatility 0.491 0.070 0.475 0.168 0.475 0.182
Page 13

High volatility 0.520 0.076 0.537 0.137 0.535 0.151

Low volatility 0.512 0.126 0.518 0.298 0.512 0.265
2 months Medium volatility 0.487 0.121 0.487 0.230 0.488 0.203
High volatility 0.579 0.107 0.640 0.173 0.622 0.145
Low volatility 0.435 0.132 0.390 0.241 0.402 0.223
3 months Medium volatility 0.515 0.144 0.526 0.256 0.524 0.222
High volatility 0.551 0.154 0.548 0.202 0.534 0.159
UK
Low volatility 0.430 0.137 0.418 0.164 0.388 0.224
1 month Medium volatility 0.507 0.157 0.516 0.165 0.519 0.215
High volatility 0.500 0.151 0.522 0.123 0.531 0.153
Low volatility 0.398 0.106 0.362 0.147 0.333 0.181
2 months Medium volatility 0.511 0.160 0.526 0.178 0.546 0.193
High volatility 0.517 0.160 0.514 0.154 0.518 0.158
Low volatility 0.343 0.156 0.308 0.204 0.288 0.226
3 months Medium volatility 0.470 0.220 0.484 0.247 0.491 0.254
High volatility 0.566 0.201 0.638 0.173 0.638 0.162
Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics

www.journaloffinancialforecasting.com
13

44N
43
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
 9
8
7
6
5
4
3
2
1

43
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10

44N
14
JFF 07 08 06 PC

TABLE 4 Continued.
10/26/07

P. F. Christoffersen et al

Baseline Nonparametric Extended

Mean Standard Mean Standard Mean Standard

1:21 PM

deviation deviation deviation

Journal of Financial Forecasting

US
Low volatility 0.419 0.093 0.404 0.104 0.378 0.143
Page 14

1 month Medium volatility 0.468 0.129 0.488 0.127 0.489 0.144

High volatility 0.525 0.130 0.536 0.088 0.532 0.088
Low volatility 0.417 0.109 0.412 0.134 0.375 0.196
2 months Medium volatility 0.451 0.149 0.466 0.125 0.461 0.157
High volatility 0.518 0.165 0.499 0.107 0.502 0.131
Low volatility 0.433 0.162 0.405 0.172 0.379 0.231
3 months Medium volatility 0.366 0.143 0.380 0.139 0.364 0.158
High volatility 0.524 0.199 0.529 0.158 0.528 0.165

Brier(Abs)  1/T  Ttkpt1 t  zt1, where k is the start of the estimation sample, pt1t is the one-step-ahead forecast of Pr(Rt1  0) made at time t and zt1  I(Rt1  0).
At each time t, data from the first period up to time t is used to estimate the forecasting model. “Baseline” refers to forecasts generated from the unconditional empirical distribution
ˆ(R ˆ ˆ
of Rt. “Non-parametric” refers to forecasts generated using Pr t1 t  0)  1  F(t1t/ˆ t1t) , where F̂ is the empirical cdf of (R t  ˆ t) /ˆt.“Extended” refers to forecasts
ˆ
generated from Pr(Rt1t  0)  1  ( ˆ ˆ
ˆ t1t x̂t1)(0  1 x̂t1) .

Volume 1/ Number 2, Fall 2007

 JFF 07 08 06 PC
10/26/07

TABLE 5 Relative forecast performance (full sample).

Research Paper
Number of forecasts Brier(Abs) Brier(Sq)
1:21 PM

Baseline Non-parametric Extended Baseline Non-parametric Extended

Hong Kong
1 month 126 0.506 1.005 1.001 0.524 1.102 1.123
Page 15

2 months 63 0.526 1.043 1.028 0.584 1.231 1.158

3 months 42 0.500 0.975 0.973 0.544 1.081 1.027
UK
1 month 126 0.482 1.013 1.003 0.510 1.031 1.084
2 months 63 0.477 0.984 0.981 0.500 0.999 1.031
3 months 42 0.460 1.036 1.027 0.510 1.124 1.129
US
1 month 126 0.469 1.011 0.991 0.471 1.015 1.003
2 months 63 0.462 0.994 0.967 0.470 0.964 0.969
3 months 42 0.441 0.993 0.961 0.450 0.972 0.966

Brier(Abs)  1/T  Ttkpt1 t  zt1 and Brier(Sq)  1/T  Ttk2(pt1 t  zt1) 2, where k is the start of the estimation sample, pt+1t is the one-step-ahead
forecast of Pr(Rt1  0) made at time t and zt1  I(Rt1  0). At each time t, data from the first period up to time t is used to estimate the forecasting model. “Baseline” refer to
ˆ(R ˆ ˆ
forecasts generated from the unconditional empirical distribution of Rt. “Non-parametric” refers to forecasts generated using Pr t1 t  0)  1  F(t1t/ˆ t1t) , where
ˆF is the empirical cdf of (R   ˆ ˆ ˆ
t
ˆ t) /ˆt.“Extended” refers to forecasts generated from Pr(Rt1 t  0)  1  ˆ t1t x̂t1 0  1 x̂t1 . Actual Brier scores are reported for the
  
baseline forecasts. All other scores are Brier scores for the given model divided by the Brier score for the baseline forecast. Ratios below 1 are in bold print.
Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics

www.journaloffinancialforecasting.com
15

44N
43
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
 9
8
7
6
5
4
3
2
1

43
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10

44N
16
JFF 07 08 06 PC
10/26/07

TABLE 6 Forecast performance, low volatility periods (1st to 33rd percentile).

P. F. Christoffersen et al

Number of forecasts Brier(Abs) Brier(Sq)

1:21 PM

Baseline Non-parametric Extended Baseline Non-parametric Extended

Journal of Financial Forecasting

Hong Kong
1 month 42 0.507 1.004 0.993 0.526 1.148 1.178
2 months 21 0.512 1.011 1.000 0.555 1.269 1.186
Page 16

3 months 14 0.435 0.896 0.923 0.411 1.002 1.010

UK
1 month 42 0.430 0.972 0.904 0.406 0.990 0.985
2 months 21 0.398 0.909 0.836 0.338 0.896 0.839
3 months 14 0.343 0.897 0.839 0.281 0.950 0.929
US
1 month 42 0.419 0.964 0.903 0.368 0.944 0.887
2 months 21 0.417 0.989 0.899 0.370 1.011 0.956
3 months 14 0.433 0.934 0.875 0.423 0.902 0.911

See Table 5 footnote.

Volume 1/ Number 2, Fall 2007

 JFF 07 08 06 PC
10/26/07

Research Paper
TABLE 7 Forecast performance, medium volatility periods (34–66th percentile).
1:21 PM

Number of forecasts Brier(Abs) Brier(Sq)

Baseline Non-parametric Extended Baseline Non-parametric Extended

Page 17

Hong Kong
1 month 42 0.491 0.967 0.966 0.492 1.029 1.046
2 months 21 0.487 1.000 1.002 0.502 1.144 1.104
3 months 14 0.515 1.022 1.019 0.568 1.187 1.129
UK
1 month 42 0.507 1.017 1.023 0.563 1.040 1.116
2 months 21 0.511 1.028 1.068 0.572 1.072 1.168
3 months 14 0.470 1.029 1.045 0.532 1.093 1.133
US
1 month 42 0.468 1.043 1.047 0.470 1.080 1.106
2 months 21 0.451 1.032 1.023 0.449 1.031 1.052
3 months 14 0.366 1.040 0.996 0.306 1.065 1.021

See Table 5 footnote.

Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics

www.journaloffinancialforecasting.com
17

44N
43
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
 9
8
7
6
5
4
3
2
1

43
42
41
40
39
38
37
36
35
34
33
32
31
30
29
28
27
26
25
24
23
22
21
20
19
18
17
16
15
14
13
12
11
10

44N
18
JFF 07 08 06 PC
10/26/07

TABLE 8 Forecast performance, high volatility periods (66th–100th percentile).

P. F. Christoffersen et al

Number of forecasts Brier(Abs) Brier(Sq)

1:21 PM

Baseline Non-parametric Extended Baseline Non-parametric Extended

Journal of Financial Forecasting

Hong Kong
1 month 42 0.520 1.032 1.029 0.552 1.110 1.118
Page 18

2 months 21 0.579 1.105 1.075 0.692 1.265 1.176

3 months 14 0.551 0.995 0.970 0.652 1.040 0.949
UK
1 month 42 0.500 1.044 1.063 0.544 1.053 1.121
2 months 21 0.517 0.995 1.002 0.583 0.984 1.002
3 months 14 0.566 1.127 1.126 0.717 1.214 1.204
US
1 month 42 0.525 1.021 1.015 0.583 1.009 0.997
2 months 21 0.518 0.964 0.969 0.588 0.885 0.912
3 months 14 0.524 1.009 1.008 0.622 0.974 0.977

See Table 5 footnote.

Volume 1/ Number 2, Fall 2007

JFF 07 08 06 PC 10/26/07 1:21 PM Page 19

Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics 19

scores. A relative measure of less than 1 therefore implies improvement in fore- 1

cast performance. 2
Table 5 reports improvement in the performance of the non-parametric or 3
extended models over the baseline forecasts in half of the cases considered, when 4
using Brier(Abs) as a measure of performance. All of the improvements, however, 5
are very small. The situation is usually worse when the forecasts are evaluated 6
using Brier(Sq) instead. 7
The fact that the non-parametric and extended models perform better during 8
low volatility subperiods than during high volatility subperiods suggests that their 9
performance relative to the baseline model might be better during low volatility 10
subperiods than during high volatility subperiods. This is verified by the relative 11
performances reported in Tables 6–8. In Table 6, the improvements are wide- 12
spread and sometimes large. In a number of cases, the ratio of the Brier(Abs) 13
scores for the extended/parametric models to the baseline model is less than 0.9. 14
15
16
FIGURE 6 Comparative Brier(Abs) scores, low volatility (nonparametric versus 17
baseline). 18
19
20
Monthly Two months Quarterly
1 Right to wrong Both wrong 1 Right to wrong Both wrong 1 Right to wrong Both wrong 21
22
Hong Kong
23
0.5 0.5 0.5
24
25
0
Both right Wrong to right
0
Both right Wrong to right
0
Both right Wrong to right
26
0 0.5 1 0 0.5 1 0 0.5 1
27
Monthly Two months Quarterly
1 Right to wrong Both wrong 1 Right to wrong Both wrong 1 Right to wrong Both wrong 28
29
UK
30
0.5 0.5 0.5
31
32
0
Both right Wrong to right
0
Both right Wrong to right
0
Both right Wrong to right 33
0 0.5 1 0 0.5 1 0 0.5 1
34
Monthly Two months Quarterly
1 Right to wrong Both wrong 1 Right to wrong Both wrong 1 Right to wrong Both wrong
35
36
37
US
0.5 0.5 0.5
38
39
0
Both right Wrong to right
0
Both right Wrong to right
0
Both right Wrong to right 40
0 0.5 1 0 0.5 1 0 0.5 1
41
42
The horizontal axis measures individual Brier(Abs) scores for baseline forecasts. The vertical axis measures
corresponding Brier(Abs) scores for the nonparametric forecasts. A score below 0.5 indicates a correct fore-
43
cast. Only observations with volatility in the 1st to 33rd percentile range are included. 44N

Research Paper www.journaloffinancialforecasting.com

 JFF 07 08 06 PC 10/26/07 1:21 PM Page 20

20 P. F. Christoffersen et al

1 FIGURE 7 Comparative Brier(Abs) scores, low volatility (extended model versus

2 baseline).
3
4
Monthly Two months Quarterly
5 1 Right to wrong Both wrong 1 Right to wrong Both wrong 1 Right to wrong Both wrong
6
7 Hong Kong
0.5 0.5 0.5
8
9
10 0
Both right Wrong to right
0
Both right Wrong to right
0
Both right Wrong to right
0 0.5 1 0 0.5 1 0 0.5 1
11
Monthly Two months Quarterly
12 1 Right to wrong Both wrong 1 Right to wrong Both wrong 1 Right to wrong Both wrong
13
14 UK
0.5 0.5 0.5
15
16
17 0
Both right Wrong to right
0
Both right Wrong to right
0
Both right Wrong to right
0 0.5 1 0 0.5 1 0 0.5 1
18
Monthly Two months Quarterly
19 1 Right to wrong Both wrong 1 Right to wrong Both wrong 1 Right to wrong Both wrong
20
21
US
0.5 0.5 0.5
22
23
24 0
Both right Wrong to right
0
Both right Wrong to right
0
Both right Wrong to right
0 0.5 1 0 0.5 1 0 0.5 1
25
26
27 The horizontal axis measures individual Brier(Abs) scores for baseline forecasts. The vertical axis measures
28 corresponding Brier(Abs) scores for the extended forecasts. A score below 0.5 indicates a correct forecast.
Only observations with volatility in the 1st to 33rd percentile range are included.
29
30
31 When Brier(Sq) is used to measure forecast performance, there are fewer
32 instances where the non-parametric and extended models perform better than the
33 baseline. The notable differences between the Brier(Sq) scores and Brier(Abs)
34 scores occur for Hong Kong, where Brier(Sq) shows no improvements from the
35 non-parametric and extended models.
36 The ratios under Brier(Abs) also show that the extended model performs much
37 better than the non-parametric model. We note also that for both Brier(Abs) and
38 Brier(Sq), the performance of the non-parametric and extended models, relative to
39 the baseline, is generally better at the quarterly frequency than at the monthly fre-
40 quency. This is to be expected, as the theory indicates that volatility-aided predic-
41 tion depends on a sizable mean return, and the mean return increases in all
42 markets as we go from monthly to quarterly frequencies.
43 In the medium and high volatility subperiods in Tables 7 and 8, respectively,
44N much less improvement in the performance of the non-parametric and extended

Journal of Financial Forecasting Volume 1/ Number 2, Fall 2007

JFF 07 08 06 PC 10/26/07 1:21 PM Page 21

Direction-of-change forecasts based on conditional variance, skewness and kurtosis dynamics 21

models is found. It appears that volatility in these subperiods is simply too large 1
relative to the mean to be useful in guiding direction-of-change forecasts. 2
Figures 6 and 7 show a clear picture of the forecast performance of the non- 3
parametric and extended models compared to the baseline forecasts. At each fre- 4
quency, we show a scatterplot of the Brier(Abs) scores of individual forecasts. 5
We include only observations when volatility is low, as previously defined. In both 6
figures, the horizontal axis measures the Brier(Abs) scores for individual baseline 7
forecasts. In Figure 6, the vertical axis measures the Brier(Abs) scores for 8
individual non-parametric forecasts. In Figure 7, the vertical axis measures the 9
Brier(Abs) scores for individual forecasts from the extended model. In addition to 10
the scatterplots, we include a horizontal and vertical gridline at 0.5 and a 45° line. 11
As a Brier(Abs) score below 0.5 indicates a correct forecast, points in the lower 12
left quadrant indicate that both competing forecasts are correct, whereas a point in 13
the lower right quadrant indicates that the baseline forecast for this observation is 14
incorrect, with the competing forecast correct. Points below the 45° line indicate 15
improvements in the Brier(Abs) scores over the baseline. 16
In all three markets, the non-parametric and extended models clearly provide 17
better signals than the baseline model when both the baseline and the competing 18
forecasts are correct. However, for Hong Kong and UK, the performance of the 19
non-parametric and extended model is worse than the baseline model when the 20
baseline and the competing forecasts are wrong. Note that the upper left and lower 21
right quadrants of Figures 6 and 7 are mostly empty, which implies that the mod- 22
els by and large make predictions that are similar to the baseline forecasts. 23
Nonetheless, there is evidence that when volatility is low, forecasts of volatility 24
can improve the quality of the signal, in the sense of providing probability fore- 25
casts with improved Brier scores. 26
27
28
5 SUMMARY AND DIRECTIONS FOR FUTURE RESEARCH
29
Methodologically, we have extended the Christoffersen and Diebold (2006) 30
direction-of-change forecasting framework to include the potentially important 31
effects of higher-ordered conditional moments. Empirically, in an application to a 32
sample of three equity markets, we have verified the importance of allowing 33
for higher-ordered conditional moments and taken a step toward evaluating the 34
real-time predictive performance. In future work, we look forward to using our 35
direction-of-change forecasts to formulate and evaluate actual trading strategies 36
and to exploring their relationships to the “volatility timing” strategies recently 37
studied by Fleming et al (2003), in which portfolio shares are dynamically 38
adjusted based on forecasts of the variance–covariance matrix of the underlying 39
assets. 40
41
REFERENCES 42
Andersen, T. G., Bollerslev, T., Diebold, F. X., and Labys, P. (2003). Modeling and forecast- 43
ing realized volatility. Econometrica 71, 579–626. 44N

Research Paper www.journaloffinancialforecasting.com

 JFF 07 08 06 PC 10/26/07 1:21 PM Page 22

22 P. F. Christoffersen et al

1 Andersen, T. G., Bollerslev, T., Christoffersen, P. F., and Diebold, F. X. (2006). Volatility and
2 correlation forecasting. Handbook of Economic Forecasting, Elliott, G., Granger, C. W. J.
and Timmermann, A. (eds). North-Holland, Amsterdam 777–878.
3
4 Brandt, M. W., and Kang, Q. (2004). On the relationship between the conditional mean
and volatility of stock returns: a latent VAR approach. Journal of Financial Economics
5 72, 217–257.
6
Christoffersen, P. F., and Diebold, F. X. (2006). Financial asset returns, direction-of-change
7 forecasting, and volatility dynamics. Management Science 52, 1273–1287.
8 Diebold, F. X. and Lopez, J. (1996). Forecast evaluation and combination. Handbook
9 of Statistics, Maddala, G. S. and Rao, C. R. (eds). North-Holland, Amsterdam,
10 pp. 241–268.
11 Engle, R. F., Lilien, D. M., and Robins, R. P. (1987). Estimating time varying risk premia in
12 the term structure: the Arch-M model. Econometrica 55, 391–407.
13 Fleming, J., Kirby, C., and Ostdiek, B. (2003). The economic value of volatility timing using
14 realized volatility. Journal of Financial Economics 67, 473–509.
15 Lettau, M., and Ludvigson, S. (2005). Measuring and modeling variation in the risk-return
16 tradeoff. Handbook of Financial Econometrics, Ait-Shalia, Y. and Hansen, L. P. (eds).
17 North-Holland, Amsterdam, forthcoming.
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44N

Journal of Financial Forecasting Volume 1/ Number 2, Fall 2007