Master in Financial Engineering (EPFL)

Financial Econometrics

Chapter 3: Performance evaluation and ranking

Florian Pelgrin

EDHEC Business School

(florian.pelgrin@edhec.edu)

Feb. 2017 - June 2017

1. Introduction

2. Risk measures and risk-adjusted performance measures

3. Model-based risk-adjusted performance measures

4. Application
1. Introduction
Context :
Data set

X Monthly returns of 10 US industry portfolios (see Kenneth

Frenchs webpage)non durables (Non. Dur.), consumer
durables (Dur), manufacturing (Man.), energy, technology
(High tech.), telecommunications (Telecom), wholesale and
retail (Shops), healthcare (Health), utilities (Util.) and other
industries (Other) ;

X A benchmark return for the market (Market) ;

X Monthly return of a risk free rate of interest (Risk Free),

X Sample period is January 1927( !) - December 2008.

Using Lecture 1...

Table 1 : Summary statistics of monthly returns

Mean Std. Dev. Skewness Kurtosis
Market 0.590 5.455 0.1886 10.56
Risk free 0.305 0.252 0.015 1.016
Non. Dur. 0.949 4.713 -0.032 8.713
Dur. 1.000 7.665 1.098 18.180
Man. 0.981 6.380 0.918 15.340
Energy 1.063 6.030 0.212 6.114
High Tech. 1.051 7.484 0.281 8.880
Telecom 0.803 4.642 0.011 6.230
Shops 0.958 5.916 -0.032 8.390
Health 1.062 5.792 0.168 10.060
Util. 0.869 5.710 0.088 10.480
Other 0.876 6.529 0.919 16.450
 Different risk measures and risk-adjusted performance
measures can be used. Importantly, risk-adjusted performance
measures are scale-free, i.e. it allows for comparisons.

Generally, two types of measures :

I Usual measures : Volatility, Sharpe-ratio, semi-variance,
semi-volatility, information ratio, etc.

I Model-based measures : Treynor ratio, Jensens alpha,

appraisal ratio, Market-timing measure, etc. These measures
are derived from the estimation of linear models, especially
here the CAPM.
Main question : Can we rank portfolios and/or individual
assets ?
In the sequel,

R p : Proportional or simple return of a portfolio ;

R f : Simple return of a risk-free asset ;

R m : Simple return of the market (e.g. S&P500) ;

R b : Simple return of a benchmark.

2. Risk measures and risk-adjusted performance measures
Definition (Historical volatility)
The (historical) volatility is a risk measure defined by :
v
u
u 1 X T
p p
2
s(R ) = t Rt R p
T 1
t=1

Notes :
X Other measures of volatility include EWMA-based volatility,
implied volatility, etc.
X Implicit assumption is that returns are normally distributed.
X From the risk perspective, no difference between returns
above or below the mean R p .
Definition (Sharpe ratio)
The Sharpe ratio is a risk-adjusted performance measure defined
by :

Rp Rf
SR(R p ) = .
s(R p R f )

where R f is a risk-free asset.

Note : The Sharpe ratio captures the reward in terms of mean

excess return per unit of risk Risk-averse investors prefer large
Sharpe ratios !
Definition (Lower partial moment)
The historical partial moment of order a with respect to the
aspiration level (threshold, benchmark), denoted , is the
average ath power deviation of returns below :
T
1 X p
a
LPM(a, ) = max 0, Rt .
T
t=1
Definition (Semi-variance and Semi-volatility)
The (historical) semi-variance is the lower partial moment of
order a = 2 with respect to an aspiration level equal to the
average return ( = R p ) :
T
1 X p
2
SVar(R p ) = LPM(2, R p ) = max 0, R p Rt .
T
t=1

The (historical) semi-volatility is the square root of the

semi-variance :
p
SVol(R p ) = SVar(R p ).
Notes :

X Both measures are asymmetric risk measures (downside risk

measure) : more importance to downside returns than to
upside returns ;

X Both measures represent better downside risk for asymmetric

(skewed) distributions ;

X They are proportional to variance and volatility for Gaussian

or symmetric distributions.

X A risk-adjusted performance measure is provided by the

Sortino ratio :

Rp Rf
SoR(R p ) =
SVol(R p R f )

Portfolios (assets) with more downside than upside risk are

penalized by this measure.
Definition (Information ratio)
The historical information ratio is a risk-adjusted performance
measure that accounts for the overperformance of the portolio
with respect to a benchmark (another portfolio or reference
asset) R b :

Rp Rb
IR(R p ) =
s(R p R b )

where the term s(R p R b ) is the tracking error of the portfolio

with respect to the benchmark.
3. Model-based risk-adjusted performance measures
Definition (Treynor ratio)
The Treynor ratio is a risk-adjusted performance measure defined
by :

zp Rp Rf
TrR = m
=
p
(z , z ) (R R f , R m R f )
p

where (z p , z m ) is the CAPM beta coefficient :

p
zt = + zm
t + ut

p p
where zt = Rt Rtf and ztm = Rtm Rtf denote respectively the
portfolio excess return and the market excess return.

Note : An investor will prefer a portfolio with a higher Treynor

ratio.
Notes :

X The Treynor ratio accounts only for the market systematic

risk captured by the (CAPM) beta of the portfolio with respect
to the market.

X The benchmark is the (excess) return of the market (portfolio)

return, which is not observable (in the sense of CAPM). A
broad stock market index is generally considered as a proxy.

X The Treynor ratio is defined as the Sharpe ratio, but

considering the beta of the portfolio instead of historical
volatility.

X A ranking of portfolios based on the Treynor Ratio is useful if

those portfolios belong to a broader fully-diversified portfolio
because only the (market) systematic risk is priced. Said
differently, two portfolio with identical systematic risk, but
different total risk, will be rated the same.
Definition (Jensens alpha)
The Jensens alpha is a risk-adjusted performance measure that
captures the overperformance (b p > 0) or underperformance
(b p
< 0) of the portfolio with respect to the CAPM-implied
expected return :

bp = z p bp z m

1 PT p 1 PT
Rt Rtf and z m = Rtm Rtf .



where z p = T t=1 T t=1
Definition (Appraisal ratio)
The appraisal ratio is a risk-adjusted measure defined by :

bp,b
AR =
u)
se(b

where
bp,b and se(bu ) denote respectively the estimate of the
abnormal return and the standard error of the residuals
(non-sytematic or idiosyncratic measure of risk) in the following
CAPM-based regression (t = 1, , T ) :
p
zt = p,b + p,b ztb + ut

where ztb = Rtb Rtf .

Note : This is an alternative to the information ratio, especially if

the portfolio is leveraged with respect to the benchmark.
4. Application
Description :
Simple returns of 30 industry portfolios (see Excel file
Lecture 3 application.xls)

The sample size is January 1990 - December 2014 (298

monthly observations)

The benchmark portfolio is an equally-weighted portfolio

Objectives

1. Provide a ranking (and clustering) of portfolios using some

risk measures and risk-adjusted performance measures :
X The historical (semi-) volatility ;
X The Sharpe ratio ;
X The Sortino ratio ;
X The information ratio (the market return being the benchmark)

2. Provide a ranking (and clustering) of portfolios using some

model-based risk (-adjusted) measures :
X The Treynor ratio ;
X The Jensens alpha ;
X The appraisal ratio ;
 Risk (-adjusted performance) measures
Industry Volatility Sharpe ratio Semi Volatility Sortino ratio Informat. ratio
Food 0.0403 0.1674 0.2561 0.0262 0.0129
Beer 0.0492 0.1698 0.3839 0.0216 0.0445
Smoke 0.0684 0.1466 0.7482 0.0134 0.0553
Games 0.0696 0.1094 0.7970 0.0096 0.0315
Books 0.0564 0.0741 0.4643 0.0090 -0.0600
Hshld 0.0430 0.1611 0.3039 0.0227 0.0187
Clths 0.0650 0.1304 0.6538 0.0129 0.0484
Hlth 0.0443 0.1854 0.3062 0.0267 0.0558
Chems 0.0563 0.1309 0.4928 0.0150 0.0320
Txtls 0.0817 0.0753 0.9348 0.0066 -0.0013
Cnstr 0.0614 0.0975 0.6169 0.0097 -0.0070
Steel 0.0828 0.0600 1.0947 0.0046 -0.0237
FabPr 0.0666 0.1130 0.7279 0.0104 0.0341
ElcEq 0.0635 0.1583 0.6380 0.0158 0.1072
Autos 0.0779 0.0733 0.8839 0.0065 -0.0095
Carry 0.0591 0.1625 0.6107 0.0157 0.0819
Mines 0.0800 0.0498 0.9812 0.0041 -0.0320
Coal 0.1178 0.0858 1.9718 0.0051 0.0361
Oil 0.0532 0.1348 0.4247 0.0169 0.0200
Util 0.0402 0.1513 0.2817 0.0216 -0.0039
Telcm 0.0519 0.0911 0.4475 0.0106 -0.0489
Servs 0.0650 0.1352 0.6700 0.0130 0.0724
BusEq 0.0781 0.1146 0.9881 0.0090 0.0555
Paper 0.0496 0.1248 0.3913 0.0158 -0.0014
Trans 0.0512 0.1434 0.4291 0.0171 0.0339
Whlsl 0.0466 0.1295 0.3678 0.0164 -0.0073
Rtail 0.0509 0.1549 0.4008 0.0195 0.0505
Meals 0.0496 0.1524 0.4051 0.0185 0.0364
Fin 0.0565 0.1277 0.5445 0.0133 0.0314
Other 0.0568 0.0367 0.5324 0.0039 -0.1124
 Ranking
Industry Volatility Sharpe ratio Semi Volatility Sortino ratio Inform. Ratio
Food 29 3 30 2 19
Beer 25 2 25 5 9
Smoke 8 10 8 15 6
Games 7 21 7 22 15
Books 17 26 18 23 29
Hshld 28 5 28 3 18
Clths 10 15 11 18 8
Hlth 27 1 27 1 4
Chems 18 14 17 14 14
Txtls 3 25 5 25 20
Cnstr 13 22 13 21 23
Steel 2 28 2 28 26
FabPr 9 20 9 20 12
ElcEq 12 6 12 12 1
Autos 6 27 6 26 25
Carry 14 4 14 13 2
Mines 4 29 4 29 27
Coal 1 24 1 27 11
Oil 19 13 21 9 17
Util 30 9 29 4 22
Telcm 20 23 19 19 28
Servs 11 12 10 17 3
BusEq 5 19 3 24 5
Paper 23 18 24 11 21
Trans 21 11 20 8 13
Whlsl 26 16 26 10 24
Rtail 22 7 23 6 7
Meals 24 8 22 7 10
Fin 16 17 15 16 16
Other 15 30 16 30 30
Comments :

Performance measures are risk-adjusted measures !

The Sharpe ratio and the Sortino ratio provide some

consistent results : only portfolios with large downside risks
are ranked differently.

In contrast, the information ratio is less consistent relative

to the Sharpe (resp., Sortino) ratio : it points towards the
importance of the benchmark (see further).

Based on the different rankings, one could classify portfolios

and aggregate them into broader portfolios, i.e. to create
some categories of portfolios (e.g., high Sharpe ratio/Sortino
ratio versus low Sharpe ratio/Sortino ratio).

Consistency measures between two rankings can be achieved.

 Model based risk measures Ranking
Industry Jensens alpha Treynor ratio Appraisal ratio Jensens alpha Treynor ratio Appraisal ratio
Food 0.0045 0.0169 0.1195 5 4 4
Beer 0.0060 0.0195 0.1285 2 3 2
Smoke 0.0080 0.0244 0.1074 1 1 5
Games -0.0010 0.0080 -0.0626 20 20 23
Books -0.0023 0.0066 -0.1041 27 27 27
Hshld 0.0038 0.0147 0.1024 6 6 6
Clths 0.0018 0.0105 0.0018 13 13 15
Hlth 0.0046 0.0154 0.1377 4 5 1
Chems 0.0011 0.0099 -0.0198 16 17 20
Txtls -0.0019 0.0072 -0.0844 26 26 24
Cnstr -0.0014 0.0075 -0.1046 24 25 28
Steel -0.0060 0.0049 -0.1545 30 29 30
FabPr -0.0013 0.0078 -0.0922 23 21 25
ElcEq 0.0018 0.0102 0.0307 14 14 11
Autos -0.0029 0.0065 -0.1033 28 28 26
Carry 0.0035 0.0124 0.0545 8 9 10
Mines -0.0010 0.0076 -0.0481 21 24 22
Coal 0.0024 0.0108 -0.0079 11 12 17
Oil 0.0036 0.0139 0.0589 7 7 9
Util 0.0051 0.0213 0.1220 3 2 3
Telcm -0.0011 0.0076 -0.0217 22 23 21
Servs -0.0000 0.0088 0.0238 19 19 13
BusEq -0.0015 0.0077 -0.0079 25 22 16
Paper 0.0012 0.0101 -0.0112 15 15 18
Trans 0.0019 0.0109 0.0260 12 11 12
Whlsl 0.0010 0.0099 0.0033 17 16 14
Rtail 0.0025 0.0115 0.0644 10 10 7
Meals 0.0032 0.0128 0.0632 9 8 8
Fin 0.0001 0.0089 -0.0177 18 18 19
Other -0.0040 0.0047 -0.1360 29 30 29
Comments :

The three ratios provide a consistent ranking. In contrast to

unstructured risk-adjusted performance measures (Sharpe
ratio, Sortino ratio and information ratio), we do observe less
sensitivity in terms of ranking.

One should expect that the Jensens alpha and the Treynor
ratio produce consistent resultsthe two measures rest on
the CAPM specification. In the case of the information ratio
(which corrects for the presence of leveraged portfolio with
respect to the information ratio), it does depend on the
benchmark (here, the 1/N-portfolio or equally weighted
portfolio).

Based on the different rankings, one could classify portfolios

and aggregate them into broader portfolios, i.e. to create
some categories of portfolios (e.g., high Sharpe ratio/Sortino
ratio versus low Sharpe ratio/Sortino ratio).
Comments :

Note that the model-based risk-adjusted performance

measures rely on a benckmark :

I The Jensens alpha and the Treynor ratio make use of the
CAPM and thus of the market portfolio, which is not observable
in practise and is proxied by a broader stock market index
(here, S&P500) ;

I The information ratio makes use of a benchmark (tracking)

portfolio (here, the equally-weighted portfolio).
The choice of the benchmark is essential and often more
important than the choice of some specific performance
measures. Among others, it affects how portfolios behave
relative to each others.

Take-away : You must pay attention to the choice of the

benchmark when reading performance statistics and when
evaluating performances !