Quantitative Finance, Vol. 7, No.

4, August 2007, 389–396

On the feasibility of portfolio optimization

under expected shortfall
STEFANO CILIBERTI*y, IMRE KONDORz and MARC MÉZARDy
yCNRS, Univ. Paris Sud, UMR8626, LPTMS, Orsay Cedex F-91405, France
zCollegium Budapest, Szenthromsg u. 2, 1014 Budapest, Hungary

(Received 3 May 2006; in final form 24 April 2007)

We address the problem of portfolio optimization under the simplest coherent risk measure,
i.e. the expected shortfall. As is well known, one can map this problem into a linear
programming setting. For some values of the external parameters, when the available time
series is too short, portfolio optimization is ill-posed because it leads to unbounded positions,
infinitely short on some assets and infinitely long on others. As first observed by Kondor and
coworkers, this phenomenon is actually a phase transition. We investigate the nature of this
transition by means of a replica approach.

Keywords: Statistical physics; Finance; Portfolio optimization; Quantitative finance;

Correlation modelling; Critical phenomena; Risk measures

1. Introduction is reported by Frey and McNeil (2002). On the other

Among the several existing risk measures in the context
of portfolio optimization, expected shortfall (ES) has
certainly gained increasing popularity in recent years. In
several practical applications, ES is starting to replace
the classical Value-at-Risk (VaR). There are a number of
reasons for this. For a given threshold probability
, the
VaR is defined so that with probability
the loss will be
smaller than VaR. This definition only gives the
minimum loss one can reasonably expect but does not
tell anything about the typical value of that loss which
can be measured by the conditional value-at-risk (CVaR,
which is the same as ES for the continuous distributions
that we consider here).x We will be more precise with
these definitions below. The point we want to stress here
is that the VaR measure, lacking the mandatory
properties of subadditivity and convexity, is not coherent
(Artzner et al. 1999). This means that summing the
VaRs of individual portfolios will not necessarily
produce an upper bound for the VaR of the combined
portfolio, thus contradicting the holy principle of
diversification in finance. A nice practical example of
the inconsistency of VaR in credit portfolio management
hand, it has been shownn (Acerbi and Tasche 2002) that
ES is a coherent measure with interesting properties
(Pflug 2000). Moreover, the optimization of ES can be
reduced to linear programming (Rockafellar and
Uryasev 2000) (which allows for a fast implementation)
and leads to a good estimate for the VaR as a byproduct
of the minimization process. To summarize, the intuitive
and simple character, together with the mathematical
properties (coherence) and the fast algorithmic
implementation (linear programming), are the main
reasons behind the growing importance of ES as a risk
measure.
In this paper we focus on the feasibility of the portfolio
optimization problem under the ES measure of risk.
The control parameters of this problem are (i) the
imposed threshold in probability,
and (ii) the ratio
N/T between the number N of financial assets making
up the portfolio and the time series length T used to
sample the probability distribution of returns. It is
curious, albeit trivial, that the scaling in N/T had not
been explicitly pointed out before (Pafka and Kondor
2002). It was reported by Kondor et al. (2007) that, for
certain values of these parameters, the optimization

*Corresponding author. Email: ciliberti@roma1.infn.it

xSee Acerbi and Tasche (2002) for the subtleties related to a discrete distribution.
Quantitative Finance
ISSN 1469–7688 print/ISSN 1469–7696 online ß 2007 Taylor & Francis
http://www.tandf.co.uk/journals
DOI: 10.1080/14697680701422089
390 S. Ciliberti et al.

problem does not have a finite solution because, even if 1

convex, it is not bounded from below. Extended b
numerical simulations allowed these authors to determine
the feasibility map of the problem. Here, in order to
better understand the root of the problem and to study

P< (w; α)
the transition from a feasible to an unfeasible regime
(corresponding to an ill-posed minimization problem) we
address the same problem from an analytical point
of view.
The paper is organized as follows. In section 2 we
briefly recall the basic definitions of
-VaR and
-CVaR
0
and we show how the portfolio optimization problem can b-VaR(w) α
be reduced to linear programming. We introduce a ‘cost
function’ to be minimized under linear constraints and Figure 1. Schematic representation of the VaR measure of
we discuss the rationale for a statistical mechanics risk. P < ðwÞ is the probability of a loss associated with the
approach. In section 3 we solve the problem of optimizing portfolio w being smaller than . The conditional VaR
-CVaR
large portfolios under ES using the replica approach. Our (or ES) is the average loss when this is constrained to be greater
than the
-VaR.
results and a comparison with numerics are reported
in section 4, and our conclusions are summarized in
section 5.
(see figure 1), While the CVaR (or ES, in this case)
associated with the same portfolio is the average loss on
the tail of the distribution,
2. The optimization problem
R
We consider a portfolio of N financial instruments dx pðxÞ‘ðwjxÞð‘ðwjxÞ

-VaRðwÞÞ

-CVaRðwÞ ¼ R
w ¼ fw1 ; . . . ; wN g where wi is the position of asset i. The dx pðxÞð‘ðwjxÞ

-VaRðwÞÞ
global budget constraint fixes the sum of these numbers: Z
1
we impose, for example, ¼ dx pðxÞ‘ðwjxÞð‘ðwjxÞ
1


X
N

-VaRðwÞÞ: ð4Þ
wi ¼ N: ð1Þ
i¼1
The threshold
then represents a confidence level. In
We do not stipulate any constraint on short selling, so practice, the typical values of
which one considers are
that wi can be any negative or positive number. This is,
¼ 0.90, 0.95, and 0.99, but we will address the problem
of course, unrealistic for liquidity reasons, but con- for any
2 [0, 1]. What is usually called ‘exceeding
sidering this case allows us to demonstrate the essence probability’ in previous literature would correspond here
of the phenomenon. If we imposed a constraint that to (1

).
would render the domain of wi bounded (such as a ban As mentioned in the Introduction, the ES measure can
on short selling, for example), this would evidently be obtained from a variational principle (Rockafellar and
prevent the weights from diverging, but a vestige of the Uryasev 2000). The minimization of a properly chosen
transition would still remain in the form of large, objective function leads directly to (4)
although finite, fluctuations of the weights, and in a
large number of them sticking to the ‘walls’ of the
-CVaRðwÞ ¼ min F
ðw;vÞ; ð5Þ
v
domain.
Z
We denote the returns on the assets by
1
x ¼ fx1 ; x2 ; . . . ; xN g and we assume that there exists an F
ðw;vÞ  v þ ð1

Þ dx pðxÞ½‘ðwjxÞ
vþ : ð6Þ
underlying probability distribution function p(x) of the
returns. The loss P of portfolio w given the returns Here, [a]þ  (a þ jaj)/2. The external parameter v over
x is ‘ðw j xÞ ¼
N i¼1 wi xi , and the probability of that which one has to minimize is claimed to be relevant
loss being smaller than a given threshold  is in itself (Rockafellar and Uryasev 2000), since its
Z optimal value may represent a good estimate for the
P < ðw; Þ ¼ dx pðxÞð
‘ðw j xÞÞ; ð2Þ actual value-at-risk of the portfolio. We will come back to
this point when we discuss our results. We stress here that
minimizing (6) over w and v is equivalent to optimizing (4)
where () is the Heaviside step function, equal to 1 if its over the portfolio vectors w.
argument is positive and 0 otherwise. The
-VaR of this Of course, in practical cases the probability distribution
portfolio is formally defined by of the loss is not known and must be inferred from past
data. In other words, we need an ‘in-sample’ estimate

-VaRðwÞ ¼ minf : P < ðw; Þ 
g ð3Þ of the integral in (6) which would turn a well-posed
 On the feasibility of portfolio
(but useless) optimization problem into a practical
approach. We thus approximate the integral by sampling
the probability distributions of returns. If we have a
time series x(1), . . . , x(T ), our objective function simply
becomes
1 XT
F^
ðw;vÞ ¼ v þ ½‘ðw j xðÞ Þ
vþ
ð1

ÞT ¼1
" #þ
1 XT XN
¼vþ
v
wi xi ;
ð1

ÞT ¼1 i¼1
where we denote by xi the return of asset i at time .
Minimizing this risk measure is the same as the
following linear programming problem:
. given one data sample, i.e. a matrix xi ,
i ¼ 1, . . . , N,  ¼ 1, . . . , T,
. minimize the cost function
E
½Y; fxi g ¼ E
½v; fwi g; fu g; fxi g
X
T
¼ ð1

ÞTv þ u ;
t¼
. over the (N þ T þ 1) variables Y  {w1, . . . ,
wNu1, . . . , uTv},
. under the (2T þ 1) constraints
X
N X
N method that we use can be generalized easily to iid
u  0; u þ v þ xi wi  0; 8 and
i¼1 i¼1
Since we allow short positions, not all the wi are positive,
which makes this problem different from standard linear
programming. To keep the problem tractable, we impose
the condition that wi 
W, where W is a very large
cutoff, and the optimization problem will be said to be
ill-defined if its solution does not converge to a finite limit
when W ! 1. It is now clear why constraining all the wi
to be non-negative would eliminate the feasibility
problem: a finite solution will always exist because the
weights are by definition bounded, the worst case being
an optimal portfolio with only one non-zero weight
taking care of the total budget. The control parameters
that govern the problem are the threshold
and the ratio
N/T of assets to data points. The resulting ‘phase
diagram’ is then a line in the

N/T plane separating a
region in which, with high probability, the minimization
problem is not bounded and thus does not admit a finite
solution, and another region in which a finite solution
exists with high probability. These statements are
non-deterministic because of the intrinsic probabilistic
nature of the returns. We will address this minimization
problem in the non-trivial limit where T ! 1, N ! 1,
while N/T stays finite. In this ‘thermodynamic’ limit,
we shall assume that extensive quantities (like the
average loss of the optimal portfolio, i.e. the minimum
cost function) do not fluctuate, namely that their
probability distribution is concentrated around the
mean value. This ‘self-averaging’ property has been
391
proved for a wide range of similar statistical mechanics
models (Guerra and Toninelli 2004). Then, we will be
interested in the average value of the minimum of the cost
function (8) over the distribution of returns. Given the
similarity of portfolio optimization to the statistical
physics of disordered systems, this problem can be
addressed analytically by means of a replica approach
(Mézard et al. 1987).
ð7Þ
3. The replica approach
For a given history of returns xit, one can compute
the minimum of the cost function, minY E
½Y; fxit g. In
this section we show how to compute analytically the
expectation value of this quantity over the histories of
returns. For simplicity we shall keep to the case in which
the xit are independent identically distributed (iid) normal
variables, so that a history of returns xit is drawn from the
distribution
ð8Þ Y 2
pðfxit gÞ  e
Nxit =2 : ð10Þ
it
This assumption of an iid normal distribution of returns is
very restrictive, but we would like to emphasize that the
wi ¼ N: variables with other distributions, and also in some cases
to correlated variables. Certainly, the precise location of
ð9Þ the critical value of N/T separating an unfeasible from a
feasible phase depends on the distribution of returns. But
we expect that the broad features like the existence of this
critical value, or the way the fluctuations in portfolio
diverge when approaching the transition, should not
depend on this distribution. This property, called
universality, has been one of the major discoveries of
statistical mechanics in the last 50 years.
Instead of focusing only on the minimal cost, the
statistical mechanics approach makes a detour: it
considers, for a given history of returns xit, a probability
distribution in the space of variables Y, defined by
P ðYÞ ¼ 1=Z ½fxit g exp½
E
½Y; fxit g. The parameter 
is an auxiliary parameter. In physics it is the inverse of the
temperature, however in the present case it is just one
parameter that we introduce in order to have a
probability distribution on Y that interpolates between
the uniform probability ( ¼ 0) and a probability that is
peaked on the value of Y which minimizes the cost
E
½Y; fxit g (the case where  ¼ 1).
The normalization constant Z ½fxit g is called the
partition function at inverse temperature  it is defined as
Z
Z ½fxit g ¼ dY exp½
E
½Y; fxit g; ð11Þ
V
where V is the convex polytope defined by (9).
The partition function contains a lot of information on
the problem. For instance, the minimal cost can be
expressed as lim!1 ð
1Þ=ðNÞ log Z ½fxit g. We shall be
392 S. Ciliberti et al.

interested in computing the large N limit of the minimal measure P(Y) is spread. After (several) Gaussian
cost per variable: integrations, one obtains
min E ½fxit g
1 Zn ½fxit g
"½fxit g ¼ lim ¼ lim lim log Z ½fxit g:
N!1 N N!1 !1 N Z þ1 Yn Z þ1 Y Z þi1 Y
ð12Þ  dva ab
dQ dQ^ ab

1 a¼1
1 a;b
i1 a;b
In the following we will compute the average value of (
this quantity over the choice of the sample xit. Using X X X
 exp N Qab Q^ ab
N Q^ ab
ð1

ÞT va
equation (12) we can compute this average minimum cost
a;b a;b a
if we are able to compute the average of the logarithm of
Z. This is a difficult problem that is usually circumvented
Tn log  þ T log Z^  ðfv g; fQab gÞ
a

by means of the so-called ‘replica trick’: one computes the )

average of Zn, where n is an integer, and then the average T N nN

Tr log Q
Tr log Q^
log 2 ; ð16Þ
of the logarithm is obtained from 2 2 2

@Zn where
log Z ¼ lim ; ð13Þ "
n!1 @n Z þ1 Y
n
1X n
thus assuming that Zn can be analytically continued to Z^  ðfva g;fQab gÞ  dya exp
ðQ
1 Þab ðya
va Þ

1 a¼1 2 a;b¼1
real values of n. The overline indicates an average over
#
different samples, i.e. over the probability distribution X
n
(10). This technique has a long history in the physics  ðyb
vb Þ þ  ya ð
ya Þ : ð17Þ
of spin glasses (Mézard et al. 1987): the proof that it leads a¼1

to the correct solution has recently been reported We now write T ¼ tN and work at fixed t while N ! 1.
(Talagrand 2002). The most natural solution is obtained by realizing that
The partiton function (11) can be written more all the replicas are identical. Given the linear character of
explicity as the problem, the symmetric solution should be the correct
one. The replica-symmetric solution corresponds to the
Z ½fxit g
Z þ1 Z Z ansatz
þ1 Y
T þ1 Y
N
( (
¼ dv dut dwi q1 ; if a ¼ b; q^1 ; if a ¼ b;

1 0 t¼1
1 i¼1 18Q ¼ab
Q^ ¼
ab
Z " !# Z q0 ; if a 6¼ b; q^0 ; if a 6¼ b;
þi1 X
N þ1 Y
T
 d exp  wi
N dt ð18Þ

i1 i¼1 0 t¼1
Z þi1 Y " !# and va ¼ v for any a. As we discuss in detail in appendix A,
T X T X N
 d^ t exp ^ t ut þ v þ xit wi
t one can show that the optimal cost function, computed

i1 t¼1 t¼1 i¼1 from equation (12), is the minimum of
" #
X T

 exp
ð1

ÞTv
 ut ; ð14Þ 1 q0
"ðv; q0 ;
Þ ¼ þ
tð1

Þv

t¼1 2
2
Z þ1  
where the constraints are imposed by means of the t
s2
pffiffiffiffiffiffiffi
þ p ffiffiffi ds e g v þ s 2q0 ; ð19Þ
Lagrange multipliers , , . ^ The replica trick is based 2 p
1
on the idea that the nth power of the partition function
where
 lim!1 
q and the function g() is defined as
appearing in (13) can be written as the partition function
1 n 8
for n independent replicas Y , . . . , Y of the system: all the
< 0;
> x  0;
replicas correspond to the same history of returns {xit}, gðxÞ ¼ x ; 2

1  x < 0; ð20Þ
and their joint probability distribution function >
:
P n
2x
1; x <
1:
is P ðY1 ; . . . ; Yn Þ ¼ 1=Zn ½fxit g exp½
 a¼1 E
½Ya ; fxit g.
It is not difficult to write down the expression for Zn and Note that this function and its derivative are continuous.
average it over the distribution of samples xit. One Moreover, v and q0 in (19) are solutions of the saddle
introduces the overlap matrix point equations
Z pffiffiffiffiffiffiffi
X N 1
ab 1
2
Q a b
wi wi ; a; b ¼ 1; . . . ; n; ð15Þ 1

þ p ffiffiffi ds e
s g0 ðv þ s 2q0 Þ ¼ 0; ð21Þ
N i¼1 2 p
Z pffiffiffiffiffiffiffi
t 2
^ ab
as well as its conjugate Q (the Lagrange multiplier
1 þ p ffiffiffiffiffiffiffiffiffiffi ds e
s sg0 ðv þ s 2q0 Þ ¼ 0: ð22Þ
2pq0
imposing (15)), where a and b are replica indexes. This
matrix characterizes how the portfolios in different We require that the minimum of (19) occurs at a finite
replicas differ: they provide some indication of how the value of
. In order to understand this point, we recall the
 On the feasibility of portfolio
meaning of
(see also (18)):
1X N  2

= 
q ¼ ðq1
q0 Þ ¼ wð1Þ
N i¼1 i
ð23Þ
1X N

wð1Þ wð2Þ  w2
w2 ;
N i¼1 i i
where the superscripts (1) and (2) represent two generic
replicas of the system. We then find that
is proportional
to the fluctuations in the distribution of the w0 s. An
infinite value of
would then correspond to a portfolio
which is infinitely short on some particular positions and,
because of the global budget constraint (1), infinitely long
on others.
Given (19), the existence of a solution at finite

translates into the following condition:
Z þ1 pffiffiffiffiffiffiffi
q0 t 2
tð1

Þv
þ pffiffiffi ds e
s gðv þ s 2q0 Þ  0;
2 2 p
1
ð24Þ
which defines, along with equations (21) and (22), our
phase diagram.
4. The feasible and unfeasible regions
We can now chart the feasibility map of the expected
shortfall problem. Following the notation of Kondor
et al. (2007), we will use as control parameters N/T  1/t
and
. The limiting case
! 1 can be determined
analytically and one can show that the critical value t*
is given by
h i
1 1 3
ð4pð1

Þ2 Þ
1
¼
O ð1

Þ e : ð25Þ
t 2
This limit corresponds to the over-pessimistic case of
maximal loss, in which the single worst loss contributes to
the risk measure. The optimization problem is the
following:
" !#
X portfolio) over those that produced a finite solution. In
min max
wi xit : ð26Þ
w t
i
A simple ‘geometric’ argument of Kondor et al. (2007)
leads to the critical value 1/t* ¼ 0.5 in this extreme case.
The idea is the following. According to equation (26), one
has to look for the minimum of a polytope made by a
large number of planes, whose normal vectors (the xit) are
drawn from a symmetric distribution. The simplex is
convex, but with some probability it can be unbounded
from below and then the optimization problem is
ill-defined. Increasing T means that the probability of
this event decreases, because there are more planes and
1 and f(x) ! 0 if x  1, and where
thus it is more likely that for large values of wi the max
over t has a positive slope in the ith direction. The exact
law for this probability can be obtained by induction on
N and T (Kondor et al. 2007) and, as we said, it jumps
in the thermodynamic limit from 1 to 0 at N/T ¼ 0.5.
The example of the max-loss risk measure is also
helpful because it allows us to stress two aspects of
393
the problem: (1) even for finite N and T there is a finite
chance that the risk measure is unbounded from below in
some samples, and (2) the phase transition occurs in the
thermodynamic limit when N/T is strictly smaller than 1,
i.e. much before the covariance matrix develops zero
modes. The very nature of the problem is that the risk
measure there is simply not bounded from below. As for
the ES risk measure, the threshold value N/T ¼ 0.5 can be
thought of as a good approximation of the actual value
for many cases of practical interest (i.e.
0 0.9), since the
corrections to this limit case are exponentially small
(equation (25)).
For finite values of
we solve numerically
equations (21), (22) and (24) using the following
procedure. We first solve the two equations (21) and
(22), which always admit a solution for (v, q0). We then
plot the l.h.s. of equation (24) as a function of 1/t for a
fixed value of
. This function is positive at small 1/t and
becomes negative beyond a threshold 1/t*. By keeping
track of 1/t* (obtained numerically via linear interpola-
tions) for each value of
we build up the phase diagram
(figure 2, left). This diagram is in agreement with the
numerical results obtained by Kondor et al. (2007).
We show in the right panel of figure 2 the divergence of
the order parameter
versus 1/t
1/t*. The critical
exponent is found to be 1/2:

1 1
1=2


; ð27Þ
t t ð
Þ
again in agreement with the scaling found by Kondor
et al. (2007). We performed extensive numerical simula-
tions in order to check the validity of our analytical
findings. For a given realization of the time series, we
solve the optimization problem (8) by standard linear
programming (Press et al. 1992). We impose a large
negative cutoff for the w’s, that is wi4
W, and we say
that a feasible solution exists if it stays finite for W ! 1.
We then repeat the procedure for a certain number of
samples, and then average our final results (optimal cost,
optimal v, and the variance of the w’s in the optimal
figure 3 we show how the probability of finding a finite
solution depends on the size of the problem. Here, the
probability is simply defined in terms of the frequency.
We see that the convergence towards the expected 1–0 law
is fairly slow, and a finite size scaling analysis is shown in
the right panel. Without loss of generality, we can
summarize the finite-N numerical results by writing the
probability of finding a finite solution as

 
1 1 ð
Þ
pðN;T;
Þ ¼ f
N ; ð28Þ
t t ð
Þ
where f(x) ! 1 if x
(1) ¼ 1/2. It is interesting to note that these results do not
depend on the initial conditions of the algorithm used to
solve the problem: for a given sample, the algorithm finds,
in linear time, the minimum of the polytope by looking
at all its vertexes exhaustively. The statistics are taken by
repeating such a deterministic procedure on a large
number of samples chosen at random.
394 S. Ciliberti et al.

0.6 105
b = 0.95
0.5 b = 0.90
b = 0.80
104 −
0.4 non-feasible 1/√ x
N/T

0.3 103

∆
0.2
feasible 102
0.1

0 101
0 0.2 0.4 0.6 0.8 1 10−8 10−7 10−6 10−5 10−4 10−3 10−2
b N N ∗
T
−( )
T

Figure 2. Left: phase diagram of the feasibility problem for the expected shortfall. Right: the order parameter
diverges
with exponent 1/2 as the transition line is approached. A curve of slope
1/2 is also shozwn for comparison.

1 1

0.8 0.8
Probability
Probability

0.6 0.6

N = 32 N = 32
0.4 N = 64 0.4 N = 64
N = 128 N = 128
0.2 N = 256 0.2 N = 256
N = 512 N = 512
N = 1024 N = 1024
0 0
0.4 0.45 0.5 0.55 0.6 −1.5 −1 −0.5 0 0.5 1 1.5
N N N ∗
T T
−
T
( )
⋅ N 0.5

Figure 3. Left: the probability of finding a finite solution as obtained from linear programming at increasing values of N and
with
¼ 0.8. Right: scaling plot of the same data. The critical value is set equal to the analytical value, N/T ¼ 0.4945, and the
critical exponent is 1/2, i.e. that obtained by Kondor et al. (2007) for the limit case
! 1. The data do not collapse perfectly, and
better results can be obtained by slightly changing either the critical value or the exponent.

In figure 4 (left panel) we plot, for a given value of
,
the optimal cost found numerically for several values of
the size N compared with the analytical prediction at
infinite N. One can show that the cost vanishes as


1  (1/t
1/t*)1/2. The right panel of the same figure
shows the behaviour of the value of v which leads to the
optimal cost versus N/T for the same fixed value of
.
Also in this case, the analytical (N ! 1) limit is plotted
for comparison. We note that this quantity has been
suggested (Rockafellar and Uryasev 2000) to be a good
approximation of the VaR of the optimal portfolio: we
find here that vopt diverges at the critical threshold and
becomes negative at an even smaller value of N/T.
5. Conclusions
We have shown that the problem of optimizing a
portfolio under the expected shortfall measure of risk
using empirical distributions of returns is not well-defined
when the ratio N/T of assets to data points is larger than a
certain critical value. This value depends on the threshold

of the risk measure in a continuous way and this defines
a phase diagram. The smaller the value of
the longer the
length of the time series needed for portfolio optimiza-
tion. The analytical approach we have discussed in this
paper allows us to have a clear understanding of this
phase transition. The mathematical reason for the non-
feasibility of the optimization problem is that, with a
certain probability p(N, T,
), the linear constraints in (9)
define a simplex which is not bounded from below, thus
leading to a solution which is not finite (
q ! 1 in our
language), in the same way as occurs in the extreme case

! 1 discussed by Kondor et al. (2007). From a more
physical point of view, it is reasonable that the feasibility
of the problem depends on the number of data points we
take from the time series with respect to the number of
financial instruments of our portfolio. The probabilistic
character of the time series is reflected in the probability
p(N, T,
). Interestingly, this probability becomes a
threshold function at large N if N/T  1/t is finite, and
its general form is given by (28).
 On the feasibility of portfolio
1 0.4
N = 32
N = 64
0.8 N = 128
N = 256
Cost function
0.6 analytic
νopt
0.4
0.2
0
0.25 0.3 0.35 0.4 0.45 0.5
N/T
Figure 4. Numerical results from linear programming and comparison with analytical predictions at large N. Left: the minimum
cost of the optimization problem versus N/T at increasing values of N. The thick line is the analytical solution (19).
Here,
¼ 0:7; ðN=T Þ ’ 0:463. Right: the optimal value of v as found numerically for several values of N compared with the
analytical solution.
These results have practical relevance in portfolio
optimization. The order parameter discussed in this
paper is tightly related to the relative estimation error
Kondor et al. (2007). The fact that this order parameter
has been found to diverge means that, in some regions of
parameter space, the estimation error blows up, which
makes the task of portfolio optimization completely
meaningless. The divergence of the estimation error is
not limited to the case of expected shortfall: as shown by
Kondor et al. (2007), it also occurs in the case of variance
and absolute deviation, but the noise sensitivity of the
expected shortfall turns out to be even greater than that of
these more conventional risk measures.
There is nothing surprising about the fact that if there
are insufficient data, the estimation error is large and
we cannot make a good decision. What is surprising is the
fact that there is a sharply defined threshold where the
estimation error actually diverges.
For a given portfolio size, it is important to know that a
minimum number of data points is required in order to
perform an optimization based on empirical distributions.
We also note that the divergence of the parameter
at
the phase transition, which is directly related to the
fluctuations of the optimal portfolio, may play a dramatic
role in practical cases. To stress this point, we can define a
sort of ‘susceptibility’ with respect to the data,
@hwj i
tij ¼ ; ð29Þ
@xit
and one can show that this quantity diverges at the critical
point, since ij 
. A small change (or uncertainty) in xit
becomes increasingly relevant as the transition is
approached, and the portfolio optimization could then
be very unstable even in the feasible region of the phase
diagram. We stress that the susceptibility we have
introduced might be considered as a measure of the
effect of noise on portfolio selection and is very
reminiscent of the measure proposed by Pafka and
Kondor (2002).
In order to present a clean, analytic picture, we have
made several simplifying assumptions in this work.
395
0.2
0
−0.2
−0.4
−0.6
N = 32
−0.8
N = 64
−1
N = 128
−1.2 N = 256
−1.4 analytic
0.25 0.3 0.35 0.4 0.45 0.5
N/T
We have omitted the constraint on the returns, liquidity
constraints, correlations between the assets, non-station-
ary effects, etc. Some of these can be systematically taken
into account and we plan to return to these finer details in
subsequent work.
Acknowledgements
We thank O.C. Martin and M. Potters for useful
discussions, and particularly J.P. Bouchaud for critical
reading of the manuscript. S.C. is supported by the EC
through the network MTR 2002-00319, STIPCO, and
I.K. by the National Office of Research and Technology
under grant No. KCKHA005.
References
Acerbi, C. and Tasche, D., On the coherence of expected
shortfall. J. Bank. Finan., 2002, 26, 1487–1503.
Artzner, P., Delbaen, F., Eber, J.M. and Heath, D., Coherent
measures of risk. Math. Finan., 1999, 9, 203–228.
Frey, R. and McNeil, A. J., VaR and expected shortfall in
portfolios of dependent credit risks: conceptual and practical
insights. J. Bank. Finan., 2002, 26(7), 1317–1334.
Guerra, F. and Toninelli, F.L., The high temperature region of
the Viana-Bray diluted spin glass model: In honor of Gianni
Jana-Lasinio’s 70th birthday. J. Statist. Phys., 2004, 115,
531–555.
Kondor, I., Pafka, S. and Nagy, G., Noise sensitivity of
portfolio selection under various risk measures. J. Bank.
Finan., 2007, 31(5), 1545–1573.
Mézard, M., Parisi, G., Virasoro, M.A., Spin Glass Theory and
Beyond, Lecture Notes in Physics, Vol. 9, 1987 (World
Scientific: Singapore).
Pafka, S. and Kondor, I., Noisy covariance matrices and
portfolio optimization. Eur. Phys. J., 2002, B27, 277.
Pflug, G.C., Some remarks on the value-at-risk and the
conditional value-at-risk. In Probabilistic Constrained
Optimization: Methodology and Applications, edited by
S. Uryasev, 2000 (Kluwer Academic: Dordrecht).
Press, W.H., Teukolsky, S.H., Wetterling, W.T. and
Flannery, B.P., Numerical Recipes in C, 1992 (Cambridge
University Press: Cambridge).
396 S. Ciliberti et al.

Rockafellar, R. and Uryasev, S., Optimisation of conditional Talagrand, M., Spin Glasses: A Challenge for Mathematicians,
value-at-risk. J. Risk, 2000, 2, 21–41. 2002 (Springer: Berlin).

Appendix A: The replica symmetric solution The saddle point equations for q^ 0 and
q^ then allow us to
simplify this expression. The free energy ð
Þf ¼
We show in this appendix how the minimum cost function limn!0 @Zn =@n is given by
corresponding to the replica-symmetric ansatz is obtained.
The ‘Tr log Q’ term in (16) is computed by realizing 1 1
t q0
1
that the eigenvalues of such a symmetric matrix are
f ðv; q0 ;
qÞ ¼
t log  þ log
q þ
2 2 2
q
(q1 þ (n
1)q0), with multiplicity 1, and (q1
q0), with
Z
multiplicity n
1. Then,

tð1

Þv þ t dPq0 ðsÞ log B ðs; v;
qÞ;
Tr log Q ¼ log detQ ¼ logðq1 þ ðn
1Þq0 Þ
 ðA6Þ
q1
þ ðn
1Þ logðq1
q0 Þ ¼ n log
q þ þ Oðn2 Þ;

q
where the actual values of v, q0 and
q are fixed by the
ðA1Þ saddle point equations
where
q  q1
q0. The effective partition function in
(17) depends on Q
1, the elements of which are @f @f @f
¼ ¼ ¼ 0: ðA7Þ
( @v @q0 @
q

1 ab ð
q
q0 Þ=ð
qÞ2 þ OðnÞ; if a ¼ b;
ðQ Þ ¼ ðA2Þ

q0 =ð
qÞ2 þ OðnÞ; if a 6¼ b: Close inspection of these saddle point equations
allows one to perform the low temperature  ! 1
By introducing
pffiffiffiffiffiffiffiffiffiffi a Gaussian measure limit by assuming that
q ¼
/ while v and q0 do not
2
dPq0 ðsÞ  ðds= 2pq0 Þe
s =2q0 , one can show that depend on the temperature. In this limit, one can
1 ^ q1 ; q0 Þ show that
log Zðv;
n (Z
1 Y P a2 P a a 1
¼ log dxa e
ð1=2
qÞ a ðx Þ þ a ðx þvÞð
x
vÞ lim log B ðs; v;
=Þ
!1 
n a
Z ) 8
P a s þ v þ
=2; s <
v

;
ðs=
qÞ x >
> ðA8Þ
 dPq0 ðsÞ e a >
<
Z ¼
ðv þ sÞ2 =2
;
v

 s <
v;
q0 >
>
¼ þ dPq0 ðsÞ log B ðs; v;
qÞ þ OðnÞ; ðA3Þ >
:
2
q 0; s 
v:
where we have defined
If we plug this expression into equation (A6) and perform
Z 
ðx
sÞ2 the large- limit we obtain the minimum cost:
B ðs; v;
qÞ  dx exp
þ ðx þ vÞð
x
vÞ :
2
q
ðA4Þ q0
1
E ¼ lim f ¼
þ tð1

Þv
t
!1 2

The exponential in (16) now reads
exp Nn½Sðq0 ;
q; q^ 0 ;
qÞ
^ þ OðnÞ, where Z


dx 2

 pffiffiffiffiffiffiffiffiffiffi e
ðx
vÞ =2q0 x þ
Sðq0 ;
q; q^ 0 ;
qÞ
^ ¼ q0
q^ þ q^ 0
q þ
q
q^

q^
tð1

Þv
1 2pq0 2
Z
Z0

tlog  þ t dPq0 ðsÞlog B ðs;v;
qÞ t dx 2
þ pffiffiffiffiffiffiffiffiffiffi e
ðx
vÞ =2q0 x2 : ðA9Þ
 2


2pq0
t 1 q^ 0 log 2

log
q
log
q^ þ
:
2 2
q^ 2 We rescale x ! x
, v ! v
, and q0 ! q0
2, and after
ðA5Þ some algebra we obtain equation (19).