European Journal of Operational Research 207 (2010) 1599–1607

Contents lists available at ScienceDirect

European Journal of Operational Research

journal homepage: www.elsevier.com/locate/ejor

Innovative Applications of O.R.

Performance appraisal based on distance function methods

Rocío de Andrés a, José Luis García-Lapresta b,*, Jacinto González-Pachón c
a
PRESAD Research Group, Dep. de Fundamentos del Análisis Económico e H.I.E., Universidad de Valladolid, 47011 Valladolid, Spain
b
PRESAD Research Group, Dep. de Economía Aplicada, Universidad de Valladolid, 47011 Valladolid, Spain
c
Dep. de Inteligencia Artificial, Universidad Politécnica de Madrid, 28660 Boadilla del Monte, Madrid, Spain

a r t i c l e i n f o a b s t r a c t

Article history: Performance appraisal is a process used by some firms to evaluate their employees’ efficiency and pro-
Received 5 May 2009 ductivity in order to plan their promotion policy, salary policy, layoffs policy, etc. Initially this process
Accepted 11 June 2010 was just carried out by the executive staff, but recently it has evolved into an evaluation process based
Available online 18 June 2010
on the opinion of different reviewers, supervisors, collaborators, customers and the employees them-
selves (360-degree method). In such an evaluation process the reviewers evaluate some indicators
Keywords: related to employees performance appraisal. In this paper we propose an evaluation framework where
Performance appraisal
there are different sets of reviewers taking part in the evaluation process. Since reviewers have a different
Multi-criteria decision making
Finite scales
knowledge about the evaluated employee, it seems suitable to offer a flexible framework in which differ-
Extended Goal Programming approach ent reviewers can express their assessments in different finite scales according to their knowledge. The
final aim is to compute a global evaluation for each employee, that can be used by the management team
to make their decisions regarding their human resources policy. In this way, to obtain a global evaluation
for each employee, we propose a methodology able to aggregate individual valuation in a metric Lp
framework. In this context, the associated optimization problems can be reduced to an Extended Goal
Programming formulation that is very easy to compute.
Ó 2010 Elsevier B.V. All rights reserved.

1. Introduction Through an effective performance appraisal system, companies

Throughout history, organizations and companies have been
adapting themselves to established conditions to get their survival
and success. Nowadays, global competition affects most companies
and organizations. The principal organizations’ aim is to remain
competitive in this context. To adapt to changes, companies devel-
op strategies that let them be more efficient in each new context.
In this way, those strategies must be able to mould each one of
companies’ components to adapt correctly to changes. Presently,
Human Resources are a fundamental company’s component. In this
way, all strategic efforts of companies have to be headed for the
development of new methodologies by adapting human resources
to the new requirements of companies.
Little by little organizations and companies have been introduc-
ing different methods of performance appraisal to reach such sur-
vival and success (see Banks and Roberson (1985), Bretz et al.
(1992) and Miner (1998) among others). One of the objectives of
the Human Resources Department consists in obtaining effective
performance appraisal systems. Thus, companies carry out perfor-
mance appraisal processes taking into account the principal ele-
ments that affect workers in a direct or in an indirect way.
* Corresponding author. Tel.: +34 983 184 391.
E-mail address: lapresta@eco.uva.es (J.L. García-Lapresta).
can realize the following functions (see Cleveland et al. (1989)
and Fisher et al. (2006)):
 Developmental uses: to measure performance goals, to relocate
employees for developmental purposes, to identify employees
training needs, etc.
 Administrative uses: salary, promotion, retention or termina-
tion, layoffs, discipline, etc.
 Organizational maintenance: human resource planning, to deter-
mine organization training needs, to evaluate organizational goal
achievement, to evaluate human resource systems, etc.
 Documentation: document human resource decisions and help
meet legal requirements.
Many companies tend to use informal methods, where only
supervisors evaluate employees and often develop formal perfor-
mance but only used by supervisors to evaluate employees. Due
to this fact, the results of these kinds of evaluation processes are
often biased and subjective, can present the halo effect (cognitive
turn which makes to think that some limited characteristics are
applied to everything), can be based on prejudice (about religion,
race, accent, gender, nationality, etc.), etc. (see Latham and Wexley
(1981), Kerr (1985), Frunzi and Savini (1996) and Fisher et al.
(2006), among others).

0377-2217/$ - see front matter Ó 2010 Elsevier B.V. All rights reserved.
doi:10.1016/j.ejor.2010.06.012
1600 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607
There is another kind of method which uses information from
many people who can truly respond to know how an employee
performs on the job. The so-called 360-degree appraisal or integral
evaluation is a mechanism for evaluating worker’s performance
based on judgment from everyone with whom the employee
comes in contact: supervisors, collaborators, colleagues, customers
and oneself included (see Edwards and Ewen (1996)). The 360-
degree appraisal system (see Fig. 1) presents some advantages with
regard to the traditional systems and it can be emphasized (see
Banks and Roberson (1985), Edwards and Ewen (1996), Baron
and Kreps (1999), Jarman (1999), Fletcher (2001), Pfau and Kay
(2002), among others):
 It is more extensive due to the fact that integral evaluation col-
lects information from different points of view.
 It can reduce the bias and prejudice because the information
comes from several people, not just one.
 It encourages the Human Resources Department to establish
policies of internal selection more clearly based on the results
of the evaluation process.
 From this evaluation system the Human Resources Department
can define training and development plans for employees based
on individual and group performance appraisal results.
 It allows companies to identify successful people more easily
with the aim of reinforcing, recognizing and encouraging their
results.
In order to overcome the drawbacks associated with the tradi-
tional performance appraisal process and to adopt the advantages
of the integral evaluation, in this paper, we propose a 360-degree
appraisal model where different sets of reviewers have to evaluate
employees attending to different criteria and attributes. Taking
into account that the different groups of reviewers may have a dif-
ferent degree of knowledge about the evaluated employees, we
then suggest a flexible evaluation framework where the appraisers
can express their assessments in different scales according to their
knowledge and expertise; such scales can come from a quantitative
or qualitative nature (see Baron and Kreps (1999)). In the litera-
ture, it is possible to find performance appraisal processes based
on quantitative information and/or on qualitative information
depending on the kind of criterion to be evaluated, although most
of them represent such information in a numerical way. In this pa-
per, we represent the information from the reviewers in a numer-
ical way with the purpose of simplifying the computing task.
Nevertheless, the information may be represented by means of lin-
guistic variables following the methodology proposed in de Andrés
et al. (2010).
Although the aggregation of the information in performance
appraisal is an essential process, most of the companies which usu-
ally carry it out by use of weighted averages without taking into
Supervisors
Colaborators Employee Co-workers
Customers
Fig. 1. 360-degree appraisal.
account whether this type of aggregation operator has or has not
got good properties. Our proposal is to analyze the aggregation
problem from a different point of view by means of the use of a
Lp metric framework where every associated optimization problem
can be formulated as a Goal Programming problem that is very
easy to compute. For this objective, we follow the approach pro-
posed by González-Pachón and Romero (2006, 2007) for aggregat-
ing individual evaluations because this method allows us to carry
out the aggregation procedure from two different and, sometimes,
incompatible social principles: the government of the majority and
the consideration to minorities. Moreover, to consider sensitivity
with respect to extreme disagreement evaluations, alternative
solutions may be obtained through convex combinations of the
previous points of view.
The paper is organized as follows. Section 2 includes the frame-
work of our integral evaluation proposal. Section 3 is devoted to
explain the unification information phase, the aggregation phase
and exploitation phase in our model. Next, in Section 4 we propose
an illustrative example. Finally, some concluding remarks are in-
cluded in Section 5.
2. Integral evaluation framework
In this section, the framework of our proposal for an integral
evaluation model is introduced.
In order to show how a company could carry out an integral
evaluation process, let us suppose a company which wants to eval-
uate the performance of their employees. It is supposed there is a
set of employees X = {x1, . . . , xn} to be evaluated by the following
collectives:
 A set of supervisors or the management team: A = {a1 , . . . , ar}.
 A set of colleagues, partners or co-workers: B = {b1 , . . . , bs}.
 A set of customers and/or subordinates: C = {c1 , . . . , ct}.
We consider several criteria Y1, . . . , Yq for evaluating each em-
ployee. We notice there is a general set of criteria to evaluate
employees but each collective of reviewers could only assess
employees attending to a number of them following the guidelines
established by the Human Resources Department.
By means of aik
ik ik
j ; bj and c j we will denote the assessments of
ai 2 A, bi 2 B and ci 2 C on the candidate xj according to the compe-
tency Yk, respectively. Therefore, there are at most (r + s + t)q
assessments for each employee provided by the three collectives.
As we have previously mentioned, the members of the different
collectives usually have a different degree of knowledge about
evaluated employees. So, we assume that each collective can use
different finite scale of real numbers to express their assessments
to each employee for each criterion Yk:
 Supervisors
SkA ¼ fsAk Ak
1 ; . . . ; sf g  R; sAk Ak
1 <    < sf :
 Collaborators
SkB ¼ fsBk Bk
1 ; . . . ; sg g  R; sBk Bk
1 <    < sg :
 Customers and/or subordinates
SkC ¼ fsCk Ck
1 ; . . . ; sh g  R; sCk Ck
1 <    < sh :
It takes notice of each term of a numerical scale that can be de-
scribed by a linguistic label following Likert’s methodology (see
Likert (1932)).
In the literature (see Antes et al. (1998), Bouyssou et al. (2000),
Arfi (2005) and Martínez (2007)), the use of the decision analysis to
 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607
accomplish evaluation processes has got good results. Therefore, in
this paper we shall propose a performance appraisal process based
on a classical decision analysis approach (see Clemen (1995) and
Martínez (2007)) with three phases (see Fig. 3):
(1) Normalization information phase: Our proposal considers that
reviewers can express their opinions about employees in dif-
ferent scales of real numbers according to their knowledge.
Therefore and before carrying out the aggregation process,
it is necessary to standardize or normalize all gathered infor-
mation into a unique domain, the interval [0, 1].
(2) Aggregation phase: Once the normalization phase has been
achieved, the aggregation process can start. Our aggregation
process is based on the distance aggregation method pro-
posed by González-Pachón and Romero (1999) and Romero
(2001). In this way and to obtain a final value for each
employee xj is carried out an aggregation procedure with
three different steps (in this moment we do not pay atten-
tion to the way we aggregate the information; it is included
in Section 3). These steps consist of (see Fig. 2):
(a) Computing collective criteria values. The aggregation pro-
cess is initiated by obtaining a collective value for each
employee xj taking into account the assessments of every
collective for each criterion Yk.
Aggregation process
Collective criteria values
Supervisors Collaborators
Customers Employees
Global criteria values
Global values
Fig. 2. Steps of the aggregation process.
Fig. 3. Decision scheme for integral evaluation process.
1601
(b) Computing global criteria values. In the second stage we
aggregate the previous reviewers’ collective values
providing a global value for each criterion to every
employee.
(c) Computing global value. Finally, is obtained a global
assessment for each employee aggregating the previous
global criteria values.
All outcomes can be used for sorting and ranking employees in
order to establish the human resources policy.
(3) Rating phase. In the exploitation phase, companies are going
to classify and order employees x1, . . . , xn to adhere to the
Human Resources policy.
Some models of performance appraisal include the opinion of
each employee about her/himself. Although the opinion of each
employee about her/himself can be useful for the organization,
companies do not take into account this information in the aggre-
gation process because including the employees’ self-evaluation
could disturb the aggregation phase, that means the corresponding
outcomes could be biased by self-evaluations. However, opinions
of employees about themselves are used by companies in the rat-
ing phase with the purpose of improving their performance and
synchronizing companies goals with employees goals (see Fig. 3).
This part of performance appraisal procedure is so-called feedback
and coaching process (see Frunzi and Savini (1996) and Fisher et al.
(2006)).
3. Distance-based aggregation procedure for performance
appraisal
In this section we carry out in detail the different phases of per-
formance appraisal process mentioned above.
3.1. The normalization phase
In order to aggregate properly the assessments from reviewers,
all gathered information should be unified. Therefore, all informa-
tion is standardized in the interval [0, 1] in the normalization
phase.
 For supervisors. Taking into account the scale used by supervi-
sors to evaluate each employee for the criterion Yk, the normal-
ized previous scale is defined as:
SkA ¼ fsAk Ak
1 ; . . . ; sf g  ½0; 1;
where
1602 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607

sAk Ak
m  s1 Notice that different target values y* are obtained for different
sAk
m ¼ 2 ½0; 1; m ¼ 1; . . . ; f :
sf  sAk
Ak
1
metrics Lp. We also note that:

Similarly we can standardize the individual assessments in the

 If we consider low values of p, then the problem pays more
interval [0, 1] for the remaining
n collectives.
o
sBk Bk
m s1
attention to medium values.
 For collaborators. SkB ¼ sBk ; . . . ; sBk  ½0; 1, where sBk
m ¼ sBk sBk 2
1 g
g 1  If we consider high values of p, then the problem pays
½0; 1; m ¼ 1; . . . ; g.  Ck 
k Ck
more attention to the extreme values. If we consider the
 For customers and/or subordinates. SC ¼ s1 ; . . . ; sh  ½0; 1,
sCk Ck
m s1
limit case p = 1, this converts the Goal Programming problem
where sCk
m ¼ sCk sCk 2 ½0; 1; m ¼ 1; . . . ; h.
h 1 into the following MINIMAX (Chebyshev) formulation, where
the disagreement of the most displaced assessments is mini-
3.2. The aggregation phase mized (see for instance Ignizio and Caliver (1994) and Romero
(2001)):
One of the aims of the aggregation phase is to obtain a value
that assesses the performance of the evaluated worker according
min D
to the different collectives that have evaluated her/him. To do that,
the assessments provided by the members of different collectives s:t: w1 ðg1 þ q1 Þ 6 D;
will be aggregated. The aggregation procedure mentioned in ...
Section 2 can be developed through several methodologies.
Decisional problems with multiple attributes, decision variables wm ðgm þ qm Þ 6 D;
and restrictions are not easy to be solved by means of classical mul- y þ g1  q1 ¼ y1 ;
ti-objective optimization methods. For this reason it seems suitable ð2Þ
...
to use another type of more flexible methodologies, inside of these
methodology lines Goal Programming (GP) can be included. GP tech- y þ gm  qm ¼ ym ;
niques are not based on the objectives optimization but are based 0 6 y 6 1;
on ‘‘satisficing” philosophy (see Simon (1957)). This methodology af-
firms that companies decisional problems lack complete informa- g1 P 0; . . . ; gm P 0;
tion, have more than one manifold objective (that in addition q1 P 0; . . . ; qm P 0:
they present disagreement among them), limited resources, etc.
In this paper we propose an operational method to aggregate where D represents the disagreement of the social group or individ-
information in a distance-based framework by a Goal Program- ual with the opinions more displaced (i.e., more significantly differ-
ming formulation, see González-Pachón and Romero (1999, 2001). ent) from the consensus obtained.
In order to present the aggregation procedure, we consider y* to One of the main problems in this kind of multi-criteria prob-
be the target value from the individual assessments y1,. . .,ym, lems is to select the right value of p which captures the essential
characteristics of real problems. Romero (2001) proposes a general
Aggðy1 ; . . . ; ym Þ ¼ y : Goal Programming structure called Extended Goal Programming
The problem is how to determine y*. This target value represents which includes cases (1) and (2):
the unknown value of our aggregation problem. It can be obtained X
m
by formulating a distance function model as a measure of disagree- min ð1  kÞD þ k ðwi ðgi þ qi ÞÞ
ment between the individual assessments y1, . . . , ym and the desired i¼1

target value y*. Then, we deal with the following problem formu- s:t: w1 ðg1 þ q1 Þ 6 D;
lated for the Minkowski metric Lp (see Yu (1973)), for p P 1:
...
!1=p
  X
m
wm ðgm þ qm Þ 6 D;
min d ðy1 ; . . . ; ym Þ; ðy ; . . . ; y Þ ¼ min wpi jyi  y jp ;
i¼1 y þ g1  q1 ¼ y1 ; ð3Þ
where wi is the parameter attached to the discrepancy between the ...
achievement of the i-th criterion and the target value.
The resolution of this problem is not easy because of the non- y þ gm  qm ¼ ym ;
linear and non-differentiable character of the function. In order 0 6 y 6 1;
to avoid some of the above disadvantages in the preceding func-
tion, the previous problem can be transformed into the following
g1 P 0; . . . ; gm P 0;
Goal Programming problem (see Romero (1991), González-Pachón q1 P 0; . . . ; qm P 0;
and Romero (1999), Dopazo and González-Pachón (2003) and Gon-
zález-Pachón and Romero (2006)): where the value of k 2 [0, 1] states the balance between the minimi-
zation of the maximum lack of success and the minimization of the
X
m
min ðwi ðgi þ qi ÞÞp weighted sum of the deviation variables in relation to the target
i¼1 values.
s:t: y þ g1  q1 ¼ y1 ;

González-Pachón and Romero (2006, 2007) considered the Ex-
... tended Goal Programming formulation to propose a method for
ð1Þ
y þ gm  qm ¼ ym ; aggregating individual valuations. In their formulation, the p
0 6 y 6 1; parameter is interpreted as a compensatory parameter between
g1 P 0; . . . ; gm P 0; two desirable social principles in all consensus process: the gov-
q1 P 0; . . . ; qm P 0; ernment of the majority, p = 1, and the consideration to minorities,
p = 1. Thus, as p 2 [1, 1] increases, more importance is given to
where gi and qi are the negative and positive deviation variables, the biggest disagreements valuations (see Yu (1973)). To consider
respectively that measure the difference among the target value sensitivity with respect to extreme disagreement valuations, an
and the individual assessments. alternative solution may be obtained through a convex combina-
 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607 1603

tion of metric Lp for p = 1 and p = 1 using a value for k  parame- 3 

X 
ter. For k = 1, the consensus valuation is defined by minimizing min ð1  kk ÞD þ kk wki ðgji þ qji Þ
i¼1
metric Lp for p = 1 (principle of majority). For k = 0, the consensus
valuation is defined by minimizing metric Lp for p = 1 (principle s:t: wk1 ðgj1 þ qj1 Þ 6 D;
of minority). For control parameter k values belonging to the open wk2 ðgj2 þ qj2 Þ 6 D;
interval (0, 1), compromise consensus valuations (if they exist)
could be obtained. wk3 ðgj3 þ qj3 Þ 6 D;
xk k
j þ gj1  qj1 ¼ aj ;
3.2.1. The aggregation steps k
We now show how to aggregate the information provided by xk
j þ gj2  qj2 ¼ bj ;

reviewers following the steps presented in Section 2. For this pur- xk k
j þ gj3  qj3 ¼ cj ;
pose, we consider the Extended Goal Programming formulation
that appeared previously. 0 6 xk
j 6 1;

gj1 P 0; gj2 P 0; gj3 P 0;

 Collective criteria values. For each employee xj, criterion Yk and
qj1 P 0; qj2 P 0; qj3 P 0:
set of reviewers, we aggregate the individual assessments.
– For supervisors:
k
  Obviously, we can consider different values of wki for each criterion
v kA ðxj Þ ¼ Ak A a1k rk
j ; . . . ; aj ¼ ak
j 2 ½0; 1; Yk, and different values of k.
 Global value.
by solving the following problem  
q
r 
X  v ðxj Þ ¼ Ak x1
j ; . . . ; xj ¼ xj 2 ½0; 1;
min ð1  kkA ÞD þ kkA wAk
i ðgji þ qji Þ
i¼1 by solving the following problem
s:t: wAk g þ qj1 Þ 6 D; q 
X 
1 ð j1
min ð1  kÞD þ k wi ðgji þ qji Þ
... i¼1

wAk g þ qjr Þ 6 D; s:t: w1 ðgj1 þ qj1 Þ 6 D;

r ð jr
...
ak
j þ gj1  qj1 ¼ a1k
j ;
wq ðgjq þ qjq Þ 6 D;
...
xj þ gj1  qj1 ¼ x1
j ;
rk
ak
j þ gjr  qjr ¼ aj ; ...
0 6 ak
j 6 1;
xj þ gjq  qjq ¼ xq
j ;

gj1 P 0; . . . ; gjr P 0 0 6 xj 6 1

qji P 0; . . . ; qjr P 0: gj1 P 0; . . . ; gjq P 0

qj1 P 0; . . . ; qjq P 0:
Similarly we can aggregate the individual assessments for the
remaining collectives. 3.3. Rating phase
– For collaborators:
  In the exploitation phase, the management team shall classify
k 1k sk k
v kB ðxj Þ ¼ Ak B bj ; . . . ; bj ¼ bj 2 ½0; 1; and order employees x1, . . ., xn according to:

– For customers and/or subordinates: (1) The collective values for each criterion (see Table 1).
  (2) The global criteria values (see Table 2).
kkC
v kC ðxj Þ ¼ A c1k tk
j ; . . . ; cj ¼ ck
j 2 ½0; 1: (3) The global values, x1 ; . . . ; xn .
(4) The self-values of each employee for each criterion.
Although the aggregation process is similar in every set of review-
ers, it is possible to consider different values of w i for each review- The opinions of employees about themselves are taken into ac-
ers’ collective and for each criterion. Generally, the weighting count in this stage of the evaluation process due to the fact that
vectors used in the aggregation process are fixed by the Human one of the most important aspect of an integral evaluation process
Resources Department according to the goals established in their is providing performance feedback to employees (see Burns and
human resources policy. Flanagan (1955), Latham and Wexley (1981), Banks and Roberson
In the same way, different values of k  for each group of reviewers (1985), Bretz et al. (1992), Bernardin et al. (1995), Baron and Kreps
and each criterion, kkA ; kkB and kkC , can be considered. If companies (1999), Fletcher (2001) and Fisher et al. (2006), among others). In
want to obtain an equitable aggregation process, the values of k 
should be near 0, but if they want to obtain an efficient aggregation
process, the values of k  should be near 1. Thus, for several values
 Table 1
of wi and k we will obtain different points of view to get the eval-
Collective values for Yk.
uation for each employee xj.
 Global criteria values. Reviewers Reviewers’ collective values
  Supervisors ak k
k 1 ; . . . ; an
v k ðxj Þ ¼ Ak akj ; bkj ; ckj ¼ xkj 2 ½0; 1; Colleagues k k
b1 ; . . . ; bn
Customers ck k
1 ; . . . ; cn
by solving the following problem
1604 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607

Table 2 employees. In our example, the Human Resource manager selects

Global criteria values. the following groups of reviewers:
Criterion Global criteria values
Y1 x1 1  A set of four supervisors A = {a1, . . . , a4}.
1 ; . . . ; xn
   A set of eight collaborators B = {b1, . . . , b8}.
Yq xq q
1 ; . . . ; xn
 A set of twelve subordinates C = {c1, . . . , c12}.

Members of each group of reviewers express their evaluations

about workers through the following finite scales of real numbers,
order to improve the employees’ performance, workers need to
according to their knowledge about evaluated employees:
know their results in the evaluation process. Most theories of work
motivation indicate that the feedback of the results to employees is
 Supervisors:
an important part of the performance appraisal system because
   
employees must know how well they do their work and how to im- S1A ¼ sA1 A1
1 ; . . . ; s7 ¼ f1; . . . ; 7g; S2A ¼ sA2 A2
1 ; . . . ; s7
prove their weak points.
¼ f1; . . . ; 7g;
Once the company obtains the results of the performance ap-
praisal process, the Human Resources Department management
analyzes them, ranks and sorts employees taking into consider-    
ation the goals fixed by the company. S3A ¼ sA3 A3
1 ; . . . ; s9 ¼ f1; . . . ; 9g; S4A ¼ sA4 A4 A4
1 ; s2 ; s3

¼ f1; 2; 3g;
4. An illustrative example
   
In this section we present a simple example in order to show S5A ¼ sA5 A5
1 ; . . . ; s5 ¼ f1; . . . ; 5g and S6A ¼ sA6 A6
1 ; . . . ; s5
how companies could carry out their performance process by ¼ f1; . . . ; 5g:
means of the previously proposed model.
With this intention, let us suppose an engineering company  Collaborators:
dealing in the construction industry is going to carry out a 360-de-    
gree assessment over their employees. S4B ¼ sB4 B4
1 ; . . . ; s5 ¼ f1; . . . ; 5g; S5B ¼ sB5 B5
1 ; . . . ; s9
 
Before showing the scheme of the performance process for this ¼ f1; . . . ; 9g and S6B ¼ sB6 B6
1 ; . . . ; s7 ¼ f1; . . . ; 7g:
company, it is remarkable that the market of public works is small
because of the big competition among companies in this economy  Subordinates:
sector, as well as the big capital that is necessary to be invested by  
the companies in order to develop projects. Due to these facts, S4C ¼ sC4 C4
1 ; . . . ; s5 ¼ f1; . . . ; 5g:
these kinds of companies need to efficiently qualify employees
and to plan carefully investments in equipments and tools; to carry
We can note that all collectives of reviewers do not evaluate
out this training, civil engineering companies have turned to the
employees attending to the same criteria. For example, subordi-
services of Human Resources consultants to develop and realize
nates only evaluate employees with respect to the criterion Y4
performance appraisal process.
due to the fact that they do not have enough knowledge to evalu-
In this example the company’s goal is to determine the level of
ate employees attending to the other criteria.
efficiency of each employee in order to develop Human Resources
In Tables 10–12 the assessments provided by the reviewers
policies and to achieve an effective personnel management. For
about employees x1 and x2 for the different criteria are indicated.
this purpose, the company is carrying out a 360-degree assessment
Once all information has been gathered, we follow the scheme
over their employees of the Civil Engineering Department which
presented in Section 2 to obtain the different values which allow
involves evaluations from supervisors, collaborators, subordinates
the company to rank employees according to its Human Resources
and employees themselves.
policy.
To facilitate the process and the calculations we consider, with-
First, we carry out the normalization phase. All information is
out loss of generality, two employees to be evaluated, x1, x2,
standardized in the interval [0, 1] following the method showed
according to the following criteria (similar to someone used by
in Section 3.1. The following normalized scales are obtained:
other companies to carry out performance appraisal (see Fisher
et al. (2006))):
 For supervisors:
 Y1: Knowledge of position. Degree of adaptation between 
  1 2 3 4 5
employee and his/her job through the experience, level of edu- S1A ¼ sA1 A1 ¼
1 ; . . . ; s7 0; ; ; ; ; ; 1 ;
6 6 6 6 6
cation, specialized training, etc.
 Y2: Responsibility. Degree of dedication to the work within 
  1 2 3 4 5
established norms. S2A ¼ sA1 A1 ¼
1 ; . . . ; s7 0; ; ; ; ; ; 1 ;
 Y3: Initiative. Trend to contribute, develop and to carry out new
6 6 6 6 6
ideas or methods. 
  1 2 3 4 5 6 7
 Y4: Communication. Skill of making easier the relationships S3A ¼ sA3 A3 ¼
1 ; . . . ; s9 0; ; ; ; ; ; ; ; 1 ;
among supervisors, collaborators, subordinates, etc. 8 8 8 8 8 8 8
 Y5: Cooperation. Ability to facilitate the teamwork. 
  1
 Y6: Finishing projects on time. Characteristic of being able to S4A ¼ sA4 A4 A4 ¼
1 ; s2 ; s3 0; ; 1 ;
end punctually a required task work. 2

  1 2 3
The reviewers that are going to take part in the evaluation pro- S5A ¼ sA5 A5 ¼
1 ; . . . ; s5 0; ; ; ; 1 ;
cess should be people who interact daily with the evaluated 4 4 4
 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607 1605


  1 2 3 Table 5
S6A ¼ sA6
1 ; . . . ; 
s A6
5 ¼ 0; ; ; ; 1 : Collective criteria values for employee x1.
4 4 4
Supervisors Collaborators Subordinates
 For collaborators:
 Y1 v 1
A ðx1 Þ ¼1 – –
  1 2 3 Y2 v 2
¼ 0:33 – –
S4B ¼ sB4 B4 ¼ A ðx1 Þ
1 ; . . . ; s5 0; ; ; ; 1 ;
4 4 4 Y3 v 3
A ðx1 Þ ¼ 0:687 – –
Y4 v 4
A ðx1 Þ ¼1 v 4B ðx1 Þ ¼ 0:75 v 4C ðx1 Þ ¼ 0:5
 Y5 v 5
v 5B ðx1 Þ ¼ 0:687 –
  1 2 3 4 5 6 7 A ðx1 Þ ¼ 0:75
S5B ¼ sB5 B5 ¼
1 ; . . . ; s9 0; ; ; ; ; ; ; ; 1 ; Y6 v 6
A ðx1 Þ ¼ 0:75 v 6B ðx1 Þ ¼ 0:833 –
8 8 8 8 8 8 8

  1 2 3 4 5 Table 6
S6B ¼ sB6 B6 ¼
1 ; . . . ; s7 0; ; ; ; ; ; 1 : Collective criteria values for employee x2.
6 6 6 6 6
Supervisors Collaborators Subordinates
 For subordinates:
 Y1 v 1
A ðx2 Þ ¼ 0:5 – –
  1 2 3 Y2 v 2
¼ 0:75 – –
S4C ¼ sC4 C4 ¼ A ðx2 Þ
1 ; . . . ; s5 0; ; ; ; 1 :
4 4 4 Y3 v 3
A ðx2 Þ ¼ 0:812 – –
Y4 v 4
A ðx2 Þ ¼ 0:5 v 4B ðx2 Þ ¼ 0:5 v 4C ðx2 Þ ¼ 1
Y5 v 5
A ðx2 Þ ¼ 0:5 v 5B ðx2 Þ ¼ 0:625 –
After the normalization phase, the aggregation process can Y6 v 6
¼1 v 6B ðx2 Þ ¼ 1 –
A ðx2 Þ
start. The weights in each step of the aggregation phase have been
selected by the company attending to the following principles:
Table 7
 The importance of the reviewers belonging to the same collec- Global criteria values for employees x1 and x2.
tive is equal for each criterion (see Table 3). x1 x2
 For computing the global criteria values, the company considers 1
v (x1) = 1 v1(x2) = 0.5
supervisors and collaborators collectives more important than v2(x1) = 0.33 v2(x2) = 0.75
subordinates group for all criteria because the subordinates v3(x1) = 0.687 v3(x2) = 0.812
assessments do not play a fundamental role in the evaluation v4(x1) = 0.75 v4(x2) = 0.5
process (see Table 4). v5(x1) = 0.687 v5(x2) = 0.583
v6(x1) = 0.75 v6(x2) = 1
 In order to calculate the global value for each employee, the
company assigns each criterion to the same level of significant,
wi ¼ 16 for each criterion. In this way, we obtain a collective criteria value for each em-
ployee and each criterion, and a global criteria value for each em-
We note that different collective criteria values, different global ployee and each criterion.
criteria values and different global values can be obtained for dif- In order to show in detail the last step of the aggregation pro-
ferent values of k. Here we only consider k = 0.5 to calculate collec- cess, we now present the procedures of computing the global value
tive criteria values and global criteria values. for each employee.
In Tables 5–7, we now show assessments obtained for each em-
ployee in the first and second step of the aggregation process, tak-  Employee x1:
 
ing into account the different weights, groups of reviewers and v ðx1 Þ ¼ Ak x1 2 3 4 5 6
1 ; x1 ; x1 ; x1 ; x1 ; x1 ¼ x1 2 ½0; 1;
criteria.
by solving the following problem
X
6
min ð1  kÞD þ k ðwi ðg1i þ q1i ÞÞ
Table 3
i¼1
Weights for computing collective global criteria values.
s:t: w1 ðg11 þ q11 Þ 6 D;
Supervisors for Collaborators for Subordinates for
w2 ðg12 þ q12 Þ 6 D;
i = 1, . . . , 4 i = 1, . . . , 8 i = 1, . . . , 12
w3 ðg13 þ q13 Þ 6 D;
Y1 wA1 1
i ¼ 4
– –
Y2 – –
w4 ðg14 þ q14 Þ 6 D;
wA2
i ¼ 1
4
Y3 wA3 ¼ 1 – – w5 ðg15 þ q15 Þ 6 D;
i 4
Y4 wA4
i ¼ 1
4 wB4 1
i ¼ 8 wC4 1
i ¼ 12
w6 ðg16 þ q16 Þ 6 D;
Y5 wA5 ¼ 1
wB5 1
i ¼ 8
– x1 þ g11  q11 ¼ x1
i 4 1 ;
Y6 wA6 ¼ 1
wB6 1
i ¼ 8
–
i 4
x1 þ g12  q12 ¼ x2
1 ;

x1 þ g13  q13 ¼ x3

1 ;

x1 þ g14  q14 ¼ x4

1 ;
Table 4
x1 þ g15  q15 ¼ x5
1 ;
Weights for computing global criteria values.
x1 þ g16  q16 ¼ x6
1 ;
Supervisors Collaborators Subordinates
0 6 x1 6 1;
Y1 w11 ¼ 1 – –
g11 P 0; g12 P 0; g13 P 0; g14 P 0; g15 P 0; g16 P 0;
Y2 w21 ¼ 1 – –
Y3 w31 ¼1 – –
q11 P 0; q12 P 0; q13 P 0; q14 P 0; q15 P 0; q16 P 0:
Y4 w41 ¼ 25 w42 ¼ 25 w43 ¼ 15
Y5 w51 ¼ 13 w52 ¼ 23 –  Employee x2:
Y6 w61 ¼ 12 w62 ¼ 12 –  
v ðx2 Þ ¼ Ak x1 2 3 4 5 6
2 ; x2 ; x2 ; x2 ; x2 ; x2 ¼ x2 2 ½0; 1;
1606 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607

by solving the following problem Table 12

Customers assessments for Y4 and each employee.
X
6
c1 c2 c3 c4 c5 c6 c7 c8 c9 c10 c11 c12
min ð1  kÞD þ k ðwi ðg2i þ q2i ÞÞ
i¼1 x1 sC4
5 sC4
2 sC4
4 sC4
1 sC4
3 sC4
5 sC4
2 sC4
5 sC4
5 sC4
1 sC4
4 sC4
3

s:t: w1 ðg21 þ q21 Þ 6 D; x2 sC4

5 sC4
5 sC4
5 sC4
5 sC4
5 sC4
4 sC4
4 sC4
2 sC4
4 sC4
4 sC4
4 sC4
5

w2 ðg22 þ q22 Þ 6 D;
w3 ðg23 þ q23 Þ 6 D;
 If companies would like to obtain an efficiency ranking proce-
w4 ðg24 þ q24 Þ 6 D;
dure, they will consider k = 0. In this case,
w5 ðg25 þ q25 Þ 6 D;
w6 ðg26 þ q26 Þ 6 D; v ðx1 Þ ¼ 0:665 < 0:75 ¼ v ðx2 Þ:
x2 þ g21  q21 ¼ x1
2 ;
 If companies would like to obtain an equitable ranking proce-
dure, they will consider k = 1. In this case,
x2 þ g22  q22 ¼ x2
2 ;

x2 þ g23  q23 ¼ x3

2 ;
v ðx1 Þ ¼ 0:687 > 0:583 ¼ v ðx2 Þ;
x2 þ g24  q24 ¼ x4
2 ; and the order of the ranking changes.
x2 þ g25  q25 ¼ x5
2 ;
5. Conclusions
x2 þ g26  q26 ¼ x6
2 ;
06 x2 6 1;
Due to the fact that the 360-degree performance appraisal pro-
g21 P 0; g22 P 0; g23 P 0; g24 P 0; g25 P 0; g26 P 0; cess incorporates assessments from different groups of reviewers,
q21 P 0; q22 P 0; q23 P 0; q24 P 0; q25 P 0; q26 P 0: it would be necessary to use a formal methodology to suitably
manage such all information and to properly carry it out. Moreover
and taking into account that in traditional 360-degree performance
In order to show how k can be used by companies to decide appraisal methods, opinions are expressed in the same expression
about the degree of efficiency and equity in the performance ap- scale without considering the different knowledge levels of the
praisal process, we consider different values of k to calculate the reviewers on employees, in this paper we propose a flexible 360-
global values for employees x1 and x2 (see Tables 8 and 9). degree performance appraisal model which provides the following
We can note that: advantages:

Table 8  It can be adapted to different companies conditions in order to

Global values for employee x1.
facilitate the Human Resources decision making processes.
k=0 k = 0.5 k=1  It is based on the Decision Making Theory.
Global values v(x1) = 0.665 v(x1) = 0.687 v(x1) = 0.687  It has been developed on a flexible framework capable of work-
ing with different evaluation scales. Specifically, the collectives
of reviewers could use different numerical scales to evaluate
Table 9 employees attending to each criterion in order to improve the
Global values for employee x2.
precision of the process results and taking into account the
k=0 k = 0.5 k=1 degree of knowledge that the different appraisers have about
Global values v(x2) = 0.75 v(x2) = 0.75 v(x2) = 0.583 employees.

Since our proposal of the 360-degree appraisal process takes

Table 10 into account the assessments of different collective of reviewers,
Supervisors assessments for each employee and each criterion. a variety of scenarios based on social principles could be consid-
x1 x2 ered; for instance, the government of the majority or the consider-
ation to minorities. To deal with these kinds of scenarios, we have
Y1 Y2 Y3 Y4 Y5 Y6 Y1 Y2 Y3 Y4 Y5 Y6
proposed a distance-based framework (Lp-metrics) for aggregating
a1 sA1 sA2 sA3 sA4 sA5 sA6 sA1 sA2 sA3 sA4 sA5 sA6
7 4 9 3 5 5 3 4 9 3 3 5 individual valuations. In this context, the p parameter has a prefer-
a2 sA1 sA2 sA3 sA4 sA5 sA6 sA1 sA2 sA3 sA4 sA5 sA6
6 3 7 3 5 4 2 7 8 2 3 5 ential interpretation in terms of the sensitiveness towards assess-
a3 sA1 sA2 sA3 sA4 sA5 sA6 sA1 sA2 sA3 sA4 sA5 sA6
7 2 6 3 3 4 2 6 6 2 4 5
ments’ dispersion.
a4 sA1
7 sA2
3 sA3
4 sA4
2 sA5
3 sA6
4 sA1
1 sA2
4 sA3
6 sA4
1 sA5
3 sA6
4
The computational complexity of a general Lp-metric based
model can be avoided by an Extended GP formulation. Now, the
Table 11 control parameter, k, can be interpreted as a marginal rate of sub-
Collaborators assessments for each employee and each criterion. stitution between the ‘‘principle of majority” and the ‘‘principle of
x1 x2
minority”. Furthermore, the required computational burden is very
low. In fact, we only need to solve Linear Programming problems.
Y4 Y5 Y6 Y4 Y5 Y6
Thus, different values of w and k can be used for reaching aggrega-
b1 sB4
4 sB5
9 sB6
6 sB4
4 sB5
8 sB6
5 tion values by means of different points of view and sensitiveness
b2 sB4
4 sB5
8 sB6
7 sB4
4 sB5
6 sB6
5 towards assessments’ dispersion. A computational experiment for
b3 sB4
5 sB5
7 sB6
5 sB4
4 sB5
7 sB6
5 making sensitivity analysis on these parameters could be consid-
b4 sB4 sB5 sB6 sB4 sB5 sB6
4 5 6 4 5 5 ered as future research.
b5 sB4 sB5 sB6 sB4 sB5 sB6
5 6 4 4 7 5
We have obtained not only a global assessment for each em-
b6 sB4
3 sB5
7 sB6
5 sB4
4 sB5
4 sB6
4
ployee, but intermediate values according to the opinions of each
b7 sB4
3 sB5
4 sB6
6 sB4
5 sB5
4 sB6
5
set of reviewers and criteria, and a global assessment for each cri-
b8 sB4
4 sB5
5 sB6
6 sB4
3 sB5
4 sB6
4
terion. It is worth emphasizing that the proposed model is quite
 R. de Andrés et al. / European Journal of Operational Research 207 (2010) 1599–1607
flexible in allowing the management team to know how to aggre-
gate the individual opinions.
Acknowledgements
The authors thank the two anonymous referees for their
comments and suggestions in helping us to improve the paper.
de Andrés and García-Lapresta gratefully acknowledge the funding
support of the Spanish Ministerio de Ciencia e Innovación (Project
ECO2009–07332), ERDF and the Junta de Castilla y León (Conse-
jería de Educación; Project VA092A08).
References
Antes, J., Campen, L., Derige, U., Titza, C., Wolle, G.D., 1998. A model-based decision
support system for the evaluation of flight schedules for cargo airlines. Decision
Support Systems 22, 307–323.
Arfi, B., 2005. Fuzzy decision making in politics: A linguistic fuzzy-set approach.
Political Analysis 13, 23–56.
Banks, C.G., Roberson, L., 1985. Performance appraisers as test developers. Academy
of Management Review 10, 128–142.
Baron, J.N., Kreps, D.M., 1999. Strategic Human Resources, Frameworks for General
Managers. Wiley & Sons, New York.
Bernardin, H.J., Kane, J.S., Ross, S., Spina, J.D., Johnson, D.L., 1995. Handbook of
Human Resources Management. Blackwell, Cambridge.
Bouyssou, D., Marchant, T., Pirlot, M., Perny, P., Tsoukiàs, A., 2000. Evaluation and
Decision Models: A Critical Perspective. Kluwer Academic Publishers.
Bretz, R.D., Milkovich, G.T., Read, W., 1992. The current state of performance
appraisal research and practice: Concerns, directions and implications. Journal
of Management 18, 321–352.
Burns, R.K., Flanagan, J.C., 1955. The employee performance record: A new appraisal
and development tool. Harvard Business Review 5, 95–102.
Clemen, R.T., 1995. Making Hard Decisions. An Introduction to Decision Analysis.
Duxbury Press.
Cleveland, J.N., Murphy, K.R., Williams, R., 1989. Multiple uses of performance
appraisal: Prevalence and correlates. Journal of Applied Psychology 74, 130–
135.
de Andrés, R., García-Lapresta, J.L., Martínez, L., 2010. Multi-granular linguistic
performance appraisal model. Soft Computing 14, 21–34.
Dopazo, E., González-Pachón, J., 2003. Computational distance-based
approximation to a pairwise comparison matrix. Kybernetika 39, 561–568.
1607
Edwards, M., Ewen, E., 1996. Automating 360 degree feedback. HR focus 73, 3.
Fisher, C., Schoenfeldt, L.F., Shaw, J.B., 2006. Human Resources Management.
Houghton Mifflin Company, Boston.
Fletcher, C., 2001. Performance appraisal and management: The developing
research agenda. Journal of Occupational and Organization Psychology 74,
473–487.
Frunzi, G., Savini, P.E., 1996. Supervision: The Art of management. Prentice Hall,
New Jersey.
González-Pachón, J., Romero, C., 1999. Distance-based consensus methods: A goal
programming approach. OMEGA International Journal of Management Science
27, 341–347.
González-Pachón, J., Romero, C., 2001. Aggregation of partial ordinal rankings: An
interval goal programming approach. Computers and Operation Research 28,
827–834.
González-Pachón, J., Romero, C., 2006. An analytical framework for aggregating
multi-attribute utility functions. Journal of Operational Research Society 57,
1241–1247.
González-Pachón, J., Romero, C., 2007. Inferring consensus weights from pairwise
comparison matrices without suitable properties. Annals of Operations
Research 154, 123–132.
Ignizio, J.P., Caliver, T., 1994. Linear Programming. Prentice-Hall, Englewood Cliffs,
NJ.
Jarman, M., 1999. Complete turnaround 360-degree evaluations gaining favour with
workers management. Arizona Republic, D1.
Kerr, J.L., 1985. Diversification strategies and managerial rewards: An empirical
study. Academy of Management Journal 28, 155–179.
Latham, G.P., Wexley, K., 1981. Increasing Productivity through Performance
Appraisal. Addison-Wesley.
Likert, R., 1932. A technique for the measurement of attitudes. Archives of
Psychology 140, 1–55.
Martínez, L., 2007. Sensory evaluation based on linguistic decision analysis.
International Journal of Approximated Reasoning 44, 148–164.
Miner, J., 1998. Development and application of the rated ranking technique in
performance appraisal. Journal of Occupational Psychology 6, 291–305.
Pfau, B., Kay, I., 2002. Does 360-degree feedback negatively affect company
performance? HR Magazine 47, 56–60.
Romero, C., 1991. Handbook of Critical Issues in Goal Programming. Pergamon
Press, Oxford.
Romero, C., 2001. Extended lexicographic goal programming: A unifying approach.
OMEGA, The International Journal of Management Science 29, 63–71.
Simon, H., 1957. Models of Man. John Wiley and Sons, New York.
Yu, P., 1973. A class of solutions for group decision problems. Management Science
19, 936–946.