Performance Management and Operational Research: A Marriage Made in Heaven?

Author(s): P. C. Smith and M. Goddard

Source: The Journal of the Operational Research Society, Vol. 53, No. 3, Part Special Issue:
Performance Management (Mar., 2002), pp. 247-255
Published by: Palgrave Macmillan Journals on behalf of the Operational Research Society
Stable URL: http://www.jstor.org/stable/822890
Accessed: 08/11/2010 07:18

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=pal.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Palgrave Macmillan Journals and Operational Research Society are collaborating with JSTOR to digitize,
preserve and extend access to The Journal of the Operational Research Society.

http://www.jstor.org
Journal of the Operational Research Society (2002) 53, 247-255 )2002 Operational Research Society Ltd. All rights reserved. 0160-5682/02 $15.00

www.palgrave-journals.com/jors

Performancemanagementand OperationalResearch:
a marriagemade in heaven?
PC Smith* and M Goddard
University of York, York, UK

Thispapersets out a simpleframework withwhichto examinetheperformance management process.It examinesthe

contextwithinwhichperformance
organizational management andnotesthatit becomesmostimportant
is undertaken,
withinhierarchical that allow considerableautonomyamongstdevolvedunits.The paperarguesthat
organizations
performance management shouldembracefourbroadfunctions:formulation of strategy;development of performance
measurementinstruments;interpretingsuch measures;and encouragingappropriate organizational responsesto
performance information.The papernotes that operationalresearchhas made significantcontributions to all four
functions,andsuggeststhatthe performance managementmovementoffersenormouspossibilitiesfor OR.
Journal of the OperationalResearch Society (2002) 53, 247-255. DOI: 10.1057/palgrave/jors/2601279

measurement;
performance
management;
Keywords:performance structure
strategy;organizational

Introduction the commercial sector,2providing opportunitiesfor cross-

The practice, teaching and research of management is
peculiarly vulnerable to fads. A cynic might suggest that
this is the result of a lively marketin managementgurus, in
which the latest management concept is promoted as an
indispensable tool for the modem manager. However, it
might also be the case that some apparentfads do indicatea
real change in the preoccupationsand needs of managers.
The explosion in interestin performancemanagement(PM)
since the mid-1990s may indeed be one such case.
For many years the concept of performancemanagement
was synonymous with the micro-managementof employee
behaviour. It certainly has roots in Taylor's concept of
'scientific management', and Curriel claims that elements
of performancemanagementcan be tracedback to the 13th
century.The predominantearly interestwas thereforefrom
the perspective of human resource management (HRM),
although some industrial companies had quite advanced
management control systems in place by the 1970s.
However, the concept of performance management has
progressivelybroadenedin recent years, to the extent that
by the 1990s it had become closer to implying a concern
with the strategicmanagementof an entire organization,or
even economy. Unusually for a managementtrend,perfor-
mance management appears to have reached the public
sector at around the same time as it started to penetrate
*Correspondence:PC Smith, Centrefor Health Economics, Universityof
York,YorkYOIO5DD, UK
E-mail:pcsl@york.ac.uk
fertilizationof ideas and practices.
In spite of the enormous growth of interest in PM, a
precise definitionof the concept is elusive. For example, an
OECD report on PM in the public sector notes that the
growth in interest in PM has been stimulated by the
increased devolution of responsibilities in many public
sectors, and the associatedneed to develop explicit models
of accountabilityand performancemeasurement,but fails to
arrive at a specific definition.3This difficulty reflects the
differentconnotationsof PM that exist, an issue we explore
furtherin the next section by examining the organizational
context within which PM must operate. We then describe
what we perceiveto be the four fundamentalbuildingblocks
of PM: formulation of strategy; development of perfor-
mance measurement instruments;application of analytic
techniquesthat seek to interpretsuch measures;and devel-
opment of instrumentsdesigned to encourage appropriate
organizationalresponses to performanceinformation.The
paper concludes with some reflections on possible future
developments.
The context of performance management
Whereas the most common criteria for performance
measurement are probably still financial, the traditional
definition of performance management (PM) places the
individual employee as the focus of attention. In this
context, the definition of performance is assumed to be
relatively uncontentious, deriving from an organizational
strategythat is taken as given, and the principalinterestis
in the instrumentsavailable to optimize performance. It
 oftheOperational
248 Journal Research Vol.53,No.3
Society

thereforeemphasizesorganizationalcontrolwithin an estab- more dynamic and is indicated by the outer loop. It

lished set of objectives. In this vein, PM can be character- recognizes that the organizationmust continually review
ized as the context within which PM is conducted,and be prepared
to amend strategyif circumstances(such as customerfeed-
'... an integratedset of planning and review proce-
dures which cascade down throughthe organizationto back, technological developmentsor a change in priorities)
suggest it is needed. It must be emphasizedthat this simple
provide a link between each individualand the overall double loop representationis very simplified(for example,it
strategyof the organization.'4
ignores important complications such as partnerships
More recently commentatorshave recognizedthe limitation between autonomous organizations).However, it captures
of such 'top-down' philosophies, and have sought to the essence of the increasinglyimportantstrategicelement
broadenthe scope of performancemanagementto include of PM.
conscious reflection on strategy, objectives and organiza- A persistent theme in the modem PM literatureis the
tional culture.For example, Rashid5advocatesa philosophy increaseddevolutionfound in the modem organization,and
of 'cascadingup' of ideas to complementthe traditionaltop- the associated increasedneed for conscious surveillanceof
down approach.Pollitt6is typical in characterizingPM as a relatively autonomous elements of the organization.Dele-
set of five processes that extend beyond the organization's gateddecision makingis unproblematiconly if the centrehas
boundaries: setting objectives; assigning responsibility; full informationon the actions and outcomesof the devolved
measuring performance;feedback of informationto deci- unit andaway of controllingthese actions,or if the objectives
sion making; and externalaccountability. of the centre and the devolved unit coincide. In practice,
The diversityof views on what constitutes PM is reflected neitherof these requirementsis likely to be fulfilled,and so a
in the diverse disciplines that have sought to make a 'principal:agent' problem arises.8 It is the relationship
contributionto the topic. Examples beyond HRM include between the centre(the principal)and the devolved unit (the
operations management, marketing, finance, accounting, agent)that forms the centralconcernof PM.
organizationalbehaviour,economics, psychology, political In the extreme, the devolved unit might become an
science, and of course operational research.Each discipline entirely autonomousplayer in an externalmarket,compet-
contributesits own language,traditions,preoccupationsand ing without particularfavour for the business of the parent
prejudices to the topic. Neely7 estimates that a news report organization.In these circumstancesthe market becomes
or article on PM has appearedevery five hours since 1994, the principalinstrumentof performancemanagement.The
and that in the US alone a new book on performance purchasingorganizationspecifies its requirements,and the
measurementappearsevery week. intention is that-if the market is assumed to be competi-
The range of perspectiveson PM can be loosely captured tive-optimal performance in terms of the required
in diagrammaticform (Figure 1). The narrowview of the product'sprice and quality will ensue. Poor performance
performancemanagement role is representedby the inner will simply lead to financialfailureof the devolved unit. At
loop. Strategy, objectives, organizational structure and the other extreme, the organization might retain the
culture are taken as given, and the purpose of PM is to devolved units strictlywithin its direct control, and instead
ensure that the organization is 'steered' in some sense seek to optimize performanceby means of a portfolio of
optimally within that context. The broader view of PM is managerial devices, such as performance measurement,
incentives and appraisal.
The choice of organizationalform (crudely characterized
as marketor hierarchy)has been the subjectof a flourishing
strand of economic enquiry, stimulated by the work of
Williamson.9There are of course numerouspotential inter-
mediate forms, such as the 'internal' markets (or quasi-
markets)introducedin organizationssuch as the BBC and
much of the UK public sector in the 1990s,10under which
the devolved units remain within the organizationalhier-
archy,but are to a greateror lesser extent exposed to internal
or externalcompetitiveforces.
Some economists argue that those designing organiza-
tional form should seek to minimize the costs associated
1
JI
I
/ with devolution.8 These comprise two broad categories:
Exltern-al transaction costs and agency costs. Transaction costs
environmental
comprise the direct costs associated with specifying the
Figure 1 Schematic representationof the performance centre's requirements (perhaps in the form of formal
management process. contracts), monitoring performance(informationsystems),

and rewardingthe agent (perhapsin the form of incentive
payments). Agency costs comprise the indirect costs that
arise because of devolution, in the form of a divergence
betweenwhat the principalideally wants fromthe agent, and
what is actuallyprovided.
Ouchi11extends Williamson'smodel by arguingthat the
choice of optimal organizationalform in a specific situation
depends on two broad contextualconsiderations:the nature
of the productionprocess and the abilityto measureoutputs.
Markets are optimal when knowledge of the production
process is poor but outputs can be accuratelymeasured-
that is, when contracts can be readily written and perfor-
mance monitored by observing outputs. Conversely,
hierarchies might be preferred when knowledge of the
production process is good but outputs are difficult to
measure-that is, when behaviourcan be readily controlled
throughrules of process. When informationon both produc-
tion process and outputmeasurementis poor, the organiza-
tion may have to resort to what Ouchi calls 'clan control',
under which performance is determined by social and
culturalnorms ratherthan marketsor bureaucracies.
Numerous other considerations might influence the
choice of organizationalform. For example, the nature of
the uncertaintyconfrontingan organization(both in demand
for products and supply of inputs) might be an important
determinant.The optimal choice is moreoverlikely to vary
considerablybetween industries,and to change over time, as
technologies, information systems and markets develop.
Organizationalform nevertheless defines a crucial context
within which PM must take place. The PM literatureis
mainly concernedwith a hierarchicalorganizationin which
some element of devolutionhas takenplace. The rest of this
paper thereforeexamines PM in this context. However,it is
worth emphasizing that the strategic element of the PM
process should in principle keep under review whether
organizationalform and processes remain appropriate.
Four components of performance management
In this paperwe adoptthe more recent strategicapproachto
PM. Ourinterestis thereforein organizationalperformance,
and the extent to which purposive managerialaction can
contributeto such performance.Importantearlywork in this
area was undertakenwithin the OR traditionby Stafford
Beer,12 who emphasized the cybernetic nature of the
managementprocess. Howeverthe most sustainedacademic
thought on these issues has been undertakenby manage-
ment accountants interested in organizational control.13
Their accounting perspective has led to an emphasis on
financial aspects of performance, a shortcoming that is
readily acknowledged in the literature.However, many of
the models they have developedare relevantto all aspects of
performance.We exclude from considerationhere action
that might influencethe externalenvironment,most notably
in the form of marketing,and instead focus on actions that
andMGoddard-Performance
PCSmith andOR 249
management
influence activity within the organization.There are then
four broad categoriesof actions that constitutePM:
- formulationof strategywith to determinewhat constitutes
performance;
- developmentof performancemeasurementinstruments;
- application of analytic techniques to interpret such
measures;
- development of instruments designed to encourage
appropriate organizational responses to performance
information.
The success of a PM system will depend on how well these
four indispensableelements of the PM process are welded
into a coherent whole. We now consider them in turn, in
each case touching briefly on the existing OR contribution.
Formulationof strategy
As noted above, most performancemanagementendeavour
takes organizationalstrategy and objectives as given, and
seeks to develop managerialinstrumentswithin that frame-
work. It pursuesthe 'innercontrolloop' of Figure 1. Yetin a
rapidlychangingand consumer-ledenvironment,most orga-
nizations need continuously to reappraiseand reformulate
theirstrategyif they arenot to fail. Particularchallengesface
the relativelynew breedof organizationssuchas the 'dotcom'
companies where the environment can be particularly
unstable and unpredictable,and where success is possible
only if constantadjustmentsaremadeto strategy.Inthepublic
sector,organizationsarerequiredto reconsidertheir strategy
following major policy shifts or changes in the political
environment.A concern for strategy might also embrace
organizationalstructure,incentivemechanismsandorganiza-
tionalculture,each of whichmighthave importantbearingon
long runperformance.Forexample,a healthyorganizationin
the privateor public sector is likely to nurturean element of
entrepreneurialambition amongst its employees if it is to
respondsuccessfullyto a changingenvironment.14
As Dyson15shows, in principleOR has much to offer in
the development of organizational strategy. However,
numerousauthorshave bemoanedthe traditionalreluctance
of OR to addressstrategicissues.16Why should this be? A
common theme amongst commentators critical of OR's
absence from concerns of strategy,is that-in contrastto
the inner loop-the 'outer control loop' is by definition
messy, unstructuredand not amenableto quantification.The
search for strategic threats and opportunities cannot be
reduced to a mathematical programme, or be readily
describedin analytic terms, or even be easily distilled into
coherenteducationaltexts. This lack of structureis antipa-
thetic to the mainstream OR tradition. Indeed, where
attempts have been made to consider strategy,they have
tendedto be subsumedinto easily measuredaspects such as
extent of diversificationactivity,ratherthan non-traditional
metrics that focus on the strategiesthemselves.17
250 Journal
oftheOperational
Research 53,No.3
Vol.
Society
For this reason, the most promising traditionwithin OR
likely to offer a strategic component to PM is probably
representedby the flourishingvariantsof the 'soft' metho-
dological approach.18The open-mindedand eclectic philo-
sophy underlying soft OR approaches is well suited to
problems of strategy,for which the 'harder' OR tradition
is rarely likely to be relevant. In particular,soft systems
methodology has been specifically developed for use in ill-
structuredand messy problem contexts where there is no
consensus about how to define the problem nor how to
tackle it.19 It is appropriatewhere complex organisational
structuresand processes are the norm, which will certainly
be true in the public sector, and also in many other
institutionalcontexts. The embracing of 'soft OR' in the
UK in the 1980s by the OR Society greatly enhanced the
potential contributionof OR to the strategicaspect of PM.
Performancemeasurementinstruments
The revolution in information technology has led to the
potential for enormous improvementsin the quantity,qual-
ity, timeliness and cost-effectiveness of data relating to
performance. Yet in many organizations, although the
potential for such data capture is recognized, the reality
has often been that the systems structure and systems
informationmade availableto managersis partial,of poor
quality and late.
The managementliteratureis replete with instrumentsto
guide the design of information systems. The most cele-
brated is the 'balanced scorecard'.20 For most commercial
corporations,the strategicobjectiveis to maximize long run
profits. However, use of historical, short run measures of
profitability as the sole indicator of success is clearly
unsatisfactory and likely to be seriously dysfunctional.
Kaplan and Norton therefore develop a multi-dimensional
instrumentthat seeks to capturecontemporaryindicatorsof
future success in the four broad domains of finance, custo-
mer perspective,internalbusiness process, and learningand
growth. The principleunderlyingsuch methods is that past
success may not be a reliable indicator of future success.
Insteadthe methods seek out measurableaspects of current
process that are likely to be strong indicatorsof currently
unmeasurablefutureoutcome.
In many respects, the practice of performancemeasure-
ment has been led by the public sector,wherethe absence of
financial measures of success naturallyled to the develop-
ment of multi-dimensionalmeasurementinstruments.21To
date therehas often been an absence of theory to informthe
development of public sector schemes, but the general
principlesadoptedhave been analogousto their commercial
counterparts:namely, indicators of currentprocess should
captureaspects of unmeasurablefutureoutcome. The added
complexity is that there is often less consensus amongst
public sector stakeholdersas to what constitutes relevant
outcome.22
Prominent examples of public sector performance
measurementschemes in the UK include the local govern-
ment performanceindicatorsystem23and the NHS Perfor-
mance Assessment Frameworlk.24Characteristicsof such
schemes tend to be that they are incomplete (rarelycaptur-
ing all acknowledged aspects of performance), prolix
(comprising numerous indicators of performance) and
opportunistic (measuring what is measurable rather than
developing new systems for PM purposes). Possibly as a
resultof these weaknesses,it has hithertobeen rareto find a
performance measurement system completely integrated
into the managerialprocesses of a public sector organiza-
tion. The NHS Plan publishedin July 2000 seeks to remedy
this by placing the NHS PerformanceAssessment Frame-
work at the centre of NHS managerial attention.25In a
similar vein, the 'Best Value' system seeks to integrate a
nationalperformancemeasurementsystem into the strategic
planning process in English local government.26Making
these laudable objectives operational is likely to be a
formidablechallenge.
OR practitionershave often been centralto the develop-
ment of organizationalinformationsystems which are a key
element in any performancemanagementsystem. Indeed it
is instructiveto note that the US Institute for Operations
Research and the Management Sciences (INFORMS)
defines OR and ManagementSciences as 'the professional
disciplines that deal with the application of information
technology for informed decision-making'.The distinctive
role of OR is often to examine the 'whole system' within
which the information system must operate.27The OR
Society's developmentof the EuropeanJournal of Informa-
tion Systemsis testimonyto the profession'scentralinterest
in this issue.
Analytic techniques
Performancedata are frequently worthless until they are
translated into meaningful signals of performance. For
example, the accounting tradition is to convert activity
data into measures of profitability.Hospitals might report
activityin termsof averagelengths of patientstay.However,
such naive approachesare in generalwoefully inadequate-
and indeed irrelevant-for most purposes. The complexity
of the modem organizationhas thereforeled to the devel-
opment of more sophisticatedanalytictechniques,designed
to secure more sensitive measuresof performance.
Broadly speaking,analytictechniquesseek to understand
the reasons why a particularindication of performanceis
observed. For example, in orderto undertakea comparison
of variationsin the performanceof hospital surgicaldepart-
ments, as measuredby clinical outcome, it might be neces-
sary to seek to gain an understandingof the extent to which
variationsin outcome are due to:

1. variations in case mix, or the characteristicsof the
patientsbeing served;
2. variationsin the nature of resources being used-poorer
resultsmay be the result of (say) an inappropriatemix of
resources (such as an outdated configurationof capital
stock);
3. variationsin priorities regardingoutcome-some provi-
ders may choose to place a greater emphasis on
(say) quality of survival at the expense of (say) higher
mortality;
4. variationsin the externalenvironments-for example the
contributionsof other agencies, such as local personal
social services, to outcome;
5. variationsin accounting treatments-there may be some
flexibility in the way that data are collected or recorded;
6. data errors-the quality of datamight vary substantially
between providers;
7. random fluctuation-many performance measures are
highly vulnerableto such fluctuation,considered to be
entirelyoutside the control of the provider;
8. variationsin quality of services-usually the key perfor-
mance measureto most stakeholders.
Two broad approaches to disentangling these sources of
variationexist: cross-sectionaland longitudinal.The cross-
sectional approachseeks to understanda unit'sperformance
data by examining equivalent data from comparableorga-
nizations, and is intrinsic to the benchmarkingmovement.
The intention is to quantify the unit's performance(say,
quality of service) relative to other organizations-after
adjusting for all the other sources of variationnot related
to performancenoted above. In contrast, the longitudinal
approachlooks at year-on-yearperformancechanges in a
single organization,and must inevitablybe used when there
exist no comparableunits, or where comparisonis highly
problematic.
Intellectualeffort has concentratedon the cross-sectional
approach,althoughthe extent to which senior management
is interested in disentangling cross-sectional sources of
variationmay vary considerably,depending on context. In
principle, an interest in managerialperformancesuggests
that the analysis should seek to adjust for all sources of
variationnot attributableto managerialaction. This is not
always achieved. For example, the early league tables of
academicsuccess in English schools were publishedwithout
correctionfor any of the above types of variation,in spite of
impassioned appeals from academics to do so, and the
existence of techniques to address the complex analytic
problem.28'29 On the other hand there may be good reasons
for not adjusting examinationresults-for example, there
are legitimate concerns about how any 'risk adjustment'
should be undertaken. However, it is worth noting that
failure to treat devolved units (schools) fairly within a PM
system might lead to serious inefficiencies in system beha-
viour, such as not being able to attractteachersto 'problem'
PCSmith
andMGoddard-Performance andOR 251
management
schools.30In other words, an inadequatetechnical basis for
PM leads to an unsatisfactorystrategic environment.
The intellectualchallenge offeredby the need to interpret
crude performance measures has proved attractive to
academics,particularlyin the realmof productivityanalysis.
Indeed the performancemeasurementfunction has become
synonymouswith performancemanagementin much of the
literature,and an impressivebatteryof analytic techniques
has been assembled.The Journal of ProductivityAnalysis is
a good source for recent developments.Withinthe statistics
tradition, multilevel (or hierarchical)modelling has been
dominant.31These techniquesassume that outcome can be
ascribed to a hierarchyof causes (for example, the local
authority, school, class and individual in educational
outcome), and seeks to ascribe measuredoutcome to each
of these 'levels'. Clearlythese methodsare most appropriate
when hierarchicalinstitutionalstructuresare in place.
The econometricstraditionhas focused on the develop-
ment of cost functions and production functions, most
specificallythroughthe use of stochasticfrontieranalysis.32
These techniquesseek to model the determinantsof 'perfor-
mance' in line with economic models of production.Speci-
fically, they assume explicitly an element of inefficiency,
which is modelled as a negative influence on performance.
The strengthof such models is thattheirspecificationcan be
tested for consistency with theory,and they can be used to
test hypotheses. Their weaknesses are that they require
extensive data,thatthey requirepossibly restrictiveassump-
tions about model specification,and that their use becomes
impracticalwith productionprocesses of any complexity.In
particular,one of the key constraintsin PM is a lack of
detailedknowledge of the relationshipsand transformations
between inputs and outputs.
Perhapsbecause of its linear programmingformulation,
OR researchershave concentratedtheir analytic efforts on
the applicationof data envelopmentanalysis.33DEA seeks
to quantify the degree of technical inefficiency of a unit
relativeto comparableorganizationsin a multi-input,multi-
outputcontext.It has spawneda small researchindustry,and
numerous applications have been reported.34In many
respects DEA representsthe apotheosis of the performance
indicatormovement.35However,reportsof DEA being used
in earnestin the performancemanagementprocess are rare.
This may be because of its many practical weaknesses:
model specificationdependson userjudgements,and cannot
readily be tested; alternativeplausible specificationsmight
offer quite differentresults;DEA models are highly vulner-
able to data errors; if the production process is at all
complex, the ability of the technique to detect inefficiency
is quite limited (many units are designated 100% efficient).
Numerous other analytic techniques designed to help
make sense of performancedata exist. Cluster analysis is
often used to identify 'families' of comparableorganiza-
tions. Factoranalysismight be used to reducethe numberof
dimensions of performance. In health care, various risk
252 Journal Research
oftheOperational Vol.
Society 53,No.3
adjustmentmethodologieshave been used to assess clinical
performance,and analogous techniques have been devel-
oped in other industries.
Most of the approachesmentioned here are 'feedback'
techniques:historic performanceis analysed and the infer-
ences fed back to managers for appropriateaction to be
taken. However, there is also an importantrole for what
managementaccountantsrefer to as 'feedforward'techni-
ques that seek to alertmanagersto futureoutcomes.Various
forecastingtechniques offer importantOR contributionsin
this respect. In particular,critical path analysis represents
perhapsthe apotheosis of the 'inner loop' control philoso-
phy. If the objectives and structureof a projectmanagement
problemare clear,then criticalpath methodsoffer important
insights into optimal managerial responses to changed
circumstances.
In spite of this impressive intellectual endeavour (or
possibly because of it), competing analytic techniques
frequentlyyield quite differentestimates of relative perfor-
mance.36'37This uncertaintycan lead strategiststo question
their usefulness or veracity.Moreover,the techniquesoften
appearto be impenetrableto the non-expert.For these and
other reasons, their impact on policy appearsto have been
modest. One can point to some concrete influence-for
example, a suite of analytic techniques is used by the UK
water regulatorto set performancetargetsfor water compa-
nies and rewardsarebased on results;38similarly,it has been
suggested that futureefficiency targetsfor the police forces
may be devised in this way.39 However, the concrete
influence on organizationalpolicy has not yet matched the
impressivemethodologicaladvances made over the last 25
years.
Encouragingappropriateorganizationalresponses
The most finely honed performancemeasurementsystem
can prove worthless, even dysfunctional (for some stake-
holders at least), if it is not embeddedwithin an organiza-
tional environmentthat encourages appropriatebehavioral
responseson the partof managers.8Indeedthe failureof the
formerSoviet economy-the most ambitiousorganizational
hierarchy in history-was due in no small part to its
inadequate PM systems.40 The intention should be to
encourageappropriatesystem responses throughthe use of
incentives and other managerialdevices. Incentives can be
designed or accidental.
Designed incentivesseek to link a targetto some aspect of
measuredperformance,and attach a reward(or penalty) to
performanceachieved in relationto the target.Rewardscan
be at the individual or organizationallevel, and may be
financial or otherwise. The important characteristic of
'designed' incentives, is that the rules of the game are set
in advance and are observableby all parties.
In additionto these purposivesystems, thereexist in most
organizations'undesigned'(or accidental)incentives.Many
undesigned incentives arise from poor design of explicit
incentiveschemes, and can be seriouslydysfunctional.They
have been documentedin a wide variety of researchtradi-
tions. In synthesizing this experience it is possible to
identify at least nine enemies of virtuous performance
management,41listed here with some concrete examples
from health care.
Tunnelvision 'Concentrationon areasthatare includedin
the performanceindicatorscheme, to the exclusion of other
importantunmeasuredareas.'
Example:numbersof people on hospital waiting lists have
hitherto preoccupied UK health care managers as the
primary process performance measure, possibly to the
detriment of important but unmeasured dimensions of
performance(eg clinical outcomes).
Measurefixation 'Pursuitof success as measuredrather
than as intended.'
Example: a five minute waiting time performancecriterion
in Accident and Emergency departmentsled to the wide-
spread employment of a 'hallo' nurse in UK hospitals,
which is costly and may have no impact on any aspect of
patient satisfactionor outcome, serving merely to meet the
formalrequirementfor patientsto be seen by a nursewithin
5 minutes.
Sub-optimization'The pursuitof narrowlocal objectives
by managers, at the expense of the objectives of the
organizationas a whole.'
Example:use of the proportionof operationscarriedout as
day cases as an indicator of surgical efficiency does not
acknowledge the increased burden the use of day case
surgerymay place on agencies outside the hospital sector,
such as social service departmentsor primarycare.
Myopia 'Concentration on short term issues, to the
exclusion of long term considerationsthat may only show
up in performancemeasuresin many years time.'
Example: curative services (as measured by short term
process) may be given higher priority than preventive
services (as measuredby long term outcome).
Complacency'Lackof ambitionfor improvementbrought
about by an adequatecomparativeperformance.'
Example: an apparently middling performance when
judged against others on surgical survival rates may
appear satisfactory and inhibit the search for further
improvement.
Misrepresentation'The deliberatemanipulationof data
by providerstaff, including 'creative'accountingand fraud,
so that reportedbehaviour differs from actual behaviour.'
Example: adverse patient satisfaction reports might be
unaccountably'lost'.
Misinterpretation 'Incorrect inferences about perfor-
mance brought about by the difficulty of accounting for
the full range of potential influences on a performance
measurement.'
Example:is a high rate of "did not attend"at an outpatient
clinic due solely to the actions of the clinic, or do other,

uncontrollableinfluences (such as patient characteristics)
have an importantinfluence?
Gaming 'Altering behaviour so as to obtain strategic
advantage, particularlyprevalent when targets are based
on year-on-yearimprovements.'
Example: an NHS hospital might have 'gone easy' on
performanceagainst its NHS efficiency index (which was
based on an annualpercentageimprovement)so that it did
not receive unduly demandingfinancialtargetsin the future.
Ossification 'Organizationalparalysis brought about by
an excessively rigid system of measurement.'
Example: a low rate of inpatientreadmissionshas hitherto
been consideredan indicatorof good practice-however, its
use might inhibitthe take-upof an efficient new technology
that requiresfrequentreadmission.
A common theme from these observationsis that under-
standingthe full impact of PM on the behaviourof indivi-
duals within the organizationis a crucial element in the
assessmentof PM systems. The incentive stage is often seen
as merely a matter of setting appropriate targets for
managers.The adverse consequences arise because indivi-
duals and systems respond to undesigned as well as
designed incentives. Formal targets are therefore likely to
be seriouslymisleading unless viewed within the context of
the entire set of incentives, both designed and accidental,
that is built into the system. However,althoughthe subject
of a respectableaccountancytradition,most notablythrough
the journal Accounting, Organizations and Society, the
behavioural response stage in the PM process has been
under-researched.Even where research has addressed the
impactof performancemeasurement,attentionhas tendedto
focus narrowly on success as measured by the chosen
performance measures themselves, rather than on any
broadersystem-wide evaluation.42
Moreover, an emphasis on incentives may represent a
narrow view of the instruments available to encourage
appropriateresponses. For example, theremay be an impor-
tant role for consideringhow the cultureof an organization
can be nurturedso as to secure favourableresponses to
performancedata, even when they are of poor quality.43
It is possible to hypothesize numerous other organiza-
tional responsesthat can be put in place to mitigatesome of
the adverse responses suggested above. To take just one
example, 'complacency'can be challengedby basing targets
to some extent on year-on-year improvementas well as
cross-sectionalperformance-but this developmentmay in
turn increase the potential for 'gaming'. The appropriate
response is thereforelikely to be highly dependenton the
organizationand industryunder scrutiny.
Thus the 'response' stage of the PM process is very
under-developed, and has hitherto been the domain of
organizationalbehaviour specialists, management accoun-
tants, psychologists and economists, most especially in the
context of agency theory.8Nevertheless, as commentators
such as Russell Ackoff44 have long argued, it is fertile
PCSmith
andMGoddard-Performance andOR 253
management
territory for future OR activity, possibly in collaboration
with other disciplines such as psychology, sociology and
economics. This is a complex area that deserves further
attention,especiallyas the natureof incentivestructureswill
vary greatlydependingon the environmentin which the PM
system operates. Most notably, appropriateresponses may
vary substantiallybetween the private and public sectors,
where the natureof the funding and resourcemechanisms,
motivationsof employees, labour markets, and the overall
aims of the PM process can differ enormously.
Discussion
The literature on performance management is eclectic,
diffuse and confused. The definitive 'general theory' of
performancemanagementremains elusive, and is unlikely
ever to emerge. Importantcontributionscan be found in
fields as diverseas strategy,organizationalbehaviour,opera-
tions management,industrialeconomics and accountancy.A
cursorysearchof any publicationdatabaserevealsthat most
academicpublicationsstill use the expression 'performance
management' in the context of human resource manage-
ment. This connotationof PM is of course often a vitally
important determinant of organizational performance.
However,it is only one of the potentialcontributoryfactors
embracedby the broaderdefinitionof PM we adopt here.
Within the inner loop of PM described earlier, most
activity to date has been concerned with performance
measurement.In the corporatesector,numerousinstruments
along these lines can be found,amongstwhich the 'balanced
scorecard' has been dominant. In the public sector, the
instrumentsof performancemeasurementhave developed
rapidly, in the form of large information systems and
sophisticatedanalytic techniques. However,in both sectors
the more elusive behaviouralaspects of PM have received
comparativelylittle attention.We would arguethatthe entire
performance management movement might be compro-
mised if this weak link in the feedbackcycle is not afforded
increasedattention.
There have been some attemptsto integrateperformance
measurementinstrumentsinto a more generalframeworkof
performance management. For example, Kaplan and
Norton45 show how the balanced scorecard approach
might be viewed within the context of corporateorganiza-
tion, cultureand environment,and the recent developments
in British health care and local government will merit
careful scrutiny.The Business Excellence Model has also
been widely used by the governmentin the context of the
public sector.46However, such work is at best exploratory,
and we have a long way to go before a satisfactorytheoryof
performancemanagementcan be put forward.
The outer feedback loop to strategy has received least
academicattention.47This is hardlysurprising,given thatthe
natureof such second orderfeedback is-almost by defini-
tion-likely to be unpredictableand difficult to formalize.
 oftheOperational
254 Journal Research Vol.
Society 53,No.3
Nonetheless, this should nevertheless not detract from its
central importance,and there have been some notable, if
isolated, cases of successful soft systems interventions.We
can merely add our voices to the ritual to call for more
researchin this area.48
The discussion has suggested that contingent factors are
likely to play a crucial role in determining optimal
approaches to PM. This message has been a constant
theme in the important management control literature,49
and suggests that specific characteristicsmight include:
- the technology employed (for example, whether manu-
facturingor service sector; an establishedtechnology or
one subject to rapid change);
- industrialstructure(for example, monopoly or competi-
tive market,public sector,regulatedsector or competitive
sector);
- organizationalstructure(whetherimposed or chosen; the
extent and natureof internaldevolution);
- external environment(for example, demand uncertainty,
suppliercharacteristics);
- strategy;
- nature and relative power of stakeholders(shareholders,
customers,citizens, patients,politicians).
In conclusion, we have shown that there is a distinctiveOR
tradition in each of the four categories of action that
constitute performancemanagement.However, the contri-
bution of OR has been partialand is less well-developedin
some areasthan in others.We have exploredthe reasons for
this. In terms of formulation of strategy,the main obstacle
relates to the 'messiness' of the process, which prohibits
quantification and the neat formulaic approaches repre-
sented by traditionalOR. The soft systems approachis far
better equipped to deal with complex and unstructured
situations than more traditional OR approaches, and we
would expect to see furtherapplicationsof this approachto
performancemanagementissues emerge over time. OR has
contributedsubstantiallyto the developmentofperformance
measurementinstruments,especially in terms of organisa-
tional information systems and one must hope that
operationalresearchers-with their broad systems perspec-
tive-will redouble their efforts in this area. Contributions
to the development of analytic techniques in performance
managementhave come from a varietyof disciplines,but the
main emphasis in OR has been on the applicationof data
envelopment analysis. Whilst the methodological
approaches have become increasingly advanced, we have
arguedthat, for a numberof reasons (such as presentational
complexity and a tendency to get different results from
differenttechniques),the impact on policy has been rather
modest. OR analysts are not alone in needing to devise
better ways of communicating complex results to non-
experts which could strengthenthe link between research
and strategic policy, and there may be a role for some
generic social research to establish more effective links
between the production of evidence and its influence on
organizationalpolicy and practice.50The issue of beha-
viouralresponses to PM and the role of incentiveshas been
relatively neglected by OR analysts to date. Their main
focus has been on the use of target-settingto achieve
specified goals and the narrowevaluationof the success of
the PM system judged only in terms of the chosen perfor-
mance measures themselves. Again, OR analysts are not
alone in theirrelativeneglect of this vital elementin the PM
system. Thereappearsto be greatpotentialfor collaboration
with other disciplines to make progress on this in future.
The need for PM arises most urgently in hierarchical
organizationalstructureswhere no natural market exists,
precisely the circumstancesin which OR can make its most
compelling contribution.There is thereforeevery reason to
believe that OR has a great contributionto make to the PM
movement,and one must hope that the discipline will seize
the opportunities presented by the recent explosion of
interest.It will be interestingto returnin 10 years time to
see whetherthat hope has been realized.
Acknowledgements-The authors would like to acknowledge helpful
comments from Jacky Holloway at the Open University and two anon-
ymous referees.They are in partfundedby the UK Departmentof Health.
References
1 CurrieRM(1959).Work Study.PitmanInternational:
London.
2 HollowayJ (1999). Managingperformance.
In: Rose A and
Lawton A (eds). Public Services Management.PrenticeHall:
London,pp 238-259.
3 Organizationfor EconomicCooperationand Development
(1996). Performance Management in Government.OECD:
Paris.
4 Rogers S (1990). PerformanceManagementin Local Govern-
ment. Longman:London.
5 RashidN (1999). ManagingPerformancein Local Government.
Kogan Page: London.
6 PollittC (1999). IntegratingFinancialManagementand Perfor-
mance Management.OECD:Paris.
7 Neely A and WaggonerD (1998). PerformanceMeasurement:
Theoryand Practice. JudgeInstitute,Universityof Cambridge:
Cambridge.
8 MilgromP andRobertsJ (1992). Economics,Organizationand
Management.PrenticeHall: Englewood Cliffs, NJ.
9 WilliamsonOE (1975). Marketsand Hierarchies:Analysisand
AntitrustImplications.Free Press:New York.
10 BartlettW, RobertsJA and Le GrandJ (1998). A Revolutionin
Social Policy: Quasi-marketReformsin the 1990s. The Policy
Press: Bristol.
11 Ouchi WG (1979). A conceptualframeworkfor the design of
optimalcontrolmechanisms.Mngt Sci 25: 833-849.
12 Beer S (1966). Decision and Control:The Meaning of Opera-
tional Research and ManagementCybernetics.John Wiley &
Sons: London.
13 Otley D (1999). Performancemanagement:a frameworkfor
managementcontrol systems research.Mngt Account Res 10:
363-382.
14 Osborne D and Gaebler T (1992). ReinventingGovernment:
How the EntrepreneurialSpirit is Transformingthe Public
Sector. PenguinBooks: New York.

15 Dyson R (2000). Strategy, performance and operational
research.J Opl Res Soc 51: 5-11.
16 OrmerodRJ (1997). The role of OR in shapingthe future:smart
bits, helpful ways and things that matter.J Opl Res Soc 48:
1045-1056.
17 MurrayE and RichardsonP (2000). Measuringstrategicperfor-
mance: arewe measuringthe rightthings?- right?In: Neely A
(ed). PerformanceMeasurement-Past, Present and Future.
Judge Institute,Universityof Cambridge:Cambridge,pp 411-
426.
18 RosenheadJ (ed). (1989). RationalAnalysisfor a Problematic
World.John Wiley & Sons: Chichester.
19 ChecklandP (1985). Achieving 'desirableand feasible' change:
an applicationof soft system methodology.J Opl Res Soc 36:
821-831.
20 Kaplan RS and Norton DP (1992). The balanced scorecard:
measuresthat driveperformance.HarvardBus Rev 70: 71-79.
21 LikiermanA (1993). Performanceindicators:20 early lessons
from managerialuse. Pub MoneyMngt 13: 15-22.
22 SmithP (ed). (1996). MeasuringOutcomein the Public Sector.
Taylorand Francis:London.
23 Departmentof the EnvironmentTransportand the Regions
(1999). Best Valueand Audit CommissionPerformanceIndica-
tors 2000/2001. TSO: London.
24 NHS Executive (1998). The New NHS: A National Framework
for Assessing Performance.Departmentof Health:London.
25 Department of Health (2000). The NHS Plan: A Plan for
Investment,A Plan for Reform.TSO: London.
26 Audit Commission (1999). Best Valueand the Audit Commis-
sion. TSO: London.
27 ChecklandP and Holwell S (1998). Information,Systems,and
InformationSystems.John Wiley & Sons: Chichester.
28 Goldstein H and SpiegelhalterDJ (1996). League tables and
theirlimitations:statisticalissues in comparisonsof institutional
performance.JR Statist Soc Series A Soc 159: 385-409.
29 MancebonM and Molinero C (2000). Performancein primary
schools. J Opl Res Soc 51: 843-854.
30 Milgrom P and Roberts J (1990). The efficiency of equity in
organizationaldecision-processes.Am Econ Rev 80: 154-159.
31 Goldstein H (1995). Multilevel Statistical Models. Edward
Arnold:London.
32 MorrisonPaul CJ (1999). Cost Structureand the Measurement
of Economic Performance. Productivity, Utilization, Cost
Economies, and Related Performance Indicators. Kluwer:
Boston.
33 CharnesA, Cooper WW, Lewin AY and Seiford LM (1995).
Data EnvelopmentAnalysis. Theory,Methodologyand Appli-
cations. Kluwer:Boston.
34 Hollingsworth B, Dawson PJ and Maniadakis N (1999).
Efficiency measurement of health care: a review of
non-parametric methods and applications. Health Care
Mngt Sci 2: 161-172.
PCSmith
andMGoddard-Performance andOR 255
management
35 SmithP (1990). The use of performanceindicatorsin the public
sector.J StatistSoc Series A 153: 53-72.
36 Iezzoni L (1997). Risk Adjustmentfor Measuring Healthcare
Outcomes,2nd edn. HealthAdministrationPress: Chicago.
37 Jacobs R (2001). Alternative methods to measure hospital
efficiency: data envelopment analysis and stochastic frontier
analysis.Health Care Mngt Sci 4: 103-115.
38 Office of WaterTrading(1999). Future Waterand Sewerage
Charges 2000-2005, Draft Determinations.OFWAT:London.
39 SpottiswoodC (2000). ImprovingPolice Performance.A New
Approach to Measuring Police Efficiency. Public Services
ProductivityPanel, HM Treasury:London.
40 KornaiJ (1992). TheSocialist System:ThePoliticalEconomyof
Communism.ClarendonPress: Oxford.
41 SmithP (1995). On the unintendedconsequencesof publishing
performancedatain the public sector.IntJPub Admin18: 277-
310.
42 Holloway J (2000). Investigatingthe impact of performance
measurement.In: Neely A (ed). PerformanceMeasurement-
Past, Present and Future. Judge Institute, University of
Cambridge:Cambridge,pp 234-241.
43 Langfield-SmithK (1995). Organizationalcultureand control.
In: Berry AJ, BroadbentJ and Otley D (eds). Management
Control:Theories,Issues and Practices. Macmillan:London.
44 Ackoff R (1978). The Art of Problem Solving. John Wiley &
Sons: New York,pp 179-200.
45 Kaplan RS and Norton DP (1996). Using the balanced
scorecardas a strategicmanagementinstrument.HarvardBus
Rev 74: 75-85.
46 European Foundation for Quality Management (2000). The
EFQMExcellenceModel:Public and VoluntarySector Version.
EFQM:Brussells.
47 Milgrom P and Roberts J (1990). The economics of moder
manufacturing:technology,strategyand organization.AmEcon
Rev 80: 511-529.
48 Langfield-SmithK (1997). Managementcontrol systems and
strategy:a criticalreview.Account OrganSoc 22: 207-232.
49 Otley D (1980). The contingency theory of management
accounting: achievement and prognosis. Account Organ Soc
5: 194-208.
50 Davies H, Nutley S and Smith P (eds). (2000). What Works?
Evidence-basedPolicy and Practice in Public Services. Policy
Press:Bristol.
Received November 2000;
accepted August 2001 after one revision