Oxfam GB

Results-Based Management: Friend or Foe?

Author(s): Michael J. Hatton and Kent Schroeder
Reviewed work(s):
Source: Development in Practice, Vol. 17, No. 3 (Jun., 2007), pp. 426-432
Published by: Taylor & Francis, Ltd. on behalf of Oxfam GB
Stable URL: http://www.jstor.org/stable/25548228 .
Accessed: 31/07/2012 06:27
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .
http://www.jstor.org/page/info/about/policies/terms.jsp

.
JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Taylor & Francis, Ltd. and Oxfam GB are collaborating with JSTOR to digitize, preserve and extend access to
Development in Practice.

http://www.jstor.org

Development

inPractice, Volume 17, Number 3, June 2007

Results-based
foe?
Michael

Ta?VLaJ?rouP

|\

friend or

management:

J.Hatton and Kent Schroeder

Results-based management (RBM) is well entrenched as a management toolfor international

development practice. Yet after a decade of its use, many development practitioners view
RBM in a negative light, considering it to be a donor requirement that diverts time, energy,
and

resources

away

from

actually

doing

work.

development

This

article

some

provides

broad reflections on RBM from a distinctive vantage point: theperspective of theproject (or
programme) evaluator. The article reflects on challenges associated with RBM and draws
from these reflections a number of suggested strategies to improve its use. It concludes that
development practitioners need to be more aggressive in implementingRBM.

Key Words:

Aid; Methods

Results-based
Over

the past

indevelopment

management

decade,

results-based

(RBM)

management

has

become

well

entrenched

as a man

agement tool for development. It is a fact of life formany of us working in this field. Since the
mid-1990s, RBM has focused the work of numerous bilateral and multilateral agencies on
defining,

managing,

and

results.

measuring

outcomes,

outputs,

Inputs,

and

impacts

have

become a familiar refrain.Yet discussions with colleagues regularly take on a negative tone
when the topic of RBM arises. Often, and only half-jokingly, RBM is referred to as part of
the

problem,

that

requirement

consumes

time,

energy,

and

resources

actual doing of development work. In the words of one colleague:

until

the next management

fad

comes

and

obstructs

the

77/ just bide my time

along'.

While this is not a firmposition taken by all players or at all levels in development work, it is
a view not infrequently adopted - in particular by many of those directly involved in project
implementation. Donors and at least some international implementing agencies tend to be
more supportive ofRBM as a management and evaluation tool. Broadly speaking, local govern
ments and beneficiaries are less enthusiastic.
The varying opinions with respect to RBM clearly imply the need for increased efforts to

explore its effectiveness as a development tool. A considerable amount has been written
about the use of RBM by development agencies to date. Much of it is descriptive (see, for
example, Cummings 1997) or it reviews themedium-term experience with RBM from the

426

ISSN 0961-4524

Print/ISSN
DOT

1364-9213

Online 030426-07

10.1080/09614520701337160

2007 Oxfam GB

Routledge Publishing

Results-based

management:

friend

or foe?

perspective of donor agencies (see, for example, Binnendijk 2000; Office ofthe Auditor General
of Canada 2000). This is not surprising, given the relatively short time since RBM came into
general use, compared with the long-term results that it intends to generate. However, as
some development agencies have been using RBM for ten or more years, it is an appropriate
time to reflectmore deeply on RBM as a development tool. This article provides some broad
reflections on RBM from a distinctive vantage point: the perspective of the project (or
programme) evaluator. The article reflects on RBM challenges and seeks to draw from these
reflections some insights for improving the use of RBM tomanage for results.

Why RBM?
Where did results-based management come from, and why is it so widespread? RBM has its
roots in the wave of public-sector reform that swept many of the OECD
countries in the
as
to
1990s.
often
New
This
Public
referred
reform,
(NPM), was driven
early
Management
more
concerns
demands
for
about
efficient and responsive services,
by
spiralling budget defi

cits, the perceived need for increased accountability, and citizens' general discontent with
their governments. New Public Management offered a response. It injected market strategies
into public management, promoted a client-focused orientation to services, decentralised
services where appropriate, and emphasised accountability based on the effectiveness and
relevance of results. Taken in total, the scope and breadth of the reformswere so broad that
the OECD
claimed thatNPM represented a paradigm shift in public management (OECD
1995: 81).
One aspect of these reformswas the incorporation of results-based management, sometimes
called performance management, as a management strategy. This strategy focuses on the
achievement of results. In particular, RBM emphasises the importance of defining expected
resultswith the involvement of key stakeholders, assessing therisks thatmay impede expected
results,monitoring programmes designed to achieve these results through theuse of appropriate
indicators, reporting on performance in achieving results, and acting on performance infor
is at the core of this process: human and financial resources
(inputs) generate activities that produce results in the short term (outputs); in themedium,

mation. A

'results chain'

term
end-of-project,
all
management
guides
resents

and

(outcomes);

a fundamental

activities

re-orientation

long term (impacts).

the ultimate
achievement

in the

towards
away

from

previous

At

its heart,

of defined

management

RBM
results.

approaches

therefore
This

rep

that were

dominated by an emphasis on inputs and activities, the assumption being that results would
follow if the inputs and activities were appropriately robust.
These reforms extended to the bilateral development programmes of OECD
countries.
Results-based

CIDA

management

became

the management

strategy

of choice

for agencies

such

as

in Canada, DFID in theUK, USAID

in theUSA, AusAID
inAustralia, and Danida in
Denmark. Multilateral organisations, including theWorld Bank and a variety of UN agencies,
also incorporated RBM as a management strategy. RBM further trickled down to the
management

approaches

of non-government

organisations

(NGOs),

private

companies,

and

higher-education institutionsacting as the implementing agencies for bilateral and multilateral

development

organisations.

Given thiswidespread use, RBM affectsmost of us involved in development work in some

manner. Some embrace it, some deride it, and some just try to
ignore it.Complicating things
further, the power relationship among the key players
donors, recipients, implementers,
and so on - is clearly asymmetrical. In thismixed context, the key question remains: what
does our actual experience tell us about RBM?
Development

inPractice, Volume 17, Number 3, June 2007

427

Michael

and Kent

J. Hatton

RBM challenges:

Schroeder

reflections fromevaluation practice

The work of monitoring and evaluation typically involves critically assessing the process and
results of a project or programme and offering a written analysis. This gives the evaluator a
unique

perspective

on

the gaps

and

successes

in programme

management.

It also

provides

the evaluator with a potentially key role in achieving project results by contributing to learning,
decision making, and accountability. So what can the view from the evaluator's perch tell us
about the challenges associated with the use of results-based management? Our experience
suggests that the following challenges are common.

Diverse

perspectives

on RBM

It is not easy to find two people who will describe RBM

decade

or more

of its use,

the emphasis,

importance,

value,

in the same way. Even today, after a

and

action

taken with

respect

to the

employment of RBM vary tremendously, depending on the players. Individuals, different

donors, different donor managers, different recipients, and different recipient countries bring
different perspectives to what RBM means, how it is described, how it is employed, how it
should be employed, and its value. And all this is shifting, thoughmuch more slowly than
has been the case in the past. It is easy to imagine the confusion and frustrationof an executing
agency thatworks with one donor forwhich RBM is the focus, and with another donor for
is an afterthought.Or consider working with two managers from the same
which RBM
donor, and finding that the firstdirects the executing agency to use RBM as the cornerstone
inmanagement and reporting, and the second seems not to grasp its value. There is much
greater convergence with respect to the opinions about and commitment to RBM today than
in thepast, and this trendwill continue to grow. But differences of opinion, practice, and under
standing remain significant.

Lack of understanding

of RBM

as a results-focused

approach

In addition to diverse perspectives on RBM, it is striking how often within implementing

agencies there is a failure to understand RBM as a results-focused strategy.For many projects
the operational challenges are such that outputs and outcomes are simply not on the radar
screen. Evaluation does not get past such elements as the qualifications and capacity of the
fieldmanager or project manager, and the logistics of flowing financial orHR capacity, or phys
ically reaching the beneficiaries, including developing relationships with them. Not surpris
ingly, reporting on outputs and outcomes is an area of weakness. Many project reporters
prefer to work on page after page of activities. It is not uncommon for a 60-page report to
have only three pages that describe results past the activity stage. It is also not uncommon
for reports in the first two to three years of a project to emphasise that it is too early for
results, including at the output level. Breaking with the past approach of managing for inputs
and activities instead of for results is a key challenge that is still poorly understood by some.

Inclusion of the developing-country

partner inRBM planning and reporting

cannot be used effectively unless the developing-country partner also understands the
The
paradigm, is well trained in it, and has made an explicit commitment to the approach.
best RBM reporting flows from reports thathave included active participation by the develop
ing-countrypartner. RBM at a mid-level of effective application, or higher,must come from a
a
partnership. That partnership will not develop unless the developing-country partner is full

RBM

428 Development

inPractice, Volume 17,Number 3, June 2007

Results-based

management:

friend

or foe?

an example. It is
participant in both planning and reporting.This is perhaps best explained by
work plan. If
assessment
annual
of
the
for
their
to
ask
developing-country partners
interesting
itsdevelopment was a shared task, theywill speak of itas 'our plan'. Otherwise, it is 'theplan'.
'Our' plan is typically described in detail with glowing pride. 'The' plan ismost often described
in a cursory fashion, with less knowledge about the detail and little commitment to the core.
'Our' plan comes with a defence of how the resources are allocated. 'The' plan is described
with little or no information on the costs of various elements and the options that have been
considered. If 'our' plan includes RBM, the partnerwill probably be committed to it.That is

an essential step. Effective RBM practice does not happen without the partner. And when it
does happen with the partner,much else falls into place.
Yet commitment to full inclusion of the developing-country partner inRBM planning and
reporting is often weak. In some cases this is due to a lack of understanding of RBM within
the executing agency. Confusion at this level quickly translates into confusion and lack of inter

est in the developing-country partner. In other cases, RBM is perceived by implementing

on partners.
agencies as a management practice from the developed world, not to be imposed
RBM monitoring and reporting therefore remain the sole domain of the implementing
agency. Further, it may also be that different results are valued differently by different
players.When the local NGO's needs or expectations differ from the donor's, the implementing
agency is left to negotiate a solution and report accordingly. In the end, a management strategy
involving only half of a partnership will not generate relevant and sustainable results.
Identifying realistic results and unexpected

results

Results thatare realistically achievable may not always be at the frontofmanagers' minds. This is
especially the case when you ask people what can be achieved when they are in the process of
bidding on a project or tryingto sell an unsolicited proposal. Results identified at this stage are

not necessarily ramped back to the real world as theproject unfolds. But if the identifiedand tar
geted results are not realistic, then theproject cannot be managed for successful results.When a
donor requires targeted results to be part of a proposal, it is importantfor those results tobe eval
uated, at least in part, by how achievable they are, not simply on how ambitious they are.
Executing agencies and their implementing partners on the ground also regularly ignore the
identification

of

unexpected

results.

The

need

to define

expected

outputs,

outcomes,

and

impacts at theplanning stage can lock the focus of those implementing a project into achieving
those specific results.Unexpected results,which may be very significant, end up being ignored
or downplayed if they do not neatly fit into the original results framework.

Meaningful

stakeholder

participation

Stakeholder participation is a core element of RBM. You cannot effectivelymanage for results if
- do not
any of thekey stakeholders beneficiaries, development partners, and donor agencies
consider the expected results to be relevant. Stakeholder participationmust be meaningful. This
is not easy, and this element alone is interpreted inmany differentways, often so subjective
thatmaking realistic assessments can be very challenging. Some project documentation speaks
about equal partnerships or equal participation. These are impossible to measure. Even the

word 'meaningful' is open to challenge and confusion at the best of times. Ifwe were to ask all
development stakeholders, confidentially, if theirparticipation in the allocation of project expen
ditures had been 'satisfactory' or better, positive replies would probably not reach 75 per cent.
Grappling with the challenges of defining, implementing, and measuring meaningful stakeholder
participation is vital toRBM, but it is also one of thehardest things to operationalise effectively.
Development

inPractice, Volume 17, Number 3, June 2007 429

Michael

J. Hatton

and Kent

Schroeder

and effective indicators

Appropriate

Appropriate and effective indicators are critical formeasuring success and feeding project
learning. Yet constraints on time and resources often lead to the selection of simplified,
easily gathered quantitative indicators that do not measure results as deeply as they could,
especially at the outcome level. For example, the mainstreaming of gender equality has
made the inclusion of gender issues and accompanying indicatorsmandatory inmany projects.
It is common forprojects to select indicators thatmeasure nothingmore than the percentage of
project staffand participants who are women. While this is important,does it really measure

change in gender relations or power imbalances in anymeaningful sense? Simplified indicators

make itmuch more difficult tomeasure meaningful relationships between inputs and results.

Managing

risks

All project partners need to be able to take informed and timely action tomanage risks. Projects
must be nimble and flexible enough to adapt to changing conditions over the duration of their
life. Are

the executing

agency,

or partners,

the partner

and

the donor

agency

entrepreneurial

enough tomake changes in project design and capacity as needs and the environment change
over a multi-year period? If not, RBM risks being thwarted, leading to results that are inap
propriate, irrelevant, or both. This should not be confused with manipulating project ends to
meet the capacity or interestsof the executing agency or beneficiary.

Limited focus on evaluation

In many cases there is a limited focus on external evaluation within project and programme
activity, despite the critical role of evaluation inmeasuring results and generating learning.
Some

projects

and

programmes

are

of a project

is also

evaluated

annually,

but many

are

others

not. An

end-of

at
project evaluation has no influence on project or programme delivery. An external evaluation
the 60 per cent point
lenge

in maintaining

consistency

tomake

hard-pressed

in evaluation.

For

example,

a mark.

Further,

a five-year

there

project

is often

a chal

or programme

could have three differentdonor-agency project officers and two differentmonitors over that
period.

Maintaining

Performance

evaluation

consistency

in such

a context

is a considerable

challenge.

incentives and consequences

Does an executing agency that reportspoorly, with littleapparent commitment to or knowledge

of RBM, suffersignificant consequences? Often the results of not being committed toRBM are
not disastrous enough tomodify behaviour in advance, and maybe not even so problematic as to
affect behaviour significantly after the fact. Similarly, are projects rewarded appropriately for
results that have been defined, achieved, and reported through the participation of all stake
holders? Without real incentives for achieving results or consequences for poor reporting and
management, the potential of a results-focused strategy is greatly diminished.

Opportunities

for learning

One of thekey roles ofRBM reporting is toprovide informationthatcan be acted upon.Managing

and reporting
performance requires ongoing learning.Yet the connection between data collection
on theone hand, and the incorporationof learning arising from these data on theother, is often not
made. The demands of day-to-day operations frequently rob organisations of time to reflect on
430 Development

inPractice, Volume 17, Number 3, June 2007

Results-based

management:

friend

or foe?

informationgathered throughmonitoring and evaluation, to draw lessons from these reflections,

and to incorporate this learning into project management. The result is lost opportunities. The
best projects incorporate learning throughouttheir lifespan, leading to effective, efficient,and rel
evant results forbeneficiaries. Many projects simply cannot find the time to do this.

Conclusion:

furtheringsuccess

The challenges outlined above illustrate thatRBM can indeed be complex. It has not been as
easy to understand and implement as was first expected. In fact, it is very tough to plan,
analyse, evaluate, and write effectivelywith an RBM focus. But isRBM the enemy thatdetracts
from development work, as some might suggest? Is it a faddishmanagement strategy simply to
be tolerated while it lasts? Despite the challenges, our experience would suggest thatRBM,

used effectively, provides a framework that guides the delivery of real results that improve
the lives of thosemost directly affected by development initiatives. It is in fact a very powerful
tool. The agencies, organisations, and corporations thatembrace RBM typically produce strong
results. We

Being

more

not only

to stay

need

aggressive,

however,

the course
requires

with

RBM,

addressing

we
the

to become

need

challenges.

more

There

are

aggressive.
some
key

areas of focus thatwould greatly improve RBM within our development projects. We
to strive for the following:

need

To oblige executing agencies of all types to report effectively through an RBM framework
that includes identifying realistic results and appropriate indicators at the risk of losing the
project if theyfail to comply.

To compel executing agencies to include developing-country partners in all facets of project

development and implementation, including reporting for results, at the risk of losing the

project.

To engage all partners inmutually defining the depth and breadth of participation, and how
thiswill be measured.
To ensure thatall reporting includes both intended and unintended results, both positive and
negative.

To

increase the evaluation level of most projects, and specifically undertake more post
impact studies to inform longer-termproject development.
To write learning activities directly into project work plans that explicitly put aside time for
project personnel to analyse, reflect on, and incorporate the lessons learned from RBM
reporting.

To provide greater incentives and rewards to projects for achieving results, including revised
results that arise from the successful management of risks during the life of the project.

To

place

greater

on

emphasis

evaluating

executing

based

agencies

on

results,

not

just

on

activities.

Results-based management is an important tool for development work. Over the past decade
ithas proved to be both successful and challenging. But a decade of use does notmean thatwe
can afford to become complacent in our implementation of RBM. We need to redouble our
efforts tomeet the challenges of RBM head-on and adopt its blueprint for success.
References
Annette
Binnendijk,
Review
of Experience.

(2000)

'Results-based

Background

Report',

Management
Paris: DAC

in the Development
Working

Party on Aid

www.oecd.org/dataoecd/17/l/1886527.pdf (retrieved1February 2006).

Development

inPractice, Volume 17, Number 3, June 2007

431

Cooperation
Evaluation,

Agencies:
available

A
at

J. Hatton

Michael

and Kent

(1997)
Harry
Cummings,
and comparisons',
Canadian
OECD
Office

Governance

(1995)

of the Auditor

from the Literature',

Schroeder

contrasts
and results-based
'Logic models,
logical frameworks
management:
Journal of Development
Studies 28(Special
Issue): 587-96.
in Transition:
Public Management
inOECD
Paris: OECD.
Countries.
Reforms

General
available

29 January2006).

of Canada
Results-based
(2000)
'Implementing
Management:
at www.oag-bvg.gc.ca/domino/other.nsf/html/OOrbm_e.html

Lessons
(retrieved

The authors
at the Humber
is Vice President Academic
Institute of Technology
and Advanced
as a project director, monitor, and evaluator
for international devel
15 years' experience
at the project
and Asia.
In addition, he has provided
technical assistance
in Africa
opment programmes
details:
level in the areas of teacher education, management
and
Contact
training,
strategic planning.

Michael

Learning.

Humber
Ontario,

J. Hatton
He

has

Institute
Canada

and Advanced
of Technology
Learning,
5L7. <michael.hatton@humber.ca>.

M9W

managing,
evaluating
School,
Toronto,

Humber

Toronto,
Boulevard,
College
Schroeder
author)
(corresponding
of the Humber
Institute of Technology
and
Kent

in the Business
School
Project Director
for
with
NGOs
and academic
institutions in designing,
has
worked
He
years
Learning.
eight
and
of managing
international development
and evaluating
projects. He also has experience
a capacity-building
The Business
details:
in the Canadian
Arctic.
Contact
programme

is the International
Advanced

205

Humber
Ontario,

Institute
Canada

of Technology
and Advanced
Learning,
5L7. <kent.schroeder@humber.ca>

205

Humber

College

Boulevard,

M9W

432 Development

inPractice, Volume 17, Number 3, June 2007