Beruflich Dokumente
Kultur Dokumente
SOCIAL DEVELOPMENT
Edited by
David Marsden and Peter Oakley
OXFAM
OXFAM
LIBRARY
© Oxfam 1990
Foreword v
Acknowledgements vii
Introduction 1
Section 1 The Meaning and Evaluation of Social Development
1.1 The meaning of social development
David Marsden 16
1.2 The evaluation of social development
Peter Oakley 27
Section 2 Qualitative Dimensions of Social Development Evaluation
2.1 Thematic paper: Muhammad Anisur Rahman 40
2.2 Review of workshop papers and discussion 51
Section 3 The Methodology of Social Development Evaluation
3.1 Thematic paper: Ignacio Garaycochea 66
3.2 Review of workshop papers and discussion 76
Section 4 Partnership in Social Development Evaluation
4.1 Thematic paper: Rajesh Tandon 96
4.2 Review of workshop papers and discussion 102
Section 5 The Evaluator in Social Development Evaluation
5.1 Thematic paper: Delle Tiongson-Brouwers 116
5.2 Review of workshop papers and discussion 123
Section 6 Conclusions
6.1 Future issues and perspectives 138
Bibliography 157
List of participants 160
Foreword
Brian Pratt
Oxford
August 1990
Acknowledgements
This book is based upon the proceedings of an international conference
which we convened on the Evaluation of Social Development Projects in
the Third World, held at the Centre for Development Studies, University
College, Swansea, 19-22 September, 1989- We are most grateful to the
following organisations which gave us financial support: Action Aid, the
Catholic Fund for Overseas Development (CAFOD), Cebemo (Netherlands),
the UN Food and Agriculture Organisation (FAO), Misereor (Germany), the
Overseas Development Administration (ODA), and Oxfam. This generous
support allowed us not only to keep the cost of the conference to a
minimum, but also to offer ten bursaries to Third World students to attend.
The conference went smoothly and certainly gave us few headaches,
thanks to the excellent support we received. Denise Johns, as Secretary to
the conference, was wonderful; Jenny Marr and Anette Arndt gave splendid
support during the week of the conference. Our thanks also to David
Brown, Patricia Goldey, Brian Pratt, Peter Sollis, and Geoff Wood for
serving as rapporteurs and for providing us with prompt notes, and to the
Centre for Development Studies, under the direction of Professor Alan
Rew, for hosting the conference and supporting us throughout; and to Ana
Toni and Anette Arndt for very prompt research assistance in preparing this
book.
Particular thanks to Heinz Knuvener (Misereor), Tony Fernandes
(Cebemo), and Brian Pratt (Oxfam), who joined us in a planning committee
to help prepare for the conference, and gave us great support and
encouragement throughout.
Finally, our sincere thanks to the 80 or so participants for helping to
create what we believe turned out to be a warm, friendly and stimulating
conference.
David Marsden
Peter Oakley
June 1990
vn
Introduction
Background
A four-day conference was held at the Centre for Development Studies,
University College of Swansea in Wales in September 1989- It brought
together a variety of individuals and institutions to address the issue of the
evaluation of social development projects which focus on process rather
than on product and which are thus difficult to evaluate using conventional
methodologies. As Bryant and White observe:
Thematic papers
The four major themes of the conference were introduced by four invited
speakers. Anisur Rahman focused on qualitative indicators in a search for
more appropriate means whereby meaningful development might be
measured. Ignacio Garaycochea addressed the issue of appropriate
methodologies for the evaluation of social development projects and
programmes. Rajesh Tandon examined the issue of partnership in
evaluations, and the dilemmas associated with cooperation between
donors and recipients. And Delle Tiongson-Brouwers focused on the
dilemmas associated with the role of the evaluator. After the presentation
of these four papers and reflections on them and on the two introductory
papers, the conference divided into smaller groups to work through the
issues raised. The results of these discussions were brought back to a
plenary session on the last morning of the conference.
The search for more appropriate indicators
Anisur Rahman argued that indicators should be developed in the context
of the individual project's or programme's actual objectives. Central
attention should be given to the local people's culture in the formulation
of indicators. He introduced the important concept oi the project community,
which includes all concerned with the implementation of the project or
programme. He emphasised that the evaluation experience should be a
learn ing process for all in the project community. In trying to identify more
appropriate indicators, it was suggested that measurement of access to
basic needs was not necessarily as important as the measurement of human
dignity, mutual respect, and solidarity. Education, rather than material
benefits, might be the most important gain in the people's own assessment
of what they had achieved through organisation.
In addressing the question of who assesses development, Rahman
focused on the promoters of social development. How far should we, as
outsiders, impose parameters on the local people? The process of empowering
and enabling must be an essential dimension of social development and
must be undertaken by the people themselves. Three elements of
empowerment were briefly dealt with. These were the organisation of the
disadvantaged, the development of social awareness which enabled the
disadvantaged to develop a sense of equality with the formally 'educated'
classes of society, and the encouragement of self-reliance. An educational
process which focused on these three elements should question seriously
the modes by which literacy, as an important tool in dealing with public
and other agencies, is acquired. Literacy should not be another means by
which the poor surrender their independence to the literate elites. With
Introduction
ensure that evaluations are cooperative and critical, rather than instrumental
and technocratic.
7
Hrciliuithifi Social Deivlopment Projects
overestimate some of the problems that are associated with the use of
quantitative data and techniques.
Questions of accountability
It is important, in this context, to be clear about the purpose of evaluation.
At the moment the general consensus appears to be that it is too donor-
led. Accountability should be shifted in line with the re-orientation of
evaluation so that donors are themselves subject to some form of
accountability. Traditional forms of evaluation based on short-term visits
by outsiders, usually from the North, are helpful only in certain sorts of
circumstances. A much wider basis for evaluation needs to be established.
There needs to be much longer contact with beneficiary groups from which
a more participative evaluation can be constructed and from which the
groundwork for self-evaluation can be established. As a result of this longer
time perspective, much richer material can be obtained and many gaps in
the information base can be filled. In addition it provides the opportunity
to be involved in the continuous process of monitoring and evaluation.
With this base it should then be possible to set clearer and more realistic
objectives. The evaluation process itself should be seen as a way of
providing space for the negotiation of objectives between donors and
recipients which are much more likely to meet with success because they
are based on more solid groundwork and an expanding trust which can
encourage realism.
Context of evaluations
Let us not. however, divorce evaluation, and indeed the whole issue of
social development, from the broader political and economic context in
which it is embedded. We are concerned with the struggle of the poor and
with the eradication of the causes of poverty. There is a danger that the
whole social development project will be captured by regressive forces that
will merely maintain the unequal status quo that currently prevails. Social
development might then be seen as merely one element in a world-wide
system which seeks to suppress' genuine social and political grievances.
Our concern to help grassroots movements involves a significant shift in
the ways in which we approach development issues. The myth of the
separation of the social from the economic, outlined in David Marsden's
paper, allows us to marginalise social issues if we are not careful, so that
they are seen as secondary to the seemingly more important economic
concerns.
Many mistakenly assume that working for non-government organisations
means working against the state. This is a naive simplification. While many
of the policies which NGOs undertake might be oriented to the establishment,
Evaluating Social Development Projects
10
Introduction
11
Evaluating Social Development Projects
12
Introduction
13
Evaluating Social Development Projects
The text
The ensuing text is based largely upon the thematic papers presented to
the conference, and on the outcomes of the group discussions — apart
from Chapter 6, which ranges more broadly over the available literature in
seeking to give a focus to future research in the area of the evaluation of
social development. In all, six workshop papers are presented in the text
in their original form; the rest of the book brings together summaries of
other submitted papers, reports on the group discussions, and further
bibliographical research.
A feature of Chapters 2-5 is the use of boxes in the text. Their purpose
is two-fold. First, to introduce into the text material from other submitted
papers, in order to give a flavour of the range of papers presented to the
conference. Second, and most importantly, to illustrate an issue under
discussion. The boxes serve as mini-case studies or examples of a particular
aspect of the group discussions, and should be read in that light. They are
not discussed in the text, but are located in such a way as to illustrate the
immediate issue under discussion. The source of the material in the box
is given, and readers could write to the author(s) directly if they wish to
see the complete paper.
14
SECTION 1
15
1.1 The meaning of social
development
David Marsden
Introduction
Thinking about social development is still couched within a national
discourse about the provision of public facilities and the public bodies
responsible for such provision. Such a national discourse is increasingly
constricting and unrealistic, given the large numbers of multinational and
transnational agencies concerned with the provision of development
inputs, and the emergence of a 'global order'. National development
strategies, the hallmarks of planning in the 1950s and 1960s, might now be
considered to be part of the problem, rather than part of the solution.
Attitudes to the formulation, implementation, and evaluation of development
projects and programmes have also changed as the relevance of what has
been termed 'sociological' knowledge has been reappraised and, indeed,
has increased in significance (Cernea 1985 and Sutherland 1987), and as
thinking about the most effective ways of distributing scarce public
resources has changed.
Specific interests in social development date back to the 1950s, when the
'social' was separated from the 'economic' by the United Nations in their
reports on the world social situation. The 'human factor' was seen to have
been neglected in thinking about economic development, and sectoral
analyses based on concerns with expanding social welfare provision were
elaborated.
A new situation developed in the 1980s with the increasing recognition
of the complexity of development intervention, and the need to understand
social and political processes, to expand and enrich economic analyses,
and to develop more appropriate tools in rapidly changing environments.
Traditional frameworks appear to have exacerbated rather than reduced
divisions between rich and poor, to have increased environmental instability,
and to have made unrealistic or inappropriate demands on the political and
social infrastructures available in many poor countries.
Social development planning has been consistently and fundamentally
concerned with the eradication of poverty and the pursuit of equity in the
identification of more holistic solutions to development problems. The
concept of the 'unified' approach, outlined by the UN and adopted in the
16
The meaning and evaluation of social development
programmes elaborated by many of its agencies, forms a focus for this new
thinking (Wolfe 1983)- Far from being merely a question of technological
fix and economic investment in industrial productivity, effective development
can be accomplished only if the complex social structural issues which
inhibit or channel development are themselves addressed.
Social development addresses problems of access to resources, the
provision of basic needs, the distribution of those resources, the room to
manoeuvre in straitened circumstances, and the effectiveness of the use of
those scarce resources. It examines the different value premises on which
policy decisions are made, and the contexts in which they are elaborated.
It takes as its starting point the willingness of governments to intervene to
direct development efforts, and to contribute resources to the satisfaction
of basic needs and the redistribution of assets on a more egalitarian basis.
But it also recognises the inability of many governments to intervene
effectively, and the rising importance of what are termed non-governmental
organisations (NGOs) in the provision of resources, to supplement those
of governments.
A central issue is the exploration of more appropriate strategies for
development which do not rely on outside resources, which encourage
self-reliance and community participation, through more decentralised
modes of decision-making. It cannot just be about the elaboration of
specific social sectoral activities, but must aim to address wider issues in
which such social sectoral concerns are rooted. It is only through an
appreciation of the socio-cultural and political context in which development
decisions are made that more appropriate managerial and decision-making
tools can be developed.
Background
The background to the current debate is rooted in two approaches — the
sectoral and the holistic. The sectoral approach separates out different
spheres of activity for individual professional treatment. These spheres of
activity (education, health, personal social services, and housing being the
most notable) consume increasing proportions of the social product. The
sectoral approach is based on a series of questionable premises. The first
of these premises assumes that a distinction can be made between the
'public' and the 'private' domains, and that different criteria of performance
should be used in their measurement and evaluation. A second premise
assumes that particular aspects of human activity can be isolated for
separate treatment, thus distinguishing between the different social sectors
and those areas of responsibility which are governmental and those which
are deemed not to be.
17
Evaluating Social Development Projects
The sectoral (or specialist) approach and the more holistic (or generalist)
approach have been complementary foci, and attempts to balance the two
have been at the centre of interests in social development. Firstly there has
been the traditional concern with social welfare activities, and the analysis
of changes in the various social sectors and in thinking about the public
provision of such facilities and resources (Hardiman and Midgley 1982).
Secondly there is the more general concern with new directions in
development policies as traditional strategies for intervention have been
found wanting (McPherson 1982).
It is commonly believed that one of the major activities of government
is to administer and supervise these 'public' domains. The debate within
these sectoral domains is between the residualists, who maintain that
government's role should be minimal, and the substantivists, who argue
that more substantial government investment can serve a redistributivist
role in the interests of greater equity and is a guarantee against exploitation
and the exigencies of the free market. Behind all these discussions remains
the question of social responsibility and the expected role of government
in fulfilling its obligations. As is readily observable, there has been in the
last few years a major debate about the limits to government, as questions
of 'people power' are translated into issues of individual responsibility for
the provision of what was formerly publicly provided, and as the legitimacy
of governments and their ability to deliver and their legitimacy are seriously
undermined.
The 'integrated' or 'holistic' approach, it might be argued, draws its
inspiration from the tradition of political economy and is characterised by
an understanding of the history of change, in which, in the field of
interventions in the name of development, the planner, the policy maker,
and the agents of change are seen as part of the analysis; in other words,
it is recognised that they bring with them to the analysis their own biases,
the results of their own, acknowledged, cultural heritage. In such an
analysis it is recognised that 'sectoral' intervention of a narrow sort often
fails to understand the interconnectedness of phenomena and is unlikely
to achieve the desired aims. Because this approach operates 'above' the
level of direct action, it is often perceived as being of less instrumental
value.
The two approaches represent different movements in the social
sciences (movements over time as well as in terms of theoretical
underpinnings). They are rooted in different intellectual traditions. The first
tends to be much more empirically based and instrumentalist, rooted as it
is in the work of specific social sectors. It is concerned with getting a job
done. The second approach is much more questioning and reflective. It is
18
The meaning and evaluation <IJ'social development
much more concerned with the analysis of the ways in which things are
done. Traditionally the former approach has been associated with the
'positive practitioner' and the latter with the 'negative academic', and
practice and theory have thereby been artificially divorced (Chambers
1983). The two approaches, however, are not necessarily to be perceived
as mutually exclusive.
The 'sectoral' tradition has been very important in mapping out separate
areas of expertise — health, social security, education, housing, personal
social services — within what became known as the social services. It has
built up major cadres of professionals who consume large amounts of
public money and who generate their own justifications for maintenance
and expansion. It is this apparently unending increase in the needs of such
public services that has generated the debate about the limits to their
growth, and has given rise to the search for alternative forms of provision
as structural adjustments necessitate the re-alignment of government
interests. This adjustment has been accompanied by attempts to rationalise
service delivery and apply 'value for money' and more effective management
criteria to public sector enterprises. If elements of these services can be
devolved, then government expenditure can be reduced.
In the 1960s, social development meant the planning of the social
services. As the welfare tradition within the UK and Western Europe
developed, a comparative social policy also developed which used the
same sectoral underpinnings. It was assumed that the major questions
asked within Western Europe were also appropriate for the newly
independent nations of the world and also that the world could be
unquestioningly split into different sectors. The lessons that post-war
Britain was learning were also applicable to her ex-colonies. Comparative
social policy stressed continuity in provision, and took for granted the
increasing responsibility of the state for social provision.
In the 1970s a more comprehensive understanding of the cultural
differences which separated the European experience from that of the rest
of the world developed as the naive faith in the efficacy of planning and
the superiority of modern technology was questioned. There was a
realisation that the models used to interpret social activity in Europe were
often inappropriate to other contexts. There was a growing realisation that
the administrative apparatus which had been developed in the North could
not so easily be grafted on to the changing traditions of the South — cultural
circumstances were different and resources were not available.
An anthropological relativism, with its roots in nineteenth century
political economy, began to re-emerge as a knowledge of cultural
differences and a questioning of different value systems became more
19
Ertiluatiiiji Social Dcrelopmeut Projects
20
The ineeiuiiif' and era/nation of social (Icivlopnicnl
21
Evaluating Social Development Projects
22
The meaning cmd evaluation of social development
The challenge
To ensure that real changes are forthcoming, attention must be focused on
capacity building, through the development of more appropriate educational
and organisational facilities. Attention must also be focused on the
problems associated with the empowerment of the under-privileged. And
if projects and programmes are to be more successful, the question of
sustainability in an interdependent world must be addressed. This is not
merely a question of environmental sustainability, but also of social and
political sustainability.
The challenge which faces social development workers in the 1990s is
one of responding to the changing world order. A radical redefinition of
planning is required. As Sachs has suggested:
23
l:reiliuithifi Social Davlopmetit Projects
24
The meaning and eralualiou of social development
25
Evaluating Social Development Projects
26
1.2 The evaluation of social
development
Peter Oakley
27
Evaluating Social Development Projects
activity during the lifetime of a project. Arguments for evaluation stress the
importance of providing project staff with the information needed to assess
a project's progress, in terms of its objects, and to make any corresponding
mid-term adjustments; equally they point to the need to justify expenditure
and to estimate the returns to a particular level of investment. Evaluation
has become the exercise whereby an assessment can be undertaken of the
impact of a project, and decisions made concerning replication, extension,
or the closing down of project activities.
As project interventions are broadly dominated by a perspective which
sees development largely in tangible, physical terms, so the practice of
evaluation is dominated by an approach which reflects this development
perspective. This dominant interpretation of evaluation is concerned with
measurement, with giving a numerical value to the supposed outcome of
a development project. It is concerned with the effort expended, the effect
of the project in terms of its original objectives, and the efficiency of the
project in terms of the use of resources. Essentially this approach to
evaluation measures a project's tangible or material performance, and
overwhelmingly this performance tends to be the sole criterion for
judgement on the project's outcome. The most widely used analytical tool
of this form of evaluation is cost-benefit analysis (CBA), which is a quantitative
formula for assessing the merits of a project in terms of the returns to
investment. This approach to evaluation is entirely relevant to the tangible
or material outcomes of a project, and should present an acceptable
quantitative understanding of project success or failure.
Despite its widespread use and acceptance, this dominant paradigm of
evaluation is not without its critics. One area of criticism is more technical
in nature and revolves around the twin issues of attribution and causality.
In other words, when evaluating in strictly quantitative terms, what is the
cause-effect link between the project and the ostensible outcome; and
secondly, how far can perceived outcomes be attributed to project
activities? Furthermore, within a paradigm that accepts in principle the
need for cost-benefit analysis, there are criticisms of a technical nature
relating to CBA costs, procedures, and potential for manipulation. More
substantially, however, criticism of this dominant paradigm suggests that
it is essentially a limited and static form of evaluation which is unable to
reflect adequately the outcomes of projects which are not exclusively
quantitative in nature. It is also time-consuming: its techniques can often
lead to major evaluation exercises which absorb the time and energies of
project staff. Furthermore, it is suggested that this approach has a built-in
bias towards favourable quantitative outcomes, and rarely reflects the
unforeseen consequences of project interventions. Essentially it is argued
28
The meaning and evaluation of social development
29
Evaluating Social Development Projects
30
The meaning and evaluation of social development
31
Evaluating Social Development Projects
32
The meaning and evaluation of social development
33
Evaluating Social Development Projects
34
The meaning and evaluation of social development
35
luciliiciting Social Development Projects
Concluding comments
This paper has not presented a model for the evaluation of social
development programmes and projects. Even if such a model existed, it
would be difficult to argue that it was universally applicable. The purpose
has been to present the issue conceptually and also to present a broad
framework of issues which we can use in considering other material
presented in the conference. This framework inevitably gives rise to the
core material in the evaluation of social development: methodology,
indicators, partnership and the role of the evaluator — and these will each
be taken up in subsequent papers. An important issue before us here is that,
although this paper has presented a conceptual framework for the
qualitative evaluation of social development, it has drawn largely on
literature from a context entirely different from the Third World, and it has
found few practical studies of relevance. It should be seen as a guide to
our discussions and analysis in the conference with the view to formulating
a statement more relevant to social development in the Third World. Our
concern in this conference is to see if we can begin to construct an
appropriate framework for the evaluation of social development (if
'evaluation' is indeed the appropriate word), and identify and characterise
methodologies and procedures which might support this framework.
Our discussions should take place against the background of several
broader issues related to the evaluation of social development. Evaluation
is not a neutral or ideologically untainted activity, and it does not take place
in a vacuum. While we must concern ourselves in the workshops with
developing a practical framework for evaluation, this must be done with
these broader issues in mind:
1 Does the evaluation of social development imply a fundamental
questioning of the nature of evaluation? Evaluation is more commonly seen
as a structured activity, usually externally initiated, with a paraphernalia of
evaluation designs, questionnaires, and data-processing techniques. Does
36
The meaning and eraliiatioit of social ch'tvlopmciil
the evaluation of social development even question the use of the term
'evaluation', and do we need to consider an alternative terminology? In
short, does the evaluation of a process like social development demand a
different action, with a different vocabulary and different instruments?
2 Whatever its nature, evaluation is not a value-free exercise. Conventional
evaluation approaches are often criticised for the implicit value assumptions
they make in judging programme or project performance. The very term
'evaluation', with its implicit suggestion of'judgement', is value-laden, and
it would appear impossible to construct an 'evaluation' exercise which is
neutral. Will indeed it be possible to construct a framework for the
evaluation of social development which is value-free? If not, how can we
build this inherent bias into our framework at least to mitigate its impact?
3 Inevitably in evaluation exercises, some people take the lead, give
direction, and carry out the evaluation on the assumption that the results
of the exercise will authentically represent both the project outcome and
the views of those who experienced the project's activities. In evaluating
social development, how can we safeguard the authenticity of the exercise,
so that we can be unreservedly sure that its outcome is a faithful reflection
ofthe actual situation?'What pre-conditions need we establish to ensure
this authenticity, and how and by whom would these pre-conditions be
agreed? Primarily in the evaluation of social development we must strive
to avoid the accusation that the picture presented is an external construct
and not a true reflection of reality.
4 The evaluation of social development may well cause us to question
exactly what it is that we are seeking to evaluate? Evaluation exercises are
invariably dominated by objectives — normally preordinate objectives
which the evaluator uses to measure outcome — as opposed to being
equally concerned with outcomes within the broader project context.
When we evaluate social development, therefore, should we take a holistic
view and seek to understand project outcome more generally, or merely
limit the exercise to the preordinate objectives?
5 Does the evaluation of social development imply an emphasis upon an
examination of practice to help construct the theoretical basis of the
evaluation, or must established theory always determine the nature of the
evaluation? Should we think of turning things around and starting off with
the practice, or do we always need the security of this established theory
before we undertake an evaluation?
37
Social Development Projects
... no minor tinkering will suddenly bring the usually taught and
practiced methods of evaluation up to scratch: it is not a case of this
powerful school of social science research having a few easily
corrected flaws, some practical limitations requiring a little adjustment
here and there to make it fit evaluation's demands. No, what is needed
for a truly sensible evaluation methodology is an entire shift of
perspective in fundamental assumptions, working concepts,
methodological tenets and research values. When this total re-think of
evaluation occurs, the almost slavish-seeming adherence to dominating
conventions of formal research design, elaborate statistical procedures,
etc., is inevitably broken, as strategies, goals and outcomes become
redefined to accord more with the realities of everyday phenomena
and less with methodological dictates.
Whether this conference can in the space of two and a half days bring
about an entire shift of perspective' in the evaluation of social development
is debatable, but at least we can give it a try!
38
SECTION 2
QUALITATIVE DIMENSIONS
OF SOCIAL DEVELOPMENT
EVALUATION
39
2.1 Qualitative dimensions of
social development
evaluation:
Thematic paper
Muhammad Anisur Rahman
Today for the first time the tribal labourers of this area are going on
strike, stopping work on their own initiative. This may result in a wage
rise of say Rs 100 per year or RS 50 per year — What is important to
us, however, is that we are asserting that we too are human beings.
Introduction
Although the subject of this conference concerns 'projects and programmes
in the Third World', this paper offers thoughts towards an overall
orientation on the question of qualitative dimensions of social development,
at the grass roots as well as at higher levels. It is argued that, keeping in
view the considerations presented in this paper, actual indicators for
evaluating social development in any specific project or programme must
be developed in the concrete context of the project or programme's
specific objectives, its socio-economic context and the people's culture.
Furthermore any evaluation will be developed by the project community,
i.e. the people concerned and others who will have a responsibility for
implementation. Project and programme evaluation is considered also to
be a learning process for the project community, and indicators should be
subject to progressive refinement or modification as experience is gained
and ideas develop. Finally, there is no reason why the concern for social
development and its evaluation should be restricted to the Third World.
For the sake of convenience, social science and development cooperation
have had an overwhelming bias towards quantitative indicators for
assessing developmental progress. This has been unfortunate and has
generally given a distorted approach to development policy and action.
40
Qualitative dimensions of social development evaluation
41
Evaluating Social Development Projects
42
Qualitative dimensions of social development evaluation
This lifestyle is now fast disappearing along with the forest ...
(Dasgupta 1986, chapter 6)
A different dimension of people's self-assessment of progress was revealed
in an evaluation of a project in the Philippines which provided animation
work to promote organisations of various categories of rural workers in
four villages (Rahman 1983). The organisations were engaged in collective
activities which brought economic gains in different degrees to their
members. A random sample of the members of these organisations were
interviewed, and people were asked what was the most important benefit
they had gained by organising. Without exception, every one of them
replied 'education'; the gaining of knowledge, through actual experience,
that they could improve their status by organising and working together,
was the single most important benefit they had gained. No one, even out
of those whose economic status had improved substantially, mentioned
economic gain as the most important.
More such examples could be given of people's perceptions of 'progress'
or 'development'; perceptions and priorities in terms of dimensions of life
which are not readily quantifiable and are yet of profound value to the
people. But let us end this section by recalling the following revealing
observation in a discussion of the ORAP movement (Organisation of Rural
Associations for Progress) in Matabeleland, Zimbabwe:
Significantly, the translation of the concept of development into
Sindebele (local language of Matabeleland) is 'taking control over
what you need to work with'. The names of most ORAP groups also
reflect these concerns. A few chosen at random are: Siwasivuka (We
fall and stand up), Siyaphambili (We go forward), Dingimpilo (Search
for life), Sivamerzela (We're doing it ourselves), Vusanani (Support
each other to get up) ... (Chavunduka et al., 1985)
As I have observed elsewhere (Rahman 1989), in apparently naive words
these expressions of people's collective self-identity reflect deep
conceptualisations of popular aspirations. Hence they reflect implicit
popular notions of 'development'. The people want to stand up, take
control over what they need to work with, to do things themselves in their
own search for life, to move forward, supporting each other. These are as
much part of their 'basic' needs as the 'basic needs' of conventional
development thinking. These are holistic needs, or aspirations, which
cannot be quantified without distorting their basic spirit.
Finally, when I recently visited a relatively well-off village in Hungary,
members of the community claimed that efforts to promote participatory
43
Evaluating Social Development Projects
development are no less needed for the 'rich', and that 'what hurt most is
the indignity of being forced to vote for a chairman who I know is corrupt'.
This indignity is a measure of one's poverty, notwithstanding one's material
well-being, and a measure, therefore, of social underdevelopment.
44
Qualitative dimensions of social development evaluation
Elements of empowerment
A quantitative element of empowerment is control over economic resources;
but progress in this matter is by itself no indication of enhanced social
power of the underprivileged to assert their developmental aspirations and
their freedom to take initiatives for their self-development. The essential
qualitative elements of empowerment are well suggested in many writings
on participatory development, from which I would highlight three:
45
Evaluating Social Development Projects
46
Qualitative dimensions of social development evaluation
Women's development
This cannot be overemphasised in view of the almost universal phenomenon
of male development at the expense of women's development. The
question is complicated by culture and religious beliefs in many situations
where exogenously conceived norms of gender equality may not be
appropriate. However, progress towards a position where women are able
to articulate and assert their own points of view concerning gender
relations in all spheres, and the evolution of gender relations towards
greater equality as assessed by the women themselves, may be suggested
as an important indicator of social development. For most societies this
implies independent organisations for women, at least at the primary level.
Development of organic knowledge
The development of a collective human personality involves not only
doing things, but also advancing simultaneously the analytical understanding
of the evolving situation in all its dimensions — social, technological,
political, and cultural. This is necessary for an intellectual appreciation of
the unfolding experience, as well as to establish guidelines for future
action, based on systematically experienced knowledge. In most societies
the task of systematically developing such knowledge has become
separated from the actual evolution of social life, and has become
concentrated in the hands of professionals who by and large live a life very
47
Evaluating Social Development Projects
different from the lives of the ordinary people. For social development this
has created the question of the relevance of much of the knowledge thus
developed, which is not rooted in people's lives, and it has also contributed
to retarding the process of development of the people by undermining
popular knowledge and their ability to create and advance knowledge.
While professional knowledge of some kinds remains valuable for social
development at certain levels of decision making, the development of self-
knowledge by the people as an organic part of their life's activities —
organic knowledge — is perhaps more valuable. This organic knowledge
helps to develop knowledge more immediately relevant for people's self-
development, as well as for sustaining people's power to assert themselves
vis a vis other social sectors. Organic knowledge, therefore, must be
underlined as an important indicator of social development.
48
Qualitative dimensions of social development evaluation
the shaping of macro policy and in the conduct of public affairs. Neither
the democracy of the so-called free world nor that of the so-called socialist
democracies has ensured this natural right of the people. The nature of
effective political parties and the outcome of electoral processes to
determine macro-leadership of societies in the free world are critically
determined by the distribution of economic power, and the economically
underprivileged masses merely have the choice of influencing which set
of privileged elites will rule over them. On the other hand, in socialist
countries, albeit with greater economic equality, the Parry remains typically
unaccountable to the people. By one guise or another, in either type of
society the real macro power remains in the hands of privileged elites. In
this context, the entry of Solidarity of Poland, a truly workers' party, on to
the stage of power is a so-far unique event that opens up the possibility
in at least one modern society of a real sharing of power between
privileged elites and the working people. If this possibility becomes truly
fulfilled, and working people find a real voice in the affairs of the state, then
the standard of attainable democracy and for that matter of macro-social
development will have reached a new height, against which claims to
democracy by other states, of both the free and the socialist worlds, could
well be assessed.
Cultural diversity
Finally, in recent years we have witnessed an upsurge of assertions by
popular sectors of their cultural identities, in opposition to attempts by
dominant powers to impose a monolithic culture or ideology of development
upon the people. In debates on individual vs. collective ownership of
property, it is hardly even recognised that many indigenous people do not
have the concept of humans owning natural resources such as land and
water. Many such communities have instead the concept of humans
relating to nature as a partner in life (offering the rest of us a model for the
preservation of nature for 'sustainable' development, rather than its
destruction to satisfy the human lust for acquisition and conquest).
Conclusion
Social development, from the point of view of the broader society,
necessarily implies people's development at the grass roots; otherwise only
the abstract concept of the 'nation state' would be promoted, and little else
would develop except structures for manipulation and repression. People's
development in its turn implies development with a people's authentic
culture, which itself would develop in the process, absorbing elements
from other cultures with which it would interact but which cannot develop
49
Evaluating Social Development Projects
50
2.2 Qualitative dimensions of
social development
evaluation:
Review of workshop papers
and discussion
Introduction
The working group on qualitative indicators began its discussion by
reviewing the present state of evaluation. It suggested that this was
characterised by an excessive desire to quantify, by assumptions about
people's behaviour and activities which were not supportable, by the often
tenuous relationship between the results of an evaluation and what
actually was happening, and by the relatively short time that evaluators
spent with projects, which severely limited their ability to come to grips
with what was really happening at the project level. Furthermore traditional
forms of external, quantitative evaluation had introduced new forms of
status difference within the spectrum of donor-project relations, and this
had provoked increasing unease at the project level. It had also led to some
distortion and consequent devaluing of the results of evaluation, and had
introduced an overriding sense that project staff were being tested and
found deficient.
The working group also discussed the general issue of evaluation from
the point of view of the different actors involved in the process: funders,
project staff, and local people. All evaluations are coloured by this
kaleidoscope of actors, interests and interpretations. The actors all approach
evaluation from different perspectives, and have different expectations of
its outcome. In this respect, therefore, it is important to ask what does
qualitative evaluation mean for these different actors? Project staff and local
people may well see qualitative evaluation more in terms of illustrating the
political dimensions of project activities; donor agencies, on the other
hand, may be less enthusiastic and may prefer to insist upon quantitative
data and information. Certainly there currently exists a general consensus
that quantitative indicators are better able to evaluate project performance,
and this consensus is a fundamental hurdle that qualitative evaluation
faces.
51
Evaluating Social Development Projects
The working group extended its opening discussion into the broad area of
social development, its meaning and its characteristics. It concluded that social
development is essentially a dynamic process directed towards building up the
organisational base of poor people in order to give them some power to
intervene in the development process. Social development is also to do with
the satisfaction of human needs of all kinds: for emotional fulfilment, for
democratic rights, for freedom to form political parties, and the right to
participate in national life. In this respect, and although the workshop would
be concentrating on the programme and project level, qualitative indicators of
social development at the national level were also important. At both the
national and local levels, social development is to do with changing the nature
and balance of power relations, and all indicators of social development must
reflect this dynamic relationship. In this respect it has to be recognised that not
only governments but also NGOs are part of these power relations, although
they may operate at different levels and also have different perceptions of the
importance of these power relations in development.
A general framework for the working group's opening discussion was
provided by Martin's paper, which was less concerned with the detail of
qualitative indicators of social development as such; it used the case study
of food aid in Peru and Colombia to illustrate not only the misuses of social
development programmes but also the political sensitivity of the evaluation
of such programmes. In such food aid programmes, Martin argued, such
qualitative indicators as power, control and self-sustainability should be
employed; but, because of their political sensitivity, few food aid evaluations
use such indicators. Furthermore the evaluation of social development
programmes like food aid should consider the international context of
power relations between donor and recipient countries. In practice this
never happens. Martin's paper further ranged over a number of key issues:
the longer-term consequences of food aid, as opposed to its immediate
effect; the potential conflict between the political and economic objectives
of the donor agency and the supposedly developmental objectives of the
programme; the consequences of food aid programmes for the target
population; and the crucial issue of people's power as central to social
development. Martin concluded that, whatever the nature or content of a
social development programme, its validity will depend upon whether
poor people have participated both in the process of social transformation
and in the evaluation of programme consequences.
52
Qualitative dimensions of social development evaluation
53
liralnatiiifi Social Derelopiiieitt Projects
54
Qualitative dimensions of social cleivlopnient evaluation
The working group also raised the question of how far we could
distinguish between conditions and indicators of social development, or
whether they were one and the same thing? As an example, the group took
the issue of participation and asked whether people's participation was an
essential indicator, or rather whether it was an essential condition of social
development? The group concluded that the two were closely inter-related:
what are seen as essential conditions of authentic social development, for
example people's participation or a functioning people-based organisation,
will in effect be the indicators to 'evaluate' the process. The identification
of these 'essential conditions' of social development should inevitably lead
to the identification also of relevant indicators.
55
Evaluating Social Development Projects
56
Qualitative dimensions of social development evaluation
(Harding 1989:6-8)
57
Evaluating Social Development Projects
58
Qualitative dimensions of social development evaluation
59
Evaluating Social Development Projects
(Robinson 1989)
60
Qualitative dimensions of social development evaluation
— More general access for poor people to the resources necessary for
development; greater distribution of resources available.
— Greater expression of self-identity by poor people, and the right to full
involvement in national life.
— Movement towards greater social equality, measured through the
strengthening of non-hierarchical relationships among poor people.
— Improved levels of caring and concern for others at national level.
— Overall improvements in physical wellbeing and security.
— Reduction of dependency relationships among poor people.
— The development of organisational capacity among poor people.
— The creation of awareness.
These do not constitute a definite series of areas, nor a model for the
evaluation of social development. Their purpose essentially is to help
break down the overall concept of social development into a series of areas
of potential change which can be used as a guide in an evaluation exercise.
They will help us to answer the question 'What could have been the effect
or impact of a particular social development programme?' Each of the
above criteria is expressed in general terms, and will need to be interpreted
more carefully within the context within which the programme is operating.
The series of areas constitute in effect a framework for reference in
considering the issue of social development evaluation. This is how the
series should be used and refined in terms of the context of the social
development programme being evaluated.
The next and more detailed stage in the process is the identification of
relevant indicators which would illustrate the above areas of change and
form the basic structure of the evaluation exercise. Here the working group
commented that the identification of such indicators would need further
time and research. This more detailed stage could best be done by a smaller
workshop concentrating solely on the practice of qualitative evaluation at
the project level. Such an exercise would help isolate the conceptual
framework of the qualitative evaluation of social development — which
essentially has been the theme of this workshop — from an analysis of the
methods and techniques of this form of evaluation.
In its report, however, the working group presented several examples
of the kinds of indicators which would need to be identified. Indicators
6l
Evaluating Social Development Projects
The working group stressed that the above were presented as examples
of the kinds of indicators which would have to be developed at the project;
they did not constitute an authoritative checklist, nor were they necessarily
universally applicable.
The working group concluded its discussions by stressing that in project
evaluation, the responsibility of project groups to the donor agencies
remains implicit. It is the donor agency which usually initiates the demand
for evaluation, and the indicators suggested often reflected a particular
world view and particular requirements. Such views and requirements
should be negotiated with project participants before being incorporated
as reliable and appropriate indicators. These negotiations should take
place at the planning stage, rather than during the execution of a particular
project, and they should be focused on the agency, rather than on the
project. Indicators should be based upon the internally perceived realities
of the project, and not upon the desk-orientated demands of the external
agency.
62
Qualitative dimensions of social development evaluation
63
SECTION 3
THE METHODOLOGY OF
SOCIAL DEVELOPMENT
EVALUATION
65
3.1 The methodology of social
development evaluation:
Thematic paper
Ignacio Garaycochea
Introduction
Theoretical and practical aspects of social development evaluation are
largely determined by the different interpretations ascribed to social
development. In an explicit or implicit way, governments, social development
researchers, donor agents, local intermediaries, evaluators, and beneficiaries
have a particular understanding of what social development is. To some
extent, these different interpretations lead to different modes of putting
social development into practice and therefore on how to evaluate it.
Under these circumstances it would be difficult to pose one single
methodology for the evaluation of social development projects.
However many particular differences might exist in the interpretation
of social development, there has been a growing effort in the past decade
to assemble them within a broader understanding. Social development is
regarded as a process in which the sectoral and holistic aspects of
development move forward and maintain a correspondent relation. At
this point we could argue over the methodology of evaluating social
development projects as 'a process within a process'. Processes need to be
described and interpreted as they occur over time, as qualitative and
quantitative events, in order to gain a complete view of the social effect
caused by project intervention. The evaluation of qualitative objectives
(for example, participation, solidarity or popular education) represents a
particular area of interest when looking at social development projects and
programmes.
One of the key elements when considering strategies for social
development evaluation is the relationship between the concept and its
operational dimension. How social development is understood or conceived
will have in turn an effect on how it is put into practice, and therefore how
it should be evaluated. There is a vast body of literature on the concept of
social development, though still few commonly shared definitions are to
be found. We would prefer to think in terms of more than one concept
(even sub-concepts) related to the definition of social development. These
66
The methodology of social development evaluation
67
Eraluatiiig Social Development Projects
life span or by the time it has finished. Effects can appear later in the future,
and in a context where we will not be able to determine to what extent
changes or actions taking place are due to the project's past intervention.
Another issue related to this topic is to ask to what extent project
members have control of the processes taking place in it. We might know
when and how to switch on the mechanisms to start a social development
process, but we do not know when and where it is going to finish. Social
development projects are dynamic and do not necessarily follow a
predetermined direction. We should take into account that social
development projects can develop unexpected results. Even though a
project might not accomplish its initial goals (the predetermined parameters
against which evaluation is supposed to rest) the project can still be a
success. This is the case with a Peruvian Social Forestry Project (described
in detail below), of which one of the objectives was to create an
environmental consciousness among Andean peasants in order to preserve
the ecological balance of the area. However, the project's methodology of
intervention/reflection/action enabled peasants to realise that among their
priorities, the strengthening of communal organisations was something
more important to do, though they did not disregard the ecological issues.
When we describe and interpret processes taking place in a social
development project there are two distinct dimensions of where to look for
the attainment of objectives: the collective or group processes and the
individual or personal processes and changes. How to detect and describe
these features through evaluation is a major task in the arena of social
development.
68
The methodology of social development cvcihuilion
69
Evaluating Social Development Projects
becomes a new input that enriches the social experience. Thus, consciousness
raising is not an objective in itself, but just one link of the chain.
Since social development is concerned with every individual in society
and no one should be neglected, social indicators are not meant to describe
and interpret only the collective or group phenomena, but to take into
consideration also the individual processes. Social creativity, self-reliance,
or popular democracy must be described and explained at the group and
individual levels if a complete understanding of the process is sought.
However, it should be important to explore if indicators have a universal
validity or pertinence for evaluation. Is it possible to use one type of
indicator for the description of collective and individual processes? Would
it be possible to make a particular inference from an indicator that provides
the sources for a general assessment?
If methodology is going to be defined by a particular understanding or
perspective of social development and the context in which the project
operates, indicators must also have this flexible nature. Indicators for
evaluating social development may be developed in the concrete context
of the project or programme's specific objectives, its social and cultural
aspects, and its development by the 'project community'.
Concluding comments
In the search for better ways to evaluate social development projects, what
is important is to recognise the existence of more than one way to go about
it. Social evaluation should be considered as a process within a process in
which a suitable methodology can be tailored to fit project and contextual
characteristics. Social, cultural and economic considerations of the project's
internal and external milieux should help to mould the methodology.
Indicators should be considered according to the expectations of the
methodology just defined. The evaluative process is considered to be an
educational experience in which 'communal' participation is highly desired,
a learning process of knowing what it is we are evaluating and how to do
it. Different perspectives of the same process can be obtained if different
groups participate in the exercise: beneficiaries, neighbouring people, field
workers, project staff, and external facilitators. Finally, what is important
when evaluating social processes is to experience a sense and feeling of
progress and change caused by people as masters of their destiny.
70
The methodology of social development evaluation
71
Evaluating Social Development Projects
72
The methodology of social development evaluation
73
Evaluating Social Development Projects
Concluding comments
For the evaluation of social projects, such as the one described here, there
are no pre-determined methodological schemes. The methodology is
tailored in relation to the project's surrounding context and its general
scope. In the case of Arbol Andino, a number of issues should be noted:
74
The methodology of social development evaluation
75
3.2 The methodology of social
development evaluation:
Review of workshop papers
and discussion
Introduction
In the context of Garaycochea's paper, the working group reviewed a
number of other papers presented in the broad area of the methodology
of evaluation of social development. (These papers are listed at the end of
this chapter.) In general terms the working group emphasised the
relationship between the conceptual framework for the evaluation of social
development and the methodologies and instruments employed. The
former clearly influences the latter, and indeed dictates the kinds of
instruments and methodologies used. Similarly it was suggested that there
is no one universally applicable methodology for the evaluation of social
development, but rather a range of alternative approaches and elements
which can be drawn upon in relation to the demands of the local context.
Different constituencies will have different needs for evaluation, and the
different actors at the different levels within the project community will
each have different interests, which will have to be catered for in the
methodologies employed.
The evaluation of social development does not take place within a
vacuum, and it should be seen as an aid to development, rather than an
instrument of control. Inevitably, before we get involved in the methodology
of evaluation, we have to ask questions such as for whom is the evaluation
intended, and why is an evaluation necessary? Methodology comes at the
end of the evaluation process, and will be influenced by the kinds of issues
— indicators, partnership, and the role of the evaluator — which are
discussed elsewhere. The methodology of evaluation is the means whereby
we link a series of potentially disjointed activities; how we build coherence
into the evaluation exercise. The overall methodology of the evaluation
exercise should be distinguished from the methodological techniques, for
example, questionnaires, which may be used. The evaluation of social
development should be first and foremost an educational as opposed to
a managerial tool, and it should be seen as emerging from and reflecting
methodologies implicit in social development as a process. Similarly the
76
The methodology of social development evaluation
77
Evaluating Social Development Projects
78
The methodology ofsocial development evaluation
79
Evaluating Social Development Projects
80
The methodology' of social development evaluation
timing (14 days for the completion of the survey), only a small number
of villages were selected. The sample chosen was representative of
the various class and ethnic groups of each village.
The results of the evaluation were then presented in a variety of
ways: in quantitative form (tables), qualitative form (case studies),
outsider's and insider's views about the social structure, a detailed
historical account of social change and development in Barka, and
broader aspects of contemporary development in the area defined by
cadres from the Department of Public Administration and members
of the people's Committee. Appendices were attached to give further
information: maps, tables of age/productivity ratio; and amount of
availability and distribution of consumer goods.
81
Evaluating Social Development Projects
82
The methodology of social development eraliuition
between the need for information and the implications for participants'
other responsibilities.
6 Finally any methodology for the evaluation of social development must
be systematic in nature and follow a regular and agreed course. Social
development can be understood only if observations are made systematically
and not in a one-off, ad hoc fashion. A rhythm of evaluation must be
established and preferably internalised within the project, and lead to a
continuous and on-going process which is analysed periodically.
83
Efaluating Social Development Projects
85
Evaluating Social Development Projects
86
The methodology of social development evaluation
87
h'rcilitatiiifi Social Development Projects
Instruments of evaluation
The final and ultimately more detailed dimension of social development
evaluation concerns the instruments to be used in the actual evaluation
exercise. In this respect it must be said that the papers presented to the
working group contained less empirical evidence or suggestions for
innovative or imaginative instruments of evaluation. This state of affairs
tended to confirm the overall view that the evaluation of social development
is still very much at the conceptual and broad framework level, and that
88
The methodology of social development evaluation
0
u Survey Observations
T Questionnaire Checklist
sI Interviews
Content Analysis Objective
D (Files) Assessment
E
1
D Diary Life History
E
89
Hirilnatiiif> Socicil Development Projects
Prompts Recording
Questionnaires Writing notes
Check-lists Audio recording
Pictures Video recording
Project schedules Diagrams
Graphical presentations
Points (3) and (4) were not considered in any depth by the working
group, which indeed reflects the lack of substantial material to date on
these issues, and points the way to a future research agenda for donor
agencies interested in improving their evaluation of social development.
90
The methodology of social development evaluation
Concluding comments
It was suggested in the working group that the methodology of evaluation
of social development required "... an entire shift of perspective'. If this is
true, then it must be recognised that this entire shift' is still taking place
and is not yet concluded. The papers presented on methodology confirmed
that this shift of perspective is underway, but that it is still largely
exploratory. The papers were strong on conceptualisation and on a broad
framework for evaluation methodology, but more tentative on the practical
aspects of instruments, aids, and information storage and retrieval. The
papers of Sollis and Moser, Uphoff and McPhail were more substantial in
this respect, although the latter was largely quantitative in orientation.
Collectively, however, the papers emphasised that any methodology of
evaluation must place project participants centre stage; such an approach
increased the likelihood that data and information collected would not
only be appropriate but would remain local and ensure that locally owned
91
Evaluating Social Development Projects
92
The methodology of social development evaluation
93
SECTION 4
PARTNERSHIP IN SOCIAL
DEVELOPMENT EVALUATION
95
4.1 Partnership in social
development evaluation:
Thematic paper
Rajesh Tandon
Introduction
The question of partnership in social development evaluation cannot be
raised and discussed without specific reference to the meaning of social
development and social development evaluation. I will reiterate my own
understanding of social development evaluation, in order to create the
context for my comments about partnership and social development
evaluation.
For me, social development evaluation is an intervention in the ongoing
efforts of promoting developmental initiatives. In our experience, this
intervention needs to have several characteristics if it is to promote
authentic and genuine processes and initiatives of social development in
a given context. Some of these characteristics are listed below.
96
Partnership in social development evaluation
97
k'lriliicttiiig Social Development Projects
98
1'artncrsbi/i in social ilerclojmicul evaluation
99
Evaluating Social Development Projects
100
Partnership in social development evaluation
101
4.2 Partnership in social
development evaluation:
Review of workshop papers and
discussion
Introduction
In trying to find better ways to evaluate the more qualitative aspects of
social development projects and programmes, this working group considered
[he role of partnership in the evaluation process. It examined the value of
less rigid and more flexible and responsive approaches to evaluation. A list
ot papers presented for discussion by this working group is to be found
at the end of this section.
In his introductory paper, 'Partnership in Social Development Evaluation',
Rajesh Tandon argued that, to be genuine, an evaluation had to be
educational for all social groups involved. This was necessary in order to
enhance the programme or project being evaluated, as well as to give a
sense of purpose and involvement to all engaged in the work. In addition
to being retrospective, the evaluation must be forward-looking through a
consideration of the aims and vision of the project. To enable this to
happen, a partnership which brings together the different actors in the
project community must be established.
Tandon asserted that partnership is essential for social development, to
empowerthepoorto initiatetheiroivndevelopmentxhrough the development
of linkages between similar organised groups. Such a partnership is not
easy to build. The relationship between the poor, local non-government
agencies, and international donors is, in terms of their respective powers,
an unequal one. It will take considerable time to build up relationships of
trust and mutual understanding. Building such trust is part of the purpose
of the evaluation which aims at creating mechanisms for dialogue and
reflection.
There must also be a wide range of evaluation tools available to
evaluators to facilitate this process and to enable them to tailor their
evaluations to particular cases. There can be no universal rules for such a
process. This flexible approach may draw on less formal qualitative
information, such as that derived from drawings and songs, as well as
quantitative surveys and interviews. The evaluators must share in the
102
Partnership in social development evaluation
visions and hopes of the participants, and gear the evaluation to the benefit
of the people rather than to that of the resource providers. To ensure
success, the partnership must be flexible enough to include all the parties
involved. The key themes of Tandon's introductory paper are the need for
learning, the need for dialogue, contact, and communication, and thus for
qualitative forms of evaluation.
— The 'social actors', people at the grass roots, mostly grouped into formal
or informal people's Organisations or •membership organisations'.
— The 'agents' of development carrying out catalytic and support functions,
(semi-) professional staff, and employed by a local non-governmental
development organisation.
— The 'flinders', staff employed by funding agency.
103
Evaluating Social Development Projects
Some believe that evaluation should be objective and neutral, and thus
argue that the participative approach, by its very nature, cannot facilitate
this. Conversely there are those who argue that the objective and external
types of evaluation are in conflict with the principles of partnership,
equality, and reciprocity. Verhagen maintains that a balance between
external and internal elements is optimally required. In analysing the
political dimensions of evaluations, this issue of objectivity and value
neutrality is central. There are those who argue for more precise assessments
of efficiency and impact in order to satisfy the demands for accountability
in the disbursement of public funds. The demands for managerial control
over expenditure are supposedly met through impartial analysis and
objective study. Such studies, Verhagen argues, flagrantly contradict the
proclaimed ideals of partnership, equality, and reciprocity. New evaluation
paradigms are advocated and increasingly practised within the NGO
system. These new forms of evaluation are not formal substitutes for
external evaluations. They may in fact be more intrusive if all they do is
further the control that the evaluator is able to exert over that which is
evaluated. Self-evaluation is a means of appraisal whereby the organisation
is able to assess its own activities and change direction if necessary.
104
Partnership in social development evaluation
ensured. This goes against the logic of project financing, but fits well
into the logic of long-term organisation-centred support and partnership.
3 Terms of reference determining objectives, scope, and organisation
of the self-evaluation and the external evaluation cannot and should
not be similar. But differences in orientation and the distinct
responsibilities of each of the parties involved should be discussed,
with their implications for timing and organisation of field trips,
exchange of findings, consultations, and reporting. Self and external
evaluators may have different perceptions and expectations of each
other's roles.
4 External evaluators, in order to be able to assess the position of the
NGDO/POs in their societal context and with a view to diversifying
their sources of information, may wish to contact other organisations
— non-governmental or governmental — and seek opinions from
them rather than being restricted to the NGDO/POs (the evaluandum)
as the only source of information. This may considerably annoy the
NGDO/PO concerned and affect the spirit of partnership during the
joint evaluation.
5 The questioning by external evaluators (foreign or local) of an
organisation's ideology (system of values) is a sensitive issue. Does one
have the right to do this? Equally sensitive is discussion of perceived
inconsistencies between ideology and practice.
6 External evaluations can be especially helpful in the interpretation
of self-evaluation findings and their possible implications for funding
agency/NGDO or PO relationship.
7 Joint evaluation can sometimes be more effective where it takes
place in an informal manner than where it has been deliberately
organised.
8 Since virtually all evaluations take place in the Third World, there
is a tendency to under-analyse the effect of the policies and administrative
practices of funding agencies on the performance of the organisations
they finance. Funding agencies tend to remain out of scope as objects
of evaluation.
9 Evaluation in partnership does not imply that the different parties
involved should come to the same conclusions. Perfect unanimity
would rather be indicative of superficiality of the evaluation and of an
immature relationship (or of a combination of both), than of good
partnership.
(Verhagen 1989)
105
Evaluating Social Development Projects
106
Partnership in social development evaluation
looking for opportunities to save time and money, through preventing losses,
rather than earning wages through paid employment which may be regarded
as degrading. Villagers may not perceive projects in terms of the achievement
of project objectives, nor will they necessarily be so politically motivated as
those who argue for their empowerment. They may judge projects in terms of
the increased unity that they have brought to the community, rather than in
terms of increased productivity.
It is important to admit that actors, agents, and funders operate in and
retain an attachment to different cultures. Performance is likely to be
judged on the bases of different value systems. This can be partly
circumvented if the evaluator is aware of other modes of explanation and
alternative conceptions. The process of finding different answers and
solutions from a range of different sources may strengthen an evaluation.
The need for discussion and dialogue between those involved in the
evaluation of a project is a theme which carries over into the notes made
by John MacDonald. In these notes — 'Who speaks for the people in
participatory evaluations?' — MacDonald illustrates the way in which
participation and discussion form the basis of the evaluation. Evaluators
must 'dialogue with the responses' to such qualitative questions as 'What
have you achieved in this programme?' which are asked at village-level
meetings. In this way the community evaluates what has happened and
influences future decisions. But the question of the representativity of
those who 'speak for the people' remains problematic. Intermediaries
between outsiders and insiders tend to be marginal to both cultures, and
thus not necessarily the best people to trust for interpretations.
The concept of partnership associated with sharing and the development
of common concerns is dealt with in Pramod Unia's paper, entitled 'Social
Action Group Strategies for Oxfam in the Indian Sub-continent'. The
positive aspects of partnership are stressed in a discussion of social action
groups. These were formed during the 1970s as a way of mobilising and
empowering the poor in the pursuit of common interests. Emphasis is laid
on the need for shared beliefs between groups, and common goals in order
to make the partnership work.
These social action groups have become viable and important alternatives
in implementing people-oriented development programmes. The assessment
of the groups' own social development is achieved through a focus on
educational needs, an understanding of basic rights related to land issues,
wages, and environmental concerns. Partnership is founded on an
understanding that social change can be brought about only through social
and political mobilisation of and by the poor.
107
Evaluating Social Development Projects
Self-evaluation
Philippe Egger's paper, 'Village Groups Evaluate Their Activities: An
Experience in Mali', provides an illustration of the work of the International
Labour Organisation's Programme on Participatory Organisations of the
Rural Poor, and documents an experience of self-evaluation carried out by
organised village groups in three areas of Mali. The NGO responsible is
called Six S (an acronym for 'Se Servir de la Saison Seche en Savane et au
Sahel'). Six S helps organised groups through a flexible fund providing
both grants and credit on the basis of matching funds raised from groups.
Locally available ideas, energy, and resources are pooled, and traditional
mutual help practices are transformed into organised forward-looking
development initiatives. Six S stresses the importance of self-evaluation,
which it sees as consisting of five phases:
Stages (a), (b) and (c) had been completed by September 1989, and some
preliminary results were available. During the initial orientation stage, six
basic evaluation tools were elaborated. Each tool is composed of a series
of key questions. These tools are described in the accompanying box. It
is too early to draw anything but tentative conclusions about the effectiveness
of these evaluation tools, and it is certain that they will need to be adapted
in some ways in the future.
Egger, like Tandon and Unia, argued that group analysis of group
activities is fundamental, both within and between groups. It must avoid
apportioning blame for failures. Participants have to learn how to 'speak
up' and achieve a gradual agreement with others. Egger further sees self-
evaluation as a subjective learning process which encourages people to
become aware of themselves, and to take responsibility for their future
social development. This in turn will promote self-reliance and the ability
to reflect critically on past and present achievements. This is something
which echoes Tandon's understanding that evaluations must include a
vision and strategy for the future, as well as assessing historical developments.
108
Partnership in social development evaluation
(Egger 1989)
109
Evaluating Social Development Projects
train and mobilise people at community level, and to plan future social
development projects.
Finally, Bandyopadhyay's paper on problems of self-sustained agriculture
stressed the need to understand farmers' perspectives — the opportunities
and constraints that they themselves feel, and the rationalities that lie
behind certain sorts of decision-making procedures.
Conflicts of interest
On the negative side, partnership in social development evaluation is not
easy to achieve. The conflict of interests and the unequal power of the
groups involved can make dialogue and agreement difficult. The political
and cultural dichotomies which may exist between the evaluator and the
evaluated can take a long time to disappear. Verhagen stressed the need
for dialogue and broad-mindedness in these situations, and the need to
acknowledge differences. There is a constant danger of the evolution of
anti-participatory conditions, enshrined in hierarchical structures, and of
fragmentation, and thus a constant danger of losing sight of original
objectives. To avoid this, the development of appropriate communication
and information networks is vital. This can ensure that perspectives are
broadened and wider issues are explored, as groups are linked to like-
minded groups. There is a limit to what individual groups can achieve, but
links with others strengthen collective purposes.
When trying to build up partnership we must take into account regional
and national differences which offer different opportunities and constraints.
In some circumstances it is going to be difficult to build up partnerships
in the ways in which one might desire. Partnership can be attained only
when beneficiaries start perceiving the importance of evaluation and begin
the process themselves. People at different levels in the project community
have different perspectives, but partnership can be built up through
focusing on the twin processes of monitoring and evaluation. The
monitoring process helps to broaden access to information, and can be an
empowering act in itself. It also makes evaluations more productive by
overcoming the problems of time which traditionally hinder evaluations.
The process of continuous monitoring and evaluation negates the need for
110
Partnership in social development evaluation
a rushed job at the end. Monitoring should happen at all levels simultaneously;
it is not just the monitoring of the implementation process as it affects the
beneficiaries. It should also be seen not as an alternative to evaluation, but
rather as a complement.
Finally, a series of related issues which emerged from the working
groups' discussions included:
111
Evaluating Social Development Projects
112
Partnership in social development evaluation
PramodUnia: 'Social Action Group Strategies for Oxfam in the Indian Sub-
Continent'
113
SECTION 5
nr*
THE EVALUATOR IN SOCIAL
DEVELOPMENT
EVALUATION
115
5.1 The evaluator in social
development evaluation:
Thematic paper
Delle Tiongson-Brouwers
Introduction
A friend once told me that she has two descriptions of an evaluator. The
first: an evaluator is like an accountant who tells you that you have no more
money and it is too late for you to do anything about it. The second: an
evaluator is one who confuses you with many recommendations because
he himself (or she herself) is confused.
To evaluate is to assess. One can assess an activity after it is over; or one
can evaluate the plan prior to its execution. For example, a football coach
can evaluate how the game went; or he can assess his game plan even
before the game starts. In social development, the former is generally
known as project evaluation, the latter as proposal evaluation. In project
evaluation the focus is on what has been accomplished vis a vis stipulated
objectives. For proposal evaluation, the relevance of objectives and
strategies in relation to the particular milieu and problems is a key area of
inquiry.
In both proposal and project assessment, the evaluator has to deal with
two basic clients: the development funding agency (DFA) and the
proponent or project holder. Both have pragmatic needs and concerns: the
DFA has the need to channel resources to the right project; and the
proponent has the wish to get his or her project funded. Thus, the evaluator
is like a tight-rope-walker, whose credibility rests on maintaining a
scrupulous balance.
Project proposals are indispensable for getting funds. When a proponent
sends a proposal to a DFA, the request is studied, for example, according
to its content and clarity of presentation. It has been common practice for
DFAs to make decisions based solely on the information contained in the
proposal. This was not because the DFAs found it the most efficient
process, but because of the absence of or unfamiliarity with alternative
methods. In some cases decisions were made by people who had never
been to the country where the project originated. I remember meeting
years ago an Asian Desk Officer of a DFA. This particular Desk Officer for
116
The evalualur in social development evaluation
Asia had never set foot on Asian soil! It is very encouraging that this could
probably not now happen.
In the Philippines, the evaluation of proposals for social development
projects can be traced back to the early 1970s as a concrete response to the
expressed need of DFAs. Being thousands of kilometres away from the
project environment, DFAs began searching for ways by which they could
be properly advised about the urgency and relevance of a project that was
requesting assistance. The DFAs sought advice from local partners who
were considered knowledgeable about the situation of the intended
project beneficiaries.
One of my vivid memories in this work was when I evaluated a proposal
for a project in a far-flung rural area. Without leaving my desk, and without
going out of our air-conditioned office in Manila, I wrote a four-page
assessment of the proposal in a relatively short time. As a novice, I thought
this was a great accomplishment. 'What you have done is an armchair
evaluation,' my friend remarked. 'You are like the author/ she continued,
'who dared to write the history of Europe just because he flew over the
continent.' Fortunately, armchair evaluations are things of the past. Today
emphasis is given to a process that is more participative and people-
oriented. Now there is a clearer realisation that an evaluation is done not
because it is a requirement of the DFA, but because projects which can
contribute to social change deserve to be assisted and funded. From being
mainly a service to DFAs, evaluation has evolved into a means for training
people's organisations and other types of proponents in improving the
technical feasibility and social relevance of their projects.
117
Evaluating Social Development Projects
DFAs. on the other hand, might find it convenient to quote the local
evaluator and openly ascribe to him or her the observations on a project,
instead of making the comments as their own.
A third-party evaluation should be considered as merely advisory in
nature. The evaluator does not make the decision whether the proposal
should be funded or not. DFAs are the ones who make the final decision,
hence they should take the advice as their own if they believe in it, or
completely disregard the evaluator's opinion if they find it not to be factual
and not to coincide with their own assessment of the project.
In relation to the project holder, the evaluator takes on the role of a
friendly adviser. More often, proponents view the evaluator as an outsider,
an unwelcome and unavoidable intruder whose assessment can either
open or close the funding taps for the project. This is because evaluations
often contain recommendations to DFAs. Consequently the evaluator is
seen as someone who has the power to sway the decision of the DFA.
Given such a perception, the evaluator needs to show understanding for
the project holder's apprehensions.
It is important for the evaluator to get the message across to the
proponents that the existence of the evaluation exercise does not confer
on him or her the power of decision regarding the project. The proponent
has the power to decide the what and how of a project; the DFA decides
how much to fund. This power structure remains basically unchanged,
even with the entry of the evaluator into the picture. However, in the eyes
of the project holder, the evaluator has the power to have the proposal
funded or not. A responsible evaluator should consciously downplay what
proponents perceive as the evaluator's capacity to influence the results of
an evaluation study.
People directly involved in the implementation of a project necessarily
feel more conversant with the project, in the same way that local evaluators
believe they know better about the local situation than their counterparts
based in Europe or elsewhere. Since proponents are an evaluator's primary
source of data, it is important to maintain open lines of communication with
them. Especially problematic is the situation where close bilateral relations
between DFAs and proponents existed prior to the evaluator coming into
the picture. The evaluator's entry into the process can be perceived as a
negative intervention. When this is not properly handled, antagonism on
both sides can arise. The creation of communication barriers and obstacles
hinders the establishment of rapport between the project holder and
evaluator. In such instances, a continuing consultation with the project
holder can contribute to a mutually enriching evaluation exercise.
118
The evaluator in social development evaluation
In relation to the project itself and the community where the project
originates, the evaluator assesses the social relevance and technical
feasibility of a project. Crucial to this role is the evaluator's ability to
understand the project or programme within its broader context. Social
development evaluation sees the success of a project in terms of the social
change that it has effected. Furthermore one cannot look at a project in
isolation from people's lives and culture. Projects are people. Projects must
always be understood within the context of the needs and culture of their
intended beneficiaries.
For example, years ago the Philippines government decided to build
core toilets in an urban resettlement area. The proposal was assessed and
the project given the green light. What the evaluator failed to point out was
the absence of an education component regarding use of toilet bowls. The
result was that the people in the community used the toilet bowls as chairs
or flower vases. As one resident acidly pointed out, their main concern was
putting food into their stomachs; evacuation was less important.
Credibility to both DFA and proponents is vital if an evaluator is to be
effective. One factor in the evaluator's credibility is his or her knowledge
of both the technical feasibility and social desirability of the project.
Assuring a workable balance between the technical aspects and the social
dimension is a continual dilemma and challenge for the evaluator. It may
well be, for example, that the welfare of the target population has to be
given more weight than a project's technical feasibility. From a humanistic
point of view, evaluators may have to exercise more flexibility when
assessing the merits of a project. One may even sacrifice some minor
technical problems as a tra<i?-off for a long-term positive result which
could not be achieved if the r articular project were not implemented.
In evaluation it is also important to check the community's perception
of the problem against that of the project holder. There are instances when
the perceptions are not the same; in such a situation the community's
perception should prevail. For example, I once received a well-written
proposal for an awareness-raising and organisation project for upland
farmers. When I visited the targeted project area, the basic question which
the farmers raised was: considering the agro-ecological situation where
farms are rainfed and the topsoil is thin, how could they maximise their
little piece of land so that it could produce enough food for the table?
To summarise: an evaluator has several roles. To the DFA he or she
should be a source of information; to the project holder, a friendly adviser.
To both DFA and proponent the evaluator serves as a bridge. And to the
project and the community, he or she provides new ideas and alternatives.
119
Evaluating Social Development Projects
120
The evaluator in social development evaluation
target population is considered. If one does not have the sensitivity to read
between and behind the lines, then the information gathered may be
limited.
2 The gender of the interviewer.^ woman respondent may feel more at
home answering questions and giving information to a woman evaluator.
3 The nature of the topic being discussed. For example, family planning,
when taken in a Western context, may spark a lively discussion. In an Asian
setting this topic may meet a dead end even before it has got going. Why?
In the first place, sex is something not openly discussed in a society where
even married couples are very reserved in showing intimacy publicly. A
Westerner may find this difficult to comprehend, but an evaluator within
such a context has no choice but to recognise this limitation and find
creative ways of confronting the issue.
4 Regional cultural variations: culture varies not only from country to
country, but also from region to region. For example, promoters of organic
farming found no difficulty in convincing the Christian farmers of Luzon
(the northernmost island of the Philippines) to use animal manure as
fertiliser instead of chemical inputs. The same strategy would be resisted
by farmers in the southern island of Mindanao, whose religious beliefs
prevent them from having anything to do with things considered unclean'
such as animal manure. Someone insensitive to such cultural beliefs may
create setbacks in terms of the project's acceptability by its intended
beneficiaries.
5 Another Filipino cultural value which may have a strong bearing on
evaluations, whether of a project or a proposal, is utang na bob. or sense
of indebtedness. A favour given is something one has to recognise and
repay for a lifetime. An evaluator who was once indebted to a proponent
may find difficulty in assessing the project in an objective manner. To avert
problems in this relation, evaluators in the Philippines are not assigned to
a project of friends and acquaintances. Besides the embarrassment of
telling your friends the weakness of their project, there is the added burden
of positive expectations, which may be very high from the other party.
The evaluator
Equally important is the evaluator's consciousness of his or her own values,
and how they can be used positively in the evaluation of proposals and
projects. While it cannot be denied that the values of the evaluator can
influence the direction of a project, the evaluator, who has the power to
close down a project, should avoid any tendency to impose his or her
personal views. Where the values of the proponent and implementers do
not go against the evaluator's, then a smooth evaluation process is more
likely to follow. There may be cases, however, when one is faced with three
121
Evaluating Social Development Projects
different points of view and values: the proponent's, the DFA's, and the
evaluator's. When this happens, it will be important to consider the
experiences of other evaluators who have faced the same problem. It is in
such situations that dialogue, open communication and trust become
exceedingly important.
An evaluator's intervention can have both positive and negative impact.
It is up to us, practitioners of social development, to come up with a
method of evaluation which can strike a balance between a project's
technical aspect and its social merits, which would involve the subject
extensively and meaningfully, which will ensure that the project will build
on its strengths and minimise its weaknesses, and which can be instrumental
in accelerating social development and change. For many of us, evaluation
in the past was a 'hit and miss' affair. The challenge for practitioners is to
make evaluation both an art and a science. But in order to do this, an
evaluator should have the mind of a scientist and the heart of a
development activist.
122
5.2 The evaluator in social
development evaluation:
Review of workshop papers
and discussion
This workshop group addressed the important issue of the appropriate role
of the evaluator and his or her position within a re-negotiated evaluation
process. In the context of Delle Tiongson-Brouwers' paper, entitled The
Evaluator in Social Development Evaluation', a number of other papers
were presented, which are listed at the end of this chapter. This section
summarises the most important issues raised by the group, and elaborates
on some issues surrounding the role of the evaluator in social development
projects and programmes.
123
Evaluating Social Development Projects
associated with 'expert' intervention, they can be the most effective way
of gaining appropriate and useful information. This did, however, raise
fundamental questions about who was best for the job. The dilemma was
not primarily between the amount of time required for the evaluation, but
more about the fundamental rationale for evaluation, which should be seen
as an educative process.
Evaluators as judges
Not only does the evaluator have to attempt to be as objective as possible,
but other problems of place and position complicate the process. As an
outsider he or she may be perceived as an intruder. Indeed the very essence
of traditional evaluations was seen as involving interference by those who
knew little of the environment in the internal affairs of those working at
the grass roots. Somehow the evaluator has to negotiate a legitimacy which
gives credence to his or her work and on which trust can be built in order
for effective assessments to be made.
The evaluator can be successful only if all members of the project
community participate fully in the process, and a basis of trust is built up
124
The evaluator in social development evaluation
125
Evaluating Social Development Projects
Participatory evaluation
126
The evaluator in social development evaluation
127
Evaluating Social Development Projects
128
The evalualor in social development evaluation
A third stage was reached when it was recognised that the organisation
itself did not have the specialist knowledge to conduct many evaluations,
and that in-house studies might not be the most appropriate. External
consultants, employed for their objectivity, on contracts, and chosen
through 'competitive tendering' were brought in. The evaluation industry
burgeoned, producing a specialist, professional group of 'experts' anxious
to enshrine a particular type of expertise within professional barriers.
A fourth stage, of incorporating beneficiary evaluations, of putting the
people centre stage, is only now being worked out as the boundaries of
social development evaluations are explored. It is the elaboration of
evaluations at this stage which involves the complete renegotiation of what
evaluations can and should be all about.
129
Evaluating Social Development Projects
as people who can see things in different ways, allowing others to step out
of the blinkered world of everyday, taken-for-granted existence. The
biases inherent in most traditional understandings of the project environment,
and so vividly emphasised in Robert Chambers' work, need to be
understood and made explicit if there is to be any chance of success for
participatory evaluations.
130
The evaluator in social development evaluation
131
Social Development Projects
process and inhibits the formal use of techniques which alienate evaluator
from evaluand. To be successful, then, evaluation must involve all
elements of the project community, and indeed the process of evaluation
becomes enmeshed with the process of objective clarification as a
mutuality of interest is explored and partnerships developed.
Alienation from development interventions can be circumvented, as
workers' alienation from management can, by mutual appraisal which
increases a sense of common purpose and collective ownership. Only
when people feel themselves to be part of the intervention is sustainability
likely to be forthcoming. The problems with the development of appraisal
systems are how to design systems that are flexible enough to tolerate many
differences of opinion and yet robust enough to remain useful.
132
The evaluator in social deuelopmoil evaluation
133
Eraluating Social Development Projects
134
The evaluator in social development evaluation
135
SECTION 6
CONCLUSIONS
137
6.1 Future issues and
perspectives
Introduction
Evaluation is important for a variety of reasons. These reasons necessarily
reflect the different world views of those approaching the issue, and the
changing climate of opinion within the field of development, but are
essentially associated with the desire to improve practice and the perceived
centrality of processes of evaluation in this process of improvement.
Evaluation provides a key word (in the sense that Raymond Williams and
Ivan Illich talk of key words) around which discussions about more
appropriate practice might revolve, and from which a community of
interest with a common purpose might evolve. It is important to bear this
in mind, given the different motives attributed to the 'negative academic'
and the 'positive practitioner', to the distinctions between hard and soft
techniques, and to the controversy that still surrounds the notion of
objectivity in any discussions about evaluation. All these dichotomies are
current in the literature and a hindrance to clear thinking.
In order to make progress in the construction of this more appropriate
practice, it is necessary to examine critically the different world views that
inform current thinking; to clear away some of the undergrowth that
inhibits the growth of new ideas. Rather than taking for granted the often
implicit views of others, incorporated into what are seen as authoritative
texts, it is important to strip them .of their pretensions and point out their
partialities as the first step in the construction of purposive actions.
On this basis one might identify two broad approaches to evaluation.
Firstly, there is the specific instrumental approach: the desire to know how
effective interventions in the name of social development have actually
been. This approach might be termed instrumental/technocratic. It usually
attempts to extend current quantitatively based practices into the new.
uncharted seas of social development. Secondly, there is that approach
which attempts to confront the dominant instrumental paradigm through
an exploration of why evaluations are currently being afforded considerable
centrality; to examine the latent functions that evaluations serve, in
reinforcing certain sorts of control. This emphasises the political processes
involved in evaluations and is not normally characterised by an overt
concern with the elaboration of more appropriate practice. For want of a
better word, this approach might be term interpretative. An interpretative
138
Conclusions
approach provides the basis for what one might call a 'practical' evaluation
methodology, something which underpinned much of the thinking of the
conference, and something which holds the issues of participation,
capacity-building, sustainability, and empowerment as central to the
elaboration of more appropriate development strategies.
It is recognised that evaluations are crucial moments in institutional life
and are thus deeply political. As crucial moments they offer the opportunities
for the reappraisal of practice and the re-formation of objectives, not just
at the micro level, but also at the macro level of thinking about the nature
of development tasks.
These different approaches are based on distinct ideological premises,
or world views. They bring different and separate questions to, the analysis
of evaluation, and thus engender distinct problematics and distinct
programmes for future action. It is important to know where we are coming
from if we are to chart successfully our course for the future.
139
Evaluating Social Development Projects
140
Conclusions
141
Evaluating Social Development Projects
What we may be witnessing is not just the end of the cold war, or the
passing of a particular period of post-war history, but the end of
history as such: that is the end point of mankind's ideological
evolution and the universalization of western liberal democracy as the
final form of human government.
142
Conclusions
143
Evaluating Social Development Projects
144
Conclusions
145
Evaluating Social Development Projects
1 The project: At the level of the project, the major concern is to unravel the
politics of distribution of resources, by identifying those individuals who are
able to capture and divert benefits which are targeted on particular groups —
for example poor women, or landless labourers. It also aims to expose the ways
in which resources accrue differentially to different groups, through an
historical and political appreciation of how relationships of inequality become
established and legitimated. A good example of this sort of evaluation is
afforded by the investigations of power networks in rural Bangladesh
undertaken by the Bangladesh Rural Advancement Committee (BRAC).
2 Project management: At the level of project management, the main aim
is to expose the ways in which resource allocation processes are manipulated
by 'non-rational considerations', related to sectional interests and values,
and motivated by what have been termed the 'politics of affection' (Hyden).
It is assumed that rationality is dominated by considerations of short-term
political and financial advantage, and individual gains associated with status
improvement. A major aim of this sort of evaluation is to expose corruption,
paternalism, and nepotism. By stripping off the cultural layers that hide 'bad
practice', the stage is set for the elaboration of more honest, just, and
egalitarian strategies. A good example of this sort of evaluation study is
provided by Wade's analysis of corruption in South India (Wade 1984).
This sort of evaluation of project management can be extended to an
examination of internal dynamics, by exposing sectional and special
146
Conclusions
147
Evaluating Social Development Projects
sorts of media opportunities are there? What types of message are likely to
receive more favourable attention? Who might be identified as spokespeople?
What sorts of coalitions need to be built in order for intervention strategies
to be more effective? What sorts of competitors are likely to be pursuing
the same sorts of objectives? What sorts of opposition is the project likely
to encounter? An evaluation of the external environment before the
implementation of projects (what in business might be termed 'market
research') might facilitate the speedy implementation of the project and
enhance its perceived legitimacy. This evaluation should not be confined
just to pre-project stages, but should be a feature of all stages. This is no
less a part of the evaluation process than attempts to measure whether
resources committed have been used effectively and efficiently, but it has
usually been associated with some form of monitoring.
148
Conclusions
149
Eralualing Social Development Projects
150
Conclusions
151
Evaluating Social Development Projects
Conclusion
The way forward, as underlined by many of the contributions to the
conference, lies in the construction of a practical evaluation that embodies
many of the central concerns of an interpretative enquiry. But one of the
very serious assumptions under which interpretative evaluations labour is
that of the cultural relativity which their emphasis on subjectivity forces
them into. While the sanctity of different cultural interpretations is
maintained, no mechanism exists to construct or impose a meta-interpretation
based on an over-arching morality. The dangers of this are manifold.
Firstly, those who have less compulsion to respect cultural differences can
proceed unchecked in the advancement of their particular projects, based
on dogmatic beliefs in their own mission (free-market capitalists, for
example, or fundamentalist Muslims). Secondly, a realisation of the
partiality of Western knowledge, and the associated guilt occasioned by
past imperialist ventures, immobilise many well-intentioned liberals, and
disable many very worthwhile development interventions. And, thirdly, as
a result of this paralysis, accusations of paternalism, elitism, and racism
remain unchallenged. The Western liberal establishment gives the stick to
others, who are allowed to brandish it and beat the indecisive cultural
relativist: the tail begins to wag the dog.
Also associated with this assumption of cultural relativism and the central
importance given to subjectivity in interpretative evaluations is the
confusion that surrounds the notion of 'belonging'. This gets bound up
with the issue of objectivity, as well as with the issue of control. To 'belong'
152
Conclusions
153
Evaluating Social Development Projects
154
Conclusions
units, one is also disassembling the structures which provided security for
their members. The rules and regulations which governed such organisations
are a two-edged sword. They may have discouraged initiative, but they also
protected employees against the arbitrary dictates of management. What
is the situation inside non-governmental organisations? An interpretative
evaluation of the social development programme which here is the NGO,
with a focus on these issues, remains to be written. But it might be argued
that a significant feature of employment in many such organisations is the
large turn-over of staff, en route to better opportunities, and the external
resources that many individuals possess which cushion them against
relatively poor remuneration. It makes it very important, therefore, to
situate these non-governmental organisations within a much more general
socio-political environment; they are not so separate as is often imagined.
155
Evaluating Social Development Projects
156
Bibliography
157
Evaluating Social Development Projects
158
Bibliography
United Nations (1984), Guiding Principles for the Design and Use ofM &
E in Rural Development Programmes, Rome: UN.
Wade, R. (1982), The system of administrative and political corruption:
canal irrigation in South India', Journal of Development Studies, 18/3-
Weiss, C. (1972), Evaluation Research, New Jersey: Prentice Hall.
Wildavsky, A. (1979), The Art and Craft of Policy Analysis, Basingstoke:
Macmillan.
Wolfe, M. (1982), Illusive Development, Geneva: UNRISD.
159
List of conference participants
160
List of conference participants
161
Evaluating Social Development Projects
162