Key concepts in ELT

Evaluation
Although evaluation has been increasingly used in a variety of public spheres, including education, for about 50 years, attention to evaluation and its uses in ELT is relatively recent. Essentially, evaluation is carried out to determine the extent to which a programme or intervention is worthwhile, and to aid decision-making through the purposeful gathering of information which is analysed and reported to stakeholders - interested parties who have a stake in the activity evaluated. The evaluation may be carried out by professional evaluators or by a group made up from the stakeholders; its scale may be nationwide or within an institution; and it may take a few days or several months to complete. Evaluators use the methods of social science research, and the discipline is sometimes referred to as evaluation research. It is usual to distinguish different functions of evaluation (Patton 1987, Weir and Roberts 1994, Fink 1995). An evaluation can be carried out to see if a programme has met its targets (summative evaluation), checking, for example, whether or not a predetermined proportion of students have achieved a specified level of language proficiency. The focus of such evaluations is usually accountability. The evaluation may investigate how far a programme is on track to achieve its targets (formative evaluation), looking, say, at how teachers implement training in methodology within a new curriculum; also, within a formative approach, and starting out with no pre-set criteria, the evaluators can, for example, study teachers attitudes and practice in order to understand how a programme works. In cases such as these the focus is usually on development. The complementary and overlapping nature of the focuses is increasingly recognized, and developmental evaluations can contribute valuable information to summative reviews (Mackay 1994). The results of the evaluation may indicate that the programme is working satisfactorily, but in general there is an expectation that they will reveal areas for improvement. However, this does not guarantee that the stakeholders will act on the findings, or that the results will be available to them in time. As this remark implies, evaluations often give rise to practical difficulties, which are further complicated by the political use that may be made of them, especially where those who 210 commission the evaluations (usually the funding body) either want them to show results favourable to their view of the programme, or will use the results for their own ends rather than those of other stakeholder groups. Two areas in particular have aroused controversy within evaluation. In the past, questions about approach have focused on the use of quantitative versus qualitative measures associated with experimental or naturalistic designs. However, the use of both is now more widely accepted, and the use of mixed methods is also advocated (e.g. Lynch 1996). Current debate on approach is concerned with how far evaluation deals with an objective world, or whether it can only ever deal with a socially constructed reality (positivism versus constructivism - see Lynch 1996). Controversy over the conduct of evaluation has to do with who carries it out, and how findings are used. Early proponents of the use of evaluation in education suggested that teachers should learn to evaluate, but in many contexts the conduct and results of evaluations have been kept in the hands of administrators and inspectors, creating an air of mystique and exclusion which has led people to regard evaluation with suspicion. More recent advocates of teachers use of evaluation have tended to focus on evaluation for development, and some practitioners in ELT have adopted this approach (e.g. Rea-Dickins and Lwaitama 1995). This move has been prompted partly by a greater consciousness of change and of the need for change, as well as by researchers and educators turning their attention to what happens in the classroom, and to teacher development. Dissatisfaction with the management of programmes and the implementation of new curricula has also brought increased attention to the role that insider evaluation may play in their development. Ministries and officials, as well as practitioners, now encourage the wider use of evaluation by insiders such as teachers. However, the mechanical implementation of evaluation instruments by untrained users is not fruitful. Even though teachers may not be trained in all aspects of evaluation, and may only have limited skills, the crucial element is that they should be able to control what they do in evaluation (Murphy 1996). Evaluations need to be designed for the context in which they are used, sensitive both to local

ELT Journal Volume 54/2 April 2000 Oxford University Press 2000

articles

welcome

conditions and to the audience that will use them. In essence, evaluation is either a practical, goaloriented activity or it is not worthwhile - a point which applies as much to large-scale evaluations as to those conducted by teachers working on their own or in a small group. The increasing body of case studies of teachers using evaluation shows that valuable results can be achieved for improving and developing curricula and professional skills where limited resources are intelligently used (e.g. Rea-Dickins and Lwaitama 1995). Dermot F. Murphy Fink, A. 1995. Evaluation for Education and Psychology. Thousand Oaks: Sage Publications. Lynch, B. K. 1996. Language Program Evaluation. Cambridge: Cambridge University Press. Mackay, R. 1994. Undertaking ESL/EFL programme review for accountability and improvement. ELT Journal 48/2: 142-9.
References

Murphy, D. F. 1996. The evaluators apprentices. Learning to do evaluation. Evaluation 2/3: 321-38. Patton, M. Q. 1987. How to Use Qualitative Methods in Evaluation. Newbury Park: Sage Publications. Rea-Dickins, P., and A. F. Lwaitama (eds.). 1995. Evaluation for Development in English Language Teaching. Review of English Language Teaching 3/3 1993. London: Modern English Publications in association with the British Council. Weir, C., and J. Roberts. 1994. Evaluation in ELT. Oxford: Blackwell. Dermot Murphy teaches in the School of Education at Kings College London. He has experience of conducting evaluations, and training teachers to evaluate in Africa, Latin America, South-East Asia, and Europe. Email: <dermot.murphy@tvu.ac.uk>
The reviewer

Key concepts in ELT

211

articles

welcome