Participant-Oriented Evaluation

Prepared by: ENGR. ROSIE JANE P. SIOSAN PhD Curriculum Development

According to Elspeth Huxley

The best thing to find things out is not to ask questions at all. If you fire off a question, it is like firing off a gun-bang it goes, and everything takes flight and runs for shelter. But if you sit quite still and pretend not to be looking, all the little facts will come and peck around your feet, situations will venture forth from thickets, and intentions will creep out and sun themselves on a stone; and if you are very patient, you will see and understand a great deal more than a man with a gun does.

Participant-Oriented Evaluation

Evaluation approach focusing on the participants Participants have interest in the results Developed in response to the needs and interests of those associated with the evaluation

PARTICIPANT ORIENTED EVALUATION

Uses instruments as its methodology through observation, interview, survey, test and experiments. According to Fitzpatrick, in this evaluation, The evaluator is in the role of the learner and the stakeholders serve as teachers.

Characteristics of Participant Oriented Evaluation

Depend in inductive reasoning [observe, discover, understand] Use multiple data sources [subjective, objective, quantitative, qualitative] Do not follow a standard plan [process evolves as participants gain experience in the activity] Record multiple rather than single realities [e.g., focus groups]

Participant-Oriented Evaluation Approaches

Stakes Countenance Framework Transactional Evaluation Illuminative Evaluation Democratic Evaluation Responsive Evaluation Naturalistic Evaluation

STAKES COUNTENANCE MODEL

Introduced by ROBERT STAKE, first evaluation theorist to introduce participant oriented evaluation in the field of education.

STAKES COUNTENANCE MODEL

This model attempts to describe the thing being evaluated and render judgment about the things value or worth. It has two basic acts of evaluation the description and judgment.

STAKES COUNTENANCE MODEL

Evaluation Framework

STAKES COUNTENANCE MODEL

Evaluation Framework

STAKES COUNTENANCE MODEL

Evaluation Framework

STAKES COUNTENANCE MODEL

Using the framework, the evaluator, Provide background, justifications, and description of program rationale List intended antecedents, transactions, and outcomes Record observed antecedents, transactions, and outcomes Explicitly state standards for judging program antecedents, transactions, and outcomes Record judgments made about antecedent conditions, transactions, and outcomes

TRANSACTIONAL EVALUATION

TRANSACTIONAL EVALUATION

According to Rippey, transactional evaluation gives emphases on the process of program improvement, for example by encouraging anonymous feedback from those that a change would affect and then a group process to resolve the differences.

TRANSACTIONAL EVALUATION

It involves not only the protagonists and the designers of the innovation, but also a representative sample of people likely to be affected by the consequence of change.

TRANSACTIONAL EVALUATION
Contributions 1. Formative evaluation is improved through the involvement of a wider range of opinions and values in the evaluation design. 2. Increased organizational efficiency and greater program benefits result because of attentiveness to potential role threats.

TRANSACTIONAL EVALUATION
Contributions 3. The concern of the evaluator for human values as well as program outcomes places him in a better relationship with personnel involved in the change, bringing greater honesty of interchange and thus more valid data

TRANSACTIONAL EVALUATION
Contributions 4. Involvement of a wider range of interested personnel in evaluation leaves a residue of organizational and evaluative skills that are potential for the organization, persisting beyond either termination or the solidification of the original change

TRANSACTIONAL EVALUATION
Steps followed in a complete and comprehensive transactional evaluation

Initial Instrumentation Program development Program monitoring Recycling

ILLUMINATIVE EVALUATION

ILLUMINATIVE EVALUATION

The basic idea is for the investigator to hang out with the participants (students, teachers, etc.) to pick up how they think and feel about the situation, and what the important underlying issues are. Ethnography

This process creates a dynamic community of learners as people engage in the art and science of evaluating themselves.

ILLUMINATIVE EVALUATION
Approach that followed a social anthropology paradigm

Observation - to explore and become familiar with the day-to-day reality of the setting being studied Further inquiry - to focus study by inquiring further on selected issues Explanation - to seek to explain observed patterns and cause-effect relationships

ILLUMINATIVE EVALUATION
Approach that followed a social anthropology paradigm

Holistic - evaluators attending closely to the various contexts of a program being evaluated and seeking to portray the program as a working whole, as an individual organizational construction that needs to be examined simultaneously from different perspectives.

ILLUMINATIVE EVALUATION
Approach that followed a social anthropology paradigm

Responsive - with researchers working closely to provide all concerned with a program with a genuinely helpful report, that might take many different forms and draw on many diverse sources and methods, but is designed to interest, to inform, to add to their understanding.

Implementation of ParticipantOriented Evaluation

When to use When cost is not an issue When time allows When responding to stakeholders When evaluator can take on the role of the learner When not to use When detailed quantitative data is essential When there is controlled, laboratory or contrived setting (Jacobs, 1985).

Strengths of Naturalistic and Participant Oriented Approach

Emphasizes human element Gain new insights and theories Flexibility, attention to contextual variables Encourages multiple data collection methods Provides rich, persuasive information Establishes dialogue with and empowers quiet, powerless stakeholders

Weaknesses of Naturalistic and Participant Oriented Approach

Subjective Expensive Labor intensive Potential for evaluators to lose objectivity Can Take some time

OTHER INTRIGUING PARTICIPANT APPROACHES

Development evaluation evaluation process and activities that support program or organizational development. The evaluator is part of a team whose members collaborate to conceptualize, design, and test new approaches in a long term, on-going process of continuous improvement, adaptation, and intentional change. The evaluators primary function in a team is to elucidate team discussions with evaluative data and logic, and to facilitate data-based decision-making in the developmental process. (Patton, 1994)

1.

OTHER INTRIGUING PARTICIPANT APPROACHES

Stakeholder-based evaluation an evaluation in which selected stakeholders representing all legitimate groups were consulted at the planning and data interpretation phases of the evaluation.(Bryk and Mark and Shotland,1985b) Emancipatory evaluation proposed to free those persons who are the most marginalized, or oppressed, or with the least power, and empower them to control their own destiny by the use of the results of the study.( Mertens 1999)

1.

2.

OTHER INTRIGUING PARTICIPANT APPROACHES

4.Participatory

evaluation cousins and earl (1995) defined participatory evaluation as evaluation that involves trained evaluation personnel and practice based decision makers working in partnership.
4.

5.

Includes training key organizational personnel in the technical skills of evaluation while they are working in partnership with evaluators. Seeks to use evaluation data for practical problem solving, not theory development or empowerment individuals or groups, or rectification of social inequities.

OTHER INTRIGUING PARTICIPANT APPROACHES

Utilization-focused evaluation begins with the premise that evaluations should be judged by their utility and actual use. Also evaluators are seen to have a responsibility for training users in evaluation processes and the uses of information. Empowerment evaluation vaulted onto evaluations center stage by Fettermans (1994) presidential address delivered to the American Evaluation Association. In it, he urges evaluators to help participants to take evaluation into their own hands and conduct self evaluations,,,seeking assistance of an evaluator to act as a coach.

4.

5.

REFERENCES

Program evaluation third edition alternative approaches and practical guidelines by Jody L. Fitzpatrick et. Al http://www.okstate.edu/ag/agedcm4h/academic/aged6220/6 220class/6220class/naturali.htm Education Research Information center, http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp ?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED060071& ERICExtSearch_SearchType_0=no&accno=ED060071 Observing, measuring, or evaluating courseware: A conceptual introduction Stephen Draper, http://www.icbl.hw.ac.uk/ltdi/implementingit/measure.htm

Other online resources

Finish..