Advances in Health Sciences Education (2007) 12:169186 DOI 10.

1007/s10459-005-5046-6

Springer 2006

Peer Assessment in Problem-Based Learning: A Qualitative Study

TRACEY PAPINCZAK*, LOUISE YOUNG and MICHELE GROVES
Mayne Medical School, School of Medicine, University of Queensland, Herston, 4006, Brisbane, Queensland, Australia (*author for correspondence, E-mail: traceypapinczak@optusnet.com.au)
Received 26 June 2005; accepted 10 November 2005

Abstract. Peer assessment provides a powerful avenue for students to receive feedback on their learning. Although student perceptions of peer assessment have been studied extensively in higher education, little qualitative research has been undertaken with medical students in problem-based learning (PBL) curricula. A qualitative study of students attitudes to, and perceptions of, peer assessment was undertaken within the framework of a larger study of metacognition with rst-year medical students at the University of Queensland. A highly structured format for provision of feedback was utilised in the study design. Many recommendations from the higher education literature on optimal implementation of peer-assessment procedures were put into practice. Results indicated the existence of six main themes: (1) increased responsibility for others, (2) improved learning, (3) lack of relevancy, (4) challenges, (5) discomfort, and (6) eects on the PBL process. Five of these themes have previously been described in the literature. However, the nal theme represents a unique, although not unexpected, nding. Students expressed serious concerns about the negative impact of peer assessment on the cooperative, nonjudgmental atmosphere of PBL tutorial groups. The practical implications of these ndings are considered. Key words: assessment, peer assessment, peer evaluation, problem-based learning, qualitative study

Introduction In recent years, the quality of medical education has attracted considerable attention, especially in light of the requirement to make education more relevant to the needs of modern society. Medical education must prepare students to deal with problems in the future, equipping them with skills necessary to become active, self-directed learners, rather than passive recipients of information (Dolmans and Schmidt, 1996). Recognition of this need was responsible, in part, for the development of problem-based learning (PBL) (Barrows and Tamblyn, 1980). Within the eld of medical education, PBL is a curriculum innovation that involves students in authentic learning activities using ill-dened medical

170

TRACEY PAPINCZAK ET AL.

problems as the stimulus and focus for learning (Norman and Schmidt, 1992). The pedagogical appeal of PBL is its perceived capacity to foster, through these learning processes, enhanced clinical reasoning skills, and the development of both an adaptable knowledge base to use in professional settings and the skills in self-directed learning necessary to become lifelong learners in that profession (Kelson and Distlehorst, 2000). Four critical conditions for a deep approach to learning are encompassed within the PBL approach: a well-structured knowledge base, active learning, collaborative learner interaction, and a context designed to promote internal motivation through the provision of pragmatic goals (Margetson, 1994). Evaluation of student progress in such a student-centred curriculum, however, has remained a challenge (Eva, 2001). Assessment protocols within PBL curricula have sometimes sought to include participative (or peer) assessment, in which students contribute to the evaluation of each others work, in order to match evaluation procedures with the curricular philosophy. Peer assessment helps to develop the acquisition of self-directed learning skills (a key objective of PBL) as students participate in the assessment experience (Ballantyne et al., 2002). As PBL emphasizes the development of prociency in the resolution of clinical problems, the assessment of student skills, processes and attitudes would take place most appropriately within the tutorial setting (Eva, 2001). There are several advantages to employing tutorial-based peer assessment, including: prolonged interaction between peers for provision of constructive feedback based on multiple observations of performance; and opportunity to assess areas of prociency (such as communication skills, self-directed learning, and respect for others) not readily evaluated by more traditional forms of assessment (Eva, 2001). Higher education literature suggests that peer assessment has many additional benets, including enhanced metacognitive skills (Ballantyne et al., 2002), and improved understanding of subject matter (McDowell, 1995). Peer assessment encourages students to reect on their own approaches to assessment tasks (Dochy et al., 1999), to develop critical reasoning skills (Hanrahan and Isaacs, 2001; Searby and Ewers, 1997) and skills of reection (Searby and Ewers, 1997; Somervell, 1993). Although strong support for peer assessment is evident in the literature, diculties and limitations have repeatedly been reported. Students perceptions that peer assessment can be unreliable and unfair has been raised in several studies in higher education (McDowell, 1995; Norton and Brunas-Wagsta, 2000). Sluijmans et al. (2001) conrmed the existence of bias in peer marking due to interpersonal relationships between students. Students frequently report feeling uncomfortable carrying out peer assessment, often because they feel unqualied to make these judgements (Orsmond and Merry, 1996), lack faith in their own objectivity (Hanrahan

PEER ASSESSMENT IN PROBLEM-BASED LEARNING

171

and Isaacs, 2001), or feel the task is too challenging (Mowl and Pain, 1995; Topping et al., 2000). The time-consuming nature of peer assessment is also regarded negatively by students (McDowell, 1995; Topping et al., 2000). While studies in medical education of students attitudes to peer assessment in PBL have, to date, proven elusive, the results of quantitative studies suggest that peer ratings are not a reliable means of tutorial-based assessment in PBL (Reiter et al., 2002; Sullivan et al., 1999). Qualitative studies of peer assessment within PBL have the capacity to provide insights into the nature and extent of benets to learning. Research into medical education has largely emphasized quantitative rather than qualitative evaluation, particularly in the area of peer assessment. Van Rosendaal and Jennett (1992) reported that peer evaluation was viewed as an intrusion into the relationship among medical colleagues. Dueld and Spencer (2002) surveyed medical students attitudes to assessment procedures and found meaningful feedback about their progress in the course was considered very important. Peer assessment provides one avenue for such feedback to be delivered within the setting of the PBL tutorial. Given the lack of research into student attitudes to peer assessment within the PBL tutorial setting, this study has the potential to provide important insights and to expand upon the ndings from studies in non-PBL curricula reported above. There have also been calls for greater qualitative exploration of the social and educational aspects of PBL, in particular, ethnographic accounts of PBL which endeavour to explore the inuence of the culture of the group (Leung, 2002). Such research has the potential to enhance our awareness of all aspects of the PBL process. The aim of this study was to explore student attitudes to, and perceptions of, peer assessment concerning student fullment of roles and responsibilities within their PBL tutorials. The PBL environment, with its emphasis on selfdirected and collaborative learning, provides a unique context in which to study students attitudes to, and perceptions of, their self-directed learning experiences. Qualitative methods allow the researcher to study these dynamic processes. Description of the Study Data were gathered as part of a larger study of metacognitive processes undertaken with rst-year students enrolled in the Bachelor of Medicine and Bachelor of Surgery (MBBS) Program at The University of Queensland. Only the qualitative results of the peer assessment portion of the study will be reported here. Methods associated with participatory action research (Kemmis and Wilkinson, 1998) and ethnography (Kamil et al., 1985) were adopted to gain insight into student attitudes to, and perceptions of, peer assessment. Collins

172

TRACEY PAPINCZAK ET AL.

(1991) recommends the following epistemological grounding for engaging in research among adult learners: Instead of deploying intrusive research designs based on independent learning project protocols, (we) might invite willing individuals to engage in exploratory conversations, as part of a pedagogical and hermeneutic process, to discover with what meanings adults endow their own learning experiences. (Collins, 1991, p. 114) Ethnographic inquiry endeavours to understand social behaviour from the participants frame of reference (Kamil et al., 1985). In this context, the experiences, behaviours, and attitudes of students learning medicine in their PBL tutorials are observed and documented. The role of this researcher (a PBL tutor) was based on participantobserver inquiry (Bishop, 1999). PBL tutors are in a fortunate position to be able to observe the culture of the PBL tutorial and to participate as much or as little as desired in the ensuing discussion and debate. The context and the participants were used to guide research questions (Bishop, 1999) the classroom realities, developing knowledge, and developing perceptions of the group allowed the researcher to adapt the study design. The resulting design aimed to more fully reect the reality of the experience of students in their PBL tutorials. Action research, involving overlapping cycles of planning, acting and observing, and reecting (Kemmis and Wilkinson, 1998), was utilised to maintain a responsive study design. Data collection and analysis were combined and used to shape ongoing data gathering in order to develop a participative-assessment procedure responsive to student priorities and concerns. The study design drew on literature in the area of peer assessment and action research, incorporating numerous elements deemed important including: 1. assessment of process rather than learning outcomes (see Boud et al., 1999); 2. use of assessment criteria targeting ve signicant areas of performance in PBL (refer Das et al., 1998; Willis et al., 2002), including assessment of participation (Dancer and Kamvounias, 2005); 3. use of multiple specic criteria on the assessment instrument, targeting discrete areas of student performance (as endorsed by Miller, 2003) rather than a few global indicators; 4. student feedback helping to guide nal criteria selection, and explicit details of implementation of the peer-assessment process (refer Ballantyne et al., 2002); 5. practice in peer assessment procedures to improve student familiarity (in keeping with the recommendations of Ballantyne (2002), Sluijmans et al. (2001), and Sullivan et al. (1999);

PEER ASSESSMENT IN PROBLEM-BASED LEARNING

173

6. provision of examplars of good and/or poor work to students in initial weeks of orientation (in accordance with the recommendations of Ballantyne et al., 2002; Hanrahan and Isaacs, 2001); 7. constant monitoring of student perceptions in order to guide the action research process (Kemmis and Wilkinson, 1998). Setting The MBBS Program introduced a 4-year, graduate-entry PBL curriculum in 1997. First-year students, in small groups of 9 or 10, undertake 5 h of PBL tutorial time each week for 33 weeks of the year. Working in collaboration with their small group of peers, students analyse a problem of practice, formulate hypotheses, and undertake self-directed learning to try to explain the patients disease process. Subjects The study was conducted with 165 rst-year medical students and 20 tutors in two phases over 2 years (refer Table I). In the rst year, small student focus groups were invited to participate in renement of the study design and peer-assessment instrument. In the following year (2004), 215 self-selected subjects, representing 69% of the student group, were recruited for the main study through an information session in the rst week of the academic year. Statistical analysis showed that study subjects were representative of the entire student cohort on measures of age, gender, and primary degree. As part of a larger study of metacognition, students were allocated to either control or intervention groups. Allocation to the intervention cohort, based on the criterion of total participation of group members within each PBL
Table I. Involvement of two cohorts of rst-year medical students in the two phases of the research Cohort Timing of qualitative research Phase 1: April & June Phase 2: July to September Number of student participants 20 rst-year medical students 10 10 rst-year medical students 10 rst-year medical students 125 rst-year medical students Research activities

Cohort 1 (2003)

Cohort 2 (2004)

Phase 3: February to September

Focus group discussions to identify areas of need Negotiation of criteria for peer assessment Trial of peer assessment instrument and procedure and re-negotiation Implementation of peer assessment procedure Student feedback

174

TRACEY PAPINCZAK ET AL.

tutorial, resulted in 13 intervention (125 students) and 16 control tutorial groups (90 students). All tutors assigned to intervention tutorial groups agreed to participate. Intervention students took part in educational activities within their PBL tutorials, including peer assessment, for a period of ve months (from which both qualitative and quantitative data were generated).

PROCEDURE Ethical approval was obtained from the University of Queenslands Behavioural and Social Sciences Ethical Review Committee. Initial exploration of peer-assessment and development of the instrument Several focus groups from the rst cohort of students (2003) were engaged in an initial exploration of peer assessment options, some negotiation of the criteria to be included in the assessment instrument (based on the relevant literature in this eld), and a short trial of the peer assessment procedure. Qualitative data were collected from direct observation and focus group interviews. Initially, two dierent groups of 10 students were consulted within the PBL tutorial setting in order to test the water regarding student acceptability of participative assessment. An unstructured interview arrangement, with the explicit use of group interaction to produce greater insight (Punch, 1998), was adopted. Remarks such as: Marks allocated to PBL would recognize the amount of work done during this time and help motivate people to work each week indicated student support for the overall concept. Subsequently, another group of ten students were invited to have input into selection of appropriate marking criteria for participative assessment. Numerous authors support the involvement of students in discussion and negotiation of criteria to be used in assessment as it encourages them to clarify their objectives and think critically about their work. Students were distributed with a list of criteria proposed by relevant literature such as criteria for self-evaluation in medical education described by Das et al. (1998), and student perceptions of motivational inuences and assessment within PBL (Willis et al., 2002). With discussion and negotiation, students agreed that all characteristics listed were valid and important in describing students who are good participants, with some reservations concerning selfawareness items. The negotiated Peer Assessment Instrument (with 19 items) was then trialled with another group of 10 medical students and rated as easy to use and understand by all participants. Student dissatisfaction with two items resulted in their removal from the nal version of the instrument. The resulting scale measures several features of successful adult education, such

PEER ASSESSMENT IN PROBLEM-BASED LEARNING

175

as participation, punctuality, respect for others, eective communication, and critical analysis. However, the inclusion of items specically targeting self-directed learning and self-awareness, core features of PBL, allows it to be dierentiated from others which may be appropriate for open-ended, but less student-centred, approaches - such as case-based instruction (Hay and Katsikitis, 2001). The instrument consists of 17 items scored on a Likert scale of one to ve across ve sub-scores: responsibility and respect, information processing, communication, critical analysis, and self-awareness. These criteria were designed to address some facets of the important aspects of learning in PBL, including self-directed learning, eective skills in collaboration and group-work, higher level thinking skills, and the ability to identify ones weaknesses and strengths. In order to gain some measure of face validity, three experienced PBL facilitators were asked to indicate whether each of the 17 items on the instrument was relevant to PBL performance and able to be adequately assessed using the item in question. Unanimous face validity was obtained for all items in the four sub-scores: responsibility and respect, information processing, communication, and critical analysis. Some dissent about the validity of the self-awareness sub-score was evident. Construct validity describes the degree to which the items used in the instrument dene the constructs (Pressley and McCormick, 1995). The ve constructs or domains of performance were reported extensively in the medical and nursing education literature. Each of the three PBL tutors and ten PBL students were asked to categorize the 17 items into the ve specied domains. In all cases, the items were distributed in accordance with the domains as dened on the instrument. Values for Cronbachs alpha ranged from 0.76 to 0.84, indicating good internal consistency among the ve sub-scores. Acceptable reliability was found, with Pearson correlation coecients for peer- and tutor-assessment ranging from 0.40 to 0.60. Notably, self-awareness items were problematic with a signicant number of students consistently entering not applicable for those two items. Unfortunately, time constraints prevented further renegotiation of the Peer Assessment Instrument or the details of the peer-assessment procedure with the second cohort prior to the commencement of the main study. Main study The main study was undertaken with the following cohort of medical students, using the instrument and the rened design of the previous cohort. Written student feedback was collected during, and at the conclusion of, the main study. This feedback was applied to modify aspects of the larger study framework, only some of which included the peer-assessment process itself.

176

TRACEY PAPINCZAK ET AL.

The intervention phase of the study involved participation of intervention groups, for 20 weeks, in an activity designed to promote metacognition and enhance learning. As part of this larger framework, students were given practice in peer assessment for approximately 20 min each week as an integral part of the wrap-up phase of PBL. It was anticipated that repeated exposure to a system of participative assessment would create a culture of enthusiastic acceptance, with greater student support for the process, leading perhaps to greater satisfaction with the PBL process operating within tutorial groups. Control groups were not involved in this process. The intervention itself comprised two key components, both of which were readily integrated into the existing tutorial format with minimal additional time required from tutors or students: 1. Reection on learning: Each week one student from each intervention tutorial group was asked to compose a summary of the weeks problem incorporating the clinical reasoning and collaborative learning occurring in their PBL tutorial group. One student was encouraged to present the summary to the group as a concept map or in mechanistic case-diagramming format (see Guerrero, 2001) to give a visual representation of both the content and the clinical reasoning (a knowledge object (Entwistle and Marton, 1994)). 2. Participative assessment: The student presentation, in association with fullment of PBL roles and responsibilities for that week, was assessed using the Peer Assessment Instrument (rened through student negotiation with the previous cohort). Both peer- and tutor-assessment took place concurrently. Scores from these worksheets were compared to look for correlation between pairs of scores as part of a quantitative study of peer assessment. Constant monitoring of student perceptions of, and attitudes to, the intervention helped to monitor the eects of the dual activities to maximize student learning. This is in keeping with action research. In the rst week of the main study (week four of the academic year), students in the intervention cohort were distributed two documents: a copy of the Peer Assessment Instrument to enable students to become familiar with the evaluation criteria, and an exemplar outlining good and poor outcomes for the criteria. Tutors assigned to each intervention group then led their group in a practice session, with a tutor-led presentation of a summary of the previous weeks medical case, in order to establish familiarity with the instrument. The summary itself was written by the researcher and presented, with explanation, to each tutor in the week prior to the trial. In the ensuing weeks (526 of the academic year), intervention tutorial groups implemented the summarization and peer-assessment activities at the start of each week as part of the wrap-up of the previous weeks medical

PEER ASSESSMENT IN PROBLEM-BASED LEARNING

177

case. Tutors were encouraged to give concise feedback (based on written peer comments on the reverse of the assessment sheet) to students as soon as possible after the complete on of the peer-assessment procedure. Scores obtained from the Peer Assessment Instrument were collated and analysed. These quantitative results are to be reported elsewhere. Student feedback about the peer-assessment exercise was regularly invited as part of the action research process. Many groups chose not to reveal their opinions, therefore more in-depth feedback from selected intervention tutorial groups was obtained using one-on-one interviews. One major criticism of the peer-assessment process was identied student preference for more prompt, anonymous, and private feedback from the tutor (based on peer scores and written responses). This feedback was used to modify the main study design and tutors were advised to provide this form of feedback to students as much as possible. The attitudes of a randomly chosen control tutorial group to peer assessment were also sought at this time. Tutor feedback was also collected at 10-week intervals. During the course of the implementation, three tutorial groups withdrew from the study. Their justication for withdrawal and perceptions of their experiences of peer assessment will be reported. The remaining ten groups in the intervention cohort continued until week 28 of the academic year. Qualitative data were sought from these groups. DATA ANALYSIS Responses to all open questions about peer-assessment and concise observations of focus groups were collated into a single document. Raw data themes, in the form of direct quotations, functioned as the essential unit of analysis. In keeping with the context of qualitative research, inductive data analysis strategies were used with themes and categories emerging from the data rather than the data being grouped into pre-determined categories (Patton, 1990). Results The inductive data analysis procedure resulted in the identication of six main themes. The rst two are positive perceptions of peer assessment increased responsibility for others, and improved learning. The nal four themes reect negative student perceptions lack of relevancy, challenges, discomfort, and eects on the PBL process. While the literature supports ve of these themes, the nal theme which encompasses the potentially negative consequences of implementing peer assessment on the PBL process has not previously been described. However, Eva (2001) hypothesized that this eect may become evident in PBL tutorial groups when peer assessment is implemented.

178 PERCEIVED BENEFITS OF PEER ASSESSMENT Increased responsibility for others

TRACEY PAPINCZAK ET AL.

Concern for the potential eects of their feedback on the learning of their peers led students to make comments such as: Peer review is helpful (because) it keeps me mindful of the fact that all my input is ... important to the learning of the whole group. Another students comment: It keeps you on your toes and challenges you to come up with (feedback) that helps others shows how strongly some students felt about producing quality feedback for peer benet in the collaborative learning environment of PBL. Improved learning In this research, assessment of participation and leadership, in which the contribution of students towards the process of learning was evaluated and rewarded, was undertaken as a meaningful form of assessment of process (as supported by Armstrong and Boud, 1983). Two main types of advantages were described: immediate benets to learning and long-term, sustainable benets. Immediate benets. The positive consequences of constructive criticism were reported on several occasions by participants. Students felt their peers were in an appropriate position to judge their performance as functioning members of the PBL group: (peer-assessment) gives you a good idea of.... how you could improve and gives lots of ... ideas and it is good to get feedback from peers so that you know how you are going and where your strengths and weaknesses are. Through feedback from peers, opportunities for students to engage in self-evaluation were provided. This assisted students to identify deciencies in their understanding and skills that may otherwise not become readily apparent, thereby enhancing self directed learning. Long-term benets. Comments on future benets to learning from the practice of peer assessment were made by a small number of participants. Mindful of a future as a medical professional, one student explained the benets of her peer-assessment experience: ... good to learn how to do this appropriately, as I think we will need to be able to assess our peers performance, as well as our own, throughout our careers. Students made references to reecting on strengths and weaknesses as a means of improving future performance.

CRITICISMS OF PEER ASSESSMENT Lack of relevancy Despite the process of negotiation of assessment criteria with a previous cohort of students followed by repeated practice at peer assessment,

PEER ASSESSMENT IN PROBLEM-BASED LEARNING

179

perceptions that the exercise lacked relevancy persisted. Two groups of responses were able to be identied in this category: irrelevant criteria, and sceptical attitudes. Irrelevant criteria. Some students failed to see the relationship between the criteria on the feedback sheet and the aspects of PBL performance they were expected to assess. One student did not view the activities of PBL as suciently important to their learning as to warrant assessment: Most of the questions were irrelevant to the learning process. Another respondent was dissatised with the criteria: (I) dont feel that the assessment sheets enable us to give relevant feedback. Sceptical attitudes. Frequent comments such as: Not taken too seriously and Not too much thought goes into the marking reect a casual attitude to the peer-assessment process. Some students were overt in their scepticism, with 23% of the intervention cohort (three groups) withdrawing from the peer-assessment activity before its completion partly for that reason. Challenges The dicult nature of peer assessment was frequently described by students. Qualitative data analysis of students comments allowed two lower order categories to be identied: newness of peer assessment and lack of motivation. Newness of peer assessment. A common complaint from student assessors was diculty with the scoring system of the feedback sheet. These problems were lack of familiarity and diculties with the Likert scale. In addition, a small number of comments were made concerning the alien nature of peer assessment. One student noted: Unless you have a couple of weeks to get used to it, students dont really know how to. Nevertheless, students were given 20 weeks to become accustomed to the practice and this comment was received in the nal week. Lack of motivation. A minority of students expressed a dislike for peerassessment attributable to apathy. One individual comment highlighted the nature of this perception: I found myself consistently giving ves for most areasIt was easier than reading through every criteria and grading the personI think Im not alone in doing this. Discomfort Two lower order categories encompassing discomfort were identied: problems with objectivity and need for greater anonymity.

180

TRACEY PAPINCZAK ET AL.

Problems with objectivity. A strong reaction to peer assessment was the widespread perception that this process could be corrupted by bias due to friendship marking or lack of honesty. Analysis of data from students who left prematurely from the study indicated this was a very important inuence on the decision to withdraw. The following set of comments sums up the attitude among many study participants: I nd it dicult to downgrade my peers. (It is) hard to criticize friends. Relationships between students can colour opinions. No one wants to criticise others in PBL. Most people are too afraid to honestly mark their peers. An often reported criticism of peer assessment among students in this study was lack of condence in the evaluation skills of peers, for example, Peers are not qualied to mark....They can be biased or picky or too lenient. One student expressed greater condence in his own assessment abilities: I think my judgement is better than that of my peers. Anonymity. A minority of students expressed concern at the potential for others to discern the source of individual scores or comments. Two signicant remarks include: I think (peer assessment) is great but (it) needs to be done more anonymously and It is very dicult to assess peers within the connes of the PBL room, especially when we hand the forms in to the tutor through many sets of hands. Some students may be more sensitive than others to the negative consequences of anonymity being compromised (an unlikely scenario given the study design which preserved anonymity to a high degree). It was notable that one student praised the highly condential nature of her peer assessment experience: Its good to be able to do this anonymously a bit too confrontational face-to-face. Eects on the PBL process Serious reservations about the negative impact of peer assessment on the integrity and interactive functioning of the PBL tutorial group were demonstrated in this research. Both students and tutors revealed their concerns that the family atmosphere may be compromised by peer evaluation. One student commented: It promotes judgmental attitudes (and) tension, and destroys harmonious learning. By not having to rate my peers, (we could) learn in a friendly, non-judgemental environment. We (could) ask questions without fear of embarrassment or humiliation.

PEER ASSESSMENT IN PROBLEM-BASED LEARNING

181

Within their PBL tutorials, students need to feel free to hypothesize, to ask questions and request clarication of points raised by others in order to create a dynamic learning environment (Woods, 1994). Some students recognized that the collaborative learning within their small group tutorials, so dependent on aable working relationships, could be disrupted by ill-feeling brought about by negative peer evaluations. This susceptibility was also recognized by an experienced PBL tutor: I would not like to see a competitiveness develop in a PBL group. Everybody should feel comfortable to express their opinions, ask questions, and generally just have a go without a fear of being assessed. These potentially negative consequences were not only identied by students and tutors experiencing peer assessment, but were anticipated by students in a randomly selected control group. One student expressed her expectation: The lack of a threatening environment is what allows us to discuss ideas and concepts without fear of being rated or judged. In an environment like this, feeling safe and respected in your group is paramount... this would be screwed up by a scheme that asked you to rate your friends.

Discussion Much of the data is congruent with ndings in all disciplines of higher education. Students have expressed appreciation for the manner in which peer assessment enhances their learning, both short-term and in anticipation of future learning. Peer assessment was considered to strengthen the sense of responsibility that group members have for one another. These positive aspects of peer assessment can be regarded as endorsing the PBL approach, with its emphasis on collaborative and meaningful learning. Some students were enthusiastic and committed to assessment of their peers, endeavouring to give valid and helpful feedback to support others learning and to graciously receive feedback from other PBL group members. Initial quantitative data analysis suggests that these individuals were more accurate peer assessors. Several studies undertaken with students in non-medical courses have also revealed strong positive perceptions of peer assessment encouraging students to take a more active and constructive role in cooperative learning (Hanrahan et al., 2001; Orsmond and Merry, 1996), and to develop attitudes of responsibility for the learning of other group members (Burnett and Cavaye, 1980; Topping et al., 2000). Furthermore, studies show peer assessment enhances learning by allowing students to observe how others

182

TRACEY PAPINCZAK ET AL.

study and write (Ballantyne et al., 2002; Hanrahan and Isaacs, 2001), and by training students to dierentiate high from poor quality work (Race, 1998; Searby and Ewers, 1997). Negative perceptions were also strongly apparent in this research. A signicant number of respondents believed that peer assessment was irrelevant to the important aspects of learning in PBL. Scepticism and feelings of discomfort were very commonly reported. Of all ve sub-scores, self-awareness was the most problematic during negotiation of items for the instrument and in terms of student and tutor acceptability. Upon reection, omission of this sub-score may have improved student condence in the peer-assessment process (despite the principal role of self-awareness in PBL). While the PBL tutorial group oers the students an opportunity to embrace unfamiliar learning activities, such as collaborative small group learning with its emphasis on peer-to-peer interaction and communication, peer assessment may be the nal straw for those students struggling to adapt to this learning environment. It may be that learning environments that are highly dependent on eective collaboration are not well suited to peer assessment. Alternatively, the nature of peer assessment undertaken in highly collaborative learning environments may dictate a requirement for dierent criteria. Negative perceptions of peer assessment were anticipated in this research based on the ndings of other studies in higher education. Studies in non-medical courses have found widespread scepticism among students concerning the validity of peer assessment aligned with beliefs that assessment should be the sole responsibility of sta (Brindley and Scoeld, 1998; Orsmond and Merry, 1996; Searby and Ewers, 1997). Apprehension about the arbitrary nature of marking are a major drawback for students experiencing peer assessment (Ballantyne et al., 2002; Cheng and Warren, 1999). Pond and Rehan (1997) discuss the need to consider the problems of friendship marking (leading to over-marking) and collusive marking (resulting in lack of dierentiation within groups) when considering the introduction of peer assessment. In his inuential article on student attitudes towards assessment, Williams (1992) reported that students viewed peer assessment as criticism of friends. The recommendations made to address this perception, which included anonymous feedback from groups of peers with sta involvement to reduce bias, were adopted in this study with little apparent impact. It is a common nding that students struggle with the newness of peer assessment as a formal assessment tool (Cheng and Warren, 1999; Sluijmans et al., 2001). They often have little prior experience with the procedure. Criteria for peer assessment can create additional problems as some students have dierent understandings of individual criteria compared to their peers and tutor, particularly criteria requiring higher order thinking (Orsmond et al., 1996,2000). Lack of familiarity may also lead to feelings of

PEER ASSESSMENT IN PROBLEM-BASED LEARNING

183

awkwardness. Students in many studies, including studies of cooperative learning in postgraduate courses (Divaharan and Atputhasamy, 2002) and PBL courses in education (Segers and Dochy, 2001; Sluijmans et al., 2001) describe the discomfort of peer assessment. A signicant concern unique to peer assessment in PBL settings (and apparently fairly unique to this study) was revealed: students perceived that peer assessment had the potential to negatively impact on PBL. The intimate quality of relationships within PBL tutorials, which has often been embraced for its ability to improve self-evaluation and communication, also makes students reluctant to rate their peers for the purposes of evaluation (Van Rosendaal and Jennett, 1992). As Eva (2001) comments in his critique of tutorial-based assessment: There is a real concern that conducting evaluation in the tutorial setting may negatively impact on the process itself. So, paradoxically, while the tutorial is the best place to observe these behaviours, the act of observation and evaluation may inhibit their demonstration. (p. 244) There are few studies exploring the eects of peer assessment on the process of problem-based or case-based learning. Sluijmans et al. (2001) explored several aspects of peer assessment in a PBL course, collecting mostly quantitative data, but also noting that students were aware of the serious consequences of negative critiques within the small group environment. Divaharan and Atputhasamy (2002) described the unease experienced by students in the face-to-face assessment of peers within the connes of cooperative small groups, which had the potential to interfere with peer relationships. Boud et al. (1999), in their discussion of the limitations and contradictions of peer assessment, report that formal assessment of others within small group tutorials can inhibit cooperation, pitting one student against another. However, peers are a source of valuable and valid feedback which can assist learners to make judgements about themselves and their learning in PBL. Conclusion The results from this study support and extend research in higher education on the positive and negative perceptions of students about peer assessment. However, the qualitative data, gathered through ethnographic enquiry, presents a more in-depth representation of student attitudes to the introduction of peer assessment into a PBL course within a medical school. The study design adopted in this research followed many guidelines promoted in the literature. This included, but was not limited to, the use of student-negotiated criteria for assessment, provision of exemplars of

184

TRACEY PAPINCZAK ET AL.

performance, and student practice in peer assessment to gain familiarity. Despite implementation of these recommendations, student perceptions of their peer-assessment experience remained quite negative. It may be that students need years of practice in peer assessment in order to become comfortable with the process. Even though students were aware that peerassessment criteria were developed through negotiation with individuals in the previous cohort, time constraints early in the academic year prevented participating students from renegotiating criteria. This minimised ownership of the assessment process, an important consideration in enhancing student acceptance of participative assessment (Boud, 1995; Strahan and Wilcox, 1996). Student ownership of criteria for peer evaluation may be crucial to its acceptance as a meaningful activity. The learning advantages oered by peer assessment are quite signicant and warrant further research. Productive benets may be gained through both receipt of feedback and provision of meaningful feedback to others. These types of activities can improve condence and enhance learning (Black and Wiliam, 1998; Woods, 1994). Peer-assessment needs to be further explored with students in PBL and other cooperative small group settings, including evaluation of fullment of the roles and responsibilities of group members (perhaps utilising criteria reecting what students value most from their PBL experience). While the highly structured format of peer assessment was viewed negatively by many students, an unprompted, anonymous and open-ended design may be more acceptable. Such an approach may enable students to give feedback which is relevant to their perceptions of valuable learning and meaningful evaluation in PBL. Black and Wiliam (1998) found that an approach to providing student feedback that involved writing comments on work, but not assigning scores, resulted in greater learning. An integral part of the PBL tutorial process is the willingness and ability of students to embrace their duties as active members of a group of learners. Principles such as critical analysis, self-awareness, and information processing play an important role in the development of self-direction as a learner. One of the gauges of quality in tertiary education is the extent to which selflearning ability is nurtured (Oldeld et al., 1995). A key aspect of this is developing the ability to judge objectively the quality of their own and others work.
References
Armstrong, M.T. & Boud, D. (1983). Assessing class participation: An exploration of the issues. Studies in Higher Education 8(1): 3344. Ballantyne, R., Hughes, K. & Mylonas, A. (2002). Developing procedures for implementing peer assessment in large classes using an action research process. Assessment & Evaluation in Higher Education 27(5): 427441.

PEER ASSESSMENT IN PROBLEM-BASED LEARNING

185

Barrows, H.S. & Tamblyn, R.M. (1980). Problem-Based Learning: An Approach to Medical Education, New York: Springer. Bishop, W. (1999). Ethnographic Writing Research, Portsmouth, NH: Boynton/Cook Publishers. Black, P. & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education 5(1): 774. Boud, D. (1995). Enhancing Learning Through Self-Assessment, London: Kogan Page. Boud, D., Cohen, R. & Sampson, J. (1999). Peer learning and assessment. Assessment & Evaluation in Higher Education 24(4): 413427. Brindley, C. & Scoeld, S. (1998). Peer assessment in undergraduate programmes. Teaching in Higher Education 3(1): 7989. Burnett, W. & Cavaye, G. (1980). Peer assessment by fth year students of surgery. Assessment in Higher Education 5(3): 273278. Cheng, W. & Warren, M. (1999). Peer and teacher assessment of the oral and written tasks of a group project. Assessment & Evaluation in Higher Education 24(3): 301314. Collins, M. (1991). On contemporary practice and research: Self-directed learning to critical theory. In: R. Edwards, A. Hanson & P. Raggatt (eds.), Boundaries of Adult Learning, London: Routledge/Open University Press. Dancer, D. & Kamvounias, P. (2005). Student involvement in assessment: A project designed to assess class participation fairly and reliably. Assessment & Evaluation in Higher Education 30(4): 445454. Das, M., Mpofu, D., Dunn, E. & Lanphear, J.H. (1998). Self and tutor evaluations in problem-based learning tutorials: is there a relationship? Medical Education 32(4): 411418. Divaharan, S. & Atputhasamy, L. (2002). An attempt to enhance the quality of cooperative learning through peer assessment. Journal of Educational Inquiry 3(2): 7283. Dochy, F., Segers, M. & Sluijmans, D. (1999). The use of self-, peer-, and co-assessment in higher education. Studies in Higher Education 24(3): 331350. Dolmans, D.H. & Schmidt, H.G. (1996). The advantages of problem-based curricula. Postgraduate Medical Journal 72(851): 535538. Drew, S. (2001). Student perceptions of what helps them learn and develop in higher education. Teaching in Higher Education 6(3): 309331. Dueld, K.E. & Spencer, J.A. (2002). A survey of medical students views about the purposes and fairness of assessment. Medical Education 36(9): 879886. Entwistle, N.J. & Marton, F. (1994). Knowledge objects: Understandings constituted through intensive academic study. British Journal of Educational Psychology 64(1): 161178. Eva, K.W. (2001). Assessing tutorial-based assessment. Advances in Health Sciences Education 6(3): 243 257. Gale, K., Martin, K. & McQueen, G. (2002). Triadic assessment. Assessment & Evaluation in Higher Education 27(6): 557567. Guerrero, A.P.S. (2001). Mechanistic case diagramming: A tool for problem-based learning. Academic Medicine 76(4): 385389. Hanrahan, S.J. & Isaacs, G. (2001). Assessing self- and peer-assessment: The students view. Higher Education Research and Development 20(1): 5370. Hay, P.J. & Katsikitis, M. (2001). The expert in problem-based and case-based learning: Necessary or not?. Medical Education 35(1): 2228. Kamil, M., Langer, J.A. & Shanahan, T. (1985). Understanding Reading and Writing Research, Boston: Allyn & Bacon. Kelson, A.C.M. & Distlehorst, L.H. (2000). Groups in problem-based learning (PBL): Essential elements in theory and practice. In: D.H. Evenson & C.E. Hmelo (eds.), Problem-Based Learning: A Research Perspective on Learning Interactions, Mahwah, NJ: Lawrence Erlbaum Associates. Kemmis, S. & Wilkinson, M. (1998). Participatory action research and the study of practice. In: B. Atweh, S. Kemmis & P. Weeks (eds.), Action Research in Practice: Partnerships for Social Justice in Education, pp. 2136. London: Routledge. Leung, W. (2002). Why is evidence from ethnographic and discourse research needed in medical education: the case of problem-based learning. Medical Teacher 24(2): 169172. McDowell, L. (1995). The impact of innovative assessment on student learning. Innovation in Education and Training International 32(4): 302313.

186

TRACEY PAPINCZAK ET AL.

Margetson, D. (1994). Current educational reform and the signicance of problem-based learning. Studies in Higher Education 19(1): 519. Miller, P.J. (2003). The eect of scoring criteria specicity on peer and self assessment. Assessment & Evaluation in Higher Education 28(4): 383394. Mowl, G. & Pain, R. (1995). Using self and peer assessment to improve students essay writing: A case study from geography. Innovation in Education and Training International 32(4): 324335. Norman, G. & Schmidt, H.G. (1992). The psychological basis of problem-based learning: A review of the evidence. Academic Medicine 67(9): 557565. Norton, L., Brunas-Wagsta, J. (2000). Students perceptions of the fairness of assessment. Paper presented at the Institute for Learning and Teaching in Higher Education Annual Conference, College of Ripon and York, St John. Oldeld, K.A., MacAlpine, J. & Mark, K. (1995). Peer and self-assessment at tertiary level: An experiential report. Assessment & Evaluation in Higher Education 20(1): 125132. Orsmond, P. & Merry, S. (1996). The importance of marking criteria in the use of peer assessment. Assessment & Evaluation in Higher Education 21(3): 239250. Orsmond, P., Merry, S. & Reiling, K. (2000). The use of student derived marking criteria in peer- and selfassessment. Assessment & Evaluation in Higher Education 25(1): 2338. Patton, M.Q. (1990). Qualitative Evaluation and Research Methods, Newbury Park, CA: Sage Publications. Pond, K. & Rehan, U.H. (1997). Learning to assess students using peer review. Studies in Educational Evaluation 23(4): 331348. Pressley, M. & McCormick, C.B. (1995). Advanced Educational Psychology for Educators, Researchers and Policymakers, New York: Harper Collins College Publishers. Punch, K.F. (1998). Introduction to Social Research, London: Sage. Race, P. (1998). Practical pointers on peer-assessment. In: S. Brown (ed.), Peer Assessment In Practice, SEDA Paper 102, Birmingham: SEDA. Reiter, H.I., Eva, K.W., Hatala, R.M. & Norman, G.R. (2002). Self and peer assessment in tutorials: Application of a relative-ranking model. Academic Medicine 77(11): 11341139. Sambell, K. & McDowell, L. (1998). The value of self and peer assessment to the developing of lifelong learning. In: C. Rust (ed.), Improving Student Learning: Improving Students as Learners, Oxford: Oxford Centre for Sta and Learning Development. Searby, M. & Ewers, T. (1997). An evaluation of the use of peer assessment in higher education: A case study in the school of music. Assessment & Evaluation in Higher Education 22(4): 371383. Segers, M. & Dochy, F. (2001). New assessment forms in problem-based learning: The value-added of the students perspective. Studies in Higher Education 26(3): 327343. Sluijmans, D.M.A., Moerkerke, G., Merrienboer, J.J.G.van & Dochy, F.J.R.C. (2001). Peer assessment in problem-based learning. Studies in Educational Evaluation 27(2): 153173. Somervell, H. (1993). Issues in assessment, enterprise and higher education: The case for self-, peer- and collaborative assessment. Assessment & Evaluation in Higher Education 18(3): 221233. Strahan, I.B. & Wilcox, S. (1996). Peer and self-assessment of group work: Developing an eective response to increased enrolment in a third-year course in microclimatology. Journal of Geography in Higher Education 20(3): 343353. Sullivan, M.E., Hitchcock, M.A. & Dunnington, G.L. (1999). Peer and self assessment during problembased tutorials. The American Journal of Surgery 177(3): 266269. Topping, K.J., Smith, E.F., Swanson, I. & Elliot, A. (2000). Formative peer assessment of academic writing between postgraduate students. Assessment & Evaluation in Higher Education 25(2): 146169. Van Rosendaal, G.M.A. & Jennett, P.A. (1992). Resistance to peer evaluation in an internal medicine residency. Academic Medicine 67(1): 63. Williams, E. (1992). Student attitudes towards approaches to learning and assessment. Assessment & Evaluation in Higher Education 17(1): 4558. Willis, S.C., Jones, A., Bundy, C., Burdett, K., Whitehouse, C.R. & ONeill, P.A. (2002). Small-group work and assessment in a PBL curriculum: A qualitative and quantitative evaluation of student perceptions of the process of working in small groups and its assessment. Medical Teacher 24(5): 495501. Woods, D.R. (1994). Problem-based Learning: How to Gain the Most from PBL, Waterdown, ON: Donald R. Woods.