Peer evaluation: 1 am not the

teacher*
Sima Sengupta

This article is based on an exploratory investigation of a secondary school

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
writing class in Hong Kong. Through examination of the way learners in
this study viewed the roles of the teacher and learner as 'readers' of the
compositions they had written, it explores the extent to which the broader
educational context and its belief system shaped six ESL students' per-
ception of peer evaluation. Finally, the article questions whether notions of
collaborative construction of knowledge in the classroom are viable
options within an examination-driven, accuracy-oriented L2 curriculum
which may preclude learners (and teachers) from re-conceptualizing their
traditional roles.

Introduction Many researchers have studied peer and self-evaluation in LI and L2

writing (for example Mangelsdorf 1992, Mendon?a and Johnson 1994).
Findings suggest that student writers take selective account of peer
comments when they revise, preferring to depend more on their own
knowledge. Mendonca and Johnson (1994: 762) also discovered that
student writers may not always trust their peers, but the same comment
from a teacher will be taken into account when they revise. Mangelsdorf
(1992: 280) reports that peer reviews were always rated negatively by
Asian students, and raises the question of the effect of teacher-centred
cultures on the way students regard peer comments. Carson and Nelson
(1996: 18) believe that cultural factors, such as harmony-maintenance
strategies, guide Chinese learners when they participate in peer
evaluation. Even in Western cultures, Freedman et al. (1986), for
example, found that even when peer evaluation is planned and
controlled by the teacher, there may be social implications behind the
responses which are determined by the way students maintain social
relations (ibid.: 15). They point out that peer evaluation often turns out
to be an exercise in futility because students are more busyfiguringout
easy ways to complete the evaluation sheets than evaluating the text
(ibid.: 21). So, learners may pay only lip-service to the task (Mangelsdorf
1992: 280).

However, it must be stressed that the peer evaluation described in this

study was not implemented in the same way as the peer conferencing/
review procedures in the studies mentioned above. In the context of this
study in Hong Kong, the demands of the English language syllabus
allowed only limited time and resources to be spent on peer and self-
evaluation.
ELT Journal Volume 52/1 January 1998 © Oxford University Press 1998 19
 Background In the first five years of secondary school in Hong Kong, reading,
writing, speaking, and listening skills are taught separately. There are
regular examinations at the end of each term, and at the end of the fifth
year, English language skills are assessed by means of a public
examination (the Hong Kong Certificate of Education Examination).
The examination includes a composition paper that accounts for 25 per
cent of the total grade. The examination is marked using double
impression holistic grading.
Tse (1993) points out that in Hong Kong schools studying to pass
examinations is common, and the teaching of writing is very examina-
tion-oriented. Almost all compositions are done in the classroom and, in

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
many respects, are considered tests. Language teachers give a topic, set a
time limit, and specify the number of words to be written. The usual
practice is to correct the grammatical errors in each composition and
then move on to a new topic (Sengupta 1996). The product-centred
culture of Hong Kong emphasizes accuracy at the expense of self-
expression (Harris 1993: 516).
At the outset, the learners with whom this study was conducted matched
this description. For this study, however, they received additional
instructional input aimed at developing reader awareness both before
and after writing the first draft. At the post first draft stage, input
consisted of audience analysis with questions such as, 'What does the
reader need to know in order to understand the writer?'.
The learners were first taught revision strategies with guided tasks, such
as analysing a text by posing questions on the content. Then two copies
of the evaluation sheet for the task (see Appendix) were handed out,
and the teacher elicited what the questions required, sometimes
changing, correcting, or simplifying them, where necessary. Students
were first asked to read their own compositions and complete the
evaluation sheets. Then they exchanged their first draft with the person
sitting next to them, read each other's first drafts, completed the
evaluations sheet, and then returned the evaluation sheet and the
composition to the writer. Each student read the evaluation sheet
completed by the other student and revised the first draft. The teacher's
role was that of a facilitator, explaining difficult words or demonstrating
how to answer the questions in the evaluation sheets.
In this way peer evaluation sheets were used as a support for revising
first drafts. The evaluation students were expected to do was not in itself
extensive but it was hoped that by giving them an opportunity to
evaluate their peers' texts, a number of pedagogic and practical goals
would be accomplished. Firstly, it would help students understand a
reader's perspective because someone other than the writer would read
the text. Secondly, it would enable students to become independent
writers invoking the authority of personal experience as 'potential
readers' (Reid 1993: 46). Thirdly, it would enable the teachers and the
researcher to examine whether the strategies taught in order to develop
writing and revising skills had actually been learnt.
20 Sima Sengupta
 The research The research project was designed to answer the following questions:
project
1 Did the evaluation sheets and the resulting revisions indicate that
there were textual changes arising from peer evaluation?
2 Did the students believe that peer evaluation led to awareness of
themselves as real readers?

Sample The sample consisted of a class of fifteen to sixteen year-old girls, whose
native language is Cantonese. The majority of the girls came from
middle or lower-middle class backgrounds. The medium of instruction in
the school was English. The teacher was a native English speaker.

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
Data collection and Firstly, the self and peer-evaluation sheets were compared in order to
analysis identify peer suggestions that were distinct from those made by the
writers themselves. Then the revisions were examined to see whether
there was any evidence that the peer suggestions had been incorpo-
rated.1 The aim of the analysis was to separate the changes that reflected
peer suggestions from those that did not. Twelve students' compositions
(6 pairs) were chosen for this analysis, and six students agreed to be
interviewed. Arising out of the analysis of the evaluation sheets, semi-
structured interview questions were framed and analysed within the
following inter-dependent constructs2:
1 perceived usefulness
2 perception of ability to respond and way of responding (i.e. how the
writer evaluated the text of a peer)
3 perception of real reader
4 concerns (e.g. the public examination)
5 beliefs about knowledge and perception of teacher's role.

Findings Self and peer-evaluation of the same composition did not differ
Changes made on substantially in the compositions selected. Although there were differ-
the basis of peer ences in the way these students executed peer evaluation, there were
evaluation notable similarities in their revision behaviour. Not one of the twelve
seemed to perform any revisions arising out of their peers' suggestions
unless they were problems the learners had already detected themselves.

Interviews For the interview the researcher started with a general question about
what the interviewee thought of peer evaluation. To this question all six,
with varying duration of pauses, said that the experience was okay or
useful. Only on further probing did the ideas that are reported emerge.
None of the six interviewees believed that peer evaluation led to
awareness of themselves as real readers. Moreover, the interviews
revealed a number of consistent concerns. Table 1 shows that four
categories of comments were made by all six subjects. The examination
is mentioned most often, followed by beliefs about the role of the
teacher as an evaluator, and a related perception of their own inability
to correct grammatical errors. These findings indicate how student
perceptions of the value of peer evaluation are shaped.
Peer evaluation 21
 Table 1:
Construct Comments Number of Number of
Summary of the main
(see interview interviewees times
concerns expressed by
data analysis) who mentioned
the students
mentioned it
4 Concerns voiced in the context 32
of the examination
5 Evaluation is the teacher's job 29
2 Cannot evaluate without
knowledge of grammar 19
3 and 5 The real reader is the teacher or
examiner, not the peer 14
2 Do not know how to read in

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
order to evaluate other's 11
composition; can only evaluate
own composition
2 Embarrassed to have peers read 18
the composition
2 Evaluation depended on aspects 13
that could be dealt with in the
revision
1 Not taken seriously by peers and 10
so did not do it seriously
2 Do not have schemata for text
type
2 Perceiving self less able 19
compared with peers
4 No similarity with real reading in
school

The interviewees seemed to imply that the instructional input had not
prepared them for peer evaluation because they could not be the real
reader. In these students' world, the real reader is the teacher or the
examiner 'who knows correct English', not a peer with a questionable
command of English. Although the students were being taught how to
look at discourse-related aspects of writing, they seemed to be so
conditioned by their experience of error correction as a follow-up to the
first draft that they had come to believe that accuracy could solve most
problems in writing, and that accuracy is the province of the teacher.

Discussion The interviews seemed to indicate that these writers did not perceive
Perceived any value in peer evaluation, and two learners called the whole exercise
usefulness a 'waste of time'. This typical comment makes it clear that these students
were completing the evaluation sheets to please the teacher:
Extract 1
Student: I think organization is better if teacher tells me what to
do. I think I do not like my neighbour to read my
composition. I have many mistakes. I am not . . . I do
not like . . . my class friend will laugh. So we read
quickly because . . . it is because [the teacher] tells me.
But I do not want to read many times and think many
22 Sima Sengupta
 times . . . I am not the real reader. So I read quickly and
write something on—in—the sheet.
Interviewer: What do you mean by 'something'?
Student: Something, I mean, a comment or something. I fill
it—ah—just fill it.
Intrinsically merged with the perception of usefulness was the context of
the approaching public examination and the role of the teacher in
preparing these students for it.

Role of the teacher Every interviewee felt that by completing these evaluation sheets they
were being asked to take on the role of the teacher. It seemed that the

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
students viewed the evaluation as something to be carried out by an
expert, and the teacher, as the one who 'knows English', was therefore
better able to evaluate than the students, whose English was seen as
'weak'.
Extract 2
Student 1: No not at all useful for me. I think it was useless. It is
because I want to know from the teacher how to make
this composition better . . . to get more . . . to get good
HKCE pass. I cannot tell Sarah and Rachel how to get
a better result. . . she is already better than me.
Interviewer: But you can read and see if there are questions you ask
as reader, just like the exercises in the notes—the way
you were taught. Did these exercises help you?
Student 2: But they helped me to read my own composition but
only [the teacher] can tell me how to make my
composition better. It is her . . . it is her work . . . job.
Since linguistic feedback from the teacher was readily available before
the study began, the strategies for evaluation were seen as unnecessary.3
The interviewer's suggestions that even grammatical errors could be
corrected by discussing them with peers or referring to a grammar book
were met with comments such as, 'But the teacher must tell me'.

The reader and There was constant reference to the readers' English proficiency, as if to
writing purpose indicate that there was an idealized target reader in the writer's mind
who did not in any way match the peer reader. This idealized reader
derived his or her status not only from knowing English, but also because
he or she gave the grade. The students' perception of themselves as real
readers was closely connected with this conceptualization of the idealized
reader, as seen in Extract 3:
Extract 3
Interviewer: So you think the real reader can only be someone
whose English is excellent?
Student: Of course.
Interviewer: Why?
Student: It is because the reader is giving me a F. (Laughs) Yes
because this reader is not reading like reading the
Peer evaluation 23
 newspaper but reading to pass or fail. So she must know
English.
Providing them with a real reader and a real writing purpose was seen as
peripheral by these students. It seemed that they were so conditioned by
the examination and their expectation that the teacher would prepare
them for it that they did not give the imaginary reader or the purpose a
second thought. Extract 4 is an example of what one student thought of
the reader and the purpose of Task 2 (see Appendix).
Extract 4
Interviewer: But if the police received such a letter what will they

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
expect? Did you read thinking of yourself as a
policeman?
Student: We know that the police are not going to read this, only
the teacher will, so we have to try to read like the
teacher .. . but my English is not so good, so how can I
read like the teacher? Why will my classmate learn
from me? She knows I do not know how to write ... not
good in this subject. I cannot help her get a good result.
The idea that learning is only possible from an authority figure who
knows 'correct' English is not surprising in a context where the focus on
accuracy naturally precludes risk taking. Even in LI writing, Tse (1993:
389) observes that students seemed not to have been encouraged in
school to express themselves freely and to attempt uncertain forms of
expression.

Classroom reading The mental configuration of the idealized real reader's perspective was
not only influenced by the students' perception of the teacher's role in
'helping them get a good result'; they also felt that neither reading in an
English lesson in school nor reading in real life was performed in this
evaluation mode. As pointed out by one student:
Extract 5
Interviewer: So you think that your neighbour was not a real reader?
Student: Of course no. How can she be a real reader? We don't
read like this. We read a passage and answer questions.
The passage is in correct English and good writing—it is
because it is in a book. When we read the passage we
do not give suggestions or see which paragraph is not so
good. This is not reading. I do not read anything and
think if it is helping me as a reader. I read a passage to
answer questions.
Reading in school is done to retrieve information and thus, although
they were learning to evaluate texts in their writing lesson, these
students failed to perceive the connection between the task of reading
and evaluating and reading done in the L2 classroom.4
Summary The demands of the educational context made these students unable to
value learning to write beyond the instrumental need for a good grade.
24 Sima Sengupta
 The whole exercise of writing, evaluating, and revising was reduced to one
main concern: the grade. The belief that there was only one real reader—
the person who gave the grade—was voiced repeatedly by all the
interviewees, indicating that the source of their perception of the teacher's
role in the L2 curriculum is intrinsically linked with the importance
attached not only to the examination but also to accuracy. In the classroom
the teacher, with her knowledge of 'correct English', gives the grades, is
thus the only 'real' reader, and is responsible for teaching accurate writing.
It seemed clear from the interviews that the learners see their
responsibility as to do what the teacher asks. The possibility of learning
from their peers by looking at a potential reader's suggestions, or asking
for genuine reader impressions, did not occur to these learners as their

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
'lack of knowledge' prevented such a possibility within the accuracy-
oriented context.

Conclusion Clearly peer evaluation was not able to bring a real reader's perspective.
A number of reasons for this may be considered. It is likely that the way
instruction in revision was designed and executed had failed to help
students to become the real reader, and that the input may not have
prepared students with adequate linguistic and cognitive maturity to
evaluate and act upon the evaluation.5
In spite of the negative feedback from students, the evaluation sheets
demonstrated that students were, in fact, reading for evaluation, and
were able to do this adequately in many cases. However, providing
students with an evaluation sheet may have encouraged a prescriptive
stance (see Mangelsdorf and Schlumberger 1992: 248) rather than a
collaborative one. The question is that if peer evaluation were
introduced as a collaboration between writer and peer, would students
value the experience more positively?
Although much more focused research is essential to address this
question in any depth, the data regarding students' perceptions seem to
indicate that these students were not able to benefit from this exercise
because, to them, the teacher was the only reader. The traditional roles
of the teacher and learner in the school curriculum seem so deep-rooted
that the only possible interpretation of knowledge appears to be that it is
transmitted from the teacher to the student, and not constructed by the
classroom community. Unless these perceptions regarding teachers'
roles are addressed, it is probable that little value will be attached to
peer evaluation, and collaborative and autonomous learning by
secondary school ESL students may not become a reality. As Onore
(1989: 231-2) points out
As long as judgements of what may be 'better' or 'worse'—that is, of
what constitutes improvement in writing—remain the province of the
teacher alone, then the writer cannot fully and authentically engage in
choice making and problem solving.
Received May 1996
Peer evaluation 25
Notes
1 However, through this analysis it is claimed that
the revision changes performed on the texts
were solely due to peer evaluation.
2 Segments of transcripts were given to a col-
league to categorize in order to establish
reliability. For eight out of the ten segments
there was agreement.
3 The fact that the feedback received prior to the
study was predominantly linguistic did not seem
to matter to these learners, to whom 'accuracy'
was synonymous with a 'good' composition, in
spite of the discourse-related input.
4 This may be partly due to the current division of
the English curriculum into the four skill areas
in Hong Kong schools.
5 Beach (1989: 28) points out that it is proble-
matic to demonstrate to large groups how
evaluation should be done because students
have disparate problems and abilities. However,
considering the realities of the large classrooms
of Hong Kong, it is essential to get teaching to a
mixed ability class. How this can be done
effectively needs further investigation.
References
Anson, CM. (ed.). 1989. Writing and Response.
Urbana, 111.: National Council of Teachers of
English.
Beach, R. 1989. 'Showing students how to assess:
demonstrating techniques for response in the
writing conference' in C. M. Anson (ed.).
Carson, J.G. and G.L. Nelson. 1996. 'Chinese
students' perception of ESL peer response
group interaction. Journal of Second Language
Writing. 5/1: 1-19.
Freedman, S., C. Greenleaf, and M. Sperling.
1986. Response to Student Writing (Research
Report No. 23). Urbana, 111.: National Council
of Teachers of English.
26 Sima Sengupta
Harris, J. 1993. 'I see what I mean! Exploring the
relationships between writing and learning and
learning to write' in N. Bird, J. Harris, and
M. Ingham (eds.). Language and Content. Hong
Kong: Institute of Languages in Education.
Mangelsdorf, K. 1992. 'Peer reviews in the ESL
composition classrooms: what do the students
think?' ELT Journal 46/3: 274-84.
Mangelsdorf, K. and A. Schlumberger. 1992. 'ESL
student response stances in a peer-review task'.
Journal of Second Language Writing. 1/3: 235-4.
Mendonca, CO. and K.E. Johnson. 1994. 'Peer
review negotiations: revision activities in ESL
writing instruction'. TESOL Quarterly 28/4:
Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
745-69.
Onore, C. 1989. 'The student, the teacher and the
text: negotiating meaning through response and
revision' in CM. Anson (ed.).
Reid, J. 1993. 'Historical perspectives on writing
and reading in the ESL classroom' in
J.G. Carson and I. Leki (eds.). Reading in the
Composition Classroom. Boston: Heinle and
Heinle.
Sengupta, S. 1996. 'A Study of the Effects of the
Systematic Teaching of Revision on the L2
Learners in a Secondary School in Hong Kong'.
PhD thesis, Department of Curriculum Studies,
University of Hong Kong.
Tse, S.K. 1993. 'The Composing Process of Hong
Kong Children in Primary Schools.' PhD thesis,
University of Nottingham.
The author
Sima Sengupta is a lecturer in the Department of
English, Hong Kong Polytechnic University,
where she teaches English to undergraduate
students. She has just completed her PhD in
writing and revision. Her research interests
include second language writing, reading, and
teacher education.
 Appendix: Taskl
Evaluation sheets Your views are sought. Please respond:
To keep the Hong Kong beaches clean it has been suggested that people should
have to buy a ticket worth HK$ 50 to enter the beach. They will not be allowed
to carry food, drink, or plastic bags. Do you think that this is a good idea?
Why/Why not?

Read the composition and fill in the evaluation sheet.

Name of writer Name of evaluator

Intention

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
Please circle the answers unless lines are provided for longer answers
1 Is the intention clear? Yes/No
2 What is the intention? Agrees/Disagrees with the policy statement
3 How soon does the reader know about your intention? In the first/second/
third/last paragraph.

Macrostructure
4 Does the composition have a definite macrostructure? Yes/No
5 Has the macrostructure been signalled with specific words e.g. problem,
solution? Yes/No
6 Do the problems follow a logical sequence?
a. Yes, the main problems are mentioned first.
b. Yes, the problem of the HK$ 50 charge has been dealt with first, and then
the question of plastic bags has been discussed.
c. No, no sequence.
d. Other sequence, e.g.
7 Do you have any suggestions about how to improve the macrostructure?

Arguments
8 What are the main arguments? (please write in note form)

(i)

(ii)

(iii)
9 Do the arguments have the three elements: claim, data, and warrant?

Argument (I)

Argument (ii)

Argument (iii)
10 Do you have any suggestion about revising the arguments?
11 Does each paragraph have a main point? What is it?

Paragraph 1

Paragraph 2

Paragraph 3

Paragraph 4

Peer evaluation 27
 12 Do you have any suggestions for changes in the paragraphs?

13 What is the best feature of this composition? (Circle one only)

a. Ideas e. Paragraphs are well developed
b. Macrostructure f. Purpose is clear
c. Arguments g. Readers finds it easy to read
d. Clearly stated main points h. Other
14 What is the weakest feature? Choose one from the options for question 13.

Downloaded from http://eltj.oxfordjournals.org/ at Cultura Inglesa - São Paulo on April 27, 2014
Task 2
Context: A series of bullying incidents which led to the hospitalization of a
schoolboy. The police are seeking information. (Three picture prompts are
given, with details such as clothes, appearance, time, street name, etc.)
Write a report to the police describing what you saw on three separate occasions.

Read the composition and fill in the evaluation sheet.

1 Have I/my classmate given the police enough information?
Whole essay: Yes/No (please circle)
Paragraph 1 Yes/No e.g.
Paragraph 2 Yes/No e.g.

Paragraph 3 Yes/No e.g.

Paragraph 4 Yes/No e.g.

If any detail can be added put a plus sign (+)
If you feel any important information can be added, put a cross (x) next to the
paragraph number and give examples.
If you feel some information is irrelevant put a question mark (?).
2 Have I/my classmate made it easy for the police to read and picture the
events as they happened, or will they have to read the report many times to
know exactly what happened?
Whole essay: Yes/No
Can the organization be made more reader-friendly by re-organizing
paragraphs?
Which paragraphs?
Why?
a. Confusing for the reader.
b. Too long.
c. The police must be told the most important details first.
d. The order or sequence of events is wrong.
e. This is a report and thus must have facts not personal views.
3 Does the language help the police understand what is meant?

28 Sima Sengupta