LANGUAGE ASSESSMENT QUARTERLY, 3(4), 333–368

Copyright © 2006, Lawrence Erlbaum Associates, Inc.

Watching for Washback: Observing the

Influence of the International English
Language Testing System Academic
Writing Test in the Classroom
Anthony Green
University of Cambridge ESOL Examinations

Previous studies of washback (the influence of a test on teaching and learning) have
provided insights into the complexity of educational systems and test use, especially
in relation to the role of the teacher, but have given insufficient attention to the rela-
tionship between observed practices and test design features. In this article a wash-
back model is proposed that incorporates both test design and participant characteris-
tics. The model is used to predict behaviour on preparation courses directed toward
the Academic Writing component of the International English Language Testing
System (IELTS) test. 197 learners and 20 teachers were observed over 51 classroom
hours. These encompassed 22 IELTS preparation classes and, for comparison, 13
classes of English for academic purposes (EAP). Evidence was found for substantial
areas of common practice between IELTS and other forms of EAP but also for some
narrowing of focus in IELTS preparation classes that could be traced to test design
features.

Before offering a place to an international student, most universities in Eng-

lish-speaking countries will require evidence of the student’s language ability. As
increasing numbers of students choose to travel to access global educational op-
portunities, there has been rapid growth in the use of language tests for this pur-
pose. In the United Kingdom the most widely recognised test of English for aca-
demic purposes (EAP) is the International English Language Testing System
(IELTS). Between 1995 and 2005 the number of candidates rose from under
50,000 to over half a million per year (International English Language Testing

Correspondence should be addressed to Anthony Green, Validation Unit, University of Cambridge

ESOL, 1 Hills Road, Cambridge, CB1 2EU, UK. E-mail: green.a@cambridgeesol.org
334 GREEN

System, 2005). The rapid expansion of the test has brought with it increased de-
mand for test preparation books and courses.
IELTS is used as a means of determining whether candidates should be ac-
cepted into English-medium courses and whether they will require further lan-
guage support. However, concern has been expressed that preparation for tests like
IELTS, because of the limitations on what can realistically and equitably be tested
in a few hours, may not develop the full range of skills required for successful uni-
versity study, particularly in the area of academic writing (Deakin, 1997; Read &
Hirsh, 2005). J. Turner (2004), for example, argues that “what the IELTS test or the
TOEFL test delivers underspecifies the complexity of language issues in the aca-
demic context” (p. 98). Her concern is that education in academic literacy is being
supplanted by training in test taking. But what influence does the IELTS writing
test really have on teaching, and how different are writing classes in IELTS prepa-
ration courses from other forms of EAP?
To investigate these questions, this article compares the practices observed in
writing classes of two types: IELTS preparation classes directed at success on the
test and, as a suitable point of comparison, presessional EAP writing classes pro-
vided by universities to prepare learners for academic study.

LITERATURE REVIEW

Washback studies, investigating the effects of tests on the teaching and learning di-
rected toward them, have often involved direct observation of the behaviour of
teachers and learners in the classroom. The inclusion of an observational element
in such studies has been recommended as a means of contextualising, corroborat-
ing, or correcting data from surveys and interviews (Alderson & Wall, 1993; C.
Turner, 2001; Wall, 1996; Watanabe, 2004).
Table 1 summarises the methods and findings of recent case study investiga-
tions of washback in language education that have included an observational ele-
ment. These studies covered a wide range of educational contexts, with observa-
tion either focussing on a small number of participants observed intensively over a
sustained period (Alderson & Hamp-Lyons, 1996; Read & Hayes, 2003) or on a
lighter sampling of classes to allow for observation of larger numbers of teachers
and a broader perspective (Hawkey, 2006; Wall, 2005).
With the exception of Burrows (1998, 2004) and Hawkey (2006), all included
comparisons between different types of class. Wall (1996, 2005) and Cheng (2005)
focused on changes over time as a new test is introduced. This approach also in-
formed work relating to the recent update of the Test of English as a Foreign Lan-
guage (TOEFL; Hamp-Lyons, 2005; Wall & Horák, 2004). Alderson and
Hamp-Lyons (1996) and Watanabe (1996, 2004) compared the practices of teach-
 TABLE 1
Studies of Washback in Language Education That Include Observation and Interview Data

Study Exam Institutions Observations N and Frequency Interviews Key Findings

Alderson and Test of English as Specialised Purpose-designed 16 classes: 8 3 focus group TOEFL washback may be
Hamp-Lyons a Foreign language instrument, general English, interviews with generated more by
(1996) Language institute in 2 teachers 8 TOEFL 3–12 students; teachers, materials, writers,
(TOEFL) United States preparation over 9 teachers in and administrators than by
1 week group and test. “Amount” and “type”
individual of washback vary
sessions according to test status,
relation of test to nontest
practice, degree to which
teachers and material
writers think about
appropriate methods and
their willingness to
innovate.
Brown (1998) International University-based Records of All classes Not reported IELTS preparation instruction
English language instructional designated for closely focussed on the test
Language institute focus and writing over 10 with regular timed practice
Testing System materials, weeks; 70 hours of IELTS tasks. IELTS
(IELTS) 2 teachers in IELTS and 30 preparation was more
hours in successful than EAP
non-IELTS EAP course in improving IELTS
course scores from entry to exit,
but small samples (9
IELTS, 5 EAP) limit
interpretation.

335
(continued)
 TABLE 1 (Continued)

Study Exam Institutions Observations N and Frequency Interviews Key Findings

336
Burrows (1998; Certificates in Adult English COLT, 4 teachers Two 4-hour lessons 30 teacher Teachers vary in responses to
2004) Spoken and Migrant English per teacher interviews— innovations in assessment
Written English Programme including those in line with individual
(CSWE) (AMEP) in observed differences and experience.
Australia (conducted With Cheng (2003) and
prior to the Wall (2005), links
observation washback to theories of
phase) change.
Cheng (2005) Hong Kong 3 HK secondary Adaptation of 31 70-min classes Follow-up Following innovation,
Certificate schools (main COLT, c. 6 months interviews with teachers engaged in
Examinations study) 3 teachers before exam over observed exam-like activities e.g.,
in English 2 years teachers role-plays, and used
(HKCEE) (conservative) exam prep
materials in class, but
beliefs and attitudes to
learning did not change.
Differences in how
teachers cope with change.
Extensive support required
in implementation of
innovations.
Hawkey (2006) IELTS 10 language Cambridge IIS 10 IELTS 120 students, 21 Teacher preference for
schools in UK, instrument, 10 preparation teachers, and 15 task-based, often
Japan, teachers classes receiving interrelated macroskills
Cambodia institution activities, involving
administrators microskills relevant to
in focus groups IELTS. Focus on IELTS
but willingness to use a
range of materials and
methods.
 Read and Hayes IELTS 2 language COLT and 22 hours of 32-hour 23 teacher Differential teacher practices
(2003); Hayes schools in New Cambridge IIS course and 28 interviews; at least partially linked to
and Read Zealand instrument, hours of 320- weekly institutional context.
(2004) 2 teachers hour (8-month) interviews with Greater pressure to “teach
course the teachers to the test” in private
observed language schools.
Wall (2005) Sri Lankan ‘O’ c. 50 secondary Purpose-designed 5 lessons observed 64 teachers in Teachers used textbook
level schools in 11 observation over 2 years focus groups; content but failed to follow
areas of Sri checklist (39–64 different follow-up suggested methods, e.g.,
Lanka classes observed interviews with reading for gist.
per round) at each class Proportion of classes was
varying periods observation dedicated to exam
ahead of exam preparation. Neglect of
speaking skills traceable to
exam content.
Watanabe (1996; Various Japanese 3 high schools in Adaptation of 964 min of exam Teacher follow-up Teachers vary in their
2004) university Japan COLT, 5 preparation and interviews with approaches to exam
entrance teachers 833 min of each preparation—informed by
examinations regular classes observation attitudes towards the exam.
over 6 months School culture an
important factor. Material
designed for exam
preparation may
sometimes be used for
other purposes.

337
338 GREEN

ers in test preparation and non-test-preparation classes, while Read and Hayes
(2003; Hayes & Read, 2004) compared two approaches to test preparation.
Three of the observational washback studies listed in Table 1 investigated
IELTS preparation classrooms. Brown (1998) compared practices in two courses
provided by the same institution: IELTS preparation and non-IELTS EAP. Read
and Hayes (2003) compared two IELTS preparation courses at different institu-
tions: one an intensive preparation course, the other combining IELTS preparation
with other forms of EAP. The ongoing IELTS Impact Study (IIS; Hawkey, 2006)
also includes observational data of IELTS preparation classes.
These studies all found IELTS to affect behaviour. Brown (1998) found that stu-
dents in the 10-week IELTS preparation course used IELTS preparation textbooks,
completed one Task 1 and one Task 2 essay each week (and no other writing), per-
formed three timed practice examinations, were informed about IELTS scoring
criteria, received feedback on the accuracy of their written work, and were in-
structed in strategies for writing under timed conditions. In contrast, students in the
EAP course worked on a 1,000-word project, did no timed writing, were instructed
in strategies for writing in academic contexts, and were encouraged to develop re-
search skills. Read and Hayes (2003) combined a broad survey with targeted class-
room observation of two teachers to provide mutual corroboration of findings. The
briefer and more intensive of the two IELTS classes they observed was more
narrowly concerned with the test and included more test practice under timed
conditions.
In common with other washback studies, Hawkey (2006) found variation be-
tween the 10 teachers he observed in how they conducted their classes, notably in
the number of opportunities they provided for learners to communicate together in
English. Teachers were willing to employ a variety of teaching methods and to use
material both within and beyond the textbook. However, both the institutions pro-
viding the courses and the students, who were motivated to succeed on the test, ap-
peared to constrain teachers to focus their instruction on IELTS. In class, teachers
showed a preference for task-based activities, targeting microskills they believed
to be relevant to the test.
Data from observational studies have informed insights into the complexity of
educational systems and test use, especially in relation to the role of the teacher.
However, a shortcoming identified by Bachman (2005) is the lack of a coherent ev-
idential link between test design characteristics and the practices observed or re-
ported in the classroom. In studies that involve predicting the effects of a test on in-
struction or learning, appeal is more often made to the views of educational
authorities (Cheng, 2005; Ferman, 2004; Qi, 2004), teachers (Banerjee, 1996), or
widely held public perceptions (Watanabe, 1996) than directly to the design of the
test instrument (as evidenced by available test materials or test specifications).
How might the design of the IELTS Academic Writing component be expected to
influence instruction?
 WATCHING FOR WASHBACK 339

Watanabe (2004) suggested two sources of evidence that may be used to relate ob-
served practices to the influence of a test. One is evidence that test design features are
reflected in teaching or learning. The other is the absence of such features in teaching
or learning not directed toward the test (or directed toward an alternative test). Of the
three IELTS-related studies, only Brown (1998), in common with Alderson and
Hamp-Lyons’s (1996) study of TOEFL preparation, incorporated a comparison
with courses that were not directed toward the test. The inclusion of just two classes
in each of the Australian studies limits their generalisability, while the lack of a
nontest comparison in Hawkey (2006) makes it difficult to disentangle test influence
from teacher variables. This study involves observation of a larger number of teach-
ers and learners than Brown (1998) and Read and Hayes (2003) but includes the
comparison with nontest EAP classes missing from Hawkey (2006).
In considering the mechanisms of washback, a growing body of theory relates
test design, test use, and classroom behaviours, although as Wall (2005) argued,
too little is sufficiently informed by empirical evidence. Most of this work takes
the form of recommendations to test developers. Chapman and Snyder (2000) pro-
vided a framework for relating tests to educational practices, and Brown (2000)
cited Hughes (1989), Heyneman and Ransom (1990), Kellaghan and Greaney
(1992), Bailey (1996), and Wall (1996) in identifying features of a test that may be
manipulated in efforts to improve instruction. These embrace both contexts for test
use and technical qualities of the test instrument.
Drawing together these two elements in washback theory, Green (2003) pro-
posed the predictive model of test washback set out in Figure 1. The model starts
from test design characteristics and related validity issues of construct representa-
tion identified with washback by Messick (1996) and encapsulated in Resnick and
Resnick’s (1992) formulation of overlap, or the extent of congruence between test
design and skills developed by a curriculum or required in a target language use
domain. Test design issues are most closely identified with the direction of
washback—whether effects are likely to be judged beneficial or damaging to
teaching and learning.
The model relates design issues to contexts of test use, including the extent to
which participants (including material writers, teachers, learners, and course pro-
viders) are aware of and are equipped to address the demands of the test and are
willing to embrace beliefs about learning embodied therein. These features are
most closely related to washback variability (differences between participants in
how they are affected by a test) and washback intensity. Washback will be most in-
tense—have the most powerful effects on teaching and learning behaviours—
where participants see the test as challenging and the results as important (perhaps
because they are associated with high stakes decisions, such as university entrance;
Bailey, 1999; Cheng, 2005; Hughes, 1993; Watanabe, 2001).
A survey of IELTS preparation courses conducted in tandem with this study
(Green, 2003) indicated that the test was regarded both as important and challeng-
340 GREEN

FIGURE 1 A model of washback direction, variability, and intensity adapted from Green
(2003).

ing by a majority of learners in the IELTS preparation courses involved (70% rated
success on the test as “very important”). It also indicated that almost all the learn-
ers were taking the test because they intended to enter higher education in the
United Kingdom. Most demonstrated at least a basic knowledge of the test format
and viewed the direct test of writing as a positive feature. In short, the conditions
for intense washback to a majority of participants would seem to be in place.
Beyond a general prediction that the direct testing of writing in IELTS would
encourage the teaching of writing in test preparation programs, the washback
model suggests that features of task design will impact on the nature of that in-
struction. The IELTS Academic Writing component (AWC) is one of four
skill-based subtests in the IELTS battery intended to assess “the language ability of
candidates who intend to study or work where English is used as the language of
communication” (International English Language Testing System, 2005). On the
AWC, candidates are required to compose two short essays in 1 hour. The first
 WATCHING FOR WASHBACK 341

(Task 1) is a description of a diagram or table, and the second (Task 2) is a discur-

sive essay written in response to a point of view, argument, or problem.
Drawing both on the available literature (Banerjee, 2000; Chalhoub-Deville &
Turner, 2000; Coffin, 2004; Douglas, 2000; Hale et al., 1996; Mickan & Slater,
2003; Moore & Morton, 1999; Thorp & Kennedy, 2003) and on the views of
course providers, a framework developed by Weigle (2002) was used to relate the
design of the IELTS AWC to theories of academic literacy. Information about the
test was derived from sample test materials published by the IELTS partners and
from unpublished specifications for item writers obtained from Cambridge Eng-
lish for Speakers of Other Languages (Cambridge Assessment, Cambridge, UK)
under a research agreement. The review highlighted the similarities between the
two IELTS tasks and paradigmatic reports or essays assigned by participant teach-
ers in universities but also highlighted differences between the design of the IELTS
AWC and the broader concerns of EAP. Briefly, the following areas emerged as the
key differences between the two.
IELTS tasks involve

• abstract and impersonal topics, but these are not targeted at learners’ chosen
academic subjects
• a limited range of text types (descriptions of iconic data and five-paragraph
persuasive essays)
• composition of texts based on personal opinions about how the world should
be.

They do not involve

• literature reviews, summaries, and other genres of relevance to the academic

context
• arguments based in reading and research (as university assignments
often do).

In short, IELTS may imply an approach to instruction that passes over features
of canonical EAP, such as the integration of source material in learners’ writing,
learning of subject- specific vocabulary and text types, and strategies for coping
with the length of university-level written assignments. Preparing for IELTS may
involve learning how to shape texts to meet the expectations of examiners (as ex-
pressed through the scoring criteria) rather than those of university staff. The scor-
ing criteria may imply a focus in the classroom on grammar, vocabulary, and or-
ganisation of text, with limited attention given to quality of content. The timed
conditions may lead to practice in composing short texts under timed conditions,
perhaps in response to past (or imitation) examination papers.
342 GREEN

It should be noted that the IELTS AWC has undergone continual modification
and revision since its inception in 1989. The test has changed in certain respects
since this research was conducted. Where these changes are of relevance to the
study, this is indicated in the text. Regular updates on IELTS developments can be
found on the IELTS Web site (http://www.ielts.org).

METHODS

Methodology
The literature review has pointed to specific features that might indicate the influ-
ence of IELTS AWC test design on instruction. To investigate whether these pre-
dictions about the likely impact of IELTS would be borne out in this context and to
explore how instruction varied between IELTS preparation and EAP classes in ac-
ademic writing, a series of classroom observations were carried out at selected
U.K. centres. The observations provided evidence of how preparation for the
IELTS AWC was conducted in practice and how this compared with other forms of
EAP writing provision. The courses included in the study involved IELTS prepara-
tion in combination with varying proportions of EAP and general English classes.
However, comparisons were made at the narrower level of the writing class: be-
tween classes directed toward the IELTS AWC and EAP classes directed at prepar-
ing learners for writing in English at U.K. universities.

Instrumentation
One observation instrument that has been widely used in washback studies (Bur-
rows, 1998; Cheng, 2005; Read & Hayes, 2003; Watanabe, 1996) is the Communi-
cative Orientation to Language Teaching (COLT) observation schedule (Spada &
Fröhlich, 1995). The scheme is designed to be used flexibly and in real time, de-
scribing classroom events at the level of activities and their constituent episodes.
These are described qualitatively and recorded quantitatively under a series of
headings to build a picture of the balance of patterns of classroom organisation,
content, and student modality. Spada and Fröhlich (1995) explain the terms thus:
“Separate activities include such things as a drill, a translation task, a discussion or
a game. Three episodes of one activity would be: teacher introduces dialogue,
teacher reads dialogue aloud, individual students read parts of dialogue aloud”
(p. 14).
As Read and Hayes (2003) discovered, the COLT schedule could not, without
adaptation, identify features of direct test preparation, such as learning test-taking
strategies. Read and Hayes (2003) chose to supplement the COLT with the draft
IIS observation schedule, an instrument developed by the IELTS partners in col-
laboration with researchers from the University of Lancaster at the inception of the
IIS (Saville, 2000). This schedule includes lists of text types and activities antici-
 WATCHING FOR WASHBACK 343

pated to occur in preparation classes and is specifically designed to record in-

stances of IELTS preparation. To address the needs of this study, but avoiding du-
plication, elements of the IIS instrument were incorporated into the COLT
schedule, and the resulting instrument was further refined through piloting. The
observation schedule appears in the Appendix.
Among the modifications to the COLT, a Test References section was included
with three categories: IELTS, Other, and Test Strategies. The IELTS section was
used to record mentions of the IELTS test, a separate note being made of the con-
text. The Other section was used to record mention of tests or assessments other
than IELTS (such as course exit tests). The Test Strategies section was used as a re-
cord of test-taking strategy instruction, and the specific strategies were also noted.
Copies of all materials used in class were collected for later analysis, and details
were recorded separately on a second page of the observation form (see the Appen-
dix). Under the heading “content,” the distinction made in the COLT between per-
sonal or broad topics was extended to further differentiate broad from academic.
An academic topic was indicated where teachers and students treated the topic as
academic subject matter: The primary focus would be learning about the topic,
rather than exploiting the topic to learn about language. Additional notes were
made of aspects of instruction, such as homework assignments, that were of inter-
est but not captured by the schedule.
The adapted schedule was piloted with five classes, including 240 min of
presessional EAP and 150 min of IELTS preparation (60 in an IELTS intensive
course and 90 in a course combining IELTS preparation with EAP). Observations
were recorded in real time at intervals of 1 min, and the time in hours and minutes
was entered at each episode boundary.
To provide an estimate of internal consistency, two classes observed at the be-
ginning of the data collection exercise were video recorded and reanalyzed in real
time 3 months later. The 82.5% level of agreement between the two sets of obser-
vations suggests that the instrument was being used consistently during the study.
Interrater reliability was investigated in cooperation with an IIS consultant, who
was trained in the use of the schedule and independently observed the two
videocassettes. These ratings showed complete agreement on the number of activi-
ties observed, with minor discrepancies in timing. There was agreement on 72% of
the observed categories, with most of the differences being on the minor focus of
an activity. For example, the first observer recorded that one activity was led by the
teacher, but the second also recorded that this involved learners working in small
groups.

Participants and Settings

To provide a cross section of courses, a range of institutions were approached by
telephone to invite participation. These institutions were selected following an
344 GREEN

earlier survey of U.K. course providers; they had indicated willingness to partic-
ipate in further research and were conveniently located. Three of these (two uni-
versities and one private language school) declined the invitation. The remaining
institutions—six universities, three colleges of further education, and four pri-
vate language schools—represented a variety of courses for students intending
to enter higher education in the United Kingdom, both in IELTS preparation
and, for purposes of comparison, presessional courses in EAP offered by univer-
sities to prepare international students for the language demands of academic
study.
The intention was to observe a minimum of one writing-focused class for each
month of a course, although this did not always prove possible. As summarised in
Tables 2 and 3, some 36 scheduled classes were observed covering over 51 hours,
involving a total of 197 learners and given by 20 different teachers. The classes in-
cluded eight different EAP groups. Each of these was observed either once or
twice over periods ranging from 8 to 12 weeks to give a total of 13 classes. There
were 12 IELTS preparation groups. Each of these was observed on between one
and three occasions over periods of between 4 and 12 weeks to give a total of 22
preparation classes.
Following each observation, teachers were briefly interviewed about the class.
The interviews focused on five issues: the aims of the class, the extent to which the
aims had been met, the place of the focal class in a teaching sequence, the extent to
which the class could be described as typical of writing classes on the course, and
the influence of the IELTS AWC. Because of other commitments, teachers did not
always have the time to take part in these interviews. Nonetheless, 22 of the classes
were accompanied by interview data (8 EAP and 14 IELTS).

Analysis
Comparisons were made between writing-focused IELTS preparation classes and
EAP classes across courses (Table 4). The length of time spent on each activity
provided an index of how much time and what proportion of class time (calculated
as a percentage) was given to each form of participant organisation, content focus,
content control, and student modality. Mentions of tests or of test-taking strategies
were treated as simple frequency data. As the data were not normally distributed,
differences between classes were evaluated for significance (p < .05) through non-
parametric Mann–Whitney U tests.
The activities observed were also reviewed qualitatively in the light of the
washback predictions to explore (a) how far the predictions were borne out in
IELTS preparation and (b) how far they differentiated between writing-focused
IELTS preparation and EAP classes. Follow-up interviews provided an opportu-
nity to probe aspects of teachers’ behaviour.
 TABLE 2
Summary of IELTS Preparation Classes Observed

Course
No. of Hours/ Hours/ Obs No. of
Centre Course Type Class Title Teacher Week Weeks Week Week Activities Min Ss

College A IELTS and EAP IELTS 1 (F) 3 6 8 23 6 82 7

College B IELTS and Gen English IELTS 2 (M) 2 12 8 23 7 137 7
College B 6 23 7 92 9
College B 10 23 4 94 4
College C IELTS and Gen English IELTS 3 (M) 3 4 6 21 4 95 8
College D IELTS and Gen English IELTS 4 (F) 2 8 8 23 6 122 10
College D 7 23 7 118 7
College E IELTS IELTS writing 5 (M) 1 6 10 25 6 148 12
College E 5 25 3 114 8
College E IELTS IELTS writing 6 (M) 1 6 10 25 4 67 12
College E 3 25 6 119 11
College E 5 25 4 114 12
University A IELTS and EAP IELTS 7 (F) 3 6 4.5 22.5 9 93 7
University A 5 22.5 5 96 7
University B IELTS and EAP IELTS 8 (M) 2 8 8 20 5 55 9
University B 6 20 5 46 7
University B IELTS and EAP IELTS 9 (M) 2 8 8 20 2 50 8
University B 6 20 3 27 7
University C IELTS and EAP IELTS 10 (F) 2 8 7.5 25 6 90 8
College F IELTS and EAP IELTS 11 (M) 3 8 9 21 5 79 14
College G IELTS and EAP IELTS 12 (F) 4 10 10 25 3 52 5
College G 6 25 3 38 6
Average 7.50 8.08 22.95 5.00 87.64 8.41

345
346
TABLE 3
Summary of EAP Classes Observed

Course
No. of Hours/ Hours/ Obs No. of
Centre Course Type Class Title Teacher Week Weeks Week Week Activities Min Ss

College G IELTS and EAP Study skills 12 (F) 4 10 10 25 6 61 11

College F 6 25 5 58 11
College A IELTS and EAP Reading and writing 13 (M) 3 12 8 23 9 127 7
College A 11 23 5 79 5
University D EAP Reading and writing 15 (F) 4 12 7.5 20 5 123 15
University D 8 20 5 126 11
University E EAP Academic writing 16 (M) 2 8 10 28 4 89 13
University E 6 28 3 52 14
University E EAP Academic writing 17 (F) 2 8 9 28 7 97 14
University E 6 28 7 95 14
University F EAP Reading and writing 18 (M) 7 8 7.5 25 4 93 12
University F EAP Reading and writing 19 (F) 7 8 7.5 25 3 98 10
University F EAP Reading and writing 20 (M) 2 8 7.5 25 4 73 12
Average 9.25 8.38 24.85 5.15 90.08 11.46
 TABLE 4
A Comparison of Timing, Episodes, and Activities in IELTS-
and EAP-Focused Classes (Minutes)

Episodes
Observed Min per per
Course Type Centre time Activities Episodes Activity Activity

IELTS Preparation College A 82 6 22 13.67 3.67

College B 137 7 26 19.57 3.71
College B 92 7 27 13.14 3.86
College B 94 4 18 23.50 4.50
College C 95 4 14 23.75 3.50
College D 122 6 14 20.33 2.33
College D 118 7 20 16.86 2.86
College E 148 6 20 24.67 3.33
College E 114 3 8 38.00 2.67
College E 67 4 16 16.75 4.00
College E 119 6 14 19.83 2.33
College E 114 4 16 28.50 4.00
University A 93 9 23 10.33 2.56
University A 96 5 11 19.20 2.20
University B 55 5 14 11.00 2.80
University B 46 5 18 9.20 3.60
University B 50 2 8 25.00 4.00
University B 27 3 9 9.00 3.00
University C 90 6 16 15.00 2.67
College F 79 5 15 15.80 3.00
College G 52 3 10 17.33 3.33
College G 38 3 6 12.67 2.00
Average IELTS 87.64 5.00 15.68 18.32 3.18
Preparation
EAP College G 61 6 13 10.17 2.17
College G 58 5 20 11.60 4.00
College A 123 9 32 14.11 3.56
College A 126 5 16 15.80 3.20
University D 89 5 15 24.60 3.00
University D 52 5 22 25.20 4.40
University E 97 4 15 22.25 3.75
University E 95 3 10 17.33 3.33
University E 93 7 25 13.86 3.57
University E 98 7 20 13.57 2.86
University F 73 4 14 23.25 3.50
University F 127 3 12 32.67 4.00
University F 79 4 11 18.25 2.75
Average EAP 90.08 5.15 17.31 18.67 3.39
Significance 0.68 0.89 0.61 0.84 0.39
(Mann–Whitney
U test)

347
348 GREEN

RESULTS: FREQUENCY DATA

The observed IELTS and academic writing classes were of similar length. The ra-
tio of episodes to activities was also very similar across class types; there were ap-
proximately 10 episodes to every 3 activities, with each activity taking up just over
18 min on average.
Table 5 compares participant organisation by course type. The results of the
Mann–Whitney test displayed in the final row of Table 5 indicate that there were
no significant (p < .05) differences between class types in participant organisation.
The predominant form across classes was Teacher–Students/Class. Calculating
this as a percentage of total class time for each course type gives 56% of
presessional EAP class time and 54% of IELTS preparation class time (see Table
5). This did not generally involve lectures, but whole-class interactions centred on
the teacher. The differences found by Read and Hayes (2003) between teachers in
how they organised their classes were not repeated across the broader sample here,
nor did students spend as much of their time in class on individual test practice.
However, the proportion of class time spent on individual work on the same exer-
cise did increase as IELTS classes progressed: an average of 20% of class time in
the first half of these courses compared with 33% in the second half.
Reflecting the focused atmosphere of the classes observed, discipline was not
recorded as the main focus of any activity, although occasionally, a teacher re-
minded students to speak English rather than their first language. Procedural issues
took up 9% of time in IELTS and 12% of time in presessional EAP courses. Ob-
served variation in the amount of time spent on procedures also appeared to be a
matter of teacher style, rather than of test focus.
The results displayed in Table 6 reveal some evidence of differences between
class types in content. Forty-six percent of IELTS class time involved a major fo-
cus on language form (grammar and vocabulary) as compared with 22% of
presessional EAP class time. However, this did not prove to be significant in the
analysis (p = .07). Across classes observed, topics were mostly broad: relating to
issues of general rather than personal interest. Fifty-eight percent of time in IELTS
preparation classes and 48% of time in presessional EAP classes was spent work-
ing with topics in this category (p = .41). Little time (4% of IELTS and 1% of
presessional EAP classes) was spent on immediate personal topics (and these typi-
cally occurred only during brief introductory episodes). These proportions are
similar to those observed by Read and Hayes (2003). Academic topics—those top-
ics that became the focus of the class in their own right—occurred only in two
presessional EAP classes, making up 2% of the total presessional EAP class time
observed. Again, the differences between courses were not significant (p = .19).
Across the classes observed, topics were generally limited to one or two activi-
ties. Switches of topic would occur with each change of task. However, there was
evidence that topics were pursued in more depth in at least some EAP classes. Four
 TABLE 5
A Comparison of Participant Organization in IELTS- and EAP-Focused Classes (Minutes)

Observed Individual Individual Group Group

Course Type Centre Time T–C/S S–C/S Same Different Same Different

IELTS preparation College A 82 40 0 39 0 0 0

College B 137 48 0 71 0 0 0
College B 92 51 5 26 0 13 0
College B 94 64 0 36 0 6 0
College C 95 53 0 4 0 38 0
College D 122 26 0 22 0 74 0
College D 118 61 0 9 0 33 17
College E 148 57 4 17 0 43 0
College E 114 37 0 0 0 75 0
College E 67 66 0 0 0 0 0
College E 119 73 0 0 0 46 0
College E 114 59 0 0 0 55 0
University A 93 66 0 10 0 17 0
University A 96 67 0 0 0 29 0
University B 55 22 0 22 0 11 0
University B 46 15 0 15 0 16 0
University B 50 38 0 9 0 3 0
University B 27 11 0 0 0 15 0
University C 90 58 0 25 0 5 0
College F 79 49 0 5 0 25 0
College G 52 38 0 0 0 14 0
College G 38 29 0 9 0 0 0
(continued)

349
350
TABLE 5 (continued)

Observed Individual Individual Group Group

Course Type Centre Time T–C/S S–C/S Same Different Same Different

Average IELTS preparation 87.64 46.73 0.41 14.50 0.00 23.55 0.77
EAP College G 61 31 0 3 0 26 0
College G 58 41 0 8 0 18 0
College A 123 64 14 49 0 0 0
College A 126 34 0 12 0 20 0
University D 89 28 0 13 0 43 0
University D 52 64 0 17 0 45 0
University E 97 62 0 16 0 7 0
University E 95 50 2 0 0 0 0
University E 93 78 0 16 0 6 0
University E 98 53 0 35 0 6 0
University F 73 52 24 6 0 20 0
University F 127 36 0 17 5 40 0
University F 79 20 23 0 22 0 3
Average EAP 90.08 47.15 4.85 14.77 2.08 17.77 0.23
Significance 0.68 0.90 0.09 0.68 0.06 0.69 0.73
(Mann–Whitney U test)
 TABLE 6
A Comparison of Content in IELTS- and EAP-Focused Classes (Minutes)

Observed
Course Type Centre Time Procedural Discipline Form Function Discourse Sociolinguistic Immediate Broad Academic

IELTS preparation College A 82 3 0 0 0 0 0 0 72 0

College B 137 20 0 81 10 13 0 0 105 0
College B 92 18 0 0 0 74 0 0 78 0
College B 94 0 0 82 10 0 0 0 44 0
College C 95 7 0 37 21 37 0 4 50 0
College D 122 13 0 101 47 0 0 0 113 0
College D 118 15 0 40 0 34 0 0 77 0
College E 148 7 0 0 13 0 0 29 48 0
College E 114 8 0 15 0 0 0 0 60 0
College E 67 3 0 57 57 9 0 0 8 0
College E 119 5 0 100 26 8 4 0 53 0
College E 114 5 0 29 0 3 0 0 41 0
University A 93 4 0 37 0 10 0 0 45 0
University A 96 9 0 87 43 29 0 0 38 0
University B 55 3 0 28 22 8 0 0 48 0
University B 46 4 0 22 0 0 0 40 0 0
University B 50 9 0 44 0 0 0 0 44 0
University B 27 1 0 0 0 24 0 0 25 0
University C 90 14 0 40 51 19 0 0 71 0
College F 79 15 0 14 52 0 0 9 41 0
College G 52 5 0 13 2 24 0 0 19 0
College G 38 3 0 35 0 0 0 0 10 0
(continued)

351
352
TABLE 6 (Continued)

Observed
Course Type Centre Time Procedural Discipline Form Function Discourse Sociolinguistic Immediate Broad Academic

Average IELTS 87.64 7.77 0.00 39.18 16.09 13.27 0.18 3.73 49.55 0.00
preparation
EAP College G 61 8 0 0 0 54 0 0 0 0
College G 58 7 0 25 3 37 3 0 29 0
College A 123 18 0 0 0 90 0 0 58 0
College A 126 3 0 24 16 45 0 0 73 0
University D 89 2 0 33 0 6 29 2 99 0
University D 52 16 0 47 45 0 0 0 35 0
University E 97 7 0 22 74 0 0 0 0 0
University E 95 5 0 0 0 47 0 0 0 23
University E 93 28 0 48 0 3 0 5 9 0
University E 98 7 0 0 0 92 0 0 0 0
University F 73 3 0 0 0 89 0 0 89 0
University F 127 21 0 0 0 5 0 0 71 0
University F 79 10 0 38 0 0 0 0 60 0
Average EAP 90.08 10.38 0.00 18.23 10.62 36.00 2.46 0.54 40.23 1.77
Significance 0.68 0.36 1.00 0.07 0.22 0.08 0.28 0.72 0.41 0.19
(Man–Whitney
U test)
 WATCHING FOR WASHBACK 353

presessional EAP classes (29% of those observed) remained focused on a single

topic for the duration of a class, while IELTS classes tended to switch topic more
frequently; only one IELTS class (5%) remained with a single topic throughout.
Eleven of the IELTS classes (50% of those observed) included more than five top-
ics, while this was true of just three (21%) of the presessional EAP classes.
IELTS was mentioned by participants a total of 129 times during IELTS classes,
compared with 10 times during presessional EAP classes (Table 7). Of these 10
mentions, 9 were on combination IELTS/EAP courses (courses in which students
were studying EAP but were also following an IELTS preparation course strand).
In these cases, teachers mentioned how the class content could be applied to the
test, or students asked for information about the test. Specific test strategies or
“test-taking tips” were provided by teachers on a total of 67 occasions, or just over
three times per class on average. Just 2 of the 22 IELTS classes observed (both
given by the same teacher) included no explicit mention of the IELTS test, al-
though even here, it remained the implicit focus for class activities and did appear
in class materials. References to IELTS and mention of test-taking strategies were
the only quantitative features recorded on the observation schedule to show signifi-
cant differences between class types (p = .00012 for references to IELTS and p =
.00002 for mention of test-taking strategies).
Tests other than IELTS were mentioned just five times during presessional EAP
classes on courses unrelated to IELTS. One class accounted for four of these men-
tions. This included 50 min of explicit preparation for a course exit test to be held 3
days after the observation. The teacher introduced the test format, describing the
timing and format of the tasks, and gave 25 min to a practice writing exercise.
Results for content control are also displayed in Table 7. In both IELTS and
presessional EAP classes the teacher or text most often held control of class con-
tent (82% of the time in IELTS and 73% of the time in presessional EAP classes).
There was no significant difference between course types (p = .21). Students did
not hold sole control of content during any of the classes observed, but control was
shared between teacher, text, and students a little more often in presessional EAP
(27% of class time) than in IELTS classes (18% of class time). Again, the differ-
ences were not significant (p = .07), but the degree of teacher control was some-
what higher and student control lower than that observed by Read and Hayes
(2003).
Modality was similar across course types (Table 8). Listening (mostly during
teacher-centred activities) took up just over half of class time in both types of class
(p = .93), while writing activities took up around 10% (p = .81).
IELTS teachers were, like those observed by Read and Hayes (2003) and by
Hawkey (2006), eclectic in their use of materials, using a variety of books and
self-produced materials. Four of seven books used in IELTS classes included a ref-
erence to IELTS in their titles. Of the remaining three, two were intended as prepa-
ration material for other tests. The four IELTS titles were all course books directed
 TABLE 7
A Comparison of Test Focus (Frequency) and Content Control (Minutes) in
IELTS- and EAP-Focused Classes

Course Type Centre IELTS Other Strategies T/Text T/S/Text

IELTS Preparation College A 7 0 6 45 37

College B 7 0 4 72 0
College B 19 0 3 66 3
College B 2 0 3 89 0
College C 2 0 1 63 31
College D 6 0 3 95 26
College D 4 0 3 105 0
College E 3 0 1 54 58
College E 3 0 1 114 0
College E 14 0 6 66 0
College E 5 0 4 119 0
College E 14 0 6 114 0
University A 9 0 7 75 18
University A 5 0 3 61 35
University B 3 0 2 46 6
University B 3 0 3 19 30
University B 0 0 0 50 0
University B 0 0 0 26 0
University C 7 0 7 81 9
College F 5 0 1 79 0
College G 8 0 9 52 0
College G 3 0 0 38 0
Average IELTS 5.86 0.00 3.32 69.50 11.50
Preparation
EAP College G 0 0 0 35 26
College G 3 0 0 47 11
College A 3 0 0 27 7
College A 3 0 0 59 20
University D 0 0 0 43 62
University D 0 1 0 77 31
University E 0 0 0 85 0
University E 0 0 0 31 21
University E 1 0 0 97 0
University E 0 0 0 95 0
University F 0 0 0 15 75
University F 0 4 1 61 37
University F 0 0 0 65 23
Average EAP 0.77 0.38 0.08 56.69 24.08
Significance 0.00 0.06 0.00 0.21 0.07
(Mann–Whitney
U test)

354
 TABLE 8
A Comparison of Student Modality in IELTS- and EAP-Focused Classes (Minutes)

Observed
Course Type Centre time Listening Speaking Reading Writing S&R S&W L&R R&W Other

IELTS Preparation College A 82 40 12 0 0 12 18 0 0 0

College B 137 46 16 0 23 0 0 0 24 0
College B 92 33 45 14 0 0 0 0 0 5
College B 94 47 19 0 28 0 0 0 0 8
College C 95 39 18 29 0 0 0 0 9 0
College D 122 22 16 30 26 6 0 0 22 15
College D 118 48 9 4 0 9 0 0 48 0
College E 148 91 17 0 40 0 0 0 0 0
College E 114 39 0 60 0 0 0 0 15 0
College E 67 67 0 0 0 0 0 0 0 0
College E 119 72 0 0 0 0 0 0 46 0
College E 114 70 0 0 41 0 0 0 0 0
University A 93 53 10 10 3 0 0 3 14 0
University A 96 66 0 29 0 0 0 0 1 0
University B 55 24 9 16 0 0 0 0 6 0
University B 46 11 15 8 4 0 0 0 8 0
University B 50 37 9 4 0 0 0 0 0 0
University B 27 10 0 17 0 0 0 0 0 0
University C 90 61 10 0 0 19 0 0 0 39
College F 79 38 7 29 0 5 0 0 0 0
College G 52 39 0 14 0 0 0 0 0 0
College G 38 29 0 9 0 0 0 0 0 0
(continued)

355
356
TABLE 8 (Continued)

Observed
Course Type Centre time Listening Speaking Reading Writing S&R S&W L&R R&W Other

Average IELTS 87.64 44.64 9.64 12.41 7.50 2.32 0.82 0.14 8.77 3.05
Preparation
EAP College G 61 31 3 26 0 3 0 0 0 0
College G 58 46 8 2 0 0 0 0 0 3
College A 123 50 12 1 12 0 0 13 39 0
College A 126 15 0 1 20 27 0 16 0 0
University D 89 48 0 13 62 0 0 0 0 0
University D 52 47 8 17 25 17 10 0 0 0
University E 97 62 0 4 11 5 0 0 7 0
University E 95 52 0 0 0 0 0 0 0 0
University E 93 36 6 0 0 6 3 0 0 0
University E 98 53 0 7 0 12 0 5 22 0
University F 73 56 7 0 20 3 0 7 0 0
University F 127 36 17 41 0 0 0 0 4 0
University F 79 22 29 8 0 7 0 0 7 0
Average EAP 90.08 42.62 6.92 9.23 11.54 6.15 1.00 3.15 6.08 0.23
Significance 0.68 0.93 0.35 0.46 0.81 0.61 0.22 0.16 0.96 0.34
(Mann–Whitney
U test)
 WATCHING FOR WASHBACK 357

toward the test, rather than collections of practice test material. In presessional
EAP courses observed, none of the titles included the acronym IELTS, and none
was intended primarily as a test preparation text.

ACTIVITIES AND EPISODES: QUALITATIVE

OBSERVATIONS RELATING TO TEST DESIGN ISSUES

The frequency data were indicative of broad similarities between courses in teach-
ing method. There was little variation associated with the type of course in partici-
pant organisation, content control, or student modality. In contrast, the qualitative
observations captured in the activities and episodes sections of the schedule high-
lighted differences in the content that were not well captured in the frequency data.
These qualitative observations are presented here in relation to the task description
categories developed by Weigle (2002).

Subject Matter and Stimulus

During work in class, topics were similar in both IELTS and EAP classes. EAP
classwork did not generally relate to specific academic disciplines to any greater
extent than did IELTS preparation, although, as noted previously, the IELTS
classes did involve more frequent changes in topic. Where learners were directed
to explore topics further, IELTS teachers suggested reading magazines such as The
Economist to learn about topics that might occur in IELTS; EAP teachers set pro-
jects relating to students’ academic subjects. To this extent, the EAP classes did, as
predicted, take greater account of subject specialisms.

Genre, Rhetorical Task, and Pattern of Exposition

IELTS-directed classes, across teachers and institutions, involved activities with a
clear and direct relationship to the IELTS AWC. Frequently observed activities in-
cluded question analysis, brainstorming ideas, forming an outline plan with topic
sentences and sentence and paragraph building, all apparently directed toward
test-like writing practice exercises. In comparison with IELTS preparation, there
was a greater variety of activities on EAP courses, and these appeared to take ac-
count of more features of academic writing.
Limitations on the selection of topics for Task 2 and the choice of data for pre-
sentation in Task 1 may have attracted the test preparation teaching strategies ob-
served, including

• providing lists of topics

• encouraging learners to read about potential IELTS topics in the media
358 GREEN

• practice in planning and writing essays (but without incorporating source

material in ways consistent with academic writing)
• encouraging memorisation of formulaic phrases
• teaching relevant vocabulary and structures (such as—for AWC Task 1—
phrases for describing trends and reporting on information in graphs and dia-
grams or learning past rather than present forms of new verbs).

However, IELTS preparation and EAP courses also had much in common. EAP
classes incorporated concern with the discursive essay genre; the rhetoric of de-
scription and evaluation; and cause and effect, comparison, and problem and solu-
tion patterns of exposition.

Cognitive Demands
As observed by Brown (1998), Hayes and Read (2004), and Hawkey (2006),
IELTS classes were explicitly directed toward test success. In preparation classes,
there were frequent mentions of IELTS and of strategies for dealing with the tasks.
Most of the materials used were either taken from IELTS textbooks or chosen by
the teacher to reflect the test. Frequent essay writing practice involved test practice
under timed conditions and completing tasks closely modelled on IELTS (often as-
signed as homework). Learners were encouraged to become thoroughly familiar
with the test tasks.
Instruction in EAP courses included stages that did not feature in IELTS prepa-
ration. As these courses progressed, learners were often encouraged to go beyond
the classroom to locate material and to integrate this into their writing. Learners
were never observed to leave an IELTS preparation classroom to carry out other
activities, but EAP classes observed included one group visit to a library, and in an-
other, individuals left the classroom to carry out research work elsewhere. Atten-
tion was given in EAP classes to issues of plagiarism, the appropriate use of
sources, and the compilation of bibliographies.

Specification of Audience, Role, Tone, and Style

In IELTS classes, primary attention was given to the expectations of examiners,
rather than to university lecturers or educated nonspecialist readers: the audiences
specified at the time in the task instructions (since January 2006, audiences are no
longer specified in the IELTS task rubrics). Audience and the role of the writer
were not a major focus of instruction in the observed EAP classes, although one
did include a discussion of university teachers’ expectations of written work. Both
IELTS preparation and EAP classes gave time to formal language, but explicit
teaching of features of academic style (hedging) was observed only in EAP
classes.
 WATCHING FOR WASHBACK 359

Performance Conditions: Length and Time Allowed

As predicted, reflecting the test format, timed writing activities were more fre-
quent in IELTS preparation classes, but these were also observed in two EAP
classes. Some IELTS classes did, contrary to expectations, provide opportunities
for redrafting and error correction, though there would be limited opportunities for
these under test conditions. When questioned about this in a follow-up interview,
one teacher reported that he saw the relevance of these exercises to IELTS in the
awareness of essay structure they developed. This may be an instance of teacher
beliefs about language learning and skill building outweighing the direct influence
of the test format in guiding behaviour.
The length of the test tasks directly influenced the requirements set by teachers
for written work: Essays assigned in IELTS preparation classes were to be 150 or
250 words in length. Two IELTS teachers observed provided formulaic openings
to Task 1, and most included sentence expansion activities as methods of increas-
ing the number of words in a response. Teachers sometimes related such activities
directly to the IELTS length requirements.

Prompts and Transcription Mode

Writing task prompts assigned in IELTS classes employed similar wording to stan-
dard test tasks. In addition to producing full-length essays, IELTS preparation stu-
dents were given practice in writing essay plans based on a range of IELTS prompts.
Analyzing prompts for key words was another frequently observed activity.

Rating Scales and Criteria

The IELTS rating scales were updated in 2005. At the time of this study, these in-
cluded task fulfilment; coherence and cohesion (Task 1); arguments, ideas, and ev-
idence; communicative quality (Task 2); and vocabulary and sentence structure
(both tasks). The new scoring criteria are task achievement (Task 1); task response
(Task 2); coherence and cohesion; lexical resource; and grammatical range and ac-
curacy. Bridges and Shaw (2004) provided an overview of their revision.
Although the differences were not statistically significant, the frequency data
suggested the possibility of a greater focus on form in IELTS classes. Qualitative
data suggested that more attention was given to grammar exercises in IELTS
classes, compared with a stronger focus on discourse in EAP classes. This might
reflect the importance IELTS afforded to vocabulary and sentence structure, the
only criterion used to score both Task 1 and Task 2. However, attention to form
could also reflect assumptions about learning and the needs of students; teachers
might assume that learners could quickly improve their language skills through in-
struction focused on this area. It is also possible that IELTS learners made more
grammatical errors and that teachers were reacting to this with remedial activities.
 TABLE 9
Summary of Findings Related to Weigle’s (2002) Writing Task Characteristics

Dimension Observations

Subject matter Lists of typical IELTS topics were prepared for students on IELTS
preparation courses. IELTS topics were broad and were not explored
beyond task demands. Topics were sometimes sustained for longer in
EAP courses, and these included some attention to students’ academic
disciplines.
Stimulus IELTS-like task stimuli heavily used in IELTS courses. Focus on topics of
general interest. Students were encouraged to read newspapers or serious
magazines outside class to learn about relevant topics. This was not the
case in EAP courses.
Genre IELTS classes limited to writing only Task 1–and Task 2–type essays. EAP
courses involved wider range.
Rhetorical task Rhetorical tasks limited to those required for the test: predominantly
description for Task 1 and evaluation or hortation for Task 2. No
integration of academic sources (unlike EAP, where this was a focus).
Pattern of exposition IELTS classes limited to Task 1 and Task 2. EAP involved greater variety,
for example, classes on writing definitions
Cognitive demands IELTS classes involved reproducing information: from graph or from
personal knowledge. EAP classes included more writing from source
material.
Specification
Audience Some mention in IELTS classes of university lecturers (the specified
audience), but more attention given to examiner expectations, including
teaching of “ways to impress the examiner.”
Role IELTS classes did not focus on the role of the university student. This was
an occasional focus in EAP classes.
Tone, style IELTS preparation taught formal tone, but not including features of an
academic style such as hedging, which were included in EAP classes.
Length Essays for the IELTS classes were short (100–300 words). Writing
assignments were generally longer in EAP classes. IELTS students were
encouraged to count words. Teaching focus in some IELTS preparation
classes on how to use more words—sentence expansion.
Time allowed Timed essay practice activities were more frequent in IELTS classes.
However, redrafting of work and error correction were common practice
in both EAP and IELTS classes.
Prompt wording IELTS preparation involved question analysis based on the generic IELTS
task prompts.
Choice of prompts Students were sometimes given a choice of topic for IELTS practice essays,
but this was always limited to IELTS-like tasks.
Transcription mode Little word processing observed in IELTS writing (and none in class). Use
of IT was observed in EAP classes.
Scoring criteria Feedback on essays was mostly in the form of band scores in IELTS
preparation classes.
IELTS preparation involved
(a) Teaching of organisational templates for coherence, argument
structure
(b) Teaching grammar points relevant to test—error analysis/useful
structures
(c) Encouraging use of more formal and varied vocabulary

360
 WATCHING FOR WASHBACK 361

Postobservation interviews with teachers did not cast light on this as teachers did
not generally seem to be aware of an emphasis on grammar in IELTS classes.

RESULTS: POSTOBSERVATION TEACHER INTERVIEWS

Although the analysis of the frequency data revealed few differences between
classes in organisation, modality, or content, interviews with teachers indicated
that they approached the two class types very differently. Thirteen of the 14
IELTS classes for which interview data were available included mention of the
IELTS writing test as an aim of the class. The one IELTS preparation class said
not to be influenced by the test came in the first week of a 6-week course. It
emerged from the postobservation interview that the central activity (finding
supporting examples for popular proverbs from students’ countries) was in-
tended as a means of preparing students for the demands of Task 2 without di-
rectly introducing test-like questions. Thus, although the teacher maintained that
the test did not directly influence the class, the demands of Task 2 were the ulti-
mate goal.
Class aims in IELTS preparation courses included both practice in performing di-
rectly test-derived tasks (Task 1 or Task 2 writing practice) and a wide variety of
other activities intended to build test-relevant skills. These aims were categorised as
follows:

• gaining an overview of test demands

• building grammar and vocabulary related to test demands
• analyzing Task 2 questions
• learning about thesis statements, topic sentences, and paragraph structure
• supporting propositions with evidence
• selecting data in response to Task 1
• focussing on specific areas of difficulty through self- or peer correction
• understanding the IELTS assessment criteria.

Some of the aims for presessional EAP classes were similar to those reported
for IELTS:

• learning how to describe processes

• learning how to construct paragraphs
• learning about a problem and solution essay structure
• reviewing the tense system
• debating an issue.
362 GREEN

Others seemed to have no parallel among the IELTS classes:

• learning how to write definitions

• learning about hedging in academic writing
• distinguishing one’s own ideas from others’
• learning to integrate source material
• learning how to construct a bibliography.

Both IELTS and presessional EAP classes were said to follow a similar cycle,
with input from the teacher, practice writing tasks, and diagnostic feedback. From
the interview data it appeared to be the content of this cycle, not the process, that
differentiated presessional EAP from IELTS preparation classes. This reported
emphasis was borne out in the content of the activities and episodes observed. As
Brown (1998) found in the courses he observed, the IELTS preparation cycle was
closely tied to test content and practice of test tasks. Teachers said that the fre-
quency of test practice intensified as the courses progressed, and this was again
consistent with the observational data reported previously. In the presessional EAP
classes, teachers built toward longer writing tasks, with learners being given
greater independence (for research activities and library work) as the courses
neared completion. Where courses involved a final test (as with University F),
there might be some attention given to this, with students having opportunities for
test practice. However, in contrast to the IELTS preparation classes, which focused
on the test throughout, the teacher of the test familiarisation class observed at Uni-
versity F reported that this would be the one session (in week seven of the
eight-week course) to concentrate on the test.
In one combination course the teacher reported that as the test date approached,
IELTS preparation leaked across the curriculum, exerting an influence on the con-
tent of classes beyond the identified IELTS component, with students requesting
practice in test tasks during non- IELTS lessons. This was consistent with the ob-
served mention of IELTS in non-IELTS classes at College A and University E,
where learners requested more information about the AWC in nontest classes.
Teachers of IELTS courses claimed that writing class content was entirely dic-
tated by the AWC. Conversely, those of EAP courses either dismissed the idea that
IELTS had any influence on their classes or suggested that it served as a useful
baseline for their teaching; they could assume that learners arriving in their courses
with an IELTS score would have some knowledge of how to write a basic
five-paragraph essay.

Student Work
It was plain that the work collected from IELTS and EAP classes differed in the va-
riety of tasks completed by learners. Reflecting the focus reported by teachers and
seen in the classes observed, all written work collected from IELTS classes con-
 WATCHING FOR WASHBACK 363

sisted of responses to practice IELTS tasks (with varying degrees of guidance from
supporting materials). EAP tasks ranged from timed writing exercises on broad
topics of general interest based on personal knowledge or experience (similar to
IELTS Task 2) to extended projects on topics relating to students’ academic disci-
plines and including tables of contents, references, and bibliographies.
The task responses also differed in their presentation. All but two of the practice
IELTS tasks were handwritten, while the work collected from presessional EAP
courses, with the exception of work done in class under time constraints, was all
word processed. As anticipated, there was a much greater range in the length of
EAP task responses than of IELTS responses. IELTS tasks collected ranged in
length from 98 to 445 words (compared with the IELTS requirements of 150 and
250 words), while the presessional EAP essays ranged from 128 to 3,495 words. It
will be interesting to see whether the recent introduction of a computer-based
IELTS, which offers test takers the option of word processing their responses, will
lead to greater use of computers in IELTS preparation courses.
Of the six IELTS teachers, five marked student work using IELTS band scores,
with one providing a breakdown of the score by the criteria used on the test. The
five teachers giving scores often added a comment to the awarded score, such as
“good 6” or “5.0+.” Two of the three EAP teachers provided scores (one as marks
out of 20, the other as percentages), while the third made written comments but did
not give a score. One of the teachers giving scores used an analytical style of re-
porting that might have been influenced by IELTS. This used the criteria “content
and task achievement,” “organisation and coherence,” “range and accuracy of lan-
guage,” and (clearly beyond the scope of IELTS) “improvement between drafts.”

CONCLUSIONS

Although the primary concern of this article has been differences between IELTS
and EAP classes, it should be emphasised that, as the washback model would pre-
dict, there were found to be considerable areas of overlap. Organisation, content
control, and student modality were all very similar across classes. Differences in
language content were nonsignificant. Excluding the references to the test, many
of the activities in the IELTS classes observed might not have been out of place in
the EAP classes. Both class types involved brainstorming and planning, with fre-
quent practice in extensive writing. Although there was some variation, both en-
couraged a formal, objective style of writing; offered instruction in discourse-level
organisation; were concerned with the requirements implicit in task instructions;
and involved work (often in the form of remediation) on grammar and vocabulary.
There was also evidence that teacher variables (such as common beliefs about
the value of editing and redrafting) may encourage practices that cannot be pre-
dicted from test design. Given the lack of differences between class types in the
frequency data, it appears that many of the differences observed between classes
364 GREEN

might be linked rather to teacher or institutional variables, such as levels of profes-

sional training and beliefs about effective learning, than to the influence of the test.
In the evidence from qualitative observations and from examples of student work,
self-correction of essays emerged as another area of variation between teachers,
rather than between IELTS and EAP classes. Further research is needed to account
for the relative influence of tests, training, resources, and prior beliefs on teaching
methods. It seems clear, in this study as elsewhere, that tests exert a less direct in-
fluence on this aspect of teaching than on content.
Although there was evidence of common practice across course types, and al-
though courses were taught by teachers with varying levels of experience and atti-
tudes toward the test, IELTS preparation classes differed in consistent and salient
ways from EAP classes. As predicted by the washback model, differences between
classes could be traced to test design features, and test preparation focused nar-
rowly on these features could not be said to offer the same range of skills and
knowledge as EAP programs. The focus on the test in IELTS preparation classes
directed learners away from their academic subjects and toward the topics and text
types featured in the test. Writing in IELTS preparation classes was time con-
strained and brief, as it is in the test, while EAP learners also worked toward more
extensive and less speeded assignments.
This study may also point to some of the limitations of “watching for washback.”
Superficially at least, as reflected in the frequency data, the IELTS classes looked
very similar to the EAP classes. However, watching classes may not tell us enough
about how they are experienced: about which aspects of a class are attended to by the
learners, what they learn from them, and whether the attention given to the test is re-
warded with improvements in test scores. The teacher interviews showed that even if
the content often appeared similar, they approached the two class types in very dif-
ferent ways. This may have been equally true of the learners. There are also limita-
tions in the explanatory power of the observations. Interviews provided some oppor-
tunity to probe why teachers believed certain activities might be useful in preparing
for IELTS, but why do learners choose to study in IELTS courses? And why do
courses and course materials take the shapes seen here?
Although its effectiveness has not been demonstrated, among the teachers ob-
served, there was a consistent approach to preparing for the IELTS AWC that
centred on building relevant writing skills. The shared nature of the approach
and the relationship of this to the test design are indicative of washback. As the
skills required by the IELTS AWC are, for the most part, relevant to writing in
higher education, there was much in common between IELTS and other forms of
EAP instruction. However, the restrictions on the IELTS tasks outlined in the re-
view of the literature were also reflected in the narrower focus of
IELTS-directed classes. This suggests that learners will need to pass beyond
IELTS preparation if they are to be fully equipped with the language skills they
will need for academic study.
 WATCHING FOR WASHBACK 365

REFERENCES

Alderson, J. C., & Hamp-Lyons, L. (1996). TOEFL preparation courses: A study of washback. Lan-
guage Testing, 13, 280–297.
Alderson, J. C., & Wall, D. (1993). Does washback exist? Applied Linguistics, 14, 115–129.
Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2,
1–34.
Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing.
Language Testing, 13, 257–279.
Bailey, K. M. (1999). Washback in language testing. Princeton, NJ: ETS.
Banerjee, J. (1996). The design of the classroom observation instruments (Internal Report). Cambridge,
UK: University of Cambridge Local Examinations Syndicate.
Banerjee, J. (2000). Using English language screening tests in your institution. Paper presented at the
BALEAP Professional Issues Meeting, Trent University, Nottingham, UK.
Bridges, G., & Shaw, S. (2004). IELTS writing: Revising assessment criteria and scales. Research
Notes, 18, 8–12.
Brown, J. D. H. (1998). An investigation into approaches to IELTS preparation, with particular focus on
the Academic Writing component of the test. In S. Wood (Ed.), IELTS research reports (Vol. 1, pp.
20–37). Sydney: ELICOS/IELTS.
Brown, J. D. (2000). University entrance examinations: Strategies for creating positive washback on Eng-
lish language teaching in Japan. Shiken JALT Testing and Evaluation SIG Newsletter, 3(2), 4–8.
Burrows, C. (1998). Searching for washback: An investigation of the impact on teachers of the imple-
mentation into the adult migrant English program of the assessment of the certificates in spoken and
written English. Unpublished doctoral dissertation, Macquarie University, Sydney.
Burrows, C. (2004). Washback in classroom-based assessment: A study of the washback effect in the
Australian Adult Migrant English Program. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback
in language testing: Research contexts and methods (pp. 113–128). Mahwah, NJ: Lawrence
Erlbaum Associates, Inc.
Chalhoub-Deville, M., & Turner, C. E. (2000). What to look for in ESL admission tests: Cambridge cer-
tificate exams, IELTS, and TOEFL. System, 28, 523–539.
Chapman, D. W., & Snyder, C. W. (2000). Can high stakes testing improve instruction: Re-examining
conventional wisdom. International Journal of Educational Development, 20, 457–474.
Cheng, L. (2005). Studies in language testing: Vol. 21. Changing language teaching through language
testing: A washback study. Cambridge, UK: Cambridge University Press.
Coffin, C. (2004). Arguing about how the world is or how the world should be: The role of argument in
IELTS tests. Journal of English for Academic Purposes, 3, 229–246.
Deakin, G. (1997). IELTS in context: Issues in EAP for overseas students. EA Journal, 15, 7–28.
Douglas, D. (2000). Assessing languages for specific purposes. Cambridge, UK: Cambridge University
Press.
Ferman, I. (2004). The washback of an EFL national oral matriculation test to teaching and learning. In
L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Research contexts and
methods (pp. 191–210). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Green, A. (2003). Test impact and English for academic purposes: A comparative study in backwash
between IELTS preparation and university presessional courses. Unpublished doctoral dissertation,
University of Surrey, Roehampton, UK.
Hale, G., Taylor, C., Bridgeman, B., Carson, J., Kroll, B., & Kantor, R. (1996). A study of writing tasks
assigned in academic degree programs. Princeton, NJ: ETS.
Hamp-Lyons, L. (2005). The effect of the newly developed TOEFL format on EFL/ESL teaching and
learning in four countries. ILTA Online Newsletter, 1. Retrieved November 4, 2005, from http://
www.iltaonline.com
366 GREEN

Hawkey, R. (2006). Studies in language testing: Vol. 24. Impact theory and practice: Studies of the
IELTS test and Progetto Lingue 2000. Cambridge, UK: Cambridge University Press.
Hayes, B., & Read, J. (2004). IELTS test preparation in New Zealand: Preparing students for the IELTS
Academic Module. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.), Washback in language testing: Re-
search contexts and methods (pp. 97–112). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Heyneman, S. P., & Ransom, A. W. (1990). Using examinations and testing to improve educational
quality. Educational Policy, 4, 177–192.
Hughes, A. (1989). Testing for language teachers. Cambridge, UK: Cambridge University Press.
Hughes, A. (1993). Backwash and TOEFL 2000. Unpublished manuscript, University of Reading,
Reading, UK.
International English Language Testing System. (2005). The IELTS handbook. Cambridge, UK: Uni-
versity of Cambridge Local Examinations Syndicate.
Kellaghan, T., & Greaney, V. (1992). Using examinations to improve education: A study of fourteen Af-
rican countries. Washington, DC: The World Bank.
Messick, S. (1996). Validity and washback in language testing. Language Testing, 13, 241–256.
Mickan, P., & Slater, S. (2003). Text analysis and the assessment of academic writing. In R. Tulloh
(Ed.), IELTS research reports (Vol. 5, pp. 59–88). Canberra: IELTS Australia.
Moore, T., & Morton, J. (1999). Authenticity in the IELTS Academic Module Writing Test: A compara-
tive study of Task 2 items and university assignments. In R. Tulloh (Ed.), IELTS research reports
(Vol. 2, pp. 64–106). Canberra: IELTS Australia.
Qi, L. (2004). Has a high-stakes test produced the intended changes? In L. Cheng, Y. Watanabe, & A.
Curtis (Eds.), Washback in language testing: Research contexts and methods (pp. 171–190).
Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Read, J., & Hayes, B. (2003). The impact of the IELTS test on preparation for academic study in New
Zealand. In R. Tulloh (Ed.), IELTS research reports (Vol. 5, pp. 153–206). Canberra: IELTS Austra-
lia.
Read, J., & Hirsh, D. (2005). English language levels in tertiary institutions (Export Levy Programme
Project E4). Wellington: Education New Zealand.
Resnick, L. B., & Resnick, D. P. (1992). Assessing the thinking curriculum: New tools for educational
reform. In B. G. Gifford & M. C. O’Conner (Eds.), Changing assessments: Alternative views of apti-
tude, achievement and instruction (pp. 37–75). Boston: Kluwer Academic.
Saville, N. (2000). Investigating the impact of international language examinations. Research Notes, 2,
4–7.
Spada, N., & Fröhlich, M. (1995). COLT observation scheme: Coding conventions and applications.
Sydney: National Centre for English Language Teaching and Research, Macquarie University.
Thorp, D., & Kennedy, C. (2003). What makes a “good” IELTS answer in academic writing? (IELTS
Research Report). London: British Council.
Turner, C. (2001). The need for impact studies of L2 performance testing and rating: Identifying areas
of potential consequences at all levels of the testing cycle. In C. Elder, A. Brown, A. K. Hill,
N. Iwashita, T. Lumley, T. McNamara, & K. O’Loughlin (Eds.), Experimenting with uncertainty: Es-
says in honour of Alan Davies (pp. 138–149). Cambridge, UK: Cambridge University Press.
Turner, J. (2004). Language as academic purpose. Journal of English for Academic Purposes, 3,
95–109.
Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education and
from innovation theory. Language Testing, 13, 334–354.
Wall, D. (2005). Studies in language testing: Vol. 22. The impact of high-stakes testing on classroom
teaching: A case study using insights from testing and innovation theory. Cambridge, UK: Cam-
bridge University Press.
Wall, D., & Horák, T. (2004). TOEFL impact study: The influence of next generation TOEFL on prepa-
ration classes in Central and Eastern Europe. Language Testing Update, 36, 40–41.
 WATCHING FOR WASHBACK 367

Watanabe, Y. (1996). Does grammar translation come from the entrance examination? Preliminary
findings from classroom-based research. Language Testing, 13, 318–333.
Watanabe, Y. (2001). Does the university entrance examination motivate learners? A case study of
learner interviews. In A. Murakami (Ed.), Trans-equator exchanges: A collection of academic pa-
pers in honour of Professor David Ingram (pp. 100–110). Akita, Japan: Akita University.
Watanabe, Y. (2004). Methodology in washback studies. In L. Cheng, Y. Watanabe, & A. Curtis (Eds.),
Washback in language testing: Research contexts and methods (pp. 19–36). Mahwah, NJ: Lawrence
Erlbaum Associates, Inc.
Weigle, S. C. (2002). Assessing writing. Cambridge, UK: Cambridge University Press.
368