3 C7

117
Longman Handbooks for Language Teachers
J. B. Heaton
New Edition
Consutant editors! Jerem" Harmon and #e" $ingsbur"
London and New %ork
11
1
.ongman &rou' L($ Limited
ongman House) Burnt *i) Harow)
sse+ C*,-,JE) Engand
rid .ssociated Com'anies throughout the word.
ubished in the /nited 0tates of .merica
1 Longman (nc.) New %ork
Longman &rou' /$ Limited 1233
(t rights reser1ed4 no 'art of this 'ubication ma" be re'roduced) stored in a retrie1a s"stem) r
transmitted in an" form or b" an" means) eectronic) mechanica) 'hotoco'"ing) recording)
)5otherwise) without the 'rior written 'ermission of the 6ubishers.
7first 'ubished 1278
hird im'ression 122-
#(T(0H L(B#.#% C.T.L9&/(N& (N 6/BL(C.T(9N :.T.
eaton) J. B.
;riting Engish anguage tests. 5 New ed. 5 <Longman handbooks for anguage teachers=.
1. Engish anguage 5 0tud" and teaching 5 >oreign s'eakers
,. Engish anguage 5
.biit" testing
1.
Tite
?,3.,@?@-7A 6E11,3..,
.0BN -588,5--,375-
(B#.#% 9> C9N&#E00 C.T.L9&/(N& (N 6/BL(C.T(9N :.T.
eaton) J. B. <John Brian=
;riting Engish anguage tests.
<Longman handbooks for anguage teachers=
Bibiogra'h"! '.
(ncudes inde+.
1.
Engish anguage 5 0tud" and teaching 5 >oreign s'eakers.
,. Engish anguage 5
+aminations 5 .uthorshi'. 3. Engish anguage 5 .biit" testing. (. Tite. ((. 0eries.
11,3..,H32? 1233 ?,3.-7A 3758,73
45in Times #oman
5oduced b" Longman &rou' <>E= Ltd.
5inted in Hong $ong
iustrated b" :a1id 6arkins
.C$N9;LE:&E*ENT0
e are gratefu to the foowing for 'ermission to re'roduce co'"right materia!
he author) John Bright and the /ni1ersit" of Cambridge Loca E+aminations 0"ndicate for
n e+tract from his critiBue on s'ecimen e+amination Buestions4 Har'er C #ow 6ubishers
for a tabe from '. 1?- DE0L Com'osition 6rofie@ from Teaching E0L Com'ositi
on b"
ene B. Hughe". :eanna #. ;ormuth. E. >a"e Hartfied and Ho" L. Jacobs <Newbur"
5ouse= Co'"right F 1233 b" Newbur" House 6ubishers (nc4 the author) #osaind Hawkins)
Thief E+aminer for /CLE0 6reiminar" Engish Test and the /ni1ersit" of Cambridge Loca
+aminations 0"ndicate for e+tracts from sam'e test materias4 Hong $ong Education
Ge'artment for e+tracts from the Hong $ong Engish 0choo Certificate E+amination 12A3
nd the Hong $ong 0econdar" 0choos Entrance E+amination 12A34 Longman &rou' /$ Ltd
=r e+tracts from Com'osition Through 6ictures b" J. B. Heaton)
0tud"ing in Engish b" J. B.
caton and
;riting Through 6ictures b" J. B. Heaton4 The author) .nthon" Tucker for an
0treet from his artice in The &uardian 8th 0e'tember 12A24 and the foowing e+amination
oards for 'ermission to re'roduce Buestions from 'ast e+amination 'a'ers! Joint
Htatricuation Board4 North ;est #egiona E+aminations Board4 The #o"a 0ociet" of .rts
+aminations Board! /ni1ersit" of Cambridge Loca E+aminations 0"ndicate4 /ni1ersit" of
I=+ford :eegac" of Loca E+aminations and the .res E+aminations Trust.
o dents
1 (ntroduction to anguage testing 8
1. 1 Testing and teaching 8
1., ;h" testG A
1 3 ;hat shoud be tested and to
what standardG 7
1.? Testing the anguage skis 3
1.8 Testing anguage areas 2
1 A Language skis and anguage
eements 1-
1 7 #ecognition and 'roduction 11
1 3 6robems of sam'ing 1,
1 2 .1oiding tra's for the students 1? 8
, .''roaches to anguage testing 18
,.1 Background 18
,., The essa"5transation a''roach 18
,. 3 The structuraist a''roach 18
,.? The integrati1e a''roach 1A
,.8 The communicati1e a''roach 12
3 9bJecti1e testing ,8 A
3.1 0ubJecti1e and obJecti1e testing ,8
3., 9bJecti1e tests ,A
3.3 *uti'e5choice items! genera ,7
3.? *uti'e5choice items! the stemK
the correct o'tionKthe distractors 3-
3.8 ;riting the test 33
? Tests of grammar and usage 3?
?.1 (ntroduction 3?
?., *uti'e5choice grammar items! 7
item t"'es 3?
?.3 Constructing muti'e5choice items 37
?.? Constructing error5recognition
muti'e5choice items 32
?.8 Constructing rearrangement items ?1
?.A Constructing com'etion items ?,
?.7 Constructing transformation items ?A
?.3 Constructing items in1o1ing the
changing of words ?3
?.2 Constructing @broken sentence@ items ?2
?.1- Constructing 'airing and
matching items ?2
?.11 Constructing combination and
addition items 8-
Testing 1ocabuar" 81
8.1 0eection of items 81
8., *uti'e5choice items <.= 8,
8.3 *uti'e5choice items <B= 558A
8.? 0ets <associated words= 83
8.8 *atching items 83
8.A *ore obJecti1e items A-
8.7 Com'etion items A,
Listening com'rehension tests A?
A.1 &enera A?
A., 6honeme discrimination tests A8
A.3 Tests of stress and intonation A3
A.? 0tatements and diaogues A2
A.8 Testing com'rehension through
1isua materia4s 7G
A.A /nderstanding taks and
ectures 3,
9ra 'roduction tests 33
7.1 0ome difficuties in testing the
s'eaking skis 33
7., #eading aoud 32
7.3 Con1ersationa e+changes 2-
7 ? /sing 'ictures for assessing ora
'roduction
7 8 The ora inter1iew
7 A 0ome other techniBues for ora
e+amining
3 Testing reading com'rehension
3 1 The nature of the reading skis
3 , (nitia stages of reading!
matching tests
3.3 (ntermediate and ad1anced55555
stages of reading! matching tests
3.? TrueKfase reading tests
3.8 *uti'e5choice items <.=! short
te+ts
3.A *uti'e5choice items <B=!
onger te+ts
3.7 Com'etion items
3.3 #earrangement items
3.2 CoLe 'rocedure
3. 1- 9'en5ended and misceaneous
items
3.11 Cursor15reading I
2 Testing the writing skis
2.1 The writing skis
2., Testing com'osition writing
2.3 0etting the com'osition
2.? &rading the com'osition
2.8 Treatment of written errors
2.A 9bJecti1e tests! mechanics
2.7 9bJecti1e tests! st"e and
register
2.3 Controed writing
?
1- Criteria and t"'es of tests 182
2, 1-.1 Eaidit" 182
2A 1-., #eiabiit" 1A,
1-.3 #eiabiit" 1ersus 1aidit" 1A?
1-, 1-.? :iscrimination 1A8
1-.8 .dministration 1A7
1-8 1-.A Test instructions to the
1-8 candidate 1A3
1-.7 Backwash effects 17-
1-7 1-.3 T"'es of tests 171
11- 11 (nter'reting test scores 17?
113 11.1 >reBuenc" distribution 17?
11., *easures of centra tendenc" 178
11A 1 1 .3 *easures of dis'ersion 17A
11.? (tem ana"sis 173
117 11.8 *oderating 138
1,? 11.A (tem cards and banks 138
1,2
131 0eected bibiogra'h" 1 33
(nde+ 1 21
133
133
138
138
13A
133
1??
1?2
18-
18,
18?
(ntroduction to anguage testing
1.1 Testing and
. arge number of e+aminations in the 'ast ha1e encouraged a tendenc" ti
teaching se'arate testing from teaching. Both testing and teaching are so cose"
interreated that it is 1irtua" im'ossibe to work in either fied without
being constant" concerned with the other. Tests ma" be constructed
'rimari" as de1ices to reinforce earning and to moti1ate the student or
'rimari" as a means of assessing the student@s 'erformance in the
anguage. (n the former case) the test is geared to the teaching that has
taken 'ace) whereas in the atter case the teaching is often geared arge"
to the test. 0tandardised tests and 'ubic e+aminations) in fact) ma" e+ert
such a considerabe infuence on the a1erage teacher that the" are often
instrumenta in determining the kind of teaching that takes 'ace before
the test.
. anguage test which seeks to find out what candidates can do with
anguage 'ro1ides a focus for 'ur'osefu) e1er"da" communication
acti1ities. 0uch a test wi ha1e a more usefu effect on the earning of a
'articuar anguage than a mechanica test of structure. (n the 'ast e1en
good tests of grammar) transation or anguage mani'uation had a
negati1e and e1en harmfu effect on teaching. . good communicati1e test
of anguage) howe1er) shoud ha1e a much more 'ositi1e effect on earnin
and teaching and shoud genera" resut in im'ro1ed earning habits.
Com'are the effect of the foowing two t"'es of test items on the
teaching of Engish!
1
%ou wi now hear a short tak. Listen carefu" and com'ete the
foowing 'aragra'h b" writing one word on each ine!
(f "ou go to ... . on hoida") "ou ma" ha1e to wait a ong time
at the ... . as the 'orters are on ...
. Howe1er) it wi
not be as bad as at most ...
<etc.=
, %ou wi now hear a short weather and tra1e re'ort on the radio.
Beforr
"ou isten to the tak) choose one of the 'aces .) B or C and 'ut a cross
<M= in the bo+ ne+t to the 'ace "ou choose.
6ace .5 0outhern 0'ain <b" air=.
6ace B 5 Northern >rance <b" car=.

6ace C 5 0witLerand <b" rai=.
6ut crosses in the correct bo+es beow after istening to the 'rogramme.
#emember to concentrate on" on the information a''ro'riate to the 'ace
which "ou ha1e chosen.
No tra1e 'robems
. few tra1e 'robems
0erious tra1e 'robems
0unn"
>ine but coud"
#ain
&ood hotes
.1erage hotes
6oor hotes
<etc.=
>ortunate") a number of we5known 'ubic e+amining bodies now
attem't to measure the candidates@ success in 'erforming 'ur'osefu and
ree1ant tasks and their actua abiit" to communicate in the anguage. (n
this sense) such e+aminations undoubted" e+ert a far more beneficia
infuence on s"abuses and teaching strategies than in the 'ast. Howe1er)
e1en the best 'ubic e+aminations are sti 'rimari" instruments for
measuring each student@s 'erformance in com'arison with the 'erformance
of other students or with certain estabished norms.
1., ;h" testG The function indicated in the 'receding 'aragra'h 'ro1ides one of the
answers to the Buestion! ;h" testG But it must be em'hasised that the
e1auation of student 'erformance for 'ur'oses of com'arison or seection is
on" one of the functions of a test. >urthermore) as far as the 'ractising
teacher is concerned) it shoud rare" be either the soe 'ur'ose or e1en the
chief 'ur'ose of testing in schoos.
.though most teachers aso wish to e1auate indi1idua 'erformance)
the aim of the cassroom test is different from that of the e+terna
e+amination. ;hie the atter is genera" concerned with e1auation for
the 'ur'ose of seection) the cassroom test is concerned with e1auation
for the 'ur'ose of enabing teachers to increase their own effecti1eness b"
making adJustments in their teaching to enabe certain grou's of students
or indi1iduas in the cass to benefit more. Too man" teachers gear their
teaching towards an i5defined Da1erage@ grou' without taking into account
the abiities of those students in the cass who are at either end of the scae.
. good cassroom test wi aso he' to ocate the 'recise areas of
difficut" encountered b" the cass or b" the indi1idua student. Just as it is
necessar" for the doctor first to diagnose the 'atient@s iness) so it is
eBua" necessar" for the teacher to diagnose the student@s weaknesses and
difficuties. /ness the teacher is abe to identif" and ana"se the errors a
student makes in handing the target anguage) he or she wi be in no
'osition to render an" assistance at a through a''ro'riate antici'ation)
remedia work and additiona 'ractice.
A
1
The test shoud aso enabe the teacher to ascertain which 'arts of the
anguage 'rogramme ha1e been found difficut b" the cass. (n this wa")
the teacher can e1auate the effecti1eness of the s"abus as we as the
methods and materias he or she is using. The test resuts ma" indicate. fo
e+am'e) certain areas of the anguage s"abus which ha1e not taken
sufficient account of5foreign earner difficuties or which. for some reason.
ha1e been gossed o1er. (n such cases the teacher wi be concerned with
those 'robem areas encountered b" grou's of students rather than bN the
indi1idua student. (f) for e+am'e) one or two students in a cass of 3- or
?- confuse the 'resent 'erfect tense with the 'resent sim'e tense <e.g.
aread" see that fim@=) the teacher ma" sim'" wish to correct the error
before mo1ing on to a different area. Howe1er) if se1en or eight students
make this mistake) the teacher wi take this 'robem area into account
when 'anning remedia or further teaching.
. test which sets out to measure students@ 'erformances as fair" as
'ossibe without in an" wa" setting tra's for them can be effecti1e" used
to moti1ate them. . we5constructed cassroom test wi 'ro1ide the
students with an o''ortunit" to show their abiit" to 'erform certain task!
in the anguage. 6ro1ided that detais of their 'erformance are gi1en as
soon as 'ossibe after the test) the students shoud be abe to earn from
their weaknesses. (n this wa" a good test can be used as a 1auabe teachi
de1ice.
1.3
;hat shoud be
5The de1eo'ment of modern inguistic theor" has he'ed to make anguaO
tested and to what
teachers and testers aware of the im'ortance of ana"sing the anguage
standardG
being tested. *odern descri'ti1e5grammars <though not "et 'rimari"
intended for foreign anguage teaching 'ur'oses= are re'acing the oder
Latin5based 'rescri'ti1e grammars! inguists are e+amining the whoe
com'e+ s"stem of anguage skis and 'atterns of inguistic beha1iour.
(ndeed) anguage skis are so com'e+ and so cose" reated to the tota
conte+t in which the" are used as we as to man" non5inguistic skis
<gestures) e"e5mo1ements) etc.= hat it ma" often seem im'ossibe to
se'arate them for the 'ur'ose of an" kind of assessment. . 'erson awa.)
s'eaks and communicates in a 'articuar situation at a 'articuar time.
;ithout this kind of conte+t) anguage ma" ose much of its meaning.
Before a test is constructed) it is im'ortant to Buestion the standards
which are being set. ;hat standards shoud be demanded of earners of
foreign anguageG >or e+am'e) shoud foreign anguage earners after a
certain number of months or "ears be e+'ected to communicate with the
same ease and fuenc" as nati1e s'eakersG .re certain habits of second
anguage eaine%s5regar@d5ed asmistakes5when these5saute5habits5wou nn.
constitute mistakes when beonging to nati1e s'eakersG ;hat. indeed) is
Dcorrect@ EngishG
E+aminations in the written anguage ha1e in the 'ast set artificia
standards e1en for nati1e s'eakers and ha1e often demanded skis simi to
those acBuired b" the great Engish essa"ists and critics. (n imitating t
anguage e+aminations of written Engish) howe1er) second anguage
e+aminations ha1e 'ro1ed far more unreaistic in their e+'ectations of tt .
'erformances of foreign earners) who ha1e been reBuired to rewrite son of
the greatest iterar" master'ieces in their own words or to write origin! essa"s
in anguage be"ond their ca'acit".
1.? Testing the >our maJor skis in communicating through anguage are often broad"
anguage skis defined as istening) istening and s'eaking) reading and writing. (n man"
situations where Engish is taught for genera 'ur'oses) these skis shoud
be carefu" integrated and used to 'erform as man" genuine"
communicati1e tasks as 'ossibe. ;here this is the case) it is im'ortant for
the test writer to concentrate on those t"'es of test items which a''ear
direct" ree1ant to the abiit" to use anguage for rea5ife communication)
es'ecia" in ora interaction. Thus) Buestions which test the abiit" to
understand and res'ond a''ro'riate" to 'oite reBuests) ad1ice)
instructions) etc. woud be 'referred to tests of reading aoud or teing
stories. (n the written section of a test) Buestions reBuiring students to
write etters) memos) re'orts and messages woud be used in 'ace of man"
of the more traditiona com'ositions used in the 'ast. (n istening and
reading5tee5B stiatas5in5whic @maw their abiit"5to5e+tract
s'ecific information of a 'ractica nature woud be 'referred to Buestions
testing the com'rehension of unim'ortant and irree1ant detais. .bo1e
a) there woud be no rigid distinction drawn between the four different
skis as in most traditiona tests in the 'ast) a test of reading now being
used to 'ro1ide the basis for a reated test of writing or s'eaking.
0uccess in traditiona tests a too often sim'" demonstrates that the
student has been abe to 'erform we in the test he or she has taken 5 and
1er" itte ese. >or e+am'e) the traditiona reading com'rehension test
<often in1o1ing the com'rehension of meaningess and irree1ant bits of
information= measures a ski which is more cose" associated with
e+aminations and answering techniBues than with the abiit" to read or
scan in order to e+tract s'ecific information5for a 'articuar 'ur'ose. (n this
sense) the traditiona test ma" te us reati1e" itte about the student@s
genera fuenc" and abiit" to hande the target anguage) athough it ma"
gi1e some indication of the student@s schoastic abiit" in some of the skis
he or she needs as a student.
;a"s of assessing 'erformance in the four maJor skis ma" take the
form of tests of!
5 istening <auditor"= com'rehension) in which short utterances) diaogues)
taks and ectures are gi1en to the testees4
5 s'eaking abiit") usua" in the form of an inter1iew) a 'icture
descri'tion) roe 'a"5 and a 'robem5so1ing task in1o1ing 'air work or
grou' work4
5 reading com'rehension) in which Buestions are set to test the students@
abiit" to understand the gist of a te+t and to e+tract ke" information on
s'ecific 'oints in the te+t4 and
5 writing abiit") usua" in the form of etters) re'orts) memos) messages)
instructions) and accounts of 'ast e1ents) etc.
(t is the test constructor@s task to assess the reati1e im'ortance of
these skis at the 1arious e1es and to de1ise an accurate means of
measuring the student@s success in de1eo'ing these skis. 0e1era test
writers sti consider that their 'ur'ose can best be achie1ed if each
se'arate ski can be measured on its own. But it is usua" e+treme"
difficut to se'arate one ski from another) for the 1er" di1ision of the four
skis is an artificia one and the conce't itsef constitutes a 1ast o1er5
sim'ification of the issues in1o1ed in communication.
0
1.8 Testing anguage (n an attem't to isoate the anguage areas earnt) a considerabe number
areas of tests incude sections on!
5 grammar and usage)
5 1ocabuar" <concerned with word meanings) word formation and
coocations=4
5 'honoog" <concerned with 'honemes) stress and intonation=.
Tests of grammar and These tests measure students@ abiit" to recognise a''ro'riate grammatica
usage
forms and to mani'uate structures.
@.though it <1= ... . Buite warm now) <,= ... . wi change
ater toda". B" tomorrow morning) it <3= ... . much coder and
there ma" e1en be a itte snow ... <etc.=
<1= .. seems B. wi seem C. seemed :. had seemed
<,= .. weather B. the weather C. a weather :. some
weather
<3= .. is B. wi go to5be C. is going to be :. woud be <etc.=
Note that this 'articuar t"'e of Buestion is caed a muti'e5choice
item. The term muti'e5choice is used because the students are reBuired to
seect the correct answer from a choice of se1era answers. <9n" one
answer is norma" correct for each item.= The word item is used in
'reference to the word Buestion because the atter word s"tggests the
interrogati1e form4 man" test items are) in fact) written in the form of
statements.
Not a grammar tests) howe1er) need com'rise muti'e5choice items.
The foowing com'etion item iustrates Just one of se1era other t"'es of O
grammar items freBuent" used in tests!
.! . does Eictor Luo ... G
B! ( think his fat is on the outskirts of $uaa Lum'ur.
<etc.=
Tests of 1ocabuar" . test of 1ocabuar" measures students@ knowedge of the meaning of
certain words as we as the 'atterns and coocations in which the" occur.
0uch a test ma" test their acti1e 1ocabuar" <the words the" shoud be abe
to use in s'eaking and in writing= or their 'assi1e 1ocabuar" <the words
the" shoud be abe to recognise and understand when the" are istening to
someone or when the" are reading=. 9b1ious") in this kind of test the
method used to seect the 1ocabuar" items <7 sam'ing= is of the utmost
im'ortance. .
(n the foowing item students are instructed to circe the etter at the side
of the word I;hich best com'etes ttie Isentence.I I
:id "ou ...
. that book from the schoo ibrar"G
.. beg B. borrow C. hire
:. end
E. ask
(n another common t"'e of 1ocabuar" test students are gi1en a
'assage to read and reBuired to re'ace certain words isted at the end of the
'assage with their eBui1aents in the 'assage.
Tests of 'honoog" Test items designed to test 'honoog" might attem't to assess the foowing
sub5skis! abiit" to recognise and 'ronounce the significant sound
contrasts of a anguage) abiit" to recognise and use the stress 'atterns of a
2
anguage) and abiit" to hear and 'roduce the meod" or 'atterns of the
tunes of a anguage <i.e. the rise and fa of the 1oice=.
(n the foowing item) students are reBuired to indicate which of the
three sentences the" hear are the same!
0'oken!
Just ook at that arge shi' o1er there.
Just ook at that arge shee' o1er there.
Just ook at that arge shi' o1er there.
.though this item) which used to be 'o'uar in certain tests) is now 1er"
rare" incuded as a se'arate item in 'ubic e+aminations) it is sometimes
a''ro'riate for incusion in a cass 'rogress or achie1ement test at an
eementar" e1e. 0uccessfu 'erformance in this fied) howe1er) shoud not
5555berc5tar5?et5N5as necessari" indicating an5abiit" to s'eak. 5
1.A Lwaguage skis and (tems designed to test areas of grammar and 1ocabuar" wi be e+amined
anguage eements
in detai ater in the a''ro'riate cha'ters. The Buestion now 'osed is! to
what e+tent shoud we concentrate on testing students@ abiit" to hande
these eements of the anguage and to what e+tent shoud we concentrate
on testing the integrated skisG 9ur attitude towards this Buestion must
de'end on both the e1e and the 'ur'ose of the test. (f the students ha1e
been earning Engish for on" a reati1e" brief 'eriod) it is high" ike"
that we sha be chief" concerned with their abiit" to hande the anguage
eements correct". *oreo1er) if the aim of the test is to sam'e as wide a
f i ed as 'ossibe) a batter" of tests of the anguage eements wi be usefu
not on" in 'ro1iding a wide co1erage of this abiit" but aso in ocating
'articuar 'robem areas. Tests designed to assess master" of the anguage
eements enabe the test writer to determine e+act" what is being tested and to
're5test items.
Howe1er) at a e1es but the most eementar") it is genera"
ad1isabe to incude test items which measure the abiit" to communicate in
the target anguage. How im'ortant) for e+am'ie) is the abiit" to
discriminate between the 'honemes Ki!K and Kt
KG E1en if the" are confused
.b" a testee and he or she sa"s
Look at that shee' saiing sow" out of the
harbour) it is unike" that misunderstanding wi resut because the conte+t
'ro1ides other cues to the meaning. . anguages contain numerous so5
caed Dredundancies@ which he' to o1ercome 'robems of this nature.
>urthermore) no student can be described as being 'roficient in a
anguage sim'" because he or she is abe to discriminate between two
sounds or has mastered a number of structures of the anguage. 0uccessfu
communication in situations which simuate rea ife is the best test of
master" of a anguage. (t can thus be argued that fuenc" in Engish 5 a
'erson@s abiit" to e+'ress facts) ideas) feeings and attitudes cear" and
with ease) in s'eech or in writing) and the abiit" to understand what he or
she hears and reads 5 can best be measured b" tests which e1auate
'erformance in the anguage skis. Listening and reading com'rehension
tests) ora inter1iews and etter5writing assess 'erformance in those
anguage skis used in rea ife.
Too great a concentration on the testing of the anguage eements ma"
indeed ha1e a harmfu effect on the communicati1e teaching of the
anguage. There is aso at 'resent insufficient knowedge about the
weighting which ought to be gi1en to s'ecific anguage eements. How
im'ortant are artices) for e+am'e) in reation to 're'ositions or
1-
'ronounsG 0uch a Buestion cannot be answered unti we know wore abw5B the
degrees of im'ortance of the 1arious eements at different stages of earning
a anguage.
1.7 #ecognition and *ethods of testing the recognition of correct words and forms of ae
'roduction
often take the foowing form in tests!
Choose the correct answer and write .) B) C or :.
(@1e been standing here ... . haf an hour.
.. since B. during C. whie :. for
This muti'e5choice test item tests students@ abiit" to recognise t tc
correct form! this abiit" is ob1ious" not Buite the same as tGe +bm" t o
'roduce and use the correct form in rea5ife situations. Howe1er) this t"'e
of item has the ad1antage of being eas" to e+amine statistica".
(f the four choices were omitted) the item woud come coser to being a
test of 'roduction!
Com'ete each bank with the correct word.
(@1e been standing here ... . haf an hour.
0tudents woud then be reBuired to 'roduce the correct answer <7 for=. (n
man" cases) there woud on" be one 'ossibe correct answer) but
'roduction items do not awa"s guarantee that students wi dea with the
s'ecific matter the e+aminer had in mind <as most recognition items ?-=. (n
this 'articuar case the test item is not entire" satisfactor") for students are
com'ete" Justified in writing near"KamostKo1er in the bank. (t woud
not then test their abiit" to discriminate between for with 'eriods of time
<e.g. for haf an hour) for two "ears= and since with 'oints of time <e.g.
since ,.3-) since Christmas=.
The foowing e+am'es aso iustrate the difference between testing
recognition and testing 'roduction. (n the first) students are instructed to
choose the best re'" in List B for each sentence in List . and to write the
etter in the s'ace 'ro1ided. (n the second) the" ha1e to com'ete a
diaogue.
<i= List . List B
1. ;hat@s the forecast for
a0oon after unch) ( think.
tomorrowG ...
,. ;oud "ou ike to go swimmingG
b;e can take our
umbreas.
3. ;here sha we goG ...
c. afternoon.
?. >ine. ;hat time sha we set offG
d%es) that@s a good ism.
8. How ong sha we s'end thereG e(t@ be Buite Not.
A. ;hat sha we do if it rainsG ...
fHow about Cearwater
Ba"G
<ii=
;rite B@s 'art in the foowing diaogue.
1.
.@ ;hat@s the forecast for tomorrowG
a! (t@ be Buite hot.
,. .! ;oud "ou ike to go swimmingG
a!
11
3. .! ;here sha we goG
3!
<etc.=
. good anguage test ma" contain either recognition5t"'e items or)
'roduction5t"'e items) or a combination of both. Each t"'e has its
uniBue functions) and these wi be treated in dctai ater.
1.3 6robems of The actua Buestion of what is to be incuded in a test is often difficut
sam'ing sim'" because a master" of anguage skis is being assessed rather than
areas of knowedge <i.e. content= as in other subJects ike geogra'h")
'h"sics) etc. .though the construction of a anguage test at the end of the
f i rst or second "ear of earning Engish is reati1e" eas" if we are famiiar
with the s"abus5co1ered) the construction of a test at a fair" ad1anced
e1e where the s"abus is not cear" defined is much more difficut.
The onger the test) the more reiabe a measuring instrument it wi be
<athough ength) itsef) is no guarantee of a good test=. >ew students
woud want to s'end se1era hours being tested 5 and indeed this woud be
undesirabe both for the tester and the testees. But the construction of
short tests which function efficient" is often a difficut matter. 0am'ing
now becomes of 'aramount im'ortance. The test must co1er an adeBuate
and re'resentati1e section of those areas and skis it is desired to test.
(f a the students who take the test ha1e foowed the same earning
'rogramme) we can sim'" choose areas from this 'rogramme) seeking to
maintain a carefu baance between tense forms) 're'ositions) artices)
e+ica items) etc. .bo1e a) the kind of anguage to be tested woud be
the anguage used in the cassroom and in the students@ immediate
surroundings or the 5anguage reBuired for the schoo or the work for which
the student is being assessed.
(f the same mother5tongue is shared b" a the testees) the task of
sam'ing is made sight" easier e1en though the" ma" ha1e attended
different schoos or foowed different courses. The" wi a e+'erience
'robems of a simiar nature as a resut of the interference of their first5
anguage habits. (t is not a difficut matter to identif" these 'robem areas
and to incude a cross5section of them in the test) 'articuar" in those
sections of the test concerned with the anguage eements. The foowing
two e+am'es based on interference of first5anguage habits wi suffice at
this stage. The first e+am'e concerns the use of the 'resent sim'e for the
'resent 'erfect tense! man" students from certain anguage backgrounds
write such sentences as Tee1ision e+ists on" for the ast fort" or fift" "ears
instead of Tee1ision has e+isted on" for the ast fort" or fift" "ears.
. test
item based on this 'robem area might be!
;rite down .) B) C) : or E according to the best aternati1e needed to
com'ete the sentence.
Tee1ision ... . on" for the ast fift" "ears.
. e+ists :. e+isted
B. was e+isting
E. is e+isting
C. has e+isted
The second e+am'e has been taken from a test of 1ocabuar" and
concerns confusion in the use of
ook for4 it is directed chief" at .rabic
and Chinese earners of Engish. The word fetched has been incuded in the
ist of choices because there is no distinction in .rabic between the two
1,
conce'ts e+'ressed in Engish b" fetch and ook for) whie account has aso
been taken of the difficut" man" Chinese earners e+'erience as a resut of
the ack of distinction in *andarin between ook for and find. Choices :
and E might aso a''ear 'ausibe to other students unsure of the correct
use of ook for.
@Here@s "our book) John. %ou eft it on m" desk.@
@Thanks. (@1e ... . it e1er"where.@
.. ooked for :. attended to
3. fetched E. watched.o1er
C. found
(t must be em'hasised that items based on contrasti1e ana"sis can
on" be used effecti1e" when the students come from the same anguage
area. (f most of them do not share the same first anguage) the test must be
uni1ersa b" nature and sam'e a fair cross5section of the anguage. (t wi
scarce" matter then if students from certain anguage areas find it easier
than others! in actua anguage)earning situations the" ma" ha1e an
ad1antage sim'" because their first anguage ha''ens to be more cose"
reated to Engish than certain other anguages are. >ew woud wish to
den" that) gi1en the same anguage5earning conditions) >rench students
earning Engish wi e+'erience fewer difficuties than their Chinese
counter'arts.
Before starting to write an" test items) the test constructor shoud
draw u' a detaied tabe of s'ecifications showing as'ects of the skis being
tested and gi1ing a com'rehensi1e co1erage of the s'ecific anguage
eements to be incuded. . cassroom test shoud be cose" reated to the
ground co1ered in the cass teaching) an attem't being made to reate the
different areas co1ered in the test to the ength of time s'ent on teaching
those areas in cass. There is a constant danger of concentrating too much
on testing those areas and skis which most easi" end themse1es to being
tested. (t ma" be he'fu for the teacher to draw u' a rough in1entor" of
those areas <usua" grammatica features or functions and notions= which
he or she wishes to test) assigning to each one a 'ercentage according to
im'ortance. >or e+am'e) a teacher wishing to construct a test of grammar
might start b" e+amining the reati1e weighting to be gi1en to the 1arious
areas in the ight of the teaching that has Just taken 'ace! sa") the) contrast
between the 'ast continuous and 'ast sim'e tenses <?- 'er cent=) artices
<18 'er cent=) time 're'ositions <18 'er cent=) wish and
ho'e <1- 'er cent=)
concord <1- 'er cent=) the infiniti1e of 'ur'ose <1- 'er cent=.
.nother teacher wishing to ado't a more communicati1e a''roach to
anguage testing might consider the foowing s'ecifications in the ight of
the earning 'rogramme5greeting 'eo'e <85'er cent=)) int oducing onesef
<8 'er cent=) describing 'aces <18 'er cent=) taking about the future <,-
'er cent=) making suggestions <8 'er cent=) asking for information <,- 'er
cent=) understanding sim'e instructions <18 'er cent=) taking about 'ast
e1ents <18 'er cent=. <(t must be em'hasised that these ists are mere" two
e+am'es of the kinds of in1entories which can be drawn u' beforehand
and are not intended to re'resent a 'articuar set of 'riorities.= (n e1er"
case) it is im'ortant that a test refects the actua teaching and the course
being foowed. (n other words) if a more traditiona) structura a''roach
to anguage earning has been ado'ted) the test s'ecifications shoud
cose" refect such a structura a''roach. (f) on the other hand) a
communicati1e a''roach to anguage earning has been ado'ted) the test
13
i
g)B ee fr
'
Notes and nre s
1?
(
s'ecifications shoud be based on the t"'es of anguage tasks incuded in the
earning 'rogramme. (t is cear" unfair to administer a test de1ised entire"
aong communicati1e ines to those students who ha1e foowed a course
concentrating on the earning of structures and grammar.
. good test shoud ne1er be constructed in such a wa" as to tra' the students
into gi1ing an incorrect answer. ;hen techniBues of error ana"sis are used)
the setting of deiberate tra's or 'itfas for unwar" students shoud be
a1oided. *an" testers) themse1es) are caught out b"
constructing test items which succeed on" in tra''ing the more abe students.
Care shoud be taken to a1oid tra''ing students b" incuding grammatica and
1ocabuar" items which ha1e ne1er been taught.
(n the foowing e+am'e) students ha1e to seect the correct answer
<C=)5but5ti55P5whoe5itenris5constructed so as to tra' them into making55555
choice B or :. ;hen this item actua" a''eared in a test) it was found that
the more 'roficient students) in fact) chose B and :) as the" had
de1eo'ed the correct habit of associating the tense forms ha1e seen and ha1e
been seeing with since and for. The" had not been taught the com'ete 'attern
<as used in this sentence=. 0e1era of the ess 'roficient students) who had not
earnt to associate the 'erfect tense forms with since and for) chose the
Dcorrect@ answer.
;hen ( met Tim "esterda") it was the first time ( ... . him since
Christmas.
.. saw C. had seen
B. ha1e seen :. ha1e been seeing
0imiar") the foowing item tra''ed the more 'roficient students in
a
grou' b" encouraging them to consider the correct answer) Dsafet"@) as too sim'e to be
right. *an" of these students seected the res'onse
Dsaturation@ since the" knew 1ague" that this word was concerned with immersion in water.
The ess 'roficient students) on the other hand) sim'" chose Dsafet"@ without further
thought.
The animas tried to find ... . from the fire b" running into the
ake.
.. sanitation C. saturation
B. safet" :. sautation
To summarise) a tests shoud be constructed 'rimari" with the
intention of finding out what students know 5 not of tra''ing them. B"
attem'ting to construct effecti1e anguage tests) the teacher can gain a
dee'er insight into the anguage he or she
is testing and
the
anguage5
earning 'rocesses in1o1ed.
*uti'e5choice items of this nature ha1e ong been used in the /nited 0tates b"
such we5known testing organisations as T9E>L <Test of Engish as a >oreign
Language) Educationa Testing 0er1ice) 6rinceton) New Jerse"= and the
*ichigan Test of Engish Language 6roficienc" </ni1ersit" of *ichigan) .nn
.rbor) *ichigan= to test grammar and 1ocabuar". *uti'e5choice items ha1e
aso been wide" used in modern anguage testing in Britain and esewhere
throughout the word. #obert La5do <Language Testing) Longman 12A1) 12A?=
was one of the first to de1eo' the muti'e5choice techniBue in testing the s'oken
anguage.
.''roaches to anguage testing
,.1 Background Language tests can be rough" cassified according to four main a''roaches
to testing! <i= the essa"5transation a''roach4 <ii= the structuraist
a''roach4 <iii= the integrati1e a''roach4 and <i1= the communicati1e
a''roach. .though these a''roaches are isted here in chronoogica.
order) the" shoud not be regarded as being strict" confined to certain
'eriods in the de1eo'ment of anguage testing. Nor are the four
a''roaches awa"s mutua" e+cusi1e. . usefu test wi genera"
incor'orate features of se1era of these a''roaches. (ndeed) a test ma"
ha1e certain inherent weaknesses sim'" because it is imited to one
a''roach) howe1er attracti1e that a''roach ma" a''ear.
,., The essa"5
This a''roach is common" referred to as the 're5scientific stage of
transation a''roach anguage testing. No s'ecia ski or e+'ertise in testing is reBuired! the
subJecti1e Judgement of the teacher is considered to be of 'aramount
im'ortance. Tests usua" consist of essa" writing) transation) and
gramrr...tica ana"sis <often in the form of comments about the anguage
being earnt=. The tests aso ha1e a hea1" iterar" and cutura bias. 6ubic
e+aminations <e.g. secondar" schoo ea1ing e+aminations= resuting from
the essa"5transation a''roach sometimes ha1e an aura ora com'onent at
the u''er intermediate and ad1anced e1es 5 though this has sometimes
been regarded in the 'ast as something additiona and in no wa" an
integra 'art of the s"abus or e+amination.
,.3 The structuraist
This a''roach is characterised b" the 1iew that anguage earning is chief"
a''roach concerned with the s"stematic acBuisition of a set of habits. (t draws on the
work of structura inguistics) in 'articuar the im'ortance of contrasti1e
ana"sts and5f the need to i enh " an measure the earner@ s master i9f the
se'arate eements of the target anguage! 'honoog") 1ocabuar" and
grammar. 0uch master" is tested using words and sentences com'ete"
di1orced from an" conte+t on the grounds that a arger sam'e of anguage
forms can be co1ered in the test in a com'arati1e" short time. The skis of
istening) s'eaking) reading and writing are aso se'arated from one
another as much as 'ossibe because it is considered essentia to test orse
thing at a time.
0uch features of the structuraist a''roach are) of course) sti 1aid for
certain t"'es of test and for certain 'ur'oses. >or e+am'e) the desire to
concentrate on the testees@ abiit" to write b" attem'ting to se'arate a
18 5
>
,.? The (ntegrati1e
a''roach
1A
i
com'osition test from reading <i.e. b" making it who" inde'endent of the
abiit" to read ong and com'icated instructions or 1erba stimui= is
commendabe in certain res'ects. (ndeed) there are se1era features of this
a''roach which merit consideration when constructing an" good test.
The 's"chometric a''roach to measuretent with its em'hasis on
reiabiit" and obJecti1it" forms an integra 'art of structuraist testing.
6s"chometrists ha1e been abe to show cear" that such traditiona
e+aminations as essa" writing are high" subJecti1e and unreiabe. .s a resut)
the need for statistica measures of reiabiit" and 1aidit" is
considered to be of the utmost im'ortance in testing! hence the 'o'uarit"
of@the muti'e5choice item 5 a t"'e of item which ends itsef admirab" to
statistica ana"sis.
.t this 'oint) howe1er) the danger of confusing
methods of testing
with 5a''ruachesto5testing5shoud be5st"5ssed. The issue is not basica" a
Buestion of muti'e5choice testing 1ersus communicati1e testing. There is
sti a imited use for muti'e5choice items in man" communicati1e tests)
es'ecia" for reading and istening com'rehension 'ur'oses. E+act" the
same argument can be a''ied to the use of se1era othei item t"'es.
This a''roach in1o1es the testing of anguage in conte+t and is thus
concerned 'rimari" with meaning and the tota communicati1e effect of
discourse. ConseBuent") integrati1e tests do not seek to se'arate anguage
skis into neat di1isions in order to im'ro1e test reiabiit"! instead) the"
are often designed to assess the earner@s abiit" to use two or more skis
simutaneous". Thus) integrati1e tests are concerned with a goba 1iew of
'roficienc" 5 an under"ing anguage com'etence or Dgrammar of
e+'ectanc"I) which it is argued e1er" earner 'ossesses regardess of the
'ur'ose for which the anguage is being earnt. (ntegrati1e testing in1o1es
Dfunctiona anguage@, but not the use of functiona anguage. (ntegrati1e
tests are best characterised b" the use of doLe testing and of dictation.
9ra inter1iews) transation and essa" writing are aso incuded in man"
integrati1e tests 5 a 'oint freBuent" o1erooked b" those who take too
narrow a 1iew of integrati1e testing.
The 'rinci'e of doLe testing is based on the &estat theor" of
Dcosure@ <cosing ga's in 'atterns subconscious"=. Thus) doLe tests
measure the reader@s abiit" to decode odP @interru'ted@ or Dmutiated@ messages b" making the
most acce'tabe substitutions from a the conte+tua cues a1aiabe. E1er" nth word is
deeted in a te+t <usua" e1er" fifth) si+th or se1enth word=) and students ha1e to com'ete
each ga' in the te+t) using the most a''ro'riate word. The foowing is an e+tract from an
ad1anced5
e1e doLe 'assage in which e1er" se1enth word has been deeted!
The mark assigned to a student ... . surrounded b" an area of
uncertaint" ... . is the cumuati1e effect of a ... . of
sam'ing errors. 9ne sam'e of ... . student@s beha1iour is
e+hibited on one ... . occasion in res'onse to one
sam'e ... . set b" one sam'e of e+aminers ...
'ossib" marked b" one other. Each ... . the sam'ing errors is
amost insignificant ... . itsef.5Howe1er) when each sam'ing
error ... . added to the others) the tota ... . of 'ossibe
sam'ing errors becomes significant.
The te+t used for the doLe test shoud be ong enough to aow a
reasonabe number of deetions 5 idea" ?- or 8- banks. The more banks
contained in the te+t) the more reiabe the doLe test wi genera" 'ro1e.
There are two methods of scoring a doLe test! one mark ma" be
awarded for each acce'tabe answer or ese one mark ma" be awarded for
each er.icr answer. Both methods ha1e been found reiabe! some argue
that the former method is 1er" itte better than the atter and does not
rea" Justif" the additiona work entaied in defining what constitutes an
acce'tabe answer for each item. Ne1ertheess) it a''ears a fairer test for
the student if an" reasonabe eBui1aent is acce'ted. (n addition) no
student shoud be 'enaised for miss'eings uness a word is so bad" s'et
that it cannot be understood. &rammatica errors) howe1er) shoud be
'enaised in those doLe tests which are designed to measure famiiarit"
with the grammar of the anguage rather than reading.
;here 'ossibe) students shoud be reBuired to fi in each bank in the
te+t it)ef. This 'rocedure a''ro+imates more cose" to the rea5ife tasks
in1o1ed than an" method which reBuires them to write the deeted items
on a se'arate answer sheet or ist. (f the te+t chosen for a doLe test
contain) a ot of facts or if it concerns a 'articuar subJect) some students
ma" be abe to make the reBuired com'etions from their background
knowedge without understanding much of the te+t. ConseBuent") it is
essentia in doLe tests <as in other t"'es of reading tests= to draw u'on a
subJect which is neutra in both content and anguage 1ariet" used. >ina".
it is awa"s ad1antageous to 'ro1ide a Dead5in@! thus no deetions shoud he
made in the first few sentences so that the students ha1e a chance to
become famiiar with the author@s st"e and a''roach to the subJect of the
te+t.
CoLe 'rocedure as a measure of reading difficut" and reading
com'rehension wi be treated brief" in the ree1ant section of the cha'ter
on testing reading com'rehension. #esearch studies) howe1er) ha1e shown
that 'erformance on doLe tests correates high" with the istening) writing
and s'eaking abiities. (n other words) doLe testing is a good indicator of
genera inguistic abiit") incuding the abiit" to use anguage a''ro'riate"
according to 'articuar inguistic and situationa conte+ts. (t is argued that
three t"'es of knowedge are reBuired in order to 'erform successfu" on a
doLe test! inguistic knowedge) te+tua knowedge) and knowedge of the
word.@ .s it resut of such research findings) doLe tests are now used not
on" in genera achie1ement and 'roficienc" tests but aso in some
cassroom 'acement tests and diagnostic tests.
:ictation) another maJor t"'e of integrati1e test) was 're1ious"
regarded soe" as a means of measuring students@ skis of istening
OcoOm'rehension. Thus) the com'e+ eements in1o1ed in tests of dictation
were arge" o1erooked unti fair" recent". The inhegratte5a
is tn1oI5 Q1e
in tests of dictation incude auditor" discrimination) the auditor" memor"
s'an) s'eing) the recognition of sound segments) a famiiarit" with the
grammatica and e+ica 'atterning of the anguage) and o1era te+tua
com'rehension. /nfortunate") howe1er) there is no reiabe wa" of
assessing the reati1e im'ortance of the different abiities reBuired) and
each error in the dictation is usua" 'enaised in e+act" the same wa".
:ictation tests can 'ro1e good 'redictors of goba anguage abiit"
e1en though some recent research, has found that dictation tends to
measure ower5order anguage skis such as straightforward
17
2
com'rehension rather than the higher5order skis such as inference. The
dictation of onger 'ieces of discourse <i.e. 7 to 1- words at a time= is
recommended as being 'referabe to the dictation of shorter word grou's
<i.e. three to fi1e words at a time= as in the traditiona dictations of the 'ast.
/sed in this wa") dictation in1o1es a d"namic 'rocess of ana"sis b"
s"nthesis) drawing on a earner@s @grammar of e+'ectanc"I and resuting in
the constructi1e 'rocessing of the message heard.
(f there is no cose reationshi' between the sounds of a anguage and
the s"mbos re'resenting them) it ma" be 'ossibe to understand what is
being s'oken without being abe to write it down. Howe1er) in Engish)
where there is a fair" cose reationshi' between the sounds and the
s'eing s"stem) it is sometimes 'ossibe to recognise the indi1idua sound
eements without fu" understanding the meaning of what is s'oken.
(ndeed) some a''ied inguists and teachers argue tha) dictation
encourages the student to focus his or her attention too much on the
indi1idua sounds rather than on the meaning of the te+t as a whoe. 0uch
concentration on singe sound segments in itsef is sufficient to im'air the
auditor" memor" s'an) thus making it difficut for the students to retain
e1er"thing the" hear.
;hen dictation is gi1en) it is ad1isabe to read through the whoe
dictation 'assage at a''roaching norma con1ersationE ( s'eed first of a.
Ne+t) the teach5r shoud begin to dictate <either once or twice= in
meaningfu units of sufficient ength to chaenge the stude. is short5term
memor" s'an. <0ome teachers mistaken" fee that the" can make the
dictation easier b" reading out the te+t word b" word! this 'rocedure can
be e+treme" harmfu and on" ser1es to increase the difficut" of the
dictation b" obscuring the meaning of each 'hrase.= >ina") after the
dictation) the whoe 'assage is read once more at sight" sower than
norma s'eed.
The foowing is an e+am'e of 'art of a dictation 'assage) suitabe for
use at an intermediate or fair" ad1anced e1e. The obiBue strokes denote
the units which the e+aminer must obser1e when dictating.
Before the second haf of the nineteenth centur" K the taest bocks of
officesKwere on" three or four store"s high.KK.s business e+'andedK
and the need for office accommodation grew more and more acute)K
architects began to 'an taer buidings.KK;ood and iron) howe1er)K
were not strong enough materias from which to construct ta buidings.KK
>urthermore) the in1ention of stee now made it 'ossibe Kto construct
frames so strong Kthat the" woud su''ort the 1er" taest of buidings.
Two other t"'es of integrati1e tests <ora inter1iews and com'osition
writing= wi he treated at ength ater in this hook. The remaining t"'e of
integrati1e test not "et treated is transation. Tests of transation) howe1er)
tend to be unreiabe because of the com'e+ nature of the 1arious skis
in1o1ed and the methods of scoring. (n too man" instances) the unreaistic
e+'ectations of e+aminers resut in the setting of high" artificia sentences
and iterar" te+ts for transation. 0tudents are e+'ected to dis'a" an abiit"
to make fine s"ntactica Judgements and a''ro'riate e+ica distinctions 5
an abiit" which can on" be acBuired after achie1ing a high degree of
'roficienc" not on" in Engish and the mother5tongue but aso in
com'arati1e st"istics and transation methods.
;hen the tota skis of transation are tested) the test writer shoud
endea1our to 'resent a task which is meaningfu and ree1ant to the
situation of the students. Thus) for e+am'e) students might be reBuired to
write a re'ort in the mother5tongue based on information 'resented in
Engish. (n this case) the test writer shoud constant" be aert to the
com'e+ range of skis being tested. .bo1e a) word5for5word transation of
difficut iterar" e+tracts shoud be a1oided.
,.8 The The communicati1e a''roach to anguage testing is sometimes inked to
communicati1e the integrati1e a''roach. Howe1er) athough both a''roaches em'hasise
a''roach
the im'ortance of the meaning of utterances rather than their form and
structure) there are ne1ertheess fundamenta differences between the two
a''roaches. Communicati1e tests are concerned 'rimari" <if not tota"=
with how anguage is used in communication. ConseBuent") most aim to
incor'orate tasks which a''ro+imate as cose" as 'ossibe to those facing
the students in rea ife. 0uccess is Judged in terms of the effecti1eness of
the communication which takes 'ace rather than forma inguistic
accurac". Language @use@3 is often em'hasised to the e+cusion of anguage
Dusage@. D/se@ is concerned with how 'eo'e actua"
use anguage for a
mutitude of different 'ur'oses whie @usage@ concerns the forma 'atterns
of anguage <described in 'rescri'ti1e grammars and e+icons=. (n 'ractice.
howe1er) some tests of a communicati1e nature incude the testing of usage
and aso assess abiit" to hande the forma 'atterns of the target anguage.
(ndeed) few su''orters of the communicati1e a''roach woud argue that
communicati1e com'etence can e1er be achie1ed without a considerabe
master" of the grammar of a anguage.
The attem't to measure different anguage skis in communicati1e
tests is based on a 1iew of anguage referred to as the di1isibiit"
h"'othesis. Communicati1e testing resuts in an attem't to obtain different
'rofies of a earner@s 'erformance in the anguage. The earner ma". for
e+am'e) ha1e a 'oor abiit" in using the s'oken anguage in informa
con1ersations but ma" score Buite high" on tests of reading
com'rehension. (n this sense) communicati1e testing draws hea1i" on the
recent work on a'titude testing <where it has ong been caimed that the
most successfu tests are those which measure se'arate" such ree1ant
skis as the abiit" to transate news re'orts) the abiit" to understand radio
broadcasts) or the abiit" to inter'ret s'eech utterances=. The score
obtained on a communicati1e test wi thus resut in se1era measures of
'roficienc" rather than sim'" one o1era measure. (n the foowing tabe.
for e+am'e) the four basic skis are shown <each with si+ bo+es to indicate
the different e1es of students@ 'erformances=.
A 8 ? 3 , 1
I5L@iDstening
#eading
Listening and s'eaking
;riting
0uch a tabe woud norma" be ada'ted to gi1e different 'rofie)
ree1ant to s'ecific situations or needs. The degree of detai in the 1arious
'rofies isted wi de'end arge" on the t"'e of test and the 'ur'ose for
which it is being constructed. The foowing is an e+am'e of one wa" in
which the tabe coud be ada'ted.
@2
i
A 8 ? 3 ), 1
Listening to s'eciaist subJect ectures
#eading te+tbooks and Journas
Contributing to seminar discussions
;riting aborator" re'orts
;riting a thesis
>rom this a''roach) a new and interesting 1iew of assessment
emerges! name") that it is 'ossibe for a nati1e s'eaker to score ess than a
non5nati1e s'eaker on a test of Engish for 0'ecific 6ur'oses 5 sa") on a
stud" skis test of *edicine. (ris argued that a nati1e s'eaker@s a 4.iit"5to
use anguage for the 'articuar 'ur'ose being tested <e.g. Engish for
stud"ing *edicine= ma" actua" be inferior to a foreign earner@s abiit".
This is indeed a most contro1ersia caim as it might be Justifiab" argued
that ow scores on such a test are the resut of ack of moti1ation or of
knowedge of the subJect itsef rather than an inferior abiit" to use Engish
for the 'articuar 'ur'ose being tested.
/nike the se'arate testing of skis in the structuraist a''roach)
moreo1er) it is fet in communicati1e testing that sometimes the assessment
of anguage skis in isoation ma" ha1e on" a 1er" imited ree1ance to
rea ife. >or e+am'e) reading woud rare" be undertaken soe" for its
own sake in academic stud" but rather for subseBuent transfer of the
information obtained to writing or s'eaking.
0ince anguage is decoiite+tuaised in 's"chometric5structura tests) it
is often a sim'e matter for the same test to be used goba" for an"
countr" in the word. Communicati1e tests) on the other hand5) must of
necessit" refect the cuture of a 'articuar countr" because of their
em'hasis on conte+t and the use of authentic materias. Not on" shoud
test content he tota" ree1ant for a 'articuar grou' of testees but the
tasks set shoud reate to rea5ife situations. usua" s'ecific to a 'articuar
countr" or cuture. (n the ora com'onent of a certain test written in
Britain and triaed in Ja'an) for e+am'e) it was found that man" students
had e+'erienced. difficut" when the" were instructed to com'ain about
someone smoking. The reason for their difficut" was ob1ious! Ja'anese
'eo'e rare" com'ain) es'ecia" about something the" regard as a fair"
tri1ia matterR .though unintended) such cutura bias affects the
reiabiit" of the test being administered.
@6erha's the most im'ortant criterion for communicati1e tests is that
the" shoud be based on 'recise and detaied s'ecifications of the needs of
the earners for whom the" are constructed! hence their 'articuar
suitabiit" for the testing of Engish for s'ecific 'ur'oses. Howe1er) it
woud be a mistake to assume that communicati1e testing is best imited to
E06 or e1en to adut earners with 'articuar" ob1ious short5term goas.
.though the" ma" contain tota" different tasks. communicati1e tests for
"oung earners foowing genera Engish courses are based on e+act" the
same 'rinci'es as those for adut earners intending to enter on high"
s'ec aised courses of a 'rofessiona or academic nature.
>ina") communicati1e testing has introduced the conce't of
Buaitati1e modes of assessment in 'reference to Buantitati1e ones.
Language band s"stems are used to show the earners e1es of
,n
.'erformance in the different skis tested. :etaied statements of each
'erformance e1e ser1e to increase the reiabiit" of the scoring b"
enabing the e+aminer to make decisions according to carefu" drawn5u'
and we5estabished criteria. Howe1er) an eBua" im'ortant ad1antage of
such an a''roach ies in the more humanistic attitude it brings to anguage
testing. Each student@s 'erformance is e1auated according co his or her
degree of success in 'erforming the anguage tasks set rather than soe" in
reation to the 'erformances of other students. Suaitati1e Jugements are
aso su'erior to Buantitati1e assessments from another 'oint of 1iew. ;hen
'resented in the form of brief written descri'tions) the" are of considerabe use
in famiiarising testees and their teachers <or s'onsors= with much5
needed guidance concerning 'erformance and 'robem areas. *oreo1er) such
descri'tions are now reati1e" eas" for 'ubic e+amining bodies to 'roduce in
the form of com'uter 'rintouts.
The foowing contents of the 'reiminar" e1e of a we5known test
show how Buaitati1e modes of assessment) descri'tions of 'erformance
e1es4 etc. can he incor'orated in e+amination brochures and guides.8
;#(TTEN EN&L(0H
6a'er 1 5 .mong the items to be tested are! writing of formaKinforma
etters4 initiating etters and res'onding to them4 writing connected 'rose)
on to'ics ree1ant to an" candidate@s situation) in the form of messages)
notices) signs) 'ostcards) ists) etc.
6a'er , 5 .mong the items to be tested are! the use of a dictionar"4
abiit" to fi in forms4 abiit" to foow instructions) to read for the
genera meaning of a te+t) to read in order to seect s'ecific information.
069$EN EN&L(0H
0ection 1 5 0ocia Engish
Candidates must be abe to!
<a= #ead and write numbers) etters) and common abbre1iations.
<b= 6artici'ate in short and sim'e cued con1ersation) 'ossib" using
1isua stimui.
<c= #es'ond a''ro'riate" to e1er"da" situations described in 1er"
sim'e terms.
<d= .nswer Buestions in a directed situation.
0ection , 5 Com'rehension
Candidates must be abe to.
<a= /nderstand the e+act meaning of a sim'e 'Tce of s'eech) and
indicate this com'rehension b"!
5 marking a ma') 'an) or grid4
555555.557choosiagOthe5nost.a''ro.'riateOof.ae.G..")isuas.4)OOO
5 stating whether or not) or how) the aura stimuus reates to the
1isua4
5 answering sim'e Buestions.
<b= /nderstand the basic and essentia meaning of a 'iece of s'eech too
difficut to be understood com'ete".
0ection 3 5 E+tended 0'eaking
Candidates wi be reBuired to s'eak for ?85A- seconds in a situation or
situations ike" to be a''ro'riate in rea ife for a s'eaker at this e1e.
This ma" incude e+'anation) ad1ice) reBuests) a'oogies) etc. but wi
not demand an" use of the anguage in other than mundane and
,1
i
(
'ressing circumstances. (t is assumed at this e1e that no candidate
woud s'eak at ength in rea ife uness it were rea" necessar") so that)
for e+am'e) narrati1e woud not be e+'ected e+ce't in the conte+t of
something ike an e+'anation or a'oog".
.fter isting these contents) the test handbook then describes brief" what a
successfu candidate shoud be abe to do both in the written and s'oken
anguage.
The foowing s'ecifications and format are taken from another wide"
used communicati1e test of Engish and iustrate the o'erations) te+t t"'es
and formats which form the basis of the test. >or 'ur'oses of com'arison. the
e+am'es incuded here are confined to basic e1e tests of reading and
s'eaking. (t must be em'hasised) howe1er) that s'ecifications for a four
skis are incuded in the.S''ro'riate test handbook. together with other
ree1ant information for 'otentia testees.fi
TE0T0 9> #E.:(N&
9'erations 5 Basic Le1e
a. 0can te+t to ocate s'ecific information.
b. 0earch through te+t to decide whether the whoe or 'art is ree1ant to
an estabished need.
c. 0earch through te+t to estabish which 'art is ree1ant to an
estabished need.
rR. 0earch through te+t to e1auate the content in terms of 're1ious"
recei1ed information.
Te+t T"'es and To'ics 5 Basic Le1e
>orm T"'e
Leafet .nnouncement
.&uide :esc i'tion
.d1ertisement Narration
Letter Comment
6ostcard .necdoteKJoke
>orm #e'ortK0ummar"
0et of instructions
:iar" entr"
Timetabe
*a'K6an
>ormat
a. 9ne 'a'er of 1 hour. (n addition) candidates are aowed ten minutes
before the start of the e+amination to famiiarise themse1es with the
contents of the source materia. The Buestion 'a'er must not be
ooked at during this time.
b. Candidates wi be 'ro1ided with source materia in the form of
authentic bookets) brochures) etc. This materia ma" be the same at
a e1es.
c. Suestions wi be of the foowing forms!
i= *uti'e choice
ii= True >ase
iii = ;rite5in <singe word or 'hrase=
d. *onoingua or biingua dictionaries ma" be used free".
TE0T 9> 9#.L (NTE#.CT(9N
9'erations 5 Basic Le1e
E+'ressing! thanks
reBuirements
o'inions
comment
attitude
confirmation
a'oog"
wantKneed
information
Narrating!
seBuence of e1ents
Eiciting!
information
directions
ser1ice
<and a areas abo1e=
T"'es of Te+t
.t a e1es candidates ma" be e+'ected to take 'art in diaogue and
muti5'artici'ant interactions.
The interactions wi norma" be of a face5to5face nature but tee'hone
con1ersations are not e+cuded.
The candidate ma" be asked to take 'art in a simuation of an"
interaction deri1ed from the ist of genera areas of anguage use.
Howe1er) he wi not be asked to assume s'eciaised or fantas" roes.
>ormat
The format wi be the same at each e1e.
a. 5rests are di1ided into three 'arts. Each 'art is obser1ed b" an
assessor nominated b" the Board. The assessor e1auates and score
the candidate@s 'erformance but takes no 'art in the conduct of the
test.
b. 6art ( consists of an interaction between the candidate and an
interocutor who wi norma" be a re'resentati1e of the schoo or
centres where the test is hed and wi norma" be known to the
candidate. This interaction wi norma" be face5to5face but tee'ho5
formats are not e+cuded. Time a''ro+imate" 8 minutes.
c. 6art (( consists of an interaction between candidates in 'airs <or
e+ce'tiona" in threes or with one of the 'air a non5e+amination
candidate=. .gain this wi norma" be face5to5face but tee'hone
formats are not e+cuded. Time a''ro+imate" 8 minutes.
d. 6art ((( consists of a re'o
rt from the candidates to the interocutor
<who has been absent from the room= of the rnteraction rom 6art511
Time a''ro+imate" 8 minutes.
.s 'ointed out at the beginning of this cha'ter) a good test wi
freBuent" combine features of the communicati1e a''roach) the
integrati1e a''roach and e1en the structuraist a''roach 5 de'ending on the
'articuar 'ur'ose of the test and aso on the 1arious test constraints. (f) for
instance) the 'rimar" 'ur'ose of the test is for genera 'acement 'ur'oses
and there is 1er" itte time a1aiabe for its administration) it m. be
necessar" to administer sim'" a 8-5item coLe test.
1
Language testing constant" in1o1es making com'romises between
what is idea and what is 'racticabe in a certain situation. Ne1ertheess this
shoud not be used as an e+cuse for writing and administering 'oor tests!
whate1er the constraints of the situation) it is im'ortant to maintain
ideas and goas) constant" tr"ing to de1ise a test which is as 1aid and
reiabe as 'ossibe 5 and which has a usefu backwash effect on the
teaching and earning eading to the test.
Notes and references
( 9er) J ; 127, :ictation as a test of E0L 6roficienc". (n Teaching Engish as a
0econd Language! . Book of #eadings. *c&raw5Hi
, Cohen) . : 123- Testing Language .biit" in the Cassroom. Newbur" House
3 ;iddowson) H & 1273 Testing Language as Communication. 9+ford /ni1ersit"
6ress
? Carro)B T1273 .n Engish Qanguage testing ser1ice! s'ecifications.
The British
Counci
8 The 9+ford5.res E+aminations in Engish as a >oreign Language! #eguations
and 0"abuses
A #o"a 0ociet" of .rts! The Communicati1e /se of Engish
as a >oreign Language
<0'ecifications and >ormat=
,?
9bJecti1e testing
<with s'ecia reference to muti'e5choice techniBues=
3.1 0ubJecti1e and
0ubJecti1e and obJecti1e are terms used to refer to the scoring of tests. .
obJecti1e testing
test items) no matter how the" are de1ised) reBuire candidates to e+ercise
a subJecti1e Judgement. (n an essa" test) for e+am'e) candidates must
think of what to sa" and then e+'ress their ideas as we as 'ossibe4 in a
muti'e5choice test the" ha1e to weigh u' carefu" a the aternati1es and
seect the best one. >urthermore) a tests are constructed subJecti1e" b"
the tester) who decides which areas of anguage to test) how to test those
'articuar areas) and what kind of items to use for this 'ur'ose. Thus) it is
on" the scoring of a test that can be described as obJecti1e. This means
that a testee wi score the same mark no matter which e+aminer marks the
test.
0ince obJecti1e tests usua" ha1e on" one correct answer <or) at east. a
imited number of correct answers=) the" can be scored mechanica".
The fact that obJecti1e tests can be marked b" com'uter is one im'ortant
reason for their e1ident 'o'uarit" among e+amining bodies res'onsibe for
testing arge numbers of candidates.
9bJecti1e tests need not be confined to an" one 'articuar ski or
eement. (n one or two we5known tests in the 'ast) attem'ts ha1e e1en
been made to measure writing abiit" b" a series of obJecti1e test items.
Howe1er) certain skis and areas of anguage ma" be tested far more
effecti1e" b" one method than b" another. #eading and 1ocabuar") for
e+am'e) often end themse1es to obJecti1e methods of assessment.
Cear") the abiit" to.write can on" be satisfactori" tested b" a subJecti1e
e+amination reBuiring the student to 'erform.a writing task simiar to that
reBuired in rea ife. . test of ora fuenc" might 'resent students with the
foowing stimuus!
%5ouwenrto5tiEe inICairo tIwo "earsDagDo.50omeoneaskU "oiiOho11ong"ou
ha1e i1ed there. ;hat woud "ou sa"G
This item is arge" subJecti1e since the res'onse ma" be whate1er students
wish to sa". 0ome answers wi be better than others) thus 'erha's causing a
'robem in the scoring of the item. How) for instance) ought each of the
foowing answers to5be markedG
.N0;E# 7! (@1e been i1ing in Cairo since 123A.
.N0;E# ,! ( didn@t ea1e Cairo since 123A.
.N0;E# 3! ( ha1e i1ed in the Cairo Cit" for abo1e two "ears.
.N0;E# ?! >rom 123A.
,8
(T
.N0;E# s! ( came to i1e here before 123A and ( sti i1e here.
.N0;E# A! 0ince 123A m" home is in Cairo.
.though the task itsef attem'ts to simuate to some degree the t"'e
of task students might ha1e to 'erform in rea ife) it is more difficut to
achie1e reiabiit" sim'" because there are so man" different degrees of
acce'tabiit" and wa"s of scoring a the 'ossibe res'onses. Carefu
guideines must be drawn u' to achie1e consistenc" in the treatment of the
1ariet" of res'onses which wi resut.
9n the other hand) reiabiit" wi not be difficut to achie1e in the
marking of the foowing obJecti1e item. The Buestion of how 1aid such an
item is. howe1er. ma" now be of considerabe concern. How far do items ike
this refect the rea use of anguage in e1er"da" ifeG
Com'ete the sentences b" 'utting the best word in each bank. @(s
"our home sti in CairoG@
@%es) (@1e been i1ing here ...
123A.@
.. for
B. on C. in
:. at E. since
Language sim'" does not function in this wa" in rea5ife situations.
ConseBuent") the ast item tests grammar rather than communication! it is
concerned with students@ knowedge of forms of anguage and how
anguage works rather than with their abiit" to res'ond a''ro'riate" to
rea Bu@stions.
9n the whoe. obJecti1e tests reBuire far more carefu 're'aration
than subJecti1e tests. E+aminers tend to s'end a reati1e" short time on
setting the Buestions but considerabe time on marking. (n an obJecti1e test
the tester s'ends a great dea of time constructing each test item as
carefu" as 'ossibe. attem'ting to antici'ate the 1arious reactions of the
testees at each stage. The effort is rewarded) howe1er) in the ease of the
3., 9bJecti1e tests 9bJecti1e tests are freBuent" criticised on the grounds that the" are
sim'er to answer than subJecti1e tests. (tems in an obJecti1e test)
howe1er) can be made Just as eas" or as difficut as the test constructor
wishes. The fact that obJecti1e tests ma" genera"
ook easier is no
indication at a that the" are easier. The constructor of a standardised
achie1ement or 'roficienc" test not on" seects and constructs the items
carefu" but ana"ses student 'erformance on each item and rewrites the
items where necessar" so that the fina 1ersion of his or her test
discriminates wide". 0etting the 'ass5mark) or the cutting5off 'oint. ma"
de'end on the testers subJecti1e Judgement or on a 'articuar e+terna
situation. 9bJecti1e tests <and) to a smaer degree) subJecti1e tests= can he
're5tested before being administered on a wider basis! i.e. the" are gi1en
to a sma but tru" re'resentati1e sam'e of the test 'o'uation and then
each item is e1auated in the ight of the testees@ 'erformance. This
'rocedure enabes the test constructor to cacuate the a''ro+imate degree
of difficut" of the test. 0tandards ma" then be com'ared not on" betN1een
students from different areas or schoos but aso between students taking
the test in different "ears.
.nother criticism is that obJecti1e tests of the muti'e5choice t"'e
encourage guessing. Howe1er) four or.fi1e aternati1es for each item are
sufficient to reduce the 'ossibiit" of guessing. >urthermore) e+'erience
Or)
shows that candidates rare" make wid guesses! most base their guesses on
'artia knowedge.
. much wider sam'e of grammar) 1ocabuar" and 'honoog" can
genera" be incuded in an obJecti1e test than in a subJecti1e test.
.though the 'ur'osi1e use of anguage is often sacrificed in an attem't to
test students@ abiit" to mani'uate anguage) there are occasions
<'articuar" in cass 'rogress tests at certain e1es= when good obJecti1e
tests of grammar) 1ocabuar" and 'honoog" ma" be usefu 5 'ro1ided that
such tests are ne1er regarded as measures of the students@ abiit" to
communicate in the anguage. (t cannot be em'hasised too strong")
howe1er) that test obJecti1it" b" itsef 'ro1ides no guarantee that a test is
sound and reiabe. .n obJecti1e test wi be a 1er" 'oor test if!
5 the test items are 'oor" written4
5 irree1ant areas and skis are em'hasised in the test sim'" because the"
are @testabe@! and
5 it is confined to anguage5based usage and negects the communicati1e
skis in1o1ed.
(t shoud ne1er be caimed that obJecti1e tests can do those tasks
which the" are not intended to do. .s aread" indicated) the" can ne1er
test the abiit" to communicate in the target anguage) nor can the"
e1auate actua 'erformance. . good cassroom test wi usua" contain
both subJecti1e and obJecti1e test items.
3.3 *uti'e5choice
(t is usefu at this stage to consider muti'e5choice items in some detai) as
items! genera
the" are undoubted" one of the most wide" used t"'es of items in
obJecti1e tests. Howe1er) it must be em'hasised at the outset that the
usefuness of this t"'e of item is imited. /nfortunate") muti'e5choice
testing has 'roiferated as a resut of attem'ts to use muti'e5choice items to
'erform tasks for D hich the" were ne1er intended *oreo1er) since the
muti'e5choice item is one of the most difficut and time5consuming t"'es of
items to construct) numerous 'oor muti'e5choice tests now abound.
(ndeed) the ength of time reBuired to construct good muti'e5choice items
coud often ha1e been better s'ent b" teachers on other more usefu tasks
connected with teaching or testing.
The chief criticism of the muti'e5choice item) howe1er) is that
freBuent" it does not end itsef to the testing of anguage as
communication. The 'rocess in1o1ed in the actua seection of one out of
four or fi1e o'tions bears itte reation to the wa" anguage is used in most
rea5ife situations. .''ro'riate res'onses to 1arious stimui in e1er"da"
situations are 'roduced rather than chosen from se1era o'tions.
Ne"etthe5tess)Omutti'eOchoiceOitemsOcan5'ra1ide aOusefu means of
teaching and testing in 1arious earning situations <'articuar" at the ower
e1es= 'ro1ided that it is awa"s recognised that such items test
knowedge
of grammar) 1ocabuar") etc. rather than the abiit" to use anguage.
.though the" rare" measure communication as such) the" can 'ro1e
usefu in measuring students@ abiit" to recognise correct grammatica
forms) etc. and to make im'ortant discriminations in the target anguage.
(n doing this) muti'e5choice items can he' both student and teacher to
identif" areas of difficut".
>urthermore) muti'e5choice items offer a usefu introduction to the
construction of obJecti1e tests. 9n" through an a''reciation and master"
of the techniBues of muti'e5choice item wnting is the woud5be test
,7
(
(
constructor fu" abe to recognise the imitations im'osed b" such items
and then em'o" other more a''ro'riate techniBues of testing for certain
'ur'oses.
The o'timum number of aternati1es) or o'tions) for each muti'e5
choice item is fi1e in most 'ubic tests. .though a arger number) sa"
se1en) woud reduce e1en further the eement of chance) it is e+treme"
difficut and often im'ossibe to construct as man" as se1en good o'tions.
(ndeed) since it is often 1er" difficut to construct items with e1en fi1e
o'tions) four o'tions are recommended for most cassroom tests. *an"
writers recommend using four o'tions for grammar items) but fi1e for
1ocabuar" and reading.
Before constructing an" test items) the test writer must first determine
the actua areas to be co1ered b" muti'e5choice items and the number of
items to be5incuded in the55test5The test must be ong enough to aow for
a reiabe assessment of a testee@s 'erformance and short enough to be
'racticabe. Too ong a test is undesirabe because of the administration
difficuties often created and because of the menta strain and tension
which ma" be caused among the students taking the test. The number of
items incuded in a test wi 1ar" according to the e1e of difficut") the
nature of the areas being tested) and the 'ur'ose of the test. The teacher@s
own e+'erience wi genera" determine the ength of a test for cassroom
use) whie the ength of a 'ubic test wi be affected b" 1arious factors) not
east of which wi be its reiabiit" measured statistica" from the resuts of
the tria test.
Note that conte+t is of the utmost im'ortance in a tests.
:econte+tuaised muti'e5choice items can do considerabe harm b"
con1e"ing the im'ression that anguage can be earnt and used free of an"
conte+t. Both inguistic conte+t and situationa conte+t are essentia in
using anguage. (soated sentences in a muti'e5choice test sim'" add to
the artificiait" of the test situation and gi1e rise to ambiguit" and
confusion. .n awareness of the use of anguage in an a''ro'riate and
meaningfu wa" 5 so essentia a 'art of an" kind of communication 5 then
becomes irree1ant in the test. ConseBuent") it is im'ortant to remember
that the foowing muti'e5choice items are 'resented out of conte+t here
sim'" in order to sa1e s'ace and to draw attention to the saient 'oints
being made.
The initia 'art of each muti'e5choice item is known as the
stem4 the
choices from which the students seect their answers are referred to as
o'tionsKres'onsesKaternati1es.
9ne o'tion is the answer) correct o'tion or
ke")
whiA the other o'tions are
distractors. The task of a distractor is to distract the
maJorit" of 'oor students <i.e. those who do not know the
answer= from the correct o'tion.
0ta" here unti *r 0hort ... . "ou to come. 7 stem
.. tod
B. wi te
C. is teing
:. tes
o'tionsK
7 res'onsesK
aternati1es
7 distractors
7 answerKcorrect o'tionKke"
The foowing genera 'rinci'es shoud be obser1ed when
muti'echoice items are constructed!
1 Each muti'e5choice item shoud ha1e on"
one answer. This answer
must be absoute" correct uness the instruction s'ecifies choosing the best
o'tion <as in some 1ocabuar" tests=. .though this ma" seem an eas"
matter) it is sometimes e+treme" difficut to construct an item ha1ing on"
one correct answer. .n e+am'e of an item with two answers is!
@( sta"ed there unti John ...
.. had come
C. came
B. woud come
:. has come
, 9n" one feature at a time shoud be tested! it is usua" ess confusing
for the testees and it he's to reinforce a 'articuar teaching 'oint.
9b1ious") few woud wish to test both grammar and 1ocabuar" at the same
time) but sometimes word order and seBuence of tenses are tested
simutaneous". 0uch items are caed
im'ure items!
(
ne1er knew where ...
.. had the bo"s gone
C. ha1e the bo"s gone
B. the bo"s ha1e gone :. the bo"s had gone
<Note that it ma" sometimes be necessar" to construct such im'ure items at
the 1er" eementar" e1es because of the se1ere" imited number of
distractors genera" a1aiabe.=
3 Each o'tion shoud be grammatica" correct when 'aced in the stem)
e+ce't of course in the case of s'ecific grammar test items. >or e+am'e)
stems ending with the determiner a) foowed b" o'tions in the form of nouns
or noun 'hrases) sometimes tra' the unwar" test constructor. (n the item
beow) the correct answer C) when mo1ed u' to com'ete the stem)makes the
sentence grammatica" incorrect!
0omeone who designs houses is a .. O ...
.. designer B. buider
C. architect :. 'umber
The item can be easi" recast as foows!
0omeone who designs houses is ...
.. a designer B. a buider C. an architect
:. a 'umber
0tems ending in are) were) etc. ma" ha1e the same weaknesses as the
foowing and wi reBuire com'ete rewriting!
The bo"@s hobbies referred to in the first 'aragra'h of the 'assage were
.. cam'ing and fishing
B. tennis and gof
C. c"cing ong distances
:. fishing) rowing and swimming
E. coecting stam's
.n" fair" inteigent student woud soon be aware that o'tions C and E
were ob1ious" not in the tester@s mind when first constructing the item
abo1e because the" are ungrammatica answers. 0uch a student woud)
therefore) reaise that the" had been added ater sim'" as distractors.
0tems ending in 're'ositions ma" aso create certain difficuties. (n the
foowing reading com'rehension item) o'tion C can be rued out
immediate"!
John soon returned to ...
.. work B. the 'rison
C. home :. schoo
,2
? . muti'e5choice items shoud be at a e1e a''ro'riate to the 5
'roficienc" e1e of the testees. The conte+t) itsef) shoud be at a ower
e1e than the actua 'robem which the item is testing! a grammar test item
shoud not contain other grammatica features as difficut as the area being
tested) and a 1ocabuar" item shoud not contain more difficut semantic
features in the stem than the area being tested.
8 *uti'e5choice items shoud be as brief and as cear as 'ossibe <though it
is desirabe to 'ro1ide short conte+ts for grammar items=.
A (n man" tests) items are arranged in rough order of increasing difficut". (t
is genera" considered im'ortant to ha1e one or two sim'e items to Dead in@
the testees) es'ecia" if the" are not too famiiar with the kind of test being
administered. Ne1ertheess) areas of anguage which are tri1ia and not worth
testing. shoud.beOe+cuded from the5test.
3.? *uti'e5choice
items! the stemKthe
correct o'tionKthe
distractors
The stem
9
( The 'rimar" 'ur'ose of the stem is to 'resent the 'robem cear" and
concise". The testee shoud be abe to obtain from the stem a 1er" genera
idea of the 'robem and the answer reBuired. .t the same time) the stem
shoud not contain e+traneous information or irree1ant cues) thereb"
confusing the 'robem being tested. /ness students understand the 'robem
being tested) there is no wa" of knowing whether or not the" coud ha1e
handed the 'robem correct". .though the stem shoud be short) it shoud
con1e" enough information to indicate the basis on which the correct o'tion
shoud be seected.
, The stem ma" take the foowing forms!
<a= an incom'ete statement
He accused me of ... ies. 5 O
.. s'eaking B. sa"ing C. teing :. taking
<b= a com'ete statement
E1er"thing we wanted was to hand.
.. under contro C. we cared for
B. within reach :. being 're'ared
<c= a Buestion
.ccording to the writer) what did Tom immediate" doG
.. He ran home. C. He began to shout.
B. He met Bob. :. He 'honed the 'oice.
3 The stem shoud usua" contain those words or 'hrases which *oud
otherwise ha1e to he re'eated in each o'tion.
The word @astronauts@ is used in the 'assage to refer to
.. tra1eers in an ocean iner
B. tra1eers in a s'ace5shi'
C. tra1eers in a submarine
:. tra1eers in a baoon
The stern here shoud be rewritten so that it reads!
The word @astronauts@ is used in the 'assage to refer to tra1eers in
.. an ocean iner C. a submarine
B. a s'ace5shi' :. a baoon
The same 'rinci'e a''ies to grammar items. The foowing item!
( enJo" ... . the chidren 'a"ing in the 'ark.
.. ooking to C. ooking at
B. ooking about :. ooking on
shoud be rewritten in this wa"!
e?o" ooking ...
. the chidren 'a"ing in the 'ark.
to B. about C. at :. on
(f) howe1er) one of the errors made b" students in their free written work
has been the conission of the 're'osition aft4 r
ook <a common error=) then
it wi be necessar" to incude
ook
in the o'tions.
enJo" ... . the chidren 'a"ing in the 'ark.
.. ooking on C. ooking at
B. ooking :. ooking to
? The stem shoud aow for the @number of choices which ha1e been
decided u'on. This is 'articuar" ree1ant) for e+am'e) when com'arisons
are in1o1ed in reading com'rehension. There is no 'ossibe fourth o'tion
which can be added in the foowing item!
Tom was ... . the other two bo"s.
.. taer than
B. smaer than
5 C. as ta as
The correct >or norma 'ur'oses of testing) this shoud be cear" the
correct or best
o'tion! thus) it is most im'ortant that each item shoud be checked b"
another 'erson.
(t can be argued that a greater degree of subtet" is sometimes gained
b" ha1ing more than one correct o'tion in each item. The correct answers
in the foowing reading com'rehension and grammar items are circed!
.ccording to the writer) Jane wanted a new racBuet because .
her od one was damaged sight"
E she had ost her od one
C. her father had gi1en her some mone" for one
9 *ar" had a new racBuet
E. .nn often borrowed her od racBuet
;ho ... . "ou c"ce here to see usG
.. or5dere 55 B!55caused 5 C9 made : asked 9 et
(t is 1er" im'ortant) howe1er) to a1oid confusing the students b" ha1ing a
different number of correct o'tions for each item) and this 'ractice is
not
recommended. Each of the two muti'e5choice test items abo1e actua"
com'rises a grou' of trueKfase <i.e. rightKwrong= items and) therefore)
each aternati1e shoud be marked in this wa"! e.g. in the first item) the
testee scores ( mark for circing .) 1 mark for not circing B) 1 mark for
not circing C) ( mark for circing :) and ( mark for not circing E <tota
score 7 8=.
The correct o'tion shoud be a''ro+imate" the same ength as the
distractors. This 'rinci'e a''ies es'ecia" to 1ocabuar" tests and tests of
31
i
The distractors
3,
i
reading and istening com'rehension) where there is a tendenc" to make
the correct o'tion onger than the distractors sim'" because it is so often
necessar" to Buaif" a statement or word in order to make it absoute"
correct. .n e+am'e of such a Dgi1eawa"@ item is!
He began to choke whie he was eating the fish.
.. die
B. cough and 1omit
C. be unabe to breathe because of something in the wind'i'e
:. grow 1er" angr"
Each distractor) or incorrect o'tion) shoud be reasonab" attracti1e and
'ausibe. (t shoud a''ear right to an" testee who is unsure of the correct)
o'tion. (tems shoud be constructed in such a wa" that students obtain the
correct o'tion5b55direct seection5rather than b" the eimination of
ob1ious" incorrect o'tions. Choice : in the foowing grammar item is
much beow the e1e being tested and wi be eiminated b" testees
immediate"! their chances of seecting the correct o'tion wi then be one
in three.
The 'resent ta+ reforms ha1e benefited ... . 'oor.
.. that B. the C. a :. an
>or most 'ur'oses) each distractor shoud be grammatica" correct when
it stands b" itsef! otherwise testees wi be e+'osed to incorrect forms. (n the
abo1e item <and in a grammar items= it is on" the wrong choice) and its
im'ied insertion into the stem) which makes a 'articuar 'attern
ungrammatica. >or e+am'e) o'tion . is grammatica" correct on its own
and on" becomes incorrect when inserted into the stem.
The foowing item <which actua" a''eared in a cass 'rogress test of
reading com'rehension= contains two absurd items!
How did 6icard first tra1e in s'aceG
.. He tra1eed in a s'ace5shi'. C. He went in a submarine.
B. He used a arge baoon. :. He Jum'ed from a ta buiding.
/ness a distractor is attracti1e to the student who is not sure of the
correct answer) its incusion in a test item is su'erfuous. 6ausibe
distractors are best based on <a= mistakes in the students@ own written
work) <b= their answers in 're1ious tests) <c= the teacher@s e+'erience) and
<d= a contrasti1e ana"sis between the nati1e and target anguages.
:istractors shoud not he too difficut nor demand a higher 'roficienc" in
the anguage than the correct o'tion. (f the" are too difficut) the" wi succeed
on" in distracting the good student. who wi be ed into
considering the correct o'tion too eas" <and a tra'=. There is a tendenc"
for this to ha''en) 'articuar" in 1ocabuar" test items.
%ou need a ... . to enter that miitar" airfied.
.. 'ermutation B. 'erdition C. 'ermit :. 'ers'icuit"
Note that ca'ita etters are on" used in o'tions which occur at the
beginning of a sentence. Com'are the foowing!
Has ... . of 'etro increasedG
.. the 'rice B. 'rice C. a 'rice
of 'etro has actua" faen.
.. The 'rice
B. 6rice
C. . 'rice
3.8 ;riting the test ;here muti'e5choice items are used) the testees ma" be reBuired to
'erform an" of the foowing tasks!
1 ;rite out the correct o'tion in fu in the bank.
He ma" not come) but we@ get read" in case he ...
.. wi B. does C. is :. ma"
, ;rite on" the etter of the correct o'tion in the bank or in a bo+ <which
ma" a''ear at the side of the Buestion) etc.=.
He ma" not come) but we@ get read" in case he ...
.. wi B. does C. is :. ma"
3 6ut a tick or a cross at the side of the correct o'tion or in a se'arate bo+.
He ma" not come) but we@ get read" in case he ...
.. wi ..
B. does B.
C. is C.
:. ma" :.
? /nderine the correct o'tion.
He ma" not come) but we@ get read" in case he ...
.. wi
B. does C. is :. ma"
8 6ut a circe round the etter at the side of the correct o'tion.
He ma" not come) but we@ get read" in case he ...
.. wi
9.
Bdoes C. is :. ma"
*uti'e5choice items are rare" o'tiona in a test) for the testees
woud then s'end considerabe time in unnecessar" reading before
choosing the items the" wished to answer. *oreo1er) uness there are good
reasons for weighting different items <using the a''ro'riate statistica
methods=) it is ad1isabe to award eBua marks for each item.
The correct o'tion shoud a''ear in each 'osition <e.g. .) B) C) : or
E= a''ro+imate" the same number of times in a test or sub5test. This can
usua" be achie1ed if it is 'aced at random in a certain 'osition or if a
the o'tions are 'aced in a'habetica order <i.e. according to the first
etter of the first word in each o'tion=. Howe1er) if the o'tions ha1e a
natura order <e.g. figures)5dates=)5it is ad1isabe to kee' to this order.
Backwe started his career as a aw"er in
.. 12,1 B.
12,8 C. 12,A :. 123,
Note that few tests consist entire" of muti'e5choice items. *an"
we5known tests strike a ha''" baance between obJecti1e items <incuding
muti'e5choice items= and subJecti1e items. The" test famiiarit" with the
grammatica and e+ica com'onents of the anguage as we as the abiit" to use
the anguage 'roducti1e".
33
(
Tests of grammar and usage
?.1 (ntroduction The foowing are some of the most common t"'es of obJecti1e items used
to test awareness of the grammatica features of the anguage. Each t"'e of
item wi be treated in greater detai in this cha'ter.
5 muti'e5choice items
5 error5recognition items
5 rearrangement items
5 com'etion items
5 transformation items
5 items in1o1ing the changing of words
5 @broken sentence@ items
5 'airing and matching items
5 combination items
5 addition items
DO.i
) . A5)I
(t shoud awa"s be remembered that such items as the abo1e test the
abiit" to recognise or 'roduce correct forms of anguage rather than the abiit"
to use anguage to e+'ress meaning. attitude) emotions) etc.
Ne1ertheess) it is essentia that students master the grammatica s"stem of the
anguage the" are earning. Thus) cassroom tests of grammar and
usage can 'a" a usefu 'art in a anguage 'rogramme.
?., *uti'e5choice The t"'e of muti'e5choice item fa1oured b" man" constructors of
grammar items!item grammar tests is the incom'ete statement t"'e) with a choice5 of four or
t"'es i f1e o'tions. This item ma" be written in an" of the foowing wa"s!
T"'e 1 Tom ought not to ... <.. te
B. ha1ing tod C. be
teing. :. ha1e tod= me "our secret) but he did.
T"'e , Tom ought not to ... . me "our secret) but he did.
.te
B.ha1ing tod
1
C.be teing
:.ha1e tod
.. te
B. ha1ing tod
T"'e 3 Tom ought not to
3?
C. be teing
:. ha1e tod
me "our secret) but he did.
T"'e ? Tom ought not to ha1e tod me "our secret) but he did.
.. No change
B. te
C. ha1ing tod
:. be teing
(tem t"'es , and 3 are 'referabe to 1 because the o'tions do not
interru't the fow of meaning in the sentence! these items 'resent the
entire sentence so that it can be read at a gance. /nfortunate") t"'e 1
confuses the reader because of the ong 'arenthesis <i.e. the four o'tions
occurring between ought not to and me=. (tem t"'e ? shows the correct <or
an incorrect= form as 'art of the sentence in such a wa" that it cannot be
com'ared on eBua terms with the other o'tions! a correct o'tion) for
instance) is genera" easier to recognise when it a''ears in the framework
of the sentence than as 'art of a ist of distracters.
.nother item t"'e a''ears beow) but it is not recommended since it
reBuires the testees to s'end time on unnecessar" reading. Not on" is it
uneconomica but it does not 'resent the @'robem@ <i.e. the o'tions= as
cear" as item t"'e , does.
T"'e 8 .. Tom ought not to te me "our secret) but he did.
B. Tom ought not to ha1ing tod me "our secret) but he did.
C. Tom ought not to be teing me "our secret) but he did.
:. Tom ought not to ha1e tod me "our secret) but he did.
The foowing method is usefu for testing short answers and
res'onses!
T"'e A @Tom ought not to ha1e tod an"one the secret.@
.. @0o ought "ou.@
C. @Neither "ou oughtn@t.@
B. @Neither ought "ou.@
:. @0o oughtn@t "ou.@
(tem t"'e 7 reBuires the students to seect the aternati1e which is
true according to the information con1e"ed in each sentence. 0uch an item
ma" he incuded either in a test of reading com'rehension or in a test of
grammar! a knowedge of the 'articuar s"nta+ is necessar" for the
understanding of the sentence.
T"'e 7 @Tom ought not to ha1e tod me.
.. Tom did not te me but he shoud.
B. 6erha's Tom ma" not te me.
C. Tom tod me but it was wrong of him.
:. (t was necessar" for Tom not to te me.
(t ma" be argued that an understanding of s"ntactica 'atterning is Just as
necessar" for the foowing item!
. was #obert ate ast weekG@
Three times.@
.. How much
C. How often
B. How man"
:. How ong
38
11
1
(tems which a''ear in a test of grammar and structure shoud be made
to sound as natura as 'ossibe. The foowing mechanica test item!
This book beongs to 6eter. (t is ...
.. 6eter@s book
C. the book of 6eter
B. the book to 6eter :. the book of 6eter@s
can be rewritten as foows!
This book beongs to 6eter) but that is ...
.. *ar"@s book C. the book of *ar"
B. the book to *ar" :. the book of *ar"@s
Note that distractors shoud genera" be correct both in writing and in
.s'eech5 T5he..foio.wing5item5'roO1..edOunsuccessfu. when it was incuded in a
test because man" of the more abe students seected o'tion :. the reason
being that the" 'ronounced used to Buite correct" as use toKJu!staK!
. to go to m" unce@s farm e1er" weekend.
.. am used C. was used
B. used :. use
Note that the sam'e items discussed in this section ha1e so far taken
the form of short deconte+tuaised items. (n 'ractice) howe1er) such items
woud a form 'art of a 'aragra'h or series of 'aragra'hs of descrir)ti1e)
narrati1e or e+'ositor" 'rose. The 'ro1ision of a detaied conte+t in this
wa") howe1er) often imits the range of grammatica features being tested.
(t is usua" im'ossibe) for e+am'e) to test the future continuous tense in
a narrati1e set in the 'ast <uness direct s'eech is used=. 0imiar") a
'aragra'h describing a sim'e manufacturing 'rocess ma" not 'ro1ide the
test writer with the o''ortunit" to test a the 1erb forms and tenses he or
she ma" wish to test. This is the 'rice to be 'aid for incuding more
natura) Corttc+tuah0eu test (LC(/0. 9n the other hand) the ad1antage of
such items as that on 'age 37 ies in the interesting and fair" authentic
conte+t <i.e. a news'a'er artice= which contains the items. This is rea
anguage used for a 'articuar 'ur'ose. >urthermore) the 'ro1ision of
conte+t he's to ensure that there is on" one correct o'tion in each case.
0hort deconte+tuaised sentences can ead to ambiguit" as the" are usua"
o'en to se1era inter'retations when used as stems for muti'e5choice
items. >or e+am'e) o'tion : in the foowing deconte+tuaised item might
be correct <as we as o'tion B= if the student ha''ens to know of a
medica research estabishment which 'a"s 1ounteers to assist with
research and deiberate" catch a cod so as to be abe to test 1arious curesR
coudn@t take the test ast week because ( ... . a cod.
.. ha1e caught C. woud catch
B. had caught :. was catching
*uch better5 for testing 'ur'oses is the foowing item. The 'assage is
taken straight from a news'a'er artice and thus the anguage is authentic
and unatered in an" wa". The conte+t 'ro1ides students with enough
background knowedge and detais to a1oid ambiguit" and aternati1e
inter'retations) and the news'a'er re'ort itsef is 1er" interesting. :oes it
rea" matter if it wi not aow us the o''ortunit" to test e1er" 'oint of
grammar which we ma" want to testG 0tudents taking this test are being
gi1en a rea fee for the anguage the" are earning.
. ong wa" from home
. 7,5"ear5od 0amoan who <1= ...
. no Engish at a s'ent
thirteen da"s <,= ...
. on buses in the 0an >rancisco area after
had become se'arated <3= ...
. his fami") 'oice said.
said that >aaitua Logo) <8= ...
mo1ed to the
/nited 0tates two "ears ago) eft his son and daughter5in5aw
<
A= . afewminutes inamarket in0anJose <7=...
something at a nearb" sta. ;hen he tried to return to th
em) he coud
not remember where the" <3= ...
. for him.
<3= )
O . first) he began to wak to their home in 6ao .to) ,-
kiometres <1-=...
)butater he<11= . onabus. H
changed from bus to bus <1,= .. .
bushes and trees) 'oice said.
<1= .. is s'eaking B. s'eaks
<,= .. to ride B. was riding
<3= .. with B. from C. b"
<?= .. He B. The" C. 9ne
. the da"time and se't unde
C. has s'oken :. was s'eakir
C. ride :. riding
:. off
:. (t
<8= .. which B. that
C. who :. what
<A= .. in B. for C. since
:. at
<7= .. to bu" B. for bu"ing C. and bu"
:. bu"ing
<3= .. waited
<2= .. >or
B. were waiting. C. ha1e waited :. wait
B. 9n C. (n :. .t
?.3 Constructing
muti'e5choice items
<1-= .. far B. from C. near :. awa"
<11= .. woud Jum' B. Jum'ed C. has Jum'ed :. woud h)
Jum'ed
<1,= .. on B. at C. for :. during
.though it is not awa"s 'ossibe to use sam'es of students@ own writte
work to 'ro1ide the basis for the test items) it shoud not be too difficut for
constructors of cassroom tests and schoo achie1ement tests to take
ad1antage of the t"'es of errors made b" students in their free
com'ositions and o'en5ended answers to Buestions.
The foowing e+tract from a student@s etter is used here and in ate
sections to show how test items can be constructed. The etter was actuLR
written b" a student in a countr" where Engish is earnt as a foreign
anguage. The errors ha1e not been Dmanufactured@ for the 'ur'ose of
iustration) but the" do re'resent errors made b" students from on" on
'articuar anguage background. The mistakes) therefore) wi not he t"'ica
of mistakes made b" students from man" other anguage
backgrounds and thus the distractors a''earing ater ma" be useess for
such students.
There is 1er" much time ( didn@t write "ou) and now ( ha1e a itte free
time. ;inter is behind us and therefore ( ho'e that "ou woudn@t min
on such a ong 'eriod between m" ast etter and this one. %ou know
how is it. 0un is shining) trees become green and it@s difficut to sta" is 1
cosed among was. 0ometimes whenethe
nJo" ooking the chidren
to
through the 'ark near m" odging
'a"ing. %ou know) the da" before "esterda" whie ooking through r
window ( saw the wet street and 'eo'e with umbreas rushing for
mone" and 'restige. ( sudden" remembered
beor
me what ha''ened.
to us. ( su''ose that "ou were not angr"
that it is not good to discuss about 'assed feeings.
>
(tem 1
Let us ignore the error in the first sentence for the time being and
concentrate on the error of tense after
ho'e.
0te' 1!
The first ste' is to reduce the ength of the sentence and to correct the
error <and an" other errors in the origina sentence=. Thus)
( ho'e that "ou woudn@t mind on such a ong 'eriod between m" ast
etter and this one.
becomes ( ho'e "ou won@t mind waiting for so ong.
0te' ,! Ne+t we write out the sentence) substituting a bank for the area being
tested. ;e write in the correct o'tion and the distractor which the student has
'ro1ided for us. Howe1er) we ha1e to add a sentence because. in certain.5
<rare= conte+fs woudn@t ma" be correct.
( ho'e "ou ... . mind waiting for so ong. ( 'romise to re'"
sooner in future.
.. won@t
B. woudn@t
0te'53! ;e now add another two distractors. .gain) we go to the written
work of our students to 'ro1ide these distractors. But if we cannot ocate
an" suitabe errors without too much difficut") we use our own e+'erience
and knowedge of the target and nati1e anguages. Thus) two usefu
distractors which woud aso baance the e+isting two o'tions might be
shoudn@t and shan@t.
(
ho'e "ou ... . mind waiting for so ong. ( 'romise to re'"
sooner in future.
.. won@t B. woudn@t
C. shoudn@t
:. shan@t
(t ma" be argued) howe1er) that shan@t is acce'tabe usage amongst certain
s'eakers) thus gi1ing us two correct o'tions instead of one. Though it is high"
im'robabe that 'eo'e in most areas woud use
shan@t) there is a sight shadow of
doubt. This is enough to make it desirabe to remo1e
shan@t from our ist of o'tions.
0te' ?! 9ne suggestion ma" be that we re'ace
shan@t with can@t. (f
students from a 'articuar anguage background make such mistakes as
can@t mind)
can@t shoud be used as a distractor) and 'ossib"
shoudn@t
changed to coudn@t.
.s can be seen at this ear" stage) the actua 'rocess of item writing is
e+treme" subJecti1e.
( ho'e "ou ...
. mind waiting for so ong. ( 'romise to re'"
sooner in future.
.. won@t B. woudn@t C. coudn@t :. can@t
.n aternati1e suggestion for a fourth o'tion might be
don@t or didn@t!
(
ho'e "ou don@t mind waiting for so ong.
( ho'e "ou didn@t mind waiting for so ong.
/nfortunate") both don@t and didn@t are correct. Howe1er) in the
foowing conte+t) didn@t is not acce'tabe!
@How ong are "ou going to beG@
@.bout haf an hour. ( ho'e "ou ...
. mind waiting for so ong.
.. won@t B. woudn@t C. shoudn@t :. didn@t
33
11
1
(t ma" be argued that didn@t stands out too much. (f so 5 and if it is eBua"
usefu to test the use of don@t <instead of
won@t= after ho'e 5 the item couc be
rewritten as!
@How ong wi "ou beG@
@.bout haf an hour. ( ho'e "ou ... . mind waiting for so ong.
.. don@t B. woudn@t
C. shoudn@t :. didn@t
9b1ious") there are 1ar"ing degrees of refinement in the constructioi of
muti'e5choice items. >urthermore) some items are much more difficu to
construct than others. The foowing two items based on errors in the
student@s etter are fair" sim'e to write.
(tem ,
Error!
. and enJo" ooking the chidren 'a"ing.
(tem! 9d *r Jones enJo"s ...
. the chidren 'a"ing.
.. ooking C. ooking on
B. ooking at :. ooking to
0ome test constructors might be tem'ted to use
for as a distractor. (t can be
argued) howe1er) that ooking for is correct! od *r Jones might enJo" ooking for the
chidren 'a"ing <i.e. he might enJo" waking throug the 'ark) chatting to his friends) etc.
whie he is in the 'rocess of ooking for his grandchidren. who are 'a"ing=.
Note that the correct o'tion is now in the third 'osition) C. (t is
im'ortant to 1ar" its 'osition. Note aso that the word ooking a''ears in each o'tion!
in some tests the item might a''ear as foows!
9d *r Jones enJo"s ooking ...
. the chidren 'a"ing.
.. 5 B. on C. at :. to
Howe1er) when this format incudes a dash <5=) it is unnatura and not
recommended since the insertion of a dash in the stem woud not be
norma 'ractice in rea ife.
(tem 3
Error! @( su''ose that "ou were not angr" to me.@
(tem!
( do ho'e "ou weren@t angr" ... . me.
.. to
B. with C. on :. about
Note that at is aso incorrect and ma" be used as a 'ossibe distracto4
9n the other hand) it ma" be fet that a number of nati1e Engish s'eake do
sa" angr" at a 'erson. The decision whether or not to incude
at in the ist of
incorrect o'tions is .again a "er" subJecti1e one.
?.? Constructing error5
The fourth sentence of the etter on 'age 37 begins 0un
is shining) trees
recognition muti'e5 become green and ...
. The error caused b" the omission of t!
choice items
artice ma" be tested as foows) using a muti'e5choice item!
. is shining bright" toda".
.. 0un B. The sun
C. . sun :. 0ome sun
(t ma" be argued) howe1er) that the choice here is strict" between
o'tions . and B at certain e1es where students ha1e earned to a1oid
using @a@ and @some@ with @sun@O (n such instances) one usefu de1ice <sti
using the muti'e5choice format= is the error5recognition t"'e of item.
#
1
T"'e 1
Each sentence contains four words or 'hrases underined) marked .) B) C
and :. 0eect the underined word or 'hrase which is incorrect or
unacce'tabe.
1. 1 do ho'e "ou woudn@t mind waiting for such a ong time.
. B C :
,. (@m worried that "ou@ be angr" to me.
. B C :
3. ( didn@t see Bi since he went into hos'ita ast month.
. B C :
?. *" car had broken down) so ( went there b" foot.
.55 OOE.. C. %)
T"'e ,
There is a mistake in grammar in each of the foowing sentences. ;rite the
etter of that 'art of the sentence in which it occurs.
. B C :
1. 0unKis shiningKbright" toda"K) isn@t itG
. B C :
,. 9d *r JonesKenJo"sKooking the chidrenK'a"ing in the 'ark.
. B C :
3. Ton"@s fatherKwoud not et himKto sta" outKate at night.
. B C :
?. :idn@tK0usan te "ouKshe woudn@t mind to comeKwith us on the
'icnicG
(tem t"'e , aows the test writer to test errors caused b" omission!
e.g. 0un is shining and ooking the chidren. This t"'e of error cannot be
tested b" the first item of the error5recognition t"'e. Howe1er) there are
different wa"s of correcting man" sentences. >or e+am'e) students ma"
write B or C to denote the incorrect 'art of the third sentence abo1e)
according to which of these correct 1ersions is in their mind!
Ton"@s father woud not 'ermit him to sta" out ate. <7 B=
Ton"@s father woud not et him sta" out ate. <7 C=
>or this reason) the test writer is strong" ad1ised to a1oid items of the
second t"'e.
0ometimes Vstudents are gi1en correct sentences together with the
incorrect ones! the" are then reBuired to write the etter E if the sentence
does not contain an" error. (n 'ractice) this method does not work too we
since man" students tend to regard e1er" sentence as containing an error.
(ndeed) another argument against this t"'e of item is that it em'hasises the
more negati1e as'ects of anguage earning. (t is cear" not sufficient for
students sim'" to recognise sources of error! the" ought to be encouraged
at a times to concentrate on recognising and 'roducing the correct forms.
This argument is su''orted b" man" 's"choogists and teachers who hod
that it is undesirabe for students to be e+'osed too much 5to incorrect
forms. 9n the other hand) this item t"'e is cose" reated to those skis
reBuired when students check) edit or 'roof5read an" re'ort) artice) 'a'er
or essa" the" ha1e Just written.
?.8 Constructing #earrangement items can take se1era forms) the first of which to consider
rearrangement items
here wi be the muti'e5choice t"'e.
The student who wrote the etter in ?.3 ob1ious" e+'erienced
considerabe difficut" with word order in re'orted s'eech) es'ecia" after
the 1erbs know and wonder. Here are two of the errors he made!
@%ou know how is it.
<3rd sentence=
@( wonder did "ou grow more fatter since summer.@
<ater in the same etter=
(f we attem't to test the first error b" means of an ordinar"
muti'echoice item) we are faced with the 'robem of being restricted to
on" two o'tions! the correct o'tion and the distractor <i.e. the error=.
%ou know how ...
.. it is
B. is it
.s the item stands here) we cannot 'ossib" construct other o'tions. (t
becomes necessar") therefore) to engthen the origina statement to! %ou know
how warm it is toda". The item woud then read!
@;on t ( need a coatG@
@;e) "ou know how ...
.. warm is it toda"
B. toda" it is warm
C. is it warm toda"
:. warm it is toda"
E. toda" is it warm
There seems to be a danger here of confusing the testees b" 'resenting
them with the 'robem in such a wa" that a certain amount of menta
Jugging becomes necessar" on their 'art. . 'referabe item t"'e@ is the
foowing word5order item!
Com'ete each sentence. b" 'utting the words beow it in the right order. 6ut
in the bo+es on" the etters of the words.
@;on@t ( need a coatG@
@;e) "ou know how ...
.. it B. toda" C. warm
:. is
wonder if ...
. since summer.
.. grown B. "ou C. fatter
:. ha1e
;ord order items are usefu for testing other structures and features
in1o1ing in1ersion!
E1er"one@s forgotten ...
.. cu' B. he C. which
:. used
Not on" ...
. but he took me to his house.
.. me B. he C. did :. meet
Howe1er ... . "ou@ ne1er 'ass that test.
.. "ou B. tr" C. hard :. ma"
Leeds /nited shoud ha1e won! Just think ...
.. unuck"
B. were C. how :. the"
?1
1
5i don@t know how ong ...
.. going B. Jim C. is :. to be
The order of adJecti1es and the 'osition of ad1erbs can be tested in
this wa") as indeed can se1era other grammatica areas!
The 'oice are ooking for ...
.. big B. two C. cars :. back
;oud "ou ike to read :a1id Brown@s ...
.. short B. new C. stor" :. e+citing
Tom said ... . ceaning his car.
.. had B. finished C. he :. Just
9n" ... . been rude to "ouR
.. e1er B. ( C. ha1e :. once
*rs ;aker made .nn ...
.. her new 'en B. to C. show :. me
0omeone warned #ob ... D . thie1es.
.. for B. to C. out :. ook
(n man" cases it wi be usefu to change from a muti'e5choice item
format to a format in1o1ing some actua writing. The rearrangement item
can be used to test the same features of word order) but the item format
becomes a itte ess artificia. The students are sim'" reBuired to
unscrambe sentences and to write out each sentence) 'utting the words or
constituent 'arts in their correct order!
1. Not on" ...
55
Kthe e+aminationK1er" difficutKunfairKwasKbutKitKwasKaso
,. (t is not ad1isabe
Kthe e+aminationKateKu'Kthe nightKto sta"Kbefore
3. The best wa" to 're'are ...
K i sK"oursefK'ast 'a'ersKtimed 'racticeKfor the 'a'erKto gi1eKin doing
Note that rearrangement items can be used for sentences as we as for
words and 'hrases. ;hen used for this 'ur'ose) such items can offer a
means of testing an understanding of connecti1es and reference de1ices.
0tudents ma" be reBuired to write out a the sentences in their correct
seBuence or sim'" to 'ut the etters of the 1arious sentences in their
correct order!
.. ConseBuent") "ou shoud make e1er" effort to com'ete the 'a'er.
B. 0cribbe them down as Buick" as "ou can) if necessar".
C. (f "ou find "oursef running out of time before "ou can com'ete it)
howe1er) don@t worr" about writing "our answers neat".
:. #emember that it is im'ossibe to score marks on Buestions which
"ou ha1e not attem'ted.
?.A
Constructing
Carefu" constructed com'etion items are a usefu means of testing a
com'etion items student@s abiit" to 'roduce acce'tabe and a''ro'riate forms of anguage.
The" are freBuent" 'referabe to muti'e5choice items since the" measure
?,
'roduction rather than recognition) testing the abiit" to insert the most
a''ro'riate words in seected banks in sentences. The words seected for
omission are grammatica or functiona words <e.g. to) it) in) is) the=!
content words ma" be seected in a 1ocabuar" or reading test.
The error 0un is shining in the e+tract from the student@s etter in ?.3
iustrates one <minor= difficut" of constructing satisfactor" com'etion
items. .though on" one answer is 'ossibe here) this com'etion item
woud ha1e to a''ear as!
. 0un is shining toda".
or as! . sun is shining toda".
The former item suggests to the testees that no determiner is necessar"
<since 0un is written with a ca'ita etter= whie the atter item suggests that
a determiner is necessar" <because sun is written without a ca'ita=.
The item can be sim'" rewritten as a Buestion to o1ercome this
'robem!
(s
. sun shining toda"G
Here are two more e+am'es of com'etion items based on the student@s
etter!
;rite the correct word in each bank.
1. The od man enJo"s ooking ... . the chidren 'a"ing.
,. That car beongs ... . Heen@s mother.
3. ( ho'e "ou@re not angr" ...
me.
6ut a) the) or some in each bank on" where necessar". (f "ou think that
no word shoud be 'aced in the bank) 'ut a cross <+= there.
1. Can "ou see ... . sun shining through the coudsG
,. ( saw "our unce ... . da" before "esterda".
3. ;hat ha1e "ou been doing since ( saw "ou ...
. ast summerG
Com'etion items cannot) of course) be machine5marked but the" are
1er" usefu for incusion in cassroom tests and for e+ercise 'ur'oses.
Howe1er) sometimes the most straightforward com'etion items can cause
'robems in the scoring. (n the foowing e+am'e was 're'aring and
're'ared are eBua" correct. (t can be argued that had 're'ared is aso
correct if as is regarded as meaning because! i.e. 'eo'e gas'ed because
the" didn@t e+'ect him to 're'are for the Journe" 5 the" thought he woud
go without 're'aration.
6#E6.#E 1. He heard a gas' behind him as he ... . to go.
/ne+'ected wa"s. of com'eting banks are shown "et again in the
foowing e+am'e!
.s soon as 'ossibe the ne+t da" ( sent m" stor" ... . the
editor ... . the magaLine ... . which m" best work
usua" a''eared.
(t is Buite 'ossibe to write a stor" about an editor. and send the stor" about
the editor to a magaLine. .though such an inter'retation ma" sound
somewhat absurd) it iustrates the engths to which the test writer must
sometimes go to make certain that testees 'roduce on" the answer he or she
wants to be used in each bank. >or cass tests) such a critica attitude might
we be harmfu if it took the teacher@s attention off more im'ortant and
urgent 'robems in teaching and testing.
?3
The foowing e+am'e, indicates the wide range of 'ossibiities for
one com'etion item!
( go to the cinema reguar") but ( ... . to the theatre for months.
The answer ob1ious" reBuired b" the tester is
ha1en@t been4 howe1er)
'ossibe answers are!
ha1en@t been shan@t be going
hadn@t been
can@t go
<sometimes= don@t go ha1en@t been abe to go
ma" not go am not going
don@t know whether (@1e been didn@t go
shan@t go
ha1en@t gone
won@t go ha1en@t been going
(f the aim of this 'articuar item is to force the use of the 'resent 'erfect
tense) there are)three wa"s of restricting the choice a1aiabe to the testees
<athough the first two wa"s de'end hea1i" on reading com'rehension=!
<a= b" 'ro1iding a conte+t!
$im usua" goes to the cinema about once a week but she ...
four fims aread" this month and it@s on" the ,-th toda". <6ossibe
answers! has seenKwi ha1e seenKmust ha1e seen=
<b= b" 'ro1iding data!
( go to the cinema reguar") but it@s ages since ( ast saw a 'a".
( go to the cinema reguar") but ( ...
. to the theatre for
months.
<6ossibe answers! ha1en@t beenKha1en@t goneKha1en@t been
goingKha1en@t been abe to go=
<c= b" using muti'e5choice techniBues!
( ) )
. to the theatre three times since ( ast saw "ou.
.. go C. had gone
B. ha1e been :. went
There are two maJor ad1antages in using a 'assage of continuous
'rose rather than se'arate sentences when gi1ing a com'etion t"'e test.
>irst") the use of conte+t often a1oids the kinds of ambiguit" referred to in
the 're1ious 'aragra'hs. 0econd") the students e+'erience the use of
grammar in conte+t) being reBuired to use a the conte+t cues a1aiabe in
order to guess man" of the missing words. .s a conseBuence) the" are
genera" ad1ised to read or gance at the whoe 'assage before starting to
f i in an" of the banks.
The foowing are e+am'es of two different t"'es of com'etion items
in conte+t. (n e+am'e <a= the banks are indicated whie in e+am'e <b=
<sometimes referred to as a modified coLe 'assage= the omissions
themse1es are not indicated. (n e+am'e <a= on" one word shoud be
written in each bank) whie in e+am'e <b= one word has been omitted
from each ine. (n this atter t"'e of e+am'e) the students are reBuired to
'ut an obiBue stroke <K= at the 'ace where the word has been omitted and
then to write the missing word in the a''ro'riate s'ace.
<a= (t <1 =
. awa"s usefu
<,= . 'ractise answering
the t"'es of Buestions <3= ...
O O
"ou ma" <?= ...
. asked. Howe1er)
<8=
. is not enough 5
sim'" <A= ...
. gance through
a 'ast 'a'er <7= ...
. answer
the Buestions <3= ...
. "our head.
This wi gi1e "ou <2= ...
indication
at a <1-=
. what "ou can do.
<b= Historians and anthro'oogists used think
<1=
that i heath and disease 're1aent among
<,=
'rimiti1e tribes. #esuts recent in1estigations.
<3=
howe1er) ha1e shown we much to earn
<?=
from 'rimiti1e 'eo'e. ;ithout benefit of
<8=
immunisation or 1accination) 'rimiti1e often
<A=
acBuired immunit" the diseases 'resent in
<7=
their societ". *oreo1er) the diseases now beset
<3= .O.
modern societ" were uncommon 'rimiti1e
<2=
man. .though he did i1e a ong ife) <1-=
according to research) he ha1e been far
<11=
heathier 're1ious" thought.
<1,=
(n the foowing e+am'e) 'art of a sentence haO =een omitted.
.though sentence com'etion items of this t"'e mai concentrati orc
s'ecific grammatica 'oints more difficut) the" ne1e5heess offer a usefu
wa" of testing an abiit" to hande structures accordi7J to certain 'atterns.
;hether such items are regarded as tests of grammaO or of controed
writing is of minor im'ortance! the" demand an abii! to use a''ro'riate
grammatica forms for a 'articuar 'ur'ose in a 'articuar conte+t.
*an" Loo officias are worried b" the increasing ii5treatment of animas
b" 1isitors to the Loo) es'ecia" b" chidren. *ost risitors e+'ect to see
the animas 1er" acti1e in their encosures. ;hen 5e animas fai to
'erform in the wa"s e+'ected ...
Crocodies seem to be the chief 1ictims of such arscks because
the" ...
. The bot .s) cans) sticks and
stones that ...
. enc J' as unsight" itter
in the encosures. 0ome animas) howe1er) swae)w
. *ost of the 1isors wttu5
H H H H H H H H
H H . (
are not rea" 5aicious but
sim'" ..
. Qoo ofPcias are
constant" ...
. (ndiscriminate feeding of
the animas is not aowed because ...
The com'etion of diaogues can aso 'ro1ide a uOs.fu wa" of testing
the abiit" to mani'uate the grammar and 'atterns of the anguage in
conte+t. .gain) meaning 'a"s a ke" roe in determinir@ the students@
abiit" to 'ro1ide satisfactor" answers. /nfortunate". howe1er. if its 9
W
1er" difficut to write natura diaogues and at the same time 'rcre
students with usefu cues.
B! 0o do (. ( genera" watch it for an hour or two e1er" e1ening.
.!
B! (mmediate" after (@1e finished m" homework) at about eight5thirt" or
nine.
B! 0o are mine. The funnier) the better. ( ike .merican ones best of a 5
"ou know) 'rogrammes ike @Benson@ and @:ifferent 0trokes .
. !
B! Neither do 1. ( can@t stand an"thing that@s too frightening.
B! ( agree about educationa 'rogrammes) but there are sti 1er" few of
themO
?.7 Constructing K The transformation t"'e of item is e+treme" usefu for testing abiit" to
transformation items 'roduce structures in the target anguage and he's to 'ro1ide a baance
when incuded in tests containing muti'e5choice items. (t is the one
obJecti1e item t"'e which comes cosest to measuring some of the skis
tested in com'osition writing) athough transforming sentences is different
from 'roducing sentences. 0ubJecti1e decisions) of course) ma" ha1e to be
made in the scoring 'rocess! e.g. how shoud s'eing errors be markedG
The foowing transformation items ha1e been based on errors which
occurred in the student@s etter) an e+tract of which was gi1en in 0ection ?.3.
#ewrite each of the foowing sentences in another wa") beginning each
new sentence with the words gi1en. *ake an" changes that are
necessar" but do not change the genera meaning of the sentence.
1. ( ha1en@t written to "ou for a ong time.
(t@s a ong time ..
,. (n sunn" weather ( often go for a wak.
;hen the weather ...
3. 9d *r Jones ikes to ook at the chidren 'a"ing.
9d *r Jones enJo"s ...
9ther transformation items gi1ing some idea of the range of areas that
can be co1ered in this wa" are!
1. (t was im'ossibe to work under those conditions.
;orking ...
,. ( don@t think it@s necessar" for "ou to sta" an" onger.
( don@t think "ou ...
3. ( was abe to ea1e the office ear" "esterda".
(t was 'ossibe ...
?. Joe can sing better than "ou.
%ou cannot ...
8. This book is too big to go on an" of the she1es.
This book is so big ...
A. >rances is 1er" good at tennis.
>rances 'a"s ...
7. 6oor 6eter was bitten b" a mosBuito.
. mosBuito ...
?A
3. @;hen wi "ou 1isit LondonG@ *r 0trong asked me.
*r 0trong asked me ...
.s with com'etion items) it is often difficut to restrict the number of
'ossibe answers. Howe1er) such restrictions are not essentia for
constructors of cassroom tests) 'ro1ided that the" are fu" aware of a the
'ossibe correct answers and of the s'ecific area the" are testing. The
foowing e+am'es indicate some of the aternati1es 'ossibe for four of
the 'receding items!
( ha1en@t written to "ou for a ong time.
(t@s a ong time since K <ast= wrote <to= "ou
since "ou recei1ed a etter from me) etc.
( don@t think it@s necessar" for "ou to sta" an" onger.
( don@t think "ou need <to= sta" an" onger 7
e+'ected answer
wi find it necessar" to sta" an" onger 7 'ossibe
answer
Joe can sing better than "ou.
%ou cannot sing as we as Joe 7 e+'ected answer
sing better than Joe 7 'ossibe answer
>rances is 1er" good at tennis.
>rances 'a"s tennis 1er" weKe+treme" we)
etc.
7 e+'ected answer
1er" good tennis 7 'ossibe answer
/nfortunate") the aternati1es in the ast three e+am'es defeat the
'ur'ose of the tests as the" stand at 'resent) since students can a1oid using
the actua grammatica 'atterns being tested. Howe1er) it is a sim'e
matter to rewrite them as foows!
(s it necessar" for us to sta" an" ongerG
Need ...
G
%ou cannot sing as we as Joe.
Joe can sing ...
>rances is 1er" good at tennis.
>rances 'a"s tennis ...
0ometimes it is difficut to eicit the 'articuar form we wish to test.
>or e+am'e!
( fee miserabe e1en though ( shoudn@t.
( know ( shoudn@t fee miserabe but ( do.
.though51 do is the answer5reBuired) we coud scarce" faut!
( know ( shoudn@t fee miserabe but ( certain" don@t fee ha''".
know ( shoudn@t fee miserabe but
1 am miserabe.
(n some tests)3 students ma" be reBuired to rewrite a sentence
beginning with a certain word underined in the origina sentence. >or
e+am'e!
The" beie1ed that the earth was fat.
The earth was beie1ed to be fat.
This item t"'e is a usefu 1ariation of the 're1ious t"'e discussed. but
sometimes restricts the kind of transformation 'ossibe since the first word
?7
11
of the new sentence has to a''ear in the origina sentence. Thus it becomes
im'ossibe to test the reBuired transformation of a sentence ike (s it
necessar" for us to sta" an" ongerG <7 Need we sta" an" ongerG=
Transformation can aso be effected b" reBuiring students to substitute
a
gi1en 1erb in a sentence)3 the new 1erb necessitating a change in the
structura 'attern.
Ten essons make u' the course. <C9N0(0T=
The course consists of ten essons.
coudn@t go swimming "esterda". <.LL9;=
( wasn@t aowed to go swimming "esterda".
.s with a the t"'es of items treated in the 're1ious sections) the
transformation t"'e of item is im'ro1ed if it can be 'ut into a conte+t <i.e. if the
sentences for transformation can be written in seBuenee to form 'art of a
continuous 'iece of 'rose=. /nfortunate") howe1er) the 'ro1ision of a conte+t
does not aow for the range of sentence 'atterns the test writer
ma" wish to test. *oreo1er) most students tend to treat each sentence in
isoation for the 'ur'oses of the test.
The foowing e+am'es iustrate how each of the sentences for
transformation can be made to form 'art of a continuous sentence.
1 Changing sentences according to a gi1en 'attern
<a= Eer" few obJecti1e tests aow for) choice.
%ou ha1e ...
<b= Howe1er) the instructions shoud be carefu" checked.
Howe1er) "ou ...
<c= :ifferent t"'es of Buestions on the same 'a'er wi necessitate
changes in the instructions.
The instructions
, Changing sentences b" using seected words
<a= #emember that it is not necessar" to answer the Buestions in the
order set. <NEE:=
<b= %ou are ad1ised to check "our answers carefu" after each Buestion.
<.:E(0.BLE=
<c= *ost teachers aso recommend "ou to ea1e fi1e minutes s'are at
the end of the e+amination in order to check "our 'a'er. <0/&&E0T=
?.3
Constructing items This t"'e of item is usefu for testing the student@s abiit" to use correct
in1o1ing the changing
tenses and 1erb forms. (t is a traditiona t"'e of test but the a"out is
of words
im'ro1ed in this 'articuar case b" 'ro1iding banks on the right of the te+t
for com'etion. The continuit" of the te+t is not im'aired more than
necessar" b" ha1ing both banks and underined words inserted in the
sentences. Thus the risk of obscuring the meaning of the te+t is reduced.
( Eerbs! tenses) etc.
#esearchers <1= to con1ince that a drug <1=
the" <,= to test can im'ro1e the memor" and that <,=
@0
it <3= to be the forerunner of other drugs which
<3=
e1entua" <?= to im'ro1e menta abiit"
<?=
, ;ord buiding
0tudents who were gi1en the drug for a fortnight did
considerab" <1. we= in tests than others. The tests
<1=
incuded the <,. memorise= of ists of words as we <,=
as of <3. inform= from two messages transmitted at <3=
the same time. :uring the first week there was no
<?. notice= difference between the two grou's) but <?=
after a fortnight the grou' on the drug was found to
ha1e increased its <8. abe= to earn b" amost <8=
twent" 'er cent.
?.2 Constructing
This t"'e of item@ tests the student@s abiit" to write fu sentences from a
@broken sentence@ series of words and 'hrases) and thus does not aow the test writer to
items
concentrate e+cusi1e" on testing those 'articuar grammatica features
which ma" ha1e Just been 'ractised in cass. (t is ne1ertheess a usefu
de1ice for testing grammar 'ro1ided that the tester is aware that se1era
other areas of the anguage are being tested in addition to those on which
he or she wishes to focus attention. (ndeed) this 1er" fact ma" be
considered an ad1antage. 0o man" students are abe to score high marks
on grammar items when each item is set in isoation and concentrates on
on" one area of grammar. Errors are made) howe1er) when the attention
of the student is concerned with the meaning of the conte+t as a whoe and
with 'erforming a number of different grammatica tasks necessar" to
achie1e5that meaning.
;hen setting this item) make sure that the instructions are 1er" cear
indeed and 'ro1ide one or two e+am'es. 0tudents unfamiiar with this
'articuar item format freBuent" ha1e difficut" in knowing e+act" what to
do) es'ecia" as the 're1ious e+'erience of man" ma" ead them to think
that the 'resence of an obiBue stroke indicates the omission of a word
instead of signaing mere" a fragmented sentence or series of notes. (n the
rubric) students shoud be instructed to make whate1er changes are
necessar" to form good sentences) adding artices) 're'ositions) etc. where
reBuired and 'utting 1erbs in their correct tense.
TakeK drugs and stimuantsK kee' awakeKwhie re1ise e+aminationK
often be 1er" harmfu.K (t be far betterK
eadK baanced ifeK and get
enough see'K e1er" night.KThereK beKimitKdegree and s'anK
concentration Kwhich "ou be ca'abeKe+ert.K BrainK
need restKas much
bod".K (ndeed)K it be Buait"K than Buantit" workK that be im'ortant.
?.1- Constructing This t"'e of item usua" consists of a short con1ersation! e.g. a stimuus in
'airing and matching the form of a statement or Buestion foowed b" a res'onse often in the
items form of a statement. (t is used to test the abiit" to seect a''ro'riate
res'onses to stimui which woud be 'resented ora" in norma e1er"da"
situations. The item is more usefu for testing students@ sensiti1it" to
a''ro'riac" and their awareness of the functions of anguage rather than
their knowedge of grammar <athough grammatica cues ma" 'ro1e
im'ortant in com'eting this item satisfactori"=. To 'erform the task
reBuired) students are sim'" reBuired to write the etter of the correct
res'onse in the s'ace 'ro1ided.
?2
=
Coumn 1
&oing to see a fim tonightG
How was the fimG.
can@t stand war fims) can "ouG
0o "ou went to the cinema.
:on@t "ou find war fims too 1ioentG
Ha1e "ou e1er seen a Ja'anese war fimG
( ike war fims.
(s e1er"one going to see the fimG
;hat5a@)tout going to see a cowbo" fim insteadG
;h" didn@t "ou come with us to see the fimG is
that wh" "ou don@t ike war fimsG
Letter Coumn ,
>. .. No) ( didn@t.
B. *ost are) ( think.
C. (t@s one of the reasons.
:. ( had a ot of work to do.
E. .ctua") ( Buite ike them.
>. %es) ( 'robab" wi.
&. No) ( ha1en@t.
H. ;hat a good ideaR ( 'refer them to war fims.
(. 0o do 1.
J. . right. Nothing s'ecia.
$. Not rea". ( Buite ike them.
(t shoud be remembered) of course) that this is not authentic
discourse. Howe1er) athough the anguage and situation here are
ine1itab" artificia) the item does ser1e to he' students to associate the
anguage the" are earning with rea5ife situations) abeit to a imited
e+tent.
?.11 Constructing These obJecti1e5t"'e items ha1e ong been used in 'ast tests. The" shoud
combination and be. used s'aring") howe1er) as the" in1o1e arge" mechanica res'onses
addition items on the 'art of the student. Note that athough the se'arate sentences are
inked to one another b" theme) the items can hard" be described as being
conte+tuaised in an" rea wa".
1 Combination items
<0tudents are instructed to Join each 'air of sentences) using the word in
brackets.=
<a= %ou finish the 'a'er. Then check "our answers carefu". <.>TE#=
<b= 0ome Buestions ma" be 1er" difficut. The" shoud be eft unti ater.
<;H(CH=
<c= %ou shoud usua" write answers in com'ete sentences for a the
Buestions on "our 'a'er. Howe1er) write notes for those Buestions
which "ou do not ha1e time to answer. <.LTH9/&H=
, .ddition items
<0tudents are instructed to insert the word in ca'itas in the most
a''ro'riate 'ace in each sentence.=
<a= %ET Ha1e "ou answered a the BuestionsG
<b= 0T(LL 0ome students had not mastered the correct
techniBues for answering e+amination Buestions.
<c= 9CC.0(9N.LL% There ma" be itte choice of Buestions.
Notes and references
1 0imiar t"'es of test items a''eared in 'ast 'a'ers of the Joint *atricuation
Board! Test in Engish <91erseas=.
, ( am indebted to *r John Bright for this e+am'e and the 'ossibe soutions.
3 0imiar items a''eared in the 'ast in the /ni1ersit" of Cambridge Loca
E+aminations 0"ndicate! Certificate of 6roficienc" in Engish.
? 0imiar items ha1e a''eared in the /ni1ersit" of Cambridge Loca E+aminations
0"ndicate! >irst Certificate in Engish.
so
Testing 1ocabuar"
8.1 0eection of items
. carefu seection) or sam'ing) of e+ica items for incusion in a test is
genera" a most e+acting task. *an" of the more traditiona t"'es of
1ocabuar" tests are designed in such a wa" that the" test a knowedge of
words which) though freBuent" found in man" Engish te+tbooks) are rare"
used in ordinar" s'eech.
The first task for the writer of a 1ocabuar" test is to determine the
degree to which he or she wishes to concentrate on testing the students@
acti1e or 'assi1e 1ocabuar". The ne+t task is to decide whether the e+ica
items in the test shoud be taken from the s'oken or the written anguage.
0eection of 1ocabuar" can thus be thought of as faiing into the foowing
rough5di1isions according to the four maJor anguage skis!
Listening! 'assi1eKs'oken
#eading! 'assi1eKwritten
0'eaking! acti1eKs'oken
;riting! acti1eKwritten
. four di1isions can be incuded in a singe test) of course) but e1en
then carefu consideration shoud be gi1en to the different weighting each
di1ision wi carr" in the test! for e+am'e) shoud there be a greater
concentration on those e+ica items seected from the students@ reading
materiaG &enera" s'eaking) the more eementar" the e1e of the test)
the greater the number of e+ica items associated with the s'oken
anguage.
The test constructor@s task is made much easier if a the students ha1e
foowed a 'articuar s"abus. Le+ica items can then be seected from!
5 the s"abus <incuding a word freBuenc" ist if a1aiabe=4
5 the students@ te+tbook <'ro1ided the items a''ro+imate to those used in
natura s'eech situations=4
5 the students@ reading materia <e.g. sim'ified readers) iterar" te+ts=4 and
5 e+ica errors taken from students@ free5written work <or from students@
incorrect answers in a coLe test=.
The foowing error) howe1er) ma" be one of 1erb 'atterning or sim'" the
wrong choice of 1erb!
(s the go1ernment going to contribute the new industr"G
81
(
(f an error of 1erb 'atterning) the correct 1ersion woud be
(s the go1ernment going to
contribute to the new industr"G
(f caused b" the wrong choice of 1erb) it woud be
(s the go1ernment going to
subsidise the new industr"G
*oreo1er) according to the findings of research@ conducted into the
effecti1eness of distractors in muti'e5choice 1ocabuar" tests) those
distractors based on students@ incorrect answers in coLe tests <though
moderate" usefu= were found to be ess 'owerfu than
<a= the use of fase s"non"ms <i.e. words with eBui1aent meanings to the
ke" word underined or shown in itaics in the sentence but
ina''ro'riate in the 'articuar conte+t=!
(@d ike to book two ...
. in the circe) 'ease.
seats 7 correct word
chairs 7 fase s"non"m
<b= conte+tua" ree1ant items <i.e. words reated to the conte+t but
different in meaning to the ke" word in the sentence=!
@How much is a ... . to Tok"o) 'easeG@
@Nine hundred "en) and a return is si+teen hundred "en.
singe 7 correct word
ticket 7 conte+tua" ree1ant
The test constructor is faced with a difficut 'robem if the testees
ha1e foowed different s"abu T@s. 0uch a situation is genera" associated
with 'roficienc" tests) in which.student@s suitabiit" and 'otentia for a
certain task are tested <e.g. uni1ersit" studies in the medium of Engish=.
(n these cases) the tester ma" wish to base the seection of e+ica items on
those used in the tasks for which the student is being tested. .n aternati1e
method) a''ro'riate for a kinds of tests) is the seection of items from
such we5known word ists as
. &enera 0er1ice List of Engish ;ords
<*ichae ;est 5 Longman=.
Cambridge Engish Le+icon
<#oand
Hindmarsh 5 Cambridge= and
The ;right >reBuenc" Count. These <and
other= word ists) howe1er) are based entire" on the written anguage4
furthermore) no account is taken of difficut" e1es <as o''osed to
freBuenc" e1es= and of areas where interference is encountered between
the 1ernacuar and the target anguage.
But testing the e+tent of a student@s 1ocabuar" is on" one as'ect of
the whoe 'robem! contro of the 1ocabuar" at his or her dis'osa must
aso be measured. .n abiit" to discriminate between words is of the
utmost im'ortance at a but the eementar" e1es. (n broader terms) this
abiit" to discriminate ma" be regarded as de1eo'ing a fee for the
anguage.
Tests of 1ocabuar" shoud a1oid grammatica structures which the
students ma" find difficut to com'rehend. 0imiar") tests of grammar
shoud contain on" those e+ica items which 'resent no difficut" to the
students.
8., *uti'e5choice (t is usefu to distinguish between the foowing two maJor kinds of
items <.= muti'e5choice 1ocabuar" items!
4)
&rou' . Choose the etter of the word which is the nearest in meaning to
the word in itaics.
He@s been 1er" feebe since his iness.
.. unwe B. thin C. fooish :. weak
&rou' B Choose the etter of the correct or best word to com'ete each
sentence.
Ha1e "ou heard the 'anning committee@s ...
. ror so1ing
the cit"@s traffic 'robemsG
.. theor"
B. design C. 'ro'asa :. 'ur'ose
This section concentrates on &rou' . items and the ne+t section on
&rou' B. The foowing item t"'es are e+am'es of four 1ocabuar"
recognition items which fa within the first grou'.
T"'e 1 (n this t"'e of recognition item the stem is re'aced b" a 'icture.
The testees see the 'icture and ha1e to seect the most a''ro'riate word
reating to the 'icture from four or fi1e o'tions. This t"'e of item is cear
1er" a''ro'riate at the eementar" stages.
.. running
B. Jum'ing
C. standing
:. kicking
T"'e , Here the stem consists of a definition! the testees ha1e to seect th
correct o'tion to which the definition refers. )
a 'erson who recei1es and 'a"s out mone" in a bank
.. broker
B. accountant C. creditor :. cashier
T"'e 3 The stem consists of a e+ica item! the testees ha1e to seect the
best s"non"m or definition.
ad1ocate
.. su''ort B. ad1ise
C. contradict :. damage
diator"
.. growing gradua" arger C. showing care and effort
B. sow in getting things done
:. hea1" with dro's of water
T"'e ? The stem here consists of a sentence. Hence) this t"'e of
recognition item is genera" to be 'referred to the 're1ious three t"'es ir. so
far as the @'robem@ word a''ears in conte+t. Eocabuar" is much more
usefu" tested in conte+t since it is the conte+t that gi1es s'ecific meaning
and ree1ance to a word) thus creating a situation which is as inguistica".
1aid as 'ossibe in the circumstances.
(t@s rained continuous" for
two whoe da"s.
.. without sto''ing C. reguar"
B. hea1i" :. at odd moments
((
0ince subte shades of meaning are often determined on" b" the
s'ecific conte+t in which a 'articuar word a''ears) it is genera" ad1isabe to
'ro1ide fair" fu conte+ts for 1ocabuar" testing) es'ecia" at an.
ad1anced e1e. The fuer the conte+t) howe1er) the) more difficut it
sometimes can 'ro1e to find 'ausibe distractors. >ew good distractors) for
e+am'e) can be found for the foowing item!
;e@1e had to 'ut off the meeting unti ne+t week.@<'ost'one=
0"non"ms are not awa"s interchangeabe in a conte+t <without
atering the meaning=. Howe1er) where a word ma" be re'aced b" another inHa
'articuar conte+t) testees ma" easi" be mised into regarding
s"non"ms as being genera" interchangeabe.
&uideines for writing 5 ( (f the5'robem5area5being tested.is ocated in the o'tions. <as5in T"'e ,=)
items the stem shoud be ke't sim'e. (f) howe1er) the 'robem area is incuded
in the stem <as in T"'es 3 and ?=) the o'tions themse1es shoud be sim'e in
so far as the" shoud contain on" those 1ocabuar" items which the testees can
understand.
, Each o'tion shoud beong to the same word cass as the word in the
stem) 'articuar" when the word a''ears in the conte+t of a sentence. (f
this rue is obser1ed) there wi be ess danger of the conte+t 'ro1iding
im'ortant grammatica cues for the testees. >or e+am'e) athough the
f i rst of the foowing test items is usabe) o'tions .. B and C in the second
item woud be grammatica" incorrect when 'ut in the conte+t.
contem'tuous
.. dee' in thought C. sef5satisfied
B. without a sense of humour :. scornfu
(an was contem'tuous of the efforts of his friends to raise some mone"
for the charit".
.. dee' in thought C. sef5satisfied
B. without a sense of humour
:. scornfu
3 The correct o'tion and the distractors shoud be at a''ro+imate" the
same e1e of difficut".@ (f the correct o'tion is more difficut than the
distractors) the testees wi arri1e at the correct answer b" 'rocess of
eimination. Thus) the test ma" ha1e a negati1e effect on the testees! i.e.
the" wi seect the correct o'tion not because the" know it is correct but
on" because the" know the other o'tions are wrong. The foowing item
measures the testees@ knowedge of the distractors rather than their
famiiarit" with the correct o'tion!
theatrica
.. angr" B. histrionic C. 'roud :. fooish
The con1erse aso hods good. (f the distractors are more difficut than the
correct o'tion) the item ma" be eBua" unreiabe. (n such a case)
there wi usua"5be a tendenc" for the more abe students to think that the
correct o'tion is too eas" and therefore wrong! the" are thus tricked into seect
g one of the more difficut o'tions!
suffice
.. be adeBuate
B. harass C. acBuiesce :. be contrite
? There is some disagreement concerning the reationshi' of the o'tions tk
the 'robem area being tested. 0ome test writers argue that the o'tions
shoud be reated to the same genera to'ic or area) whie others 'refer as
wide a range of associations as 'ossibe. /ness the 1ocabuar" item being
tested has a 1er" ow freBuenc" count <i.e. is 1er" rare" used=) howe1er.
the item writer is ad1ised to imit the o'tions to the same genera area of
acti1it" where 'ossibe.@5
(tem 1 (tem5,
a''arition a''arition
..skeeton ..scener"
B.ghost
B.ghost
C.nightmare C.magician
:.cor'se
:.caste
(f item , were set in a test) students who had read a few ghost stories
woud 'robab" seect o'tion B because the" woud associate
a''arition
with the stories the" had read. (n item (. howe1er) students are reBuired t
show a much greater contro o1er 1ocabuar".
8 . the o'tions shoud be a''ro+imate" the same ength., There is a
tem'tation both in 1ocabuar" and in reading com'rehension tests to mak!
the correct o'tion much onger than the distractors. This is 'articuar"
true in a 1ocabuar" test item in which the o'tions take the form of
definitions! the item5writer tends to take great 'ains to ensure that the
o'tion is absoute" correct) Buaif"ing it at great ength. Howe1er) the
item5writer rare" takes such troube o1er the distractors) since the" are
deiberate" wrong and need not be Buaified in an" wa".
a hitch5hiker
.. a man who makes ro'es
B. a 'erson who tra1es about b" asking motorists to gi1e him free ride
C. an od5fashioned saior
:. a bo" who waks ong distances
.n" student who did not know Die meaning of
hitch5hiker woud cear"
choose o'tion B 5 and woud be correct in doing so. ConseBuent") if it is
e1er necessar" to Buaif" a definition at some ength) either one distractor
or a three or four distractors shoud be made eBua" ong. (n this wa")
the correct o'tion wi be disguised a itte more effecti1e".
(t is ad1isabe to a1oid using a 'air of s"non"ms as distractors! if the testees
recognise the s"non"ms) the" ma" reaise immediate" that neither is the correct o'tion)
since there can be on" one correct answer.
The od woman was awa"s
courteous when an"one s'oke to her.
.. 'oite B. gad C. kind :. 'eased
E1en such near s"non"ms as
gad and 'eased are sufficient to indicat)
to inteigent students that the choice must be between 'oite and kind)
since if gad were correct)
'eased woud 'robab" aso be correct.
(t is aso dangerous to D'air off@ o'tions b" 'ro1iding an anton"m as !
distractor. 9'tions . and C in the foowing 1ocabuar" item immediate)
stand out4 again) ce1er students wi be abe to narrow their choice down
to two o'tions once the" reaise that . means the o''osite of C.
ascend
.. go u' B. tak
C. come down :. fetch
8.3 *uti'e5choice
The guideines gi1en in 8., for constructing 1ocabuar" items a''" eBua"
items <B= for the &rou' B items now being treated. (n certain wa"s) the items shown
in this section are more difficut to construct than those in the 're1ious
section. The 'robem is chief" one of conte+t! too itte conte+t is
insufficient to estabish an" meaningfu situation) whie too much conte+t
ma" 'ro1ide too man" cues <both grammatica and semantic=.
1. ( saw a nast" ... between two cars this morning.
.. ha''ening B. danger C. damage :. accident
,. ( was s'eaking to Cath" on the 'hone when sudden" we
were ...
.. hung u' B. run out C. broken down :. cut off
3. ( shoud ha1e returned this book ast Tuesda"! it is now fi1e
da"s ...
.. 'ost'oned B. e+cessi1e C. o1erdue :. dea"ed
?. Nothing had been organised and confusion seemed ...
.. ine1ident
B. inefficient C. ineigibe :. ine1itabe
8. Tom awa"s tries to he' 'eo'e) but recent" he has been
. kind and generous.
.. chief" B. es'ecia" C. 'rinci'a" :. fundamenta"
*an" muti'e5choice 1ocabuar" test items of the t"'e being deat
with in this section re" on the conte+t itsef to 'ro1ide grammatica cues
which automatica" rue out at east one of the o'tions. These kinds of test
items are usefu in man" res'ects but ma" 'ossib" beong5more to tests of
grammar and structure rather than to 1ocabuar". Ne1ertheess) there can
be itte obJection to introducing) sa") a few items on 1erb 'atterning in a
test of 1ocabuar".
A. (@m . . of getting a new Job! ( don@t ike m" 'resent one.
.. contem'ating B. thinking C. desiring :. ho'ing
7. .nn ...
. me of a gir ( used to know.
.. recas B. reminds C. remembers :. recoects
(t is sometimes argued that mar.@ iTiuiii' c Cu1iCc 1ocabuar" tests
consist arge" of items such as the foowing and that these test on" a
knowedge of coocation.
3. The tee1ision station was ...
. with etters and 'hone cas
after the announcement.
.. drowned B. stormed C. deuged :. absorbed
0ince this item ignores the abiit" to create une+'ected coocations) it can
aso be argued that an imaginati1e use of the anguage is discouraged.
.though there ma" be some truth in this argument) une+'ected
coocations resut from a creati1e and intuiti1e handing of anguage)
which in turn demands an im'icit understanding of e1er"da" coocations. (t
is usua" the writer@s 1er" awareness of the degree of incongruit" which
makes a new coocation 1igorous and meaningfu.
.though the coocations in such items as the foowing ma" be tested
eBua" we without a conte+t) it is usua" ad1isabe to test them in
sentences.
8A
2. :r Heston charges a high ... . for his ser1ices.
.. fee B. 'rofit C. saar" :. 'a"ment
<Coocations being tested here) for e+am'e) are! charge a feeKmake a
'rofitKrecei1e a saar"Kmake or recei1e a 'a"ment 5 athough it is
'ossibe to charge a 'a"ment to an account.=
1-. ( don@t beie1e "ou! ( think "ou@re ...
ies.
.. sa"ing B. taking C. s'eaking
:. teing
11. (ron wi e1entua" ... ! . if grease is not a''ied.
.. wear B. corrode C. damage :. corru't
1,. *" dri1ing icence ... . at the end of this month.
.. e+'ires B. 'asses out C. retires :. concudes
(f se'arated from such conte+ts as the 'receding ones) these test items
woud read!
2. charge a feeK'rofitKsaar"K'a"ment
1-. sa"taks'eakte ies
11. iron wearsKcorrodesKdamagesKcorru'ts
1,. a icence e+'iresK'asses outKretiresKconcudes
(n this t"'e of item) howe1er) each conte+t reBuires a Dnorma@
reaction and takes no account of cutura differences. >or e+am'e) in the
foowing item B or : woud be correct in certain societies since it is
im'oite to acce't a gift without first 1ehement" refusing it.
Emma cried out with ... at the beautifu 'resent *rs ;hite
ga1e her.
.. deight B. horror C. disma"
:. anger
(n 1iew of such ambiguit") it is e1en more im'ortant than usua to
'ro1ide a conte+t for this 'articuar kind of item. The foowing is a L"'ica
e+am'e of this t"'e of muti'e5choice item as it a''ears in se1era
tests. The diaogue takes 'ace in a doctor@s surger" which has a
'harmac".
6./L LEE! Can "ou te me what time the doctor@s <1= ...
. o'ensG
*#0 $(N&! (t@s o'en now. The <,= ... . wi he' "ou.
6./L LEE! E+cuse me. ( Just want to coect a <3= ...
*#0 $(N&! (s it for some <?= ... . for a headacheG
6./L LEE! No) it@s for some cough <8= . . .
*#0 $(N&! Here it is. This shoud soon <A= ...
. "our bad cough.
<1= .. office B. surger" C. hos'ita
:. ward
<,= .. 'orter B. hostess C. waitress :. rece'tionist
<3= .. 'rescri'tion
B) reci'e C. cure :. direction
<?= .. 're1ention B. iBuid C. medicine
:. soution
<8= .. mi+ture
B. drink C. wash :. com'ound
<A= .. 're1ent B. treat C. refresh : cure
This is undoubted" the most common t"'e of muti'e5choice 1ocabuar"
item. The 'ro1ision of a conte+t) howe1er) imits the test constructor to
testing on" the 1ocabuar" associated with a 'articuar to'ic. Hence) man"
we5known tests sti tend to incude singe sentence 1ocabuar" items
rather than fu" conte+tuaised ones in order to co1er the range of
87
11
1
1ocabuar" considered@desirabe for sam'ing. The choice between the use
of singe sentences and the use of 'aragra'hs 'ro1iding a far fuer conte+t
wi be determined b" the 'ur'ose of the test and the test writer@s own
a''roach to the communicati1e as'ects of anguage earning. The Buestion
is sim'" whether the testing of anguage in conte+t is worth the sacrifices
demanded) and the answer must differ according to each 'articuar
situation.
8.? 0ets <associated *an" of the difficuties arising from the testing of coocations are a1oided
words= b" the testing of word sets. (n such tests the students@ famiiarit" with a
range of associations is measured.
T"'e 1! #ecognition
#ead each of the foowing ists of four words. 9ne word does not
beong in each ist5!56ut5a cTree round the odd word in each ist.
son ha''" arri1e
father married de'art
bo"
engaged go awa"
brother singe ea1e
T"'e ,! 6roduction
Each grou' of words is reated to a 'articuar subJect. ;rite down the
'articuar subJect which is connected with each grou' of words.
hand theatre
1oume nurser"
wrist sister track ift
dia
bed head so'e
face ward s'oo snow
<7 watch=
<7 hos'ita=
<7 ta'e recorder= <7 skiing=
8.8 *atching items
T"'e ( of the foiuwing est items suffers from testing e+ica items from
different word casses) whie T"'e , tests a mi+ed hag) of tense forms. etc.
The resut is that for both t"'es of test items grammatica cues assume
great im'ortance) since the" are instrumenta in imiting the range of
choices facing the testees for each bank. >or e+am'e. athough there ma"
a''ear to be ,- words for seection for bank <1= in T1'e t . in 'ractice
there are on" three which woud fit grammatica"!
turned <down=) broken
<down=) knocked <down=. 0imiar") in the first sentence of T"'e , there are
on" two o'tions
<'u through) get wa"=)
since a the other o'tions are
either 'ast tense forms or 'artici'es. Both items need to he rewritten)
therefore) if a higher degree of reiabiit" is to he obtained.
T"'e (
;rite the correct word from the foowing ist at the side of each number on
"our answer sheet. /se each word once on".
road accident tra1eing turned side
broken know
knocked midde
ooked
orr" 'oiceman 'a1ement running hurt
"ing
crossed eft forgot tak
6oor Tom ;right was <1= down b" a <,= ast week when he was crossing the
<3=. He was Buite bad" <?= and he5had to go into hos'ita for a few da"s.
His eft eg was <8= and both his arms were cut. ;hie he was <A= in. bed in
the hos'ita) a <7= came to <3= to him.
83
@;as the orr" <2= 1er" Buick"G@ he asked Tom.
Tom tod him a about the <1-=.
@( was <11= home from schoo and 1 <1,= the road. 1 <13= right but 1 <1?= to
ook <18=. (n the <1A= of the road ( sudden" saw a orr". ( didn@t <17= what
to do) so ( began to run to the other <13= of the road. The orr" <12= but it
hit me when ( was near the <,-=.@
T"'e ,
Com'ete the foowing sentences with the most suitabe 1erb 'hrase
from the ist.
came about 'u through
broken out faing out
running into brought u' get awa" 'ut off
1. @:id the 'risoner manage to ... G@ @%es) the 'oice are sti
ooking for him.@
,. The doctor thought *r Benson woud ... . after the
o'eration.
3. The cou'e are awa"s ... . and causing a disturbance.
?. .nd so it ... . that we e1entua" 'arted.
<etc.=
(t is much more efficient to test words from the same word cass <e.g.
nouns on" in T"'e 1=) or 'arae tense forms <e.g. the 'ast sim'e tense in
T"'e ,=. Thus the T"'e , item coud be rewritten as foows!
came about ran into 'ued through
got awa"
1. @( hear the 'risoner ... . "esterda" and the 'oice are sti
ooking for him.@
,. @;e were a reie1ed that *r Benson ... . after the
o'eration.@
<etc.=
T"'e 3
>rom the ist of words gi1en) choose the one which is most suitabe for
each bank. ;rite on" the etter of the correct word after each number on
"our answer sheet. </se each word once on".=.)
.. com'ete" C. busi" E. Buick"
B. 'oite" :. carefu" >. angri"
@;rite <1= . . @the teacher shouted <,= ... . @but don@t
waste time. %ou must get used to working <3= ...
@6ease) sir)@ a student said <?= ... (@1e finished.@
@No) "ou ha1en@t)@ answered the teacher. @%ou ha1en@t <8= ...
finished unti "ou@1e rued a ine at the end.@ *eanwhie) the bo" sitting
ne+t to him was <A= ... . engaged in drawing a ma'.
This t"'e is satisfactor" in man" wa"s because a the e+ica items
tested are ad1erbs. Howe1er) ike the other two t"'es) this t"'e gi1es the
student5too itte choice. >or instance) there wi be on" one word eft for
the ast number. Thus) it coud be im'ro1ed considerab" b" the addition of
a few other ad1erbs. The ist might then read as foows!
..com'ete" E.deiberate" (. Buick"
B.hea1i" >.busi" J.hard"
C.ab" &.hasti" $.angri"
:.'oite" H.carefu" L.suitab"
82

The first attem't to construct this ist incuded the ad1erbs
sient" and
aread") but it was then found that either of these coud be used at <A=
instead of the correct o'tion busi". This iustrates one of the dangers of this
'articuar testing de1ice! cear" the more distractors there are) the
greater is the chance that one of the distractors might be a correct o'tion for
at east one of the other items.
T"'e ?
The most usefu t"'e of matching item is undoubted" that based on a
reading com'rehension 'assage. The students are gi1en a ist of words at
the end of the 'assage and reBuired to find words of simiar meaning in the
'assage. 0ince a detaied conte+t is 'ro1ided b" the 'assage and itte
additiona materia is reBuired) this is an economica method of testing
1ocabuar". The chief. risk. here) howe1er) is the du'ication of Buestions! if
one of the reading com'rehension Buestions de'ends for its answer on a
knowedge of the meaning of a 'articuar word) care must be e+ercised
not
to
test that word again in the 1ocabuar" section.
(n most we5known tests in which this t"'e of item is incuded) a
different reading te+t from the com'rehension te+t is used as the basis for
the matching 1ocabuar" test. Thus) the reading te+t contains on"
Buestions on 1ocabuar" and does not incude com'rehension Buestions as
such. (n this wa" the test constructor can be sure that the abiit" to answer
the com'rehension Buestions does not de'end on a knowedge of the
indi1idua words seected for the 1ocabuar" test.
(n the foowing e+am'e3 candidates in the test are instructed to
re'ace the words isted beow with the a''ro'riate words contained in the
'assage without changing the meaning.
grou's
owned
s'ecific H H
H H H H H H H H
made u' H H
H H
chief
knowedge
simiar"
H H H H H
cose to each other
were incined
work together
THE TEH/ELCHE0
The Tehueches i1ed in a band 5 usua" of between fift" and a hundred
'eo'e. Each band had e+cusi1e rights to a 'articuar hunting area and
no other band was abe to hunt there without 'ermission. Each band
was com'osed of famiies reated through the mae ine and the man
who ed them was the hunter who had the greatest e+'erience of the
hunting grou's. Each man married a woman from another band and his
sister woud aso marr" men outside his band. (n this
wa" bands in a
neighbourhood were inked b" ties of marriage and so tended to co5
o'erate with each other in hunting and other tasks.
8.A *ore obJecti1e
This section contains e+am'es of t"'es of 1ocabuar" items which ha1e
items
a''eared in certain tests. ;hie T"'es 1 and , are usefu for cassroom
testing. T"'es 3 and ? are rather artificia) and shoud be a1oided where
'ossibe.
11
1
T"'e 1! ;ord formation test items
<a= ;rite a word in each bank. The word "ou write must be the correct
form of the word on the eft.
<i=C.#E Be ... . when "ou cross the road.
<ii= C#/EL
To mistreat animas is a form of ...
<iii= (NTE#E0T:o "ou think this book is ... G
<i1=ENTE# Can "ou show me the ...
. to the ca1eG
<b= #ewrite the foowing 'aragra'h 'utting in each bank the correct
form of the word in ca'ita etters.
*9*ENT
Can "ou s'are a ... G@ 6eter asked his brother. He thought he
coud detect a ... ook of im'atience on his eder brother@s
face) but it was gone in an instant.
@(@m 1er" bus" at the ...
.@ his eder brother said.
;hat is it "ou want to s'eak to me aboutG@ he asked 6eter.
6eter@s mind ... . went bank. @(@1e forgotten he said.
@;e) then it must ha1e been nothing of ... . im'ortance)@ his
eder brother said rather sarcastica".
T"'e ,! (tems in1o1ing s"non"ms
<a= ;rite in each s'ace the best word to re'ace the words underined in
each sentence.
<i= Tom went at once to the doctor@s.
immediate"
<ii= . of a sudden there was a oud cr".
<iii= ( came across an interesting book.
<i1= The boat is o1er fourteen feet in ength.
<b= (n each s'ace write one word that means amost the same as the
word on the eft. The word "ou write must rh"me with the word on
the right.
E+am'e! ear" soon moon
<i= 'urchase ... . die
<ii= miserabe ... . bad
. simiar item ma" be constructed so as to in1o1e anton"ms rather
than s"non"ms. The 'honoogica eement <rh"ming= in ,<b=) howe1er)
ma" on" confuse testees instead of he'ing them. ;ords are tested in
isoation) so) a'art from its sheer no1et") the item is of itte use and is not
to be recommended for most 'ur'oses. The acti1it" in1o1ed is more a
game than a test
T"'e 3! #earrangement items
#earrange the foowing etters to make words. Then use each word in a
sentence of "our own so as to show the meaning of the word.
6LE.6 #9L#% CEL6.
0/H9E (#.CH
E&/#.
.s can be seen) this item is itte more than a crossword 'uLLe. (t ma")
'erha's) be of some use in an inteigence test) but it is of doubtfu use in a
anguage test.
A1
((
8.7 Com'etion items
A,
(
T"'e ?! :efinitions
<a= /se each of the foowing words in a sentence so as to show the
meaning of the word.
econom" 'oitics industrious <etc.=
<b= E+'ain the meaning of each of the underined words in the
foowing 'hrases.
an archaic word a fortuitous e1ent
These item t"'es are of 1er" itte use. The" test writing abiit" in
addition to a knowedge of word meanings. >urthermore) it is e+treme"
difficut e1en for nati1e s'eakers to 'roduce sentences Dto show the
meaning@ of words 5 and it is certain" not a usefu task. . student ma"be
famiiar5with5the meaning5of aword and ma".usG).it.uorrect") without being
abe to e+'ress this meaning cear" in a sentence <es'ecia" under test
conditions=.
The foowing t"'es of com'etion items can be used for the testing of
1ocabuar". Tests which 'resent such items in a conte+t are genera"
'referabe to those which re" on singe words or on definitions.
T"'e 1
#ead through the foowing 'assage containing a number of incom'ete
words. ;rite each com'eted word on "our answer sheet at the side of
the a''ro'riate number. <Each dash re'resents one etter.=
0nakes are one of the <1= d5m5n55t grou's of <,= r5't5555! there
are at east ,)--- different <3= s'5c55s of snakes <?= sc5t5555d o1er
a wide area of the earth. Not a snakes are <8= '55s5 n55s! in fact) the
<A= m5J5555 " are Buite harmess. Contrar" to <7= '5'5(55 beief) a
snake@s <3= f55k5d tongue is not <2= d5ng 55555 to human beings! it
is mere" for touching and smeing <1-= s5bs55n55s. 0nakes
<11= in55ct 'oison into their) <1,= 1i5555@s bod" b" <13= b5t55g him
with their <1?= f55gs.
T"'e ,
<a= Com'ete each bank with the most a''ro'riate word to re'ace each
number in the te+t.
#osN.H! ;hat@s the <1= toda"G <1=
*9H.*E:! (t@s the se1enth.
#osN.H! .t what <,= does the concert startG <,=
*9H.*E:! 0e1en o@cock) ( think. Just a moment.
( made a note of it in m" <3=. <3=
#osN.H! How ong do "ou think it@ <?=G <?= H H H
*9H.*E:! (t finishes about ten.
#90N.H! That@s Buite a ong <8=) isn@t itG <8=
*9H.*E:! ( su''ose so. (t@s three hours.
Note the range of 'ossibe answers) es'ecia" with 3) ? and 8 <e.g. 3!
diar") notebook) e+ercise book4 ?! ast) take4 8! time) concert)
'erformance=.
<b= Com'ete the foowing 'aragra'h on 'robems caused b"
weightessness b" writing 9NE word in 'ace of each bank.
(ncreasing ...
. is now being ... . on the effects of
weightessness on man. >or ... . scientists are ..
. the
5roe of gra1it" on the wa" ces function. E1en in the first manned s'ace5
fights doctors were arge" unaware of the 1arious
'robems ... .. b" absence of gra1it". The" found that
weightessness ... . in the redistribution of bood and other
fuids from the egs to the to' of the bod". The" were abe
to . . how astronauts@ egs actua" shrank and their faces
sweed during the first few da"s of s'ace fight. *oreo1er) doctors had
a chance to ... . astronauts both at the time of the fight
b" ... . of tee1ision cameras and after the fight during
e+tensi1e medica ... . 0cientists are now ...
. into the
effects of diet and e+ercise as a ...
. of reducing some of the
'robems ... . b" weightessness . . most of the
fundamenta scientific Buestions wi ne1er be satisfactori" ... b"
scientists working on the Earth.
These test items come cose to the kind of item often used to test
reading com'rehension <described in Cha'ter 3=. Cear") a degree of
com'rehension is necessar" before each of the banks can be com'eted.
The items ha1e been incuded in this cha'ter because there is a deiberate
attem't to concentrate on the testing of 1ocabuar" 5 in the first cas5 the
1ocabuar" associated with information about the time and date) and in the
second case with the anguage of research and inBuir".
Notes and references
( &oodrich) H C 1277 :istractor Efficienc" in >oreign Language Testing. TE09L
Suarter" 11
Cohen) . : 123- Testing Language .biit" in the Cassroom.
Newbur" House
, 0ee Harris) : 6 12A2
Testing Engish as a 0econd Language.
*c&raw5Hi)
''. 8?587.
3 North ;estern #egiona .d1isor" Counci for >urther Education) .'ri 1233)
Engish as a 0econd Language 6a'er 3 C?
A3
i
Listening com'rehension tests
A.1 &enera .n effecti1e wa" of de1eo'ing the istening ski is through the 'ro1ision
of carefu" seected 'ractice materia. 0uch materia is in man" wa"s
simiar to that used for testing istening com'rehension. .though the
auditor" skis are cose" inked to the ora skis in norma s'eech
situations) it ma" sometimes be usefu to se'arate the two skis for
teaching and testing) since it is 'ossibe to de1eo' istening abiit" much
be"ond the range of s'eaking and writing abiit" if the 'ractice materia is
not de'endent on s'oken res'onses and written e+ercises.
.n awareness of the wa"s in which the s'oken anguage differs from
the written anguage is of crucia im'ortance in she testing of the istening
skis. >or instance) the s'oken anguage is much more com'e+ than the
written anguage in certain wa"s) as a resut of the arge eement of
Dredundanc"@ that it contains. .n e+am'e can be seen in the s'oken
Buestion DHa1e "ou got tc go nowG@) the Buestion being signaed b" the
rise in 'itch on go now and b" the in1ersion of the word order <i.e. b" both
'honoogica and grammatica features4. Thus)5if the istener did not hear
the Buestion signa Ha1e "ou) the rise in 'itch woud indicate that a
Buestion was being asked. (f the s'eaker surred o1er
got to) the Buestion
woud sti be inteigibe. (n addition) meaning might aso be con1e"ed)
em'hasised and @re'eated@ b" means of gestures) e"e mo1ements) and
sight changes in breathing. 0uch features of redundanc" as those described
make it 'ossibe for mutiated messages to be understood) e1en though the
fu message is on" 'artia" heard. >urthermore) the human brain has a
imited ca'acit" for the rece'tion of information and) were there no such
features buit into the anguage) it woud often be im'ossibe to absorb
information at the s'eed at which it is con1e"ed through ordinar" s'eech.
0uch con1ersationa features as re'etition) hesitation and grammatica re5
'atterning are a e+am'es of this t"'e of redundanc") so essentia for the
understanding of s'oken messages.
;hat is the significance of these features for testing 'ur'osesG >irst")
the abiit" to distinguish between 'honemes) howe1er im'ortant) does not
in itsef im'" an abiit" to understand 1erba messages. *oreo1er)
occasiona confusion o1er seected 'airs of 'honemes does not matter too
great" because in rea5ife situations isteners are abe to use conte+tua
cues to inter'ret what the" hear. .though isteners re" on a the
'honoogica cues 'resent) the" can often afford to miss some of them.
A?
0econd") im'rom'tu s'eech is usua" easier to understand than
carefu" 're'ared <written= materia when the atter is read aoud. ;ritten
tests genera" omit man" of the features of redundanc" and im'art
information at a much higher rate than norma s'eech does. ConseBuent")
it is essentia to make 'ro1ision for restating im'ortant 'oints) rewriting
5and re'hrasing them when writing materia for aura tests. The ength of
the segments in each breath grou' shoud be imited during dei1er") for
the onger the segment the greater the amount of information and the
greater the strain on the auditor" memor". The 'auses at the end of each
segment shoud aso be engthened to com'ensate for the ack of
redundant features.
.though not awa"s 'ossibe when istening com'rehension tests are
conducted on a wide scae) it is he'fu if the s'eaker can be seen b" the
isteners. Howe1er e+ceent the Buait" of a ta'e recorder) a disembodied
1oice is much more difficut for the foreign earner to foow. (n 'ractice)
most ta'e recorders are not of a high Buait" and are used in rooms where
the acoustics are unsatisfactor". (f the Buait" of the re'roduction is 'oor)
the test wi be unreiabe) es'ecia" when such discrete features as
'honeme discrimination) stress and intonation are being tested.
.'art from the use of 1ideota'e) howe1er) the ta'e recorder is the on"
wa" of ensuring com'ete uniformit" of 'resentation and thus a high degree
of reiabiit". (t is aso 'ossibe to use recordings made b" nati1e s'eakers
and thus 'resent 'erfect modes of the s'oken anguage 5 an im'ortant
ad1antage in countries where nati1e s'eakers are not a1aiabe to
administer the test. *oreo1er) ta'e recorders are essentia for the
'roduction and use of authentic materia.
>or 'ur'oses of con1enience) auditor" tests are di1ided here into two
broad categories! <i= tests of 'honeme discrimination and of sensiti1it" to
stress and intonation) and <ii= tests of istening com'rehension.
A., 6honeme T"'e 1
discrimination tests
<a= This t"'e of discrimination test consists of a 'icture) accom'anied b"
three or four words s'oken b" the e+aminer in 'erson or on ta'e.
1 , 3
The testees hear!
1. .. 'in
B. 'en
C. 'air :.'ain
,. .. shark B. sock C. sack :.shock
3. .. thin
B. tin C. fin :.din
.fter each grou' of four words the testees write the etter of the most
a''ro'riate word for that 'icture. >or e+am'e!
1. . ,. B 3. B
A8
<b= Con1erse") four 'ictures ma" be shown and on" one word s'oken. (n
this case) it is usua" better if the word is s'oken twice.
. B
C :
-
B
C
The testees hear!
1. 'ain 5 'ain <7 :=
,. sock 5sock<7 B=
<etc.=
T"'e ,
The testees hear three sentences and ha1e to indicate which sentences are
the same and which are different.
1. .. There@s a bend in the midde of the road.
B. There@s a bend in the midde of the road.
C. There@s a band in the midde of the road.
,. .. (s that sheet o1er there ceanG
B. (s that seat o1er there ceanG
C. (s that seat o1er there ceanG
3. .. (@1e Just ocked the car in the garage.
B. (@1e Just knocked the car in the garage.
C. (@1e Just ocked the car in the garage.
<etc.=
T"'e 3
<a= (n each of these items one word is gi1en on ta'e whie three or four
words are 'rinted in the answer booket. The testees are reBuired to
choose the written word which corres'onds to the s'oken word.
1. 0'oken! den
;ritten! .. ten
B. den C. Ben :. 'en
,. 0'oken! win
;ritten! .. when
B. one C. wane :. win
3. 0'oken! 'a"s
;ritten! .. 'a"s B. 'ra"s C. 'a"s :. bra"s
<etc.=
AA
<b= This t"'e of item is simiar to the 're1ious one4 this time) howe1er) the
words s'oken b" the tester occur in sentences. The four o'tions ma"
then be either written or s'oken.
1. 0'oken! (@ thread it for "ou.
;ritten or s'oken! ..
thread B. tread C. threat :. dread
,. 0'oken! :id John manage to catch the trainG
;ritten or s'oken! ..
drain B. chain C. 'ane :. train
3. 0'oken! 6ut the 'an in some hot water.
;ritten or s'oken! .. 'an
B. 'en C. 'in :. 'ain
<c= This item t"'e@ is simiiar to T"'e 3<a=! one word is s'oken b" the
tester <'referab" twice=. Howe1er) instead of a choice of four words)
testees ha1e in front of them a choice of four definitions. The" ha1e thus
to seect the correct definition for the word the" hear.
1. 0'oken! cot 5 cot
;ritten! .. sto''ed and hed
B. a bab"@s bed
C. 'ued b" horses
:. a sma 'et anima co1ered with fur
,. 0'oken! threw 5 threw
;ritten! .. made something mo1e through the air
B. not fase
C. some but not man"
:. made a 'icture or diagram on 'a'er
3. 0'oken! bud 5 bud
;ritten! .. 'art of a tree or a fower
B. a creature with wings
C. something to see' on
:. not good
The test items described in this section are a of imited use for
diagnostic testing 'ur'oses) eiiabing the teacher to concentrate ater on
s'ecific 'ronunciation difficuties. The items are 'erha's more usefu when
testees ha1e the same first anguage background and when a contrasti1e
ana"sis of the mother tongue and the target anguage can be used. *ost of
the item t"'es described are short) enabing the tester to co1er a wide
range of sounds.
T"'e 3<c=) howe1er) tests not on" the abiit" to discriminate between
the different sounds of a anguage but aso a knowedge of 1ocabuar". .
testee who ma" be abe to discriminate accurate" wi ne1ertheess find the
test 1er" difficut if he or she cannot understand the definitions in the
o'tions. 0imiar") T"'e 3<a= is a test of 'honeme discrimination and
s'eing abiit". (n T"'e 3<b= 'roficienc" in grammatica structure wi
fa1our the testee. Thus) for e+am'e) a testee who cannot discriminate
between thread) tread) threat
and dread ma" immediate" rue out the
distractors
threat and dread
since the" cannot be 'ut in the 'attern
(@
. it for "ou.
Each indi1idua test item in a the t"'es described must be ke't fair"
sim'e. 9bscure e+ica items shoud be a1oided. This ma" seem to be a
sim'e enough 'rinci'e to obser1e) but the a1oidance of difficut e+ica
items freBuent" makes it im'ossibe to test a the sound contrasts that
need to be incuded in the test. >or e+am'e) the contrasts shark)
sock)
A7
(
sack) shock woud not be suitabe for incusion in a test intended for
eementar" earners of Engish.
*uch of the materia in such tests is unfortunate" 1er" artificia)
differing great" from s'ontaneous s'eech. >reBuent" there is a tendenc"
for the tester to ado't a certain tone5'attern and rh"thm which ma" be a
source of irritation to the isteners or affect their concentration. Howe1er)
if the tester changes 'itch <e.g. i1e) ea1e) i1e= this wi on" confuse the
isteners. Thus) the tester must attem't to 'ronounce e1er" s"abe using
the same stress and 'itch 'atterns.
The abiit" to discriminate between certain 'honemes ma".sometimes
'ro1e 1er" difficut for nati1e s'eakers. *an" Engish diaects fai to make
some of the 1owe and consonant contrasts and thus) in addition to a the
other 1ariabes <e.g.5the acoustics of the room) the Buait" of the ta'e
recorder) etc.=) these tests are affected b" the 'ronunciation differences of
nati1e s'eakers.,
A.3 Tests of stress and .though features of stress) intonation) rh"thm and Juncture are genera"
intonation
considered more im'ortant in ora communication skis than the abiit" to
discriminate between 'honemes) tests of stress and intonation are on the
whoe ess satisfactor" than the 'honeme discrimination tests treated in the
're1ious section. *ost tests are im'ure in so far as the" test other skis at
the same time4 man" are aso 1er" artificia) testing the rarer <but more
@testabe@= features.
T"'e ( The foowing item t"'e3 is designed to test the abiit" to recognise
word stress or sentence stress. The testees isten to a sentence <usua"
s'oken on ta'e= and are reBuired to indicate the s"abe which carries the
main stress of the whoe structure. The" show the main stress b" 'utting a
cross in the brackets under the a''ro'riate s"abic.
0'oken! (@1e Just gi1en TH#EE books to Bi.
;ritten! (@1e Just gi1en three books to Bi.
< = < =9h = <M( < = 9< (
0'oken! *" >.ther wi he' "ou do it.
;ritten! *" father wi he' "ou do it.
9<M=9 = <
= 9 99
/nfortunate") this test acks conte+t and is 1er" artificia. (t tests on"
recognition of stress and is of imited use for ear5training 'ur'oses.
T"'e , The e+aminer makes an utterance and the testees ha1e to seect the
a''ro'riate descri'tion to indicate whether the" ha1e understood the
origina utterance. The utterance is s'oken once on") but the test is based
on the 'rinci'e that the same utterance ma" be s'oken in se1era different
tone5'atterns indicating a 'ain statement) a Buestion) sarcasm) sur'rise)
anno"ance. etc.
0'oken! Tom@s a fine goakee'er.
;ritten! Tom@s a fine goakee'er.
The s'eaker is
.. making a straightforward statement
B. being 1er" sarcastic
C. asking a Buestion
A3
0'oken! %ou wi@send me a cou'e of tickets.
;ritten! %ou wi send me a cou'e of tickets.
This is 'robab"
.. a reBuest
B. a command@
C. an e+'ression of disbeief
0'oken! (@ he' .nn.
;ritten! (@ he' .nn.
The s'eaker is
.. reuctant to he' .nn
B. eager to he' .nn
C. making a 'ain statement
This t"'e of test item is sometimes difficut to construct. 0ince the
conte+t must he neutra. it is often hard to a1oid ambiguit". There is aso a
danger of in1enting 5odd inter'retations or of concentrating on the rarer
meanings! e.g. sarcasm) iron") increduit". *oreo1er) it can be argued that
the item tests 1ocabuar" and reading com'rehension in addition to
sensiti1it" to stress and intonation.
A.? ia0tatements and
diaogues
These items designed
short sam'es a of s'eech 'and deaswith a 1ariet" ofa gnas on theceL ca and
gram grammatica e1es of 'honoog". The" ).re
1er"
suitabe for use in tests
administered in the anguage aborator"
but the" do not resembe natura
discourse. The s'ontaneit") redundanc") hesitations) fase
starts and
ungrammatica forms) a of which constitute such an im'ortant 'art of
rea5ife s'eech) are genera" absent from these t"'es of items sim'"
because the" ha1e been 're'ared 'rimari" as written anguage to be
read
aoud.
*oreo1er) the res'onses reBuired on the 'art of the isteners are not
communicati1e res'onses in an" sense at a. The isteners are not reBuired
to res'ond b" inter'reting what the" ha1e heard or b" adding further
information) as in rea ife. 0uch communicati1e res'onses. athough idea
for man" teaching situations) woud be difficut to incor'orate in such
istening tests) es'ecia" those intended for 'articuar diagnostic 'ur'oses.
Ne1ertheess) the im'ortance of such res'onses in tests of istening shoud
be borne in mind when communicati1e 'roficienc" tests are being
constructed 5 in other words) when the test writer is interested in finding
out what students can
do with the anguage the" are earning.
T"'e ( This item t"'e ma" be incuded in a test of grammar) a test of
reading com'rehension or a test of istening com'rehension) de'ending on
whether the item is written or s'oken. (t tests the abiit" to understand
both e+ica
a
hear a statement <usua" on t
a'e= and then choose the best o'tion from
four written 'ara'hrases.
0'oken! ( wish "ou@d done it when ( tod "ou.
;ritten! .. ( tod "ou and "ou did it then.
B. ( didn@t te "ou but "ou did it then.
C. ( tod "ou but "ou didn@t do it then.
:. ( didn@t te "ou and "ou didn@t do it then.
11
1
0'oken! (t took .an a ong time to find he coudn@t mend m" bic"ce.
;ritten! .. .fter a ong time) .an reaised he was unabe to mend m"
bic"ce.
B. .an s'ent a ong time mending m" bic"ce but he was at
ast successfu.
C. .an was a ong time before he found m" bic"ce.
:. (n s'ite of searching for a ong time) .an coudn@t find m"
bic"ce and) therefore) coudn@t mend it.
;hen constructing these items) it is ad1isabe to kee' the
grammatica) e+ica and 'honoogica difficuties in the stem) ea1ing the
written o'tions free of such 'robems and at a ower e1e of grammatica
and e+ica difficut" than the s'oken stimuus.
T"'e , These item t"'es are more5satisfactor" than T"'e 1 insofar as the"
are an attem't to simuate s'eech situations. The testees isten to a short
Buestion and ha1e to seect the correct res'onse from a choice of four
'rinted ones.
0'oken! ;h" are "ou going homeG
;ritten! .. .t si+ o@cock.
B. %es) ( am.
C. To he' m" mother.
:. B" bus.
Each o'tion shoud be so constructed as to a'
'ear correct in some wa"
to the testee who has not recognised the correct signas in the Buestion.
Thus) in the 're1ious item) o'tion . woud a''ear correct if the testee had
confused
;h" with ;hen) and o'tion : if he or she had @heard@
How
signa the Buestion. (f) on the other hand) a testee had faied com'ete" to
'ick u' the ;h5Buestion signa) he or she woud be tem'ted to seect
o'tion B) considering it the answer to a %esKNo Buestion.
The Buestion t"'es shoud be 1aried as much as 'ossibe and %esKNo
Buestions incuded as we as ;h5Buestions.
0'oken!:oes .ison mind "ou 'a"ing the 'ianoG
;ritten!.. %es) she@s awa"s thinking about it.
B. No) she rather ikes it.
C. No) she doesn@t 'a" the 'iano.
:. %es) she must be carefu.
(n this item two of the distractors <. and := are based on confusion
reating to mind in order to tem't an" testee who has faied to understand
the Buestion accurate". :istractor C has been incuded to attract an"
testee who has genera" misunderstood the Buestion and thinks it is about
.ison 'a"ing the 'iano.
(t can be argued that for certain tests the 'rimar" 'ur'ose of a
istening com'rehension item shoud be to test com'rehension aone and
not the abiit" to seect an a''ro'riate re'" to a stimuus. (t is 'ossibe
that a student who fais to answer this t"'e of item correct" ma" ha1e
actua" understood the statement or Buestion but faied to seect the
correct re'".
>urthermore) each5statement or Buestion which 'ro1ides the stimuus
in this t"'e of item usua" takes the form of an isoated item 'resented out
of conte+t and occurring with other unreated items. Conseniuent"
7-
11
A.8 Testing
com'rehension
through 1isua
materias
1
students who answer a these items in a test ha1e to accom'ish an
intricate seBuence of menta g"rations) sudden" Jum'ing from one
situation to another. Ne1ertheess) this item t"'e is usefu for se1era
'ur'oses in cass 'rogress tests 'ro1ided that the imitations indicated here are
recognised and the item t"'e is not o1er5used.
*ost of the item t"'es in this section are more a''ro'riate for the
eementar" stages of earning Engish. The" are) howe1er. 'referabe to
the discrimination items 're1ious" discussed as the" in1o1e the testing of
grammar and e+is through 'honoog". 6ictures) ma's and diagrams can he
used effecti1e" for testing such skis) thereb" making the testee@s
'erformance ess de'endent on other skis <e.g. s'eaking. 1ocabuar" and
reading=.
T"'e 1 (n this item t"'e a 'icture is used in conJunction with s'oken
statements. The statements are about the 'icture but some are correct and
others incorrect. The testees ha1e to 'ick o o ut the true <i.e.
statements and write T <or 'ut a tick 1= a side a''ro'riate '6ro6
numbers. The" ;rite > <or 'ut a cross ")
fase <i.e. incorrect= statements.
0'oken!
= at the side of the numbers of the
1. The orr"@s on the eft of the motorc"cist.
,! The car@s tra1eing in the same direction.
3. . dog@s running in front of the car.
?. . itte gir@s running after her mother.
8. 0he@s hoding a do.
A. Her mother@s carr"ing a bag.
7. The two bo"s are ooking in a sho' window.
3. . 1er" sma bo"@s he'ing the od woman.
2. The od woman@s going into a sho'.
1-. . ta man@s 'osting some etters.
11. There are a ot of cars in the street.
1,. The two bo"s are on the same side of the street as the itte gir.
@i
i
(
;ritten!
1. ,. 3. ?. 8. A.
7. 3. 2. 1-.
11.
1,.
T"'e , (n the foowing istening tests students ha1e fi1e 'ictures in front of
them. The" isten to four sentences) at the end of which the" are
reBuired to seect the a''ro'riate 'icture being described.
(
The testees hear!
1. Both car doors are o'en.
,. (t@s da"ight but both headam's are on.
3. The man who@s re'airing the car is "ing underneath it.
?. .though the gir sees the man working hard) she doesn@t he' him.
Thus the testees are abe to narrow down the choice a1aiabe to them as
foows!
1 . B C : E
<9n" . shows one door o'en=
,.B : E <9n" C shows the headam's off=
3.B : <9n" E shows the man standing u'=
?.: <9n" B shows the gir he'ing the man=
T"'e 3 The foowing t"'e of test item? is used in a number of istening
com'rehension tests. The testees see a set of three or four 'ictures and
hear a statement <or a short series of statements=) on the basis of which
the" ha1e to seect the most a''ro'riate 'icture. (n the test the testees
often see a tota of ten or twe1e such sets of 'ictures.
1.
3.
?.
The testees hear!
1. The car@s going to crash into a tree.
,. :ann" can@t run as fast as Caire.
3. Tom wishes his sister coud 'a" tennis with him.
?. The switch is so high that $atie can@t reach it.
T"'e ? 0im'e diagrams <consisting of ines) sBuares) rectanges) circes
and trianges= can be drawn to function as o'tions in a test of eementar"
com'rehension.8 (ustrations of this nature end themse1es in 'articuar to
testing such grammatica features as com'arison) 're'ositions and
determir. rs.
Look carefu" at each of the four diagrams. %ou wi hear a series of
statements about each of the diagrams. ;rite down the a''ro'riate
etter for each statement.
. B C :
9
9
-
0'oken!
1. .! LookR ;hat@s that inside the sBuareG
B! (t@s a white circe.
,. .! (s that a back circeG
B! ;hereaboutsG
.! .bo1e the sBuare.
B! %es) it is. (t@s a back circe abo1e the sBuare.
73
3. .! (s the white circe on the eft of the sBuareG
a! No) it@s on the right of the sBuare.
?. .! (s there an"thing at a in the sBuareG
a! No) it@s com'ete" em't". There@s neither a white nor a back
circe in the sBuare.
8. .! There@s nothing at a under the sBuare) is thereG
a! No) "ou@re wrong. There@s a back circe under the sBuare.
A. .! ;hat are "ou ooking atG
a! (@m ooking at a sBuare.
c! ;hich sBuareG
:! The one under the back circe) of course.
7. .! (s the circe on the eft of the sBuareG
555a!5No) it5isn@t5The5sBuare@s on the. eft of the circe.
3. .! ;hat@s unusua about this drawingG
B! ;e) there@s a back circe inside the sBuare.
.! That@s not unusua. There are two sBuares with back circes inside.
a! But this one (@m ooking at has a white circe Just outside the
sBuare.
. kinds of sha es and forms can be used to test isteningt)
(
com'rehension. The foowing e+am'e iustrates how an understanding C9
com'e+ structures can be tested in this wa". Howe1er) there is often a
tem'tation for the test writer to be too Dce1er@ and set an item testing
inteigence <or menta agiit"= rather than anguage acBuisition 5 as in the
foowing e+am'e.
.
B C
:
0'oken! (f the thick ine had been on" a miimetre onger) it woud ha1e
been the ongest of the three ines.
0hort con1ersationa e+changes as in the first e+am'e of T"'e ? are
far 'referabe to singe sentences <as in T"'es 1) , and 3=. . four t"'es)
howe1er) are scarce" 1aid tests of the abiit" to understand natura
discourse. Ne1ertheess) such items are of some use for certain 'ur'oses
and are usua" Buite reiabe guides to 'articuar as'ects of the istening
abiit".
T"'e 8 This test is designed to assess the testee@s abiit" to understand
sim'e instructions. .n" street ma' can be used or ada'ted for this
'ur'ose <see the e+am'e at the to' of the ne+t 'age=.
0'oken! %ou come out of schoo into Centra #oad and wak in the
direction of &reen Lane. Howe1er) "ou take the eft turning Just before
"ou reach &reen Lane. .t the end of the street "ou turn right and
continue unti "ou come to the second turning right.. %ou cross this road
and "ou wi see on "our right ... <;hich buiding wi "ou
seeG=
7?
11 1
;est >err"
East >err"
1
0ea1iew &ardens
North 0treet
6ost
9ffice
Centra #oad
0choo
Head .1enue
Cinema
&o1ernment
9ffices
0'orts &round
0tadium
The foowing istening com'rehension test is in the form of a
diaogue. The idea on which it is based was suggested b" an actua robber"
and a 'oice5 chase. (n this wa") the istener is gi1en a greater sense of
reaism and an added interest in the diaogue.
&reen Lane
>ootba 0tadium
Bramho'd .1enue .:
co
-
#egents Lane P
Centra
(ndoor
*arket
#itL Cinema
Cit" Ha
a;oo
9+ford 0treet
;ison@s
Jeweer@s 0ho'
( X n (
0outh 0treet
Centra
Librar"
#aiwa"
0tation
.! Ha1e "ou heard about the raid on the Jeweer" sho' in Cifton
#oadG
B! %es) in fact) ( saw 'art of the chase. (t was e+traordinar".
.! (@1e on" heard a 1er" 1ague re'ort about it. ;hat e+act"
ha''enedG
B! ;e) the thie1es 'anned to rob the sho' 5 "ou know).;ison@s in
Cifton #oad 5 Just after it@d o'ened ear" "esterda" morning.
78
i
.! (n broad da"ightG ( didn@t know that.
B! The" 'anned to arri1e as the Jeweer" was being taken from the
safe into the big dis'a" window. The" arri1ed in a arge red car
which the" 'arked on the o''osite side of the road. Can "ou see
the 'ace on this ma' (@1e gotG
1. +! ;rite the etter . on "our ma' at the 'ace where the thie1es
'arked the car.
.! How man" robbers were thereG
B! Three. 9ne waited inside the car and the other two waked o1er
to the sho') carr"ing arge briefcases. 9nce the" were in the
sho') the" made the manager and his assistant ie down on the
foor whie the" fied the briefcases with Jeweer". ;hat the"
didn@t know) though) was that another assistant was in the room
at the back of the sho'. He had caught sight of the two thie1es
and had 'ressed a sma aarm be. .t that 'recise moment) a
'oice 'atro car was at the Junction between *ain 0treet and the
road that runs 'ast the ibrar" 5 "ou know) near the #itL.
,. +! ;rite the etter B at the Junction referred to b" the s'eaker.
.! 0o things went wrong for the robbers from the startG
a! %es. B" the time the" were ea1ing the Jeweer@s) the 'oice car
was aread" turning into Cifton #oad. The two men hadn@t e1en
time to cose one of the car doors 'ro'er" as the" set off in the
direction of the footba stadium. . 'asser5b" heard one of the
men te the dri1er to take the first turning off Cifton #oad. Just
after that) one of the briefcases fe out as their car swung eft.
3. +! ;here did the thie1es ose one of the briefcasesG
;rite the etter C on the s'ot.
B! But that wasn@t the end of it a. ;hen the" ooked back) the" saw
the 'oice car gaining on them.
.! But the" didn@t gi1e u'G
a! No) the" acceerated. The" turned eft and then the" turned right.
Then the" swung into a narrow street and sto''ed a few "ards
down it at the side of a second car 5 most ike" their getawa"
car.
?. +! ;here was the getawa" car 'arkedG ;rite : on "our ma'.
.! Had the" managed to throw off the 'oice carG
B!
No. .s the" were about to change cars) the" heard it coming u'
behind them. 0o the" changed their minds and started off again
in the red car. .t the end of the narrow street) the" turned eft
into Hiiard #oad again and s'ed off in the direction of the
stadium. .t the ne+t but one Junction before the stadium 5 "ou
know) on the south side of it 5 a second 'oice car sudden" cut
across their 'ath and forced them to sto'.
8. +! ;rite E at the 'ace where the robbers were forced to sto'.
.! ;hat on earth did the" do thenG
B! ;e) b" this time) the" were rea" des'erate. The dri1er of the
red car got out and fired a 'isto at the 'oice car. But this didn@t
sto' the 'oice. 9ne of them scrambed o1er the bonnet of the
'oice car and chased the man with the 'isto down Hiiard #oad.
7A
11
1
The man ran in the direction of the Cit" Ha and then took the first
turning eft after #egent Lane. Then he made as if to turn right) but
Just at that moment) he tri''ed and fe. (n a matter of seconds)
two 'oicemen were on him.
A. +! ;here was the man caughtG ;rite the etter > to show the 'ace.
.! ;here were the other two robbers whie a this was ha''eningG
a! ;e) b" this time one was hafwa" down #egent Lane with two
'assers5b" chasing him. He si''ed through the narrow ae" at
the end of the ane and turned right before dashing across the
road. He got most of the wa" down the road to the station before
one of the 'assers5b" fina" caught u' with him and
o1er'owered him.
7. +! ;here was this robber caughtG ;rite the etter & to show the
'ace.
.! .maLingR ( su''ose the 'oice soon caught the third man.
a! No) the" didn@t. The third robber had a shotgun and he@d s'rinted
aong #egent Lane and into *ain 0treet. He was about to set uff
running in thedirection of the market. Then he caught sight of a
butcher@s 1an tra1eing towards him. He stood Buite sti in the
midde of the road) 'ointed his gun at the bewidered dri1er and
shouted to him to sto' and get out.
3. +! ;rite the etter H to show where the robber sto''ed the 1a51.
a! Then he got into the 1an) started off down *ain 0treet and
turned eft on" to find himsef in the midde of the bus stationR
He Buick" turned round and headed u' *ain 0treet. Ne+t he took
the road eading to the main entrance of the footba stadium.
Hafwa" down this road) howe1er) he saw two 'oicemen on
motorc"ces in front of him at the end of the road.
2. +! ;rite the etter ( where the two 'oice motorc"cists were.
.! ;e) he must ha1e been we and tru" cornered b" now.
a! %es) but he sti fired se1era shots at the motorc"cists. Then he
re1ersed and Jum'ed out of the 1an at the end of the road. He
turned in the direction of the Cit" Ha. He hadn@t got more than a
"ard or two when he found himsef surrounded b" a doLen
'oicemen.
1-. +! ;rite the etter J to show where@the third robber was caught.
.! .nd so at ast he was caughtR
a! %es) and so was the manager of the Jeweer" sho'.
.! ;hat on earth do "ou meanG
a! ;e) the 'oice ha1e Just5found that it wasn@t rea Jeweer" at a.
(t was imitation stuff. 0o !the Jeweer@s been arrested for fraudR
T"'e A There are man" other wa"s@of e+'oiting 1isua materias for testing
sim'e istening com'rehension. The foowing kind of item ma" be usefu in
the testing of the istening abiit" of students of mathematics.
1.
:raw a straight ine .B three centimetres ong.
,. Continue the ine .B to 'oint C so that .C is twice as ong as .B.
3. :raw a 'er'endicuar from 'oint B.
?. *easure an ange ?8P to be caed :.C.
5
77
i
(
8. Now draw the ine .: unti it meets the 'er'endicuar at 'oint :.
Suestion 1! How ong is .:G
A. Now Join :C.
Suestion ,! ;hat does ange .:C measureG
7. :raw a ine from 'oint BT'arae to .: and mark the 'oint M 11heTe it
bisects C:.
Suestion 3! How ong is BMG
3. Now draw a ine from M 'arae to .C so that it bisects .: at %.
Suestion ?! How ong is .%G
Suestion 8! How man" figures ha1e "ou drawnG
;hat are the"G
T"'e 7 .nother usefu test item <and e+ercise= which is inde'endent of the
s'eaking) reading and writing skis is that in which the testees are
'resented with an incom'ete 'icture <usua" a sim'e ine drawing= and
are reBuired to add to it 'ieces of 1isua information according to certain
ora instructions the" are gi1en. The foowing is an e+am'e of such an
item!
<The testees ook at the 'icture=
The testees hear!
1. :raw a tabe and two chairs in front of the cafe.
,. :raw two traffic ights on the o''osite side of the road.
73
3. :raw a Lebra5crossing between the 9+ford 0choo and the cinema.
?. :raw a orr" tra1eing in the o''osite direction to the car Just before
the Junction.
8. . 'oiceman directing traffic is standing in the midde of the Junction.
:raw him.
A. .though there@s on" one tree at the side of the office buiding) there
are two trees on the o''osite side of the road. :raw them.
7. 0ome 'eo'e ha1e com'ained about the danger of crossing the road
between the cafe and the cinema. . 'edestrian footbridge has now
been buit at this 'oint. :raw it.
3. . man who has been ceaning the windows of the second foor of the
office buiding o''osite the cafe has forgotten to take his adder awa".
(t@s sti eaning against the window on the e+treme right of the front of
the buiding. :raw it.
(t is cear" im'ortant to kee' an" such drawing sim'e so that too
much is not reBuired from the testees. Basic 'ractice in matchstick
drawings woud be a usefu 're'aration for such istening com'rehension
tasks in cass 'rogress tests. . sim'e countr" scene in1o1ing the drawing
of cows) trees) tents) 'ots and 'ans) ri1ers) bridges) fish) birds or an indoor
scene in1o1ing the 'ositions of furniture and sim'e obJects might form a
usefu basis for such a istening test. *oreo1er) it is im'ortant to tr" out
this kind of acti1it" before gi1ing it as a test. item to students. This kind of
're5testing of items wi he' to a1oid such 'robems as ea1ing students
insufficient room to draw a the 1arious obJects reBuired.
(t is aso usefu to buid u' an interesting stor" instead of imiting the
com'rehension test to instructions in singe sentences. The foowing
istening test) constructed b" a teacher some time ago) coud ha1e been
much more interesting had it been 'ut in the form of a sim'e stor" or
seBuence of e1ents.
5.a
Look at the drawing and isten to the sentences. %ou wi see *r 6eanut
sitting at a tabe. :o what ( te "ou and com'ete the 'icture. (t doesn@t
matter if "ou cannot draw 1er" we. .re "ou read"G Now begin.
1. *rs 6eanut is sitting on a chair o''osite *r 6eanut.
6/T HE# (N.
,. The door is behind *r 6eanut@s back.
6/T (T (N.
3. *r 6eanut can see *rs 6eanut but he can@t see Charie because
Charie@s standing behind him. 6/T H(* (N...
<etc.=
72.
The test was ater 'ut into the form of a short narrati1e and found to be far
more interesting!
9ne da" *r and *rs 6eanut were sitting down to ha1e a mea when
something strange ha''ened. *r 6eanut had his back to the door and
*rs 6eanut was sitting o''osite him. :raw them in "our 'icture.
<6ause=
Their sma 'et dog was asee' under the tabe hafwa" between them 5
a itte nearer *r 6eanut than *rs 6eanut. :raw the 'et dog.
<6ause=
0udden" the door was fung o'en and a "oung man rushed into the
room and stood behind *r 6eanut) hoding a ong knife. Can "ou draw
himG <6ause= <etc.=
;hen 'ut into the form of a sim'e narrati1e) the test at once
becomes interesting. The foowing diagram is used for a istening test
based on the reconstruction of an actua batte on a warehouse roof
between two "ouths and the 'oice.
*c :9N.t.9! n *c
H.##L0oni! JH
mrLE s ! J..
The testA consists of two 'eo'e discussing what ha''ened) minute b"
minute) in the e1ents eading u' to the shooting of a 'oiceman. 0tudents
are asked to 'ot the 'ositions of the 'oice and the "ouths on the roof of
the warehouse at 1arious times in this gun batte) according to the
information the" hear on the ta'e. 0e1era co'ies of the 'an of the
warehouse roof are gi1en to each student so that the" can draw the
'ositions of the indi1iduas at the 1arious times stated in the discussion.
(ndeed) it is on" through buiding u' a 'icture of the seBuence of e1ents in
this wa" that it becomes 'ossibe to gain an understanding of what rea"
ha''ened on the night of the shooting. .s a resut of attem'ting to do this)
students e+'erience a dee' interest and a rea sense of 'ur'ose when
istening to the discussion.
3-
T"'e 3 0e1era we5known e+amining bodies freBuent" use 'ictures based on
a istening com'rehension tak in order to test students@ abiit" to
identif" and cassif" information which the" ha1e heard. The foowing is an
e+am'e@ of such a t"'e of item.
(n this 'art "ou wi hear a tak about drawings made b" chim'anLees and
chidren. %ou wi hear information twice. Then "ou shoud!
1. identif" the nine sentences) from . to $ beow) which are about the
de1eo'ment of a chid. Number them in the order the" occur. ;rite
the numbers 152 in the bo+es.
,. cassif" the diagrams b" writing abo1e each diagram an a''ro'riate
s"mbo from the ke".
$e"
C: Chid on"
C6 Chim' on"
B Both
M Not in the two
stages described
.. sha'es inside circes
&. unconnected ines
B. singe ines H. e1er"da" obJects
11
C. human figures rows of sha'es
:. singe sha'es J. masses of connected ines
E. o1era''ing sha'es $. human faces

>. reguar 'atterns
T"'e 2 0im'e 'a'er5foding and drawing can aso be used to measure
abiit" to understand instructions. >or e+am'e!
<0'oken= 6rint "our name in the to' eft5hand corner of "our 'a'er! draw a
one5inch ine si+ inches under it. :raw a sma circe on the ine and on the
right of it draw a sBuare rough" the same siLe as the circe. Now
take the to' right5hand corner and the bottom eft5hand corner and fod "our
'a'er so that the drawing a''ears on the outside.
6ractica considerations) howe1er) shoud not he ignored in the
administration of this t"'e of test. .though usefu for ordinar" cassroom
'ur'oses) such tests are difficut to administer in 'ubic e+aminations.
Co'"ing is a sim'e matter and test reiabiit" ma" thus be great" affected.
31
(
*oreo1er) there is a tendenc" for such tests to become tests of inteigence
rather than of anguage 'roficienc". The item writer must be carefu to test
on" the student@s abiit" to understand a s'oken message 5 not the abiit" to
inter'ret it and see hidden reationshi's.
A.A /nderstanding The abiit" to understand both informa taks and forma ectures is an
taks and ectures
im'ortant ski for students stud"ing subJects in the medium of Engish at
intermediate and ad1anced e1es.
T"'e 1 Testees isten to a short tak and seect the correct answer about
the tak.
0'oken! There@s a marked tendenc" for most de1eo'ed countries to
grow steadi" noisier each "ear. This continua" increasing amount5of
noise is uncomfortabd and) what is more im'ortant) can affect our
heath. The noise of machines) hea1" traffic and aero'anes constitutes
'erha's the most serious threat to 'ubic heath. 0uch noise can
interfere with our abiit" to con1erse) it can disturb our see') and it care
Buick" make us become ner1ous wrecks. . oud bast or an e+'oYion
ma" e1en cause damage to our hearing. But there@s another danger 5
Just as great. This is the gradua damage which ma" be caused if we@re
continua" e+'osed to noise o1er se1era "ears. 0uch e+'osure to noise
can undermine our heath 5 as we as our 'erformance and efficienc".
>ortunate") technoog" is 'rogressing at a 1er" ra'id rate. 0ome
manufacturers are now designing new siencing mechanisms in their
'roducts) and 'anning e+'erts are e1en beginning to 'an cities
according to sound Lones.
;ritten! 9n" one of the foowing statements about the tak "ou ha1e
Just heard is correct. 6ut a circe round the etter ne+t to the correct
statement.
.. *odern technoog" is now making towns in de1eo'ing countries
free of oud noise.
B. The increase in noise is a 'robem which cannot "et be so1ed b"
modern technoog".
C. &radua noise o1er a ong 'eriod ma" ha1e Just as harmfu an effect
as oud or sudden noise.
:. There is no rea soution to the 'robem of increasing noise in
modern ife.
T"'e , Like T"'e ( this test combines istening com'rehension with
reading co4).'reiieii i@ n. The testees hear a short tak and then read a
summar" containing 'anks. The" must then com'ete the banks from the
tak the" ha1e heard. The danger here) howe1er) is that testees coud
successfu" com'ete the written summar" of the tak e1en if on" itte had
been understood.
0'oken! ;oud "ou ike a robot in "our houseG (t@s now genera"
acce'ted that in the future robots wi take o1er man" of our tasks)
es'ecia" Jobs of a re'etiti1e nature. But it@s high" doubtfu if robots
wi e1er be abe to do an" of the more creati1e t"'es of work 5 or
indeed if 'eo'e woud want them to. (n the home) robots woud
'robab" be used to do the ceaning) tabe5a"ing) scrubbing and
washing5u') but
it@s considered unike" so far that the"@ be used to do
the cooking 5 at east) not in the near future. .ccording to engineers)
11
1
robots wi do nothing more origina or so'histicated than the" ha1e
been 'rogrammed to do b" human beings. .nd so robots in the home
might not be creati1e enough to do the cooking 'an the meas) and so on.
The" woud be used as sa1es) thereb" freeing 'eo'e to do more of the
things the" wanted.
(n factories) mobie robots woud carr" out a the distribution and
assemb" tasks whie human beings carried out research and drew u'
'ans for new 'roducts. Human beings woud sti be res'onsibe for
diagnosing fauts and for re'airing and maintaining machiner". 9n the
farm) robots woud 'robab" dri1e tractors4 the"@d be 'rogrammed to
kee' their e"es on the ground. in front to guide the tractor aong a
straight ine or between rows of 1egetabes.
The robots themse1es woud 'robab" not ook at a ike human
beings because their design woud be chief" functiona. >or instance) it
woud not be at a sur'rising to find a robot with an e"e in the 'am of its
hand and a brain in one of its feetR
;ritten! The foowing 'assage is a written summar" of the short tak
"ou ha1e55Just5heard. &i1e the correct word which can be used in 'ace of
each number.
(n future <1= wi do man" Jobs) 'articuar" those which are <,= b"
nature..it is genera" doubted if the" coud do <3= work and in the home
the" woud 'robab" not do things ike <?=. #obots wi do nothing more
<8= than the" ha1e been <A= to do b" human beings. . robot woud be a
kind of <7=) freeing human beings so that the" coud do whate1er the"
wanted. .though robots woud be used in factories) human beings
woud 'robab" <3= the machiner". 9n farms) robots woud 'robab"
dri1e <2=. The robots woud ook <1-= human beings because the" woud
be <11= in design. (t woud e1en be 'ossibe for a robot to ha1e an e"e in
its hand or a <1,= in one of its feet.
T"'e 3 The testees hear a short tak or ecture and are reBuired to answer
Buestions on it. /ness the" are aowed to take notes on the tak) the test
ma" 'ut too hea1" a oad on the memor". (n certain instances) in fact) it
ma" be desirabe to gi1e them some guidance for note5taking. The
'ro1ision of a ist of 'oints on which Buestions wi be asked ma" im'ro1e
the test.
The foowing is an e+am'e of a test based on a <fictitious= no1eist4 the
Buestions that foow reate to the no1eist@s 'ace of birth) ear"
infuences on his chidhood) the books he read at schoo) his first
'ubications) his tra1es) etc. The sheet gi1en to each testee a few minutes
before the ecture reads as foows!
N9TE 6.6E#
%ou are going to hear a tak about Chares Edward Backwe) a writer of
chidren@s books. %ou are being tested on "our abiit" to isten and
understand. .fter the tak "ou wi be asked ,8 Buestions about Chares
Edward Backwe.
This sheet of 'a'er is for an" notes which "ou wish to take whie
"ou are istening to the tak. The notes wi not be marked in an" wa" b"
the e+aminer.
The Buestions "ou wi be asked after the tak wi be about the
'oints isted beow. . s'ace has been eft to enabe "ou to write notes for
each 'oint.
3G
((
1. ;hat Backwe enJo"s doing
,.. Backwe@s birth
3. His age at the time of the economic de'ression
?. The books Backwe read
<etc.=
The testees ma" take notes during the ecture. The" wi ater recei1e the
foowing answer sheet.
(N0T#/CT(9N0
%ou ha1e Just heard a tak about Chares Edward Backwe) a writer of
chidren@s books. %ou are being tested on "our abiit" to isten and
understand. %ou now ha1e 18 minutes to answer the Buestions which
foow. The ,8 Buestions foow the order of the tak and "ou shoud
com'ete each statement with the best answer. ;rite @.@) @B@) @C@) or @:@ on
the ine 'ro1ided at the side of each Buestion.
:9 N9T ;#(TE 9/T THE >/LL
.N0;E#.
Here is an e+am'e!
This tak is about
.. writers of chidren@s books.
B. chidren@s reading.
C. Chares Edward Backwe.
:. Leeds /ni1ersit".
1. Backwe enJo"s
.. writing books for chidren.
B. gi1ing ectures for writers.
C. reading books to chidren.
:. taking about himsef.
,. ;hen Backwe was born) his father was
.. a cricketer.
B. an inn5kee'er.
C. a writer.
:. a factor" worker.
3. .t the time of the great economic
de'ression Backwe was
.. three "ears od.
B. fi1e "ears od.
C. twent"5fi1e "ears od.
:. thirt" "ears od.
;hen Backwe was a bo") he read
.. books about chid geniuses.
B. Tosto"@s ;ar and 6eace.
C. stories written for bo"s of his age.
:. ad1ice about writing for chidren.
<etc.=
E+. C
1.
,.
3.
?.
This t"'e of tCZt is genera" administered in one of the foowing wa"s! 1
The testees recei1e note 'a'er and take notes whie the" isten to the
ecture. The" are then gi1en the Buestion 'a'er <usua" consisting of
muti'e5choice items=.
, The testees recei1e the Buestion 'a'er first and are gi1en a few minutes
to dance through it. The" then hear the ecture and work through the
r1
Buestions. The Buestions are genera" in the form of <a= muti'e5choice
items) or <b= trueKfase items) or <c= incom'ete sentences! Com'etion)
howe1er) is not usua" to be recommended as the testees are faced with
the tasks of istening) reading and writing simutaneous" 5 an e+treme"
difficut o'eration e1en for nati1e s'eakers. E1en muti'e5choice items
ma" cause confusion <es'ecia" if not carefu" s'aced out throughout
the ecture=) since the testees ha1e to isten whie reading carefu"
through a the o'tions and making their seection. (ndeed) if this
'articuar 'rocedure is to be ado'ted at a) it is 'erha's best to use
trueKfase t"'e items since this reduces the amount of reading and the
seection to be made.
3 The testees isten to the ecture and then recei1e the Buestion 'a'er.
The" read it through and then isten to the ecture gi1en a second time.
.though the testees wi be istening with a 'ur'ose during the second
reading of the ecture) the criticisms made 're1ious" sti a''".
*oreo1er) this test does not a''ro+imate as cose" to a norma ecturinO
situation as does method 1.
(n writing such tests) it is ad1isabe to gi1e a tak from rough notes or to
record a tak and then to work from a transcri't of the tak in setting
suitabe Buestions. The im'ortance of 'resenting rea s'eech instead of
written 'rose s'oken aoud cannot be em'hasised too strong". This) of
course) ma" be 1er" difficut for non5nati1e s'eaking teachers) but access to
radio broadcasts in Engish or recordings of taks gi1en b" nati1e
s'eakers wi be of rea he' in such cases.
Therewi ine1itab" he times when the non5nati1e teacher of Engish
wi be forced to use written te+ts as the basis of a istening com'rehensio
test. (n these cases) it is im'ortan) to kee' to the norma dei1er" rate)
increasing the ength of 'auses at the end of breath segments <i.e. units of
meaning such as 'hrases. causes and short sentences=. (n addition) the
written te+t itsef shoud be ada'ted to assume the features of s'eech as fa
as 'ossibe! the im'ortant 'oints can be restated in 1arious wa"s and
com'e+ sentences can be rewritten in the form of short com'ound or
sim'e sentences.
(t is most inad1isabe to attem't to introduce other essentia features
of s'oken discourse when ada'ting written te+ts for the 'ur'oses of
reading aoud. The more the test writer tries to incor'orate such features
as hesitation) fase starts) and ungrammatica sentences) the more artificia
the tak wi become. (n other words. the deiberate introduction of those
s'ontaneous eements inherent in a s'eech wi on" increase the
artificiait" of the situation. (n an e+'eriment conducted se1era "ears ago.
a re'ort about the e1ents in an imaginar" East .frican countr" was read
aoud and recorded for aPistt ring test b" a teacher 'a"ing the 'art of a
re'orter who had Just returned from the countr" in Buestion. The test
'ro1ed fair" successfu e1en though the students were reBuired at first to
isten to written discourse carefu" 're'ared and read aoud. The same
re'ort was then read aoud to another grou') the s'eaker deiberate"
introducing hesitation features) making fase starts) etc. Though taking
more time to dei1er and im'arting information. at a sower rate. the re'o
was not on" e+treme" difficut to understand but aso 1er" aboured and
irritating for the isteners. . third 'resentation of the same re'ort was th".
gi1en) but this time it took the form of an inter1iew. The @re'orter@ began
in a sef5conscious wa") acting his 'art and s'eaking we5rehearsed ines.
Howe1er) the 'erson conducting the inter1iew became so interested in the
subJect that he 'ut the scri't aside and asked Buestions which the Dre'orter@
was not e+'ecting. 0udden" the inter1iew came to ife and was far more
natura and s'ontaneous. The inter1iew itsef asted much onger than
either of the 're1ious taks and new Buestions were reBuired) but the
istening test had at once become 1aid.
. number of istening tests in1o1e e+tra5inguistic factors 5 memor")
knowedge of a to'ic and interest in that to'ic. (t is thus im'ortant to a1oid
testing memorisation of unim'ortant and irree1ant 'oints in a tak <e.g.
;hen was the writer@s grandfather bornG=. There is itte Justification for
setting Buestions on such 'oints in a reading com'rehension test 5 and far
ess in a test of istening com'rehension. The taking of notes aso
minimises the memor" factor) but the test itsef ma" then become more a
5555testof5note taking5skisOO5[
.bo1e a) remember that it is the 'ro'ositiona meaning of sentences
which is retained b" the istener <i.e. their genera meaning and intention= and
not the actua words or grammatica forms used to e+'ress that
meaning53 >or e+am'e) the genera meaning is sti the same whiche1er of the
foowing two sentences is used!
The wear" band of e+'orers managed to cross the wide ri1er in a 1er"
sma "acht.
The tired 'art" of e+'orers succeeded in getting to the other bank of the
great ri1er in a tin" dingh".
;e are rare" caed u'on to remember the e+act words someone s'oke in
rea ife uness in 1er" unusua circumstances) e.g. e1idence gi1en in a
court case) in which a s'eaker@s e+act words ma" ha1e great significance
E1en in such circumstances) indi1iduas usua" ha1e great difficut" in
recaing the actua words s'oken e1en though the" can remember
'erfect" the genera meaning of what the 'erson said. Therefore) a1oid
setting Buestions which in1o1e the memorisation of indi1idua words in
sentences. (f a summar" of a tak is gi1en for com'etion) the words
omitted in the summar" shoud be those words essentia to the meaning of
d
the whoe tak <e.g. the word Drobots@ in T"'e , in this section=.
T"'e ? (t is im'ortant that the content of the te+t itsef shoud determine
the t"'e of Buestion or item used to test com'rehension. There is nothing
intrinsica" either right or wrong about the use of muti'e5choice items for
a istening com'rehension test. (t is essentia) howe1er) that the most
a''ro'riate t"'e of item shoud be used. Certain te+ts wi end themse1es
far more to muti'e5choice items than others4 other te+ts wi end
themse1es to Buestions set in a tabuar form4 others wi suggest 1isuas.
whie "et others are better e+'oited if foowed b" o'en5ended Buestions.
The foowing e+am'e2 is incuded to show how trueKfase items can
be de1eo'ed to incude a third choice <no information a1aiabe=. This is a
'articuar" usefu de1ice in istening and reading com'rehension tests
since freBuent" the information being sought is not contained in a te+t. (t
is Just as usefu to test the abiit" to be aware of im'ortant information
not
gi1en in a tak as it is to test information gi1en in the tak. >urthermore.
the choice becomes no onger a two5wa" one but a three5wa" one) thereb"
reducing the effect of guessing.
3A
1
%es
No
7
Notes and references
(
,
The The ;e ha1e no
statement statement information
is true. is fase. about it.
The" had a 'robem
taking off because the"
were carr"ing so much
fue.
The" started on June
18th) 1212.
(t was fogg" when the"
took off.
The" had a 'robem
when something fe off
the 'ane during the
fight.
The" had difficut"
finding their wa".
The" had no heating in
the 'ane. . snowstorm
affected the engine.
The" were inJured when
the" anded.
There were a ot of 5
'robems on the fight.
0uch item t"'es ha1e been used in the /ni1ersit" of
E+aminations 0"ndicate Lower Certificate in Engish
but arednow noaonger used in the
>irst Certificate in Engish
e+amination.
Eisabeth (ngram re'orts that nati1e s'eakers of .merican Engish made u' to (t= 'er cent
errors in the ELB. 'honeme sub5test as com'ared with an a1erage of , 'er cent errors made
b" nati1e British s'eakers <Language Testing
0"m'osium)
9+ford /ni1ersit" 6ress 12A3=.
3 This item t"'e was de1ised b" Eisabeth (ngram for use in the
ELB. Test
<Engish Language Batter"=.
? #obert Lado made use of this techniBue in his
Test of .ura Com'rehension.
8
. 1ersion of this test item used to be incuded in the
&raded .chie1ement Tests
in Engish
<&.TE= oW thWH..merican Language (nstitute) &eorgetown /ni1ersit".
A The test item was written b" :a1id Bonam" and John Be1ere".
7 Joint *atricuation Board)
Test in Engish <91erseas=) 9ra 6a'er) June 1233. 3 >or further
detais) see #ichards) J C 1238 The Conte+t of Language Teaching.
Cambridge /ni1ersit" 6ress
2 The #o"a 0ociet" of .rts!
E+aminations in the Communicati1e /se of Engish as
a >oreign Language)
(ntermediate e1e) Test of Listening. *a" 1233.
37
rah 'roduction tests
7.1 0ome difficuties in
Testing the abiit" to s'eak is a most im'ortant as'ect of anguage testing.
testing the s'eaking Howe1er) at a stages be"ond the eementar" e1es of mimicr" and
skis
re'etition it is an e+treme" difficut ski to test) as it is far too@com'e+ a
ski to 'ermit an" reiabe ana"sis to be made for the 'ur'ose of obJecti1e
testing. Suestions reating to the criteria for measuring the s'eaking skis
and to the weighting gi1en to such com'onents as correct 'ronunciation
remain arge" unanswered. it is 'ossibe for 'eo'e to 'roduce 'ractica"
a the correct sounds but sti be unabe to communicate t it ideas
a''ro'riate" and effecti1e". 9n the other hand) 'eo'e don make
numerous errors in both 'honoog" and s"nta+ and "et succeed in
e+'ressing themse1es fair" cear". >urthermore) success in
communication often de'ends as much on the istener as on the s'eaker! a
'articuar istener ma" ha1e a better abiit" to decode the foreign s'eaker@s
message or ma" share a common ne+us of ideas with him or her. thereb"
making communication sim'er. Two nati1e s'eakers wi not awa"s.
therefore) e+'erience the same degree of difficut" in understanding the
foreign s'eaker.
(n man" tests of ora 'roduction it is neither 'ossibe nor desirabe to
se'arate the s'eaking skis from the istening skis. Cear") in norma
s'eech situations the two skis are interde'endent. (t is im'ossibe to hod
an" meaningfu con1ersation without understanding what is being said and
without making onesef understood at the same time. Howe1er) this 1er"
interde'endence of the s'eaking and istening skis increases the difficut"
of an 1 serious attem't to ana"se 'recise" what is being tested at an" one
time. *oreo1er) since the s'oken anguage is transient) it is im'ossibe
without a ta'e recorder to a''" such 'rocedures as in the marking of
com'ositions) where e+aminers are abe to check hack and make an
assessment at eisure. The e+aminer of an ora 'roduction test is working
under great 'ressure a the time) making subJecti1e Judgements as Buick"
as 'ossibe. E1en though sam'es of s'eech can he recorded during a test)
the ta'e5recording) b" itsef) is inadeBuate to 'ro1ide an accurate means of
reassessing or checking a score) since it cannot reca'ture the fu conte+t of
the actua situation) a of which is so essentia to an" assessment of the
communication that takes 'ace.
%et another <though not insu'erabe= difficut" in ora testing is that of
administration. (t is freBuent" im'ossibe to test arge numbers of students
because of the imited time in1o1ed. (t is not difficut to a''reciate the
huge 'robems reating to a test situation in which thousands of students
ha1e. to be e+amined b" a handfu of e+aminers. each student being tested for
a 'eriod of) Vsa") ten or fifteen minutes. .though the use of anguage
aboratories for such tests has made it 'ossibe in some cases to administer
more reiabe ora 'roduction tests to arge numbers of students) the actua
scoring of the tests has not been so easi" so1ed.
E+cuding tests of reading aoud and one or two other simiar tests.
ora tests can ha1e an e+ceent backwash effect on the teaching that takes
'ace 'rior to the tests. >or e+am'e) in one countr" the ora test was
retained as 'art of a schoo5ea1ing e+amination sim'" to ensure that at
east some Engish woud he s'oken in the ast two "ears of the secondar"
schoo 5 e1en though the test itsef was considered an unreiabe measuring
instrument as a resut of the arge number of unBuaified e+aminers who
had to administer it. >or this reason) and indeed) because ora
communication is genera" rated so high" in anguage earning) the testing
of ora 'roduction usua" forms an im'ortant 'art of man" anguage
testing 'rogrammes.
The foowing sections in this cha'ter wi gi1e an idea of the range of
'ossibe t"'es of ora tests. 0ome of the e+ercises <e.g. 'icture descri'tions=
ha1e 'ro1ed 1er" usefu in man" tests whie others <e.g. 'enci5and5'a'er
tests= ha1e met with 1ar"ing degrees of success. (n s'ite of its high
subJecti1it") an e+treme" good test is the ora inter1iew. (n man" cases)
one or two sub5tests <or ora acti1ities= are used together with the ora
inter1iew to form a com'rehen)i1e test of ora 'roduction skis.
7., #eading aoud
*an" 'resent5da" ora tests incude a test of reading aoud in which the
student is gi1en a short time to gance through an e+tract before being
reBuired to read it aoud. The abiit" to read aoud differs great" from the
abiit" to con1erse with another 'erson in a fe+ibe) informa wa".
.though reading aoud ma" ha1e a certain usefuness) on" a few
newsreaders and teachers ma" e1er reBuire training and testing in this
'articuar ski. The maJorit" of students wi ne1er he caed on to read
aoud when the" ha1e eft schoo. (t is a 'it") therefore) that students are
reBuired to sacrifice their enJo"ment of sient reading in order to 'ractise
reading aoud. ;e read 'rimari" for information or enJo"ment) and the
sient reading skis so necessar" for this 'ur'ose differ great" from those
of reading aoud. The backwash effects of this kind of test ma" he 1er"
harmfu) es'ecia" in areas where the reading skis are misguided"
'ractised through reading aoud. >ina") how man" nati1e s'eakers can
read aoud without making an" errorsG
Tests in1o1ing reading aoud are genera" used when it is desired to
assess 'ronunciation as distinct from the tota s'eaking skis. (n order to
construct suitabe tests of reading aoud) it is he'fu to imagine actua
situations in rea ife in which the testees ma" be reBuired to read aoud.
6erha's one of the most common tasks is that of reading aoud directions or
instructions to a friend) coeague or feow5worker! e.g. how to wire a 'ug)
how to trace fauts in a car engine) how to.cook certain dishes. >or e+am'e)
the foowing instructions reate to a situation in which a teacher or cass
monitor ma" be asked to read aoud!
>irst 'ut the headset on. *ake sure it is in its most comfortabe 'osition
with the headband o1er the centre of the head. The micro'hone shoud be
about it inches from the mouth.
32
((
To record) 'ut the white switch to the 'osition marked
;ork. 6ut the red
switch to
0'eak and 'ress the red recording button) which wi now ight
u'.
<etc.=
.nother situation which might occur in rea ife is that in which the
student is asked to read aoud <'art of= a etter he has recei1ed. >or a the
different e+tracts) howe1er) it is ad1isabe to draw u' certain features
which must be incuded in each 'assage! e.g. one %esKNo Buestion) one ;h5
Buestion) two sentences each containing a subordinate cause) one Buestion
tag) the 'honeme contrasts is5t) '5b. o5Q=!) etc. (n this wa") some degree of
consistenc" can be achie1ed.
. test more usefu in man" wa"s than reading aoud is the reteing of
a short stor" or incident. (n this t"'e of e+amination) the students are
reBuired to rete5a5ster5"5the" ha1e.Justsead. (f carefu" constructed) such5
a test can assess most of the 'honoogica eements which are otherwise
tested b" reading aoud. /nfortunate". it often measures other skis such
as reading com'rehension. memor" and organisation) too.
7.3 Con1ersationa
These dris are es'ecia" suitabe for the anguage aborator" and can ser1e
e+changes to focus attention on certain as'ects of the s'oken anguage) es'ecia" in
those countries where Engish is taught as a foreign anguage and the
5
em'hasis is 'rimari" on the reading skis. Howe1er) se1era of the test
items themse1es are far from communicati1e in an" sense at a and do not
aow for authentic interaction of an" kind. The essentia)eement of
constructi1e inter'a" with un'redictabe stimui and re'onses is ahsent
from a these items as a resut of the attem't to contro the interaction
taking 'ace. The item t"'es range from items 'resenting the testees with
situations in which the" initiate con1ersations to incom'ete comersations
with the 'art of one s'eaker omitted <i.e.a one5sided diaoDVue=. Tests
containing such item t"'es are on the whoe reiabe) but the" cannot h)O
described as being 1aid tests of s'eaking. (f an o''curtunit) is 'ro1ided in
other 'arts of the test for rea ora interaction <i.e. genuine con1ersation
and discussion=. howe1er) these controed test items can he of some use in
directing the attention of the students to s'ecific anguage areas and skits.
T"'e 1
The testees are gi1en a series of situations and are reBuired to
construct sentences on the ines of a certain 'attern or grou' of 'atterns.
.gain. it is essentia that two or three modes be gi1en to the testees sro
that the" know e+act" what is reBuired. <The testees read or hear the
situation and then make the a''ro'riate res'onses) shown in the hraekets.(
E+am'es!
*rs &reen i1es in a fat. 0he doesn@t ike i1ing in a fat and woud ike to
i1e in a sma house with a garden. <0he wishes she
i1ed in a sma
house with a 4arden.=
(t@s raining hea1i". Tom and .nna are waiting im'atient" at home to set off
on their 'icnic. <The" wish it woud
sto' raining.=
1. *r Back has a sma car but his neighbours a ha1e arge cars. He
woud ike a arBe car) too.
,. .nna hasn@t earnt how to swim "et but most of her friends can swim.
3. Tom is waiting for Bi outside the cinema. The show is Just about to
start but Bi has not arri1ed "et.
2-
?. *rs #obinson doesn@t. ike i1ing in towns4 she wants to i1e in the
countr".
<etc.=
T"'e , This t"'e of test item is simiar to the 're1ious t"'e but not as
strict" controed.\ No mode5res'onses are gi1en b" the e+aminer and the
students are free to use whate1er 'atterns the" wish.
. friend of "ours has forgotten where he has 'ut his gasses. He cannot see
too we without them. ;hat wi "ou sa" to himG <Let me he' "ou to ook
for them) etc.=
%ou are on "our wa" to schoo when it starts to rain hea1i".
/nfortunate") "ou and "our friend ha1e no raincoats. There is nowhere
to sheter but "our schoo is on" a hundred "ards awa". ;hat do "ou
sa" to "our friendG <0ha we make a dash for itGKLet@s run the rest of the
wa".=
1. %ou are tr"ing to get to the 'ubic ibrar" but "ou are ost. .sk a
'oice officer the wa".
,. %our friend has Just returned from a hoida" abroad. ;hat do "ou sa" to
himG
3. . waitress has Just brought "ou the bi but has totaed it u'
incorrect". ;hat do "ou sa" to herG
?. . friend of "ours wants to see a fim about a murder. %ou ha1e
aread" arranged to see it another e1ening) but "ou know she woud be
hurt if she knew. *ake u' an e+cuse.
T"'e 3 The students hear a stimuus to which the" must res'ond in an"
a''ro'riate wa". <This test often reies on con1entiona greetings)
a'oogies) acce'tabe wa"s of e+'ressing 'oite disagreement) etc.=
:o "ou mind if ( use "our 'enci for a momentG
<Not at aKCertain"K6ease doK&o ahead) etc.=
;hat about a game of tennisG
<%es) (@d o1e a gameK. right. ( don@t mindK:on@t "ou think it@s a bit too
hotG) etc.=
1. 6ease don@t go to a ot of troube on m" behaf.
,. 9h dear) it@s raining again. ( ho'e it sto's soon.
3. ;e shan@t be ate) sha weG
?. @$aren asked m. e to sa" she@s sorr" she can@t come tonight.
T"'e ? This is simiar to the 're1ious t"'e of item) but the stimui and
res'onses form 'art of a onger diaogue and the situation is thus
de1eo'ed. Because5e) its tota 'redictabiit") howe1er) this t"'e of item is
sometimes referred to as a diaogue of the deafR The man in the diaogue
beow continues regardess of what the testee5sa"s.
%ou@re on "our wa" to the su'ermarket. . man comes u' and s'eaks to
"ou.
*.N! E+cuse me. ( wonder if "ou can he' me at a. (@m ooking for a
chemist@s.
6./0E >9# TE0TEE@0 #E6L%
*.N! Thank "ou. :o "ou know what time it o'ensG
6./0E >9#. TE0TEE@0 #E6L%
*.N! Thanks a ot. 9h) er) b" the wa") is there a 'hone bo+ near hereG
21
6./0E >9# TE0TEE@0 #E6L%
*.N! 9h dear) (@ need some coins. :o "ou ha1e an" change for a ]8
noteG
6./0E >9# TE0TEE@0 #E6L%
*.N! ;e) thanks a ot. %ou@1e been most he'fu.
This diaogue cear" becomes absurd if) when asked where there is a
chemist@s) the testee re'ies) D(@m sorr") ( don@t know)@ and the man
'rom't" thanks him and asks what time it o'ens. Ne1ertheess) the use of
're5recorded materia of this kind makes it 'ossibe to use the anguage
aborator" to test arge numbers of students in a 1er" short time.
T"'e 8 This item@5 takes the form of an incom'ete diaogue with 'rom'ts
<shown in brackets in the foowing e+am'e= whis'ered in the students
%ou are at the rece'tion desk of a arge hote. The rece'tionist turns to
address "ou!
#ECE6T(9N(0T! Can ( he' "ouG
<%ou want to know if there is a singe5room a1aiabe.=
%ou!
#ECE6T(9N(0T! %es) we ha1e a singe room with an attached bathroom.
<.sk the 'rice.=
%ou!
#ECE6T(9N(0T! Thirt"5four 'ounds fift" a night.
<%ou want to know if this incudes breakfast.=
%ou!
#ECE6T(9N(0T! %es) that@s with continenta breakfast.
<%ou ha1e no idea what @continenta breakfast@ is.=
%ou.
#EcE6TiorasT! it@s fruit Juice) coffee or tea and bread ros.
<The rece'tionist is s'eaking too Buick". ;hat do "ou sa"G=
%ou!
#ECE6T(9N(0T! >ruit Juice) coffee or tea) and bread ros.
<Book the room for two nights.=
%ou.
#ECE6T(9N(0T! Certain". #oom ,1A. The 'orter wi take "our bag and show "ou
where it is.
<Thank the rece'tionist.=
%ou!
7.? /sing 'ictures for 6ictures) ma's and diagrams can he used in ora 'roduction tests in simiar
assessing ora wa"s to those described in the 're1ious cha'ter on testing the istening
'roduction skis. 6ictures of singe obJects can be used for testing the 'roduction of
significant 'honeme contrasts) whie a 'icture of a scene or an incident can
be used for e+amining the tota ora skis. This section wi concentrate on
the use of 'ictures for descri'tion and narration.
The students are gi1en a 'icture to stud" for a few minutes! the" are
then reBuired to describe the 'icture in a gi1en time <e.g. two or three
minutes=5. 9ccasiona") the number of words each student s'eaks is counted
b" one e+aminer in the room. whie the other e+aminer counts the number
of errors made. The score is thus obtained on the basis of the number of
words s'oken and the errors made <hut this 'rocedure is 1er"
unreiabe.= 0e'arate scores for genera fuenc". grammar. 1ocabuar".
L=
1
'honoog") and accurac" of descri'tionKnarration are far better.
.d1ertisements) 'osters and stri' cartoons ma" be used in this wa" for
cass tests. 'ro1ided that there are enough a1aiabe to 're1ent the students
from 're'aring one or two set 'ieces.
Carefu seection of the 'ictures used for the e+amination wi he' in
controing the basic 1ocabuar" reBuired and ma". to some e+tent)
determine the t"'e of sentence structure that 'redominates. :ifferent
st"es and registers can be tested b" incuding ma's and diagrams as we as
'ictures for com'arison) 'ictures for instructions and 'ictures for
descri'tion and narration. (f the 'ictures de'ict a stor" or seBuence of
e1ents) it is usefu to gi1e the testees one or two sentences as a @starter@)
thereb" famiiarising them with the tense seBuencing the" shoud em'o".
E+aminer! Last summer Luc" s'ent a few da"s with her unce and aunt in
the countr". ;hen it was time for her to return home) her unce and aunt
took her to the station. Luc" had made a ot of friends and she fet sad on
ea1ing them. 0he got on the train and wa1ed goodb"e to
them... Now "ou continue to te this stor".
23
((
((
The most effecti1e t"'e of ora e+amination using 'ictures reBuires not
on" narration or 'icture descri'tion on the 'art of the students but aso a
discussion about the 'icture<s= concerned. (f the e+aminer asks
Buestions and discusses the 'icture<s= with each student) the forma s'eech
situation is combined with the reci'roca s'eech situation and two different
t"'es of ora 'roduction skis can thus be measured. E1en if no discussion
is incuded in the e+amination) the e+aminer woud be we ad1ised to
'rom't the student whene1er he or she a''ears to need encouragement. (t
is awa"s im'ortant to find out what a student knows 5 not what he or she
doesn@t know! ong 'eriods of sience wi te the e+aminer 1er" itte.
. simiar techniBue3 to that described in the 're1ious cha'ter can be
used to test ora 'roduction. The student and the e+aminer ha1e fi1e
'ictures in front of them) each 'icture differing in on" one res'ect from
the other four 'ictures. The student is gi1en a card bearing a etter <.) B)
C) : or E=4 the e+aminer cannot see the etter. The student is reBuired to
describe the a''ro'riate 'icture <according to the etter=. The e+aminer
then seects a 'icture according to the descri'tion) assessing the)student not
on" on the correctness and fuenc" of his or her s'eech but aso on the
ength of time taken before the student@s descri'tion resuts in the
identification of the a''ro'riate 'icture. The e+aminer then checks the
card.
*
2?
.nother effecti1e wa" of assessing a student@s abiit" to) s'eak)
howe1er) is to gi1e 'airs or grou's of students a sim'e task to 'erform.
;orking in 'airs) students can describe their own 'icture before istening
to their 'artner@s descri'tion of a simiar <but not identica= 'icture. The
two students cannot see each other@s 'icture. The" can then discuss in
which wa"s the. two 'ictures are the same and in which wa"s the" differ.
The 'ictures used for such discussion ma" range from amost identica ones
to Buite different ones. The most successfu 'ictures for this acti1it")
howe1er) wi genera" de'ict fair" simiar subJects but differ considerab"
in their treatment of the subJect. The foowing two 'ictures of a harbour
ha1e been used 1er" successfu" to generate discussion between two
grou's of students.
.
B
;hen scoring students@ 'erformances) the e+aminer shoud
concentrate on what indi1idua students are doing with the target anguage
and how the" are using it to achie1e their 'ur'ose. Language errors which
interfere with successfu communication wi thus be 'enaised hea1i". 9n
the other hand) those minor errors which) though anno"ing in certain
res'ects) do not seem to im'ede communication to an" degree wi not be
'enaised in the same wa". Cear") the successfu outcome of the acti1it"
wi thus be im'ortant for both e+aminer and students.
(f students are e+amined in sma5grou's) one of the most usefu
acti1ities in1o1ing 'ictures is for them to be gi1en a seBuence of 'ictures
to rearrange. 0tudents shoud begin b" describing their own 'icture
without showing it to the other members of the grou'. .fter each 'icture
has been described and discussed in reation to the other 'ictures) the
grou' decides on an a''ro'riate seBuence. Each member of the grou'
then 'uts down his or her 'icture in the order decided u'on. Comments
are made and the order changed if the grou' considers it desirabe. .gain)
students are using anguage to achie1e a certain 'ur'ose in this t"'e of test.
The foowing e+am'e@ shows how sim'e it is to base such an acti1it"
on 'ictures aread" 'ubished) es'ecia" those foowing the comic stri'
'rinci'e.
28
Two mues
7.8 The ora inter1iew Like man" other e+aminations of ora 'roduction) the scoring of the ora
inter1iew is high" subJecti1e and thus. sometimes has on" ow reiahiit".(n
addition) the 'erformance of a student in a 'articuar inter1iew ma" not
accurate" refect his or her true abiit".
2A
0u''orters of the ora inter1iew caim that the e+amination at east
a''ears to offer a reaistic means of assessing the tota ora ski in a
Dnatura@ s'eech situation. 9thers) howe1er) argue that the e+amination
ne1ertheess is artificia and unreaistic! students are 'aced not in natura.
rea5ife s'eech situations but in e+amination situations. The" are thus
susce'tibe to 's"choogica tensions and aso to constraints of st"e and
register necessar" in such a situation. >or e+am'e) man" students ado't a
Buiet and coouress tone in inter1iews4 some e1en de1eo' a guarded
attitude) whie others become o1er5friend".
9ne soution to this 'robem is to ha1e the cass teacher as the
inter1iewer! if an e+terna e+aminer is reBuired. he or she ma" sit at the
back of the room or in an" other obscure 'ace. The inter1iewer <whether
teacher or e+aminer= shoud endea1our to 'ut the student at ease at the
beginning of the inter1iew) ado'ting a s"m'athetic attitude and tr"ing to
hod a genuine con1ersation <i!onstant" making his or her own contribution
without) at the same time) taking too much=. The inter1iewer shoud
ne1er
attem't to note down marks or comments whie the student. is sti engaged
in the inter1iew. The dua roe <i.e. of both anguage 'artner and assessor=
which the e+aminer is reBuired to assume in the ora inter1iew is awa"s a
most difficut one.
.nother soution to 'robems caused b" tension and anguage
constraints is to inter1iew students in 'airs or e1en threes) thus not on"
'utting them more at ease through the 'resence of a friend or cassmate
but aso enabing them to s'eak to each other as members of the same 'eer
grou'. ConseBuent") the whoe atmos'here wi become more rea+ed and
the constraints of register wi disa''ear) resuting in ess artificia and
stited anguage being used. 0tudents wi use the anguage which the"
norma" use in most s'eech situations in e1er"da" ife. No onger wi an
inferior <i.e. the student= be reBuired to address a su'erior <i.e. the
teacher= throughout the entire inter1iew. .though the teacher) or
e+aminer) ma" interru't or direct the discussion whene1er necessar") he or
she wi aso be abe to ado't a more 'assi1e roe in the discussion. (n this
wa") students wi fee free to con1erse and use anguage in a more natura
and 'ur'osefu wa".
The chief danger in conducting inter1iews with 'airs of students is that
resuting from 'ersonait" conficts or the dominance of one of the
members of the 'air. (t is therefore 1er" im'ortant for the teacher or
e+aminer to ensure that the two students forming a 'air ha1e simiar or
s"m'athetic 'ersonaities and ha1e simiar e1es of anguage abiit". The
anguage task itsef can aso he' since) if the two students are 'resented
with a rea 'robem to so1e) there wi be a greater chance of them co5
o'erating and working together in the target anguage. (n addition to the
use of 'ictures for com'arison and contrast) students can be gi1en sim'e
'uLLes and 'robem5so1ing tasks. . short BuiL ma" e1en be 're'ared b"
one of the students and gi1en to the other. (n a these cases) howe1er) the
e+aminer shoud be concerned more with the students@ use of the target
anguage to achie1e their goas rather than with their knowedge or their
actua abiit" to accom'ish the task gi1en. Thus) a student@s inabiit" to
answer se1era Buestions in a BuiL shoud not weigh too hea1i" 'ro1ided
that he or she is abe to gi1e some answers <right or wrong= and conduct a
reasonabe con1ersation) etc. with the student gi1ing the BuiL. (n short)
each student shoud be assessed not on" on such features of the s'oken
anguage as grammatica acce'tabiit" and 'ronunciation) but aso on
27
a''ro'riac" of anguage and effecti1eness of communication 5 and) where
a''ro'riate) the time taken to accom'ish the task gi1en.
The ora inter1iew shoud be scored on" after the student has eft the
room <uness two or more e+aminers are 'resent 5 in which case one of
them can sit behind the student and score=. .though.this settes the
'robem of when the inter1iew shoud be scored) the Buestion of how it
shoud be scored sti remains. >or e+am'e) how shoud the re'ies to this
Buestion be scoredG
<Tester= @;hat are "ou going to do this weekendG@
<0tudent .= @(@m Buite we) thank "ou.@
<0tudent B= @( go to fish. ( fish in ri1er near the big wood.
.@s re'" is 'erfect" correct but it is ne1ertheess Buite ina''ro'riate! the
student sim'" hasn@t heard.)n !)understood the Buestion correct". 9n the
other hand. B@s re'" shows a rea attem't to answer the Buestion but
unfortunate" contains se1era errors.
The scoring of the inter1iew can range from an im'ression mark to a
mark arri1ed at on the basis of a fair" detaied marking scheme <showing
accurac" of 'ronunciation) grammar) 1ocabuar" a''ro'riac") fuenc" and
ease of s'eech=. The foowing marking schemes <using a A5'oint scae= is
gi1en as Just one e+am'e of a number of such schemes in 'resent5da" use.
#ating .biit" to communicate ora"
A E+ceent! on a 'ar with an educated nati1e s'eaker. Com'ete" at
ease in his use of Engish on a to'ics discussed.
8 Eer" good! athough he cannot be mistaken for a nati1e s'eaker) he
e+'resses himsef Buite cear". He e+'eriences itte difficut" in
understanding Engish) and there is no strain at a in
communicating with him.
? 0atisfactor" 1erba communication causing itte difficut" for nati1e
s'eakers. He makes a imited number of errors of grammar) e+is
and 'ronunciation but he is sti at ease in communicating on
e1er"da" subJects. He ma" ha1e to correct himsef and re'attern his
utterance on occasions) but there is itte difficut" in understanding
him.
3 .though 1erba communication is usua" fair" satisfactor") the
nati1e s'eaker ma" occasiona" e+'erience some difficut" in
communicating with him. #e'etition) re5'hrasing and re5'atterning
are sometimes necessar"4 ordinar" nati1e s'eakers might find it
difficut to communicate.
,
*uch difficut" e+'erienced b" nati1e s'eakers unaccustomed to
@foreign@ Engish. His own understanding is se1ere" imited) but
communication on e1erda" to'ics is 'ossibe. Large number of
.errors of 'honoog") grammar and e+is.
1
E+treme difficut" in communication on an" subJect. >aiure to
understand adeBuate" and to make himsef understood.
Note that an e1en numbered scae is often 'referred because it he's
e+aminers to a1oid awarding the midde mark <a tendenc" in man" cases=.
Thus) athough marks ma" custer round the median 3 on a 85'oint scae)
e+aminers using a A5'oint scae wi ha1e to decide whether to award 3 <Just
beow the midde 'oint on the scae= or ? <Just
abo1e the midde 'oint on
the same scae=. (t is aso ad1isabe to a1oid a narrow scae <i.e. a ?5'oint
scae= as this wi not aow for the range of discriminations "ou ma" wish
to make. 9n the other hand) a wide scae <e.g. a ,-5'oint scae= is not
recommended as few markers seem to make use of either the u''er or ower
ends of this scae) most scores tending to custer around 251,.
*an" e+amining bodies 'refer fair" short descri'tions of grades in
order to enabe the e+aminer to gance Buick" through the marking
scheme. E+aminers who are faced with a ot of reading in the assessment
wi be tem'ted to re" soe" on the numerica grade itsef <e.g. 8= 5 which
makes their scoring e+treme" subJecti1e and iabe to fuctuation.
;here1er 'ossibe) it is usefu in 'ubic e+aminations to ha1e two or more
e+aminers istening for 'articuar areas or features before ater 'ooing
their assessments. The im'ortance of e+aminers@ meetings and D'ractice@
inter1iews <using ta'e recorders and marking guides= cannot be too great"
em'hasised. 0uch sessions) when conducted with a arge number of
e+aminers) are of considerabe he' in increasing the marker reiabiit" of
ora inter1iews. 0am'e recordings and scores are discussed and some
degree of standardisation of marking is thus achie1ed.
The 're1ious 'Lragra'h arge" concerns 'ubic e+aminations or
achie1ement tests set outside the cassroom. >or most cassroom and
schoo tests) howe1er) the teacher shoud de1ise his or her own rating scae
after ha1ing carefu" considered the e1e and kind of ora skis the
students shoud be e+'ected to achie1e. The o'timum 'erformance
e+'ected) therefore) wi not necessari" be near nati1e5s'eaker fuenc".
Cear") students who ha1e been s'ending a few hours a week earning
Engish o1er a 'eriod of two "ears cannot 'ossib" be e+'ected to achie1e
band A on the scae incuded here! hence the scae itsef woud be arge"
ina''ro'riate for the 'ur'oses of most cassroom tests of ora Engish.
(n order to de1ise a suitabe scae) the teacher shoud first begin to
describe cear" the criteria for assessing ora abiit". The teacher ma") for
e+am'e) wish to consider each student@s achie1ement in terms of accurac".
a''ro'riac" and fuenc"4A accurac") a''ro'riac") range) fe+ibiit" and
siLe47 or fuenc") com'rehensibiit") amount of communication) Buait" of
communication) and effort to communicate.@ ;hate1er the criteria
seected) the teacher shoud begin b" describing in one or two sentences
e+act" what he or she e+'ects the a1erage successfu student to ha1e
5achie1ed under each5of the headings b" the time the test is taken. These
descri'tions wi then form band ? <if a A5'oint scae is being used=. The
teacher shoud then re'eat the same 'rocedure for the student who is
sight" beow a1erage in his or her e+'ectations <i.e. band 3=. Ne+t) the
teacher writes the descri'tions for the successfu student <abo1e a1erage=)
assigning these descri'tions to band 8 on the scae. .fter this) the teacher
describes the 'erformance of the unsuccessfu student who is beow
a1erage <band ,=) then the 'erformance of the 1er" successfu student
<band A=) and fina" that of the east abe and most unsuccessfu student
<band 1=.
The foowing is an e+am'e of a teacher@s rating scae for the ower
in
termediate e1e.
22
i
.
A 6ronunciation is on" 1er"
sight" infuenced b" the
mother5tongue. Two or three
minor grammatica and e+ica
errors.
8 6ronunciatior is sight"
infuenced b" the mother5
tongue. . few minor
grammatica and e+ica errors
but most utterances are
correct.
? 6ronunciation is sti
moderate" infuenced b" the
mother5tongue but no serious
'honoogica errors. . few
grammatica and e+ica errors
but on" one or two maJor
errors causing confusion.
3 6ronunciation is infuenced b"
the mother5tongue but on" a
few serious 'honoogica
errors. 0e1era grammatica
and e+ica errors) some of
which cause confusion.
, 6ronunciation serious"
infuenced b" the mother5
tongue with errors causing a
breakdown in communication.
*an"@basic@ grammatica and
e+ica errors.
1 0erious 'ronunciation errors
as we as man" @basic@
grammatica and e+ica errors.
No e1idence of ha1ing
mastered an" of the anguage
skis and areas 'ractised in
the course.
>uenc"
0'eaks without too great an
effort with a fair" wide range of
e+'ression. 0earches for words
occasiona" but on" one or two
unnatura 'auses.
Has to make an effort at times to
search for words. Ne1ertheess)
smooth dei1er" on the whoe
and on" a few unnatura 'auses.
.though he has to make an
effort and search for words)
there are not too man" unnatura
'auses. >air" smooth dei1er"
most". 9ccasiona"
fragmentar" but succeeds in
con1e"ing the genera meaning.
>air range of e+'ression.
Has to make an effort for much
of the time. 9ften has to search
for the desired meaning. #ather
hating dei1er" and fragmentar"
#ange of e+'ression often
( nited.
Long 'auses whie he searches
for the desired meaning.
>reBuent" fragmentar" and
hating dei1er". .most gi1es u'
making the effort at times.
Limited range of e+'ression.
>u of ong and unnatura
'auses. Eer" hating and
fragmentar" dei1er". .t times
gi1es u' making the effort. Eer"
imited range of e+'ression.
1
Com'rehensibiit"
Eas" for the istener to
understand the s'eaker@s
intention and genera meaning.
Eer" few interru'tions or
carifications reBuired.
The s'eaker@s intention and
genera meaning are fair" cear. .
few interru'tions b" the
istener for the sake of
carification are necessar".
*ost of what the s'eaker sa"s is
eas" to foow. His intention is
awa"s cear but se1era
interru'tions are necessar" to
he' him to con1e" the message
or to seek carification.
The istener can understand a ot
of what is said) but he must
constant" seek carification.
Cannot understand man" of the
s'eaker@s more com'e+ or
onger sentences.
9n" sma bits <usua" short
sentences and 'hrases= can be
understood 5 and then with
considerabe effort b" someone
who is used to istening to the
s'eaker.
Hard" an"thing of what is said
can be understood. E1en when
the istener makes a great effort
or interru'ts) the s'eaker is
unabe to carif" an"thing he
seems to ha1e said.
;hate1er the criteria chosen) the brief descri'tions can be made much
more s'ecific at each e1e in order to refect the contents of the course
being foowed. The most im'ortant 'oint to bear in mind) howe1er) is that
for most cassroom 'ur'oses the rating scae shoud not ha1e nati1e5
s'eaker 'erformance as the desired coa. (nstead) it shoud be based on
reaistic e+'ectations of what successfu earners can achie1e at a 'articuar
stare in their de1eo'ment.
>ina") ora inter1iews do not sim'" ha''en s'ontaneous".
.though each ora inter1iew shoud simuate as natura and reaistic a
s'eech situation as 'ossibe) it is essentia that a certain amount of materia
be 're'ared beforehand. The inter1iew ma" be carefu" structured or ma4 be
structured Buite oose"! in both cases) the e+aminer shoud ha1e 'ent" of
materia on which to fa back if necessar". He or she ma" ead in to the
inter1iew b" asking a %esKNo Buestion) foowed at some stage or other b"
certain ;h5Buestions and Buestion tags.
There are dangers) howe1er) in adhering to a 1er" rigid structure or
'an. >or e+am'e) a student ma" de1eo' Da certain to'ic and be
'roceeding ha''i" in one direction when the e+aminer interru'ts and
sto's the whoe fow of the inter1iew in order to incude
a How often
Buestion. .gain) the demand for certain set res'onses ma" reach an absurc
stage! in a certain ora e+amination) for e+am'e) it was agreed that
;hereabouts . . G ought to signa a different res'onse to
;here .
ConseBuent". the more he'fu students) who antici'ated the in
ter1iewer.
Dfaied@ on this item!
e.g.
(NTE#E(E;E#! ;hereabouts do "ou i1eG
<#eBuiring the name of the
district where the testee i1ed .
0T/:ENT! 3?1 $ing@s #oad) North 6oint.
Had the student re'ied
North 6oint in a natura s'eech situation) the othc 'erson
woud 'robab" ha1e asked! ;here<abouts= in North 6ointG (n the inter1iew the student
mere" antici'ated such a reaction <conscious" or subconscious"= and ga1e a fu answer
initia".
6ro1ided that fe+ibiit" can be retained) it is usefu to 're'are a serie of Buestions
on a wide range5 of. to'ics. The foowing ist of ,- to'ics is gi1en here to he') not to
inhibit) the e+aminer!
fami") home) schoo) s'orts) hobbies) books) fims) trans'ort)
weekends) hoida"s) radio) heath) teeth) sho''ing) traffic) crime)
friends) mone") fines) careers) etc.
.t east ten or twe1e Bi estions can be asked on each to'ic) but the
e+aminer shoud ne1er attem't to @work through@ ists of Buestions.
(ndeed) the e+aminer shoud contribute to the inter1iew from time to tim4O
E+am'es of the t"'es of Buestions which can be asked on one of the to'i)
are!
0'orts and games
:o "ou 'a" an" gamesG
;hat@s "our fa1ourite s'ortG
How often do "ou 'a"G
.re "ou in "our schoo teamG
:o "ouike watching s'ortG
;hich do "ou think is the most interesting s'ort to watchG
Can "ou swimG
How did "ou earn to swimG
;hereabouts do "ou go swimmingG
;hich game woud "ou ad1ise me to take u'G ;h"G
;hich is the most difficut game to 'a"G
;hich is better as an e+ercise! basketba or footbaG
;here considered necessar") current affairs and high" contro1ersia
issues ma" be introduced in an inter1iew to stimuate or 'ro1oke a stude!
'ro1ided that some aowance is made for the emoti1e content of the
discussion. (t is e1en more im'ortant here that the e+aminer shoud remain
fe+ibe and 1ar" the range of Buestions for discussion.
(f the ora inter1iew is recorded on ta'e) the e+aminer can score the
inter1iew at eisure) 'a"ing and re'a"ing sections where necessar".
E+tensi1e notes can be taken from the recordings) com'arisons made and
confirmation sought where there is doubt concerning a 'articuar mark. (f
inter1iews are recorded) the e+aminer must of course take care to identif"
each student at the beginning of the inter1iew) es'ecia" when
com'arisons in1o1ing rewinding and re'a"ing are to be made.
7.A 0ome other
techniBues for ora
e+amining
The shor5t takO
&rou' discussion and
roe 'a"ing
i S,
(n certain e+aminations students are reBuired to 're'are a short tak on a
gi1en to'ic. The" ma" be aowed se1era da"s or on" a few minutes in
which to 're'are the tak and) in some cases) the" ma" be 'ro1ided with
notes or reference materia. This is cear" a reaistic test of sustained
s'eech but it constitutes an e+treme" difficut e+amination for second5
anguage earners at a but the most ad1anced stages. (ndeed) this
'articuar t"'e of e+amination is genera" 1er" difficut for first5anguage
s'eakers. The e+amination can be im'ro1ed sight" b" reducing the time
aotted for the tak and asking students Buestions based on their tak) thus
introducing a reci'roca s'eech situation. The Buestions might be asked either
b" the e+aminer or b" a grou' of students <if the tak is gi1en in front of an
audience=. (n whate1er situation the tak is gi1en) howe1er) the e+aminer
must make e1er" attem't to 'ut the s udents at ease.
Care must be taken to 're1ent students from earning whoe sections of
their tak! 0ubJects about which an indi1idua student knows 1er" itte
shoud be a1oided. E+'erience of such e+aminations has shown that
candidates tak better when the" ha1e something worthwhie to sa" and can
bring into the tak a genuine interest in the subJect cou'ed with e+'erience
and imagination. . co5o'erati1e audience aso he's great".
Eague subJects are best a1oided4 man" to'ics are best 'resented as
Buestions!
0houd cJuntries s'end huge sums of mone" on s'ace e+'orationG
:o demonstrations ser1e an" usefu 'ur'oseG
:o 'eo'e e1er rea" earn an"thing from the mistakes the" makeG
&rou' discussion and roe 'a"ing are two other im'ortant techniBues for
assessing ora 'roduction. Through grou' discussion and roe 'a"ing the
teacher can disco1er how students are thinking and using the target
anguage. >or e+am'e) are the" using the anguage the" are earning to
e+'ore conce'ts and ideasG 9r are the" sim'" using the foreign anguage
to 'resent ideas aread" we5formedG (n this wa") grou' acti1ities in both
teaching and testing can be used to 'ro1ide an o''ortunit" for meaningfu
and acti1e in1o1ement. 0tudents are thus gi1en an o''ortunit" to use what
can be termed De+'orator" tak@!@ i.e. the anguage 'eo'e use when tr"ing
to communicate rather than when the" are engaged in the mechanica
'roduction of 1erba formuae or 'atterns. (n short) anguage becomes a
means to an end rather than an end in itsef. &rou' anguage acti1ities
then become ego5su''orting <unike the ego5threatening e+'erience of too
man" students in traditiona tests=) as the other members of the grou' are
interested chief" in the message rather than in forma correctness.
0e1era usefu books ha1e been written on grou' work <deaing with
the siLe and nature of 1arious grou's= and on acti1ities for grou'
discussion. &enera" s'eaking) the t"'e of acti1it" most suitabe for grou'
work is that in which the e1e of difficut" in1o1ed makes the task a itte
too difficut for the indi1idua student to accom'ish aone) but not so
difficut as to discourage a grou' of students <es'ecia" gi1en a the
constraints and tensions of a test=. (n addition to a the 'robem5so1ing
acti1ities and 'uLLes which abound in books and materias of 1arious
kinds) the t"'e of task in1o1ing concensus5seeking is 'articuar" suitabe
for grou' discussion. (n this atter t"'e of acti1it". the members of the
grou' are gi1en a 'articuar situation and instructed to make 1arious
decisions. (t is necessar" for them to use the target anguage to Justif" their
decisions and seek agreement from the other members of the grou'.
I#adioacti1it" from a nucear 'ower station accident wi reach "our area in
a few hours. There is a sma but 1er" safe nucear faout sheter
nearb") but there is room for on" si+ 'eo'e out of a tota of twe1e.
;hich si+ 'eo'e from the foowing ist do "ou think it woud be most
usefu to sa1e in the interests of future generationsG List them in order of
'riorit". <Note! * 7 mae4 > 7 femae.=
5 a marine bioogist) aged 8A <>=
5 a 'h"sicist) aged ,8 <*=
5 a famous musician) aged 33 <>=
5 a farmer) aged 3, <*=
5 an eectrician) aged ?2 <*=
5 a mathematics teacher) aged 3? <>=
5 a we5known footbaer) aged ,, <*=
5 a doctor) aged A3 <>=
5 a uni1ersit" student of socioog") aged 12 <>=
5 a fireman) aged 33 <*= 5555555. 5 5
5 a factor" worker) aged ,3 <>= 5
5 a garage mechanic) aged ,7 <*=
#oe5'a" acti1ities can aso be used successfu" to test ora
communicati1e abiit". The students in1o1ed are assigned fictitious roes
and are reBuired to im'ro1ise in anguage and beha1iour. (t is ad1isabe for
the students to be gi1en fictitious names before the roe 'a") as these
usua" 'ro1e 1er" he'fu in encouraging them to act out the roes assigned
to them. The roe 'a"s used for the test ma" 1ar" from short sim'e roe
'a"s in1o1ing on" two or three students to far onger roe 'a"s in1o1ing
se1era students. The students needn@t be informed that the" are being
obser1ed and assessed in their use of Engish! in fact) it is usua"
im'ortant that the teacher or e+aminer does this as discreet" as 'ossibe.
The foowing is an e+am'e of an e+treme" sim'e roe 'a" suitabe
for use at the eementar" e1e.
9ne student acts the 'art of a 'oic
e officer) another a bus conductor) a
third a bus5dri1er) a fourth a 'assenger hurr"ing to 1isit a sick friend in
hos'ita) and a fifth a b"stander who wants to be he'fu. The 'assenger
hurries to get on the bus and Jum's on as it is mo1ing off. The
conductor sto's the bus and tes him that the bus is fu and that he
must get off. The 'assenger can see an em't" seat and he begins to
argue. The bus is now in the midde of the road and is a danger to other
traffic. .ct the roes gi1en.
(t is usua" ad1isabe for each @character@ to be gi1en a card on which
there are a few sentences describing what kind of a 'erson he or sheds. etc
*oreo1er. it is often he'fu if the e+aminer can take a minor roe in order
1-3
((
&enera concusions
Notes and references
w?
(
to be abe to contro or infuence the roe 'a" if necessar". (ndeed) e1en in
grou' discussions and acti1ities) the e+aminer shoud awa"s fee free to
interru't or contro the discussion in as di'omatic a wa" as 'ossibe in order to
ensure that each member of the grou' makes a contribution. Both grou'
discussions and roe 'a"s are best assessed if the" are incuded as 'art of the
anguage 'rogramme rather than as a forma test.
&enera" s'eaking) a reiabe method of obtaining measurements of ora
'roduction skis is that which in1o1es the students@ cass teacher. The
tensions and artificiaities that inhibit the students in an ora e+amination
conducted b" an e+terna e+aminer wi now be a1oided since the teacher is a
famiiar figure and the cassroom a reaistic 'art of the students@ ife.
Continuous assessment b" the teacher) with a his or her cassroom
e+'erience4i.5genera" <but5not awa"s= a reiabe method of measuring the
ora skis. %et the ora inter1iew <conducted b" a s"m'athetic
e+aminer= is sti a usefu e+amination to retain) 'articuar" for its
beneficia backwash effects on teaching. . com'rehensi1e and baanced
e+amination of ora 'roduction might thus consist of!
5 an ora inter1iew in1o1ing two students4
5 a short 'robem5so1ing acti1it" in1o1ing the com'arison or seBuencing
of 'ictures) etc.4
5 a onger acti1it" com'rising grou' discussion <concensus5seeking acti1it"=
or a roe 'a".
The first two com'onents isten here ma" be gi1en as 'art of a forma test
whie the atter com'onent is much better gi1en during the course itsef and
assessed b" the cass teacher in as informa a wa" as 'ossibe. The
e+aminer must freBuent" consider the effects of the e+amination on
teaching 5rid earning) howe1er) and if) for instance) the reading aoud
section is considered harmfu in its effect on teaching)
tier) it shoud be
omitted from the e+amination.
1 This t"'e of Buestion has been used effecti1e" in the .#EL0 9ra E+amination
<.ssociation of #ecognised Engish Language 0choos=.
, The e+am'e is based on an item t"'e in the /ni1ersit" of Cambridge Loca
E+aminations 0"ndicate! 6reiminar" Engish Test <re1ised 1ersion=.
3 This techniBue is deri1ed from a simiar one used b" .. 0. 6amer and described
in the artice DTesting Communication@ in (#.L <(nternationa #e1iew of
.''ied Linguistics in Language Teaching=) (ssue $K1) >ebruar" 127,.
? #ichardson) # <ed.= 127A Learning for Change in ;ord 0ociet". 9ne ;ord
Trust
8 Based on the marking scheme de1ised b" :r >rank Cha'en for use in assessing
the ora skis of o1erseas students at British uni1ersities. <The scheme is) in
turn) deri1ed from the .merican >oreign 0er1ice (nstitute rating scae.=
A *orrow) $ 123, Testing 0'oken Language. (n Language Testing <ed. Heaton=.
*odern Engish 6ubications
7 #o"a 0ociet" of .rts) Tests of 9ra (nteraction! :egrees of 0ki. The
Communicati1e /se of Engish as a >oreign Language.
3 BartL and 0chutL) 127?
2 Barnes) : 1273 Language in the Cassroom. 6enguin5
1- Based on an idea in #ichardson) # <ed.= 127A Learning for Change in ;ord
0ociet". 9ne ;ord Trust
Testing reading com'rehension
3.1 The nature of the /nti recent" the man" and di1erse reading skis and strategies for use in
reading skis e1er"da" situations ha1e been arge" subordinate to a narrower range of
skis reBuired for deaing with sim'ified readers) es'ecia" at the
eementar" e1es. >urthermore) on a few anguage courses) efficient
reading skis ha1e been 'ushed into the background in an attem't to
de1eo' ora fuenc" skis. .ttem'ts at deaing with the man" com'e+
reading skis freBuent" come too ate) at the tertiar" e1e <i.e. at
uni1ersit") technica coege=) when students sudden" find themse1es
confronted with 'rofessiona and technica iterature in the foreign
anguage.
5
(n s'ite of the wide range of reading materia s'ecia" written or
ada'ted for Engish anguage earning 'ur'oses) there are few
com'rehensi1e s"stematic 'rogrammes which ha1e been. constructed from
a detaied ana"sis of the skis reBuired for efficient reading. *uch test
materia is sti imited to short reading e+tracts on which genera
Dcom'rehension@ Buestions are based. .s with istening com'rehension)
reading com'rehension test materia is 1er" cose" reated to the t"'e of
'ractice materia used b" the teacher to de1eo' the reading skis. >ew
anguage teachers woud argue against the im'ortance of reading! what is
sti urgent" reBuired in man" cassroom tests is a greater awareness of the
actua 'rocesses in1o1ed in reading and the 'roduction of a''ro'riate
e+ercise and test materias to assist in the master" of these 'rocesses.
Before reading tests in the second or foreign anguage can be
successfu" constructed) the first anguage reading skis of the students
must. be ascertained.OCear".there.is often itte 'ur'ose in testing in the
second anguage those basic reading skis which the students ha1e not "et
de1eo'ed in their own anguage. Howe1er) the mere fact that a student
has mastered some of the reBuired reading skis in the first anguage is no
guarantee at a that he or she wi be abe to transfer those skis to reading
another anguage.
.t this stage in our e+amination of reading difficuties) it woud be
he'fu to attem't to identif" some of the s'ecific skis in1o1ed in
reading.@ Broad" s'eaking) these can be defined as the abiit" to!
5 recognise words and word grou's) associating sounds with their
corres'onding gra'hic s"mbos4
1-8
1
((
(
5 deduce the meaning of words b"
<a= understanding word formation <roots) affi+ation) deri1ation and
com'ounding=4
<b= conte+tua cues <e.g. 9ne of the members of the grou'
e+'osed the
'ot) and the 'oice were soon abe to arrest the eaders.=4
5 understand e+'icit" stated information <e.g.
( wish .nn had come. O
.nn did not come 5 hence m""
wish.=4
5 understand reations within the sentence) es'ecia"
<a= eements of sentence structure
<b= negation
<c= fronting and theme
<d= com'e+ embedding4
5 understand reations between 'arts of a te+t through both e+ica de1ices
.<e.g..re'etition) s"non"ms).antithesis= and grammatica cohesi1e de1ices)
es'ecia" ana'horic and cata'horic reference <e.g. he)
the") it4 the
former) the atter=
and connecti1es <e.g.
since) after) because) athough.
howe1er) in addition=!
'ercei1e tem'ora and s'atia reationshi's) and aso seBuences of ideas4
5 understand conce'tua meaning) es'ecia"
<a= Buantit" and amount
<b= definiteness and indefiniteness
<c= com'arison and degree
<d= means and instrument
<e= cause) resut) 'ur'ose. reason) condition) addition) contrast.
concession4
5 antici'ate and 'redict what wi come ne+t in the te+t!
identif" the main idea and other saient features in a te+t4
generaise and draw concusions!
understand information not e+'icit" stated b"
4a= making inferences <it5. reading between the ines=
<b= understanding figurati1e anguage!
5 skim and scan <ooking for the genera meaning and reading for s'ecific
information=4
5 read critica"4
5 ado't a fe+ibe a''roach and 1ar" reading strategies according to the
t"'e of materia being read and the 'ur'ose for which it is being read.
No mention has been made here of reading aoud) since this 'articuar
ski is uniBue in that it in1o1es different skis from sient reading.
Two different kinds of com'ementar" reading acti1ities to which
students are usua" e+'osed are genera" cassified as
intensi1e and
e+tensi1e reading. 0hort reading e+tracts of a moderate degree of difficut"
and containing features which merit detaied stud" form a basis for
intensi1e reading 'ractice. ;hoe artices) cha'ters and books <usua"
sim'ified readers= are used for e+tensi1e reading 'ractice4 in this case.
howe1er) the materia seected is genera" sight" beow the anguage
attainment e1e of the students using it. /nfortunate") most reading tests
concentrate on intensi1e reading to the e+cusion of e+tensi1e reading.
'robab" because it is more economica to ha1e a arge number of items
based on a short reading e+tract than a few items based on a much onger
one. Howe1er) these are insufficient grounds for negecting to test
e+tensi1e reading at certain e1es.
(n most tests) es'ecia" tests of genera 'roficienc". it is usefu to
incude a 1ariet" of te+t t"'es for reading com'rehension in addition to the
(t7)
usua) more iterar" 'rose e+tracts! e.g. news'a'er artices) instructions for
using a''iances and machiner") director" e+tracts) 'ubic notices)
timetabes and ma's) ad1ertisements) etc. The incusion of such te+t t"'es
wi not on" 'ro1ide a more reaistic and reiabe means of assessment but
wi aso he' to moti1ate students b" demonstrating how the target
anguage is used in rea5ife situations. ConseBuent") it becomes im'ortant
that the actua 'resentation of the materia shoud be as authentic as
'ossibe. (n short) a news'a'er artice shoud a''ear in the actua form of
a news'a'er artice) thereb" gi1ing a genuine fee to the materia.
0e1era wa"s of testing reading com'rehension are treated in the
foowing sections of this cha'ter. Certain of the item t"'es wi be more
suited to testing com'rehension of a 'articuar te+t than other item t"'es.
(ndeed) there are numerous wa"s of testing reading com'rehension)
ranging from muti'e5choice items to o'en5ended Buestions <i.e. Buestions
which reBuire students to write an answer in a sentence of their own=.
.though muti'e5choice items are sometimes the most suitabe
instruments for testing reading com'rehension) the" shoud not be o1er5
used. >reBuent") other item t"'es are far more interesting and usefu. The
te+t itsef shoud awa"s determine the t"'es of Buestions which are
constructed. Certain te+ts ma" end themse1es to muti'e5choice items)
others to trueKfase items) others to matching items) others to re5
arrangement items) others to ordinar" com'etion items) others to the
com'etion of information in tabes) and "et others to o'en5ended
Buestions. (ndeed) sometimes the same te+t wi demand at east two or
three different t"'es of items.
3., (nitia stages of The tests described in the first haf of this section are concerned 'ure"
reading! matching with word and sentence recognition. The" test students@ abiit" to
tests discriminate 1isua" between words which are s'et in fair" simiar wa"s.
if used in e+ercise materia and 'rogress tests) these test items wi assist in
de1eo'ing word recognition s'eed. Though not administered as s'eed
tests in the strict sense in the 1er" ear" stages) word and sentence
matching items shoud be co1ered b" the students as Buick" as 'ossibe.
9nce the students ha1e gained famiiarit" and confidence with this t"'e of
test) their 'erformances shoud be timed so that the" are forced to read
under some 'ressure. .t first) it is ad1isabe to confine the words used in
the items to those aread" encountered ora"4 ater a number of words not
encountered ora" shoud be introduced.
;ord matching The testees are reBuired to draw a ine under the word which is the same as
the word on the eft.
riow bowKnotKhowKnow mow
shee'
sho'Ksha'eKsee'Khea'Kshee'
e1er ne1erKo1erKe1erKfewerKe1en
to' to'Ksto'Kta'K'otKton
wonder wanderKwonderKwindowKfonderKwon
has gone is goneKhas wonKhas goneKhis gameKhad gone
cothes cothKcothingKcothsKcotsKcothes
most most 'easantKmore 'easantKmost 'resentKnot 'easantKmost
'easant 'easants
0entence matching This item is simiar to the word5matching item. The testees are reBuired to
recognise as Buick" as 'ossibe sentences which consist of the same words
1-7
i
6ictures and sentence
matching
1-3
(
in the same order. The" read a sentence) foowed b" four simiar
sentences) on" one of which is e+act" the same as the 're1ious one.
1. Tom is not going to "our schoo.
.. Tom is not going to "our 'oo.
B. Tom is going to "our schoo.
C. Tom is not coming to "our schoo.
:. Tom is not going to "our schoo.
,. The thief can hide in the Junge.
.. The thief can die in the Junge.
B. The thie1es can hide in the Junge.
C. The thief can be hidden in the Junge.
5. OO..:.rQhe+bi.efOcanhide in the Junge.
(n the remainder of this section the items wi concentrate on word and
sentence com'rehension) using 'ictures to test this ski.
T"'e 1 This t"'e of item is simiar to that used to test istening
com'rehension and described under T"'e 3 in 0ection A.8. The testees ook
at four 'ictures and then read a sentence about one of the 'ictures. The"
are reBuired to identif" the correct 'icture.
The" are c"cing to work.
He is showing her the 'hotogra'h.
T"'e , This t"'e is simiar to the 're1ious one but is much more
economica in that on" one 'icture is reBuired for each item <instead of
four=. The testees ook at a 'icture and read four sentences. on" one of
which is about the 'icture. The" then ha1e to seect the correct sentence.
.. Jenn" is throwing the ba to 6eter.
B. 6eter is kicking the ba to Jenn".
C. 6eter is throwing the ba to Jenn".
:. Jenn" is kicking the ba to 6eter.
.. The man under the tree is reading
his book.
B. The man resting under the tree is
ooking at his book.
C. The man with the book is see'ing
under the tree.
:. The man carr"ing the book is going
to sit down under the tree.
T"'e 3 .though this item t"'e is referred to here as a matching item) it
coud eBua" we take the form of a trueKfase item <in which the testees
write T or > at the side of each sentence according to whether or not the
sentence agrees with the contents of the 'icture=. (n this 'articuar
instance) testees ha1e to seect the <four= sentences which match the
'icture.
>our of the foowing sentences agree with the 'icture. 6ut a circe round
the etter of each of the. four sentences.
.. The dog on the foor is asee'.
B. The bab" is 'a"ing with the dog.
C. The bab" has Just broken a to" car.
:. The tee1ision set is on fire.
1-2
11
E. The dog is behind the bab".
>. The woman has taken the fowers out of the bow.
&. 9ne of the two bo"s is he'ing the woman.
H. The woman is going to 'ick u' a book.
1. The two bo"s are istening to the radio.
J. The radio is on the tabe but the book is on the foor under the tabe.
3.3 (ntermediate and
ad1anced stages of
reading! matching
tests
C9rre
. ... B ...
T"'e 1 The foowing matching item, shows how 1isuas can be used to test
the com'rehension of definitions of certain words. Testees are reBuired to
match the meaning of certain terms in a dictionar" with the a''ro'riate
sha'es which those terms denote. The e+am'e shows how the matching
techniBue can be used at a more ad1anced e1e and how it can end itsef to a
more communicati1e testing of reading. .bo1e a) this 'articuar item
measures the abiit"5of)the testee to understand the kinds of definitions
usua" found in a dictionar" 5 an essentia ski reBuired in earning and
using a foreign anguage.
0ome of the sha'es are described in the dictionar" e+tracts. Name
on" those that5 are described in the e+tracts. This first one has been
done for "ou.
*
c ... : ... E >
N.*
;
& ...
H ... ( J $ ... L
* ...
N ...
- ... 6 ...
( 1 t=
coi T-iK
1t)1i wind or twist into a continuous
circuar or s'ira sha'e4 cur round and round!
The snake 5ed <itsef= round the branch. a n \CX 1
something coied4 a singe turn of something
coied! the thick 555s of a '"thon.
, ength of wire
wound in a s'ira to conduct eectric current.
coneKkaunK n \CX 1 soid bod" which narrows to a
'oint from a round) fat base. , something of this
sha'e whether soid or hoow. 3 fruit of certain
e1ergreen trees <fir) 'ine) cedar=.
cube KkJubK n
\CX 1 soid bod" ha1ing si+ eBua
sBuare sides4 bock of something so sha'ed or
simiar" sha'ed. , <maths=. 'roduct of a number
muti'ied b" itsef twice! The 5 of 8 <83= is 8 + 8
+ 8 <1,8=. 1t muti'" a number b" itsef twice!
1- 55d is 1 ---.
cu5bicK@kJubikK adJ ha1ing the sha'e of a cube4 of
a cube! one 55 metre) 1oume of a cube whose
edge is one metre.
cu.bi5caK@kJubikaK adJ 7 cubic <the usua word=.
c"(5inHderK@siinda<r=K n \CX 1 soid or hoow bod"
sha'ed ike a 'oe or og. , c"inder5sha'ed
chamber <in an engine= in which gas or steam
works a 'iston! a si+ engineKmotor5car.
c"indrica Ksi@indnkaK adJ c"inder5sha'ed.
ei'seKi@i'sK n \CX reguar o1a.
e(5(i'HticKi@(1'tikK) e5i'5tiHca K5kaK adJ sha'ed ike
an ei'se.
funne K@fnnaK n \CX 1 tube or 'i'e wide at the to'
and narrowing at the bottom) for 'ouring iBuids
or 'owders through sma o'enings. , outet for
smoke of a steamer) raiwa" engine) etc.

1t) 1i
(( 5) /0 5(5= <cause to= mo1e <as if= through a
funne.
at5ticeK@(aetisK n \CX framework of crossed aths or
meta stri's as a screen) fence or door) or for
cimbing 'ants to grow o1er! a 55 window.
at5ticed adJ
'"ramid K@'iramidK n \CX 1 structure with a
trianguar or sBuare base and so'ing sides
meeting at a 'oint) es' one of those buit of stone
in ancient Eg"'t. , 'ie of obJects in the sha'e of
a '"ramid.
s'hereKsfia<r=K n \CX 1 form of a gobe4 star4 'anet.
music of the s'heres) <m"th= music 'roduced b"
the mo1ement of hea1en" bodies which men
cannot hear. , gobe re'resenting the earth or the
night sk".
s'herica K@sfenkaK adJ sha'ed ike a s'here.
wedgeKwedsK n \CX 1 E5sha'ed 'iece of wood or
meta) used to s'it wood or rock <b" being
hammered=) to widen an o'ening or to kee' two
things se'arate) the thin end of the wedge) <fig= a
sma change or demand ike" to ead to big
changes or demands. , something sha'ed ike or
used ike a wedge! 55 hees <on shoes=. 1tfi+
tight" <as= with a wedge! 55 a door o'en) b"
'acing a wedge under it. be tight"55d between
two fat women on the bus.
T"'e , The foowing item t"'e3 is incuded to 'ro1ide another e+am'e of
how reading com'rehension matching tests can be based on the dictionar".
.gain) the item is intended for use at a fair" ad1anced e1e.
0ettements
Encosed hut grou's are characteristic@settements in the area and
remains5of more than a. hundred sti e+ist. The o'en settements) the
1iages of 'redominant" unencosed huts) are not numerous but on" a
doLen grou's ha1e sufficient numbers of co5ordinated huts to be
described as 1iages. Though there ma" be some was in these 1iages)
the" are on" fragmentar".
*an" encosed settements ha1e disa''eared but one sti e+tant is
#ides #"s. (t consists of two encosures) one rough" sBuare and a arger
one rough" obong in 'an with a shared wa. .n area of si+ acres was
encosed containing more than thirt" buidings. The arge encosure) as
in the case of other muti'e settements) had been buit on to the
smaer and indicates an e+'anding communit".
111
The foowing dictionar" definitions are for words used in the 'assage
0ettements. ;rite the words from the 'assage ne+t to the a''ro'riate
definition.
ha1ing or in1o1ing more than one 'art as an indi1idua
ser1ing as re'resentati1e
becoming greater in siLe
H H H
enough to meet a need or 'ur'ose
H
ha1ing something in common
H H H H H
in most cases
is a sign of
1er" near to4 a''ro+imate
broken off or incom'ete
in e+istence4 sur1i1ing
T"'e 3 The foowing item? is simiar to the 're1ious t"'e but here testees are
reBuired to match a''ro'riate words or information in the te+t with the correct
'arts of a diagram.
1 A
, 7
3 ...
3 ..
? .. 2
8 ...
1-
T"'e ? This item t"'e in1o1es the matching of a 'ara'hrase of 'hrases
and sentences in a te+t with the origina words in the te+t) each item
starting with D(nstead of . .@. .though of use occasiona" in achie1ement
and 'roficienc" tests) this t"'e of item is more suited to teaching than to
testing as it can be used to he' students to increase their understanding of
a te+t.
(n the foowing e+am'e) students are asked to read a te+t and then
com'ete each sentence. The item) howe1er) cas for the matching of
information and is thus not actua" a com'etion t"'e of item.
11,
(t is im'ortant for each student to obtain at reguar inter1as a rough
idea of his or her 'rogress. How are goas being achie1ed week b"
weekG *ethods of continuous assessment of students@ work are
re'acing e+aminations 5 or 'arts of e+aminations 5 on certain courses.
There are sti doubts about the ad1antages of continuous assessment it
the earning 'rocess but) if a''ied with care and discretion) continuous
assessment can be a far more 1auabe means of assessing standards
than an e+amination. 6ro1ided that methods of continuous assessment
do not im'art a feeing of tension and strain) the" can be used to guide
students in their work and to inform them of the 'rogress the" are
making. (f no means of continuous assessment is a1aiabe) students
shoud attem't to e1auate and summarise their 'rogress 1er" brief"
week b" week. Cear") such an attem't is more difficut in a subJect
which teaches skis <e.g. earning a anguage) 'a"ing a musica
instrument= than in a content subJect <e.g. histor") chemistr"=. E1en as
far as skis are concerned) howe1er) it is a sim'e matter for students to
go back to an od e+ercise and do it again. The ease with which the" can
do what 're1ious" seemed a difficut e+ercise is often Buite remarkabe.
1. (nstead of taking about using such methods carefu" and wise") the
writer taks about a''"ing them
,. (nstead of sa"ing that it is harder for the students to assess 'rogress
made in earning a anguage than in histor") the writer sa"s that
3. (nstead of referring to wa"s of measuring students@ 'rogress at
reguar inter1as) the writer uses the 'hrase ...
...).
?. (nstead of sa"ing that students shoud tr" to assess and re'ort brief"
on their 'rogress e1er" week) the writer sa"s that
5 ...
8. (nstead of sa"ing that continuous assessment is usefu uness it
makes students fee u'set and worried) the writer sa"s that it is usefu
3.? TrueKfase reading
The trueKfase test is one of the most wide" used tests of reading
tests
com'rehension. Not on" is the scoring of such a test straightforward and
Buick) but the scores obtained b" the testees can be 1er" reiabe indices of
reading com'rehension 'ro1ided that the items are we constructed and
that there are enough of them. TrueKfase tests are of considerabe use for
incusion in cass 'rogress tests chief" because) unike muti'e5choice Lest
items) the" can be constructed easi" and Buick") aowing the teacher
more time for other tasks.
The trueKfase test) howe1er) has two main disad1antages! first") it can
encourage guessing) since testees ha1e a 8- 'er cent chance of gi1ing a
correct answer for each item. 0econd") as the base score is 8- 'er cent
<7 - 'er cent= and thus the a1erage test difficut" genera" in the region of 78
'er cent <7 8- 'er cent=) the test ma" fai to discriminate wide" enough
among the testees uness there are a ot of items.
(t is) of course) 'ossibe to 'enaise the testee for guessing) and
instructions on the ines of the foowing ma" be incuded in the rubric!
113
(
Each correct answer wi be awarded two marks. Howe1er) for each
wrong answer) one mark wi be deducted from "our score. (t is better)
therefore) not to guess bind" and to ea1e a bank if "ou do not know
the correct answer.
0uch 'enaties) howe1er) are of dubious 1aue) and the whoe subJect
of guessing is treated in greater detai in Cha'ter 11.
.nother soution to the 'robem of guessing is to incude a third
Buestion in addition to the trueKfase o'tion! e.g. true) fase) not stated <i.e.
the reBuired information is not gi1en in the 'assage=. Thus) a rubric for
this item t"'e woud read!
.ccording to the 'assage) are the foowing statements true or fase) or is
it im'ossibe to draw an" concusionG
(n addition to the ease and s'eed with which the items can be
constructed) the great merit of the trueKfase reading test ies in the ease
with which suitabe test 'assages can be seected! a short reading e+tract)
for e+am'e) can 'ro1ide a basis for numerous items. *oreo1er) the
trueKfase test can be used as a 1auabe teaching5de1ice with which the
students@ attention is directed to the saient 'oints in the e+tract b" means
of the trueKfase items.
(f the students@ com'rehension of the trueKfase reading e+tract <and
not the trueKfase items themse1es= is being tested) each of the trueKfase
items shoud be as cear and concise as 'ossibe. (n such cases) it is
essentia that the 'robem 'osed b" each item is fu" understc .. (n man"
trueKfase reading tests some indication of the number of correct and
incorrect statements is gi1en to the students. .though this ma" make the
test sight" easier for them) it does at east 'resent them with a cear
statement of the 'robem.
TrueKfase reading tests fa into two genera categories! those which
are inde'endent of a reading te+t <T"'e 1= and those which de'end on a
te+t <T"'e ,=.
T"'e 1 (t is 'ossibe to construct trueKfase items which are com'ete in
themse1es! a testee@s com'rehension of each trueKfase item is tested b"
means of a series of genera truths. >or e+am'e!
6ut a circe round the etter T if the statement is true. (f it is not true) 'ut a
circe round the etter >.
1. The sun rises in the west. T >
,. >ish can@t f") but birds can.
T >
3. Engand is as arge as .ustraia. T >
?. ;hen ice mets) it turns into water. T >
T"'e , The construction of trueKfase items based on a reading e+tract
forms one of the most wide" used t"'es of reading tests. (t is often used at
eementar" e1es of reading com'rehension) but it can be used eBua"
effecti1e" at more ad1anced e1es. The foowing e+am'e iustrates its use
at a fair" ad1anced e1e!8
E"e5gaLing and e"e5a1oidance ha1e meanings and 'atterns of
'rofound significance. &aLing at others@ e"es genera" signas a reBuest
for information and 'erha's affection) but embarrassment can resut
from too ong a mutua gaLe. (n fact) in intimate situations) there seems
to be an eBuiibriumm in1o1ing 'ro+imit") e"e contact) intimac" of to'ic)
11?
and smiing. (f one com'onent is changed) the others tend to change in the
o''osite direction.
But the e+tended gaLe seems to ha1e a function much dee'er than
that of maintaining a baance or ensuring a smooth fow of con1ersation.
(t signas) not sur'rising") an intensification of reationshi') not
necessari" aong amorous ines. (t ma" be a threat) or a chaenge for
dominance.
. definite 'ecking order of dominance and submission emerges
from the 1er" first e"e contact of strangers. Curious") when
con1ersation is 'ossibe) it turns out that the one who ooks awa" first
tends to be dominant. The a1erted e"e is a signa that its owner is about
to take the foor. ;hen con1ersation is not 'ossibe) howe1er) the first to
ook awa" wi be the submissi1e one.
.bnorma use of e"e contact or a1ersion ma" we indicate an
abnorma 'ersonait". .dut schiLo'hrenics tend to use their e"es at a
the wrong 'oints in a con1ersation) and the bod iar can hod a stead"
gaLe far onger than his truthfu coeague when both are caught in the
same misdemeanour.
.ccording to the 'assage) si+ of the foowing statements are true and si+
are fase. (t is not 'ossibe to draw an" concusions from the
information in the 'assage about the remaining three statements. 6ut T >
or G in each bo+) as a''ro'riate.
1. Looking at someone ese@s e"es or ooking awa" from them
means a 'erson is thinking 1er" dee'".

,. ;e genera" ook towards another 'erson@s e"es when we
want information or a sign of affection from that 'erson.
11
3. (f two 'eo'e ook too ong at each other@s e"es) the" wi
usua" become embarrassed.
?. ;e are usua" 'uLLed b" someone who freBuent" ooks into
our e"es during a con1ersation.
8. ;hen engaged in 1er" friend" con1ersation) a cou'e wi
'robab" ook ess at each other@s e"es the more the" smie
and the coser the" sit.
A. (t is easier to tak to someone in a friend" wa" if the 'erson we
are taking to does not gaLe too ong at the s'eaker.
7. Looking for a ong time at the other 'erson@s e"es is on" a
means of continuing a con1ersation smooth".
3. ;hen two 'eo'e gaLe for a ong time at each other@s e"es) it
555is a sign that5the"5are going to argue.
2. .n e+tended gaLe can signa a threat or a bid for authorit" o1er
the other 'erson.
1-. ;hen two strangers meet) the" use their e"es to contro or
infuence the other or to show their surrender to the other@s
authorit".
11. ;hen it is 'ossibe to tak) the first 'erson who ooks awa" is
the one who submits to the other 'erson.
1,. . 'erson shows that he or she wants to tak b" ooking
towards the other 'erson@s e"es.
11
118
13. 6eo'e with 'oor e"esight genera" stand or sit 1er" cose to

the 'erson the" are addressing.
1?. .bnorma 'eo'e usua" turn their e"es awa" from the other
'erson@s e"es more often than norma 'eo'e do.

18. (t is 'ossibe to te the difference between a iar and an honest
'erson b" their e"e5gaLing 'atterns when both are tr"ing to te
a ie.
The reading te+t in the 'receding e+am'e contained anguage at a
higher difficut" e1e than that used in the statements which foowed. (t is
'ossibe <though not common 'ractice=. howe1er) to construct a reati1e"
sim'e te+t foowed b" more difficut statements! in such cases) the
com'rehension 'robems wi be contained in the statements themse1es
rather than in the te+t. 5
3.8 *uti'e5choice T"'e 1 (t can be argued that the t"'e of item in this section is in man"
items <.=! short te+ts
wa"s a test of 1ocabuar" rather than of reading com'rehension. These
'articuar items) howe1er) ha1e been incuded here because it is fet that a
com'rehension of the te+t is genera" of at east as much im'ortance as an
understanding of the meaning of the words for seection. This) of course) is true
of an" 1ocabuar" item 'resented in conte+t! howe1er) here the
em'hasis is more on the correct understanding of the conte+t. The
foowing three e+am'es show the use of this item t"'e at eementar")
intermediate and ad1anced e1es res'ecti1e".
1. The e"es are wonderfu teachers 5 e1en musicians) who dea with
sound) earn as much b"
<doing) 'a"ing) watching) 'ractising= as b"
istening.
,. The housewife who coud not afford to bu" cothes woud s'end
hours at her s'inning whee) s'inning her woo into "arn 5 a Job
which took itte ski but reBuired a ot of <abiit") 'atience) taent)
wisdom=
and was done b" the fireside during the ong winter
e1enings.
3. Two5thirds of the countr"@s
<fue) endea1our) industr") energ"= comes
from im'orted oi) whie the remaining one5third comes from coa.
*oreo1er) soon the countr" wi ha1e its first nucear 'ower station.
T"'e , Just as the 're1ious item t"'e is cose" reated to the testing of
1ocabuar") so this t"'e is 'erha's more accurate" described as a test of
com'rehension of grammatica structure. The testees are reBuired to
identif" the correct 'ara'hrase of a statement from a choice of four or fi1e.
The" are tod in the rubric that the <four= statements ma" refer to the
entire sentence or on" 'art of the sentence. .gain) e+am'es are 'ro1ided for
each of the three genera e1es.
1.
John is not as ta as 0a" but he@s a itte taer than #ick.
.. 0a" is taer than John and #ick.
B. John is not as ta as #ick.
C. 0a" is taer than John but not as ta as #ick.
:. #ick is taer than John and 0a".
11A
,. (n s'ite of the oud music) ( soon managed to fa asee'.
.. The oud music soon he'ed me to fa asee'.
B. ( soon fe asee' as a resut of the oud music.
C. The oud music made me unabe to fa asee' soon.
:. ( soon fe asee' e1en though the music was oud.
3. (f "ou@d forgotten to 'ut out "our hand) "ou woudn@t ha1e 'assed
"our dri1ing test.
.. %ou didn@t forget to 'ut out "our hand and "ou 'assed "our
dri1ing test.
B. %ou forgot to 'ut out "our hand and "ou faied "our dri1ing test.
C. %ou forgot to 'ut out "our hand but "ou 'assed "our dri1ing test.
:. %ou didn@t forget to 'ut out "our hand but "ou didn@t 'ass "our
dri1ing test.
T"'e 3 This item t"'e consists of a 1er" short reading e+tract of on" a few
sentences <or sometimes of on" one sentence=. The testees are reBuired to
answer on" one com'rehension test item on each reading 'assage. The
actua construction of muti'e5choice reading com'rehension items based on
a reading e+tract wi be treated in greater detai in the ne+t section.
*eanwhie) here are two e+am'es of the use of muti'e5choice items for
testing reading com'rehension) the first being at a fair" eementar" e1e and
the second at a more ad1anced e1e.
1.
The 'resident was taking to a "oung woman in the crowd when Tim
sudden" caught sight of a man standing se1era "ards behind her.
The man had something in his hand! it was a short stick.
;hat made Tim notice the man in the crowdG
.. He was 1er" cose to Tim.
B. The 'resident was taking to him.
C. He was standing in front of the woman.
:. He was carr"ing a stick.
,. There were on" two wa"s of mo1ing aong the narrow edge! face
outwards or face to the wa. ( concuded that e1en the smaest of
bottoms woud 'ush a 'erson with his back to the wa far enough
out to o1erbaance him and so) with arms outstretched in the sha'e
of a cross and with chin 'ointed in the direction ( was heading) (
inched m" wa" aong.
The writer managed to cross the narrow edge b"
.. crawing aong on his knees with his arms stretched out in front c
him.
B. mo1ing sidewa"s inch b" inch with his back to the wa.
C. working his wa" forward on his stomach with his face amost
touching the edge.
:. waking sow" with his face and stomach cose to the wa.
3.A *uti'e5choice
The muti'e5choice test offers a usefu wa" of testing reading
items <B=! onger te+ts
com'rehension. Howe1er) not a muti'e5choice reading tests are
necessari" good tests of reading com'rehension. .s was cear" indicated
earier) the e+tent to which a test is successfu in measuring what it sets ou4
to measure de'ends arge" on the effecti1eness of each of the items used.
(ndeed) certain genera as'ects of man" reading tests ma" be sus'ect. >or
instance) does the usua brief e+tract for reading com'rehension
concentrate too much on de1eo'ing on" those skis reBuired for intensi1e
reading) encouraging freBuent regressions and a word5b"5word a''roach to
readingG
The sam'ing of the reading 'assage is of the utmost im'ortance and
must be reated to the broader aims of the anguage teaching situation.
*an" of the te+ts in both schoo and 'ubic e+aminations concentrate too
much on a iterar" kind of Engish. (f certain students are earning Engish
in order to read technica Journas) for e+am'e) then the sam'ing of the
reading e+tract shoud refect this aim. (dea") in a test of 'roficienc" the
te+t shoud contain the t"'e of reading task which wi be demanded of the
testees in ater rea5ife situations. (f the test is a cass 'rogress or
achie1ement test) the reading 'assage shoud be simiar to the t"'e of
reading materia with which the students ha1e been confronted in their
work at schoo. (n other words) if other subJects are being taught in the
medium of Engish <as in man" second anguage situations=) the te+t shoud
freBuent" <though not awa"s= refect the t"'e of reading the students are
reBuired to do in histor" or chemistr") etc.
(n this section) it is assumed that on" intensi1e reading skis are being
tested. Thus) the ength of the reading e+tract recommended might 1ar"
from 8- to 1-- words at the eementar" e1e) ,-- to 3-- words at the
intermediate e1e) and ?-- to A-- words at the ad1anced e1e. These
f i gures are) of course) e+treme" rough guides and ma" not be a''ro'riate
for man" reading situations. *oreo1er) the e+tract seected shoud be
ca'abe of 'ro1iding the. basis for a sufficient number of muti'e5choice
com'rehension items. (t is not an eas" task to find an e+tract which wi
i
su''ort a number of muti'e5choice items 5 e1en though the same e+tract
ma" form a basis for a arge number of trueKfase items or o'en5ended
Buestions. &enera" s'eaking) 'assages deaing with a series of e1ents) a
coection of facts) or different o'inions and attitudes make the best t"'es
of te+ts for testing 'ur'oses4 those deaing with a si nge idea or main thane
are rare" suitabe.
The ength of the e+tract shoud aso be reated to its e1e of
difficut"! a 'articuar" difficut or com'e+ 'assage woud 'robab" be
considerab" shorter than a more straightforward one. 9n the whoe) the
difficut" e1e of the te+t) howe1er) shoud coincide with the e1e of the
students@ 'roficienc" in Engish) but we must remember that the reading
matter used outside the test situation <e.g. sim'ified readers= shoud be
seected for enJo"ment and shoud thus be at a sight" ower e1e than the
actua standard of the reading skis acBuired. <The difficut" e1e of a te+t
de'ends chief" on the degree of the structura and e+ica com'e+it" of
the anguage used.=
;hen writing test items based on a reading te+t) the tester shoud
attem't to construct more items than the number actua" reBuired. .fter
the construction of the items) it is usefu to secure the ser1ices of one or
two coeagues so that a the items can be moderated. (n1ariab" this
'rocess brings to the attention of the item writer certain faws in some of
the items. .though a number of the faws wi be easi" rectified) in certain
cases it wi be necessar" to dis'ense with entire items. (n tests of grammar
and 1ocabuar") new items can awa"s be constructed in 'ace of the
discarded items) but this does not foow with reading com'rehension
items. The te+t itsef has to be rewritten) certain sections added and others
deeted in order to obtain the reBuired number of items. 0uch 'rocesses
are difficut and time5consuming! thus) it is awa"s an ad1antage toH@b@
113
construct in the first instance more items than are actua" reBuired. (f the
te+t wi not aow for more items) another more suitabe te+t shoud be)
chosen to a1oid wasting time at a ater stage.
(t is usefu to incude items testing the students@ abiit" to recognise5
5reference features in a te+t) no matter whether muti'e5choice) com'etion or
o'en5ended items are being constructed. . reference5word item can 'ro1ide the
e+aminer with s'ecific information about reading difficuties. (f a student fais
to 'ercei1e what the reference de1ice Dit@ refers to in the .. te+t) for e+am'e) the
e+aminer immediate" knows the reason for his or her faiure to understand
that 'art of the te+t.
The griLL" bear roams some 1, miion acres in rugged 'arts of the
/nited 0tates. .nd this great bear sti roams our imagination at wi!)
it is 'art of its natura habitat.
The word it in ine 3 refers to
.. @the /nited 0tates@ <ines 15,=
B. @this great bear@ <ine ,=
C. @our imagination@ <ine ,=
:. @its natura habitat@ <ine 3=
How man" muti'e5choice items shoud be set on one te+tG Cear")
the number of items wi de'end on the ength and com'e+it" of the te+t.
Howe1er) tests of reading com'rehension genera" contain fewer items
than other ski tests. >urthermore) the testees reBuire much more time! to
work through a reading com'rehension test since the" first ha1e to read the
te+t carefu" once or twice before the" can begin to answer the items
based on it. ;hie as itte as ten or fifteen seconds for each item can be
aowed in muti'e5choice tests of. grammar and 1ocabuar") at east one or
two minutes must be aowed for each item in the a1erage readir test) <if
the time reBuired to read the te+t is taken into account=. Conseghent")
such tests) though ong in terms of time) must of necessit" be short in terms
of items and) therefore) ess reiabe.
The construction of items de'ending sim'" on a matching of wordsand
'hrases shoud be a1oided. (tems shoud test more than a su'erficia
understanding of the te+t and shoud reBuire the testees to digest and
inter'ret what the" ha1e read. The foowing e+am'es show how
ineffecti1e items can be if testees are sim'" reBuired to match the
words in the items with the words in the te+t.
.t four o@cock on 0e'tember 3-th two men armed with iron bars
attacked a sodier in 6rior" 0treet.
;hat ha''ened at four o@cock on 0e'tember 3-thG
.. Two neminsi deraden with rinot babbers tackened a derisodt.
(magine that a testee did not understand much of the sentence to the 5e+t.
(n order to a''reciate this fu") it necessar"
to change
sight") and the te+t might a''ear to
u ike ti
.t four o@cock on 0e'tember 3-th two neminsi deraden with rinotH
babbers tackened a derisodt.
;hat ha''ened at four o@cock on 0e'tember 3-thG
.. Two neminsi deraden with rinot babbers tackened a derisodt5
. sight" better item stem woud be!
;hat ha''ened one afternoon at the end of 0e'temberG
112
i
(
Howe1er) to be com'ete" satisfactor") it woud be necessar" to rewrite
both the te+t and the item) as in the foowing e+am'e!
6au was sur'rised when he met 0ue at the 'art". He was under the
im'ression she had gone awa" from the ocait". The ast time he saw
her was when Jane was teaching her to dri1e. . few da"s afterwards she
had sudden" become i.
<first 1ersion= 6au was sur'rised when
.. 0ue went awa".
B. he met 0ue at the 'art".
C. Jane was teaching 0ue to dri1e.
:. 0ue sudden" became i.
<second 1ersion= 6au did not e+'ect to see 0ue because
.. he knew she was at the 'art".
B. he thought she had eft the district.
C. he had seen Jane teaching her to dri1e.
:. he had heard she was i.
There is often a tem'tation to concentrate too much on facts) figures
and dates when constructing test items based on a factua te+t. &enera"
s'eaking) figures and dates are incuded in a te+t chief" for the 'ur'ose of
iustration or to show the a''ication of a genera 'rinci'e. (t is usefu in
such cases to construct items which reBuire the testees to use the figures in
the te+t to state <or restate= the genera 'rinci'e behind them. E.g.!
>rom Januar" to :ecember ast "ear) ,21 'eo'e were kied and A),?3
were inJured in road accidents on the cit"@s roads. 187 of a the fata
accidents in1o1ed motorc"cists or their 'iion 'assengers) whie 28 O
in1o1ed 'edestrians and the remaining 32 the dri1ers and 'assengers of
motor 1ehices.
91er haf of a the 'eo'e kied in road accidents ast "ear were
.. motorc"cists and 'iion 'assengers.
B. 'edestrians.
C. dri1ers of buses) cars and orries.
11
:. both 'edestrians and motorists.
Testees can aso be encouraged to use the figures the" are gi1en in a te+t
and to work out sim'e arithmetica sums and 'robems. Cear") there is a
imit to the tasks which the testees ma" be reBuired to 'erform! otherwise the
test writer wi be testing something other than anguage skis. The
foowing is an e+am'e of an item which tests students@ abiit" to hande
sim'e facts and figures in Engish! the stem 'resents a usefu task 'ro1ided
that this kind of reading e+ercise is not o1erdone.
Latest re'orts from the northeast 'ro1inces state that at east si+teen
'eo'e ost their i1es in 0aturda"@s foods. . further nine 'eo'e) most"
chidren) are re'orted missing) beie1ed dead. 0e1en sma bo"s)
howe1er) had a miracuous esca'e when the" were swe't onto the
branches of some ta trees.
The tota number of 'eo'e re'orted dead or missing as a resut of
0aturda"@s foods is
.. 7 B. 2 C. 1A :. ,8 E. 3,
1,-
(
The choice of the5correct o'tion in each muti'e5choice item must de'end
on a testee@s com'rehension of the reading te+t rather than on genera
5
knowedge or inteigence. The foowing item) for e+am'e) can be
answered without an" knowedge of the te+t on which it has been based.
*emorising is easier when the materia to be earnt is
.. in a foreign anguage.
B. aread" 'art" known.
C. unfamiiar and not too eas".
:. of no s'ecia interest.
Care must be taken to a1oid setting distractors which ma" be true) e1en
though the" ma" not ha1e been e+'icit" stated b" the writer. (n the
foowing test item based on a reading te+t about the /nited Nations
and the dangers of war) C is the reBuired answer4 howe1er) a four o'tions
are correct 5 e1en though not stated in so man" words b" the writer.
;hat woud ha''en if there was a goba warG
.. Nations woud train men for war.
B. Lots of terribe wea'ons woud be made.
C. The whoe human race woud be com'ete" destro"ed.
:. 6eo'e woud grow 1er" des'erate.
The correct o'tion must .be rough" the same ength as the distractors. (n the
foowing test item the correct o'tion has been modified to such a
degre. that it a''ears as the ob1ious answer without e1en necessitating an"
reference to the te+t.
The curricuum at the new coege is a good one in man" wa"s because
it
.. incudes man" science courses.
B. offers a we5baanced 'rogramme in both the humanities and the
sciences.
C. is reaistic.
:. consists of usefu technica subJects.
. the o'tions must be grammatica" correct! there is a tendenc"
es'ecia" in reading com'rehension to o1erook the grammatica
a''ro'riateness of some of the distractors used. 9'tion : in the foowing
item can be rued out immediate" because it is ungrammatica.
The writer sa"s that he had studied engineering for
.. a ong time.
B. on" a 1er" short 'eriod.
C. se1era "ears.
:. ne1er.
:oube negati1es are on" confusing and such items as the foowing <based on
the e+tract on 'age 1,-= are best a1oided!
6au did not e+'ect to see 0ue because
.. he did not know she was at the 'art".
B. no one knew she had eft the district.
C. he hadn@t seen Jane teaching her to dri1e.
:. he didn@t reaise she was we.
. usefu de1ice in muti'e5choice tests of reading com'rehension is the
o'tion .LL 9> THE0E or N9NE 9> THE0E!
1,1
1
.ccording to the 'assage) what do some 'eo'e think there shoud be
outside a modern cit"G
.. Buses
B. Car 'arks
C. 9ffice buidings
:. Ta+is
E. .LL 9> THE0E
(f an o'tion ike E is used) it is ad1isabe to ha1e it as the correct answer in
at east one of the items. The testees shoud not be encouraged to think
that it has been incuded sim'" to make u' the reBuired number of
o'tions.
The foowing te+t and com'rehension itemsA iustrate some of the
guideines aid down. in.this section!
0tud" the foowing 'assage and then answer the Buestions set on it.
The Ca'ti1e is a strange but sincere and tender fim) as indeed one
woud e+'ect from a director of the caibre of *arce L"me. (n addition
to his keen sensiti1it") L"me has a strong feeing for historica
atmos'here) so a''arent in his earier fim
/nder the 0hadow of the
&uiotine)
in which the e1ents of the >rench #e1oution are de'icted
8
with sur'rising reaism and 1itait". (n
The Ca'ti1e L"me manages to
e1oke the atmos'here of an Engish town in the ear" 'art of the
nineteenth centur") not so much through the more ob1ious de1ices of
stage5coaches) od inns) and thatched cottages as through minute
attention to detais of s'eech) dress) customs) and mannerisms. 0imiar 1
in theme to
.dam Brown) The Ca'ti1e
is distinguished b" a sincerit"
which the former acks and which he's to transform this fim from an
ordinar" ad1enture stor" into a memorabe and a 1er" mo1ing traged".
Es'ecia" unforgettabe is the farewe scene at 6"mouth) when 5
Jonathan #obson sees Catherine ;insome on his wa" to the grim)
sBuaid shi' which is waiting to take him to .ustraia. #obson breaks
oose from his ca'tors for a feeting moment to bid farewe to
Catherine. @(@ 'ro1e m" innocence)@ he cries 1ehement" as he shakes
his fist at Catherine@s cousin.
.s the shi' sets sai) one enters a grotesBue nightmare word in
which e1i seems trium'hant. 9ur identification with #obson becomes
so 'ersona that we fee e1er" stroke of the fogging after he has been
caught steaing medicine for his sick com'anion. ;e share his s"m'ath"
for Joe Biggs as the od saior is haued under the shi'@s kee. (ndeed)
e1ents might we ha1e become unbearabe but for the ight reief
'ro1ided b" the comica antics of Bobo) the sma cabin bo" who ski's
about uncom'aining" doing whate1er task he is gi1en. ;e know) of
course) that utimate" e1i wi be 1anBuished) and so we are gi1en
strength to endure the ad1ersities which confront the hero. The mutin"
and the conseBuent esca'e of Jonathan #obson) therefore) come as no
sur'rise.
Suestions
<a= >or each of the foowing statements choose the word or 'hrase that
best com'etes the statement according to the informationO
contained in the 'assage. ;rite the number of the Buestion and the
answer "ou ha1e chosen in "our answer book.
1,,
<i= The Ca'ti1e was directed b"
.. Jonathan Brown.
B. .dam Brown.
C. *arce L"me.
:. Catherine ;insome.
<ii=
(n The Ca'ti1e *arce L"me con1e"s the atmos'here of the
nineteenth centur" chief" through
.. cose attention to sma detais.
B. the use of con1entiona scener".
C. stage5coaches) od inns) and thatched cottages.
:. de'icting dramatic e1ents of the time.
<iii= The 'assage im'ies that .dam Brown
was
.. a 1er" mo1ing fim.
B. a reaistic and 1ita fim.
C. an ordinar" ad1enture fim.
:. a sincere fim.
<i1= Jonathan #obson is angr" as a resut of
.. ha1ing to wait to go to .ustraia.
B. being wrong" con1icted.
C. meeting Catherine.
:. being reca'tured.
<1= 9n the 1o"age to .ustraia #obson
.. becomes i.
B. begins to ha1e nightmares.
C. is haued under the shi'@s kee.
:. recei1es a fogging.
<1i= Bobo is introduced into the stor" to he' us to bear the grim e1ents
b"
.. beha1ing in a strange but interesting wa".
B. making us augh.
C. doing e1er"thing without com'aining.
:. acting kind" toward the hero.
<1ii= ;e can endure the hero@s sufferings because we know
.. things cannot get worse.
B. the crew wi mutin".
C. good wi win in the end.
:. the hero is 1er" bra1e.
<1iii= The writer@s attitude to this fim is
OO
.. a''reciati1e
5 B. 'atronising.
C. scornfu.
:. critica.
<i+= The word @his@ in ine 3 refers to
.. @The Ca'ti1e@ <ine 1=
B. @one@ <ine 1=
C. @a director@ <ine ,=
:. @L"me@ <ine 3=
<+= The words @the former@ in ine 1, refer to
.. @theme@ <ine 11=
B. @.dam Brown@ <ine 11=
1,3
C. @The Ca'ti1e@ <fine 11=
:. @a sincerit"@ <ine 11=
<+i= The word @his@ in the 'hrase @;e share his s"m'ath"@ in ine ,3
refers to
.. @#obson@ <find ,1=
B. @his sick com'anion@ <ine ,3=
C. @Joe Biggs@ <fine ,?=
:. @the od saior@ <ine ,?=
<+ii= The word @he@ in ine ,7 refers to
.. @#obson@ <ine ,1=
B. @Joe Biggs@ <ine ,?=
C. @the comica antics@ <ine ,A=
:. @Bobo@ <ine ,.ALO
<b= Each of the foowing words and 'hrases can be used to re'ace one
word in the 'assage. >ind the words and write them in "our answer
book. Number "our answers.
<i= dragged
<ii= conBuered
<iii= troubes and misfortunes
<i1= 1er" brief
<1= fina"
3.7 Com'etion items Com'etion items measure reca rather than recognition. .though such
items are simiar in man" wa"s to o'en5ended Buestions in tests of reading
com'rehension) the" are often regarded as beonging more to the obJecti1e
categor" of test items. There is 1er" itte difference) howe1er) between the
foowing o'en5ended reading com'rehension Buestion!
;h" was the author sur'rised to meet :r 0hortG
and the eBui1aent com'etion item!
The author was sur'rised to meet :r 0hort because ...
/sua") com'etion items reBuire the testees to su''" a word or a
short 'hrase. /ness great care is taken to ensure that there is on" one
correct answer) the mar+ing wi 'ro1e 1er" difficut when the tester is
confronted with a 1ariet" of answers ranging from acce'tabe to
unacce'tabe. . 1aid inter'retations) whether or not these were in the
test writer@s mind at the time of the construction of the test) must be
regarded as correct.
T"'es of com'etion items for testing reading com'rehension are
di1ided into two grou's for ease of treatment! T"'e 1 consisting of banks for
com'etion in the items foowing the te+t) and T"'e , consisting of banks
in the te+t itsef.
T"'e 1 /ness carefu" constructed) this t"'e of com'etion test can
become mere" a matching e+ercise in which the words and 'hrases
reBuired in the com'etion are determined after a 'rocess of matching the
whoe item with the a''ro'riate 'art of the te+t. Howe1er) ess em'hasis is
'aced on the writing skis in such a test than in a test consisting of o'en5
ended Buestions. The first e+am'e of this item t"'e iustrates how a short
informa etter can be used to 'ro1ide the basis for com'etion items.
1,?
E+am'e 1
,8A ;eeton #oad)
,nd >oor)
Hong $ong.
7th June) 1233.
:ear :a1id)
( am 1er" sorr" that ( coud not meet "ou ast night. ( ho'e that "ou did
not wait too ong outside the New %ork Theatre. ( had to ook after m"
sma brother unti m" mother returned home. 0he was a ong time at
the doctor@s and she arri1ed home 1er" ate. ( ran a the wa" to the bus
sto') but ( had aread" missed the bus. ( decided to get on a tram and (
arri1ed at the New %ork Theatre at eight o@cock. ( did not think that "ou
woud sti be there because ( was three5Buarters of an hour ate. ( do
ho'e that "ou wi forgi1e me.
%our friend)
6eter
;rite one word or more in each bank.
1.
6eter i1es at ...
,. He wrote the etter on ...
3. 6eter coud not ea1e home because he had to wait for
. ( . to return.
?. His mother had been to the ...
8. 6eter went to the New %ork Theatre b" ...
5
A. He thought that :a1id ...
7. The word ...
. means e+cuse.
3. 6eter had5arranged to meet :a1id at ...
se1en on June ...
E+am'e , The second e+am'e gi1es the stimuus in the form of a theatre
ad1ertisement. (t has been incuded here as a reminder of the im'ortance of
1ar"ing te+t t"'es in a reading com'rehension test and of using the t"'e of
reading materia which the student ma" meet in 'aces where Engish is used
as an e1er"da" means of communication.
#ead the foowing ad1ertisement and com'ete each sentence. ;rite
one word or more in each s'ace.
NE;T9N THE.T#E
>9#THC9*(N& .TT#.CT(9N0
*onda") 3 Januar" for , weeks 5
*% >.T >#(EN:
Chares Laurence@s 'o'uar comed"
;ednesda") ,? Januar" to
0aturda") ,7 Januar"
0hanghai >esti1a Baet
'resents
0;.N L.$E
*onda") ,2 Januar" for one week
#/N .N: $(LL
Tim :anb"@s thriing m"ster"
1. The 0hanghai >esti1a Baet wi 'erform
on ... e1enings.
, . . wi be the
O O most amusing 'a".
3. (f "ou ike dancing) "ou shoud see
?. The 'a" written b"
.is 1er"
e+citing.
1,8
(
<
E+am'e 3 (n this com'etion item the students are reBuired to fi in a
tabe based on a 'assage for reading com'rehension. The information
e+tracted from the te+t is best tested b" reBuiring the students to 'ut it in
tabuar form.
The cit" with the highest tem'erature "esterda" was 0inga'ore. .t noon
the tem'erature in 0inga'ore was 33PC and at midnight the tem'erature
there was ,8PC. Tok"o had the second highest tem'erature. (t was on"
,PC ower there than in 0inga'ore at noon. The tem'erature in #ome at
noon was 3-PC) the same as in New %ork. Howe1er) New %ork@s
tem'erature at midnight was one degree ower than #ome@s. The noon
tem'erature in Cairo was ,2PC) one degree higher than that in .thens and
Hong $ong. The tem'erature at midnight in 6aris was 11PC ess than that at
noon in 6aris "esterda". .though 0tockhom@s tem'erature at midnight5w5
as5tire same as the tem'erature at midnight in 6aris) its noon tem'erature
was 1?PC. The cooest cit" was London) with a tem'erature of on" ,,PC at
noon and 13PC at midnight)
The fi1e cities with the highest tem'eratures had a ot of sunshine
throughout the da". .though the sun did not shine at a in .thens and
6aris) it did not rain. (t was aso coud" and dr" in 0tockhom) but it
rained hea1i" in both London and Hong $ong.
%esterda"@s ;eather .round the ;ord
0 7 0unn" C 7 Coud" # 7 #ain
Noon *idnight
,APC
Cairo
,APC 0
Hong $ong ,8PC
,,PC #
3-PC
6aris ,8PC
#ome ,8PC
0
0inga'ore
33PC ,8PC 0
0tockhom
C
Tok"o ,APC
0
T"'e , (n this item t"'e the testees are reBuired to com'ete the bank
s'aces in a reading te+t. The banks ha1e been substituted for what the test
writer considers are the most significant content words. ConseBucntZ) a
'ossibe weakness of such a test ma" resut from the faiure to su''"
adeBuate guideines to the testees4 the foowing is an e+am'e of a 't+r item
because the framework is insufficient to guide them. The inguistic cues are
thus inadeBuate and the testees are faced with the task of has ing to guess
what was in the e+aminer@s mind.
;hen we <1= ...
. something aong the <,= ...
.. it wi
cause<3=...
1,A
The foowing te+t iustrates how banks shoud be inters'ersed4 the
testee@s degree of success in com'eting the banks de'ends amost entire" on
his or her com'rehension of the whoe te+t.
;hen we side something aong the foor) it wi cause
<1=
. (f something is 1er" <,= ...
. there wi be a ot
of friction between it and the ground. Howe1er) friction is
<3=
. when something ros instead of <?= ...
. The
in1ention of the <8= ... . was rea" an attem't to reduce friction.
/ness there is snow or ice) it is much harder to <A= ...
something on a bo+ or sedge than in a cart. Ba5bearings are used a ot
in machiner" to <7= ... . friction. (t is friction which causes
<3= . to machiner" as the 1arious 'arts <2= ...
against one another. >riction is reduced if we 'ut oi onto the
<1-= . (t shoud not be forgotten) howe1er) that
<1 1=
. is aso usefu to us! it is necessar") for instance) for car
whees to gri' the <1,= ...
E+am'e 1 (n some tests certain etters of missing words are gi1en. (n these
cases) the testees are genera" informed that each dash in the bank
signifies a etter.
The might" .maLon f555s into the .tantic near the EBuator! its
es5555" is about 17- mies wide. The w55th55 is often so mist" that
the b555s of the ri1er cannot be seen from a shi') e1en if it is
'5s555g Buite cose to them.
(t is aso 'ossibe to 'ro1ide on" the initia etter of the missing word. This
item is not to be recommended for most 'ur'oses as the incusion of etters
can often create menta bocks and on" confuse students if the" fai to
think of the e+act word reBuired. This is cear" disad1antageous for
reiabe assessment as a student might understand the 'assage and woud
otherwise ha1e been abe to com'ete the 'assage with a suitabe s"non"m.
E+am'e , . 1ariation of this t"'e of reading com'rehension ma"
incor'orate the muti'e5choice techniBue!
.stroog" is the ancient <1= ...
. of teing what wi
<,= . in the future b" stud"ing the <3= ... . of the stars
and 'anets. <?= ...
.. astroogers thought that the stars and
'anets infuenced the <8= ... . of men) the" caimed the" coud
te <A=
<1= custom business magic knowedge art
<,= coincide ha''en chance come forete
<3= 'aces sha'es times 'ositions ight
<?= Howe1er Because .though >or ;hie
<8= affairs matters businesses chances times
<A= horosco'es future ad1ice fortunes uck
E+am'e 3 The foowing t"'e of item is used to most ad1antage when the
item itsef is reated to the kinds of tasks the testees are reBuired to
'erform in their studies or in rea ife. >or e+am'e) testees can be
'ro1ided with the dictionar" definitions of a number of words. <This
materia is sim'" e+tracted from a good dictionar") together with
information concerning 'ronunciation) 1erb 'atterns) 'arts of s'eech) etc.
1,7
i
and e+am'es of use5where a''ro'riate.= /nderneath the dictionar"
e+tract are 'rinted incom'ete sentences) which testees are reBuired to
com'ete with the most a''ro'riate word.
E+am'e ? (n the foowing t"'e of item the reading com'rehension te+t is
reated to information contained in a diagram or tabe.
Look at the foowing diagram! (t shows the 'aces to which trains tra1e
from London. (t aso shows the times taken to tra1e to these 'aces.
N
; ZE
0
Ho"head
?hr ,2 Chester
,hr 32
Hereford
,hr ??
0wansea
Edinburgh
Z?hr 3?
Hu
,hr 37
:oncaster
1 hr 32
Norwich
1 hr 83
,hr ?1
A
( N
Cardiff
1 hr ?3 0outham'ton
;e"mouth 1 hr 2
,hr ?, 5@5Q
L9N:9N
Cochester
8- min
:o1er
1 hr ,7
(t takes two hours and se1en minutes to tra1e to <1= <1=
from London) haf an hour ess than it takes to
<,=. (t takes one hour and thirt"5nine minutes to <,=
tra1e to <3=) which is aso in the north of Engand. (t <3=
takes the same time to reach 1?= as it does to reach
<?=
*anchester. <8= is the furthest 'ace from London. <8=
<A=) a sma town in the west of the countr") is on" <A=
fi1e minutes nearer London than Edinburgh is. Just
under two hours@ tra1eing time from London is <7=) <7=
a cit" to the east of London. (t takes a itte under an
hour and a haf to reach <3=) a 'easant town to the <3=
southeast of London. To the southwest of London is
<2=) which is one hour and nine minutes from there <2=
and is on the wa" to <1-=. (f "ou tra1e to <11=) it wi <1-=
take two hours and fort"5one minutes. %ou wi 'ass <11=
through <1,= amost an hour before "ou reach there. <1,=
<13= is the second argest cit" in Engand) one hour <13=
and thirt"5four minutes from London. (f "ou i1e in
<1?=) "ou can catch the Birmingham train) the <1?=
*anchester train or the Ho"head train. . three
trains wi 'ass through this o1e" countr" town.
E+am'e 8 Com'etion items are aso usefu for testing the abiit" to
deduce word meanings from the cues a1aiabe in the conte+t. 0uch an
abiit" is of the utmost im'ortance in reading and shoud be encouraged in
both teaching and testing at a e1es. The foowing is an e+am'e of an
1,3
1
item designed to test this abiit" at the intermediate e1e. 0tudents are
instructed to read the entire te+t before attem'ting to com'ete each
bank. The first e+am'e contains on" one bank whie the second e+am'e
contains three banks. each reBuiring the same word.
<9ne bank! one word=
The &reat 6"ramid at &iLa in Eg"'t is without doubt sti one of the
greatest ...
. in the word) e1en though it has been studied 1er"
thorough" o1er the ast centur" or so. (t ma") howe1er) be 'ossibe to
find out more about the '"ramid b" coser stud" o1er the ne+t few
"ears. 5
<Three banks! one word=
Ja'an) in the interests of both the Ja'anese and the word ...
needs to be assured that efforts wi be made to stabiise the "en. >or its
'art) Britain is now taking acti1e ste's to im'ro1e its ...
. b"
de1eo'ing new industries and cutting down on im'orted goods of a
kinds. Taks now being hed in 6aris between >rance and &erman" are
directed at wa"s of increasing the manufacturing ca'abiities of the two
countries in the ho'e that this wi ead to a genera im'ro1ement in the
Euro'ean ...
. and create more Jobs.
3.3 #earrangement These two item t"'es are 'articuar" usefu for testing the abiit" to
items
understand a seBuence of ste's in a 'rocess or e1ents in a narrati1e. ;hie
in an e+ercise for cassroom 'ractice the students wi often be reBuired to
rewrite the Jumbed sentences in their correct seBuence) it is ob1ious"
'referabe for testing 'ur'oses to instruct them to write sim'" the
numbers or etters of the Jumbed sentences. (t is aso ad1isabe to 'ro1ide
them with one or two answers! if students start off b" 'utting the first two
or three sentences in the wrong order) it ma" be im'ossibe for them to 'ut
the remaining sentences in the correct order. (n other words) one wrong
answer wi ine1itab" ead to a second wrong answer and 'ossib" a third)
and so on. This weakness must be borne in mind when marking! for
instance) shoud two incorrect answers <i.e. one incorrect answer
automatica" resuting in another= be 'enaised b" the oss of two marks@
T"'e ( The first of these items takes the form of Jumbed sentences which
the students ha1e to @unscrambe@ and arrange in the correct order. The
students are instructed to rearrange the etters of the sentences in their
correct order in the bo+es beow.
.. The dogs were se'arated from their human masters and were 'ut
into arge fieds.
B. >or instance) the" wagged their tais) the" barked and growed) and
the" fawned on animas which 'ossessed food.
C. 6s"choogists obser1ing them found that the" reacted towards each
other in much the same wa" as the" used to res'ond to 'eo'e.
:. #ecent" an e+'eriment was conducted with a grou' of dogs to find
out how much their beha1iour was) in fact) sim'" a resut of human
en1ironment.
E. 6u''ies born to those dogs and ke't out of contact with 'eo'e
showed the same beha1iour 'atterns! the" were e+treme" wid and
afraid of human beings.
1,2
T"'e , The Jumbed sentences in the second item are based on a reading
com'rehension te+t and shoud be unscrambed in the ight of the
information contained in the te+t. The students must write the etters of
the sentences in their correct order in the bo+es at the end of the item.
.gain) howe1er) the fact that one mistake ine1itab" resuts in another
reduces the item@s reiabiit".
;hen a customer waks into a tra1e agenc" to make a booking) the cerk
behind the counter turns immediate" to the sma com'uter unit on the
desk. The unit ooks ike a s"nthesis of a tee1ision set and a t"'ewriter.
The customer states the date and destination of the fight reBuired) and
the cerk t"'es this information on the ke"board. .s each ke" is 'ressed)
a etter is formed on the screen of the unit. The cerk then checks the
information onthe screen before transmitting the data to a centra
com'uter. This com'uter contains a the information about current
bookings and destinations) and ra'id" estabishes whether the new
booking is 'ossibe. The com'uter immediate" sends a re'") indicating
the number of 1acancies on the fight reBuested or showing that the
fight is fu" booked. ;hie these figures are being dis'a"ed on the
screen) the" ma" change to indicate that another booking has been
made in another 'art of the word.
The cerk now t"'es in the customer@s reser1ation) after which the
com'uter wi reBuest his or her name and address and then other
information. This information) incuding an indication of how the ticket
wi be 'urchased <cash or credit card=) is then t"'ed onto the screen.
Ne+t the com'uter confirms the booking and reBuests that 'a"ment be
made. ;hen the customer has 'aid for the ticket) the cerk t"'es this
information into the com'uter as we. >ina") if modern eBui'ment is
being used) it is 'ossibe for the com'uter to 'rint out a ticket on the
s'ot.
Now 'ut these sentences in the correct order. ;rite on" the
etter of
each sentence in each bo+. <Three bo+es ha1e been com'eted for "ou.=
.. :etais about the seats a1aiabe are sent back.
B. The com'uter then wants to be informed about the method of
'a"ment.
C. The tickets are issued.
:. The com'uter asks for 'ersona detais.
E. The customer makes his or her reBuest to the tra1e agent.
>. The reBuest is sent to the main com'uter.
&. The customer goes to a tra1e agent.
H. The com'uter reBuests 'a"ment.
1. The tra1e agent feeds the initia reBuest into a sma com'uter unit.
J. The booking is t"'ed into the com'uter.
$. The booking is confirmed.
&
> J
.s in the case of trueKfase t"'e reading items) it is 'ossibe to im'ro1e
the reiabiit" of the item b" introducing an additiona feature! i.e. D:on@t
know@) D(nformation not gi1en@) DNot a''icabe@. The foowing item7 is based
on an artice contained in a se'arate 'u5out sheet using a
news'a'er format <but not shown here=.
13-
11
%ou woud ike to earn something about migraine and so read the
artice @9ne5sided Headache@.
(n what order does the writer do the foowing in her articeG To answer
this) 'ut the number 1
in the first answer coumn ne+t to the one that
a''ears first) and so on. (f an idea does not a''ear in the artice) write
NK. <not a''icabe= in the answer coumn.
;rite in e1er" bo+
a. 0he gi1es some of the histor" of migraine. a.
b. 0he recommends s'ecific drugs.
b.
c. 0he recommends a herba cure. C.
d. 0he describes migraine attacks. d.
e. 0he gi1es genera ad1ice to migraine sufferers. e.
3.2
CoLe 'rocedure .though simiar in a''earance to com'etion items) coLe tests shoud not
be confused with sim'e bank5fiing tests. (n ordinar" com'etion tests the
words for deetion are seected subJecti1e" <consisting arge" of structura
words in tests of grammar and ke" content words in 1ocabuar" or reading
tests=. (n coLe tests) howe1er) the words are deeted s"stematica". Thus)
once the actua te+t has been chosen) the construction of a coLe test is
'ure" mechanica! e1er" nth word is deeted b" the test writer. Certain
test writers argue that the bank substituted for the deeted word shoud
corres'ond to the ength of the missing word but in most coLe tests a the
banks are now of uniform ength. /ness a 'hotostat co'" of the actua
'rinted te+t is being used <in which case the words are deeted before the
'hotoco'"ing 'rocess=) it is sim'e5 to insert banks of uniform ength.
The inter1a at which words are deeted is usua" between e1er" fifth
and e1er" tenth. word. Howe1er) if e1er" se1enth word has been deeted in
the first few sentences) then e1er" se1enth word must be deeted for the
rest of the te+t. The fifth) si+th and se1enth words are the most wide"
fa1oured for deetion in doLe tests. (f e1er" third or fourth word is deeted)
the student wi ha1e e+treme difficut" in understanding the te+t as
insufficient cues wi be a1aiabe. (f e1er" tenth or twefth word is deeted)
it wi be necessar" to ha1e a ong te+t. >or e+am'e) if there are ?-
deetions e1er" se1enth word) the te+t wi be ,3-53-- words in ength <i.e.
?- + 7=4 if) howe1er) there are ?- deetions e1er" twefth word) the ength
of the te+t wi be ?3-58-- words <i.e. ?- + 1,=.
The choLe test was origina" intended to measure the reading difficut"
e1e of a te+t. /sed in this wa") it is a reiabe means@ of determining
whether.or not certain te+ts are at an a''ro'riate e1e for 'articuar
grou's of students <both nati1e s'eakers and non5nati1e s'eakers=. The
reading te+t being e1auated is gi1en to a grou' of students and the
a1erage score of a the students in the grou' is obtained. (f the mean score
of the grou' is o1er 83 'er cent) the materia can be used b" the students
for reading at Dthe inde'endent e1e@) the te+t being considered eas"
enough for students to read on their own without an" he'. (f the mean
score obtained is between ?? and 83 'er cent) howe1er) the materia is
suitabe for use at Dthe instructiona e1e@ 5 i.e. with the he' of the
131
1
teacher. (f the mean score is beow ?? 'er cent) the te+t is described as
being at Dthe frustrationa e1e@ and shoud not be used e1en with the he'
of a teacher. Later research) howe1er) has shown that this range of scores
ma" 1ar" considerab") the instructiona e1e on some occasions ranging
from 33 to 8- 'er cent and on other occasions from ?7 to A1 'er cent.
6erha's the most common 'ur'ose of the doLe test) howe1er) is to
measure reading com'rehension. (t has ong been argued that doLe
measures te+tua knowedge! i.e. an awareness of cohesion in a te+t)
in1o1ing the interde'endence of 'hrases) sentences and 'aragra'hs within
the te+t.I /nike the other t"'es of items treated here) howe1er) a true
doLe test <i.e. the mechanica deetion of words in the te+t= is genera"
used to measure goba reading com'rehension 5 athough insights can
undoubted" be gained into 'articuar reading difficuties. Howe1er) it is
aso argued that doLe measures an under"ing goba inguistic abiit" <aid55
indeed e1en knowedge of the word= rather than sim'" those skis
associated with reading com'rehension. .s the doLe 'rocedure is now
such an im'ortant feature in anguage testing <es'ecia" in the /nited
0tates=) the whoe subJect is treated at much greater ength in the foowing
cha'ter.
*eanwhie) the foowing e+am'es ha1e been incuded in this section
to demonstrate how doLe 'rocedure can be a''ied to the testing of
reading com'rehension at both the eementar" and the more ad1anced
e1es.
E+am'e 1 <Eementar"=
9nce u'on a time a farmer had three sons. The farmer was rich and
had man" fieds) but his sons were aL". ;hen the farmer was d"ing) he
caed his three sons to him. @( ha1e eft "ou ...
which wi
make "ou ... .. he tod them. @But .. must dig in
a . fieds to find the .
. where the treasure
is
.fter the od man ... his three aL" sons ...
out
into the fieds . . began to dig. @(@ . the first to
find . 'ace where the treasure ...
buried)@ cried the
edest .. . @That@s the fied where ... . father 'ut the
treasure)@ . . another son. The three ...
. dug a the
fieds . se1era "ears) but the" ... no treasure.
Howe1er) man" ... . grew in the fieds ..
. the sons had
dug. . 1egetabes made them 1er" ...
E+am'e , <.d1anced=
(t is estimated that in the ast two thousand "ears the word has ost
more than a hundred s'ecies of animas. . simiar number of s'ecies of
birds has aso become e+tinct. The rea significance of ...
figures) howe1er) ies in ...
. fact that amost three5
Buarters ... . a the osses occurred .. . the 'ast
hundred "ears ... . as a direct resut .
man@s
acti1ities on this ... (t is essentia for . . whoe
'rocess of e1oution ...
. the e+tinction of certain ...
shoud occur o1er a ... . of time. But e+tinction ..
occur b" nature@s design ...
. not as a resut ... 5. . the
acti1ities of man ...
. is b" no means ... . to the
'reser1ation of ... . s'ecies of anima and ...
ife.
1 3,
Conser1ation means the ... . of a heath"
en1ironment ... . a whoe. (f conser1ation ...
ignored)
then within a ...
. short time our water ... . wi be
found inadeBuate. .. . seas and ri1ers wi ..
. fewer
fish) our and ...
'roduce fewer cro's) and ... . air we
breathe wi ... . 'oisonous. (t becomes on" ... .O!
matter
of time before ...
. heath deteriorates and before ...
3.1- 9'en5ended and
misceaneous items
3.11 Cursor" reading
together with e1er" other ... . thing) disa''ears from
the ... . of the earth.
The term Do'en5ended@ is used to refer to those Buestions which eicit a
com'ete" subJecti1e res'onse on the 'art of the testees. The res'onse
reBuired ma" range from a one5word answer to one or two sentences!
<9ne5word answer=
&i1e the name of the town where the writer had a bad accident.
<.nswer in a few words=
%ou ha1e a friend who is keen on cross5countr" running. ;hich e1ent
can he enter at the end of the monthG
<0entence answer=
.ccording to the artice) wh" do "ou think so few foreign cars ha1e bee n
im'orted into 0inga'ore recent"G
;hen marking o'en5ended items which reBuire answers in sentences) it is
freBuent" ad1isabe to award at east two or three marks for each correct
answer. (f the ma+imum for a correct answer is three marks) for e+am'e) the
marking guide might be as foows!
Correct answer in a grammatica" correct sentence or a sentence
containing on" a minor error
Correct answer in a sentence containing one or two minor errors <but
causing no difficut" in understanding= ,
Correct answer but 1er" difficut to understand because of one or more
maJor grammatica errors ... 1
(ncorrect answer in a sentence with or without errors ... ...-
(t is awa"s usefu to write down 'recise" how marks shoud be awarded) e1en
if on" one 'erson is marking the items. This marking scheme wi then ser1e as
a reminder at a times. (t is) of course) essentia to write brief guideines if
more than one e+aminer is s'arking the items.
>ina") when constructing a reading com'rehension test) the test
writer shoud remember to et the te+t itsef determine the t"'es of items
set. (ndeed) if this 'rinci'e is obser1ed) it foows that different 'arts of a
'articuar te+t wi freBuent" reBuire different item h"'es. (t is now
becoming common 'ractice for more than one t"'e of item to be used to
test com'rehension of the same te+t. Thus) a reading com'rehension
'assage ma" be foowed b" one or two muti'e5choice items) se1era
trueKfase items) a few com'etion items and one or two o'en5ended items.
The tite of this section ser1es as a genera term to denote the skis in1o1ed in
reading Buick") skimming and scanning. The term
skimming is used to denote the
method of gancing through a te+t in order to become famiiar with the gist of the content4
scanning refers to the skis used when reading in order to ocate s'ecific information.
1353
(
(n tests of reading s'eed the students are genera" gi1en a imited
time in which to read the te+t. Care must be taken to a1oid constructing
Buestions on the ess ree1ant 'oints in the te+t) but the students shoud be
e+'ected to be famiiar with the successi1e stages in which the te+t is
de1eo'ed. The actua reading s'eed considered necessar" wi be arge"
determined b" the t"'e of te+t being read and wi 1ar" according to the
'ur'ose for which it is being read. (t is sufficient to note here that 'oor
readers <nati1e s'eakers= genera" read beow ,-- words 'er minute4 a
s'eed of between ,-- and 3-- words 'er minute is considered to be an
a1erage s'eed4 and 3-- to 8-- words is considered fast. 9n the whoe) it is
reaistic to e+'ect no more than a reading s'eed of 3-- words from man"
ad1anced earners of a second anguage. *ost 'eo'e tend moreo1er to
read at a sower rate under test conditions or in an" situation in which the"
are reBuired to answer Buestions5on a te+t.
(n tests of skimming) the rubrics genera" instruct the students to
gance through the te+t and to note the broad gist of the contents. The" are
then gi1en a sma number of Buestions concerning on" the maJor 'oints
and genera outine of the te+t. 0ometimes at the end of the skimming the
students are aowed a few minutes to Jot down an" notes the" wish to
make) but the" are not usua" aowed to refer back to the te+t. (f the
students are aowed to retain the te+t) the time for answering the 1arious
Buestions on it wi be imited to order to discourage them from referring
back too often to the te+t.
(n scanning tests) the Buestions are gi1en to the students before the"
begin to read the te+t) thus directing them to read the te+t for s'ecific
information. (n such cases) it is he'fu to set sim'e o'en5ended Buestions
<e.g. D;hat is the writer@s 1iew of modern trans'ortationG@= rather than
muti'e5choice items. The atter t"'e of item tends on" to confuse the
students since the" then find it necessar" to kee' in mind four or fi1e
o'tions for each item whie the" are reading.
Tests of s'eed reading shoud be administered on" when students
ha1e been adeBuate" 're'ared for the tasks in1o1ed in such tests. (t is
gross" unfair to test those reading strategies which ha1e ne1er 're1ious"
been 'ractised.
Notes and references
1
Based in 'art on *unb") J 1273 Communicati1e 0"abus :esign. Cambridge
/ni1ersit" 6ress
, /ni1ersit" of 9+ford :eegac" of Loca E+aminations! E+amination in Engish
0tudied as a >oreign Language) 6reiminar" Le1e) 1231
3 Joint *atricuation Board! Test in Engish <91erseas=) *arch 123,
? Joint *atricuation Board! Test in Engish <91erseas=) *arch 1233
8 The te+t is sight" ada'ted from an artice b" .nthon" Tucker in The &uardian)
0e'tember 8th 12A2.
A This com'rehension test was first constructed b" the author for the Hong $ong
Engish 0choo Certificate E+amination <Education :e'artment) Hong $ong= in
12A3) but was ater e+'anded to test@awareness of reference de1ices.
7
#o"a 0ociet" of .rts! The Communicati1e /se of Engish as a >oreign Language)
Test of #eading) .d1anced Le1e) June 123?
3 0ee Cohen) . : 123- Testing Language .biit" in the Cassroom. Newbur"
House
138
Testing the writing skis
2.1
The writing skis
The writing skis are com'e+ and sometimes difficut to teach) reBuiring
master" not on" of grammatica and rhetorica de1ices but aso of
conce'tua and Judgementa eements. The foowing ana"sis attem'ts to
grou' the man" and 1aried skis necessar" for writing good 'rose into fi1e
genera com'onents or main areas.
5 anguage use! the abiit" to write correct and a''ro'riate sentences4
5 mechanica skis! the abiit" to use correct" those con1entions 'ecuiar to
the written anguage 5 e.g. 'unctuation) s'eing4
5 treatment of content! the abiit" to think creati1e" and de1eo'
thoughts) e+cuding a irree1ant information4
5 st"istic skis! the abiit" to mani'uate sentences and 'aragra'hs) and
use anguage effecti1e"4
5 Judgement skis! the abiit" to write in an a''ro'riate manner for
a
'articuar 'ur'ose with a 'articuar audience in mind) together with an
abiit" to seect) organise and order ree1ant information.
The actua writing con1entions which it is necessar" for the student to
master reate chief" <at the eementar" stages= to 'unctuation and s'eing.
Howe1er) in 'unctuation there are man" areas in which 'ersona
Judgements are reBuired) and tests of 'unctuation must guard against being
too rigid b" recognising that se1era answers ma" be correct. /nfortunate".
tests of 'unctuation and s'eing ha1e often tended to inhibit writing and
creati1it".
9f far greater im'ortance in the teaching and testing of writing are
those skis in1o1ing the use of Judgement. The abiit" to write for a
'articuar audience using the most a''ro'riate kind of anguage is essentia
for both nati1e5s'eaking and foreign student aike. The use of correct
registers becomes 5an5im'ortant ski at ad1anced e1es of writing. >aiure
to use the correct register freBuent" resuts in incongruities and
embarrassment. ;hereas nati1e s'eakers earn to make distinctions of
register intuiti1e") students of foreign anguages usua" e+'erience
'robems in mastering this com'e+ area of anguage. The 1arious kinds of
register incude cooBuiaisms) sang) Jargon) archaic words) ega
anguage) standard Engish) business Engish) the anguage used b"
educated writers of Engish) etc. The 'ur'ose of writing wi aso he' to
estabish a 'articuar register! for e+am'e) is the student writing to
entertain) inform) or e+'ainG
13@
. 'iece of continuous writing ma" take the form of a narrati1e)
descri'tion) sur1e") record) re'ort) discussion) or argument. (n addition to
the subJect and the format) the actua audience <e.g. an e+aminer) a
teacher) a student) a friend= wi aso determine which of the 1arious
registers is to be used. ConseBuent") the use of a''ro'riate register in
writing im'ies an awareness not on" of a writing goa but aso of a
'articuar audience.
.fter the 'ur'ose of writing and the nature of the audience ha1e been
estabished) Judgement is again reBuired to determine the seection of the
materia which is most ree1ant to the task at hand <bearing in mind the
time a1aiabe=. 9rganisation and ordering skis then foow seection.
2.,
Testing .n attem't shoud be made to determine the t"'es of writing tasks with
com'osition writing
which the students are confronted e1er" da". 0uch tasks wi 'robab" be
associated with the writing reBuirements im'osed b" the other subJects
being studied at schoo if the medium of instruction is Engish. 0hort
artices) instructions and accounts of e+'eriments wi 'robab" form the
main bod" of writing. (f the medium of instruction is not Engish) the
written work wi often take the form of consoidation or e+tension of the
work done in the cassroom. (n both cases) the students ma" be reBuired to
kee' a diar") 'roduce a magaLine and to write both forma and informa
etters. The concern of students foowing a 'rofession or in business wi
be chief" with re'ort5writing and etter5writing) whie at coege or
uni1ersit" e1e the" wi usua" be reBuired to write <technica= re'orts
and 'a'ers.
9ne arge 'ubic e+amining bod"@ e+'icit" states the kinds of writing
tasks its e+aminations test and the standards of writing e+'ected in the
'erformance of those tasks!
. successfu candidate wi ha1e 'assed an e+amination designed to test
abiit" to 'roduce a seection of the foowing t"'es of writing!
Basic Le1e! Letter4 6ostcard4 :iar" entr"4 >orms
(ntermediate Le1e! .s Basic Le1e) 'us &uide4 0et of instructions
.d1anced Le1e! .s (ntermediate Le1e) 'us News'a'er re'ort4 Notes
The candidate@s 'erformance wi ha1e met the foowing minimum
criteria!
Basic Le1e! No confusing errors of grammar or 1ocabuar"4 a 'iece of
writing egibe and readi" inteigibe4 abe to 'roduce sim'e
unso'histicated sentences.
(ntermediate Le1e! .ccurate grammar) 1ocabuar" and s'eing) though
'ossib" with some mistakes which do not destro" communication4
handwriting genera" egibe4 e+'ression cear and a''ro'riate) using a
fair range of anguage4 abe to ink themes and 'oints coherent".
.d1anced Le1e! E+treme" high standards of grammar) 1ocabuar" and
s'eing4 easi" egibe handwriting4 no ob1ious imitations on range of
anguage candidate is abe to use accurate" and a''ro'riate"4 abiit" to
'roduce organised) coherent writing) dis'a"ing considerabe
so'histication.
(n the construction of cass tests) it is im'ortant for the test writer to f i
nd out how com'osition is tested in the first anguage. .though the
em'hasis in the teaching and testing of the skis in Engish as a
13A
foreignKsecond anguage wi of necessit" be Buite different to the
de1eo'ment of the skis in the first anguage) a com'arison of the abiities
acBuired and methods used is 1er" he'fu. (t is cear" udicrous. for
instance) to e+'ect in a foreign anguage those skis which the students do not
'ossess in their own anguage.
(n the 'ast) test writers ha1e been too ambitious and unreaistic in
their e+'ectations of testees@ 'erformances in com'osition writing! hence
the constant com'aint that reati1e" few foreign earners of Engish attain
a satisfactor" e1e in Engish com'osition. >urthermore) the backwash
effect of e+aminations in1o1ing com'osition writing has been unfortunate!
teachers ha1e too often antici'ated e+amination reBuirements b" beginning
free com'osition work far too ear" in the course. The" ha1e @'rogressed@
from controed com'osition to free com'osition too ear") before the basic
writing skis ha1e been acBuired.
Howe1er) once the students are read" to write free com'ositions on
carefu" chosen reaistic to'ics) then com'osition writing can be a usefu
testing too. (t 'ro1ides the students with an o''ortunit" to demonstrate
their abiit" to organise anguage materia) using their own words and
ideas) and to communicate. (n this wa") com'osition tests 'ro1ide a degree
of moti1ation which man" obJecti1e5t"'e tests fai to 'ro1ide.
(n the com'osition test the students shoud be 'resented with a cear"
defined 'robem which moti1ates them to write. The writing task shoud be
such that it ensures the" ha1e something to sa" and a 'ur'ose for sa"ing it.
The" shoud aso ha1e an audience in mind when the" write. How often in
rea5ife situations do 'eo'e begin to write when the" ha1e nothing to
write) no 'ur'ose in5writing and no audience in mindG Thus) whene1er
'ossibe) meaningfu situations shoud be gi1en in com'osition tests. >or
e+am'e) a brief descri'tion of a rea5ife situation might he gi1en when
reBuiring the students to write a etter!
%our 'en5friend is going to 1isit "our countr" for a few weeks with her
two brothers. %our house is big enough for her to sta" with "ou but
there is not enough room for her brothers. There are two hotes near
"our house but the" are 1er" e+'ensi1e. The third hote is chea'er) but it
is at east fi1e mies awa". ;rite a etter to "our 'en5friend) e+'aining
the situation.
Com'osition tites which gi1e the students no guidance as to what is
e+'ected of them shoud be a1oided. E+am'es of 'oor tites which fai to
direct the students@ ideas are . 'easant e1ening) *" best friend) Look
before "ou ea'. . good fim which
( ha1e recent") seen.
;ith the em'hasis on communicati1e testing) there is a tendenc" for
test writers to set tasks asking5the students to write notes and etters in
their own roe <i.e. without 'retending to be someone ese=. Tasks
reBuiring the students to act the 'art of another 'erson are often a1oided
as it is fet the" are ess reaistic and communicati1e. Howe1er) this is
usua" far from being the case. (t is usefu to 'ro1ide the students not on"
with detais about a s'ecific situation but aso with detais about the
'articuar 'erson the" are su''osed to be and the 'eo'e about <or to=
whom the" are writing. Com'are) for e+am'e) the two foowing tasks!
<a= ;rite a etter) teing a friend about an" interesting schoo e+cursion on
which "ou ha1e been.
<b= %ou ha1e Just been on a schoo e+cursion to a nearb" seaside town.
137
Howe1er) "ou were not taken to the beach and "ou had no free time at
a to wander round the town. %ou are 1er" keen on swimming and "ou
aso enJo" going to the cinema. %our teacher often tes "ou that "ou
shoud stud" more and not waste "our time. 9n the e+cursion "ou
1isited the aw courts) an art gaer" and a big museum. (t was a 1er"
boring a'art from one room in the museum containing od5fashioned
armour and scenes of battes. %ou found this room far more interesting
than "ou thought it woud be but "ou didn@t tak to "our friends or
teacher about it. (n fact) "ou were so interested in it that "ou eft a sma
camera there. %our teacher tod "ou off because "ou ha1e a re'utation for
forgetting things. 9n" "our cousin seems to understand "ou. ;rite a etter
to him) teing him about the e+cursion.
.though the former task is one which students ma" concei1ab" ha1e
to 'erform in rea ife) the atter task wi resut in far more reaistic and...
natura etters from the students sim'" because the s'ecific detais make
the task more meaningfu and 'ur'osefu. The detaied descri'tion of both
the situation and the 'erson in1o1ed he's the students to sus'end their
disbeief and gi1es the task an immediac" and reaism which are essentia
for its successfu com'etion.
Two or more short com'ositions usua" 'ro1ide more reiabe guides
to writing abiit" than a singe com'osition) enabing the testing of different
registers and 1arieties of anguage <e.g. a brief) forma re'ort=. (f the
com'osition test is intended 'rimari" for assessment 'ur'oses) it is
ad1isabe not to aow students a choice of com'osition items to be
answered. E+amination scri'ts written on the same to'ic gi1e the marker a
common basis for com'arison and e1auation. >urthermore) no time wi
be wasted b" the testees in deciding which com'osition items to answer. (f)
on the other hand) the com'osition test forms 'art of a cass 'rogress test
and actua assessment is thus of secondar" im'ortance) a choice of to4 ics
wi cater for the interests of each student.
>ina") the whoe Buestion of time shoud be considered when
administering tests of writing. ;hie it ma" be im'ortant to im'ose strict
time imits in tests of reading) such constraints ma" 'ro1e harmfu in tests
of writing) increasing the sense of artificiait" and unreait". *oreo1er) the
fact that candidates are e+'ected to 'roduce a finished 'iece of writing at
their 1er" first attem't adds to this sense of unreait". 0tudents shoud be
encouraged to 'roduce 'reiminar" drafts of whate1er the" write) and this
means gi1ing them sufficient time in an e+amination to do this. 9n" in this
wa" can writing become a genuine communicati1e acti1it".
2.3 0etting the (n addition to 'ro1iding the necessar" stimuus and information reBuired
com'osition
for writing) a good to'ic for a com'osition determines the register and
st"e to be used in the writing task b" 'resenting the students with a
s'ecific situation and conte+t in which to write. 0ince it is easier to
com'are different 'erformances when the writing task is determined more
e+act") it is 'ossibe to obtain a greater degree of reiabiit" in the scoring of
com'ositions based on s'ecific situations. >urthermore) such
com'osition tests ha1e an e+ceent backwash effect on the teaching and
earning 're'arator" to the e+amination.
The difficut" in constructing such com'ositions arises in the5writing
of the rubrics. 9n the one hand) if the descri'tion of the situation on which
the com'osition is to be based is too ong4 then the te+t becomes more
133
of a reading com'rehension test and there wi be no common basis for
e1auation. 9n the other hand) howe1er) sufficient information must be
con1e"ed b" the rubric in order to 'ro1ide a reaistic) he'fu basis for the
com'osition. (t is im'ortant) therefore) that e+act" the right amount of
conte+t be 'ro1ided in sim'e anguage written in a concise and ucid
manner. The foowing rubric) for e+am'e) can be sim'ified considerab"!
%ou ha1e been directed b" "our su'erior to com'ose a etter to a
'otentia cient to ascertain whether he might contem'ate entering an
undertaking that woud concei1ab" be of mutua benefit ...
The foowing are 'ro1ided as e+am'es of situationa com'ositions
intended to be used in tests of writing!
T"'e (
(magine that this is "our diar" showing some of "our acti1ities on certain
da"s. >irst) fi in "our acti1ities for those da"s which ha1e been eft
bank. Then) using the information in the diar") write a etter to a friend
teing him or her how "ou are s'ending "our time. ;rite about 1--
words. The address is not necessar".,
T"'e ,
<Suestion 1=
5 ;hie "ou are awa" from home) some5.merican friends are coming to
sta" in "our house. %ou are ea1ing before the" are due to arri1e) so "ou
decide to ea1e them some notes to he' them with a the things the"
wi need to know whie sta"ing in the house. %our friends ha1e ne1er
been to "our countr" before so there is Buite a ot of ad1ice "ou need to
'ass on. ;rite "our message on the noteet 'ad sheet beow.
<. bank noteet foows.=
<Suestion ,=
;hie "our .merican friends are sta"ing in "our house) the" write to
sa" that the" are enJo"ing themse1es so much that the" woud ike to
s'end two weeks) 1isiting some other 'arts of the countr". The" woud
ike "our ad1ice about what to go and see and where to sta".
132
;rite to "our friends gi1ing the best 'ossibe ad1ice "ou can from "our
own knowedge and e+'erience) with whate1er s'ecia hints and
warnings ma" be necessar". *ake sure "our friends know who the" can
write to for further information of an @officia@ kind to he' them to 'an the
best 'ossibe hoida".
;rite "our etter in the s'ace beow. (t shoud be between 18- and ,--
words in ength.3
<. bank s'ace foows.=
T"'e 3
#ead the foowing etter carefu".
17A ;ood Lane
London N;,
18th *a"
:ear *r Johnson)
wish to com'ain about the noise which has come from "our home ate
e1er" night this week. ;hie ( reaise that "ou must 'ractise "our
trum'et some time) ( fee "ou ought to do it at a more suitabe time. Ten
o@cock in the e1ening is rather ate to start 'a"ing. E1en if "ou coud
'a" we) the noise woud sti be unbearabe at that time.
( ho'e that in future "ou wi b a itte more considerate of the
feeings of others.
%ours sincere")
;. #obinson
Now write a re'" to this etter. %ou do not 'a" the trum'et but on two
or three occasions recent" "ou ha1e 'a"ed some trum'et music on
"our record 'a"er. %ou did not 'a" the ecord 1er" oud" 5 certain"
not as oud" as *r #obinson@s tee1ision. %ou want to te him this but
"ou do not want to become enemies so "ou must be reasonab" 'oite in
"our etter.
Care must be taken in the construction of etter5writing tasks to imit
the amount of information to which the student must re'". (f this is not
done scoring can become e+treme" difficut.
T"'e ? . diaogue can be 1er" usefu in 'ro1iding a basis for com'osition
work. (n such a writing task) students must demonstrate their abiit" to
change a te+t from one register to another) as in the foowing e+am'e!
#ead the foowing con1ersation carefu".
*# BL.C$! ;hat was the weather ike whie "ou were cam'ingG
L(N:.! Not too bad. (t rained the ast cou'e of da"s) but most" it was
fine. ;e weren@t abe to 1isit the &orge ;aterfas on the ne+t
to the ast da") but ...
*# BL.C$! ;hat a 'it"R
L(N:.! ;e) a'art from that we did e1er"thing we wanted to 5
waking) cirr bing and Just sitting in the sun. ;e e1en managed a
1isit to Hock Ca1e.
*# BL.C$! How on earth did "ou get that farG
L(N:.! ;e c"ced. 9h ... and we went to the beach Buite a few times.
*# BL.C$! :id "ou take "our bikes with "ouG
1 ?-
L(N:.!@ No) we borrowed some from a 'ace in the 1iage.
*# BL.C$! ;hereabouts were "ouG
L(N:.! 9h) in a o1e" 1ae"5 ots of woods and about twent" mies
from the sea. Just north of Hison.
*# BL.c$! ( remember one time when ( went5cam'ing. ;e forgot to take a
tin5o'enerR
L(N:.! That@s nothing. . goat came into our tent in the midde of the
night5 it ate a the food we had with usR
*# BL.C$! ;e) "ou seem to ha1e had a good time.
Now write an account of Linda@s hoida") using the con1ersation abo1e as
a guide. (magine other things which ha''ened to her during the
cam'ing hoida".
T"'e 8 Tabes containing information are aso usefu for situationa
com'osition since the" can genera" be read b" the students without much
difficut". *oreo1er) as on" a short written te+t is used) the students are
thus not encouraged to re'roduce 'art of the rubric for use in their
com'osition.
(magine that a oca news'a'er has asked "ou to write an artice of
a''ro+imate" ,8- words about the information in the foowing tabe.
;rite down the concusions "ou draw from the figures about the 1arious
wa"s in which 'eo'e s'ent their hoida"s in 12A3 as com'ared with
1233. .ttem't to e+'ain the reasons for these differences.
12A3 1233
Tra1eing abroad
? 17
&oing to seaside 33 31
Cam'ing 3 31
Eisiting friendsKreati1es in another town 11 1-
&oing to another town <but not to 1isit friendsKreati1es= 1A 3
0ta"ing at home ,3 3
T9T.L 1-- 1--
T"'e A (nformation con1e"ed in the form of a sim'e gra'h or histogram
?-
ma" aso 'ro1ide a suitabe stimuus for writing. 0uch writing tasks of this
nature) howe1er) are suitabe on" for more ad1anced students.
38
/se the chart together with the information
3-
beow5 to5gi1e5a(bdef sur1e".of the causes of
accidents on Link #oad between 1277 and
,8
1237.
,-
1277573 #oad not in great use
1273572 Nearb" road cosed! road now in great use
18 127253- Bus sto' mo1ed 1-- "ards
123-531 No changes
1-
123153, 0ign! Beware animas
123,533 No 'arking signs
12333? 0ign! No right turn
123?538 :oube white ine!
No o1ertaking
77
73 72
3- 31
3,
33
3? 38 3A 37 <etc.=
1?1
T"'e 7 The stimuus for writing ma" e1en take the form of notes.
/se the foowing notes to write an account of an accident for a
news'a'er. %ou shoud write about ,8- words.
C"cist about to turn right.
Not 'ut hand out.
Lorr" behind sows down.
0'orts car behind orr" o1ertakes.
0wer1es to a1oid bo".
$nocks o1er od man on 'a1ement.
T"'e 3 .n e+ceent de1ice for 'ro1iding both a 'ur'ose and content for
writing is the use of 'ictures. . 'icture or series of 'ictures not on"
'ro1ides the students with the basic materia for their com'osition but
stimuates their imaginati1e. 'owers.
The 'icture beow shows a dangerous Junction where accidents often
ha''en. ;rite a etter to "our oca news'a'er) describing the Junction
and mentioning some of the dangers and causes of accidents.
Link #oad! .n .ccident@Back 0'ot@
(f the stimuus in a situationa com'osition is 'ure" 1erba) the testees
often tend to re'roduce the 'hrases and sentences contained in it. 6ictures
and diagrams fortunate" a1oid this weakness.
T"'e 2 6ictures and diagrams ser1ing as stimui for descri'tions of obJects.
'eo'e) 'aces and 'rocesses ma" a be used to ad1antage in a test 5 as
we as those in1o1ing the writing of directions) instructions)
cassifications) com'arisons and narrati1es. The foowing e+am'e@ shows
how the iustration of two aircraft can be used as a basis for a fair"
detaied com'arison in a test.
1?,
.ngo5>rench Concorde
ength! A,.17 m
wing s'an! ,8.A- m
no. of 'assengers! 1--51??
ma+imum cruising s'eed! ,333 kmih
Boeing 7?7
ength! 7-.81 m
wing s'an! 82.A? m
width of cabin! A.1 m
no. of 'assengers! 33-58--
ma+imum cruising s'eed! 273 km.h
T"'e 1- The foowing e+am'es tests students@ abiit" to describe sha'es
and inear forms) as it is assumed that none of them wi ha1e an"
knowedge of the technica terms reBuired for reference to each 'icture. (t
is a searching test of their descri'ti1e writing abiit" and is) of course)
suitabe on" for fair" ad1anced students. The rubric is gi1en for this
'articuar item to he' readers to obtain a cearer idea of what is reBuired.
The 'ictures beow are arranged from the odest <1= to the most recent
<3=. /se them to comment on de1eo'ments in a warrior@s cothes and
eBui'ment.
1?3
1
2O? &rading the
The chief obJection to the incusion of the com'osition Buestion as 'art of
com'osition
an" test is genera" on grounds of unreiabiit". Considerabe research in
the 'ast has shown how e+treme" unreiabe markers are 5 both in their
own inconsistenc" and in their faiure to agree with coeagues on the
reati1e merits of a student@s com'osition.
*arkers ma" award their marks on <i= what a student has written!
<ii= what the" beie1e the student meant b" what he or she wrote4
<iii= handwriting and genera a''earance of what the student has written4
and <i1= 're1ious knowedge of the student. *oreo1er) two markers ma"
differ enormous" in res'ect of s'read of marks) strictness and rank order.
>or e+am'e) marker . ma" gi1e a wider range of marks than marker B
<i.e. ranging from a ow mark to a high mark=4 marker C ma" ha1e much
higher e+'ectations than marker . and thus mark much more strict")
awarding ower marks to a the com'ositions! and fina" marker : ma"
'ace the com'ositions in a different order of merit. .n e+am'e of these
differences can be seen in the foowing tabe. <The tota number of
'ossibe marks was ,-.=
0'read 0tandard
9rder
. B . C
. :
#ick 1? 1- 1? 2
1?
2
.manda 11 2 11 A 11 1,
:ebbie
1- 3 1- 8 1- 1-
Tina 7 7 7 , 7 11
:a1e 8 A 8 1 8 A
The whoe Buestion of unreiabiit". howe1er) does not sto' here.
E1en if a student were to take two com'osition e+aminations of
com'arabe u ^^ficuiti@) there woud be no guarantee whatsoe1er that he or she
woud score simiar marks on both e+aminations. This 'articuar t"'e of
unreiabiit" is more common to the com'osition 'a'er than to an"
other and is termed testKre5test reiabiit". . further com'ication resuts from a
ack of markKre5mark reiabiit"! in other words) if the same
com'osition is marked b" the same e+aminer on two occasions there ma" be a
difference in the marks awarded.
(n s'ite of a such demonstrations of unreiabiit") com'osition is sti
wide" used as a means of measuring the writing skis. The 1aue of
'ractice in continuous or e+tended writing cannot he stressed too great".
. student@s abiit" to organise ideas and e+'ress them in his or her own
words is a ski essentia for rea5ife communication. Thus) com'osition
can be used to 'ro1ide not on" high moti1ation for writing but aso an
e+ceent backwash effect on teaching) 'ro1ided that the teacher does not
antici'ate at too ear" a stage the com'e+ skis reBuired for such a task.
*oreo1er) if a more reiabe means of scoring the com'osition can he
used) sam'ing a student@s writing skis in this wa" wi a''ear a far more
1aid test than an" number of obJecti1e tests of grammar.
.s is cear" demonstrated at the end of this section) it is im'ossibe to
obtain an" high degree of reiabiit" b" dis'ensing with the subJecti1e
eement and attem'ting to score on an @obJecti1e@ basis) according to a
carefu" constructed s"stem of 'enaties. Howe1er) com'osition marking
can be im'ro1ed considerab" once the subJecti1e cement is taken into
1??
account and once methods of reducing the unreiabiit" inherent in the
more traditiona methods of assessment are em'o"ed. To start with)
testees shoud be reBuired to 'erform the same writing task. .though
there ma" sometimes be a case for a imited choice of com'osition to'ics in
the cassroom) attem'ts at accurate assessment of writing abiit" can on"
be successfu if the choice of to'ic is se1ere" restricted or aboished
com'ete". . we5defined task in terms of one or two situationa
com'ositions can he' enormous" to increase the reiabiit" of the
e+amination.
Because of the inherent unreiabiit" in com'osition marking) it is
essentia to com'ie a banding s"stem 5 or) at east) a brief descri'tion of
the 1arious grades of achie1ement e+'ected to be attained b" the cass. The
foowing are two e+am'es of descri'tions of e1es of 'erformance used b" a
we5known e+amining bod" in Britain! tabe <a= for intermediate5e1e
earners and tabe <b= for more ad1anced5e1e earners.
.s with the scoring of ora 'roduction) banding s"stems de1ised for a
'articuar grou' of students at a 'articuar e1e are far 'referabe to scaes
drawn u' for 'roficienc" tests administered on a nationa or an
internationa basis.
Tabe <a=@
135,-
1A517
1,518
E+ceent Natura Engish) minima errors) com'ete reaisation
of the task set.
Eer" good&ood 1ocabuar" and structure) abo1e the sim'e
sentence e1e. Errors non5basic.
&ood 0im'e but accurate reaisation of task. 0ufficient
naturaness) not man" errors.
O
3511 6ass #easonab" correct if awkward 9# Natura treatment
of subJect with some serious errors.
857 ;eak
Eocabuar" and grammar inadeBuate for the task set.
-5? Eer" 'oor(ncoherent. Errors showing ack of basic knowedge
of Engish.
Tabe <b=@
135,- E+ceent Error5free) substantia and 1aried materia)
resourcefu and controed in anguage and
e+'ression.
1A517
Eer" good&ood reaisation of task) ambitious and natura in
st"e.
1,518 &ood
0ufficient assurance and freedom from basic error to
maintain theme.
3511 6ass
Cear reaisation of task) reasonab" correct and
natura.
857 ;eak
Near to 'ass e1e in genera sco'e) but with either
numerous errors or too eementar" or transated in
st"e.
-5? Eer" 'oorBasic errors) narrowness of 1ocabua
r".
1?8
11
1
The foowing rating scae is the resut of considerabe and carefu research
conducted5in the scoring of com'ositions in the /nited 0tates.G 9n" a
summar" of the scae is shown here and it must be remembered that in its
origina form sight" fuer notes are gi1en after each item.
Content
3-5,7 EMCELLENT T9 EE#% &99:! knowedgeabe 5 substanti1e 5 etc.
,A5,,
&99: T9 .EE#.&E! some knowedge of subJect 5 adeBuate
range 5 etc.
,1517
>.(# T9 699#! imited knowedge of subJect 5 itte substance 5
etc.
1A513 EE#% 699#! does not show knowedge of subJect 5 non5
substanti1e 5 Etc.
9rganiLation
,-513 EMCELLENT T9 EE#% &99:! fuent e+'ression 5 ideas cear"
stated 5 etc.
1751? &99: T9 .EE#.&E! somewhat cho''" 5 oose" organiLed but
main ideas stand out 5 etc.
1351-
>.(# T9 699#! non5fuent 5 ideas confused or disconnected 5 etc.
257 EE#% 699#! does not communicate 5 no organiLation 5 etc.
Eocabuar"
,-513 EMCELLENT T9 EE#% &99:! so'histicated range 5effecti1e
5
wordKidiom choice and usage 5 etc.
1751? &99: T9 .EE#.&E! adeBuate range 5 occasiona errors of
wordKidiom form) choice) usage but meaning not obscured. 135
1- >.(# T9 699#! imited range 5 freBuent errors of wordKidiom
form) choice) usage 5 etc.
257
EE#% 699#! essentia" transation 5 itte knowedge of Engish
1ocabuar".
Language use
,85,, EMCELLENT T9 EE#% &99:! effecti1e com'e+ constructions 5
etc.
,1512 &99: T9 .EE#.&E! effecti1e but sim'e constructions 5 etc.
17511
>.(# T9 699#! maJor 'robems in sim'eKcom'e+ constructions
5 etc.
1-58 EE#% 699#! 1irtua" no master" of sentence construction rues 5
etc.
*echanics
8 EMCELLENT T9 EE#% &99:! demonstrates master" of
con1entions 5 etc.
? &99: T9 .EE#.&E! occasiona errors of s'eing) 'unctuation 5
etc.
3 >.(# T9 699#! freBuent errors of s'eing 'unctuation)
ca'itaiLation 5 etc.
, EE#% 699#! no master" of con1entions 5 dominated b" errors of
s'eing) 'unctuation) ca'itaiLation) 'aragra'hing 5 etc.
9ne 'oint worth noting in the scae is that in each cassification <e.g.
Content) 9rganiLation) Eocabuar") etc.= the owest grade described as
1?A
@Eer" 6oor@ ends with the 'hrase @9# not enough to e1auate.@ <This is not
incuded here.=
Com'ositions ma" be scored according to one of two methods! the
im'ression method or the ana"tic method. Note) howe1er) that the former
method does not in1o1e the use of a rating scae to an" arge e+tent.
The im'ression method of marking entais one or more markers
awarding a singe mark <7 muti'e marking=. based on the tota
im'ression of the com'osition as a whoe. .s it is 'ossibe for a
com'osition to a''ea to a certain reader but not to another) it is arge" a
matter of uck whether or not a singe e+aminer ikes a 'articuar scri't. .s
has been demonstrated) the e+aminer@s mark is a high" subJecti1e one
based on a faibe Judgement) affected b" fatigue) careessness) 'reJudice)
etc. Howe1er) if assessment is based on se1era <faibe= Judgements) the
net resut is far more reiabe than a mark based on a singe Judgement.
&enera") three of four markers score each 'a'er) marks being
combined or a1eraged out to re1ea the testee@s score. The foowing tabe
shows how four markers can score three com'ositions using a fi1e5'oint
scae for im'ression marking.
Com'.1 Com'., Com'. 3
*arker .! 3 8 ?
*arker B!
, ? ,
*arker C! , ?
3
*arker :! 3 ? 1
Tota! 1- 17 1-
<G=
.1erage! ,.8
? ,.8<G=
(n those cases where there is a wide discre'anc" in the marks aocated
<e.g. Com'osition 3 in the 're1ious e+am'e=. the scri't is e+amined once
again b" a four markers and each mark discussed unti some agreement is
reached. >ortunate") such discre'ancies occur on" rare" after the initia
stages in muti'e marking.
. the e+aminers 'artici'ating in a muti'e5marking 'rocedure are
reBuired to gance Buick" through their scri'ts and to award a score for
each one. The marking scae ado'ted ma" be as itte as from - to 8 or as
arge as from - to ,- <athough it has been the author@s e+'erience that
most markers 'refer to use a 85'oint scae or an" simiar scae with on" a
few categories in order to obtain a wide range of marks=. (t is most
im'ortant that a markers be encouraged to use the whoe range of an"
scae! cear") marks which bunch around 2 to 1, on a ,-5'oint scae are of
itte use in discriminating among candidates. (t is aso im'ortant that a
the markers read through a certain number of scri'ts in a gi1en time
<usua" about ,- 'er hour= and time themse1es. (f the" find themse1es
sowing down) marking fewer scri'ts 'er hour) the" are ad1ised to rest and
resume work when the" fee abe to mark at the reBuired rate. (m'ression
marking is genera" found more e+hausting than mechanica methods of
marking4 thus) it is essentia that markers sto' work when their attention
begins to wander or when the" find themse1es aborious" reading through
the content of each com'osition. (m'ression marks must be based on
im'resssion on") and the whoe obJect is defeated if e+aminers start to
reconsider marks and ana"se com'ositions. *ost e+aminers find it more
1?7
enJo"abe than an" other method of scoring com'ositions. 0ome argue in
fa1our of marking for one or two hours at a stretch in order to maintain
consistenc") but itte concusi1e research has been carried out in this area
and there a''ears to be no e1idence that marking@for a ong 'eriod
'roduces more consistent and reiabe marks than marking for short
'eriods. (m'ression marking is genera" found to be much faster than
ana"tic or mechanica marking. (f com'ositions are scored b" three or four
im'ression markers) the tota marks ha1e been found to be far more
reiabe than the marks awarded b" one ana"tic marker. <The com'arison
is a fair one) since it takes as ong for one ana"tic marker to score a
com'osition as it does four im'ression markers.= 9n the other hand) the
marks awarded b" one im'ression marker are ess reiabe than those
awarded b").one ana"tic marker.
0ince most teachers ha1e itte o''ortunit" to enist the ser1ices Iof5two
or three coeagues in marking cass com'ositions) the ana"tic method is
recommended for such 'ur'oses. This method de'ends on a marking
scheme which has been carefu" drawn u' b" the e+aminer or bod" of
e+aminers. (t consists of an attem't to se'arate the 1arious features of a
com'osition for scoring 'ur'oses. 0uch a 'rocedure is idea" suited to the
cassroom situation! because certain features ha1e been graded se'arate")
students are abe to see how their 'articuar grade has been obtained. The
foowing is re'roduced sim'" as one e+am'e of such an ana"tic scheme!
in this 'articuar case du'icate <bank= co'ies of this scheme were
stencied b" the teacher and attached to the end of each com'osition.
&rammar
Eocabuar"
*echanics M
>uenc"
#ee1ance * M
Note that *echanics refers to 'unctuation and s'eing4
>uenc" to st"e
and ease of communication4 and #ee1ance to the content in reation to the
task demanded of the student. . 85'oint scae has been used.
(f the ana"tic method of scoring is em'o"ed) it is essentia that
f e+ibiit" is maintained. .t the 1arious e1es it ma" become necessar" to
change either the di1isions themse1es or the weighting gi1en to them. .t
the eementar" e1e) for e+am'e) the tester ma" be far more interested in
grammar and 1ocabuar" than in fuenc") thus deciding to omit >uenc". .t
the intermediate e1e) the tester ma" be 'articuar" interested in
ree1ance and ma") therefore) decide to award a ma+imum of 1- marks for
this feature whie awarding on" 8 marks for each of the others. .t the
more ad1anced e1e) the tester ma" wish to incude se'arate di1isions for
organisation and register and to incude mechanics and fuenc" in one
di1ision.
. third method of scoring com'ositions is the mechanica accurac" or
error5count method. .though this is the most obJecti1e of5a methods of
scoring) it is the east 1aid and is not recommended. The 'rocedure
consists of counting the errors made b" each testee and deducting the
number from a gi1en tota! for e+am'e) a testee ma" ose u' to 1- marks
1?3
for grammatica errors).8 marks for misuse of words) 8 for miss'eings)
etc. 0ince no decision can be reached about the reati1e im'ortance of
most errors) the whoe scheme is actua" high" subJecti1e. >or e+am'e)
shoud errors of tense be regarded as more im'ort ant than certain
miss'eings or the wrong use of wordsG >urthermore) as a resut of
intuition and e+'erience) it is fair" common for an5e+aminer to fee that a
com'osition is worth se1era marks more or ess than the score he or she
has awarded and to ater the assessment according". .bo1e a) the
mechanica accurac" method unfortunate" ignores the rea 'ur'ose of
com'osition writing 5 communication4 it concentrates on" on the negati1e
as'ects of the writing task) 'acing the students in such a 'osition that the"
cannot write for fear of making mistakes. The conseBuent effect of such a
marking 'rocedure on the earning and teaching of the writing skis can be
disastrous.
2.8
Treatment of Before brief" re1iewing some of the attem'ts to identif" error gra1it") it i
written errors
interesting to note a re'ort of an e+'eriment@ in which nati1e s'eakers whc.
were not teachers scored written work b" its degree of inteigibiit" rather
than b" the errors it contained. Nati1e5s'eaking teachers) on the other hand)
e1auated written work b" the number and t"'es of errors it
contained. Non5nati1e s'eaking teachers of the anguage) howe1er)
'enaised students for what the" consider Dbasic errors@ <e.g. DHe go@= and
were genera" far stricter in their attitude to errors.
.though resuting in a far more 'ositi1e a''roach) e1auating wetter
work according to the degree of inteigibiit" is not awa"s a reiabe
method of assessment. >reBuent") the student@s 'erformance 5 and
success in accom'ishing the task 5 ma" be masked b" errors and a tired
marker ma" fai to make the necessar" effort to res'ond to the writing as
means of communication.
.n im'ortant2 distinction is now made between
goba and oca
errors. Those errors which cause on" minor troube and confusion in a
'articuar cause or sentence without hindering the reader@s com'rehensio!
of the sentence are categorised as oca errors <e.g. misuse of artices)
omission of 're'ositions) ack of agreement between subJect and 1erb)
incorrect 'osition of ad1erbs) etc.! D( arri1ed Leeds.@=. &oba errors are
usua" those errors which in1o1e the o1era structure of a sentence and
resut in misunderstanding or e1en faiure to understand the message whit
is being con1e"ed <e.g. the misuse of connecti1es! D.though the train
arri1ed ate) we missed the ast bus to the cit" centre@4 the omission) misus
and unnecessar" insertion of reati1e 'ronouns! D%ou shoud tr" to be as
heath" as the gir arri1ed on the bic"ce a short time ago@4 etc.=. This
usefu distinction) which 'ro1ides criteria for determining the
communicati1e5im'ortance of errorshas een further de1eo'ed recent"
so that it can be more readi" a''ied to the marking of free writing. .
11
(n addition) it is necessar" in cassroom tests to distinguish between
those errors which) though 'erha's not resuting in an" breakdown in
communication) indicate that the student has faied to earn something
which has Just been taught or which shoud ha1e been mastered '.Ee1ious)
(n most norma writing situations) howe1er) we can on" assess what
student writes and not
what he or she wants to write. >or this reason)
'ictures and diagrams can 'a" a 1er" usefu 'art. in testing writing) since
the" enabe the e+aminer to te immediate" what a student wishes to
write. 6ictures were recent" used b" researchersI in an e+'eriment to
15
show how L, earners <i.e. ess fuent earners= used a1oidance
strategies or
reduction strategies)
a1oiding an actua to'ic. Lt earners <and 'ossib"
more fuent L5) earners= tended to use 'ara'hrase strategies or
achie1ement strategies.
>uent 'erformance seems to be characterised b"
the use of fewer communication strategies of both kinds.
The test writer@s attitude to error gra1it" and a''roach to treating
errors in marking free writing wi 1ar" according to the students@ e1e of
attainment. .t the eementar" e1es) for e+am'e) the test writer wi
'robab" be far more toerant than at the intermediate and ad1anced
e1es. .t the ower e1es he or she wi e+'ect more a1oidance strategies
and more goba t"'es of error. ;hat is im'ortant at a e1es is an
awareness of different t"'es of error and of communication strategies.
resuting in an increased sensiti1it" to them.
2.A 9bJecti1e tests!
mechanics
6unctuation
T"'e 1 The foowing t"'e of 'unctuation item is 1er" 'o'uar and is
genera" used to co1er5a wide range of 'unctuation marks. (t is not tru"
obJecti1e) and the scoring of such an e+ercise woud take considerabe
time since 'unctuation is to a arge degree subJecti1e and one 'articuar
use of a 'unctuation mark ma" we determine the correctness of the
'unctuation mark foowing it.
(n the foowing 'assage there is no 'unctuation. ;rite out the 'assage)
'utting in a the 'unctuation and ca'ita etters.
end me "our 'en 'ease 'eter asked
i took m" 'en out of m" 'ocket
be carefu i said
i gi1e it back to "ou in a moment he 'romised
dont worr" i said "ou can kee' it as ong as ". o. .u1IPID
(t is ad1isabe) howe1er) to maintain some degree of contro o1er the task
which the testees are e+'ected to 'erform. 9ne method of doing this is b.)
substituting ines or circes for those 'unctuation marks which are being
tested) thus aso faciitating scoring.
T"'e ,
6ut the correct 'unctuation mark in each bo+.
: ;hat do "ou want) : 1 asked Henr":
: *a" ( use "our tee'honeG : he asked.
et me
Certain" n : 1 said. : ;hen "ou : 1e finished : 'ease
know : :
ered.
: 1 sha on" be a moment n : Henr" answ
: Has John Lee in1ited "ou to his 'art" :: ( asked.
: No) he hasn : t "et :: Henr" re'ied.
\ He : s in1ited 6au : :a1id : Ton" and *ar" 9n (
c continued. o
: He : s 'robab" forgotten about me : : Henr" aughed.
: How strange : : 1 answered. : (@m sure he wants "ou to go to@his
'art".
T"'e 3 . greater degree of obJecti1it" can be obtained b" using the
muti'e5choice techniBue) e.g.
6ut a circe round the etter <.) B) C) or := of the correct" 'unctuated
sentence.
.. Tom asked me if ( was going to the meetingG
B. Tom asked me) if ( was going to the meeting.
C. Tom asked me) @(f ( was going to the meetingG@
:. Tom asked me if ( was going to the meeting.
0'eing
T"'e 1! :ictation
.s with 1ocabuar" testing) sam'ing is of 'rimar" im'ortance in the
construction of s'eing tests. ;ords used in connection with the students@ free
com'osition work or e1er"da" writing form the most suitabe basis for tests
of s'eing) athough items ma" aso be drawn from the students@
reading 'ro1ided that the tester is aware of the im'ications of testing the
more 'assi1e items of the students@ 1ocabuar".
:ictation of ong 'rose 'assages is sti regarded as an essentia
method of testing s'eing. Howe1er) dictation measures a com'e+ range of
integrated skis and shoud not be regarded as constituting sim'" a test of
s'eing. The dictation of singe words) ne1ertheess) can 'ro1e a fair"
reiabe test of s'eing. 0e1era such tests consist of u' to fift" words and use
simiar 'rocedures to the foowing!
<i= Each word is dictated once b" the tester4
<ii= the word is then re'eated in a conte+t4 and fina")
<iii= the word is re'eated on its own.
T"'e ,! *uti'e5choice items
.nother fair" wides'read method of testing s'eing is through the use of
muti'e5choice items usua" containing fi1e o'tions) four of which are
s'et correct". The students are reBuired to seect the word which is
incorrect" s'et) e.g.
1. .. thief B. beief C. seiLe :. ceiing
E. decie1e
,. .. ha''ening
B. offering C. occuring
:. beginning
E. benefiting
3. .. iega B. genera" C. summar" :. beggar
E. neccessar"
?. .. interru't
B. su''ort C. answering :. ocasiona
E. command
(n some tests on" four words are gi1en as o'tions) the fifth o'tion
being No mistakes or . correct)
e.g.
.. e+hibition B. 'unctua" C. 'easure :. oba"ed
E. .
correct
T"'e 3! Com'etion items
0uch items as the foowing differ from simiar ones used in tests of
1ocabuar" because sufficient cues are 'ro1ided both in the banks and in
the definitions to enabe the students to know e+act" which word is
reBuired. The banks occur on" in those 'arts of the word which gi1e rise
to a s'eing difficut" for man" students. 9ne ad1antage of such a test is
that it does not 'resent the students with incorrect forms. <*an" nati1e
s'eakers argue that the" freBuent" fai to recognise correct forms after
e+'osure to miss'eings.=
181
;rite one or more etters in each of the s'aces in the foowing words.
<The definitions wi he' "ou to recognise the correct word.=
1. om s n something eft out
,. di uade 'ersuade someone
not to do something
3. o u ing ha''ening) taking 'ace
?. rec t a written statement
to show that a bi bas been 'aid
(t is he'fu in tests of this nature to 'ro1ide a conte+t for the word <in
addition to gi1ing a s"non"m or definition=.
1. The horse ga ed <7 ran= to the front of the race.
,. ( doubt if an"one e1er 'rof
ed <7 gained= from that business dea.
3. The schoo has an enro ment <7
number on its register= of o1er 8--
students..
?. :on@t worr"4 m" dog wi go into the water and retr 1e <7 bring
back=
"our ba.
T"'e ?! Error5recognition items
(n these items the students are reBuired to identif" <according to its etter=
the 'art of the sentence in which a word has been miss'et.
.
B C
1. The dissatisfied womanKrefused to admitKthat there was sufficentK
:
coffee for e1er"one.
. B
C
,. :on@t be decie1edKb" the new 'rocedure!Kit hard" differs fromK
:
the od corru't s"stem.
. B C
3. The man was eBui''edKwith a doube5barre shotgunKand his
C
:
coeagueKwith an innocent5ooking waking5stick.
.
B
C
?. *r &rant@s e+aggeratedKaccount of the im'ortanceKof his new a'oint5
:
mentK was Buite unnecessar".
2.7 9bJecti1e tests! The muti'e5choice items beow are concerned chief" with measuring
st"e and register
students@ sensiti1it" to st"e. 0ome of the distractors in the two e+am'es
are incorrect on grammatica grounds whie others are grammatica"
0t"e
correct but not re'resentati1e of the kind of Engish used b" an educated
nati1e s'eaker in the 'articuar conte+t in which the" a''ear. (ndeed.
some test writers distinguish tests of writing from tests of grammar and
usage in terms of the 'erformance of nati1e s'eakers! whereas a nati1e
s'eakers of a anguage woud be e+'ected to score high marks in a test of
grammar) on" certain educated nati1e s'eakers 'ossessing the reBuired
writing skis woud score high marks in an obJecti1e test of writing.
0ome of the re1iews were fa1ourabe to the new 'a"
.. and as man" were unfa1ourabe.
B. athough others of the same amount were unfa1ourabe.
VC. whie an eBua number were unfa1ourabe.
t8)
:. but the same number were unfa1ourabe.
E. in s'ite of haf being unfa1ourabe.
The weather has awa"s been an im'ortant factor in 'eo'e@s i1es
V.. because of its effects on a as'ects of farming.
B. for it has considerabe infuence o1er farming.
C. since farmers concern themse1es with it.
:. as weather constitutes the dominant worr" for farmers.
E. on account of its affecting farming affairs.
<V 7 Correct answer=
#egister
The use of the correct register denotes the abiit" to write for a s'ecific
'ur'ose with a s'ecific audience in mind. Confusion and embarrassment
resut from the use of ina''ro'riate registers. 0uch tests as the foowing)
howe1er) are not too difficut to construct and 'resent the students with an
interesting task) 'ro1ided that the e+tract used is written in a fair"
distincti1e st"e.
T"'e 1 The foowing t"'e of <ad1anced= register test reBuires the students to
identif" those words which are incongruous) re'acing each with a much more
suitabe word. The student is instructed to re'ace si+teen of the
words underined in the 'assage.
(t has now been made out
be"ond an" doubt whatsoe1er that the
nicotine
contained
in tobacco smoke is 'oisonous. 9ne minute dro' of
'ure nicotine
'unged
into the boodstream of a rat is sufficient to ki it.
(t has aso been 'ro1ed that the nicotine contained in tobacco smoke
sends u' the 'use rate and the bood 'ressure. There is aso strong
e1idence that the nicotine
content in fags is a 'rimar"
cause of oss of
weight and hungriness. (t is aso ike" that a few
hea1" smokers wi
ose contro of )heir heir finer musces and be unabe to 'a" around with
obJects with ease and 'recision. 0uch a oss of musce acti1it" ma"
widen the e"es and
s'oi 1ision. *oreo1er) smoking 'uts back growth in
adoescents and
owers
athetic abiit".
Ho)_1e1er) the most serious disease connected with smoking is
cancer of the ung! the direct connection between smoking and cancer
has
recent" been estabished so
assured" that cancer research fok and
'ubic heath authorities throughout the word ha1e begun intensi1e
cam'aigns against smoking. (n certain countries not on" are cigarette
ad1ertisements banished
from cinema and tee1ision screens but aso
makers are forced to 'rint on each 'acket a
warning concerning the
dangers of smoking.1,
T"'e , *atching tests are we5suited to tests of register4 such tests can be
constructed both at word and sentence e1e.
<a= ;ord e1e! The students are instructed to match each word in List .
with a word in List B) de'ending entire" on how forma or informa a
word is 5 not on its meaning5
13
List . List B .nswers
1.
cr" a. boss
<1e=
,. commence b. gee gee
<,c=
3. kid c. e+'ire
<3a=
?. 'uss"
d. hos'itaise <?b=
8. entrain e. draw
<8d=
183
<b= 0entence e1e! The students are instructed to 'ut the etter of the most
a''ro'riate sentence in List B with the number of each sentence in
List .. The sentences ha1e been taken from instructions) ega
documents) scientific Engish) ad1ertisements) chidren@s comics and
news'a'ers.
List .
1. Buid the assemb" formers fat on the 'an and bend the
undercarriage down to the 'attern shown.
,. The Tenant sha kee' the interior of the 'remises in good order and
condition.
3. . bic"ce 'um' is a de1ice for mo1ing air agains
t a 'ressure
difference.
?. Because the Barceno has front whee dri1e) there@s
no 'ro' shaft. 0o
"ou get big car roominess in on" thirteen feet.
8. But it@s too ateR The e1i 'an) cooked u' b" the monster Babo) has
ed Cato to *adam Qena.
A. .ce dri1er inJured in thriing race of "ear.
List B
a. There@s a new andmark for o1ers and others
at ;ateroo 0tation.
The :rum Bar and Buffet.
b. .n obJect norma" becomes hot when it is 'aced in the
sun.
c. The mi+ture shoud be taken three times dai" after meas.
d. &ang fight death 5 "outh kied when 'ushed onto eectric ine.
e. .n" amendment of this certificate or faiure to com'ete an" 'art of
it ma" render it in1aid.
f. &i1e o1er. (@m not a genius. The radio transmits a kind of buLL. .
beam that can be 'icked u' for a cou'e of mies.
2.3 Controed writing There are se1era wa"s of controing students@ freedom of e+'ression in
their written work and) as a conseBuence) increasing the reiabiit" of the
scoring. Howe1er usefu such methods are as teaching de1ices) the" wi
on" 'ro1e usefu for testing 'ur'oses if each student is com'ete" famiiar
with the 'articuar task to be 'erformed! hence the im'ortance of cear
instructions foowed b" at east one e+am'e. 0ometimes there is e1en the
danger that certain students wi fee inhibited rather than he'ed b" such
contro. E+am'es of controed writing e+ercises are incuded in this
section.
T"'e 1 The students are gi1en a short reading e+tract and then reBuired to
write a simiar 'aragra'h) using the notes the" ha1e been gi1en) e.g.
.though dogs are on" animas) the" are 1er" usefu and he' 'eo'e a
ot. >or e+am'e) certain dogs he' farmers to ook after their shee'.
0ome dogs are used for hunting and others he' to rescue 'eo'e. E1en
now 'oice officers use dogs when the" are ooking for thie1es and
criminas. 6eo'e aso teach dogs to race) and dog racing is a s'ort
which man" 'eo'e ike. . dogs ike eating meat 1er" much and ike
bones best of a.
.though 5 horses 5 animas) 5 usefu 5 a ot. >or e+am'e) 5 horses 5
'eo'e 5 catte. 0ome horses 5 hunting 5 'u things. (n the 'ast 5
sodiers 5 horses 5 fighting against the enem". 6eo'e 5 horses 5 horse
racing 5 s'ort 5 ike. . horses 5 ha" 5 oats.
18?
T"'e , The foowing item t"'e1? is set in a few wide"5used e+aminations
and can 'ro1e 1er" usefu in controing writing once students are famiiar
with the con1entions obser1ed in the item. E1en the foowing rubric and
item ma" cause difficut" if a student has not 're1ious" been gi1en 'ractice
in com'eting such items. 9biBue strokes are used in the fragmented
sentences chief" in order to reinforce the im'ression that the sentences
ha1e been gi1en in note form.
/se the foowing notes to write com'ete sentences. 6a" carefu
attention to the 1erbs underined and insert a missing words. The
obiBue ines <K= are used to di1ide the notes into sections. ;ords ma" or
ma" not be missing in each of these sections. #ead the e+am'e
carefu" before "ou start.
E+am'e! 6arachute Jum' from aero'anes and baoonsKbe 1er"
'o'uar s'or man" 'arts of word.
6arachute Jum'ing from aero'anes and baoons is a 1er"
'o'uar s'ort in man" 'arts of the word.
&reatest heightKfrom which 'arachute Jum' e1er make be o1er 31)---
metres.K12A-Kdoctor in /nited 0tates .ir >orceK
Jum' from basket of
baoonKand
fa near" ,A)--- metresKbefore
o'en 'arachute.K>a ast ?
minutes and 33 seconds)Kand bod" reach s'eed 23- kiometres hour.KHe
and safe" in fiedK13 minutes and ?8 secondsKafter he Jum'.K9n ste' of
basket of baoonKbe wor5@sK@This be highest ste' in word.
hen as if
he ike Jum' againKfrom such heightKdoctor
shake head.
T"'e 3 0e1era t"'es of writing tasks can be based on the foowing reading
e+tract.18 .n" simiar te+t can aso be used for!
5 co'"ing with minor aterations! e.g. tenseK'erson changes
5 changing the 'oint of 1iew! e.g.
;rite this stor" as seen b" .
changing the st"e and register! e.g.
;rite this stor" in the form of a
news'a'er re'ortKa humorous account) etc.
adding further information.
. "oung man who refused to gi1e his name di1ed into the ri1er
"esterda" morning to sa1e a twe1e5"ear5od bo".
The bo") who ran awa" after being rescued) had been swimming in
the ri1er and had caught his foot between two concrete 'osts under the
bridge. He shouted out for he'.
.t the time) a "oung man was riding across the bridge on his
bic"ce. He Buick" dismounted and di1ed fu" cothed into the ri1er. He
then freed the bo"@s foot5and he'ed him to the ri1er bank where a sma
crowd had coected. The bo" thanked his rescuer courteous" and
sincere") then ran off down the road. He was ast seen cimbing o1er a
gate before disa''earing o1er the to' of the hi.
The "oung man) who was about twent" "ears of age) said @( don@t
bame the bo" for not gi1ing his name. ;h" shoud heG (f he wants to
swim in the ri1er) that@s his business. .nd if ( want to he' him) that@s
mine. %ou can@t ha1e m" name eitherR@
He then ran back to the bridge) mounted his bic"ce and rode awa".
Test <i= #ewrite this stor" but imagine that "ou are actua" watching
e1er"thing that is ha''ening. Begin!
There is a sma bo"
swimming ...
188
Test <ii= #ewrite this stor" as tod b" <a= the "oung man who sa1ed the
bo" and <b= the bo" who was sa1ed.
Test <iii= ;rite this stor" as if "ou were gi1ing e1idence at a 'oice
station.
Test <i1= (t was a sunn" da" but at the time of the rescue it began
to rain
hea1i". 0e1era 'eo'e were 'assing nearb" on their wa" to a
footba match. ;hen the "oung un
thought that he had got
the rain. ;rite out the stor")
adding these facts.
T"'e ? There are aso se1era methods of 'ractising or measuring the
abiit" to ink sentences) in1o1ing subordination and co5ordination
features. 0ome tasks in
u'
<arge" determined5bN5+ 1 onectr1es gi1en=. 1en=. The foowing
is an e+am'e
of a controed writing ta
h e
skc'ractising subordination!
Join the short sentences in each of the grou's beow to form one
sentence. Then write each of the finished sentences so as to form a
'aragra'h. /se the Joining words gi1en) but note that sometimes no
Joining word is necessar"4 aso
5ing denotes the 1erb ending on".before Each
9"m'ic &ames o'ens.
.n athete a''ears.
He hods a torch. <5ing=
(t has been carried from *ount 9"m'us in &reece. which
The ceremon" was started in Berin in 123A. which
(t inks the sites of the modern &ames with the first
9"m'ic &ames.
Howe1er) the actua torch ceremon" dates back to .ncient
&reece. where
9ne of the most s'ectaci@@Har e1ents was the torch race. which
(t was awa"s run at night.
The athete entefs the stadium. ;hen
He is hoding the torch. who
He runs to the huge bow.
The sacred fame wi burn there. in which
*an" such tests do not gi1e the reBuired inkers to the testees but
ea1e them free to Join the sentences in whiche1er wa" the" consider
a''ro'riate. (ndeed) since such tests are sti 1er" subJecti1e and reBuire a
ot of time to score) it is often better not to 'ro1ide the testees with ink
ers
but to ea1e them free to so1e each 'robem in their own wa".
T"'e 8<a= (n some tests of com'osition) es'ecia" at the eementar" and
intermediate e1es) sentences and causes are 'ro1ided at first in order to he'
the students to start writing. The" ;a" then reBuired to finish the incom'ete
sentences in an" a''ro'riate
#ead these sentences. >inish each one and then com'ete the stor" in
"our own words.
9ne da" Hannah and Beck" got u' ear" to
go ...
The" caught a bus to the arge de'artm
ent store
where ...
18A
11
1
@Look) that@s 6ete 0haw o1er there)@ Beck" cried.
@Let@s .
The" shouted to 6ete but ...
@;h" doesn@t he ook at usG@ Hannah asked. He@s beha1ing as
if .@
<b= This item is simiar to the 're1ious one) but here the testees are reBuired
to write a''ro'riate sentences father than causes. The foowing e+am'e
shows how the item t"'e can be used at the u''er5intermediate e1es.
1. 0tudents who do not know a ot of Engish can take se1era ste's to
're'are for their stud" in a British uni1ersit". >or e+am'e)
,. #ecent research shows that 'ubic o'inion is di1ided on the subJect of
s'ending mone" on defence. .bout ?- 'er cent of the countr"
beie1es we shoud increase such s'ending. 9n the other hand)
. we5seected series of such items can incude sentences eiciting an
abiit" to use e+em'ification) contrast) addition) cause) resut) 'ur'ose)
concusion and summar". ConseBuent") students can be tested on their
abiit" to use whate1er s'ecific functions and notions the test writer wishes.
T"'e A 9ne of the most usefu de1ices for e+ercising contro o1er the kind
of written res'onse reBuired and "et) at the same time) gi1ing the testees
considerabe freedom of e+'ression is the two5sentence te+t designed to
measure the abiit" to form a coherent unit of anguage.18 >or e+am'e)
testees ma" be instructed to write a sentence to
'recede the statement!
*oreo1er) it was im'ossibe to o'en the windows.
0am'e res'onses coud be!
(t was 1er" hot in the sma room.
There was on" one fan in the room) but it was broken.
The door sammed behind John) and he reaised he was5ocked in the
room.
(n a cases) the students are reBuired to demonstrate an awareness of
the communicati1e nature of anguage in. genera and cohesi1e de1ices in
'articuar whie sti retaining a arge degree of freedom of res'onse.
9ther sam'e items are!
There was one outside the schoo entrance) too.
To do this) the water must first be boied.
These shoud then be carefu" sorted.
>or) widife) howe1er) there are e1en greater dangers in the 'oution
of ri1ers) akes and seas.
Howe1er) there is no reason to be 'essimistic.
187
11
Notes and references
1
The communicati1e nature of this item t"'e woud be great" reduced if
the first sentence were gi1en instead of the second sentence. The constraints
woud then be minima) the cohesi1e de1ices acking in ree1ance to a certain
degree) and the range of acce'tabe res'onses 1er" wide indeed. >or
e+am'e) after the sentence
There was a strange5ooking creature outside our door.
.n" of the foowing res'onses woud be acce'tabe!
( went u' to it and stroked it tender".
:o "ou ike itG
*rs Lee screamed.
The ne+t thing ( knew ( was "ing on m" back.
There were aso se1era cats and dogs.
.be5tee'hone5sudden"5rangt
(t was a hot da".
1 #o"a 0ociet" of .rts! The Communicati1e /se of Engish as a >oreign
Language
, /ni1ersit" of Cambridge Loca E+aminations 0"ndicate! 6reiminar" Engish
Test <re1ised 1ersion=
3 #o"a 0ociet" of .rts! The Communicati1e /se of Engish as a 0econd
Language) Test of ;riting! .d1anced Le1e) summer 123?
? Heaton J B 123A ;riting through 'ictures. Longman
8 Joint *atricuation Board! Test in Engish <91ers 5as=) *arch 1233
A /ni1ersit" of Cambridge Loca E+aminations 0"ndicate Cambridge
E+aminations in Engish <Tabe <a= 7 >irst Certificate in Engish4 Tabe <b= O
Certificate of 6roficienc" in Engish=
7 Hartfie. >a"e et. a. 1238 E0L Com'osition 6rofie. Learning E0L
Com'osition. Newbur" House
3
Hughes) . and Lascaratou) C 123, Com'eting criteria for error gra1it". ELT
Journa 3A<3=
2 Burt) * $ and $i'arsk" C
127, The &ooficon! a re'air manua for Engish.
Newbur" House
Hendrickson) J <ed.= 1272
Error .na"sis and Error Correction in Language
Teaching. #ELC 9ccasiona 6a'ers No. 1-
1- Tomi"ana. * 123-
&rammatica errors and communication breakdown.
TE09L Suarter" 1?<=
11 Eis) #
123? Communication strategies and the e1auation of communicati1e
'erformance. ELTJourna33<=.
1,
made out <'ro1ed=) 'unged <inJected=) sends u' <increases=) fags <cigarettes=)
hungriness <a''etite=) 'a" around with <mani'uate=) widen <e+tend=) s'oi
<im'air=) 'uts back <retards=) owers <reduces=) assured" <concusi1e"=) fok
<organisations=) begun <aunched=) banished <banned=) forced <reBuired=)
makers <manufacturers=
13
(n an e+ceent artice @0t"e and #egister Tests@) in 9bJekti1e Tests im
Engischunterricht der 0chue and /ni1ersitat) .thenaum Eerag) #obert 6"nsent
draws attention to the use of semantic and structura distractors in matching
tests of register 5 i.e. incuding words with the same meaning <or 'hrases with
identica structures= but in a different register.
1?
This t"'e of item has been used in the /ni1ersit" of Cambridge Loca
E+aminations 0"ndicate! >irst Certificate in Engish.
18 0amonte) . L 127A
TechniBues in Teaching ;riting. #ELC Journa 1<1=
0harwood50mith) 127A New :irections in Teaching ;ritten Engish. >orum
183 M(E<,=
Criteria and t"'es of tests
1-.1 Eaidit" This section attem'ts to summarise much of what was contained in Cha'ter
1. Brief") the 1aidit" of a test is the e+tent to which it measures what it is
su''osed to measure and nothing ese.
E1er" test) whether it be a short)
informa cassroom test or a 'ubic e+amination) shoud be as 1aid as the
constructor can make it. The test must aim to 'ro1ide a true measure of
the 'articuar ski which it is intended to measure! to the e+tent that it
measures e+terna knowedge and other skis at the same time) it wi not
be a 1aid test. >or e+am'e) the foowing test item is in1aid if we wish
soe" to measure writing abiit"! (s 'hotogra'h" an art or a scienceG
:iscuss.@ (t is ike" to be in1aid sim'" because it demands some
knowedge of 'hotogra'h" and wi conseBuent" fa1our certain students.
0imiar") man" ora inter1iew tests run the risk of assessing
'ersonait" as we as ora 'roficienc". The atter case is an interesting one)
howe1er) as it can be strong" argued that success shoud be measured b"
concentrating on whether the task set has been achie1ed using whate1er
strategies the students ha1e at their dis'osa rather than sim'" on 'ure
inguistic abiit". (n other words) wh" shoud 'ersonait" and a 1ariet" of
usefu non51erba strategies be ignored in assessing ora 'erformance in a
anguageG The answer to this Buestion) howe1er) must awa"s de'end on
the 'ur'ose of the test! an achie1ement test or a cassroom 'rogress test
might we e+cude such factors as 'ersonait" whie a 'roficienc" test or
'ubic e+amination might consider such factors as im'ortant in e1auating
ora abiit" in the target anguage. Ne1ertheess) the Buestion ser1es to
iustrate a significant difference between the rather narrow
's"chometric5structura a''roach and the broader communicati1e
a''roach to testing.
>ace
1aidit" E1en a su'erficia ins'ection of the essa" item referred to in the first
'aragra'h in this section woud be sufficient to re1ea that it was not 1aid.
This t"'e of 1aidit") in fact) is often referred to as
face 1aidit"! if a test
item ooks right to other testers) teachers) moderators) and testees) it can
be described as ha1ing at east face 1aidit". (t is) therefore) often usefu to
show a test to coeagues and friends. .s constructors of the test) we can
become so in1o1ed in the test that we sometimes fai to stand back and
ook at the indi1idua test items obJecti1e". 9n" if the test is e+amined b"
other 'eo'e can some of the absurdities and ambiguities then be
disco1ered.
182
i

Language tests whicct ha1e been designed 'rimari" for one countr"
and are ado'ted b" another countr" ma" ack face 1aidit". . 1ocabuar"
or reading com'rehension test containing such words as @t"'hoon@.
@sam'an@) @abacus@) and @cho'sticks@ wi ob1ious" not be 1aid in East
.frica no matter how 1aid and usefu a test it has 'ro1ed in Hong $ong.
The same argument a''ies to man" standardised tests designed for
immigrants in the /nited 0tates and used ater in man" other 'arts of the
word.
.though no substitute for em'irica 1aidit" in 'ubic e+aminations
and standardised tests) face 1aidit" can 'ro1ide not on" a Buick and
reasonabe guide but aso a baance to too great a concern with statistica
ana"sis. *oreo1er) the students@ moti1ation is maintained if a test has
good face 1aidit") for most students wi tr" harder if the test ooks sound.
(f) on the5!Tther hand).the.test a''ears to ha1e itte of ree1ance in the e"es
of the student) it wi cear" ack face 1aidit". 6ossib" as a direct resut)
the student wi not 'ut ma+imum effort into 'erforming the tasks set in
the test! hence the reiabiit" of the test wi be affected.
(t is 'ossibe for a test to incude a the com'onents of a 'articuar
teaching 'rogramme being foowed and 5"et at the same time ack face
1aidit". >or e+am'e) a reading test for engineers coud ha1e most of the
grammatica features of the anguage of engineering <e.g. freBuent use of
the 'assi1e 1oice) nomina strings. etc.= as we as most of the anguage
functions and notions associated with reading and writing engineering te+ts
<e.g. defining) cassif"ing) h"'othesising) drawing concusions) describing
'rocesses) e+'ressing notions of Buantit" and amount) etc.=. E1en so.
howe1er) the test wi ack face 1aidit" if the subJect of the reading te+t
concerns) sa") 'ubic institutions in Britain.
The conce't of face 1aidit" is far from new in anguage testing but the
em'5 3sis now 'aced on it is reati1e" new. (n the 'ast) face 1aidit" was
regarded b" man" test writers sim'" as a 'ubic reations e+ercise. Toda".
howe1er) most designers of communicati1e tests regard face 1aidit" as the
most im'ortant of a t"'es of test 1aidit". (ndeed) man" argue that a test
must ook 1aid e1en as far as the re'roduction of the materia itsef is
concerned! thus) a test of reading com'rehension using such authentic
tasks as reading and skimming news'a'ers must contain actua news'a'ers
5 or) at east) artices 'rinted in e+act" the same wa" as the" a''eared in
the news'a'er from which the" were taken.
Content 1aidit" This kind of 1aidit" de'ends on a carefu ana"sis of the anguage being
tested and of the 'articuar course obJecti1es. The test shoud he so
constructed as to contain a re'resentati1e sam'e of the course) the
reationshi' between the test items and the course obJecti1es awa"s being
a''arent. There is a strong tendenc". es'ecia" in muti'e5choice testing.
to test on" those areas of the anguage which end themse1es readi" to
tasting. *an" tests of 'honoog". for instance) concentrated in the 'ast. on
testing 'honeme discrimination rather than the more im'ortant features of
stress and intonation! one cannot he' but sus'ect the reason for this was
sim'" that 'honeme tests were easier to construct than items testing stress
and intonation.
;hen embarking on the construction of a test. the test writer shoud
first draw u' a tabe of test s'ecifications) describing in 1er" cear and
'recise terms the 'articuar anguage skis and areas to be incuded in the
test. (f the test or sub5test being constructed is a test of grammar) each of
the grammatica areas shoud then be gi1en a 'ercentage weighting <e.g.
1A-
the future sim'e tense 1- 'er cent) uncountabe nouns 18 'er cent) reati1e
'ronouns 1- 'er cent) etc.= as touched u'on 're1ious". (f the test or sub5
test concerns reading) then each of the reading sub5skis shoud be gi1en a
weighting in a simiar wa" <e.g. deducing word meanings from conte+tua
cues ,- 'er cent) search5reading for s'ecific information 3- 'er cent)
reading between the ines and inferring 1, 'er cent) intensi1e reading
com'rehension ?- 'er cent) etc.=. (t scarce" matters what the tota
'ercentage is! the im'ortant 'oint is that the test writer has attem'ted to
Buantif" and baance the test com'onents) assigning a certain 1aue to
indicate the im'ortance of each com'onent in reation to the other
com'onents in the test. (n this wa") the test shoud achie1e content 1aidit"
and refect the com'onent skis and areas which the test writer wishes to
incude in the assessment.
Construct 1aidit"
Em'irica 1aidit"
(f a test has construct 1aidit")
it is ca'abe of measuring certain s'ecific
characteristics in accordance with a theor" of anguage beha1iour and
earning. This t"'e of 1aidit" assumes the e+istence of certain earning
theories or constructs under"ing the acBuisition of abiities and skis. >or e+am'e) it can
be argued that a s'eed reading test based on a short
com'rehension 'assage is an inadeBuate measure of reading abiit" <and
thus has ow construct 1aidit"= uness it is beie1ed that the s'eed reading of short 'assages
reates cose" to the abiit" to read a book Buick" and efficient" and is a 'ro1en factor in
reading abiit". (f the assum'tion is
hed that s"stematic anguage habits are best acBuired at the eementar"
e1e b" means of the structura a''roach) then a test which em'hasises the communicati1e
as'ects of the anguage wi ha1e ow construct 1aidit".
Con1erse") if a communicati1e a''roach to anguage teaching and
earning has been ado'ted throughout a course) a test com'rising chief"
muti'e5choice items wi ack construct 1aidit".
. fourth t"'e of 1aidit" is usua" referred to as statistica or em'irica
1aidit". This
1aidit" is obtained as a resut of com'aring the resuts of the test with the
resuts of some criterion measure such as!
5 an e+isting test) known or beie1ed to be 1aid and
gi1en at the same
time4
or
5 the teacher@s ratings or an" other such form of inde'endent assessm ent
gi1en at the same time4 or
5 the subseBuent 'erformance of the testees on a certain task measured b"
some 1aid test4 or
5 the teacher@s ratings or an" other such form of inde'endent assessment
gi1en ater.
#esuts obtained b" either of the first two methods abo1e are
measures of the test@s concurrent 1aidit"
in res'ect of the 'articuar
criterion used. The third and fourth methods estimate the 'redicti1e
1aidit"
of a test which is used to 'redict future success. ;e coud estimate the
'redicti1e 1aidit" of a anguage test administered to ci1i engineers
embarking on a ci1i engineering course gi1en in the medium of Engish) for
instance) if we measured their subseBuent 'erformances on their academic
courses and com'ared these resuts with the resuts of the anguage test gi1en
at the beginning of their course.
The test situation or the techniBue used is awa"s an im'ortant factor in
determining the o1era 1aidit" of an" test. .though an idea test
1A1
situation wi b" no means guarantee 1aidit") a 'oor test situation wi
certain" detract from it. (s a istening com'rehension test 1aid if the
students hear on" a disembodied 1oice on) sa") a 'oor Buait" ta'e
recorderG
*oreo1er) the fact that a new test com'ares fa1ourab" with another
su''osed" 1aid test wi not necessari" ensure that the new test is 1aid)
'articuar" if the other test is not rea" a 1aid measure) itsef. (n short) how
far can we trust the criteria we use for estabishing 1aidit"G This is
one of the maJor criticisms of the whoe conce't of em'irica 1aidit" made
b" the communicati1e schoo of test writers. The argument is sim'" that the
estabished criteria for measuring 1aidit" are themse1es 1er" sus'ect! two
in1aid tests do not make a 1aid test.
1-., #eiabiit" #eiabiit" is a necessar" characteristic of an" good test! for it to be 1aid at
a) a test must first. be reiabe as a measuring instrument. (f the test is
administered to the same candidates on different occasions <with no
anguage 'ractice work taking 'ace between these occasions=) then) to the
e+tent that it 'roduces differing resuts) it is not reiabe. #eiabiit"
measured in this wa" is common" referred to as testKre5test reiabiit"
to
distinguish it from
markKre5mark reiabiit" and the other kinds of
reiabiit" described ater in this section. This atter kind of reiabiit"
denotes the e+tent to which the same marks or grades are awarded if the
same test 'a'ers are marked b" <i= two or more different e+aminers or <ii= the
same e+aminer on different occasions. (n short) in order to be reiabe) a test
must be consistent in its measurements.
#eiabiit" is of 'rimar" im'ortance in the@use of both 'ubic
achie1ement and 'roficienc" tests and cassroom tests. *ethods of
estimating the reiabiit" of indi1idua items in a test wi be indicated in thc
ne+t cha'ter. Howe1er) an a''reciation of the 1arious factors affecting
reiabiit" is im'ortant for the teacher at the 1er" outset) since man"
teachers tend to regard tests as infaibe measuring instruments and fai to
reaise that e1en the best test is indeed a somewhat im'recise instrument with
which to measure anguage skis.
>actors affecting the reiabiit" of a test are!
5 the e+tent of the sam'e of materia seected for testing! whereas 1aidit"
is concerned chief" with the content of the sam'e) reiabiit" is
concerned with the siLe. The arger the sam'e <i.e. the more tasks the
testees ha1e to 'erform=) the greater the 'robabiit" that the test as a
whoe is reiabe 5 hence the fa1ouring of obJecti1e tests) which aow for
a wide fied to be co1ered.
5 the administration of the test! is the same test administered to different
grou's under different conditions or at different timesG Cear") this is an
im'ortant factor in deciding reiabiit") es'ecia" in tests of ora
'roduction and istening com'rehension.
The wa" in which this factor differs from test situation 1aidit" can be
seen from the foowing e+am'e! if a recording for a istening
com'rehension test is initia" 'oor in Buait") then it is 'oor in Buait" for
a testees. This wi conseBuent" make for in1aidit" <uness s'eech has
been deiberate" masked with noise) as a testing de1ice=. But if the Buait"
of the recording is good and if certain grou's hear it 'a"ed under good
acoustic conditions whie other grou's hear it under 'oor acoustic
conditions) this wi make for unreiabiit" and therefore in1aidit".
1A,
5 test instructions! are the 1arious tasks e+'ected from the testees made
cear to a candidates in the rubricsG
5 'ersona factors such as moti1ation and iness.
5 scoring the test! one of the most im'ortant factors affecting reiabiit".
9bJecti1e tests o1ercome this 'robem of marker reiabiit") but
subJecti1e tests are sometimes faoed with it! hence the im'ortance of the
work carried out in the fieds of the muti'e5marking of com'ositions and
in the use of rating scaes.
Communicati1e anguage testing has recent" introduced another
dimension to the whoe conce't of reiabiit"! name") 'rofie re'orting. (n
order to obtain a fu 'rofie of a student@s abiit" in the target anguage) it
is necessar" to assess his or her 'erformance se'arate" for each of the
different areas of communication! e.g. istening com'rehension) s'eaking
and istening) reading) reading and writing <summarising) etc.=) and
writing. >urthermore) 'erformance is assessed according to the 'ur'ose for
which the anguage is to be used! e.g.@ academic) occu'ationa) socia
sur1i1a. The obJect of the sub5tests through which 'erformance is assessed.
is to indicate the e+tent of the earner@s master" of the 1arious anguage
skis which he or she wi reBuire for a 'articuar 'ur'ose. . score or
grade is gi1en for each of the skis or areas seected for testing) and an
a1erage mark is e1entua" obtained. This atter mark) howe1er) is on"
gi1en aongside the 1arious scores which ha1e contributed to it. 6rofie
re'orting is thus 1er" 1auabe for 'acement 'ur'oses) and indeed it is an
essentia feature of one of the most wide" used 'roficienc" tests set in
Britain and administered in man" countries throughout the word.@ .
student@s 'erformance on the 1arious 'arts of the test can be shown in the
form of a sim'e tabe or chart) in which the target score a''ears beside
the student@s score. (t is thus 1er" eas" to com'are a student@s 'erformance
e1es in each area with the reBuired e1es.
9ne method of measuring the reiabiit" of a test is to re5administer
the same test after a a'se of time. (t is assumed that a candidates ha1e
been treated in the same wa" in the inter1a 5 that the" ha1e either a been
taught or that none of them ha1e. 6ro1ided that such assum'tions <which
are freBuent" hard to Justif"= can be made) com'arison of the two resuts
woud then show how reiabe the test has 'ro1ed. Cear") this method is
often im'racticabe and) in an" case) a freBuent use of it is not to be
recommended) since certain students wi benefit more than others b" a
famiiarit" with the t"'e and format of the test. *oreo1er) in addition to
changes in 'erformance resuting from the memor" factor) 'ersona factors
such as moti1ation and differentia maturation wi aso account for
differences in the 'erformances of certain students.
.nother means of estimating the reiabiit" of a test is b"
administering 'arae forms of the test to the same grou'. This assumes
that two simiar 1ersions of a 'articuar test can be constructed! such tests
must be identica in the nature of their sam'ing) difficut") ength) rubrics)
etc. 9n" after a fu statistica ana"sis of the tests and a the items
contained in them can the tests safe" be regarded as 'arae. (f the
)correation between the two tests is high <i.e. if the resuts deri1ed from the
two tests corres'ond cose" to each other=) then the tests can be termed
reiabe.
The s'it5haf method is "et another means of measuring test
reiabiit". This method estimates a different kind of reiabiit" from that
1A3
11
estimated b" testKre5test 'rocedures. The s'it5haf method is based on the
'rinci'e that) if an accurate measuring instrument were broken into two
eBua 'arts) the measurements obtained with one 'art woud corres'ond
e+act" to those obtained with the other. The test is di1ided into two and
the corres'onding scores obtained) the e+tent to which the" correate with
each other go1erning the reiabiit" of the test as a whoe. 9ne 'rocedure
wide" used is to ascertain the correation between the scores on the odd
numbered items and those on the e1en numbered items. Howe1er) if the
items are graded according to increasing difficut") di1ision according to
odd and e1en numbers woud not be an accurate means of assessing
reiabiit"). since item , woud be sight" more difficut than item 1) item ?
again more difficut than item 3) and so on. . more accurate 'rocedure is to
baance the items as foows!
item 1 ? 8
3
2 1,
against item ,
3 A 7 1- 11
Howe1er) it woud be better) though ess con1enient) to aow chance to
decide which items go into one haf and which into the other.
The reiabiit" of the whoe test can be estimated b" using the formua!
O O
N < O m N5m =
ri i
N 5 ( Z 1
N+, K
where N 7 the number of items in the test4
m 7 the mean score on the test for a the testees <see 'age
178=4
+ 7 the standard de1iation of a the testees@ scores <see 'age
17A=) and
r1 7 reiabiit".
<Note that in this formua) + has to be sBuared.=
(n 0ections 11. 1 and 11., the cacuation of the mean and standard
de1iation of scores on a anguage test containing ?- items is iustrated. The
mean is found to be ,7 and the standard de1iation ?.-77. /sing these
i f gures with the abo1e formua) we obtain!
O O ?- < 5 ,7 + 13 Z
rI 32 1 ?- + 1A.AA,=
7 -.?3?
This formua is sim'e to use since <i= it a1oids troubesome
correations and <ii=) in addition to the number of items in the test) it
in1o1es on" the test mean and standard de1iation) both of which are
norma" cacuated an"how as a matter of routine.
>ina") it.shoud be noted that a test can be reiabe without
necessari" 'ossessing 1aidit". Howe1er) reiabiit" is cear" inadeBuate b"
itsef if a test does not succeed in measuring what it is su''osed to
measure.
1-.3
#eiabiit" 1ersus .s we ha1e seen) test 1aidit" and reiabiit" constitute the two chief
1aidit"
criteria for e1auating an" test) whate1er the theoretica assum'tions
under"ing the test. The fundamenta 'robem) howe1er) ies in the confict
between reiabiit" and 1aidit". The idea test shoud) of course) be both
reiabe and 1aid. Howe1er) the greater the reiabiit" of a test) the ess
1aidit" it usua" has. Thus) the rea5ife tasks contained in such 'roducti1e
skis tests as the ora inter1iew. roe5'a") etter writing) etc. ma" ha1e
been gi1en high construct and face 1aidit" at the e+'ense of reiabiit".
1 A?
9bJecti1e tests are cear" not subJect to such a degree of unreiabiit".
But does this mean that a forms of subJecti1e testing shoud be
abandoned in fa1our of obJecti1e testingG >or most 'ur'oses obJecti1e
tests are 1ast" inferior to such more meaningfu and communicati1e tasks
as free5writing) roe 'a"ing and 'robem5so1ing. Language5earning
beha1iour cannot be demonstrated soe" b" means of an abiit" to seect
correct o'tions from gi1en aternati1es. Language use sim'" does not
function in this wa".
The choice facing the test writer is) therefore) whether to attem't to
increase the 1aidit" of a test known to be reiabe or ese to increase the
reiabiit" of a test known to be 1aid. (f the test writer tries to do the
former) he or she wi be faced with an im'ossibe task because the 1er"
features which make the test reiabe <usua" muti'e5choice and
com'etion items tested out of conte+t= are those 1er" features which
render the test in1aid. ConseBuent") it is essentia to de1ise a 1aid test
f i rst of a and then to estabish wa"s of increasing its reiabiit".
9ne effecti1e wa" of increasing test reiabiit" in such cases is b"
means of a carefu" drawn u' banding s"stem or rating scae. 0uch a scae
<with a cear and concise descri'tion of the 1arious characteristics of
'erformance at each e1e= enabes the marker to identif" 'recise" what.he
or she e+'ects for each band and then assign the most a''ro'riate grade to
the task being assessed. >urthermore) markers are encouraged in this wa"
not on" to be consistent in their marking but aso to formuate Judgements
in Buaitati1e terms before ater con1erting such Judgements into
Buantitati1e assessments. .n e+am'e of such a rating scae is gi1en in the
're1ious cha'ter! it is sufficient to em'hasise here that reiabiit" can be
increased b" means of 'rofie re'orting and Buaitati1e Judgements <rather
than Buantitati1e ones=.
1-.?
:iscrimination
0ometimes an im'ortant feature of a test is its ca'acit" to discriminate
among the different candidates and to refect the differences in the
'erformances of the indi1iduas in the grou'. >or e+am'e) 7- 'er cent
means nothing at a uness a the other scores obtained in the test are
known. >urthermore) tests on which amost a the candidates score 7- 'er
cent cear" fai to discriminate between the 1arious students. Tests which
are designed for a arge test 'o'uation <and which are to be standardised=
are first tried out on a re'resentati1e sam'e of students. This sma sam'e
mirrors the much arger grou' for whom the test is intended. The resuts of
the test are then e+amined to determine the e+tent to which it
discriminates between indi1iduas who are different. ;hen the fina 1ersion
of the test is e1entua" used) therefore) its discriminator" 'owers ha1e
aread" been estabished.
ConseBuent") there wi then be itte need for
concern if) for e+am'e) it is found that the scores of indi1iduas in a grou'
custer around a centra 'oint. The test has been 'ro1ed ca'abe of
discriminating4 it does not do so in this case because there is nothing to
discriminate.
The e+tent of the need to discriminate wi 1ar" de'ending on the
'ur'ose of the test! in man" cassroom tests) for e+am'e) the teacher wi
be much more concerned with finding out how we the students ha1e
mastered the s"abus and wi ho'e for a custer of marks around the 3-
'er cent and 2- 'er cent brackets. Ne1ertheess) there ma" be occurrences
in which the teacher ma" reBuire a test to discriminate to some degree in
order to assess reati1e abiities and ocate areas of difficut".
1A8
(
E1en the best test can ne1er be so 'recise as to determine the true
score of a testee to within , or 3 'er cent of a certain mark. This ack of
'recision is often referred to as the
margin of error in a test. (t is im'ortant
to be aware of this gre") borderine area in test scores) es'ecia" when cut5
off 'oints <i.e. 'assKfai e1es= are being fi+ed. (ronica") such cut5off
'oints are usua" fi+ed around a midde range of scores <i.e. ?- 'er cent or
8- 'er cent=) affecting most of the students. (t is 'recise" in this range of
scores where the im'ortant decisions about 'ass or fai are usua" made.
There is a imit to the e+tent to which such scores can be refined in
order to enabe these 'assKfai decisions to be made with com'ete
certaint". (n other words) the gre" borderine area referred to in the
're1ious 'aragra'h wi awa"s remain. The situation can be im'ro1ed
consider .b" howe1erOiftheOscores themse1es can be s'read out o1er the
whoe range of the scae. (n this wa") fewer students wi be affected when
discriminations are made between scores on the critica 'art of the scae.
(n the foowing gra'h showing the usua 'attern of scores) the
e+aminer wi be uncertain about the arge number of students whose
scores fa within , 'er cent on either side of the 'assKfai score of 8- 'er
cent.
,-
18
1-
8
-
1- ,- 3- ?- 8- A- 7-
3-
2- 1--
(f the test can be constructed so as to discriminate as much as 'ossibe.
the scores wi then be s'read out o1er a far wider range as shown on the
foowing gra'h. (t wi sti be im'ossibe to be certain when distinguishing
between a student who has scored ?2 'er cent <and thus faied= and a
student who has scored 81 'er cent <and 'assed=. Howe1er) the number of
students in this borderine area wi now be much smaer.
The s'read of scores wi now a''ear in the gra'h ike this!
1- ,- 3-
?-
8- A- 7- C- 2-
1--
1 AA
How can test scores be s'read in this wa" and the test conseBuent"
make finer discriminations among the testeesG Brief") the items in the test
shoud be s'read o1er a wide difficut" e1e as foows!
5 e+treme" eas" items
5 1er" eas" items
5 eas" items
5 fair" eas" items
5 items beow a1erage difficut" e1e
5 items of a1erage difficut" e1e
5 items abo1e a1erage difficut" e1e
5 fair" difficut items
5 difficut items
5 1er" difficut items
5 e+treme" difficut items.
1-.8 .dministration . test must be 'racticabe! in other words) it must be fair" straight5
forward to administer. (t is on" too eas" to become so absorbed in the
actua construction of test items that the most ob1ious 'ractica
considerations concerning the test are o1erooked. The ength of time
a1aiabe for the administration of the test is freBuent" misJudged e1en b"
e+'erienced test writers) es'ecia" if the com'ete test consists of a
number of sub5tests. (n such cases sufficient time ma" not be aowed for
the administration of the test) the5coection of the answer sheets) the
reading of the test instructions) etc. (n the case of a arge5scae tests) the
time to be aowed shoud be decided on as a resut of a 'iot
administration of the test <i.e. a tr"out of the test5 to a sma but
re'resentati1e grou' of testees=.
.nother 'ractica consideration concerns the answer sheets@ and the
stationer" used. *an" tests reBuire the testees to enter their answers on
the actua Buestion 'a'er <e.g. circing the etter of the correct o'tion=)
thereb" unfortunate" reducing the s'eed of the scoring and 're1enting
the Buestion 'a'er from being used a second time. (n some tests) the
candidates are 'resented with a se'arate answer sheet) but too often
insufficient thought has been gi1en to 'ossibe errors arising from the
<menta= transfer of the answer from the conte+t of the item on the
Buestion 'a'er to the answer sheet itsef. Confusion ma" resut) for
e+am'e) if the items are numbered 1ertica" on the Buestion 'a'ers and
horiLonta numbering is ado'ted for the corres'onding answer sheet!
1. %ou@d aread" eft5b" se1en o@cock...
. "ouG
.. didn@t
B. weren@t
C. hadn@t
:. ha1en@t
,. (f "ou take swimming essons) "ou ...
. soon.
.. wi be abe to swim
B. swim
C. can swim
:. sha ha1e swum
1A7
3. :id an"one te Tim ...
.. off
B. o1er
C. on
:. about
. the careess error he madeG
6ut a cross <M= in the bo+ containing the etter of the correct answer.
1.
(.(B(C(:( ,. (.(B(C(:( 3. :
The use of se'arate answer sheets) howe1er) great" faciitates marking
<through the use of a mask or ke"= and is strong" recommended when
arge numbers of students are being tested.
(t is of 'aramount im'ortance that e+aminers are fu" con1ersant with
the test situation. (f the test is to be administered b" se1era e+aminers
5working in different test5centres) cear directions s'ecif"ing e+act" what
each e+aminer shoud sa1 and do shoud be issued in order to ensure that
e+act" the sarrie 'rocedure is foowed in each centre. .though this
'rinci'e seems ob1ious enough) it is e+treme" difficut in man" cases for
test writers to see their own test from the 'oint of 1iew of the 'eo'e
conducting the test) sim'" because the" are so cose" in1o1ed in their test
that the" are incined to take too man" things for granted. ConseBuent")
where1er 'ossibe) a test arrangements shoud be discussed in detai b"
the test writer and b" those conducting the test) the 1arious ste's in the
administering of each sub5test being stated in sim'e anguage and cear"
numbered. This is 'articuar" essentia in tests of istening com'rehension
and ora 'roduction) where the administrator@s roe in the test is so
im'ortant. .ccom'an"ing these instructions shoud be a cear statement of
aims together with a com'rehensi1e <but sim'e= marking scheme.
Before beginning to construct a test) the test writer must make certain
that the necessar" eBui'ment wi be a1aiabe in each centre and that there
wi be a high degree of standardisation in the test administration Cear")
it is useess to record taks or diaogues on ta'e if certain test centres do
not ha1e a ta'e recorder. ;hat is not so ob1ious) howe1er) is the 'otentia
unreiabiit" of a istening test resuting from the different siLes of the
rooms where the test is administered and the different degrees of
interference caused b" e+traneous noise. The Buestion of 'racticabiit")
howe1er) is not confined soe" to auraEora tests! such written tests as
situationa com'osition and controed writing tests de'end not on" on the
a1aiabiit" of Buaified markers who can make 1aid Judgements
concerning the use of anguage) etc.) but aso on the ength of time
a1aiabe for the scoring of the test.
. fina 'oint concerns the 'resentation of the test 'a'er itsef. ;here
'ossibe) it shoud be 'rinted or t"'ewritten and a''ear neat) tid" and
aesthetica" 'easing. Nothing is worse and more disconcerting to the
testee than an untid" test 'a'er) fu of miss'eings) omissions and
corrections.
1-.A Test instructions
0ince most students taking an" test are working under certain menta
to the candidate 'ressures) it is essentia that a instructions are cear" written and that
e+am'es are gi1en. /ness a students are abe to foow the instructions)
the test wi be neither reiabe not 1aid. &rammatica terminoog" shoud
be a1oided and such rubrics as the foowing rewtHritten!
1A3
6ut the correct 'ronoun in the banks.
Choose one of the foowing 1erbs to go in each bank s'ace and 'ut it in
the correct tense.
0tudents ma" be abe to 'erform a the reBuired tasks without ha1ing an"
knowedge of forma grammar. (ndeed) since their knowedge of forma
grammar is not being tested) a reference to grammatica terms shoud be
a1oided. Thus) the first of the rubrics abo1e shoud be rewritten and the
'hrase Dwords ike the foowing@ <foowed b" e+am'es= be used to re'ace
D'ronouns@4 the second rubric shoud refer to Dwords@ instead of 1erbs) and
e+am'es shoud be gi1en so that students are shown the tense changes
the" are reBuired to make. This 'rinci'e does not a''" on" to
grammatica terms! if students are instructed to 'ut a tick o''osite the
correct answer) an e+am'e of what is meant b" the word Dtick@ shoud be
gi1en 5 e.g. <(K =. The same a''ies to crosses) circes) underining) etc.
0ometimes it is difficut to a1oid writing cums" rubrics or rubrics
consisting of com'e+ sentences abo1e the difficut" e1e being tested) e.g.
.nswer each of the foowiig Buestions b" seecting the word or grou' of
words which best com'etes each sentence from the words or grou's of
words which are ettered .) B) C) and :.
>or each of the banks in thG foowing sentences choose one of the
words or grou's of words which best com'etes the sentence. ;rite in the
s'ace shown b" the dotted ine the etter corres'onding to the word) or
grou' of words) which best com'etes the sentence.
9ne 'ossibe soution is to write the rubric in short sentences 5 cear" and
concise"!
Choose the word or grou' of words which best com'etes each
sentence. Each is ettered .) B) C) or :. 6ut a circe round the etter of the
correct answer.
.nother soution is to use the students@ first anguage when the test grou' is
monoingua. Howe1er) this 'rocedure is recommended on" at the
eementar" e1e where absoute" necessar".
The rubrics in too man" e+isting tests assume that the testees aread"
know what to do. ;hie this can be e+cused to a certain e+tent in cass
tests where the teacher is 'resent to e+'ain) it is 1er" disturbing when it
occurs in more wide" used achie1ement and 'roficienc" tests where no
such he' is a1aiabe. Howe1er) it is often difficut to strike the right
baance between short) cear instructions and ong) in1o1ed rubrics. .
rubric shoud ne1er) in itsef) become a test of reading com'rehension. The
foowing are further e+am'es of cear instructions!
Com'ete each of the foowing sentences. ;rite in the bank s'ace the
etter of the correct word or words. The first one is an e+am'e.
There are ?- Buestions in this section. Each Buestion consists of an
incom'ete sentence. This incom'ete sentence is foowed b" fi1e
'ossibe wa"s of com'eting it. Each wa" is abeed .) B) C) :) or E.
Choose the answer which "ou think best com'etes the sentence. The
first one is done for "ou.
Each word shoud be carefu" considered in the writing of a rubric. >or
e+am'e) the word Dbest@ is used in certain instances instead of Dcorrect@
because man" test items <es'ecia" those in tests of 1ocabuar"= contain
1A2
se1era @correct@ answers) athough on" one is rea" acce'tabe and cear"
the reBuired answer. >ina") a rubrics shoud be tried out on 'iot grou's in
the same wa" in which items are tried out.
Because the writing of cear and concise rubrics is so difficut) it is
essentia that sim'e e+am'es are 'ro1ided for the testees. 9ne or two
e+am'es of the t"'e of. task set are recommended) uness the 'articuar
testing techniBue ado'ted is one with which testees are e+treme" famiiar.
(ndeed) if the testees are a unfamiiar with the t"'e of test being gi1en) it
is ad1isabe for the test su'er1isor to work through a few e+am'es with
them. (n one test of 'roficienc") the testees are gi1en fi1e @'ractice@ items
to work through) their answers being subseBuent" checked before the test
is commenced. This sma 'recaution ensures that a testees are con1ersant
with the t"'e of test in which the" are about to 'artici'ate. (n certain other
tests4 a7'ractice@5testHisadministered beforehand to the testees. 0uch a test
is s'ecia" constructed to incude e+am'es of a the t"'es of items in the
test 'a'er.
(f new testing techniBues are being used on a arge scae for the first
time) it is essentia for the test writers and test administrators concerned to
construct sam'e items and rubrics to send to schoos we in ad1ance)
together with sufficient detaied information about the new 'rocedures. No test
can 'ossib" be 1aid if the techniBues ado'ted are so new and
unfamiiar as to bewider the testees.
1-.7 Backwash effects
17-
(n Cha'ter ( and throughout this book the im'ortance of the backwash
effects of testing on teaching has been em'hasised. (n Cha'ter 1 reference
was made to ora e+amining) where it was 'ointed out that) in s'ite of
'ossibe unreiabiit") ora tests shoud be continued as far as 'ossibe in
certain anguage earning situations if for no other reason than the
backwash effects the" ha1e on the teaching that takes 'ace before the test.
The 'ossibe conseBuences of man" reading com'rehension tests on the
de1eo'ment of the re ading skis were cited as another e+am'e of the
backwash effects of testing. Each eement and ski has been treated in the
book in reation to its 'otentia infuence on teaching. Ne1ertheess) the
im'ortance of the infuence of testing on teaching is worth em'hasising
again) since the test constructor can so easi" become too dee'" in1o1ed in
test statistics and in other test criteria.
. arger issue at stake is the effect of obJecti1e tests on anguage
earning in genera. (m'ortant Buestions are raised4 man" as "et remain
unanswered. >or e+am'e) do obJecti1e tests freBuent" ead to a greater
em'hasis on accurac" than on fuenc"G (t is high" 'ossibe that) in s'ite of
a efforts) man" testing techniBues sti em'hasise the negati1e as'ects of
anguage earning) encouraging teachers and students to 'ace more
em'hasis on correctness than on the communicati1e as'ects of anguage
earning. . number of obJecti1e tests aso encourage the teaching of
anguage in artificia" constructed situations) thereb" reducing moti1ation
in anguage earning.
9ther issues are eBua" im'ortant in their im'ications. How much
infuence do certain tests e+ert on the com'iation of s"abuses and
anguage teaching 'rogrammesG How far is such an infuence harmfu or
actua" desirabe in certain situationsG .gain) what 'art does coaching
'a" in the test situationG (s it 'ossibe to teach effecti1e" b" re"ing soe"
on some of the techniBues used for testingG Cear") the answers to these
and other Buestions remain in some doubt. . that can be done at 'resent is
to discourage as acti1e" as 'ossibe the use of testing techniBues as the chief
means of 'ractising certain skis. ;hie coaching undoubted" 'a"s a 'art in
increasing test scores) good teaching can do far more. *oreo1er) 'ro1ided
that the students ha1e at east one o''ortunit" to 'artici'ate in a 'ractice test
'a'er before embarking on a 'articuar test) coaching b" itsef wi 'roduce
itte im'ro1ement in test resuts.
ConseBuent") whie we ma" de'ore) and must guard against) certain
backwash effects of testing on the one hand) it is fair to 'oint out on the
other hand that testing has been one of the greatest singe beneficia forces
in changing the direction of anguage teaching in man" areas and in
encouraging the more res'onsi1e teachers to e+amine not on" their own
teaching methods but aso the anguage the" are teaching.
1-.3 T"'es of tests
There is some confusion regarding the terminoog" used to denote the
different t"'es of anguage tests in use. *ost test s'eciaists) howe1er)
agree on the foowing broad di1isions! achie1ementKattainment tests)
'roficienc" tests) a'titude tests and diagnostic tests.
.chie1emenEattainment This grou' can be further subdi1ided into cass 'rogress tests and
tests
<standardised= achie1ement tests.
Cass 'rogress tests
This book has been concerned chief" with cass 'rogress tests) since these
are the most wide" used t"'es of tests. *ost teachers are) at some time or
other) reBuired to construct such tests. Each 'rogress test situation is
uniBue and can on" be e1auated fu" b" the cass teacher in the ight of
his or her knowedge of the students) the 'rogramme which the" ha1e been
foowing) and the cass teacher@s own 'articuar aims and goas. (t is
iogica to e+'ect genera 'ur'ose tests and books of tests to function as
effecti1e" as a test constructed s'ecia" for a 'articuar situation! hence
the 'ur'ose of this book in encouraging teachers to construct their own
tests.
The 'rogress test is designed to measure the e+tent to which the
students ha1e mastered the materia taught in the cassroom. (t is based on
the anguage 'rogramme which the cass has been foowing and is Just as
im'ortant as an assessment of the teacher@s own work as the student@s own
earning. #esuts obtained from the 'rogress test enabe the teacher to
become more famiiar with the work of each student and with the 'rogress
of the cass in genera. The cass 'rogress test is a teaching de1ice) its
backwash effect onteaching5and moti1ation being im'ortant features. .
good 'rogress test shoud encourage the students.to 'erform we in the
target anguage and to gain additiona confidence. (ts aim is to stimuate
earning and to reinforce what has been taught. &ood 'erformances act as
a means of encouraging the students) and athough 'oor 'erformances ma"
act as an incenti1e to more work) the 'rogress test is chief" concerned with
aowing the students to show what the" ha1e mastered. 0cores on it shoud
thus be high <'ro1ided) of course) that 'rogress has indeed been made=.
;hereas in standardised achie1ement and 'roficienc" tests) a wide range of
'erformance shoud be indicated) the 'rogress test shoud show a custer of
scores around the to' of the scae. Com'are the foowing gra'h with the
two gra'hs in 1-.?.
171
1
i
,-
18
1-
8
1- ,- 3- ?- 8- A- 7- 3- 2- 1--
.chie1ement tests
.chie1ement <or attainment= tests.) though simiar in a number of wa"s to
'rogress tests) are far more forma tests and are intended to measure
achie1ement on a arger scae. *ost annua schoo e+aminations take the
form of achie1ement tests4 a 'ubic tests which are intended to show
master" of a 'articuar s"abus are aso achie1ement tests. These tests are
based on what the students are 'resumed to ha1e earnt 5 not necessari"
on what the" ha1e actua" earnt nor on what has actua" been taught.
.chie1ement tests freBuent" take the form of secondar" schoo entrance
tests and schoo certificate e+aminations4 man" are based on a 'ubished
s"abus and e+ert a strong infuence on the teaching in schoos.
Constructors of such tests rare" teach an" of the students being tested
<often an ad1antage 'ro1ided that the test constructors are 1er" famiiar
with the teaching and earning 'robems of the testees=. (ndeed) this is
often a 'rereBuisite before an"one can be a''ointed to an" 'osition of
res'onsibiit" in connection with this t"'e of test) though this 'rinci'e
ob1ious" cannot awa"s be a''ied to schoo e+aminations.
0e1era aciiie1ement tests are standardised! the" are 're5tested) each
item is ana"sed and re1ised where necessar") norms are estabished and
com'arisons made between 'erformances of different students and
different schoos. 0ince such tests are administered "ear after "ear) it is
'ossibe to com'are 'erformances of students one "ear with those of
students taking the test another "ear.
(f the students ha1e foowed a structura a''roach to anguage
earning) it is cear" unfair to administer a communicati1e achie1ement
test at the end of their course. (t is eBua" unfair to administer a
structura5based@test to those students who ha1e foowed a communicati1e
a''roach to earning the target anguage. . good achie1ement test shoud
refect the 'articuar a''roach to earning and teaching that has 're1ious"
been ado'ted.
roficienc" tests
;hereas an achie1ement test ooks back on what shoud ha1e been earnt)
the 'roficienc" test ooks forward) defining a student@s anguage 'roficienc"
with reference to a 'articuar task which he or she wi be reBuired to
'erform. 6roficienc" tests are in no wa" reated to an" s"abus or teaching
'rogramme4 indeed) man" 'roficienc" tests are intended for students from
se1era different schoos) countries and e1en anguage backgrounds. The
'roficienc" test is concerned sim'" with measuring a student@s555 . contro of
the anguage in the ight of what he or she wi be e+'ected to do with it in
the future 'erformance of a 'articuar task. :oes the student know enough
Engish) for e+am'e) to foow a certain uni1ersit" or coege course gi1en
in the medium of EngishG :oes the student know enough Engish in order
to function efficient" in a 'articuar t"'e of em'o"mentG The 'roficienc"
test is thus concerned with measuring not genera attainment but s'ecific
skis in the ight of the anguage demands made ater on the student b" a
future course of stud" or Job.
.'titude tests
. anguage a'titude test <or 'rognostic test= is designed to measure the
student@s 'robabe 'erformance in a foreign anguage which he or she has
not started to earn! i.e. it assesses a'titude for earning a anguage.
Language earning a'titude is a com'e+ matter) consisting of such factors
as inteigence) age) moti1ation) memor") 'honoogica sensiti1it" and
sensiti1it" to grammatica 'atterning. The reati1e weighting gi1en to these
eements must de'end on man" factors and thus 1ar" considerab" from
one indi1idua to another. 0ome s'eciaists in this fied maintain that it is
neither 'ossibe nor desirabe to take an o1era measurement of anguage
a'titude4 conseBuent" a'titude is sometimes di1ided into 1arious as'ects
according to the s'ecific tasks for which a 'erson is being trained! e.g.
istening) inter'reting) transating. .'titude tests genera" seek to 'redict
the student@s 'robabe strengths and weaknesses in earning a foreign
anguage b" measuring 'erformance in an artificia anguage. The abiit" to
earn new 'honemic distinctions and aso to use anguage 'atterns in an
unfamiiar but s"stematic manner is tested b" means of the artificia
anguage. 0ince few teachers are concerned with the com'e+ fied of
a'titude testing) it is not necessar" to go into further detai here.
:iagnostic tests .though the term diagnostic test
is wide" used) few tests are constructed
soe" as diagnostic tests. .chie1ement and 'roficienc" tests45howe1er) are
freBuent" used for diagnostic 'ur'oses! areas of difficut" are diagnosed in
such tests so that a''ro'riate remedia action can be taken ater. 0ections
of tests which end themse1es 'articuar" we to diagnostic 'ur'oses are
'honeme discrimination5 tests) grammar and usage tests) and certain
controed writing tests. Cear") weaknesses indicated in a test of
1ocabuar" are not high" significant in themse1es and can on" be
regarded as indicating genera weaknesses. 0imiar") man" e+isting tests of
reading com'rehension are not 1er" suitabe for diagnostic 'ur'oses. Tests
of writing and ora 'roduction can be used diagnostica" 'ro1ided that
there is an a''reciation of the imits to which such tests can be 'ut. 0ince
diagnosing strengths and weaknesses is such an im'ortant feature of
'rogress tests and of teaching) the teacher shoud awa"s be aert to e1er"
facet of achie1ement re1eaed in a cass 'rogress test.
Note that diagnostic testing is freBuent" carried out for grou's of
students rather than for indi1iduas. (f on" one or two students make a 'articuar error)
the teacher wi not 'a" too much attention. Howe1er) if se1era students in the grou'
make a certain error) the teacher wi note the error and 'an a''ro'riate remedia
teaching.
Notes and references
( The Engish Language Testing 0er1ice <ELT0=) The British Counci
173
((
(nter'reting test scores
11.1
>reBuenc" *arks awarded b" counting the number of correct answers on a test scri't
distribution are known as raw marks. D18 marks out of a tota of ,-@ ma" a''ear a high
mark to some) but in fact the statement is 1irtua" meaningess on its own.
>or e+am'e) the tasks set in the test ma" ha1e been e+treme" sim'e and
18 ma" be the owest mark in a 'articuar grou' of scores.
T.BLE 1 T.BLE , T.BLE 3
Testee*arkTestee*ark#ank
*arkTa">reBuenc"
. ,- :
38
1 ?-
B ,8 * 3? , 32
C 33 C 33
3.8 <or
37= 33
: 38 ; 33 3.8 <or 37=
37
E ,2 L 3, 8 3A
> ,8 & 3- A.8 <or A7= 38 K 1
& 3- 0 3- A.8 <or A7= 3? K
1
H ,A E ,2 3.8 <or
37= 33
,
1 12 6 ,2 3.8 <or 37= 3, K
1
J ,7
J ,7 11 <or 1-7=
31
$ ,A N ,7 11
<or 1-7=
3-
,
L
3,
-
,7 11 <or 1-7= ,2
,
*
3?
H ,A 18 <or 137= ,3
N ,7
$
,A
18 <or 137= ,7
3
- ,7 T ,A 18
<or 137=
,A 8
6 ,2 M
,A 18
<or 137=
,8
3
S ,8 Q ,A 18 <or 137= ,?
K 1
# ,3
B
,8
12
<or 137=
,3
,
0 3- > ,8 12 <or 137= ,, K 1
T ,A
S ,8 12 <or 137= ,1
/ ,, % ,? ,1 ,- K
1
E
,3
# ,3 ,,.8 <or ,,7= 12 K 1
; 33
E
,3 ,,.8 <or ,,7= 13
M ,A
/ ,, ,? 17
% ,? .
,- ,8
1A
Q
,A 1 12 ,A 18 Tota ,A
17?
Con1erse") the test ma" ha1e been e+treme" difficut) in which case 18
ma" we be a 1er" high mark. Numbers sti e+ert a strange and 'owerfu
infuence on our societ") but the shibboeth that ?- 'er cent shoud awa"s
re'resent a 'ass mark is ne1ertheess both sur'rising and disturbing.
The tabes on the 're1ious 'age contain the imaginar" scores of a
grou' of ,A students on a 'articuar test consisting of ?- items. Tabe 1
con1e"s 1er" itte) but Tabe ,) containing the students@ scores in order of
merit) shows a itte more. Tabe 3 contains a freBuenc" distribution
showing the number of students who obtained each mark awarded4 the
strokes on the eft of the numbers <e.g. KKKK= are caed
taies and are
incuded sim'" to iustrate the method of counting the freBuenc" of
scores. Note that norma" the freBuenc" ist woud ha1e been com'ied
without the need for Tabes 1 and ,4 conseBuent") as the range of highest
and owest marks woud then not be known) a the 'ossibe scores woud
be isted and a record made of the number of students obtaining each score
in the scae <as shown in the e+am'e=.
Note that where ties occur in Tabe ,) two wa"s of rendering these are
shown. The usua cassroom 'ractice is that shown in the 'arentheses.
;here statistica work is to be done on the ranks) it is essentia to record
the a1erage rank <e.g. testees J) N and -) each with the same mark)
occu'" 'aces 1-) 11 and 1, in the ist) a1eraging 11=.
The foowing freBuenc" 'o"gon iustrates the distribution of the
scores!
8
?
35(
,7
11., *easures of
centra tendenc"
*ode
*edian
*ean
12 ,- ,1 ,, ,3 ,? ,8 ,A ,7 ,3 ,2 3- 31 3, 33 3? 38 3A
+
The mode refers to the score which most candidates obtained! in this case it is
,A) as fi1e testees ha1e scored this mark.
The median refers to the score gained b" the midde candidate in the order of
merit! in the case of the ,A students here <as in a cases in1o1ing e1en
numbers of testees=) there can ob1ious" be no midde 'erson and thus the
score hafwa" between the owest score in the to' haf and the highest
score in the bottom haf is taken as the median. The median5score in this
case is aso ,A.
The mean score of an" test is the arithmetica a1erage! i.e. the sum of the
se'arate scores di1ided b" the tota number of testees. The mode) median)
and mean are a measures of centra tendenc". The mean is the most
efficient measure of centra tendenc") but it is not awa"s a''ro'riate.
(n the foowing Tabe ? and formua) note that the s"mbo + is used
to denote the score) N the number of the testees) and m the mean. The
178
((
s"mbo f denotes the freBuenc" with which a score occurs. The s"mbo E
means the sum of.
T.BLE ?
+ f f+
38+1 38
3? + 1 3?
33 + , AA
3, + 1 3,
3- + , A-
,2 + , 83
,7+3 31 m O Mf+ 7 7-, 7
,A+8 13- N ,A
,8+3 78
,? + 1 ,?
,3+, ?A
,, + 1 ,,
,- + 1 ,-
12 + 1 12
Tota 7 7-,
7 Ef+
,7
Note that + 7 7-, is the tota number of items which the grou' of ,A
students got right between them. :i1iding b" N 7 ,A <as the formua
states=) this ob1ious" gi1es the a1erage.
(t wi be obser1ed that in this 'articuar case there is a fair" cose
corres'ondence between the mean <,7= and the median <,A=. 0uch a cose
corres'ondence is not awa"s common and has occurred in this case
because the scores tend to custer s"mmetrica" around a centra 'oint.
11.3 *easures of ;hereas the 're1ious section was concerned with measures of centra
dis'ersion
tendenc") this section is reated to the range or s'read of scores. The
mean b" itsef enabes us to describe an indi1idua student@s score b"
com'aring it with the a1erage set of scores obtained b" a grou') but it tes
us nothing at a about the highest and owest scores and the s'read of
marks.
#ange 9ne sim'e wa" of measuring the s'read of marks is based on the
difference between the highest and owest scores. Thus) if the highest score on
a 8-5item test is ?3 and the owest ,1) the range is from ,1 to ?3! i.e. ,,. (f the
highest score) howe1er) is on" 32 and the owest ,2) the range is 1-. <Note
that in both cases) the mean ma" be 3,.= The range of the ,A scores gi1en in
0ection 11.1 is! 38 5 12 7 1A.
0tandard de1iation The standard de1iation <s.d.= is another wa" of showing the s'read of
scores. (t measures the degree to which the grou' of scores de1iates from
the mean4 in other words) it shows how a the scores are s'read out and
thus gi1es a fuer descri'tion of test scores than the range) which sim'"
describes the ga' between the highest and owest marks and ignores the
information 'ro1ided b" a the remaining scores. .bbre1iations used for
the standard de1iation are either s.d. or o <the &reek etter sigma= or s.
17A
9ne sim'e method of cacuating s.d. is shown beow!
s.d. 7
1d,
N
N is the number of scores and d the de1iation of each score from the mean.
Thus) working from the ,A 're1ious resuts) we 'roceed to!
1. i fnd out the amount b" which each score de1iates from the mean <d=4
,. sBuare each resut <d,=4
3. tota a the resuts <,d,=4
?. di1ide the tota b" the number of testees <Ed,KN=4 and
8. find the sBuare root of this resut <EEd,KN=.
0core *ean :e1iation <d=0Buared <d,=
<0te' 1= 38 de1iates from ,7 b"
3
<0te' ,=
A?
3?
7 ?2
33 A 3A
33 A
3A
3, 8 ,8
3- 3
2
3-
3 2
,2 , ?
,2
, ?
,7 - -
,7
- -
,7 - -
,A 51
1
,A 51
1
,A 51 1
,A 51 1
,A 51 1
,8 5,
?
,8 5, ?
,8
5, ?
,? 53
2
,3 5? 1A
,3 5? 1A
,, 58
,8
,- 57 ?2
12 53 A?
7-,
<0te' 3=Tota
7 ?3,
<0te' ?=s.d.
O E ?3A,
<0te' 8=s.d. 7 ZK1A.A,
s.d. 7 ?.-77
?.-3
Note! (f de1iations <d= are taken from the mean) their sum <taking account of
the minus sign= is Lero _ ?, 5 ?, 7 -. This affords a usefu check on the
cacuations in1o1ed here.
. standard de1iation of ?.-3) for e+am'e) shows a smaer s'read of
scores than) sa") a standard de1iation of 3.2A. (f the aim of the test is
sim'" to determine which students ha1e mastered a 'articuar 'rogramme
177
of work or are ca'abe of carr"ing out certain tasks in the target anguage) a
standard de1iation of ?.-3 or an" other denoting a fair" narrow s'read wi be
Buite satisfactor" 'ro1ided it is associated with a high a1erage score. Howe1er)
if the test aims at measuring se1era e1es of attainment and
making fine distinctions within the grou' <as 'erha's in a 'roficienc" test=)
then a broad s'read wi be reBuired.
0tandard de1iation is aso usefu for 'ro1iding information concerning
characteristics of different grou's. (f) for e+am'e) the standard de1iation
on a certain test is ?.-3 for one cass) but 3.2A on the same test for another
cass) then it can be inferred that the atter cass is far more heterogeneous
than the former.
11.?
hem ana"sis Earier carefu consideration of obJecti1es and the com'iation of a tabe of
test s'ecifications were urged before the construction of an" test was
attem'ted. ;hat is reBuired now is a knowedge of how far those
obJecti1es ha1e been achie1ed b" a 'articuar test. /nfortunate") too
man" teachers think that the test is finished once the raw marks ha1e been
obtained. But this is far from the case) for the resuts obtained from
obJecti1e tests can be used to 'ro1ide 1auabe information concerning!
5 the 'erformance of the students as a grou') thus <in the case of cass
'rogress tests= informing the teacher about the effecti1eness of the
teaching4
5 the 'erformance of indi1idua students4 and
5 the 'erformance of each of the items com'rising the test.
(nformation concerning the 'erformance of the students as a whoe
and of indi1idua students is 1er" im'ortant for teaching 'ur'oses)
es'ecia" as man" test resuts can show not on" the t"'es of errors most
freBuent" made but aso the actua reasons for the errors being made. .s
shown in earier cha'ters) the great merit of obJecti1e tests arises from the
faci that the" can 'ro1ide an insight into the menta 'rocesses of the
students b" showing 1er" cear" what choices ha1e been made) thereb"
indicating definite ines on which remedia work can be gi1en.
The 'erformance of the test items) themse1es) is of ob1ious
im'ortance in com'iing future tests. 0ince a great dea of time and effort
are usua" s'ent on the construction of good obJecti1e items) most
teachers and test constructors wi be desirous of either using them again
without further changes or ese ada'ting them for future use. (t is thus
usefu to identif" those items which were answered correct" b" the more
abe students taking the test and bad" b" the ess abe students. The
identification. of certain difficut items in the test) together with a
knowedge of the 'erformance of the indi1idua distractors in muti'e5
choice items) can 'ro1e Just as 1auabe in its im'ications for teaching as
for testing.
. items shoud be e+amined from the 'oint of 1iew of <1= their
difficut" e1e and <,= their e1e of discrimination.
(tem difficut"
The inde+ of difficut" <or faciit" 1aue= of an item sim'" shows how eas"
or difficut the 'articuar item 'ro1ed in the test. The inde+ of difficut"
<>E= is genera" e+'ressed as the fraction <or 'ercentage= of the students
who answered the item correct". (t is cacuated b" using the formua! 55
>E7
N
172
# re'resents the number of correct answers and N the number of students
taking the test. Thus) if, out of ,A students tested obtained the correct
answer for one of the items) that item woud ha1e an inde+ of difficut" <or a
faciit" 1aue= of .77 or 77 'er cent.
>E7
,- ,A 7 .77
(n this case) the 'articuar item is a fair" eas" one since 77 'er cent of the
students taking the test answered it correct". .though an a1erage faciit"
1aue of .8 or 8- 'er cent ma" be desirabe for man" 'ubic achie1ement
tests and for a few 'rogress tests <de'ending on the 'ur'ose for which one
is testing=) the faciit" 1aue of a arge number of indi1idua items wi 1ar"
considerab". ;hie aiming for test items with faciit" 1aues faing
between .? and .A) man" test constructors ma" be 're'ared in 'ractice to
acce't items with faciit" 1aues between .3 and .7. Cear") howe1er) a
1er" eas" item) on which 2- 'er cent of the testees obtain the correct
answer) wi not distinguish between abo1e5a1erage students and beow5
a1erage students as we as an item which on" A- 'er cent of the testees
answer correct". 9n the other hand) the eas" item wi discriminate
amongst a grou' of beow5a1erage students4 in other words) one student
with a ow standard ma" show that he or she is better than another student
with a ow standard through being gi1en the o''ortunit" to answer an eas"
item. 0imiar") a 1er" difficut item) though faiing to discriminate among
most students) wi certain" se'arate the good student from the 1er" good
student.
. further argument for incuding items co1ering a range of difficut"
e1es is that 'ro1ided b" moti1ation. ;hie the incusion of difficut items
ma" be necessar" in order to moti1ate the good student) the incusion of 1er"
eas" items wi encourage and moti1ate the 'oor student. (n an" case) a few
eas" items can 'ro1ide a @ead5in@ for the student 5 a de1ice which ma" be
necessar" if the test is at a new or unfamiiar or if there are
certain tensions surrounding the test situation.
Note that it is 'ossibe for a test consisting of items each with a faciit"
1aue of a''ro+imate" .8 to fai to discriminate at a between the good
and the 'oor students. (f) for e+am'e) haf the items are answered
correct" b" the good students and incorrect" b" the 'oor students whie
the remaining items are answered incorrect" b" the good students but
correct" b" the 'oor students) then the items wi work against one
another and no discrimination wi be 'ossibe. The chances of such an
e+treme situation occurring are 1er" remote indeed4 it is high" 'robabe)
howe1er) that at east one or two items in a test wi work against one
another in this wa".
(tem discrimination The discrimination inde+ of an item indicates the e+tent to which the item
discriminates between the testees) se'arating the more abe testees from
the ess abe. The inde+ of discrimination <:= tes us whether those
students who 'erformed we on the whoe test tended to do we or bad"
on each item in the test. (t is 'resu''osed that the tota score on the test is
a 1aid measure of the student@s abiit" <i.e the good student tends to do
we on the test as a whoe and the 'oor student bad"=. 9n this basis) the
score on the whoe test is acce'ted as the criterion measure) and it thus
becomes 'ossibe to se'arate the @good@ students from the @bad@ ones in
'erformances on indi1idua items. (f the @good@ students tend to do we on
172
11
1
an item <as shown b" man" of them doing so 5 a freBuenc" measure= and
the D'oor@ students bad" on the same item) then the item is a good one
because it distinguishes the Dgood from the Dbad@ in the same wa" as the
tota test score. This is the argument under"ing the inde+ of
discrimination.
There are 1arious methods of obtaining the inde+ of discrimination! a
in1o1e a com'arison of those students who 'erformed we on the whoe
test and those who 'erformed 'oor" on the whoe test. Howe1er) whie it
is statistica" most efficient to com'are the to' ,7, 'er cent with the
bottom ,73 'er cent) it is enough for most 'ur'oses to di1ide sma sam'es
<e.g. cass scores on a 'rogress test= into ha1es or thirds. >or most
cassroom 'ur'oses) the foowing 'rocedure is recommended.
1
.rrange the scri'ts in rank order of tota score and di1ide into two
grou's of eBua siLe <i.e. the to' haf and the bottom haf=. (f there is an
odd number of scri'ts) dis'eosc with one scri't chosen at random.
, Count the number of those candidates in the u''er grou' answering the
f i rst item correct"4 then count the number of ower5grou' candidates
answering the item correct".
3 0ubtrwt the number of correct answers in the ower grou' from the
number of correct answers in the u''er grou'! i.e. find the difference in
the 'ro'ortion 'assing in the u''er grou' and the 'ro'ortion 'assing in
the ower grou'.
? :i1ide this difference b" the tota number of candidates in one grou'!
: 7 Correct / 5 Correct L
n
<: 7 :iscrimination inde+4 n 7 Number of candidates in one grou'V!
/ 7 /''er haf and L 7 Lower haf. The inde+ : is thus the difference
between the 'ro'ortion 'assing the item in / and L.=
8 6roceed in this manner for each item.
The foowing item) which was taken from a test administered to ?-
students) 'roduced the resuts shown!
( eft Tok"o ... >rida" morning.
.. in
s <
on C. at :. b"
:7
185A 2
5 8
,- 7
0uch an item with a discrimination inde+ of .?8 functions fair" effecti1e")
athough cear" it does not discriminate as we as an item with an inde+ of
.A or .7. :iscrimination indices can range from _ ( <7 an item which
discriminates 'erfect" 5 i.e. it shows 'erfect correation with the testees@
resuts on the whoe test= through - <7 an item which does not discriminate
in an" wa" at a= to 51 <7 an item which discriminates in entire" the
wrong wa"=. Thus) for e+am'e) if a ,- students in the u''er grou'
answered a certain item correct" and a ,- students in the ower grou' got
the wrong answer) the item woud ha1e an inde+ of discrimination of 1.-.
@The reader shoud carefu distineuish between n<7 the nwnber of candidates in
either the / or L grou'= and N <7 the number in the whoe grou'= as used
're1ious". 9h1ious n 7 1)K,
N.
(f) on the other hand) on" 1- students in the u''er grou' answered it
correct" and furthermore 1- students in the ower grou' aso got correct
answers) the discrt inaton inde+ woud be -. Howe1er) if none of the ,-
students in the u' 'er grou' got a correct answer and a the ,- students in
the ower grou' answered it correct") the item woud ha1e a negati1e
discrimination) shown b" 51.-. (t is high" inad1isabe to use again) or
e1en to attem't to amend) an" item showing negati1e discrimination.
(ns'ection of such an item usua" shows something radica" wrong with it.
.gain) working from actua test resuts) we sha now ook at the
'erformance of three items. The first of the foowing items has a high
inde+ of discrimination4 the second is a 'oor item with a ow discrimination
inde+4 and the third e+am'e is gi1en as an iustration of a 'oor item with
negati1e discrimination.
1 High discrimination inde+!
NE.#L% ;hen ... . Jim .
. crossed ... . the
road) he D... ran into a car.
: 7
13 5 3 7 18 7 78 >E 7
,17 -.8,8
,-
,- ?-
<The item is at the right e1e of difficut" and discriminates we.=
, Low discrimination inde+!
(f "ou .. . the be) the door woud ha1e been o'ened.
.. woud ring
C. woud ha1e rungOOO.OOO )
O
had rung
:. were ringing
>E
-78
:73,--7.18 ?-
<(ii this case) the item discriminates 'oor" because it is too difficut
for e1er"one) both Dgood@ and Dbad@.=
3 Negati1e discrimination inde+!
( don@t think an"bod" has seen him.
.. %es) someone has.
B. . %es) no one has.
C. %es) none has.
:. %es) an"one has.
: 7
? A
,- 5,- 75H1- >E7 ?- 7-.,8
<This item is too difficut and discriminates in the wrong Lircction.=
;hat has gone wrong with the third item abo1eG E1en at this stage and
without counting the number of candidates who chose each of the o'tions)
it is e1ident that the item was a trick item! in other words) the item was far
too @ce1er@. (t is e1en concei1abe that man" nati1e s'eakers woud seect
o'tion B in 'reference to the correct o'tion .. (tems ike this a too often
esca'e the attention of the test writer unti an item ana"sis actua" focuses
attention on them. <This is one e+ceent reason for conducting an item
ana"sis.=
Note that items with a 1er" high faciit" 1aue fai to discriminate and
thus genera" show a ow discrimination inde+. The 'articuar grou' of
131
((
(tem difficut" and
discrimination
(
students who were gi1en the foowing item had ob1ious" mastered the use of
for and since foowing the 'resent 'erfect continuous tense!
He@s been i1ing in Berin ... 1278.
: 7 12,-12 7 - >E 7 33 7 -.28
<The item is e+treme" eas" for the testees and has Lero
discrimination.=
>aciit" 1aues and discrimination indices are usua" recorded together in
tabuar form and cacuated b" simiar 'rocedures. Note again the formuae
used!
>EOO Correct / _5Correct L
,n
:Correct / 5 Correct L n
or >E 7 #
N
The foowing tabe) com'ied from the resuts of the test referred to
in the 'receding 'aragra'hs) shows how these measures are recorded.
(tem/ L /_L>E /5L :
1 12 12 33 .28 - -
, 13 1A ,2
.73 53 5.18
3 ,- 1, 3, .3- 3 .?-
? 13
3 ,1 83 18 .78
8 18 A ,1 .83 2 .?8
A 1A 18 31 .77 1
.-8
7 17 3 ,8 .A, 2 .?8
3 13
? 17 .?, 2 .?8
2 ? A 1- .,8 5, 5.1-
1- 1- ? 1? .38 A .3-
11 13 13 31 .73 8 .,8
1, 1,
, 1? .38 1- .8-
13 1? A ,- .8- 3 .?-
1? 8 1 A 18 ? .,-
18 7 1 3 .,- A .3-
1A 3 - 3 .-3 3 .18
Etc.
(tems showing a discrimination inde+ of beow .3- are of doubtfu use
since the" fai to discriminate effecti1e". Thus) on the resuts isted in the
tabe abo1e) on" items 3) ?) 8) 7) 3) 1-) 1,) 13 and 18 coud be safe" used in
future tests without being rewritten. Howe1er) man" test writers woud
kee' item 1 sim'" as a ead5in to 'ut the students at ease.
E+tended answer
(t wi often be im'ortant to scrutinise items in greater detai) 'articuar"
ana"sis
in those cases where items ha1e not 'erformed as e+'ected. ;e sha want
to know not on" wh" these items ha1e not 'erformed according to
e+'ectations but aso wh" certain testees ha1e faied to answer a 'articuar
item correct". 0uch tasks are reasonab" sim'e and straightforward to
'erform if the muti'e5choice techniBue has been used in the test.
1 3,
(n order to carr" out a fu item ana"sis) or an e+tended answer
ana"sis) a record shoud be made of the different o'tions chosen b" each
student in the u''er grou' and then the 1arious o'tions seected b" the
ower grou'.
(f ( were rich) ( . work.
.. shan@t B. won@t woudn@t
/ L /_L
.. 1 ? 8
B. , 8 7 >E 7
C. 1? ? 13
:. 3 7 1-
<,-= <,-= <?-=
:. didn@t
/_L O 13 7
,n ?- ?8
/5L5 1-
n ,- 7.8-
The item has a faciit" 1aue of .?8 and a discrimination inde+ of .8- and
a''ears to ha1e functioned efficient"! the distractors attract the 'oorer)
students but not the better ones.
The 'erformance of the foowing item with a ow discrimination inde+ is
of 'articuar interest!
*r ;atson wants to meet a friend in 0inga'ore this "ear.
He ... . him for ten "ears.
.. knewB. had known5 5O C!5knows
F has known
/L/_L
5
..
7 3 1-
B. ? 3 7 >E7.3,8
C. 1 2 1- : 7 .18
:.
3
8 13
<,-= <,-= <?-=
;hie distractor C a''ears to be 'erforming we) it is cear that distractors .
and B are attracting the wrong candidates <i.e. the better ones=. 9n
coser scrutin") it wi be found that both of these o'tions ma" be correct in
certain conte+ts! for e+am'e) a student ma" en1isage a situation in which *r
;atson is going to 1isit a friend whom he had known for ten "ears in Engand
but who now i1es in 0inga'ore) e.g.
He knew him <we= for ten "ears <whie he5i1ed in Engand=.
The same Justification a''ies for o'tion B.
The ne+t item shoud ha1e functioned efficient" but faied to do so! an
e+amination of the testees@ answers eads us to guess that 'ossib" man" had
been taught to use the 'ast 'erfect tense to indicate an action in the 'ast
taking 'ace before another action in the 'ast. Thus) whie the resuts
obtained from the 're1ious item refect on the item itsef) the resuts
here 'robab" refect on the teaching!
John >. $enned" ...
. born in 1217 and died in 12A3.
.. is B. has been F was :. had been
133
/L/_L
.. - , ,
B. -
3 3 >E7.A,8
C. 13 1, ,8 : 7 .-8
:. 7 3
1-
<,-=
<,-= <?-=
(n this case) the item might be used again with another grou' of students)
athough distractors . and B do not a''ear to be 'uing much weight.
:istractor : in the foowing e+am'e is ineffecti1e and cear" needs to
be re'aced b" a much stronger distractor!
.H5555.He com'ained that5he ...
O.
the same bad fim the night before.
had seen B. was seeing
C. has seen :. woud see
/L/_L
..
1?
3
,,
B. ? 7 11 >E7.88
C. , 8 7 : 7 .3-
:.
-
-
-
<,-= <,-= <?-=
0imiar") the e1e of difficut" of distractors C and : in the foowing
item is far too ow! a fu item ana!)sis suggests on" too strong" that the"
ha1e been added sim'" to com'ete the number of o'tions reBuired.
;asn@t that "our father o1er thereG
.. %es) he was.
C. %es) was he.
%es) it was. :. %es) was it.
/L/_L
.. 7 13 ,-
B. 13 7 ,- >E 7.8-
C. - - - : 7.3-
:. - - -
<,-= <,-=
<?-=
The item coud be made sight" more difficut a!.d thus im'ro1ed b"
re'acing distractor C b" %es) he wasn@t and : b" %es) it wasn@t. The item
is sti im'erfect) but the difficut" e1e of the distractors wi 'robab"
corres'ond more cose" to the e1e of attainment being tested.
The 'ur'ose of obtaining test statistics is to assist inter'retation of
item and test resuts in a wa" which is meaningfu and significant. 6ro1ided
that such statistics ead the teacher or test constructor to focus once again
on the content of the test) then item ana"sis is an e+treme" 1auabe
e+ercise. 9n" when test constructors misa''" statistics or become
uncritica" dominated b" statistica 'rocedures does item ana"sis begin to
e+ert a harmfu infuence on earning and teaching. (n the fina ana"sis)
the teacher shoud be 're'ared to sacrifice both reiabiit" and
discrimination to a imited e+tent in order to incude in the test certain
items which he or she regards as ha1ing a good Deducationa@ infuence on
u
the students if) for e+am'e) their e+cusion might ead to negect in
teaching what such items test.
11.8 *oderating The im'ortance of moderating cassroom tests as we as 'ubic
e+aminations cannot be stressed too great". No matter how e+'erienced test
writers are) the" are usua" so dee'" in1o1ed in their work that the"
become inca'abe of standing back and 1iewing the items with an" rea
degree of obJecti1it". There are bound to be man" bind5s'ots in tests)
es'ecia" in the fied of obJecti1e testing) where the items sometimes
contain on" the minimum of conte+t.
(t is essentia) therefore) that the test writer submits the test for
moderation to a coeague or) 'referab") to a number of coeagues.
.chie1ement and 'roficienc" tests of Engish administered to a arge test
'o'uation are genera" moderated b" a board consisting of inguists)
anguage teachers) a 's"choogist) a statistician) etc. The 'ur'ose of such a
board is to scrutinise as cose" as 'ossibe not on" each item com'rising
the test but aso the test as a whoe) so that the most a''ro'riate and
efficient measuring instrument is 'roduced for the 'articuar 'ur'ose at
hand. (n these cases) moderation is aso freBuent" concerned with the
scoring of the test and with the e1auation of the test resuts.
The cass teacher does not ha1e at his or her dis'osa a the faciities
which the 'rofessiona test writer has. (ndeed) it is often a too tem'ting
for the teacher to construct a test without showing it to an"one) es'ecia"
if the teacher has had 're1ious training or e+'erience in constructing
e+aminations of the more traditiona t"'e. /nfortunate") few teachers
reaise the im'ortance of making a s"stematic ana"sis of the eements and
skis the" are tr"ing to test and) instead of com'iing a ist of test
s'ecifications) tend to seect testabe 'oints at random from coursebooks
and readers. ;eaknesses of tests constructed in this manner are brought to
igiH. in the 'rocess of moderation. *oreo1er) because there is genera"
more than one wa" of ooking at something) it is incredib" eas" <and
common= to construct muti'e5choice items containing more than one
correct o'tion. in addition) the short conte+ts of man" obJecti1e items
encourage ambiguit") a feature which can 'ass b" the indi1idua unnoticed.
To the moderator) some items in a test ma" a''ear far too difficut or ese
far too eas") containing im'ausibe distractors4 others ma" contain
unsus'ected cues. 9n" b" moderation can such fauts be brought to the
attention of the test writer.
(n those cases where the teacher of Engish is working on his or her
own in a schoo) assistance in moderation from a friend) a s'ouse) or an
oder student wi 'ro1e beneficia. (t is sim'" im'ossibe for an" singe
indi1idua to construct good test items without he' from another 'erson.
11.A item cards and
.s must be 1er" cear at this stage) the construction of obJecti1e tests
banks necessitates taking a great dea of time and troube. .though the
scoring of such tests is sim'e and straightforward) further effort is then s'ent
on the e1auation of each item and on im'ro1ing those items which do not
'erform satisfactori". (t seems somewhat iogica) therefore) to dis'ense
with test items once the" ha1e a''eared in a test.
The best wa" of recording and storing items <together with an"
ree1ant information= is b" means of sma cards. 9n" one item is entered
on each card4 on the re1erse side of the card information deri1ed from an
item ana"sis is recorded! e.g. the faciit" 1aue <>E=) the (nde+ of
138
:iscrimination <:=) and an e+tended answers ana"sis <if carried out=.
.fter being arranged according to the eement or ski which the" are
intended to test) the items on the se'arate cards are grou'ed according to
difficut" e1e) the 'articuar area tested) etc. (t is an eas" task to arrange
them for Buick reference according to whate1er s"stem is desired.
>urthermore) the cards can be rearranged at an" ater date.
.though it wi ob1ious" take considerabie time to buid u' an item
bank consisting of a few hundred items) such an item bank wi 'ro1e of
enormous 1aue and wi sa1e the teacher a great dea of time and troube
ater. The same items can be used man" times again) the order of the items
<or o'tions within each item= being changed each time. (f there is concern
about test securit" or if there is an" other reason indicating the need for
new items) man" of the e+isting items can be rewritten. (n such cases) the
same o'tions5are genera"5ke't) but the conte+t is changed so that one of
the distractors now becomes the correct o'tion. *uti'e5choice items
testing most areas of the 1arious anguage eements and skis can be
rewritten in this wa") e.g.
<&rammar=
( ho'e "ou ... . us "our secret soon.
.. tod B. wi te C. ha1e tod
:. woud te
( wish "ou ... . us "our secret soon.
.. tod B. wi te C. ha1e tod :. woud te
<Eocabuar"= .re "ou going to wear "our best ... . for the
'art"G
.. cothes B. cothing C. coths :. cothings
5 ;hat kind of ...
. is "our new suit made ofG
.. cothes B. cothing C. coth :. cothings
<6honeme
beat bit beat
discrimination= 55 beat beat bit
<Listening 0tudent hears!
;h" are "ou going homeG
com'rehension=
0tudent reads! .. .t si+ o@cock.
B. %es) ( am.
C. To he' m" mother.
:. B" bus.
0tudent hears! How are "ou going to :a1id@sG
0tudent reads! .. .t si+ o@cock.
B. %es) ( am.
C. To he' him.
:. B" bus.
13A
<#eading 55V5 Two5thirds of the countr"@s <fue) endea1our) industr")
com'rehensionK energ"= comes from im'orted oi) whie the remaining
1ocabuar"=
one5third comes from coa. *oreo1er) soon the countr"
wi ha1e its first nucear 'ower station.
Two5thirds of the countr"@s <fue) endea1our) industr")
'ower= takes the form of im'orted oi) whie the
remaining one5third is coa. Howe1er) e1er"one in the
countr" was made to reaise the im'ortance of coa
during the recent miners@ strike) when man" factories
were forced to cose down.
(tems rewritten in this wa" become new items) and thus it wi be
necessar" to coect faciit" 1aues and discrimination indices again.
0uch e+am'es ser1e to show wa"s of making ma+imum use of the
1arious t"'es of test items which ha1e been constructed) administered and
e1auated. (n an" case) howe1er) the effort s'ent on constructing tests of
Engish as a second or foreign anguage is ne1er wasted since the insights
'ro1ided into anguage beha1iour as we as into anguage earning and
teaching wi awa"s be in1auabe in an" situation connected with either
teaching or testing.
137
i
0eected bibiogra'h"
Cosed tests and e+aminations in Engish as a
secondKforeign anguage
.ssociated E+amining Board <.EB=
Test in Engish for Educationa 6ur'oses <TEE6=
.ssociation of #ecogniLed Engish Language 0choos
E+amination Trust <.ET=
.#EL0 9ra E+aminations! 6reiminar")55Higher
Certificate) :i'oma
British CounciK/ni1ersit" of Cambridge Loca E+aminations
0"ndicate
Engish Language Testing 0er1ice
Cit" and &uids of London (nstitute
Communication in Technica Engish
Educationa Testing 0er1ice <6rinceton) New Jerse") /0.=
Test of Engish as a >oreign Language <T9E>L=
Engish Language Teaching :e1eo'ment /nit <ELT:/=
0tages of .ttainment 0cae and Test Batter"
Engish 0'eaking Board <(nternationa= Limited <E0B=
9ra .ssessments in 0'oken Engish as an .cBuired
Language
&enera *edica Counci <&*C=
The 6L.B Test
(nstitute of Linguists Educationa Trust
E+aminations in Engish as a >oreign Language! Le1es 1)
,. 3and?
Certificates in Engish as a >oreign Language! 6reiminar")
&rade ( and &rade 11
:i'omas in Engish as a >oreign Language! (ntermediate
:i'oma and >ina :i'oma
Joint *atricuation Board <J*B=
Test in Engish <91erseas=
London Chamber of Commerce and (ndustr" <LCC(=
Engish for Commerce! Eementar" 0tage) (ntermediate
0tage) and Higher 0tage
0'oken Engish for (ndustr" and Commerce <0E>(C=
North ;est #egiona E+aminations BoardKNorth ;estern
#egiona .d1i Counci for >urther Education
Engish as a 0econd Language! , 1ersionsKe1es
9+ford :eegac" of Loca E+aminations <9+ford=
/ni1ersit" of 9+ford :eegac"@s E+aminations in Engish as
a >oreign Language! 6reiminar" Le1e Certificate and
Higher Le1e Certificate
6itman E+aminations (nstitute <6E(=
Engish as a >oreign Language <0"abus L and 0"abus C=
and Engish Language <NCE=! Eementar") (ntermediate)
Higher (ntermediate and .d1anced
#o"a 0ociet" of .rts E+aminations Board <#0.=
Engish as a >oreign Language! 0tage 1) 0tage (( and 0tage
(((
Communicati1e /se of Engish as a >oreign Language!
Basic) (ntermediate and .d1anced
133
(
Trinit" Coege) London
0'oken Engish as a >oreign or 0econd Language! 1, grades
;ritten Engish <(ntermediate=
/ni1ersit" of Cambridge Loca E+aminations 0"ndicate
6reiminar" Engish Test <6ET=
>irst Certificate in Engish <>CE=
.OCe.rtificate of 6roficienc" in.Engish <C6E=
:i'oma of Engish 0tudies <:E0=
0choo Certificate and &CE <91erseas Centres=! Engish
Language
/ni1ersit" of London
- Le1e Engish Language 0"abus B
Books and artices
.itken. $ & 1277 /sing cone 'rocedure as an o1era
anguage 'roficienc" test. TE09L Suarter" 11<1=! 825A7
.itken) $ & 1272 TechniBues for assessing istening
com'rehension in second anguages. .udio5Eisua
Language Journa 17! 178531
.derson) J C 1273 . stud" of the coLe 'rocedure with
nati1e and non5nati1e s'eakers of Engish. /ni1ersit" of
Edinburgh 6h: thesis
.derson) J C and Hughes) . <eds.= 1231 (ssues in
Language Testing. ELT :ocuments (((. British Counci
.en J 6 B and :a1ies) . <eds.= 1277 Testing and
e+'erimenta methods. Edinburgh Course in .''ied
Linguistics 1o ?. 9+ford /ni1ersit" 6ress
.nderson) J 1271 . techniBue for measuring reading
com'rehension and readabiit". Engish Language
Teaching Journa ,8<,=! 17353,
.nderson) J 127A 6s"choinguistic e+'eriments in foreign
anguage testing. /ni1ersit" of Sueensand 6ress.
Beardsmore) H B 127? Testing ora fuenc". (#.?L
1, <?=! 3175,A
Bensoussan) * 1233 :ictionaries and tests of E>L reading
com'rehension. Engish Language Teaching Journa
37 <?=! 3?158
Brown) & 1277 Listening to s'oken Engish. Longman Brumfit)
C J 123? Communicati1e *ethodoog" in Language
Teaching. Cambridge /ni1ersit" 6ress
Bursta) C 12A2 The main stages in the de1eo'ment of
anguage tests. (n 0tern) H H <ed.= Languages and the
%oung 0choo Chid. 9+ford /ni1ersit" 6ress) 12352
Burt) * $ and $i'arsk". C 127, The &ooficon! a re'air
manua for Engish. Newbur" House) #owe")
*assachusetts
Canae) * and 0wain) * 123- Theoretica Bases of 5
Communicati1e .''roaches to 0econd Language Teaching
and Testing. .''ied Linguistics 1! 15?7
- O!1 O). .O... 4 .D
Carro) B J 123- Testing Communicati1e 6erformance.
6ergamon
Chen) Q and Henning) & 1238 Linguistic and cutura bias
in anguage 'roficienc" tests. Language Testing
, <,=! 13-5121
Cark J L : 127, >oreign Language Testing! Theor" and
6ractice. Centre for Curricuum :e1eo'ment)
6hiade'hia. 6enns"1ania
Cark) J L : <ed.= 1273 :irect Testing of 0'eaking
6roficienc"! Theor" and 6ractice. Educationa Testing
0er1ice) 6rinceton
Cohen) . : 123- Testing Language .biit" in the
Cassroom. Newbur" House) #owe") *assachusetts
Crocker) . C 12A2 0tatistics for the Teacher <or How to 6ut
>igures in their 6ace=. 6enguin
:a1ies) . <ed.= 12A3 Language Testing 0"m'osium. 9+ford
/ni1ersit" 6ress
:a1ies) . 1273 Language Testing <0ur1e" .rtices Not. 3
and ?=. (n $insea) E <ed.= Language Teaching and
Linguistics .bstracts 1o. ,! 1,7582. Cambridge /ni1ersit"
6ress
:a1ies) 0 and ;est) # 123? The 6itman &uide to Engish
Language E+aminations ,nd edn. 6itman
:ougas) : 1273 &ain in reading 'roficienc" in Engish as a
>oreign Language measured b" three doLe scoring
methods. Journa of #esearch in #eading
1<1=!A7573O5O
Eis) # 123? Communication strategies and the e1auation
of communicati1e 'erformance.
Engish Language
Teaching Journa 33<1=! 325??
>inocchiaro) * and 0ako) 0 1233 >oreign Language Testing!
. 6ractica .''roach. #egents) New %ork
>ok) .) Lord) #) Low) &) T@sou B $) and Lee)
% 6 1231 ;orking 6a'ers in Linguistics and Language
Teaching. 0'ecia (ssue on Language Testing) No. ?)
Language Centre) /ni1ersit" of Hong $ong
&annon) 6 1238 .ssessing ;riting! 'rinci'es and 'ractice
of marking written Engish. Edward .rnod
&eoghegan) & 1233 Language 'robems of non5nati1e
s'eakers of Engish at Cambridge /ni1ersit".
Be
Educationa Trust) Cambridge
&reen) J . 1278
Teacher5*ade Tests ,nd edn. Har'er and
#ow) New %ork
Hae) & .) 0tansfied) C ;) and :uran)
# 6 123? 0ummaries of studies in1o1ing the Test of
Engish as a >oreign Language)
12A35123,. Educationa
Testing 0er1ice) 6rinceton) New Jerse"
Hanania) E and 0hikhani) * 123A (nterreationshi's .mong
Three Tests of Language 6roficienc"! 0tandardiLed E0L)
CoLe) and ;riting.
TE09L Suarter"
,-<1=! 2751-2
Harris) : 6 12A2
Testing Engish as a 0econd Language.
*c&raw5Hi) New %ork
Harrison) . 1233 . Language Testing Handbook.
*acmian
Heaton) J B <ed.= 123, Language Testing. *odern Engish
6ubications
Hendrickson. 3 <ed.= Error .na"sis and Error Correction in
Language Teaching. #ELC 9ccasiona 6a'ers
1-)
0E.*E9 #egiona Language Centre) 0inga'ore
5 5
Henning) & H et. a. 1231 Com'rehensi1e .ssessment of
Language 6roficienc" and .chie1ement .mong Learners
of Engish as a >oreign Language.
TE09L Suarter"
18 <?=! ?875AA 55
Hughes) . and Lascaratou) C 123, Com'eting criteria for
error gra1it". Engish Language
Teaching Journa
A3 <3=! 17853,
Hughes) . and 6orter) : <eds.= 1233 Current :e1eo'ments
in Language Testing. .cademic 6ress
Hughes) . and 6orter) : <eds.= Language Testing.
<Journa
'ubished b" Edward .rnod in June and :ecember each
"ear.=
(be) * : 1278 . com'arison of doLe and muti'e5choice
tests for measuring the Engish reading com'rehension of
southeast .sian teachers of Engish.
#ELC Journa
A <,=! ,?53,. 0E.*E9 #egiona Language Centre)
0inga'ore
Jones) # L and 0'osk") B <eds.= 1278 Testing Language
6roficienc". Center for .''ied Linguistics) .rington)
Eirginia
Lado) # 12A1) 12A? Language Testing! the Construction and
/se of >oreign Language Tests. Longman
Lee) % 6 and Low) & : 1231 Cassif"ing tests of anguage
use. 6a'er 'resented at Ath .(L. ;ord Congress) Lund)
0weden
Lee) % 6) >ok) C % %) Lord) #) and Low. & 123, New
:irections in Language Testing. 6ergamon. Hong $ong
Lukmani) % 1!3, The communicationa testing of reading.
Engish Language Teaching Journa
3A<?=! ,175,8
*oer) . 1278 Eaidit" in 6roficienc" Testing) ELT
:ocuments 3!
8513. British Counci
*orrow) $ E 1277 TechniBues of E1auation for a Notiona
0"abus. Centre for .''ied Language 0tudies) /ni1ersit"
of #eading <for the #o"a 0ociet" of .rts=
*orrow) $ E 1272 Communicati1e Language Testing!
re1oution or e1oution. (n Brumfit) C J and Johnson) $ J
<eds.= The Communicati1e .''roach to Language
Teaching. 9+ford /ni1ersit" 6ress
*unb") J L 1273 Communicati1e 0"abus :esign.
Cambridge /ni1ersit" 6ress
9ier) J ; 127, CoLe tests of second anguage 'roficienc"
and what the" measure. Language Learning
,3<1=! 1-8513
9ier) J ; 1272
Language Tests at 0choo. Longman
9((erT15E L5and 6erkins) $ 1273 Language in Education!
testing the tests. Newbur" House) #owe") *assachusetts
9ier) J ; and 0treiff) E 1278 :ictation! a test of grammar5
based e+'ectancies. Engish Language Teaching Journa
3- <1=! ,853A
6amer) . 0 1231 Testing communication. (#.L 1-! 385?8
6amer) . 0 1231 *easures of achie1ement)
communication) incor'oration) and integration for two
casses of forma E>L earners. #ELCJourna
1, <1=! 375A1
6amer) L and 0'oisk") B <eds.= 1278.6a'ers on Language
Testing) 12A75127?. TE09L) ;ashington) :. C. 6erkins)
$ 123- /sing 9bJecti1e *ethods of .ttained
;riting 6roficienc" to :iscriminate .mong Hoistic
E1auations. TE09L Suarter" 1?<1=! A15A2
132
(( (
6erren) & E <ed.=
1277 >oreign Language Testing!
s'eciaised bibiogra'h". Centre for (nformation on
Language Teaching and #esearch
6orta) * <ed.= 123A (nno1ations in Language Testing.
N>E#5Neson
#ea) 6 * 1273 .ssessing anguage as communication! *.L0
Journa. New series) No. 3) :e'artment of Engish)
/ni1ersit" of Birmingham
#ead) J . 0 <ed.= 1231
:irections in Language Testing)
#ELC .nthoog") 0eries 2. 0E.*E9 #egiona Language
Centre) 0inga'ore
#ichards) J C 1238
The Contest of Language Teaching.
Cambridge /ni1ersit" 6ress
#i1era) C 123? Communicati1e Com'etence .''roaches to
Language 6roficienc" .ssessment! #esearch and
.''ication. *utiingua *atters) Ce1edon O
#i1ers) ; * 12A3 Teaching foreign5anguage skis.
/ni1ersit" of Chicago 6ress
#i1ers) ; * and Tem'ere") * 0 1273 . 6ractica &uide to
the Teaching of Engish as a 0econd or >oreign Language.
9+ford /ni1ersit" 6ress) New %ork
0chuL) # .
1277 :iscrete56oint 1ersus 0imuated
Communication Testing in >oreign Languages. *odern
Language Journa A1<3=! 2151-1
0immonds) 6 1238 . sur1e" of Engish anguage
e+aminations. Engish Language Teaching Journa)
32<1=! 335?,
0'oisk") B
1238 ;hat does it mean to know how to use a
anguageG .n essa" on the theoretica basis of anguage
testing. Language Testing ,<,=! 13-5121
0'osk") B with *ur'h") 6) Hom) ;) and >erret)
. 127, Three >unctiona Tests of 9ra 6roficienc".
TE09L Suarter" A<3@! ,,1538
0tubbs) J B and Tucker) & # 127? The CoLe Test as a
*easure of Engish 6roficienc". *odern Language Journa
83
<8KA=! ,325?1
Tomi"ana) * 123- &rammatica errors and communication
breakdown. TE09L Suarter" 1?<1=! 7152
/'shur) J . 1271
9bJecti1e e1auation of ora 'roficienc" in
the E09L cassroom. TE09L Suarter" 8! ?75A-
/'shur) J . and >ata) J 12A3
6robems in foreign anguage
testing. Language Learning 0'ecia (ssue) No. 3
Eaette) # *
1277 *odern Language Testing ,nd edn.
Harcourt Brace Jo1ano1ich) New %ork
Eaette) # * and :isick) # 0
127, *odern Language
6erformance 9bJecti1es and (ndi1iduaisation. Harcourt
Brace Jo1ano1ich) New %ork
12-
(nde+
.chie1ement tests 17,
.ddition items 8-
.dministration 1A751A3
.na"tic marking 1?3
.'titude items 173
.ttainment tests 171
Backwash 17-5171
Banding
5 com'ositions 1?851?A
5 ora inter1iews 2351--
Broken sentence items ?2
Centra tendenc" 178517A
Chan g in g words ?35?2
18
Error recognition items 325?-) 18,
Error5count marking 1?351?2
Errors 1?2518-
Essa"5transation a''roach 18
E1auation 7
E+tended answer ana"sis 13,5138
E+tensi1e reading 1-A
>ace 1aidit" 18251A-
>aciit" 1aue 1735172
>ragmented sentences 188
>reBuenc" distribution 17?5178
&rading com'ositions 1??51?2
&rammar 2) 3?58-
5 error5count method 1?351?2
5 im'ression method 1?751?3 5
mechanica accurac"
method 1?351?2
5 treatment of errors 1?2518-
*atchin g items
5 reading 1-75113
5 1ocabuar" 835A-
*ean 178517A
*echanica accurac" marking 1?351?2
*edian 178
*istakes 7) 1?2518-
*ode 178
*ode 'aragra'hs 18?
*oderating 13A5137
*oti1ation 7
Cassification of tests
Cassroom tests A
CoLe 'rocedure 1A517) 1315133
Combination items 8-
Communicati1e a''roach 125,?
Com'etion items
5g rammar ?,5?A
5 reading com'rehension 1,?51,2 5
sentences) te+ts 18A5187
5 s'eing 181518,
5 1ocabuar" A,5A3
Com'osition
5 genera 13A518-
5 setting 13351?3
5 tites 1375133
Concurrent 1aidit" 1A151A,
Connecti1es 18A
Construct 1aidit" 1A1
Content 1aidit" 1A-51A1
Controed writing 18?5183
Con1ersations 2-52,
Cursor" reading 133513?
:efinitions A,
:iagnostic testing A) 173
:iaogues A2571) 2-52,@
:ictation 17513) 181
:ifficut" inde+ 1735172
:iscrimination 1A8) 172513,
:is'ersion 17A5173
Em'irica 1aidit" 1A151A,
Error) test margin of 1AA
&rou' discussion 1-,51-?
(m'ression marking 1?751?3
(m'ure items ,2
(nde+ of difficut" 1735172
(nstructions 1A3517-
(ntegrati1e a''roach 1A
(ntensi1e reading 1-A
(nter'retation of scores 17?5133
(ntonation A35A2
(tems
5 ana"sis 1735138
5 cards and banks 1385137
5 difficut" 1735172
5 discrimination 172513,
<>or t"'es of items) see indi1idua
entries) e.g. add t i ion) combination)
muti'e5choice=
Judgement skis 138
Language areas 2
Language eements 1-511
Language skis 3) 1-511
Lectures 3,537
Length of te+ts 113
Linking sentences 18
Listening com'rehension A?537
*arkKre5mark reiabiit" 1A,
*arking
5 ana"tic method 1?3
5 com'ositions 1??51?2
*uti'e marking 1?751?3
*uti'e5choice items
5 construction ,75?-
5 correct o'tion 37533
5 distractors.3,533
5 error recognition 325?-
5 grammar 2
5 istening AA571
5 reading 11A51,?
5 s'eing 181
5stem 3A537
5 1ocabuar" 8,5,3
5 writing 18,5183
9bJecti1e testing ,85,7
9'en5ended items 133
9ra 'roduction 3351-?
9ra inter1iew 2A51-,
6airing and matching items ?258-
6arae test forms 1A3
6erfomn race e1es ,15,3 555555
6honeme discrimination A85A3
6honoog" 2
6ictures
5 istening com'rehension 7153, 5
matching 11-) 11,
5 s'eaking 2,52A
5 writing 1?,51?3
6racticabiit" 1A751A3
6redicti1e 1aidit" 1A151A,
6robem so1ing 1-,51-?
6roduction 11
121
i
6roficienc" tests 17,5173
6rofie re'orting 125,3) 1 A3
6rogress tests 171517,
6s"chometric a''roach 1A
6unctuation 138) 18-5181
Suaitati1e Judgements ,-5,,
#ange 17A
#ating scae
5 com'ositions 1?851?A
5 ora inter1iews 2351--
#eading
5 e+tracts <for writing= 188518A
5reading aoud5372-55 5
5 skis 1-851-A
5 test s'ecifications ,,5,3
#earrangement items
5 grammar ?15?,
5 reading 1,25131
5 1ocabuar" A1
#ecognition ((
#edundanc" A?
#egister 183518?
#eiabiit" 1,) 1A,51A8
5 istening com'rehension A8
5 markKre5mark 1A,
5 'arae test forms 1A3
5 'rofie re'orting 1A3
5 s'it haf5method 1A351A?
5 testKre5test 1A,
12,
51 1aidit" 1A?51A8
#oe 'a"ing 1-,51-?
#ubrics 1 A3517-
0am'ing 1,) 8158,) 113
0canning 133513?.
0cores 17?5133
0coring
5 ora inter1iews 2351--5
0entence matching 1-7511-
0ets 83
0kimming 133513?
0'eaking ,15,3) 3351-?
0'ecifications 13
0'eing 138. 181518,
0'it5haf method 1A351A?
0'oken anguage A?5A8
0'read of scores 1AA51A7
0tandard de1iation 17A5173
0tandards 7
0tatements A2571
0tress A35A2
0tructuraist a''roach 18
0t"e 18,5183
0ubJecti1e testing ,85,A
0"non"ms A1
Taks 3,537) 1-,
i
Teaching 8)17-517(
Test t"'es 1-A51-7) 1715173
TestKre5test reiabiit" 1A,
Transformation items ?A5?3
Transation 13512
Tra's 1?
TrueKfase tests 113511A
/sae 3?58- g
Eaidit" 18251A,
5concurrent 1A1
5 construct 1A1
5 content 1A-51A1
5 em'irica 1A151A,
5 face 18251A-
5 'redicti1e 1A1
5 1 reiabiit" 1A?51A8
Eisuas
5 istening com'rehension 7153, 5
matching 11-) 11,
5 s'eaking 2,52A
5 writing 1?,51?3
Eocabuar" 2 ) 815A3
;ord formation A1
;ord matching 1-7
;riting 1385183
5 e1es 13A
5 tasks 13A