The design of discrete vocabulary tests

Review various considerations that influence the design of discrete

vocabulary tests
These are tests that measure vocabulary as a construct separate from
other aspect of language ability
Discrete tests focus on vocabulary knowledge: whether the test-takers
know the meaning or use of a selected set of content words in the target
language

Test purpose
-

Is to define the purpose of the test a test is valid to the extent that we
are justified in drawing conclusions from its results
Three uses of language tests: for research, making decisions about
learners and making decisions about language programmes
Second language vocabulary researchers have needed assessment
instruments for studies on: - how broad and deep learners vocabulary
knowledge is
o How effective different methods of systematic vocabulary learning
are
o How incidental learning occurs through reading and listening
activities
o Whether and how learners can infer the meanings of unknown
words encountered in context
o How learners deal with gaps in their vocabulary knowledge
Assessing proficiency: placement test battery to estimate how many
high-frequency words the learners already know
o A progress test: assesses how well students have learned the
words presented in the units they have recently studied in the
coursebook
o Achievement test: how well the learners have mastered a
vocabulary skill that they have been taught

Construct definition
-

Another important foundation for test validity is the defining of the

construct we are intending to measure
2 approaches: syllabus-based and theory based
Syllabus-based: when vocabulary assessment takes place within a course
of study
Theory-based: for research purposes and in proficiency testing
o Issues such as what we mean by a word whether vocabulary
includes multi-word lexical items, what it means to know a word
and how words are influenced by context

Receptive and productive vocabulary

The number of words we can recognise and understand is rather larger

than the number we use in our own speech and writing (passive active)
The difficulty at the conceptual level is to find criteria for distinguishing
words that have receptive status from those which are part of a persons
productive vocabulary
It is generally assumed that words are known receptively first and only
later become available for productive use
The problem is to locate the threshold at which the word passes from
receptive to productive status

Recognition and recall

Recognition: test-takers are presented with the target word and are asked to
show that they understand its meaning
Recall: they are provided with some stimulus designed to elicit the target word
from their memory
Comprehension and use
Comprehension: learners can understand a word when they encounter it in
context while listening or reading
Use: the word occurs in their own speech or writing
His argument is that the terms reception and production are too broad and, in
undertaking a vocabulary assessment project which involves making the
distinction, we first need to define what specific learner ability each one refers to.

Characteristics of the test input

Selection of target words
-

Design a selective vocabulary test: how to choose the target words?

First, it is necessary to make a general point about the frequency
distribution of words in English
High-frequency category (2000 word families) foundation of the
vocabulary knowledge
Low-frequency vocabulary as a whole is of much less value to learners.
The focus of teaching and testing shifts to the question of whether
learners have effective strategies for dealing with these lexical items
Specialised vocabulary: comprises of technical terms that occur relatively
frequently in particular registers of the language
Subtechnical vocabulary: words which occur quite frequently across a
range of registers or topic areas in academic and technical language
These four categories represent different ways of specifying the
vocabulary to be learned and to be assessed, depending on the learners
proficiency level and their particular learner needs.

Presentation of words
-

Present in isolation or in context?

Words in isolation
-

in systematic vocabulary learning, students apply memorising techniques

to sets of target words and their meanings this is effective in providing a
foundation for further development of vocabulary knowledge in the second
language
Vocabulary Levels Test: words are often presented in isolation this allows
the test designer to simplify the test-takers task and cover a large sample
of words in the test
Whether the learners can show an understanding of the words when they
occur without contextual support

Words in context
-

How context dependent particular vocabulary items are?

It is pointless to present words in a context unless the test-takers are
required to engage with the contextual information in some meaningful
way in making their response to the test task
One function of a sentence context in traditional discrete-point items is to
signal the particular meaning or use of a high-frequency target word
On the other hand: the target word is a low-frequency word and the
learners infer its meaning on the basis of contextual clues
In recall-type tests of productive vocabulary knowledge, a sentence
context is one means of eliciting the word (fill in the blanks)

Characteristics of the expected response

Self-report vs. verifiable response
-

For most testing purposes we need verifiable evidence that the test-takers
have the knowledge and skills that are the focus of the assessment.

Monolingual vs. bilingual testing

-

Translation from L2 to L1 assesses receptive knowledge and L1 to L2

translation is the corresponding productive measure
In English-speaking countries teachers of English are typically working
with learners from a variety of language backgrounds, which makes it
impractical for them to use bilingual test items.
L1 is likely to play a greater role in assessing the vocabulary knowledge of
lower proficiency learners than those who are more advanced
Testing of very high-frequency words monolingually is quite a challenge.
For objects and actions that can be clearly depicted, a picture
identification or labelling format can be used
It is unwise to encourage in learners a belief that any word in L2 has
directly synonymous word of phrase in L1.

Practical examples

Matching items: the basic matching task requires learners to make a connection
between target words and their synonyms or definitions. It is a recognition rather
than a recall task, focusing on basic word meaning.
Comprehension items: Completion, or blank filling, items consist of a sentence
from which the target word has been deleted and replaced by a blank. The
function of the sentence is to provide a context for the word and perhaps to cue
a particular use of it.
Sentence-writing items: writing a sentence for each word allow the learners to
demonstrate several aspects of their vocabulary ability
Generic test items: coding the words with symbols. For example, words of more
than one syllable which may cause pronunciation problems are marked with +
and the learners respond to a specific task in the test.
Testing depth of vocabulary knowledge
The word-associates test: it was impractical to attempt to elicit the full range of
the learners vocabulary knowledge, instead, he opted for the more modest goal
of assessing how well they knew various meaning of the target words, using a
less open test format.
Word association: presenting subjects with a set of stimulus words one by one
and asking them to say the first related word that comes into their head.
He developed a word association task based on selection, rather than supplying
students own ideas
3 types of relationship between target word and associate: 1. Paradigmatic (the
two words are synonyms) 2. Syntagmatic: (two words often appear together in a
phrase) 3. Analytic (the associate represents one aspect, or component, of the
target word and is likely to form part of its dictionary definition)
While the word-associates tests were good measures of learners overall
knowledge of the University Word List, the individual item scores were not very
satisfactory measures of depth of knowledge of each target word, because of the
guessing factor.
Conclusion
Different variables need to be weighed up in order to reach the most suitable
design for a discrete vocabulary test.