Testing

Vocabulary
Jonathan Magdalena

What are we trying to MEASURE?

A word is
of

a microcosm
human consciousness
(Vygotsky)

What is vocabulary?
Language emerges first as words, both historically, and in terms of the way each
of us learned our first and any subsequent languages. The coining of new words
never stops. Nor does the acquisition of words. Even in our first language, we are
continually learning new words, and learning new meanings for old words.
Can you think of an example?

How many words can you find?

I like looking for bits and pieces like old second-hand record players and doing them up to look like new.

and is repeated once; like three times

The first like is a verb; the other two are prepositions
looking and look:
look: do they count as two different words or different forms of the same word?
second-hand:
second-hand: two words or just one?
What about record player?
player?
And bits and pieces?
pieces? Is this a self-contained unit or three separate words?
Are looking and looking for the same?
same?
Doing up means renovate,
renovate, but it is separated by the word them;
them; is it one or two words then?

What is a word then?

A word is a highly complex phenomenon.

Words have different functions.
The same word can have a variety of forms.
Words can be added to, or combined, to form new words.
Words can group together to form units that behave as if they were single words.
Many words commonly co-occur with other words.
Words may look and/or sound the same but have quite different meanings.
One word may have a variety of overlapping meanings.
Different words may share similar meanings, or may have opposite meanings.
Words can have the same or similar meanings but be used in different situations or for different effects

What does know a word mean?

Knowing a word means:

having the ability to recognize it in its spoken and written forms
knowing its different meanings.
knowing its part of speech [eg. a noun, a verb]
being able to pronounce it properly
being able to use it correctly within a sentence in an appropriate grammatical form
for technical words, recognizing it in context
being able to recognize different types of English e.g boot/trunk, lift/elevator [British/American].

How are words learned?

Without grammar very little can be conveyed, without vocabulary
nothing can be conveyed.

What makes you swerve your car?

The limo surges forward and starts to swerve wildly over the
road.
The bus driver swerved to avoid hitting the cyclists
She is one of those rare politicians whom one can trust not to
swerve from policy and principle.
Hitting the brakes would make the bikes swerve more.
The driver made no attempt to swerve out of their path
Nothing could swerve him
Last and surprisingly, for such a small town you can still get your
swerve on with the nightlife.
He managed to pass the ball with a perfect body swerve.

How are words organized?

The mind seems to store words neither randomly or in the form of a
list, but in a highly organized and interconnected fashion in what is
often called the mental lexicon.

Why should we test vocabulary?

Feedback

Backwash effect

Motivation

Recycling

How should we test vocabulary?

Separated from skills?

Discrete point or integrative items?

Focus on lexis or contextualized meaning?

What are the most common test formats for

testing vocabulary?

Recognition
Multiple choice items (Liu, 121)
Error-recognition items (Liu, 129)
Pairing and matching items
Gap-filling items
Cloze/Modified Cloze (Hughes, 144)
Guessing meaning from context

When should we go for vocabulary

recognition?
When
More material needs to be covered.
You want to test different levels of
learning.
You have little time for scoring.
You are not interested in evaluating how
well a test taker can formulate a correct
answer.
You have a large number of test takers.

What are the most common test formats for

testing vocabulary?

Production
Completion items (Hughes, 143)
Transformation items (Liu, 131)
Paraphrase (Hughes, 143)
Table completion
Combination and addition items
Items involving the changing of words

When should we go for vocabulary

production?
When
You want to evaluate a persons ability
to formulate a correct answer.
You have more time to score the items
You want to test a persons ability to
apply concepts and information to a new
situation.
You have a clear idea of the aspects and
concepts that should be tested.

Test
Formats

Multiple Choice
The flight attendant asked the passengers to ______
attention to the safety demonstration.
a. give
b. devote
c. pay
d. lend

They can be tricky or too picky

Difficult to test attitudes towards learning
Knowledge is limited to options provided
Difficult to construct at higher levels
Encourages guessing (25% chance)
More than one option may be possible
All options must be grammatically possible

Matching Word-Definition
Tangle means
a. A type of dance
b. A tropical forest
c. A confused mass
d. A kind of fruit

They are not communicative

Difficult to test attitudes towards learning
Words have diverse connotations
Encourages guessing (25% chance)
One word class at a time

Single Selection
Someone was (playing/singing) the tune and for
a moment you were happy to go (along/away)
with what seemed a reasonable idea.

They are authentic and communicative

Words are part of a broader context
Knowledge is limited to options provided
Encourages guessing (50%)
Difficult to design but easy to mark

Correction/Editing
Maria bought a skarf for the winter

Tests recognition of mistakes

Is focused on word spelling only
Mistakes are ungrammatical
Negative backwash
Context is important but not essential

Guessing Meaning from Context

Unlike the guitar, the fiddle has four strings on a
fingerboard without frets.

Tests contextualized knowledge

Context is crucial
Communicative and authentic
Positive backwash
Difficult for some students (bias)
Tests various types of word relationships

Gap-Filling
The flight attendant asked the passengers to
______ attention to the safety demonstration.

They must have a broader context

More than one option may be possible
(unless tester provides limited options or
first letter)
Tests can focus on content words
Production is tested unless options are
provided
Focus should be on the aspect assessed

Cloze
John ______ works ______Monday ______Friday

Tests students understanding of the whole

language
Many word types can be tested
Understanding of context is crucial
Difficult to test attitudes towards learning
Some words may be impossible to guess
Can test students IQ rather than language
proficiency
Difficult for some students (bias)

Table Completion
Complete the table with positive, negative and
neutral adjectives

Tests language structure

Tests word-building
Many word types can be tested
Not communicative at all
Tests use but not language usage
Difficult to test attitudes towards learning

Word Transformation
Turn verbs into nouns:
1. The opera was conducted by the (compose)____

Not very communicative (without context)

Clues will limit the answers
Difficult to test attitudes towards learning
Some context may be added
Instructions are essential
Few aspects might be assessed (word
classes, synonyms, antonyms)

Five characteristics to measure

communicative vocabulary
1.
2.
3.
4.
5.
6.

The test must provide more context than only

a single sentence.
The items must assess lexis within a limited
number of semantic fields.
The test taker should understand what the
communicative purpose of the task is.
He or she should also know who the intended
audience is.
He or she must have to focus on meaning and
not form to answer correctly.
Recognize is not sufficient. The test taker
must be able to produce grammatical
responses.
(R. Dickins)

In Conclusion

Evidence shows that semantically related

items are 'stored together'. Items are
arranged in a series of associative networks, a
large 'master file', and a variety of 'peripheral
access files' which contain information about
spelling, phonology, syntax and meaning.
Knowing a word, then, is the sum total of all
these
connections

semantic,
syntactic,
phonological,
orthographic,
morphological,
cognitive, cultural and autobiographical. It is
unlikely, therefore, that any two speakers will
'know' a word in exactly the same way.