Assessing Speaking Skills 1

Practical issues in task design

In assessing oral skills, we can use different test items according the relevant purposes
or target information to be tested. Luoma (2004) provides three frameworks about
general purposes of testing oral skills as “linguistically oriented, communication-
oriented and situation-based.”
Linguistically oriented: vocabulary, grammar and pronunciation.
This purpose can be carried out by structured speaking tasks. They are called
‘structured’ because they control quite closely what the examinees are going to say. In
reading aloud and sentence repetition, the testers know exactly what the examinee will
say, and, in short-answer questions and reacting to phrases tasks, a short list of
acceptable answers can usually be specified. Structured speaking tasks are typically
used to evaluate linguistic features, particularly pronunciation and grammar. Short-
answer questions and reactions to phrases can also be used to evaluate overall
understanding and comprehensibility. Short-answer questions test comprehension of
the questions and ability to give relevant information in response. The difference
between this and more extended simulations is that the questions and answers are
limited and all the information needed for answering the questions is usually provided
in the task materials. Reacting to phrases is another structured task that is often used in
tape-based tests. The task usually tests the examinees’ knowledge of conventional
politeness exchanges such as greetings, thanks, apologies, expressions of agreement and
polite disagreement, and so on.
Communication-oriented: the overall communication activity in the task such as telling
a narrative or expressing and defending an opinion, discussing factors that support the
chosen opinion and argue against others, comparing and contracting things through
which advanced oral skills for description is needed.
Situation-based task design: this belongs to the task-based approach to defining the test
construct. This approach is typically used in specific-purpose testing and in vocational
and professional education.
What we find significant for the early stage of Chinese language learning are the first
two, namely we evaluate how students understand linguistic rules and use them in
communication.
Here are some examples in Luoma (2004, pp. 139-169) that show the types of test items
and their functions:
Description tasks
Example 1—A one-to-one interview: Describe to me the room or area where you work.
Example 2—A pair task in an interview test (two pictures are provided):
Describe your pictures to each other and then talk about what is similar in your pictures
and what is different.
Narrative tasks
Example 3: A tape-based test (Six pictures should be provided)
Please look at the six pictures below. Tell the story based on these pictures starting from
picture number 1 and going through picture number 6. Take one minute to look at the
pictures.
The narrative is a monologue, and as the test is tape-based the examinees have to tell it
in one long stretch without any feedback from a listener.
Example 4: A face-to-face paired interaction test
You each have a set of pictures. Together they make a story. Each of you tells one part
of the story.
Narrative tasks are also frequently used in speaking tests. They show how well the
examinees can recount a sequence of events, usually in one time frame, either present or
past. Most often, the tasks are based on picture sequences, where the content of the
pictures guides what will be said. It is likely to create some interaction even though one
of the pupils is always the main speaker while the narrative is being constructed.
Information gap between the two speakers is therefore likely to occur. Also, the choice
of good sequences is a difficult matter.
Personal stories often reveal embarrassing details that speakers would be shy to discuss
in a test or, if not, they may be so uneventful that the speakers would consider them
unworthy to tell.
Instruction tasks
Example 5: A one-to-one interview
Imagine that we are standing in front of your house. Tell me how to get to the shop from
there.
Example 6: A face-to-face paired interaction test—Feeding the puppy
You cannot go home and your puppy needs to be fed. Your friend says he will do it.
Tell your partner exactly what to do, what he’ll need and where to find things. Follow
the instructions below. Fine what you need in the picture. Tasks—what you have to do;
what you need; where to find things.
The main purpose in giving directions and instructions is getting the message across the
making sure that it has been understood. This tends to mean short exchanges between
the speaker and the listener.
Comparing and contrasting tasks
Example 7: Interaction outline for a pair task in a paired interview
Candidate A compare and contrast two or three of these photographs, saying what kind
of clothing the people are wearing and why the protection might be necessary.
Explaining and predicting tasks
Example 8: A taped-based test
Explaining the contents of a graph or explaining a process is a fairly common task in
many professional and study settings. To do well on the task, the speakers need to set
the scene and identify parts of the information or stages in the process that they are
explaining and present them in coherent order. They also need to explain the
significance of the important parts or stages, so that the listeners understand what the
explanation is about and why it is the being given. Predictions go together well with
explanation tasks, and they can also be fairly monologic. As predictions involve
speculation, they may become more interactive in a face-to-face setting.
Decision tasks
Example 9: in a paired interview
Discuss and decide together: 1) what the advantages and disadvantages are of attending
trade fairs, for instance, and 2) which members of staff would most usefully represent a
company at a trade fair.
In speaking tests, the issues that need to be decided are usually not clear-cut, so that
arguments for and against different solutions are needed. The speakers express their
opinions about the concerns and justify them in order to air different viewpoints before
negotiating the conclusion.
Role-plays and simulations
Role-plays simulate different kinds of communication situations that the target group of
the test could plausibly meet outside the test. They can be completed between two
examinees or between an examinee and a tester.
Example 10: A job interview
The employer—inquiring information about candidate’s abilities, qualifications, and
character
The candidate—providing information about himself and inquiring information about
the company
Role-play tasks are a way of making communication in a test more versatile because,
rather than talking to a tester, the examinees take on a new role and a new, simulated
role relationship to their communication partner. Their performance shows their ability
to adapt to the requirements of the new role and situation. As long as the situation is
relevant for the target audience and the purpose of the test, this gives useful information
for the tester. The information may simply be a new perspective into the examinees’
linguistic resources, or the use of different functions from other tasks in the test.