EVALUATION

Test Item Construction

Structured / Short-Answer Items

Sylvester Saimon Simin

Keningau Teachers Training College
 TYPES OF ITEMS

SELECTION-TYPE ITEM SUPPLY-TYPE ITEM

(OBJECTIVE) (SUBJECTIVE)

1. MULTIPLE CHOICE 1. SHORT ANSWER

2. TRUE-FALSE (STRUCTURED)
3. MATCHING 3. ESSAY (RESTRICTED
RESPONSE)
4. INTERPRETIVE
EXERCISE 4. ESSAY (EXTENDED
RESPONSE)
 Subjective Questions

• Subjective questions are questions that require students

to respond in a written or oral manner. The response can
be either opened or limited, in one word, one sentence
or even a few paragraphs. The correct answer can only
be determined by experts in that particular field. This is
also known as the free response format.
 Structured Question
• A structured question format consists of a few questions
arranged and organized systematically to be developed
into a bigger problem all related to a theme. Mehrens &
Lehmann (1991) term this type question as the short-
answer item.
• Macintosh and Frith (1987) explain the structured
question in three different contexts.
1. Item format- questions is divided into simpler and
less complex form.
2. Stimulus for the question – the stimulus used eg.
documents, drawings, articles and diagrams are
simplified and edited
3. The task is usually presented in the form of problem
or a dilemma and not in a question form like
“discuss” or ”explain”.
 Purpose of constructing structured questions

• There is a big gap in between objective and essay

questions. The structured question can bridge this gap
as it provides the student with an outline or guideline to
respond
• This type of question can connect short and simple
questions into sets according to a theme
 Limitation of structured questions

• Questions can be too long because it consists of a

lot of words to explain the whole theme.
• They are best for measuring highly specific facts and
may encourage rote learning. This is because
question of the synthesis and evaluation may not be
restricted to one or two word answers.
• Marking the answers may not be objective as many
alternative answers may be accepted.
 Classification of structured questions
• The classification is usually based on the type of
stimulus used. The most frequently used variations
are:
– Factual type
• The questions are constructed based on the facts in a given
passage.
– Interpretive type
• The stimulus is in the form of a passage, chart, graph, diagram,
or map
 Guidelines for writing good structured
questions
• A good structured question should:
– Consists of both the objective and essay elements.
– Generally written in the essay form, that is quite
open but the responses required are short.
– Questions are of ascending difficulty i.e. from the
easiest/simplest to the hardest/most complex. The
degree of complexity is based upon the length of
response required.
– Questions are relevant and should interact with the
stimulus provided.
– All questions in one section are independent. The
student’s ability to answer one question does not
depend on his ability to answer other questions.
 WHAT ARE SHORT-ANSWER ITEMS?
• The short-answer (or completion) item requires the examinee
to supply the appropriate words, numbers, or symbols to
answer a question or complete a statement
– Eg: What are the incorrect responses in a multiple-choice item
called?
• 2 major problems in construction of short-answer items:
– Difficult to phrase the question or incomplete statement so that
only one answer is correct
• (eg: ‘incorrect alternatives’, ‘wrong answers’, ‘inappropriate options’,
‘decoys’, ‘foils’ etc)
– The problem of spelling.
• If marks are give only to answers spelled correctly, the poor spellers
will be prevented from showing their true level of acheivement. Test
scores became an uninterpretable mixture of knowledge and spelling
skill. But if spelling is ignored, is the badly spelled word represent the
intended answer?. This introduces an element of subjectivity that tends
to make the score less dependable as a measure of acheivement.
 SHORT-ANSWER ITEMS
Strengths
1. Easy to write test items
2. Guessing as less likely than
in selection-type items.
3. Well-suited to computational
problems and other learning
outcomes where supplying
the answer is important.
4. A broad range of knowledge
outcomes can be measured.
Limitations
1. It is difficult to phrase
statements so that only one
answer is correct.
2. Scoring is contaminated by
spelling ability when
responses are verbal.
3. Scoring is tedious and time
consuming.
4. Not very adaptable to
measuring complex learning
outcomes.
RULES FOR WRITING SHORT-ANSWER ITEMS
1. State the stem so that only a single, brief answer is
possible.
RULES FOR WRITING SHORT-ANSWER ITEMS
2. Start with a direct question and switch to an incomplete
statement only when greater conciseness is possible
by doing so.
 RULES FOR WRITING SHORT-ANSWER ITEMS

3. It is best to leave only one blank and it should relate to

the main point of the statement.
 RULES FOR WRITING SHORT-ANSWER ITEMS

4. Place the blanks at the end of the statement.

 RULES FOR WRITING SHORT-ANSWER ITEMS

5. Avoid extraneous clues to the answer.

 RULES FOR WRITING SHORT-ANSWER ITEMS

6. For numerical answers, indicate the degree of precision

expected and the units in which they are to be
expressed.
 Checklist for Evaluating Short-Answer Items

1. Is this type of item appropriate for measuring the

intended learning outcome?
2. Does the item task match the learning task to be
measured?
3. Does the item call for a single, brief answer?
4. Has the item been written as a direct question or a
well-stated incomplete sentence?
5. Does the desired response relate to the main point of
the item?
6. Is the blank placed at the end of the statement?
7. Has clues to the answer been avoided (eg: ‘a’, ‘an’,
length of the blank)?
8. Are the units and degree of precision indicated for
numerical answers?