Distractor Analysis

Distractor analysis is an extension of item analysis, using techniques that are similar to item
difficulty and item discrimination. In distractor analysis, however, we are no longer interested
in how test takers select the correct answer, but how the distracters were able to function
effectively by drawing the test takers away from the correct answer. The number of times
each distractor is selected is noted in order to determine the effectiveness of the distractor.
We would expect that the distractor is selected by enough candidates for it to be a viable
distractor. What exactly is an acceptable value? This depends to a large extent on the
difficulty of the item itself and what we consider to be an acceptable item difficulty value for
test times. If we are to assume that 0.7 is an appropriate item difficulty value, then we should
expect that the remaining 0.3 be about evenly distributed among the distractors. Let us take
the following test item as an example:
In the story, he was unhappy because
A. it rained all day
B. he was scolded
C. he hurt himself
D. the weather was hot
Let us assume that 100 students took the test. If we assume that A is the answer and the
item difficulty is 0.7, then 70 students answered correctly. What about the remaining 30
students and the effectiveness of the three distractors? If all 30 selected D, the distractors B
and C are useless in their role as distractors. Similarly, if 15 students selected D and another
15 selected B,

then C is not an effective distractor and should be replaced. The ideal

situation would be for each of the three distractors to be selected by 10 students. Therefore,
for an item which has an item difficulty of 0.7, the ideal effectiveness of each distractor can
be quantified as 10/100 or 0.1. What would be the ideal value for distractors in a four option
multiple choice item when the item difficulty of the item is 0.4? Hint: You need to identify the
proportion of students who did not select the correct option.
From a different perspective, the item discrimination formula can also be used in distractor
analysis. The concept of upper groups and lower groups would still remain, but the analysis
and expectation would differ slightly from the regular item discrimination that we have looked
at earlier. Instead of expecting a positive value, we should logically expect a negative value

as more students from the lower group should select distracters. Each distractor can have its
own item discrimination value in order to analyse how the distracters work and ultimately
refine the effectiveness of the test item itself. If we use the above item as an example, the
item discrimination concept can be used to assess the effectiveness of each distractor. If a
class has 100 students, we can form upper and lower groups of 30 students each. Assume
the following are observed:
The values in the last column of the table can once again be interpreted according to how
we examined item discrimination values, but with a twist. Alternative A is the key and a
positive value is the value that we would want. However, the value of 0.33 is rather low
considering the maximum value is 1. The value for distractor B is 0 and this tells us that the
distractor did not discriminate between the proficient students in the upper group and the
weaker students in the lower group. Hence, the effectiveness of this distractor is
questionable. Distractor C, on the other hand, seems to have functioned effectively. More
students in the lower group than in the upper group selected this distractor. As our intention
in distractor analysis is to identify distractors that would seem to be the correct answer to
weaker students, then distractor C seems to have done its job. The same cannot be said of
the final distractor. In fact, the positive value obtained here indicates that more of the
proficient students selected this distractor. We should understand by now that this is not
what we would hope for.
Distractor analysis can be a useful tool in evaluating the effectiveness of our distractors. It is
important for us to be mindful of the distractors that we use in a multiple choice format test
as when distractors are not effective, they are virtually useless. As a result, there is a greater
possibility that students will be able to select the correct answer by guessing as the options
have been reduced.

The values in the last column of the table can once again be interpreted according to how
we examined item discrimination values, but with a twist. Alternative A is the key and a
positive value is the value that we would want. However, the value of 0.33 is rather low
considering the maximum value is 1. The value for distractor B is 0 and this tells us that the
distractor did not discriminate between the proficient students in the upper group and the
weaker students in the lower group. Hence, the effectiveness of this distractor is
questionable. Distractor C, on the other hand, seems to have functioned effectively. More
students in the lower group than in the upper group selected this distractor. As our intention
in distractor analysis is to identify distractors that would seem to be the correct answer to
weaker students, then distractor C seems to have done its job. The same cannot be said of
the final distractor. In fact, the positive value obtained here indicates that more of the
proficient students selected this distractor. We should understand by now that this is not
what we would hope for.
Distractor analysis can be a useful tool in evaluating the effectiveness of our distractors. It is
important for us to be mindful of the distractors that we use in a multiple choice format test
as when distractors are not effective, they are virtually useless. As a result, there is a greater
possibility that students will be able to select the correct answer by guessing as the options
have been reduced.