THE EFFECT OF SCORING METHODS ON THE STABILITY OF RELIABILITY

COEFFICIENTS OF THE MULTIPLE-CHOICE TYPE TEST SCORES BASED ON THE

VARIABILITY OF EXAMINEES’ INTELLIGENCES

WIDIATMOKO

ABSTRACT
The objective of the research is to study the effect of scoring methods on the stability of
reliability coefficients of the multiple-choice type test scores based on the variability of
examinees’ intelligences. For this, two sampling techniques were employed: one employed
4021 examinees as subject population; the other employed their scores and item scores as
attribute population.
Of the 4021 examinees, they were categorized into groups of high and non-high
achievers. Each sat the English reading test which comprised 30 multiple-choice items. The
scoring was manipulated into number-correct and formula. Hence, the examinees yielded
their scores on the items responded, i.e., the high achievers’ scores from number-correct
scoring and formula scoring, the non-high achievers’ scores from number-correct scoring and
formula scoring. Those scores then assigned for the subject and attribute population used for
data simulation in the theoretical experiments. Of the 4021 examinees’ scores and 30 item
scores, 150 examinees’ scores were randomly drawn with replacement in each group and 25
item scores were randomly drawn without replacement. In terms of replication over 100
samples in each group of sample population, the estimated mean scores of samples were
gained. To some valuable extent, reliability coefficients of scores were estimated. The
reliability coefficients in each group of examinees pursued measures of variances. The
smallest variance over 100 samples with scoring manipulation indicated the stability of
reliability coefficients of the test scores.
Based on the hypotheses tested, the study proved that (1) in a group of non-high
achievers, the variance of reliability coefficients when occupying formula scoring was
significantly smaller than that of reliability coefficients when occupying number-correct
scoring, (2) in a group of high achievers, the variance of reliability coefficients when
occupying number-correct scoring was not significantly smaller than that of reliability
coefficients when occupying formula scoring, (3) in a group of non-high achievers, the
variance of reliability coefficients was significantly the same as that in a group of high
achievers when occupying number-correct scoring, and (4) in a group of non-high achievers,
the variance of reliability coefficients was significantly the same as that in a group of high
achievers when occupying formula scoring. In sum, the stability of reliability coefficients was
brought about by formula scoring in groups of non-high and high achievers, but it could not
inform any stability of reliability coefficients when comparing their variances in groups of high
and non-high achievers by occupying the same scorings.