92
Evaluating the Effectiveness of Stroke Rehabilitation:
Choosing a Discriminative Measure
Kim A. Brock, PhD, Patricia A. Goldie, PhD, Kenneth M. Greenwood, PhD
ABSTRACT. Brock KA, Goldie PA, Greenwood KM.
Evaluating the effectiveness of stroke rehabilitation: choosing
a discriminative measure. Arch Phys Med Rehabil 2002;83:
92-9.
Objective: To evaluate the discriminative ability of several
measures of physical disability used to determine quality of
outcome for poststroke rehabilitation.
Design: A comparative study, using Rasch analysis, of the
discriminative ability of functional status and mobility mea-
sures in rehabilitation patients with stroke.
Setting: A 26-bed rehabilitation unit, on site of a tertiary
teaching hospital in Melbourne, Australia.
Participants: A consecutive sample of 106 patients with
acute stroke admitted for rehabilitation.
Interventions: Not applicable.
Main Outcome Measures: Rasch analysis of the motor
subscale of the FIM™ instrument, Motor Assessment Scale,
Functional Ambulation Classification, gait velocity, and gait
endurance.
Results: The more difficult items of the FIM motor scale
adequately discriminated among higher functioning patients.
The gait velocity measure further distinguished 9% of the
sample, who functioned at a higher level than could be indi-
cated by FIM motor subscale. The other measures did not add
levels of discrimination to that provided by the FIM motor.
Ability estimates provided by Rasch analysis of the FIM motor
scale were a more accurate indication of ability than raw
scores. Raw scores underestimated change in ability observed
at higher levels of ability.
Conclusion: Rasch estimates of the FIM motor subscale
provide a discriminative measure for evaluating outcomes and
change in ability achieved in stroke rehabilitation.
Key Words: Cerebrovascular disorders; Disability evalua-
tion; Outcome assessment (health care); Rehabilitation.
© 2002 by the American Congress of Rehabilitation Medi-
cine and the American Academy of Physical Medicine and
Rehabilitation
he search for appropriate outcome measures has been a
T focus of rehabilitation research for decades. Introducing
benchmarking processes into health care, which facilitates
comparisons of resource use and outcomes, has intensified
From the Physiotherapy Department, St Vincent’s Hospital, Fitzroy, Melbourne,
Australia (Brock); and Schools of Physiotherapy (Goldie) and Psychological Science
(Greenwood), La Trobe University, Bundoora, Victoria, Australia.
Accepted in revised form March 7, 2001.
Presented at the Joint Annual Scientific Meeting of the Australasian Faculty of
Rehabilitation Medicine, the British Society of Rehabilitation Medicine, and the
International Association of Health Professionals in Rehabilitation, May 27-30, 1998,
Sydney, Australia.
No commercial party having a direct financial interest in the results of the research
supporting this article has or will confer a benefit upon the author(s) or upon any
organization with which the author(s) is/are associated.
Reprint requests to Kim Brock, PhD, Physiotherapy Dept, St Vincent’s Hospital,
Victoria Pde, Fitzroy, 3065 Melbourne, Australia, e-mail: brockka@svhm.org.au.
0003-9993/02/8301-6469$35.00/0
doi:10.1053/apmr.2002.27348
Arch Phys Med Rehabil Vol 83, January 2002
discussion about valid methods for outcome evaluation. With-
out evidence of outcomes, measures of resource use may be
used to justify spending reductions when “the true need is to
maximize health benefits for the money spent.”1 Benchmarking
processes for analyzing costs of delivering health care are more
advanced than processes for determining outcomes of health
care. Although this difference exists, changes in service deliv-
ery may be driven mainly by cost factors, with insufficient
regard to the quality of outcomes associated with the changes.
A major issue in pursuing outcome-based benchmarking is
the adequacy of the tool used to measure outcome. Tools used
for this purpose must be reliable, valid, and discriminative.
Examining discriminative ability is important to ensure that a
chosen outcome measure is able to differentiate within the
patient group and to identify meaningful differences in pa-
tients’ abilities.2 A discriminative scale will have items that
span the range of patient ability, with appropriately spaced
intervals. Failure to differentiate among patients at higher lev-
els of function may produce inaccurate evaluations of the
effectiveness of rehabilitation units.
Functional status measures such as the FIM™ instrument,3
which assess the assistance required for a person to complete
daily living tasks, are the most widely used outcome measures.
Concerns have been expressed about possible ceiling effects of
these ordinal disability scales.4-8 Dodds et al4 argue that the
FIM does not take into account elements such as speed, ease,
and quality of task performance. Johnston et al9 argue that
ordinal functional status measures are responsive to improve-
ment in a limited part of the range only, with both floor and
ceiling effects. For patients with less disability, Johnston sug-
gests that measures of speed and endurance may be important.
In light of these comments, it may be useful to compare the
discrimination of the FIM motor subscale with other measures
in the physical domain that include aspects such as speed and
quality of task performance.
The present study considered outcome measures that reflect
mobility function. In the rehabilitation of motor function, much
emphasis is placed on the quality and ease of walking and of
motor tasks such as moving from sitting to standing. This study
investigates whether the motor subscale of the FIM adequately
represents mobility function after rehabilitation.
The appropriateness of mathematically manipulating ordinal
values of functional status measures, such as summing scores
and calculating change values, has also come under question in
the literature.10-12 Item response theory, using Rasch analysis,
has been used to investigate the properties of ordinal scales,
including the FIM. With Rasch analysis, ordinal scales are
converted into interval measures by estimating the increase in
ability for each level of the scale, for each of the items.12,13
Several studies have shown the validity of developing Rasch
estimates of ability from the FIM.14-16 Linacre et al15 showed
that the steps between scores on the FIM motor scale are not
equal in level of difficulty. A change in ability at the top end of
the scale was of greater magnitude when converted into Rasch
estimates than a similar change in ability toward the middle of
the scale. A 10-point change in raw data scores from 80 to 90,
when converted to Rasch estimates, indicates 4 times as much
 CHOOSING A DISCRIMINATIVE MEASURE, Brock
change as a change in raw scores from 40 to 50. This finding
indicates that summed raw scores cannot be considered to be an
interval scale for the FIM motor subscale. However, the reha-
bilitation field seems slow to adopt the use of Rasch estimates,
and summed raw scores remain the more frequently reported
indicator of outcome.
The purpose of the present study was to compare several
outcome measures, using Rasch analysis, to distinguish which
measure or combination of measures is the most discriminative
for assessing physical disability at the end of intensive reha-
bilitation after stroke.
METHODS
Measures of Physical Ability
We examined 3 ordinal scales: the 13 items that make up the
motor component of the FIM motor subscale3 (FIM motor), the
Motor Assessment Scale17 (MAS), and the Functional Ambu-
lation Classification18 (FAC). The FIM motor assesses the level
of assistance required to perform various activities of daily
living. Extensive investigations of the FIM’s reliability and
validity have provided evidence of its interrater and test-retest
reliability,19 internal consistency,20,21 concurrent validity,22,23
and predictive validity.23-26
The MAS17 assesses 5 functional motor tasks (supine to side
lie, supine to sitting, sitting, sit to stand, walking) by using
criteria that address the quality of performance as well as the
level of assistance required. The instrument’s quality aspects
include symmetry, control, timing of movement, and use of the
affected side. The upper limb components of the MAS were not
considered in the present study. Interrater reliability,17,27,28
concurrent validity,28,29 and predictive validity30 have been
shown for the MAS.
The FAC18 addresses walking ability relevant to community
ambulation, such as ability to walk on rough ground, ramps,
and over curbs. Limited reliability and validity evaluations
have been undertaken, including interrater reliability31 and
concurrent validity with other gait measures.18,31,32
We included 2 interval scaled measures of walking ability—
gait velocity and endurance—in the present study, to compare
the discriminative ability of ordinal scales and specific interval
measures. Gait velocity is a widely used measure of function
at the end of rehabilitation. Few stroke patients achieve
normal gait velocity, with a recent study finding that 83% of
stroke patients were still impaired in terms of gait speed at
3 months poststroke.33 Studies have provided evidence of in-
terrater reliability,18,34,35 test-retest reliability,18,35,36 concurrent
validity,18,31,32,37-41 and predictive validity.42 Previous stud-
ies36,43,44 have shown gait velocity to be a discriminative mea-
sure.
Few functional status measures include an assessment of
endurance beyond the 50-meter requirement of the FIM. An
audit of stroke outcomes undertaken by Hill et al45 showed that
only 15% of patients could walk more than 500 meters at
discharge. To date, there are no simple tests of endurance with
established reliability and validity. We included a standardized
test of endurance in the present study, requiring the subject to
walk laps of a 50-meter circuit.
If the areas of function represented by the 4 ordinal scales
and 2 interval measures are shown to be measuring the same
domain (physical disability), then Rasch analysis could be used
to compare difficulty levels of items in the disability scales and
the gait measures. This comparison would then permit one to
identify the measures that most adequately differentiate be-
tween various levels of ability at the higher end of the spectrum
of functional ability, observed at the end of rehabilitation. A
93
key advantage to using Rasch analysis in the evaluation of
rating scales is the potential to include measures that do not
cover the whole spectrum of ability but are more sensitive to a
portion of the range.46 This provides a method for integrating
measures, inclusive of measures that are not as discriminative
but cover the range of ability, and measures that discriminate
well, but only in a portion of the range.
Few studies have used Rasch analysis to undertake compar-
ative analysis of the discriminative ability of scales. Fisher et
al47 used it to compare the discriminative abilities of the FIM
motor and 22 items from the motor competency components of
the Patient Evaluation Conference System, including items
assessing housework and meal preparation. Grimby et al48
combined the physical activities of the FIM with the Instru-
mental Activity Measure (IAM). The IAM assesses domestic
tasks, such as cleaning and shopping, and community mobility.
The aim of using Rasch analysis to investigate outcome
measures in the present study was to establish which measure,
or combination of measures, would yield a single ability score
for each patient that most accurately reflects mobility outcome.
This score could represent either functional level at outcome or
change in function between admission and discharge. Having
established a discriminative measure, comparisons of outcome
in relation to resource use could then be undertaken with
greater confidence.
At the time of the study (1993–1995), the funding of reha-
bilitation services in Victoria was provided by block grants,
allowing rehabilitation teams to set individual goals and pro-
grams for each patient, without being restricted by specific
requirements of funding agencies. This approach permitted
wide variation in length of stay (LOS) in the rehabilitation unit
at St Vincent’s Hospital, to achieve high levels of outcome,
with a large proportion of discharges to independent living in
the community. During the study, few alternatives to inpatient
care were available for intensive rehabilitation services in
Victoria. Almost all patients in the present study had stopped
participating in intensive rehabilitation at discharge; some pa-
tients had prolonged inpatient stays to achieve sufficient func-
tion to return home. Therefore, in this study, discharge from
inpatient rehabilitation adequately represents the end point of
intensive rehabilitation.
Subjects
All patients admitted to the St Vincent’s Hospital Rehabili-
tation Unit from 1993 to 1995 with a primary diagnosis of
stroke, infarct, or hemorrhage were included in the study.
Patients were excluded if they suffered another major incident
at onset of stroke (eg, fracture, amputation), or had cerebro-
vascular accidents related to trauma. Patients were selected for
admission to the rehabilitation unit according to criteria that
included potential to return to some form of independent living
and the ability to cope with an intensive rehabilitation program.
The study was approved by the ethics committees of St Vin-
cent’s Hospital and the La Trobe University Faculty of Health
Sciences.
One hundred six patients were included in the study. The
patients had an average age ⫾ standard deviation of 68.7 ⫾
11.3 years (range, 36 –91yr). Sixty-seven (63%) were men.
Forty-nine patients (46%) had right-hemisphere lesions, 48
(45%) left-hemisphere lesions, and 9 patients (9%) had bilat-
eral lesions. Eighteen patients (17%) had hemorrhagic strokes
and 87 patients (82%) had infarcts. The pathology was un-
known for 1 patient. Before admission, 27% of patients lived
alone, 49% lived with a spouse, 17% with their family, and 7%
lived in special accommodation houses or hostels. Table 1
shows median time from onset of stroke to admission to reha-
Arch Phys Med Rehabil Vol 83, January 2002
94 CHOOSING A DISCRIMINATIVE MEASURE, Brock
Table 1: Hospital LOS and Functional Status of Cohort
25th 75th
Median Percentile Percentile Range
LOS (d)
Onset to admission 11 8 15.5 0–52
Rehabilitation (d) 28 16 68 3–274
Functional status
(summed raw scores)
Admission FIM motor 67 51 80 18–91
Discharge FIM motor 86 81.5 89 32–91
bilitation, LOS in rehabilitation, and median admission and
discharge FIM motor scores. At the end of rehabilitation, 89%
of patients returned home, 7% to semi-independent living, and
4% to nursing home care.
Instruments
The FIM and MAS were assessed at admission and dis-
charge. Additional mobility variables of gait speed, endurance,
and rating on the FAC were assessed at discharge.
Procedure
Patients were assessed at admission and at discharge from
rehabilitation. The FIM motor scores were evaluated according
to the FIM guidelines by rehabilitation team members consid-
ering the patient’s performance over a 24-hour period. During
the study period, training and examination of FIM reliability
was performed at least twice yearly, and at least 10 staff on the
Rehabilitation Unit achieved satisfactory reliability, as mea-
sured by the Uniform Data System for Medical Rehabilitation
examination process, at any 1 time.
The MAS was assessed by 1 of 3 physiotherapist raters, all
of whom had undergone reliability testing on the MAS through
the School of Physiotherapy, University of Sydney, achieving
satisfactory reliability scores of above 80%. Walking speed
was assessed by the treating physiotherapist as unassisted,
self-selected walking speed over the central 6 meters of a
10-meter walkway. The use of aids and splints customarily
worn was allowed and recorded. The endurance test required
the patient to walk laps of a 50-meter circuit outdoors. The
patients were asked to “walk as many laps as you can, without
tiring yourself, to a maximum of 10.” The FAC was rated by
the treating physiotherapist.
Statistical Analysis
Discriminative ability was evaluated by examining the
spread of scores for ceiling effects and through Rasch analysis.
The use of Rasch analysis is dependent on the underlying
variations in behavior being dominated by 1 dimension.12 Be-
fore subjecting the data to Rasch analysis, we applied principal
components analysis49,50 to establish the unidimensionality of
the measures used, for the measures individually and for com-
bined data sets.
At the end of intensive rehabilitation, we calculated item
estimates (level of difficulty of items) and case estimates (level
of ability of persons tested) by using Rasch analysis for each
functional measure. These estimates are measured as logits,
mathematically defined units that are constant from 1 end of
a continuum to the other. Comparing estimates allowed us to
compare both the difficulty level of items on a scale and the
ability level of patients. Rasch analysis also provides fit
statistics, which indicate how the observed ratings vary from
those predicted by the model. These fit statistics show the
Arch Phys Med Rehabil Vol 83, January 2002
estimate’s internal validity and indicate the unidimensionality
of the model.51
The stability of the measures over time was investigated to
determine whether the relative level of difficulty of the items
remained constant when assessed at different points in time.15
This attribute is essential to the validity of a measure used for
quantifying change. The stability of FIM motor scores has been
shown in several studies.15,16,52,53 The stability of the measures
used in the present study was examined by using intraclass
correlation coefficients (ICCs), as recommended by Chang and
Chan.53
Rasch analysis does not provide case estimates for perfect
scores. If a considerable portion of the sample achieve a perfect
score, one must determine the level of ability this score indi-
cates. Otherwise, this group’s achievements are not considered
when assessing change. Case estimates for perfect scores on
the FIM motor and MAS were established by adding a more
difficult item from another measure to the functional scale and
performing another Rasch analysis.
The items on the functional status measures that discrimi-
nated among higher level performances were then compared by
entering them into a further Rasch analysis. The purpose of
which was to identify the functional status measure, or com-
bination of measures, that most adequately discriminated
among higher level performances.
Data analysis was performed on a personal computer, using
the computer software packages SPSS, version 6.0,a and
QUEST.b
RESULTS
Spread of Scores
The measures were examined for ceiling effects at discharge.
Table 2 shows the proportion of patients achieving the highest
score for each of the measures and on the hardest item of the
measure. The FAC had the strongest ceiling effect (table 2).
The endurance test revealed that 39% of subjects were able to
walk 500 meters on outdoor surfaces without a rest. In contrast,
the FIM motor subscale had 16% and the MAS had 25% of
persons scoring the highest score.
Gait velocity has no ceiling effect. However, to facilitate
comparison, the gait velocity variable was grouped into 8
classes, with the fastest walkers being those who walked at
more than 70m/min. This cutoff was chosen on the basis of a
study54 investigating the gait velocity required to safely cross
signalled road intersections in Melbourne. Patients with a self-
selected gait velocity of 70m/min have sufficient gait velocity
to safely cross almost all signalled intersections in Melbourne,
at their normal walking speed. Nine percent of patients
achieved this optimum speed. This test showed the least ceiling
effect.
Although the FAC and endurance showed large ceiling ef-
fects, these single-item tests may yield useful information in
combination with other tests. The validity of combining gait
Table 2: Percentage of Patient’s Achieving the Highest Score for
Each Measure
Total (%) Most Difficult Item (%)
FIM motor 16 29
MAS 25 35
Gait velocity (grouped) 9
Endurance 39
FAC 46
 CHOOSING A DISCRIMINATIVE MEASURE, Brock
endurance, the FAC, and gait velocity into a variable repre-
senting gait function was explored.
Principal Components Analysis
Principal components analysis of FIM motor scores showed
that the scale closely approached a unidimensional scale: 2
factors had eigenvalues above 1.0 (8.6, 1.2). The items of
“bowel” function and “eating” loaded most strongly onto the
second factor. Principal components analysis of MAS scores
revealed 1 factor with an eigenvalue above 1.0. We entered the
3 items assessing gait (gait velocity, gait endurance, FAC
score) into principal components analysis to establish their
unidimensionality. One factor was revealed with an eigenvalue
above 1.0. Because all the measures met the criteria of unidi-
mensionality to a reasonable degree, the measures were further
examined with Rasch analysis.
Rasch Analysis
FIM motor. Rasch analysis was performed on the dis-
charge FIM motor scores. One item, “bladder” function,
showed serious misfit, as indicated by the infit statistic (2.61).
Figure 1 shows the estimated difficulty of each level of the 13
items. Item levels that are scored similarly in the FIM motor
may have markedly different weightings in the Rasch analysis.
The FIM motor item estimates obtained in this study were
compared with those quoted in the study by Linacre et al15 to
examine how closely our small sample related to the very large
sample of Linacre. Correlation of combined admission and
discharge estimates yielded a Pearson’s r correlation of .87,
indicating substantial similarity between the item estimates.
Sixteen percent of patients had a perfect score on the FIM
motor. Review of case estimates revealed only 5 cases with
infit t values beyond 2.0 or ⫺2.0 that seriously misfit the
model. The mean case estimate was 2.34 ⫾ 1.56, indicating
that the sample was biased toward cases with higher scores, as
would be expected in the discharge scores of a selected reha-
bilitation sample. The item separation reliability51 was .85.
Although the Rasch modeling shows some areas of concern,
the overall model fit justified our continuing with the analysis.
Calculation of the ICC of each item’s level of difficulty at 2
weeks poststroke and at discharge revealed an ICC of .87.
Chang and Chan53 stipulated a cutoff of an ICC value of .90 as
an indication of the appropriateness of producing generalized
item difficulty estimates from scores at both occasions. Al-
Fig 1. Item threshold estimates for FIM motor scores at discharge.
Abbreviations: Eat, eating; Groom, grooming; Bath, bathing; Dress
up, dressing the upper body; Dress low, dressing the lower body;
Toilet, toileting; Bed tr, bed transfers; Toilet tr, toilet transfers; Tub
tr, tub transfers.
95
Fig 2. Summed raw scores and Rasch estimates of FIM motor
scores at discharge. (Rasch estimates and summed raw scores were
rescaled to range between 0 and 100.)
though the present value falls short of that .90 threshold, it is
close to the specified level. Because 3 previous studies16,52,53
found the FIM motor’s stability to be acceptable, we combined
the admission and discharge scores in the present study. The
item thresholds developed on the combined data set are used
later in discussion of Rasch transformation of the FIM motor.
As previously discussed, Rasch techniques do not provide an
estimate for perfect or zero scores. This is a problem for compar-
ison of summed raw scores and Rasch estimates and for the
calculation of change. A value for perfect scores was ascertained
by adding another variable to the Rasch analysis. The only
variable in the study that had less ceiling effect than the FIM
motor was gait velocity (grouped as previously discussed). We
performed a further Rasch analysis with the FIM motor vari-
ables and gait velocity. Before this, we assessed the unidimen-
sionality of the combined FIM motor and gait velocity by using
principal components analysis. Two factors were revealed with
eigenvalues of 7.1 and 1.7, with the gait and transfer items
loading heavily on the first factor and “eating” and “grooming”
loading heavily onto the second factor. The fit statistics of the
Rasch analysis showed an infit mean square value of 1.4 for
gait velocity. Although the fit of gait velocity to the FIM scale
was not optimum, it was considered reasonable to continue this
analysis because it was the most accurate means of obtaining
an estimate for perfect scores on the FIM motor.
The difference in Rasch case estimates between cases with
raw scores of 90 and 91 revealed that the step size between 90
and 91 in raw scores was equivalent to 11% of the Rasch
estimates. This finding indicates that a large improvement in
ability is required to move from a score of 90 to 91, compared
with an improvement of 1 point at other points of the scale.
Calculating a Rasch estimate for perfect FIM scores enables
more accurate comparison of Rasch estimates and summed raw
scores.
The relationship between the summed raw scores and Rasch
estimates at discharge displayed an ogival relationship at the
upper end (fig 2). The correlation between summed raw scores
and Rasch estimates was .92 at admission and .83 at discharge.
Rasch estimates and summed raw scores were compared as
measures of change. The correlation between change as
summed raw scores and Rasch estimates was .71 (fig 3). Figure
4 shows that for those subjects with low admission scores,
summed raw scores tended to overstate the degree of change in
ability, and for subjects with high admission scores, the
summed raw scores tended to understate the degree of change
in ability.
Arch Phys Med Rehabil Vol 83, January 2002
96 CHOOSING A DISCRIMINATIVE MEASURE, Brock
Fig 3. Summed raw scores and Rasch estimates for change in FIM
motor scores.
Motor Assessment Scale. Rasch analysis was performed
on the discharge MAS scores. One item, sitting, had an infit
mean square value above 1.3 (1.45). Figure 5 shows the esti-
mated difficulty of each level of the 5 items. As with the FIM
motor, levels of items that are scored similarly in the MAS may
have markedly different weightings in the Rasch analysis.
Review of case estimates revealed only 2 cases with infit t
values beyond 2.0 or ⫺2.0. The mean case estimate was 1.94 ⫾
1.60. The item separation reliability was .70.
Calculation of the ICC of the estimates at 2 weeks poststroke
and discharge revealed an ICC of .77. This ICC is considerably
lower than the cutoff of .90, stipulated by Chang and Chan53 as
an indication of the appropriateness of producing generalized
item difficulty estimates from scores at both occasions. The
lack of stability of the thresholds of the items over time
precludes combining admission and discharge data into a single
data set, as was performed for the FIM motor. This finding
therefore, decreases the validity of the MAS Rasch estimates
being used to measure change. For the present analysis, the
MAS was used as a measure of ability at discharge only.
To obtain a value for subjects who had a perfect score, we
added another variable to the Rasch analysis. Gait velocity was
the variable of choice, having less ceiling effect than the MAS.
A further Rasch analysis was performed with the MAS and gait
velocity. Before this analysis, the unidimensionality of the
MAS with gait velocity was explored with principal compo-
nents analysis. The results were very similar to the results for
Fig 4. Summed raw scores and Rasch estimates of change in FIM
motor scores for randomly selected cases. (Cases are ordered ac-
cording to initial FIM motor score.)
Arch Phys Med Rehabil Vol 83, January 2002
Fig 5. Item threshold estimates for the MAS at discharge.
the MAS alone, with only 1 factor having an eigenvalue above
1.0. A Rasch analysis was performed, adding gait velocity to
the MAS. The fit statistics of the Rasch analysis showed an infit
mean square value of .87 for gait velocity.
The difference in Rasch case estimates between cases scor-
ing 29 and 30 on the MAS was calculated and converted into
a percentage of the available range of Rasch estimates. Thus, it
was revealed that the difference between 95 and 100 in raw
scores was equivalent to 17% of the range of Rasch estimates.
Cases with perfect scores were recoded to reflect this relation-
ship, with a case estimate of 4.89. Rasch estimates were then
recoded to range between 0 and 100 to facilitate comparison
with summed raw scores.
The correlation between summed raw MAS scores and
Rasch estimates at discharge was .92. Although the correlation
is relatively high, summed raw scores understate the level of
ability for the cases with the highest MAS scores.
Gait measures. Rasch analysis was performed on the gait
measures. The infit mean square values of the 3 items indicate
that no items misfitted the model. Figure 6 shows the estimated
difficulty of each level of the 3 items. Ten cases had perfect
scores at discharge. Review of case estimates revealed only 3
cases with infit t values beyond 2.0 or ⫺2.0. The mean case esti-
mate was 1.43 ⫾ 1.83. The item separation reliability was .83.
Comparison of the Discriminative Properties of the
Functional Status Measures
The discriminative properties of selected items from the FIM
motor, the MAS, and the gait measures were compared by
Fig 6. Item threshold estimates for gait measures.
 CHOOSING A DISCRIMINATIVE MEASURE, Brock
Table 3: Items Chosen for Comparative Analysis
Measure Item
FIM motor Stairs
Bathing
Tub transfer
Walking
MAS Walking
Sitting
Gait Velocity
Endurance
using Rasch analysis. Because the sample size of this study
precluded including too many items within any 1 analysis, we
considered only those items of the measures that had discrim-
inated most effectively at higher levels of ability. We identified
the items with high-item thresholds in the Rasch analysis for
each measure and entered them into a combined Rasch analy-
sis. The items included in the combined Rasch analysis are
shown in table 3.
Principal components analysis of the items revealed 2 factors
with eigenvalues above 1.0. These factors had eigenvalues of
6.31 and 1.29, explaining 57.4% and 11.8% of the variance.
Although the scale revealed 2 factors greater than 1, scree
testing showed that the second factor represented the point at
which a line drawn through the points changed direction.
Subsequently, we continued with Rasch analysis because the
scale closely approaches a unidimensional scale.
Rasch analysis was performed on the combined gait and
functional status measures at discharge. Two items were found
to misfit the model, MAS sitting (1.44) and endurance (1.39).
Two patients had perfect scores at discharge. Review of case
estimates revealed 9 cases with infit t values beyond 2.0 or
⫺2.0. The mean case estimate was 2.46 ⫾ 2.35. The item
separation reliability was .90. Figure 7 shows the item thresh-
old estimates for each of the items and the number of cases at
the upper scoring levels of each scale.
Gait velocity showed the highest level of difficulty, at 4.84
logits (see fig 7). The next most difficult item thresholds, at
3.46 and 3.33 logits, were the next level down for walking
velocity and the highest level of the FIM “stairs” item. The
FIM motor items of “stairs” and “walking” provided a good
spread of item thresholds at higher levels of ability. Table 4
Fig 7. Item threshold estimates for the combined measures. For the
higher levels of each item, the number of patients with scores at
each level are provided, to facilitate comparing the discriminative
abilities of items at the upper end of the spectrum of ability.
97
Table 4: Distribution of Scores for FIM Motor Walking and Stairs
FIM Walking FIM Stairs n*
7 7 29
7 6 22
6 6 19
6 5 6
5 5 5
5 4 0
4 or ⬍4 4 or ⬍4 9
* For simplification, this table does not include 11 patients who had
more than a 2-point difference between the FIM walking and FIM
stairs item.
shows the distribution of scores for these 2 FIM motor items,
showing the capacity of these items to discriminate within the
sample. The FIM motor items of “bathing” and “tub transfer”
also differentiated among cases at this level. The MAS “walk-
ing” item (highest level threshold at 2.90 logits) discriminated
nearly as well as the FIM motor, but the other items of the
MAS contributed little to the discriminative ability of the scale.
The endurance variable’s highest threshold level was 2.52
logits, well below that of other variables.
DISCUSSION
Examining the spread of scores for the measures, we found
wide variation in the ceiling effects. Those for endurance and
the FAC showed marked ceiling effects, with 39% and 46% of
subjects achieving the highest scores. The ceiling effect of the
FIM motor was much less pronounced, with only 16% achiev-
ing a perfect score. The MAS was shown to have a mild ceiling
effect, with 25% achieving a perfect score at discharge. The
gait velocity variable showed minimal ceiling effects with only
9% of patients achieving the highest level.
In stroke rehabilitation, a small proportion of patients re-
cover with very little residual disability. Arguably, a ceiling
effect of 16% of cases, to use the FIM motor as an example, is
an acceptable level because there may be little need to discrim-
inate among patients making a near perfect recovery. It is also
possible that the disability in these cases fell in other domains,
such as cognition and/or communication. If that is so, then
there is little point in attempting to increase the discriminatory
power of a measure of physical disability.
However, examination of ceiling effects alone is insufficient
to show the suitability of a measure for evaluating higher level
outcomes. A measure may have a low ceiling effect, with
inclusion of a difficult item. The “step” size between the most
difficult and the next level down the scale is equally important.
If large gaps exist between the upper levels, the measure may
not discriminate adequately among the population. Rasch anal-
ysis investigates the “size of the steps.”
Rasch analysis showed that the FIM motor items represented
varying levels of difficulty. Step size tended to increase with
level of difficulty (see fig 1), and the step between 90 and 91
represented 11% of the scale. This value is similar to that
provided by Fiedler,55 in which the difference between 90 and
91 for summed raw scores represented 10% of the range of
Rasch estimates.
The relationship between the summed raw scores and Rasch
estimates at discharge displayed a nonlinear relationship, as
previously shown by Linacre et al.15 The upper levels of the
FIM motor require greater steps in level of ability to achieve
the next level than are required at the lower levels. The corre-
lation between summed raw scores and Rasch estimates for
Arch Phys Med Rehabil Vol 83, January 2002
98 CHOOSING A DISCRIMINATIVE MEASURE, Brock
change was .71. Change in the lower part of the disability
spectrum tends to be overstated by the summed raw scores and
change in the upper part of the disability spectrum tends to be
understated by the summed raw scores. Summed raw scores
and Rasch estimates are not readily comparable for calculating
change. To accurately represent level of ability or change in
ability, Rasch estimates are preferable to summed raw scores.
Rasch analysis of the MAS at discharge showed acceptable
model fit. For this scale, the difference between the perfect
score and the next level down occupied 17% of the range of
Rasch case estimates. Summed raw scores and Rasch estimates
were compared for MAS at discharge, yielding a correlation of
.92. We found a nonlinear relationship for scores in the upper
end of the scale. The use of Rasch case estimates, in preference
to summed raw scores, is recommended for the MAS at dis-
charge to evaluate accurately ability level.
Comparing item thresholds for admission and discharge
scores on the MAS revealed an ICC of .77. This correlation
was well below the cutoff of .90 recommended by Chang and
Chan.53 This finding suggests that it is inadvisable to calculate
change scores from the MAS because the difficulty level of the
items did not show stability over time. This is a major problem
for the use of this scale. Further development of the scale,
identifying and rectifying those item levels that do not maintain
a constant level of difficulty at admission and discharge, may
improve the stability of the MAS.
Rasch analysis of gait velocity, endurance, and the FAC at
discharge showed acceptable model fit. The case estimates
developed provide a composite score representing the patient’s
walking ability in terms of speed, endurance, level of assistance
required, and ability to walk on stairs, inclines, and nonlevel
surfaces (FAC).
Comparing items from the FIM, MAS, and the gait measures
by using Rasch analysis, we found that gait velocity had the
strongest discriminative ability among patients with higher
level abilities. However, this single item does not extend to
lower levels of ability. The item is not appropriate for patients
who cannot walk without assistance at discharge from rehabil-
itation. The next item to discriminate well among high-level
cases was FIM “stairs.” This item requires patients to go up and
down 12 to 14 stairs, without use of an aid or stair rail, safely
and in a timely manner. That the FIM motor scale achieved the
third highest level of difficulty argues against a ceiling effect
for this scale. This finding adds weight to the contention by
Linacre15 that the ceiling effect of the FIM is apparent rather
than real, and that use of Rasch converted scores clarifies the
true magnitude of the raw score differences. The 4 FIM motor
items of “stairs,” “walking,” “bathing,” and “tub transfer”
discriminated well throughout the spectrum of ability at dis-
charge.
Although the MAS item of “walking” was a good discrim-
inator, the other variables of the MAS were less useful. The
threshold level of MAS “walking” was lower than the threshold
level of the FIM “stairs” item. This is curious because both
items require the patient to go up and down 12 stairs without a
rail or a walking aid, with a specific time limit applied to the
MAS. The main difference between the tests is that the MAS
item is tested on a single occasion, whereas the FIM is rated on
24-hour performance, with safety as a consideration. Patients
may successfully complete the MAS test under supervision,
though their therapist may not rate them as safe on stairs
without supervision.
At the time of data collection in this study (1993–1995),
most rehabilitation services in Australia were delivered in the
inpatient setting. Long hospital LOS for the more severely
disabled stroke patients were not uncommon. In recent years,
Arch Phys Med Rehabil Vol 83, January 2002
other options for intensive rehabilitation services have been
developed, such as “same day” rehabilitation (an intensive
outpatient rehabilitation program) and home-based rehabilita-
tion. Although the present data set is composed solely of
inpatient data, it is arguable that outcome measurement should
be performed at discharge from intensive rehabilitation, no
matter how those services are delivered.
CONCLUSION
The FIM motor subscale proved to be the most suitable
measure for evaluating mobility outcomes, as either level of
ability at discharge, or as change in ability from admission to
discharge. Comparison of measures did not support the need to
add additional items to the FIM motor to prevent ceiling
effects. Rasch analysis of the FIM motor showed that summed
raw scores did not show the characteristics of interval mea-
sures. Small increments of change in raw scores at the higher
levels of the scale, which may be interpreted as indicating little
progress, actually denote far more substantial change. For an
accurate indication of ability level, Rasch estimates should be
used, particularly for ability level at discharge and for the
calculation of change scores. The use of the conversion table of
summed raw scores to Rasch estimates of ability calculated by
Fiedler55 provides a simple and valid means of accurately
comparing levels of ability at the end of intensive rehabilita-
tion. The present study supports the use of the FIM motor, with
Rasch-transformed item difficulty and patient ability levels as
a discriminative outcome measure of physical disability. The
use of this outcome evaluation model would permit bench-
marking of stroke rehabilitation services according to standards
of outcome, as well as according to costs incurred in stroke
rehabilitation.
References
1. Hungate RW. Purchaser quality measures: progressing from wants
to needs. Jt Comm J Qual Improv 1994;20:381-6.
2. American Education Research Association and National Council
on Research in Education. Standards for educational and psycho-
logical tests. Washington (DC): American Psychological Associ-
ation; 1985.
3. Center for Functional Assessment Research and the Uniform Data
System for Medical Rehabilitation. Guide for use of the Uniform
Data Set for Medical Rehabilitation including the Functional
Independence Measure (FIM) version 3.1. Buffalo (NY): State
Univ New York; 1990.
4. Dodds TA, Martin DP, Stolov WC, Deyo RA. A validation of the
Functional Independence Measurement and its performance
among rehabilitation inpatients. Arch Phys Med Rehabil 1993;74:
531-6.
5. Nissen MI. Emphasis placed by Cedar Court Physical Rehabili-
tation Hospital on the functional independence measure as the
overall index for successful rehabilitation. Aust Clin Rev 1989;9:
36-8.
6. Pollock C, Freemantle N, Sheldon T, Song F. Methodological
difficulties in rehabilitation research. Clin Rehabil 1993;7:63-72.
7. Tsuji T, Sonada S, Domen K, Saitoh E, Liu M, Chino N. ADL
structure for stroke patients in Japan based on the Functional
Independence Measure. Am J Phys Med Rehabil 1995;74:432-8.
8. Wade DT, Skilbeck CE, Hewer RL. Predicting Barthel ADL score
at 6 months after an acute stroke. Arch Phys Med Rehabil 1983;
64:24-8.
9. Johnston MV, Findley TW, DeLuca J, Katz RT. Research in
physical medicine and rehabilitation: XII. Measurement tools with
application to brain injury. Am J Phys Med Rehabil 1991;70:40-
56.
10. Merbitz C, Morris J, Grip JC. Ordinal scales and foundations of
misinference. Arch Phys Med Rehabil 1989;70:308-12.
11. Silverstein B, Fisher WP, Kilgore KM, Harley JP, Harvey RF.
Applying psychometric criteria to functional assessment in med-
 CHOOSING A DISCRIMINATIVE MEASURE, Brock
ical rehabilitation: II. Defining interval measures. Arch Phys Med
Rehabil 1992;73:507-18.
12. Wright BD, Linacre JM. Observations are always ordinal; mea-
surements, however, must be interval. Arch Phys Med Rehabil
1989;70:857-60.
13. Fisher AG, Bryze KA, Granger CV, et al. Applications of conjoint
measurement to the development of functional assessments. Int J
Educ Res 1994;21:579-93.
14. Heinemann AW, Linacre JM, Wright BD, Hamilton BB, Granger
C. Relationships between impairment and physical disability as
measured by the Functional Independence Measure. Arch Phys
Med Rehabil 1993;74:566-73.
15. Linacre JM, Heinemann AW, Wright BD, Granger CV, Hamilton
BB. The structure and stability of the Functional Independence
Measure. Arch Phys Med Rehabil 1994;75:127-32.
16. Wright BD, Linacre JM, Heinemann AW. Measuring functional
status in rehabilitation. Phys Med Rehabil North Am 1993;4:475-
91.
17. Carr JH, Shepherd RB, Nordholm L, Lynne D. Investigation of a
new motor assessment scale for stroke patients. Phys Ther 1985;
65:175-9.
18. Holden MK, Gill KM, Magliozzi MR, Nathan J, Piehj-Baker L.
Clinical gait assessment in the neurologically impaired: reliability
and meaningfulness. Phys Ther 1986;64:1530-9.
19. Ottenbacher KJ, Hsu Y, Granger CV, Fiedler RC. The reliability
of the Functional Independence Measure: a quantitative review.
Arch Phys Med Rehabil 1996;77:1226-32.
20. Stineman MG, Jette A, Fiedler R, Granger C. Impairment-specific
dimensions within the Functional Independence Measure. Arch
Phys Med Rehabil 1997;78:636-43.
21. Stineman MG, Shea JA, Jette A, et al. The Functional Indepen-
dence Measure: tests of scaling assumptions, structure, and reli-
ability across 20 diverse impairment categories. Arch Phys Med
Rehabil 1996;77:1101-8.
22. Granger CV, Cotter AC, Hamilton BB, Fiedler RC. Functional
assessment scales: a study of persons after stroke. Arch Phys Med
Rehabil 1993;74:133-8.
23. Oczkowski WJ, Barreca S. The Functional Independence Mea-
sure: its use to identify rehabilitation needs in stroke survivors.
Arch Phys Med Rehabil 1993;74:1291-4.
24. Heinemann AW, Linacre JM, Wright BD, Hamilton BB, Granger
C. Prediction of rehabilitation outcomes with disability measures.
Arch Phys Med Rehabil 1994;75:133-43.
25. Stineman MG, Escarce JJ, Goin JE, Hamilton BB, Granger DV,
Williams SV. A case-mix classification system for medical reha-
bilitation. Med Care 1994;32:366-79.
26. Stineman MG, Goin JE, Granger CV, Fiedler R, Williams SV.
Discharge motor FIM–function related groups. Arch Phys Med
Rehabil 1997;78:980-5.
27. Loewen SC, Anderson BA. Reliability of the Modified Motor
Assessment Scale and the Barthel Index. Phys Ther 1988;68:
1077-81.
28. Poole JL, Whitney SL. Motor assessment scale for stroke patients:
concurrent validity and interrater reliability. Arch Phys Med Re-
habil 1988;69:195-7.
29. Malouin F, Pichard L, Bonneau C, Durand A, Corriveau D.
Evaluating motor recovery early after stroke: comparison of the
Fugl-Meyer assessment and the Motor Assessment Scale. Arch
Phys Med Rehabil 1994;75:1206-12.
30. Loewen SC, Anderson BA. Predictors of stroke outcome using
objective measurement scales. Stroke 1990;21:78-81.
31. Holden MK, Gill KM, Magliozzi MR. Gait assessment for neu-
rologically impaired patients. Phys Ther 1984;66:35-40.
32. Roth EJ, Merbitz CT, Grip JC, et al. The timer-logger-communi-
cator gait monitor: recording temporal gait parameters using a
portable computerized device. Int Disabil Stud 1990;12:10-6.
33. Mayo NE, Wood-Dauphinee SA, Gordon C, Higgins J, McEwen
S, Salbach N. Disablement following stroke. Disabil Rehabil
1999;21:258-68.
34. Evans M, Goldie P, Hill K. Systematic and random error in
99
repeated measurements of temporal and distance parameters of
gait following stroke. Arch Phys Med Rehabil 1997;78:725-9.
35. Wade DT, Wood VA, Heller A, Maggs J, Hewer RL. Walking
after stroke. Scand J Rehabil Med 1987;19:25-30.
36. Collen FM, Wade DT, Bradshaw CM. Mobility after stroke:
reliability of measures of impairment and disability. Int Disabil
Stud 1990;12:6-9.
37. Bohannon RW. Gait performance of hemiparetic stroke patients:
selected variables. Arch Phys Med Rehabil 1987;68:777-81.
38. Brandstater ME, de Bruin H, Gowland C, Clark BM. Hemiplegic
gait: analysis of temporal variables. Arch Phys Med Rehabil
1983;64:583-7.
39. Corcoran PJ, Jebsen RH, Brengelmann GL, Simons BC. Effects of
plastic and metal leg braces on speed and energy cost of hemipa-
retic ambulation. Arch Phys Med Rehabil 1970;51:69-77.
40. Dettman MA, Linder MT, Sepic SB. Relationships among walk-
ing performance, postural stability, and functional assessments of
the hemiplegic patient. Am J Phys Med Rehabil 1987;66:77-90.
41. Lehmann JF, Condon SM, Price R, deLateur BJ. Gait abnormal-
ities in hemiplegia: their correction by ankle foot orthoses. Arch
Phys Med Rehabil 1987;68:763-71.
42. Friedman PJ. Gait recovery after hemiplegic stroke. Int Disabil
Stud 1991;12:119-22.
43. Richards CL, Malouin F, Dumas F, Tardif D. Gait velocity as an
outcome measure of locomotor recovery after stroke. In: Craik
RL, Oatis CA, editors. Gait analysis: theory and application. St
Louis: Mosby; 1995. p 355-64.
44. Goldie PA, Matyas TA, Evans OM. Deficit and change in gait
velocity during rehabilitation after stroke. Arch Phys Med Rehabil
1996;77:1074-82.
45. Hill K, Ellis P, Bernhardt J, Maggs P, Hull S. Balance and
mobility outcomes for stroke patients: a comprehensive audit.
Aust J Physiother 1997;43:173-80.
46. Velozo CA, Kielhofner G, Lai J. The use of Rasch analysis to
produce scale-free measurement of functional ability. Am J Occup
Ther 1999;53:83-90.
47. Fisher WP, Harvey RF, Taylor P, Kilgore KM, Kelly CK. Reha-
bits: a common language of functional assessment. Arch Phys
Med Rehabil 1995;76:113-22.
48. Grimby G, Andren E, Daving Y, Wright B. Dependence and
perceived difficulty in daily activities in community living stroke
survivors two years after stroke. Stroke 1998;29:1843-9.
49. Silverstein B, Kilgore KM, Fisher WP, Harley JP, Harvey RF.
Applying psychometric criteria to functional assessment in med-
ical rehabilitation: 1. Exploring unidimensionality. Arch Phys
Med Rehabil 1991;72:631-7.
50. Weiss DJ, Yoes ME. Item response theory. In: Hambleton RK,
Zaal JN, editors. Advances in educational and psychological test-
ing: theory and applications. Boston: Kluwer Academic; 1991. p
69-95.
51. Wright BD, Masters GN. Rating scale analysis. Chicago: Mesa Pr;
1982.
52. Grimby G, Gudjonsson G, Rodhe M, Sunnerhagen KS, Sundh V,
Ostensson ML. The Functional Independence Measure in Sweden:
experience for outcome measurement in rehabilitation medicine.
Scand J Rehabil Med 1996;28:52-62.
53. Chang W-C, Chan C. Rasch analysis for outcomes measures:
some methodological considerations. Arch Phys Med Rehabil
1995;76:934-9.
54. McGinley J. Criteria for community ambulation: implications for
stroke rehabilitation [dissertation]. Melbourne (Aust): La Trobe
Univ; 1991.
55. Fiedler R. The Rasch measurement model. UDS Update 1993;7:
1-10.
Suppliers
a. SPSS Inc, 233 S Wacker Dr, 11th F1, Chicago, IL 60606.
b. Adams R. Khoo S. QUEST: the interactive test analysis system.
Melbourne (Aust): Australian Council for Educational Research;
1994.
Arch Phys Med Rehabil Vol 83, January 2002