Sie sind auf Seite 1von 22

Phys Occup Ther Pediatr Downloaded from informahealthcare.

com by University of Louisville on 01/18/15

Goal Attainment Scaling:


Its Use in Evaluating
Pediatric Therapy Programs
Gillian A. King
Janette McDougall
Robert J. Palisano
Janet Gritzan
For personal use only.

Mary Ann Tucker

ABSTRACT. Goal attainment scaling is becoming an increasingly


popular technique for evaluating the functional goal attainment of chil-
dren receiving pediatric therapy services. This article reports on the
experiences of the authors in conducting formal program evaluations
using this individualized measurement approach. Goal attainment scal-
ing is described, its utility is assessed, and issues in its use are identi-
fied. The article considers the pros and cons of the technique, highlights
the key decisions required to use goal attainment scaling effectively,
and provides standard criteria and procedures for its use in pediatric
settings. [Article copies available for a fee from The Haworth Document Deliv-

Gillian A. King, PhD, is Investigator, CanChild Centre for Childhood Disability


Research, McMaster University and Research Program Manager, Thames Valley
Children’s Centre. Janette McDougall, MA, is Research Associate, Thames Valley
Children’s Centre. Robert J. Palisano, ScD, PT, is Professor and Director, Program in
Movement Science, Medical College of Pennsylvania, Hahnemann University, and
Co-Investigator with CanChild. Janet Gritzan, MClSc, is Speech-Language Patholo-
gist and Mary Ann Tucker, BSc, is Manager of School-Age and Adolescent Services,
both at Thames Valley Children’s Centre.
The authors sincerely thank the service providers, parents, teachers, and children
who assisted with this research. The authors also thank Paul Stolee for his useful
feedback on this article.
This research has been made possible through funding provided by CanChild and
Thames Valley Children’s Centre.
Physical & Occupational Therapy in Pediatrics, Vol. 19(2) 1999
E 1999 by The Haworth Press, Inc. All rights reserved. 31
32 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

ery Service: 1-800-342-9678. E-mail address: getinfo@haworthpressinc.com


<Website: http://www.haworthpressinc.com>]

KEYWORDS. Goal attainment scaling, therapy, program evaluation,


functional outcomes, children with special needs, rehabilitation

Many questions need to be considered when designing outcome


evaluation studies for children receiving occupational, physical, or
speech-language therapy, either in the community or in a health care
centre. One of the fundamental questions is whether to use a standard-
ized or individualized measurement approach–or both.1,2 Individual-
For personal use only.

ized methods indicate whether single individuals have achieved the


goals of intervention. These methods also provide clear goals and
priorities for intervention, ensure the ongoing relevance of the child’s
goals, and reflect a client-centered perspective to service delivery.1,3
One of the most widely-used individualized approaches is goal at-
tainment scaling (GAS),1 which provides an individualized, criterion-
referenced measure of change. The GAS procedure involves: (a) defin-
ing a unique set of goals for each child, (b) specifying a range of
possible outcomes for each goal (on a scale recommended to contain
five levels, from *2 to +2),4 and (c) using the scale to evaluate the
child’s functional change after a specified intervention period. As we
have used it, a score of *2 represents the child’s baseline level before
intervention, *1 represents improvement that is less than the ex-
pected level of attainment after intervention, 0 represents the expected
level of attainment after intervention, and +1 and +2 represent levels
of attainment that exceed expectations but represent outcomes that the
child is thought to be capable of achieving under favorable conditions.
(Examples of these five-point GAS scales written for children receiv-
ing occupational, physical, and speech-language therapy are presented
in Tables 1 to 3. The examples reflect subcategories of goals for three
areas (productivity, mobility, and communication), which are often
targeted for intervention in the school setting.)
GAS was initially used to measure the impact of intervention in the
mental health field.5 Since then, it has been used widely to evaluate
health services, educational programs, and social services.6,7 In 1979,
GAS was considered the most popular outcome evaluation technique
in the human sciences.8
There are two main reasons for measuring outcomes in the field of
King et al. 33
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

TABLE 1. Examples of Goal Attainment Scales for Children Receiving Occupa-


tional Therapy Services to Address Classroom Productivity

Goal Subcategory
Attainment Level Score Written Communication Functional Fine Organizational Skills
Motor Skills

Baseline *2 Writes some letters in Cuts within a 1/4I wide Organizes desk (all notes
isolation (i,e,u,l,t) with boundary with verbal and books stacked neatly)
verbal and visual cueing cues to turn page with with physical assistance,
nonĆdominant hand at standby supervision,
corners scheduling, and a
checklist

Less than expected *1 Writes all letters of the Cuts within a 1/8I wide Organizes desk (all notes
outcome alphabet in isolation with boundary with verbal and books stacked neatly)
For personal use only.

verbal and visual cueing cues to turn page with with standby supervision,
nonĆdominant hand at scheduling, and a
corners checklist

Expected outcome O Writes all letters of the Cuts within a 1/16I wide Organizes desk (all notes
alphabet in isolation with boundary with verbal and books stacked neatly)
visual cueing cues to turn page with with standby supervision
nonĆdominant hand at and a checklist
corners

Greater than +1 Writes all letters of the Cuts within a 1/16I wide Organizes desk (all notes
expected outcome alphabet with grouping of boundary and and books stacked neatly)
2 to 3 letters with visual independently turns page with standby supervision
cueing with nonĆdominant hand
at corners

Much greater than +2 Writes all letters of the Cuts on a regular penĆ Organizes desk (all notes
expected outcome alphabet with grouping of width line and and books stacked neatly)
3 to 4 letters independently turns page independently (general
independently with nonĆdominant hand instructions from teacher)
at corners

pediatric therapy: (1) to evaluate outcomes for a specific child (to


improve services to that child), and (2) to determine the effectiveness
of a service or program as a whole.9 GAS can be used for both
purposes–to document therapeutic change in individual children or to
examine change in groups of children. The focus of this article is on
the latter–the use of GAS in program evaluation studies.
GAS has been used in a number of studies evaluating pediatric
therapy services and programs, most fairly recent.10-17 Four studies
took place in early intervention settings and focused on the attainment
of motor development goals by infants or preschool children.14-17
Three other studies focused on the attainment of rehabilitation therapy
34 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

TABLE 2. Examples of Goal Attainment Scales for Children Receiving Physical


Therapy Services to Address Mobility

Goal Subcategory

Attainment Level Score Functional Gross Ambulation Transitions


Motor Skills

Baseline *2 Descends 6 stairs, Ambulates (with walker) Transfers self from walker
holding the handrail, from the resource room to to desk chair with verbal
utilizing a stepĆtoĆstep classroom in 8.5 minutes assistance and physical
pattern, with one hand with supervision and verbal assistance (other person
held cueing holding trunk and placing
feet on/off footrest with
weight supported)
For personal use only.

Less than expected *1 Descends 6 stairs, Ambulates (with walker) Transfers self from walker
outcome holding the handrail, from the resource room to to desk chair with verbal
utilizing a stepĆtoĆstep classroom in 6 to 8 minutes cueing and physical
pattern, with standby with supervision and verbal assistance (holding trunk
assistance cueing and placing feet on/off
footrest)

Expected outcome 0 Descends 6 stairs, Ambulates (with walker) Transfers self from walker
holding the handrail, from the resource room to to desk chair with verbal
utilizing a reciprocating classroom in 5 minutes or cueing and physical
pattern, with one hand less with supervision and assistance (placing feet
held verbal cueing on/off footrest)

Greater than +1 Descends 6 stairs, Ambulates (with walker) Transfers self from walker
expected outcome holding the handrail, from the resource room to to desk chair with verbal
utilizing a reciprocating classroom in 5 minutes or cueing and no physical
pattern, with standby less with supervision and no assistance
assistance verbal cueing

Much greater than +2 Descends 6 stairs, Ambulates (with walker) Transfers self from walker
expected outcome holding the handrail, from the resource room to to desk chair independently
utilizing a reciprocating classroom in 5 minutes or
pattern, independently less independently (no
supervision/verbal cueing)

goals by school-aged children.10,11,13 A recent study by Brown et al.12


examined the effects of physical therapy intervention on the attain-
ment of gross motor goals in individuals ranging from 3 to 30 years
with severely limited physical and cognitive abilities. Thus, there is
growing evidence that GAS is a useful way to measure therapeutic
change. GAS has shown that children receiving pediatric therapy in-
tervention achieve goals in a variety of spheres–classroom productiv-
ity, mobility, and communication. Still, questions remain about how to
properly conduct GAS and concerns are often raised about its reliabil-
King et al. 35
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

TABLE 3. Examples of Goal Attainment Scales for Children Receiving


Speech-Language Therapy Services to Address Communication

Goal Subcategory

Attainment Level Score Speech Sound Production Speech Sound Production Speech Sound Production
of Blends in Final Word Position in Initial Word Position

Baseline *2 Produces ``l'' blends with Produces ``m'' sound in Produces ``f'' sound in
85% accuracy in final word position with initial word position in
structured phrases (i.e., 80% accuracy at the imitated phrases with
17 of 20 trials correct) spontaneous phrase level 85% accuracy (i.e., 17 of
(i.e., 8 of 10 trials 20 trials correct)
correct)

Less than expected *1 Produces ``l'' blends with Produces ``m'' sound in Produces ``f'' sound in
For personal use only.

outcome 85% accuracy in imitated final word position with initial word position in
phrases 80% accuracy at the imitated sentences with
imitated sentence level 85% accuracy

Expected outcome 0 Produces ``l'' blends with Produces ``m'' sound in Produces ``f'' sound in
85% accuracy in final word position with initial word position in
structured sentences 80% accuracy at the structured sentences with
structured sentence level 85% accuracy

Greater than +1 Produces ``l'' blends with Produces ``m'' sound in Produces ``f'' sound in
expected outcome 85% accuracy in final word position with initial word position in
spontaneous sentences 80% accuracy at the spontaneous sentences
spontaneous sentence with 85% accuracy
level

Much greater than +2 Produces ``l'' blends with Produces ``m'' sound in Produces ``f'' sound in
expected outcome 85% accuracy at the final word position with initial word position
spontaneous 80% accuracy at the when describing a picture
conversational level conversational level with 85% accuracy

ity and validity.8,18 The appropriate use of GAS depends on a clear


understanding of its strengths and weaknesses19 and thoughtful deci-
sion-making concerning the issues that arise when using GAS in pe-
diatric settings.
The aim of this article is to assist potential users to decide whether
or not to use GAS and to provide information about how to implement
GAS with a minimum of bias. Practical tips and guidelines are pre-
sented based on our combined experience with five studies that used
GAS in three types of pediatric settings: (a) a multidisciplinary,
school-based therapy program encompassing occupational, physical,
and speech-language therapy services,10,11 (b) physical therapy ser-
vices for infants addressing mobility and other developmental
goals,15,16 and (c) a residential setting for children and adults with
36 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS

severely limited physical and cognitive abilities.12 These studies were


Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

conducted in the United States or in Canada. Based on feedback from


therapists involved in these studies, our own observations, and guide-
lines in the general GAS literature, we have devised standard criteria
and procedures for the application of GAS in the formal evaluation of
pediatric therapy programs.
Other articles have provided information on the technical aspects
of using GAS, specifically the steps involved in setting the scales and
rating outcomes.13-23 Most of these articles have dealt with the ap-
plication of GAS in non-therapeutic settings, such as special educa-
tion services,20,22 human services programs,21 and mental health
services.23 On a practical level, the most useful publications are
For personal use only.

those by Kiresuk and Lund,24 who discuss typical errors in creating


GAS scales and answer commonly-asked questions about biases in
goal setting and rating, and Smith,23 who provides guidelines around
who should set goals and who should rate goal attainment.
To date, publications have not addressed the specific issues that
arise in using GAS in pediatric therapy: Issues such as how to ap-
proach goal selection and goal definition for children with special
needs, and where to set the baseline on the GAS scale when a child
has a progressive or non-progressive condition. General guidelines
for the use of GAS (such as those laid out by Kiresuk et al.6) must be
adapted to particular intervention settings and populations because of
the different issues that arise in each setting.25 The present article
identifies key decision points and provides criteria and guidelines for
the systematic use of GAS in pediatric therapy settings, including
school-based therapy programs and early intervention programs.

DECIDING WHETHER OR NOT TO USE GAS


TO EVALUATE PEDIATRIC THERAPY PROGRAMS
A number of organizational conditions are necessary for the suc-
cessful implementation of a program evaluation study using GAS.
They include: (a) a motivated team whose members are committed to
the evaluation and who share a common drive toward improvement of
therapy services, (b) adequate orientation and training of therapists,
(c) the availability of people to coach therapists in the proper applica-
tion of GAS, so that both technical and practical issues are addressed
in an integrated fashion, and (d) sufficient resources allocated to do the
King et al. 37

job.24 We often have employed a study coordinator to oversee the data


Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

collection and training and to ensure the technical quality of the GAS
scaling and rating procedures.
The pros and cons of using GAS have been outlined in many ar-
ticles.1,6,8,13,15,16,18,19,23,26 A primary strength of GAS is its ability to
measure change in performance, whereas most standardized measures
are discriminative tools designed to measure post-intervention status
(based on norms for children without special needs) and have not been
validated as responsive to clinically significant change.6,16,19 Clinical
significance refers to the magnitude of an effect in real-world terms.27
GAS is criterion-referenced, rather than norm-referenced, making it
For personal use only.

potentially responsive to small changes that are perceived by children,


families, and teachers as important for daily function. GAS may be
particularly useful for children with low cognitive functioning, since
standardized measures may not be sensitive to the small but meaning-
ful changes targeted for these individuals.
Relatively few standardized measures address functional outcomes
that are appropriate for children with special needs within a context
such as school (e.g., children’s ability to walk from the bus to the
school classroom). Standardized assessments of function often are
designed to measure a broad range of abilities. Some of these areas
may not reflect therapy goals and not be relevant to particular chil-
dren. Pediatric studies provide a fair amount of evidence that GAS and
parallel standardized measures provide scores that are only moderate-
ly correlated with one another.10,11,15-17 For formal program evalua-
tion purposes, we recommend the use of both GAS and standardized
measures.10,11
In addition to the ability to measure change in the performance of
individual children, GAS has other advantages: clinical utility, rele-
vance, client involvement, and acceptability.1,23 GAS is ideally suited
to collaborative goal setting between a therapist, child, parent, and
other professionals (such as teachers). Its collaborative use reflects a
client- or family-centered approach to service delivery.1,3,28 Other
potential advantages of GAS include: (a) improved clarity of therapy
objectives for both therapists and clients, (b) improved conceptualiza-
tion and delivery of the intervention, (c) more realistic client and
therapist expectations of therapy, (d) increased client satisfaction, and
(e) increased motivation of the client toward improvement, provided
by the very existence of goals.1,6,13,26
38 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

There are a number of potential limitations in using GAS in a


program evaluation study: (a) biases in goal scaling and rating can
occur, (b) training and standardized implementation procedures are
required, which are time-consuming (when therapists are unfamiliar
with GAS), and (c) GAS can interfere with day-to-day practice be-
cause, when conducting a program evaluation, therapists should not
modify a goal in the course of the intervention. There are two reasons
for this. First, the study intervention period may not be sufficiently
long for change to be expected on a new or modified goal. Secondly,
therapists may elect to change goals they discover that they cannot
meet, which undermines the utility of GAS.
For personal use only.

The major drawback to GAS is the possibility of bias in the use of


the tool, which can affect its validity.8,13,16,18 Unintentional bias can
occur in goal scaling (so goals are overly easy to attain) or in goal
rating (showing children make improvements that are not in fact
real). Reliability and validity can be improved, however, by compre-
hensive training of raters, adequate definitions of the levels of goal
attainment, and the use of multiple raters.6,18 As well, there are
aspects of how services are delivered in pediatric settings that natu-
rally reduce the possibility and extent of bias in goal scaling and
rating. A collaborative goal setting model (a common feature of a
multidisciplinary, family-centered approach to service delivery)
helps to ensure that goal levels are meaningful and ratings are valid
because both are based on a consensus involving several individuals
who are knowledgeable about the child and invested in ensuring that
the child makes real gains.26,29 Thus, collaborative goal setting helps
to ensure that therapy goals are meaningful to the child and family
and not simply easy goals that therapists set on their own and can be
sure of attaining, which is a criticism raised by many.13,25
We have observed differences between the rehabilitation disciplines
with respect to the ease of writing appropriate functional goals. The
GAS scaling format appears easiest to apply for speech therapy goals
and harder to apply for physical therapy and occupational therapy
goals. As well, in King et al.,11 speech-language pathologists found it
easier to set goals that could be integrated into the child’s function in
the school setting. We speculate that the established hierarchy of the
development of speech sounds may assist speech-language patholo-
gists in setting goals in the area of articulation. Physical and occupa-
tional therapists needed to give more thought to the process of writing
King et al. 39
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

the various goal attainment levels. For instance, physical therapists


found it hard to set equal intervals between goal levels for goals
targeting unique mobility difficulties and found it hard to establish
relevant goal levels for high functioning children. Interestingly, there
has been relatively little application of GAS in the speech-language
area. The majority of applications have dealt with physical and occu-
pational therapy goals.12,14-17,29

HOW TO USE GAS APPROPRIATELY AND EFFECTIVELY


IN A PEDIATRIC CONTEXT
For personal use only.

Overview of GAS Procedure

We recommend that the child’s treating therapist participate in the


goal setting process and that a therapist not providing services to the
child do the goal rating. For each goal, the treating therapist provides a
written description of the child’s baseline level of performance. In a
family-centered service delivery model, this baseline level of perfor-
mance would be vetted with clients (i.e., teachers, parents, and often
the children themselves). The treating therapist, in conjunction with
clients, also provides a written description of the expected level of
performance for the child at the end of intervention (corresponding to
the 0 rating). We have found it most effective for the other levels of the
scale ( *1, +1, and +2) then to be written by the treating therapist in
conjunction with a person trained to oversee the quality of the GAS
scales (i.e., a person who understands the steps of the scaling proce-
dure, the necessary criteria, and pitfalls to avoid). The final step is a
peer review of the GAS scales by the therapists for each discipline
who do the ratings of goal attainment at the end of the intervention
(with the assistance of a person well-versed in the GAS scaling meth-
odology). If a formal program evaluation is being done, a research
assistant may be involved. However, any well-trained person could
help identify the goal attainment levels and assist in the peer review of
the GAS scales.

Ensuring the Technical Quality of the GAS Procedure

One needs to ensure that the goal scales are reliable (i.e., that a
rating made by one therapist observing the child’s performance is
40 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

comparable to the rating made by a different therapist) and that bias


does not occur in goal scaling (overly easy goals) or goal rating (im-
provements that are not real). The reliability and validity of GAS
scales can be improved by various procedures: ensuring that treating
therapists have a minimum level of experience so they can set realistic
goals in conjunction with children and parents; providing comprehen-
sive training to therapists; using collaborative goal setting and peer
review in the goal selection phase; ensuring well-written goals
through training, peer review, and use of a standard procedure and
checklist; and using independent raters (i.e., raters who do not have a
personal investment in the outcome).
For personal use only.

Table 4 outlines the questions that need to be considered in using


GAS effectively, criteria that should be met to ensure reliability and
validity of the procedure, and procedures and tools that can be used to
meet the criteria. These questions and criteria are based on the au-
thors’ experiences in using GAS and on recommendations in the liter-
ature.
How Much Clinical Experience Is Necessary to Set Appropriate
Goals? One year of full-time clinical experience in the setting of
interest (a school-based therapy program or an early intervention
program) is ideal. This amount of time provides therapists with
enough exposure to different types of goals and different children so
that they can estimate the performance levels that children will most
likely attain. We did not meet this criterion in our own study11 (only
83% of therapists had over a year experience in providing school-
based intervention) and realize that the experience level and number
of staff in the program implementing the evaluation are limiting
factors.
The ability to decide on the key variables that must change for goal
attainment is a skill that improves with experience. Experienced thera-
pists develop competency in assessing child and environmental factors
affecting performance, are able to generate various ideas about pos-
sible variables to change, and are able to focus in on the variable they
believe is the most amenable to change (based on their past knowledge
and experience). For example, speech-language pathologists working
on communication goals in the area of articulation select particular
target sounds based on knowledge of the developmental hierarchy in
attainment of speech sounds, the child’s stimulation potential for that
sound in isolation, the importance of that sound in increasing the
King et al. 41
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

TABLE 4. How to Use Goal Attainment Scaling Effectively

Questions to Consider Criteria to Be Met Procedures and Tools


to Help Ensure Criteria
Are Met

How much clinical Therapists should have a minimum of 1 Involve only experienced
experience is necessary to set year of pediatric experience. therapists in goal setting and
appropriate goals? scaling.

How much instruction and Therapists should have approximately 7 See Table 5 for
training is necessary for hours of specific training in the use of recommended orientation
therapists? GAS. and training steps.

How can one ensure adequate The process of selecting goal areas should
goal selection? ensure that:
For personal use only.

1. Goals are meaningful and relevant to Employ collaborative


others. goal setting (therapists
select broad goal areas in
conjunction with
knowledgeable others
such as the teacher,
parent, and/or child).
2. Goals make sense from a conceptual
point of view (e.g., if the goals of the Use peer review of goal
program are to improve dayĆtoĆday content.
function, then the majority of goals set
should be functional in nature, rather
than impairmentĆbased).

How can one ensure adequate Each of the levels on the scale should: Each criterion can be
goal scaling (i.e., adequately assured through the use of
written goal levels on the 1. be written as clearly as possible, in three interĆconnected
fiveĆpoint scale)? concrete behavioral terms procedures and tools:
2. specify an observable behavior of the
child Therapist training
3. be written in the present tense
4. be achievable or realistically possible Peer review to ensure the
adequacy of the goal
The scale as a whole should: scales
1. have levels that reflect clinically
meaningful gradations of Use of a standard
improvement procedure and checklist
2. have approximately equal intervals to review the technical
between the goal attainment levels adequacy of written goal
(i.e., the change from +1 to + 2 should scales (see Figure 1)
be similar to that between *2 to *1, etc.)
3. specify a time period for achievement
4. reflect a single dimension of change
(as long as a goal remains meaningful),
keeping other variables constant
5. not reflect attainment that is
dependent on the therapist's physical
assistance (unless the assistance of
others is a written part of the goal)
42 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

TABLE 4 (continued)

Questions to Consider Criteria to Be Met Procedures and Tools


to Help Ensure Criteria
Are Met

How can one ensure adequate 1. Ratings should be done by therapists Use independent raters.
goal rating? āĂnot involved in providing the intervention.
2. Reduce the ``performance demands" of
ĂāĂĂ the visiting independent rater.

How is the summary score Clients' individual outcome scores need to Ensure that you have
determined? be aggregated in some way, preferably thought of the data analysis
using TĆscores. stage and have access to a
statistical software package.
For personal use only.

child’s intelligibility, and the consensus of the caregiver. Determining


a child’s potential for change during the intervention period is based
on clinical judgment (i.e., experience, the underlying cause of the
child’s articulation disorder, the type of error with the targeted sounds,
the number of other sounds that are in error).
How Much Instruction and Training Is Necessary for Therapists?
Orientation and training of therapists is necessary for the successful
use of GAS.13 The steps and time requirements included in Table 5 are
based on Kiresuk et al.’s6 recommendations and our own experiences.
Many therapists play the dual role of treating therapist and rating
therapist (only for children to whom they do not provide intervention),
which requires approximately 12 hours of training.
The training procedures in Table 5 incorporate the recommenda-
tions of therapists involved in our evaluation studies. For instance,
therapists recommended that a list of common errors in creating GAS
scales (and solutions) would be useful (Table 6), as would an invento-
ry of potential goals.
To develop skill in writing the scales, we recommend the use of
small group sessions in which therapists practice putting goals into
GAS format (i.e., identifying five levels of possible goal attainment).
A series of case scenarios can be used for practice and discussion.
Therapists also can practise putting goals in GAS format for two or
three children from their caseload. At this point, it is useful for them to
have access to a resource binder with examples of the five-point rating
King et al. 43
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

TABLE 5. Orientation and Training of Therapists to the Goal Attainment Scaling


Procedure

All Therapists General Orientation (2 hours)

S Orientation session

S Handout
S examples of goals
S specific guidelines
S list of common errors

Treating Therapists-Goal Setting Skill Acquisition (3 hours)


S Small group practice
S reviewing types of goals
S setting goal levels
S use of actual case material
For personal use only.

S OneĆtoĆone guidance with trainer


Skill Maintenance (2 hours)
S Continued monitoring of goal setting
S Question and answer sessions

Rating Therapists-Goal Rating Skill Acquisition (4 hours)


S Review goals with trainer
S peer review of goal writing
S familiarizing raters with the goals that they will
be rating
S Review of goal rating procedure
S Handout
Skill Maintenance (1 hour)
S Question and answer sessions

scale applied to goals in particular areas (e.g., mobility goals, commu-


nication goals).
How Can One Ensure Adequate Goal Selection? It is important that
the selected goals are meaningful to the child and family and reflect
the primary focus of therapy for the child. We therefore recommend
that treating therapists be involved in goal selection rather than inde-
pendent goal setters.30
How Can One Ensure Adequate Goal Scaling? The literature refers
to six criteria for good goal writing: Goals should be relevant, under-
standable, measurable, behavioral, attainable, and time-limited.13,18
We have gone beyond these general criteria to specify criteria relevant
to each of the goal levels and those dealing with the properties of the
scale as a whole (see Table 4).
Three criteria concerning the scale as a whole require more ex-
planation. First, although the GAS scale examples in Tables 1 through
3 did not specify a time period, it should be noted that a standard time
44 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

TABLE 6. Common Errors in Creating GAS Scales

Error Description Solution

Overly Generalized If the expected level (i.e., 0 level) of a scale The expected level of a scale
Goals is written in very general terms (e.g., ``walks should be written as clearly as
a greater distance in a set period with possible (e.g., ``walks with
assistance"), it will be difficult or platform walker 100 metres in six
impossible to create the remaining scale minutes with two hands on walker
points, therefore making the goal unmeasurable. to assist with steering").

Overly Technical A goal setter may use terms specific to Write goals in common terms,
Goals his/her profession in creating a scale that the especially if the goal rater is not of
goal rater is not familiar with. the same professional background
as the goal setter.

Multiple Variables A scale may include two or more variables Decide on one variable by which to
of Change of change. This could be problematic if the measure change in performance
For personal use only.

scale is written so that change is expected to and hold others constant. If in


occur simultaneously on these variables. doing so, the goal does not remain
meaningful, two variables could
change within in a single scale,
provided each scale level differs on
only one variable.

Unequal Scale A scale may be created where the amount of Aim for clinically equal intervals
Intervals clinical change is greater between, say, the between all levels of the scale.
+1 and +2 levels than the amount of change
between the *2 and*1 levels.
Clinically Irrelevant A scale may be created where one or more The amount of change between all
or Unrealistic Scale of the levels represents an amount of change scale levels needs to be clinically
Levels that would not be clinically relevant to the relevant and all levels should be
child (i.e., the amount of change is too small achievable for the child.
to matter) or the amount of change is
unrealistic for the child (i.e., the amount of
change is too great).
Using Different A GAS scale may be written with the *2 All scale levels should be phrased
Tenses (i.e., Past, level written in one tense and all other levels in the present tense, in order for
Present, Future) in another tense, which could be confusing evaluation to make sense at
When Writing Scale and bias the goal rater. different time points (i.e., ``walks . . .'').
Levels
Redundant or A scale may be written where a child could Be careful not to create scale levels
Incomplete Scale be scored on two levels at the same time that are redundant or incomplete.
Levels (e.g., the +1 level has walking distances Careful wording (e.g., +1 would be
specified between ``40 and 50 metres'' and the ``more than 40 metres and up to 50
+2 level specifies distances between ``50 and metres" and +2 would be ``more than
metres"). If a child walks exactly 50 metres, 50 metres and up to 60 metres")
both the +1 and the +2 level would be or specific instructions to the
correct. On the other hand, a gap could be rater (e.g., if a child obtains a
present in the scale where a child could not midway point between two levels,
be scored on any level (e.g., the +1 score the child at the lower level)
specifies walking distances between ``40 and will be of benefit.
50 metres" and the +2 specifies distance
between ``60 and 70 metres"; if a child
walks 55 metres, neither the +1 nor the +2
level is correct).
Blank Scale Levels It may be difficult to write the more extreme Be careful to set goals where it is
levels of a scale, tempting the goal setter to possible to complete all scale
leave these levels blank. If a child happens to levels.
achieve an upper or lower extreme, it would
be impossible to rate the child's performance.
King et al. 45
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

period (the length of the intervention period as prescribed by the study


design) was set for all goals.
An example may clarify criterion 4. The following goal includes
three possible variables of change, namely distance, time, and level of
assistance: ‘‘Walks 100m with platform walker in 8 minutes with two
hands on walker to assist with steering.’’ In order to write an appropri-
ate scale, the therapist must decide which is the most important vari-
able of change. That variable would then be altered in each of the
written goal levels, with the other variables held constant.
With respect to criterion 5, if physical assistance is required, this
should be explicitly stated in the written goal, either as a constant
For personal use only.

factor or as the variable that changes over time. The key idea is that
goal attainment levels should reflect change in the child’s behavior,
not unacknowledged variations in the therapist’s physical assistance. It
is permissible to write goals where the physical assistance of someone
else is explicitly stated as the variable that changes over time, so that
the goal scale shows meaningful changes in the level of assistance the
child requires to perform a task. An example of the gradations that
could be included in a scale focusing on changes in level of assistance
is: physical and verbal assistance required (*2), verbal assistance
with checklist required (*1), checklist and verbal cueing or prompt-
ing required (0), verbal cueing/prompting required (+1), and com-
pletely independent (+2).
According to strict research methodology,8 the therapist who sets
the levels of goal attainment should not be the same therapist who
provides the treatment. This is a difficult criterion to meet since, in
actual clinical practice, the treating therapist is involved in goal set-
ting, often in conjunction with the client.10,11,23 This is defensible
when one involves an independent rating therapist and a trainer/study
coordinator in the review of the goal attainment levels (who follows a
standard procedure with set criteria). In our experience, even highly
experienced therapists have some difficulty identifying the baseline
and expected levels of the goals. Assistance and review by others is
useful, appreciated, and necessary. Figure 1 provides a checklist that
can be used in the review of written goals by a trainer and independent
rating therapist.
How Can One Ensure Adequate Goal Rating? Cardillo30 addresses
the selection of raters and the decision about the timing of the goal
rating session, but provides little information on how to conduct the
46 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

FIGURE 1. Goal Attainment Scaling Checklist–Goal Review Procedure

Name of Participant:

S Therapy Goal: Expected Outcome (i.e., a score of 0)

As a whole, the scale must meet the following criteria:

Criteria Criterion Criterion Comments


Met Not Met

The amount of change between the levels is


clinically important.
For personal use only.

There are approximately equal intervals


between the goal attainment levels.

There is a time period for achievement of the


goal.

The scale reflects a single dimension of


change (or, if not feasible, each scale level
reflects a single dimension of change).

Each level on the scale must meet the following criteria:

Criteria Criterion Criterion Comments


Met Not Met

Be written in concrete behavioral terms


Specify an observable behavior of the child
Be written in the present tense
Be achievable or realistically possible

actual goal rating session. Our experiences have led to some recom-
mendations. First, it is important to consider the child’s view of the
rating situation. When children are aware that a new person is coming
to watch a session, they may be very motivated to perform well for
this visiting person. The treating therapist can reduce the ‘‘hype’’
regarding the rater’s visit by informing the child in advance of the visit
and assuring the child that his/her regular performance is what is
called for. The rating therapist should minimize the effect of his/her
presence by sitting quietly in the back of the room and making notes
discreetly.
King et al. 47
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

A second issue concerns the behavior of the treating therapist in


the session. For goals whose attainment cannot be observed under
naturally occurring circumstances at school, we have found it most
appropriate for the treating therapist to interact with the child and
request performance of the behavior outlined in the goal. (Some
goals require that the child be set to the task because of infrequent
naturally occurring opportunities to display the behavior.) The treat-
ing therapist begins by orienting the child to the task. If prompting,
cueing or some other support is required, the therapist starts with the
0 level of the scale and prompts for performance up or down the scale
depending on the child’s success at the 0 level. Brown et al.12 also
For personal use only.

used a prompting procedure and allowed up to three trials per goal


(for individuals with severely limited physical and cognitive abili-
ties). Similarly, we have found that children may need more than one
attempt to demonstrate their true ability. Some children react to the
presence of the rating therapist by showing silly behavior and two or
three attempts are required before they calm down and demonstrate
true performance.
How Is the Summary Score Determined? For program evaluation
purposes, users need to calculate an appropriate summary score to
reflect the overall goal attainment of children in the therapy or early
intervention program. The recommended procedure is to convert chil-
dren’s outcome scores into aggregate T-scores (see Cardillo and
Smith,4 for a complete discussion of T-scores and other summary
scores). T-scores can be calculated using a statistical computer soft-
ware package such as the Statistical Package for the Social Sciences
(SPSS). Mean aggregate T-scores facilitate reliability analyses and
comparisons across children, and provide an overall evaluation of
children’s performance. Another advantage of using T-scores is that
they can be compared to other standardized scores.
T-scores can be computed using the formula developed by Kiresuk
and Sherman:5

(10  WiXi)
T = 50 + p
(1 * r) Wi 2 + r (Wi 2)

In this formula, 50 represents the mean and 10 is the standard


deviation. Wi represents the weighing for a particular goal and Xi
48 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

represents the score for each goal. The r represents the expected over-
all intercorrelation among outcome scores. The formula for computing
the T-score assumes a relatively low correlation among goals of .30.
Kiresuk and Sherman5 found this correlation useful because it yields a
standard deviation of 10 units. Cardillo and Smith4 strongly recom-
mend against the use of differential weighing of GAS goals. The use
of this formula may appear time-consuming and difficult, but the need
for manual computation is rare. If goals are not weighted and the
suggested intercorrelation of .30 is used, tables are available that allow
the quick and easy conversion of outcome scores into T-scores for
clients with up to eight scored scales (see Kiresuk et al.6).
For personal use only.

Conceptual Issues Arising in the Use of GAS


Validity Issues: How to Ensure that Goal Attainment Is Real. Two
basic strategies can be used to address the issue of validity:
(1) supplement the use of GAS with measures that provide more
conventional estimates of post-treatment status (i.e., standardized
measures) to provide a more comprehensive assessment of outcome,6
or (2) employ randomly selected control goals (after Brown et al.12).
In Brown et al., therapists created GAS scales both for treatment
goals that were practiced in the physical therapy setting and for
control goals that were set but not addressed in practice. The sub-
jects’ progress on treatment goals was significantly greater than their
progress on the control goals, suggesting that the therapy interven-
tion was the factor contributing to improved goal attainment and that
the goal attainment was real.
How to Set the Baseline in the GAS Scale. This is an important
consideration. When no deterioration is expected in children’s perfor-
mance, such as when the child has a non-progressive, chronic condi-
tion, *2 can be defensibly used as the child’s baseline.10,11 When
evaluating goal attainment of individuals with severely limited physi-
cal and cognitive abilities, Brown et al.12 used *2 as a baseline but
captured lower performance on a test day by using a score of *3.
When evaluating the performance of children with progressive condi-
tions who may deteriorate in function over time, it would make sense
to set the baseline at *1. If the expectation is that decreased perfor-
mance will occur and the goal is to minimize performance loss, it
would make sense to write goal levels in terms of gradations of dimin-
ished performance.
King et al. 49
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

How to Best Demonstrate Reliability. It is important to report the


reliability of the goal ratings. There are various types of reliability
including inter-rater stability (independent raters’ agreement over
time) and inter-rater reliability (independent raters’ agreement at
the same point in time).8 In King et al.,10 we computed a measure of
inter-rater stability, which involved correlating T-scores based on
the ratings of the treating and rating therapists obtained on separate
occasions. We found a correlation of .6, which indicates a moderate
level of agreement between the rating therapists and the treating thera-
pists on the amount of improvement the children made on their goals,
and is typical of estimates of stability obtained when ratings by differ-
ent individuals are obtained on different occasions.6 In our second
For personal use only.

study, we felt it was more appropriate to obtain a measure of inter-


rater reliability from two independent rating therapists who rated
the goals on the same occasion.8 Using this procedure, the inter-ra-
ter reliability (Intraclass Correlation Coefficient) was .98. To demon-
strate that the GAS scales are reliable, we recommend that others
examine and report inter-rater reliability estimates from two inde-
pendent raters.
Brown et al.12 computed measures of inter-rater reliability between
a treating therapist who directly observed levels of goal attainment in a
physical therapy setting and an independent rater who scored goal
attainment based on videotapes of the same sessions. This is a good
way to reduce performance demands or reactive behavior created by
the presence of an unfamiliar, independent rater.
What Does It Mean When the Change Exceeds Therapists’ Expecta-
tions? Studies have found that GAS ratings are often higher than the
expected level of 0 on the 5-point scale.10,11,15,16 Since the ratings
were reliable, it appears that finding change exceeding therapists’
expectations is not due to biases in the rating procedure, but rather in
the goal setting. There is therefore fairly strong evidence that expected
outcomes may be underestimated systematically by therapists. The
multiple reasons for this are hard to disentangle. Therapists may set
low levels of expected goal attainment to be cautious and ensure
success for the child. It is also possible that therapists underestimate
the therapy gains that children can make, which may be due to not
fully taking children’s motivation into account. The underestimation
of expected outcomes should be discussed with therapists during train-
ing in future studies.
50 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

How to Approach Goal Selection and Definition. When selecting a


goal, therapists need to be aware of the underlying factor that is being
targeted for change–whether the major limiting factor is the child’s
endurance or motivation, for example, or aspects of the physical envi-
ronment. Goal definition therefore is based on the therapist’s working
hypothesis of the best way to assist the child to perform the task.
The therapist’s hypothesis about the major factor limiting the child’s
successful performance is sometimes evident from looking at the goal
scale that is set. We have found that when the variable being changed
deals with a quality (such as accuracy) or a quantity (such as number of
letters formed correctly), then the working hypothesis seems to deal with
For personal use only.

some aspect internal to the child. On the other hand, when the variable
being changed deals with the level of physical or verbal assistance pro-
vided, or levels of equipment support, then the variable of change corre-
sponds to the factor of critical importance in the therapist’s view. Under
these conditions, the written goal levels provide a window on the thera-
pist’s view of the factors limiting the child’s goal achievement.

Implications for Therapists and Managers

This article should help therapists and managers make informed


decisions about whether or not to use GAS, based on an understanding
of the questions they need to ask and the requirements for using GAS
appropriately and effectively in pediatric program evaluation studies.
Moreover, the criteria and procedural guidelines presented here should
provide useful assistance to those using GAS in pediatric settings.
Information from GAS can to be used to improve programs, to assist
in meeting accreditation requirements, and to demonstrate account-
ability to governing bodies and funders.
GAS has been criticized due to its potential for bias when imple-
mented without thought and care.18 We have provided a more hopeful
or balanced perspective here. GAS can be implemented appropriately
when guidelines and standard procedures are used. Under these condi-
tions, it is well worth the effort to use GAS to evaluate pediatric
therapy programs. Service providers, managers, parents, and children
themselves benefit from the knowledge provided by goal attainment
scaling.
King et al. 51
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

REFERENCES
1. Russell D, King G, Palisano R, Law M. Measuring individualized outcomes
(Research Report No. 95-1). Hamilton, ON: McMaster University and Chedoke-
McMaster Hospitals, CanChild Centre for Childhood Disability Research; 1995.
2. Russell D, King G, Palisano R, Law M. Measuring individualized outcomes.
Proceedings of the American Academy for Cerebral Palsy and Developmental Medi-
cine. 1996; 25.
3. Zaza C, Stolee P, Prkachin K. The application of goal attainment scaling in
chronic pain settings. Journal of Pain and Symptom Management. 1999; 55.
4. Cardillo JE, Smith A. Psychometric issues. In: Kiresuk TJ, Smith A, Cardillo
JE, eds. Goal attainment scaling: Applications, theory, and measurement. Hillsdale,
NJ: Lawrence Erlbaum Associates; 1994; 173-212.
5. Kiresuk TJ, Sherman RE. Goal attainment scaling: A general method for eval-
For personal use only.

uating comprehensive community mental health programs. Community Ment Health


J. 1968; 4:443-453.
6. Kiresuk TJ, Smith A, Cardillo JE. Goal attainment scaling: Applications,
theory, and measurement. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994.
7. MacKay G, Somerville W, Lundie J. Reflections on goal attainment scaling
(GAS): Cautionary notes and proposals for development. Educational Research.
1996; 38:161-172.
8. Cytrynbaum S, Ginath Y, Birdwell J, Brandt L. Goal attainment scaling: A
critical review. Eval Q. 1979; 3:5-40.
9. Law M, King GA, MacKinnon E, Russell DJ. Quality performance: Design-
ing clinical services around person-centered outcomes. In: Gardner JF, Nudler S, eds.
Quality performance in human services: Leadership, values, and vision. Baltimore,
MD: Paul H. Brookes; 1999; 81-106.
10. King G, Tucker M, Alambets P, Gritzan J, McDougall J, Ogilvie A, Husted K,
O’Grady S, Brine M, Malloy-Miller T. The evaluation of functional, school-based
therapy services for children with special needs. A feasibility study. Phys Occup Ther
Pediatr. 1998; 18: 1-27.
11. King G, McDougall J, Tucker MA, Gritzan J, Malloy-Miller T, Alambets P,
Cunning D, Thomas K, Gregory K. An evaluation of functional, school-based thera-
py services for children with special needs. Manuscript submitted for publication;
1999.
12. Brown DA, Effgen SK, Palisano RJ. Performance following ability-focused
physical therapy intervention in individuals with severely limited physical and cogni-
tive abilities. Phys Ther. 1998; 78:934-947.
13. Clark MS, Caudrey DJ. Evaluation of rehabilitation services: The use of goal
attainment scaling. Int Rehabil Med. 1983; 5: 41-45.
14. Maloney FP, Mirrett P, Brooks C, Johannes K. Use of goal attainment scaling
in the treatment and ongoing evaluation of neurologically handicapped children. Am
J Occup Ther. 1978; 32: 505-510.
15. Palisano RJ, Haley SM, Brown DA. Goal attainment scaling as a measure of
change in infants with motor delays. Phys Ther. 1992; 72: 432-437.
16. Palisano RJ. Validity of goal attainment scaling with infants with motor de-
lays. Phys Ther. 1993; 73: 651-658.
52 PHYSICAL & OCCUPATIONAL THERAPY IN PEDIATRICS
Phys Occup Ther Pediatr Downloaded from informahealthcare.com by University of Louisville on 01/18/15

17. Stephens TE, Haley SM. Comparison of two methods for determining change
in motorically handicapped children. Phys Occup Ther in Pediatr. 1991; 11(1): 1-17.
18. Ottenbacher KJ, Cusick, A. Goal attainment scaling as a method of clinical
service evaluation. Am J Occup Ther. 1990; 44: 519-525.
19. Ottenbacher KJ, Cusick A. Discriminative versus evaluative assessment:
Some observations on goal attainment scaling. Am J Occup Ther. 1993; 47: 349-354.
20. Carr RA. Goal attainment scaling as a useful tool for evaluating progress in
special education. Except Child. 1979; October: 88-95.
21. Kiresuk TJ, Lund SH. Goal attainment scaling. In: Atkisson C, ed. Evaluation
of human service programs. New York, NY: Academic Press; 1978; 341-370.
22. Shuster SK, Fitzgerald N, Shelton G, Barber P, Desch S. Goal attainment scal-
ing with moderately and severely handicapped preschool children. Journal of the Di-
vision for Early Childhood. 1984; Winter: 26-37.
23. Smith A. Introduction and overview. In: Kiresuk TJ, Smith A, Cardillo JE,
For personal use only.

eds. Goal attainment scaling: Applications, theory, and measurement. Hillsdale, NJ:
Lawrence Erlbaum Associates; 1994; 1-14.
24. Kiresuk TJ, Lund SH. Implementing goal attainment scaling. In: Kiresuk TJ,
Smith A, Cardillo JE, eds. Goal attainment scaling: Applications, theory, and mea-
surement. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994; 119-134.
25. Lewis A, Spencer JH, Haas GL, DiVittis A. Goal attainment scaling: Rele-
vance and replicability in follow-up of inpatients. J Nerv Ment. 1987; 175: 408-417.
26. Mitchell T, Cusick A. Evaluation of a client-centered paediatric rehabilitation
program using goal attainment scaling. Australian Occupational Therapy Journal.
1998; 45: 7-17.
27. Bain BA, Dollaghan CA. Clinical forum: Treatment efficacy. The notion of
clinically significant change. Language, Speech, and Hearing Services in Schools.
1991; 22: 264-270.
28. Rosenbaum P, King S, Law M, King G, Evans J. Family-centered service: A
conceptual framework and research review. Phys Occup Ther Pediatr. 1998; 18:
1-20.
29. Stolee P, Zaza C, Pedlar A, Myers AM. Clinical experience with goal attain-
ment scaling in geriatric care. Journal of Aging and Health. 1999; 11: 96-124.
30. Cardillo JE. Goal setting, follow-up, and goal monitoring. In: Kiresuk TJ,
Smith A, Cardillo JE, eds. Goal attainment scaling: Applications, theory, and mea-
surement. Hillsdale, NJ: Lawrence Erlbaum Associates; 1994; 39-59.

Das könnte Ihnen auch gefallen