07martin (bc/t)

5/21/03

3:39 pm

Page 321

Toward the use of a standardized assessment for young children with autism
Current assessment practices in the UK
N E I L T. M A R T I N PETER BIBBY Research, UK University of Kent at Canterbury, UK

autism 2003 SAGE Publications and The National Autistic Society Vol 7(3) 321330; 035378 1362-3613(200309)7:3

Autism & Developmental Disorders Education Keele University, UK

OLIVER C. MUDFORD SVEIN EIKESETH

Akershus College, Norway

A B S T R AC T

Little is known about the progress of autistic children following specic interventions in England. Nor do we know how frequently standardized assessments are used to monitor progress or to evaluate specic educational interventions. The reports of 75 children with autism, for whom special educational provision had been determined by a local education authority, were reviewed. Parents were interviewed and educational psychologists were contacted for details of any norm-referenced assessments. Of these children, 39 percent had no standardized assessments before education authorities determined their provision, and only 9 percent had follow-up assessments that could be used to evaluate progress. Children with autism in the UK rarely have sufcient assessments to allow an objective evaluation of their progress. There is currently no standardized assessment protocol to prescribe a specic educational intervention, to evaluate the progress of children or to make comparisons between interventions. We recommend the development of such a protocol.

K E Y WO R D S

assessment protocols; autism; monitoring; statements

ADDRESS Correspondence should be addressed to: D R N E I L M A R T I N , Tizard Centre, University of Kent at Canterbury, Canterbury, Kent CT2 7LZ, UK. e-mail: N.T.Martin@ ukc.ac.uk

Introduction
In England and Wales, selection of the type of educational provision prescribed for young autistic children rests in the hands of local education authorities (LEAs) following an assessment of childrens needs. Having carried out an assessment the LEA must, if necessary, issue a statement of 321

07martin (bc/t)

5/21/03

3:39 pm

Page 322

7(3) special educational needs (Education Act 1996). The statement describes the childs learning difculties and prescribes the provision that is deemed appropriate to meet the childs needs. The literature suggests that different types of educational provision may lead to different outcomes. Some studies report progress by some young children with autism in response to specic intensive behavioural interventions. For example, there are some reports of children making such outstanding progress that they were no longer deemed autistic and/or were functioning normally (Greenspan and Wieder, 1997; Lovaas, 1987; McEachin et al., 1993; Weiss, 1999). There are also reports of young children with autism making substantial gains in IQ, language and adaptive behaviour as a result of intensive programmes (Anderson et al., 1987; Birnbrauer and Leach, 1993; Sheinkopf and Siegel, 1998). However, other studies have reported little change following intensive behavioural treatment (Smith et al., 2000; Bibby et al., 2001). Studies of other educational approaches seem to indicate less effectiveness. For example, Lord and Schopler (1989) reported that of 216 children who had followed TEACCH programmes for at least 2 years, all but three were classied as autistic during the later assessment. On average the remaining 213 children made minimal gains in IQ and regressed on Vineland Adaptive Behaviour Scales composite scores (Lord and Schopler, 1988). Given that outcome for autistic children may be dependent on the type of educational provision prescribed, it would seem imperative that the progress of children is monitored. This would allow parents to make betterinformed decisions about the type of education that they would like for their child. It would also allow other professionals working within the eld (e.g. in health, social and educational services) to evaluate the effects of a specic type of intervention and to make comparisons both between interventions and between different children who share the same or similar educational provision. This would require the use of comparable assessment protocols. The aim of this study was to determine the extent to which young autistic children are assessed prior to the commencement of a specic educational intervention, and also to determine whether data continued to be collected in a way that would allow quantiable assessment of progress. The focus of this study is on the use of standardized assessments.
AU T I S M

Method
Participants The parents of 92 children with a diagnosis of autism were invited to take part in a study that sought to evaluate parent-managed, home-based early
322

07martin (bc/t)

5/21/03

3:39 pm

Page 323

M A RT I N E T A L . : S TA N DA R D I Z E D A S S E S S M E N T intensive behavioural intervention (EIBI). The results from this study have been published separately (Bibby et al., 2001; Mudford et al., 2000). These parents had independently approached one of the authors for legal assistance in seeking a statement of special educational needs that prescribed an EIBI programme (based on those described by Lovaas, 1987). Seventy-eight parents responded to our request for participation. One family subsequently withdrew and contact with two further children was lost. The 75 remaining children were distributed across 42 different LEAs in England (representing approximately 30 percent of the total number of LEAs in England) and were born between 1989 and 1995. They had a mean age of 53.6 months (SD 14.6) and 61 (81 percent) were boys. They all had statements of special educational needs prescribing educational intervention in response to their autism. These were completed between August 1995 and February 1999.

Procedure Three different methods were used to examine the extent to which children had been assessed before and after commencing educational programmes. The rst method was to survey the les that had been collated by the familys legal representative. These included all the assessments that formed part of the childrens statements of special educational needs. The second method was to write to the psychologist who had provided advice to the LEA in the preparation of the childs statement. The third method was for a research assistant to visit each of the families to ask about any assessments that had been carried out on their children. These visits took place after completion of the childrens statements. Only assessments that were standardized or norm-referenced (and were, therefore, directly comparable between different children and different types of educational provision) were surveyed. Furthermore, assessments had to be complete in that all the elements had been administered or that the assessor had halted the assessment and recorded that the child was untestable. Partially completed assessments were not included.

Results
The mean age of the children at completion of their statements was 52 months. Almost half (48 percent) had norm-referenced assessments carried out by either the LEA or the health authority before the childs statement had been completed (see Table 1). The assessment instruments used are listed in Table 2. The most commonly used assessments prior to the issue of a statement were the Grifths Mental Development Scales (Grifths, 323

07martin (bc/t)

5/21/03

3:39 pm

Page 324

7(3) 1996), the Vineland Adaptive Behavior Scales (Sparrow et al., 1984) and the Reynell Developmental Language Scales (Edwards et al., 1997). After the childrens statements had been issued, 37 percent of the sample had norm-referenced assessments carried out by either an LEA or a health authority (see Table 3) and a further 13 percent had privately arranged assessments carried out. Nearly half of the sample (49 percent) had no further assessments. The three most commonly used assessment tools following statementing
AU T I S M

Table 1 Numbers and percentages of children assessed before the issue of a statement of special educational needs (total N = 75) Children who had: No norm-referenced assessment Only privately arranged assessments Assessments conducted by LEA and/or health authority Assessments conducted by LEA Assessments conducted by health authority N 29 10 36 20 28 % of sample 39 13 48 27 37

Table 2 Numbers of children for whom each different assessment tool was used before the issue of a statement of special educational needs Assessment instrument Private LEA Health authority 47 20 1 6 4 2 1 5 3 4 1 0 All

Totals Grifths Mental Development Scales Vineland Adaptive Behavior Scales Reynell Developmental Language Scales Bayley Scales of Infant Development Leiter International Performance Scale Wechsler Preschool and Primary Scale of Intelligence MacArthur Communicative Developmental Inventory MerrillPalmer Scale of Mental Tests Symbolic Play Test CELF Pre Sschool Language Scales British Ability Scales

30 0 17 3 0 3 2 0 1 1 2 1

18 6 2 1 3 2 3 0 1 0 0 0

95 26 20 10 7 7 6 5 5 5 3 1

324

07martin (bc/t)

5/21/03

3:39 pm

Page 325

M A RT I N E T A L . : S TA N DA R D I Z E D A S S E S S M E N T were the Vineland Adaptive Behavior Scales, the Reynell Developmental Language Scales and the Wechsler Preschool and Primary Scale of IntelligenceRevised (Wechsler, 1989) (see Table 4). The nal stage of this survey was to examine the extent to which the subsequent assessments could be related to earlier assessments in order to measure progress. This requires testretest using the same or comparable instruments. The use of the same instrument for testretests is not always

Table 3 Numbers and percentages of children assessed after the issue of a statement of special educational needs (total N = 75) Children who had: No norm-referenced assessment Only privately arranged assessments Assessments conducted by LEA and/or health authority Assessments conducted by LEA Assessments conducted by health authority N 37 10 28 17 17 % of sample 49 13 37 23 23

Table 4 Numbers of children for whom each different assessment tool was used after the issue of a statement of special educational needs Assessment instrument Private LEA Health authority 22 3 10 1 0 1 1 3 1 1 0 1 0 All

Totals Vineland Adaptive Behavior Scales Reynell Developmental Language Scales Wechsler Preschool and Primary Scale of Intelligence Bayley Scales of Infant Development British Ability Scales MerrillPalmer Scales of Infant Development Grifths Mental Development Scale British Picture Vocabulary Scales Leiter International Performance Scale CELF Pre Sschool Language Scales Symbolic Play Test MacArthur Communicative Developmental Inventory

51 20 6 6 4 2 6 0 2 2 2 0 1

23 3 0 7 4 5 1 2 0 0 0 1 0

96 26 16 14 8 8 8 5 3 3 2 2 1

325

07martin (bc/t)

5/21/03

3:39 pm

Page 326

7(3) possible because the age range of the standardization of the tests varies. For example, the Bayley Scales of Infant Development (Bayley, 1993) are only standardized up to age 312;, and older children would be expected to progress to a standard IQ test such as the Wechsler Preschool and Primary Scale of Intelligence. Some of the assessments, such as the Grifths Mental Development Scales, generate developmental age scores that may be converted to a ratio quotient (RQ) which can then be compared with a subsequent IQ test. Only seven children (9 percent of the sample) had testretests using comparable instruments. These were the Bayley Scales of Infant Development and the British Ability Scales (n = 1), the Grifths Mental Development Scales (testretest, n = 1), the Grifths Mental Development Scales and the Wechsler Preschool and Primary Scale of Intelligence (n = 3), and the Reynell Developmental Language Scales (testretest, n = 2).
AU T I S M

Discussion
Standardized and objective pre- and post-intervention assessments are crucial for assessing progress made by children with autism placed in special educational provision. Our ndings suggest that LEAs do not systematically assess the outcomes of children placed in different kinds of provision. Nor is choice of provision based on any objective measures of progress. However, such data are important in light of the increasing numbers of parents who are opting for home-based behavioural programmes (Lovaas, 1987) rather than accepting standard educational provision. Several large surveys (Carter et al., 1998; Lord and Schopler, 1988; 1989) provide normative data for children with autism following traditional programmes in the USA. These studies provide a basis against which outcomes for any particular programme may be compared, provided that the same or comparable assessment instruments are used. Despite the fact that it is important that the progress made by young children with autism following different programmes can be compared across large numbers of children (Connor, 1998; Jordan et al., 1998), the use of the same or comparable assessment instruments is not widespread in England. There are a variety of assessment instruments available but these have all been standardized for typically developing children. Moreover, the appropriate test is determined by the chronological age of the child rather than by existing cognitive/developmental ability. For example, the Bayley Scales of Infant Development covers the age range 1 to 42 months; the Wechsler Preschool and Primary Scale of Intelligence covers the age range 3 years to 7 years 3 months. Markedly different scores may be obtained according to the test used; for example, the MerrillPalmer Scale of Mental Tests 326

07martin (bc/t)

5/21/03

3:39 pm

Page 327

M A RT I N E T A L . : S TA N DA R D I Z E D A S S E S S M E N T (Stutsman, 1948) yields consistently higher scores than tests such as the Bayley Scales of Infant Development (Bibby et al., 2001; Magiati and Howlin, 2001). It is important that a standardized measure be used that allows evaluations both within and across different interventions. Few tests cover the age range of preschool to later childhood, although the Grifths Mental Development Scales covers the age range 0 to 8 years (and is currently being restandardized). We would recommend the use of such a test. Some researchers have argued that measures of adaptive behaviour could be used in situations where cognitive assessments may prove difcult to administer or may be unreliable because of the degree of intellectual disability (Meins and Smann, 1993; Moss and Hogg, 1997; Volkmar et al., 1987). The assessment of young autistic children can also present problems that are not ordinarily associated with assessing typical children, for example, poor linguistic skills and attention problems (Sattler, 1992). The correlation between adaptive behaviour and cognitive ability in individuals with autism appears to be consistently high (Carpentieri and Morgan, 1996; Freeman et al., 1988, 1991; Perry and Factor, 1989). The correlation between Bayley developmental age equivalents and Vineland Adaptive Behaviour (composite) age equivalents for the children in this study (n = 51) yielded a high correlation (r = 0.78, p < 0.001). This nding was replicated for a further 36 children (mean age 48.2 months, SD 17.2) who had not started any particular type of educational provision and who had also been tested by the same assessors used for the primary study (cf. Bibby et al., 2001; Mudford et al., 2000). The VABS developmental age score (mean 19 months, SD 6.7) and the Bayley score (mean 20.8 months, SD 8.1) were also highly correlated for this group of children (r = 0.87, p < 0.001). These data suggest that, in the absence of any other appropriate psychometric assessment, an assessment of adaptive behaviour (specically the Vineland Adaptive Behaviour Scales) could be used as a good estimate of cognitive ability. We would recommend that a standardized assessment of young autistic children include an assessment of adaptive behaviour as a minimum requirement. This study has focused on the use of comparable standardized normreferenced assessments as a simple means of evaluating educational provision for children with autism. However, it is clear that such assessments are not without their limitations and they may be relatively insensitive to the social and communicative difculties that are the key decits in autism. For example, improvements in stereotypic or problematic behaviours, themselves legitimate targets of good educational provision, may not easily be captured by assessments of cognitive ability or adaptive behaviour. The development of a range of measures that address different areas of impairment is important for the future and may include non-standardized direct

327

07martin (bc/t)

5/21/03

3:39 pm

Page 328

7(3) observational methods. Standardized assessments, however, remain the most reliable tool available at present. It is important that the assessment process, both before and after the issue of a statement of special educational needs, is applied consistently in terms of both when and how children are assessed. Currently, normally developing children are given a local standard assessment at age 5 and national standard assessments (SATS: DfES, 1999) at ages 7, 11, 14 and 16. Children with serious developmental disorders such as autism are generally not entered for these assessments. The introduction of a specic assessment protocol for children with autism (and those with other special educational needs) could be achieved by amending national legislation governing statutory assessment and annual reviews. Although such a national initiative would take time to achieve, there is no reason why local education authorities could not adopt such a practice in the short term.
AU T I S M

Acknowledgements This research was supported by a grant from Autism and Developmental Disorders Education Research (ADDER) to Keele University. We thank the following for assistance with data collection: Siobhan Breen, Barbara Cross, Jo Halden, Amanda Kemmish, Iliana Magiati, Gill McGrane and Catherine Naysmith. References
A N D E R S O N , S . R . , AV E RY, D . L . , D I P I E T RO , E . K . , E DWA R D S , G . L . & C H R I S T I A N , W. P. (1987) Intensive Home-Based Early Intervention with Autistic Children,

Education and Treatment of Children 10: 35266. (1993) Manual for the Bayley Scales of Infant Development, 2nd edn. San Antonio, TX: Psychological Corporation. B I B B Y, P. , E I K E S E T H , S . , M A RT I N , N . T. , M U D F O R D , O . C . & R E E V E S , D . (2001) Progress and Outcomes for Children with Autism Receiving Parent-Managed Intensive Interventions, Research in Developmental Disabilities 22: 42547. B I R N B R AU E R , J . S . & L E AC H , D . J . (1993) The Murdoch Early Intervention Program after 2 Years, Behaviour Change 10: 6374. C A R P E N T I E R I , S . & M O R G A N , S . B . (1996) Adaptive and Intellectual Functioning in Autistic and Nonautistic Retarded Children, Journal of Autism and Developmental Disorders 26: 61120. C A RT E R , A . S . , VO L K M A R , F . R . , S PA R ROW, S . S . , WA N G , J . - J . , L O R D , L . , DAW S O N , G . , F O M B O N N E , E . , L OV E L A N D , K . , M E S I B OV, G . & S C H O P L E R , E . (1998) The Vineland Adaptive Behaviour Scales: Supplementary Norms for Individuals with Autism, Journal of Autism and Developmental Disorders 28: 287302. C O N N O R , M . (1998) A Review of Behavioural Early Intervention Programmes for Children with Autism, Educational Psychology in Practice 14: 10917. D E PA RT M E N T F O R E D U C AT I O N A N D S K I L L S (1999) National Curriculum (Key Stage 1 Assessment Arrangements for English, Mathematics and Science) Order 1990. London: HMSO. E DWA R D S , S . , F L E T C H E R , P. , G A R M A N , M . , H U G H E S , A . , L E T T S , C . & S I N K A , I .
B AY L E Y, N .

328

07martin (bc/t)

5/21/03

3:39 pm

Page 329

M A RT I N E T A L . : S TA N DA R D I Z E D A S S E S S M E N T

(1997) Reynell Developmental Language Scales III: The University of Reading Edition. Windsor: NFER Nelson. F R E E M A N , B . J . , R A H B A R , B . , R I T VO , E . R . , B I C E , T. L . , YO KO TA , A . & R I T VO , R . (1991) The Stability of Cognitive and Behavioral Parameters in Autism: A TwelveYear Prospective Study, Journal of the American Academy of Child and Adolescent Psychiatry 30: 47982. F R E E M A N , B . J . , R I T VO , E . R . , YO KO TA , A . , C H I L D S , J . & P O L L A R D , J . (1988) WISCR and Vineland Adaptive Behavior Scale Scores in Autistic Children, Journal of the American Academy of Child and Adolescent Psychiatry 27: 4289. G R E E N S PA N , S . I . & W I E D E R , S . (1997) Developmental Patterns and Outcomes in Infants and Children with Disorders in Relating and Communicating: A Chart Review of 200 Cases of Children with Autistic Spectrum Diagnosis, Journal of Developmental and Learning Disorders 1: 87141. G R I F F I T H S , R . (1996) Grifths Mental Development Scales. Henley-on-Thames: Test Agency. J O R DA N , R . , J O N E S , G . & M U R R AY, D . (1998). Educational Interventions for Children with Autism: A Literature Review of Recent and Current Research. London: DfEE. L O R D , C . & S C H O P L E R , E . (1988) Intellectual and Developmental Assessment of Autistic Children from Preschool to School Age, in Diagnosis and Assessment in Autism. New York: Plenum. L O R D , C . & S C H O P L E R , E . (1989) The Role of Age at Assessment, Developmental Level, and Test in the Stability of Intelligence Scores in Young Autistic Children, Journal of Autism and Developmental Disorders 19: 48399. L OVA A S , I . O . (1987) Behavioral Treatment and Normal Educational and Intellectual Functioning in Young Autistic Children, Journal of Consulting Clinical Psychology 55: 39. M AG I AT I , I . & H OW L I N , P. (2001) Monitoring the Progress of Preschool Children with Autism Enrolled in Early Intervention Programmes: Problems in Cognitive Assessment, Autism 5: 399406. M C E AC H I N , S . J . , S M I T H , T. & L OVA A S , O . I . (1993) Long-Term Outcomes for Children with Autism Who Receive Early Intensive Behavioral Treatment, American Journal of Mental Retardation 97: 35972. M E I N S , W. & S M A N N , D . (1993) Evaluation of an Adaptive Behaviour Classication for Mentally Retarded Adults, Social Psychiatry and Psychiatric Epidemiology 28: 2015. M O S S , S . & H O G G , J . (1997) Estimating IQ from Adaptive Behaviour Information in People with Moderate or Severe Intellectual Disability, Journal of Applied Research in Intellectual Disabilities 10: 616. M U D F O R D , O . C . , M A RT I N , N . T. , E I K E S E T H , S . & B I B B Y, P. (2000) Parent-Managed Behavioral Treatment for Pre-School Children with Autism: Some Characteristics of UK Programs, Research in Developmental Disabilities 22: 17382. P E R RY, A . & F AC T O R , D . C . (1989) Psychometric Validity and Clinical Usefulness of the Vineland Adaptive Behavior Scales and the AAMD Adaptive Behavior Scale for an Autistic Sample, Journal of Autism and Developmental Disorders 9: 4155. S AT T L E R , J . M . (1992) Assessment of Children (Revised and Updated Third Edition). San Diego, CA: Jerome Sattler. S H E I N KO P F , J . & S I E G E L , B . (1998) Home-Based Behavioural Treatment of Young Children with Autism, Journal of Autism and Developmental Disorders 28: 1523. S M I T H , T. , G RO E N , A . & W Y N N , J . W. (2000) Randomized Trial of Intensive Early Intervention for Children with Pervasive Developmental Disorder, American Journal on Mental Retardation 104: 26985.

329

07martin (bc/t)

5/21/03

3:39 pm

Page 330

AU T I S M

7(3)

S PA R ROW, S . S . , B A L L A , D . A .

& C I C C H E T T I , D . V. (1984) Vineland Adaptive Behaviour Scales. Circle Pines, MN: American Guidance Service. S T U T S M A N , R . (1948) MerrillPalmer Scale of Mental Tests. Los Angeles, CA: Western Psychological Services. VO L K M A R , F . R . , S PA R ROW, S . S . , G O U D R E AU, D . , C I C C H E T T I , D . V. , PAU L , R . & C O H E N , D . J . (1987) Social Decits in Autism: An Operational Approach to Using the Vineland Adaptive Behavior Scales, Journal of the American Academy of Child and Adolescent Psychiatry 26: 15661. W E C H S L E R , D . (1989) Wechsler Preschool and Primary Scale of IntelligenceRevised. San Antonio, TX: Psychological Corporation. W E I S S , M . (1999) Differential Rates of Skill Acquisition and Outcomes of Early Intensive Behavioural Intervention for Autism, Behavioural Intervention 14: 322.

330