SPED 311 Assessment Review Project

Name: Kaitlyn Kolarik

Date: November 29, 2016

School/Setting: Resource/Inclusion

How does this project contribute to your knowledge about the technical aspects of
assessment?

This project really opened my eyes as to what all goes into an assessment. It was amazing to
see how much work goes into making sure the test is valid and reliable. I now feel more aware
about all of the different facets that go into making and distributing tests. This will make me
more aware and understanding when being consulted with those who are distributing this test,
along with others, to my future students.

On my honor, as an Aggie, I have neither given nor received unauthorized aid

on this academic work.

Signature____________________________________________
 Practical Evaluation
Description of Test

The Gilliam Autism Rating Scale (GARS-3) written by James E. Gilliam and published by

Pro-Ed, is an assessment tool used to identify Autism Spectrum Disorder in students ages 3-22.

The test kit includes 50 summary/response forms and two booklets: an examiners manual and

an instructional objectives booklet. The price of the kit is $163, but it is possible to buy each

item separately, as well as extra packs of summary/response forms for $59 consisting of a pack

of 50.

Discussion of Test Manuals

The test, as well as its components, was very well thought out when it was being

created. The examiners manual is broken up into very organized and neatly detailed sections.

The examiners manual opens with an explanation of Autism Spectrum Disorder, administration

procedures, conducting with reliability and validity of the test. The instructional objectives

booklet is also laid out very professionally. It is broken into sections that suggest instructional

objectives in regards to specific academic and functional goals.

Discussion of Test Materials

The actual test comes in the form of a two-page front and back summary/response

booklet. The layout of the summary/response booklet is very organized, which makes filling out

answers very easy. The paper of the booklet is a little thin, but due to the fact that just the

examiner touches this paper this is not a huge problem. The test measures six different
subscales: restricted/repetitive behavior, social interaction, social communication, emotional

response, cognitive style, and maladaptive speech. The front of the first page is a description of

the student. The remaining pages of the booklet contain questions in regards to each of the six

subscales for the examiner to answer by circling the appropriate response.

Discussion of Test Protocols

The explanation of test protocols can be found in the examiners manual. Administration

is particularly simple, as the administrator is to just observe the student. This observation is

guided by the six subscales (stated in the discussion of test materials). Scoring is also self-

explanatory as the instructor just has to circle a number from 0-3 that resembles the students

behaviors and characteristics in the summary/response form. The summary and response form

is physically durable, laid out intuitively, and is easy to navigate through. The test usually takes

about five to ten minutes to complete. One major drawback to this test is that if the

administrator is not able to score a test item, they are to observe the student for a longer

period of time. Potentially, the examiner can observe the student for up to 6 hours to answer

the question.

Discussion of Test Items

The test includes 58 questions that follow the six subscales of restricted/repetitive

behavior, social interaction, social communication, emotional response, cognitive style, and

maladaptive speech. The administrator is to observe the student and answer each question by

using a rating scale of 0-3. The rating scale resembles: 0 - not at all like the individuals, 1 - not

much like the individual, 2 - somewhat like the individual, and 3 - very much like the individual.
 Technical Evaluation

Norms

Normative data was collected in 2010 and 2011 on 1,859 who have been diagnosed with

Autism Spectrum Disorder. The norming criterion was that the individual had to have a

diagnosis of Autism Spectrum Disorder, be between the ages of 3-19, and reside in the United

States. Of all of the students being normed, 1,139 of them have only Autism Spectrum Disorder

as an identified disability. The remaining 720 individuals had another diagnosis as well. Most of

the students who were included in this normative sample were either white or black/African

American (Gilliam 2014).

Overall, this normative sample is not very effective due to the fact that many individuals

of multiple ethnicities were not studied in this normative data. Due to the fact that this sample

is not very representative of different ethnicities (especially Hispanics), this test may not be the

best test to administer throughout the state of Texas.

Reliability

Test-retest and inter-rater reliability types were used to determine the reliability of the

GARS-3. According to James E. Gilliam, test-retest reliability was tested with 122 participants in

a two-week span and the test-retest reliability fell within the average range of .76 to .87. The

inter-rater reliability test was completed by 232 individuals rating 116 of the participants with

Autism Spectrum Disorder. The inter-rater reliability was found to fall in the .71-.85 range

which is considered acceptable (Gilliam 2014).

Validity
 James E. Gilliam mentions in the examiners manual that validity of the GARS-3 was

tested with construct validity, criterion validity, and content validity. When it comes to

construct validity, Gilliam wanted to make sure the assessment was actually able to

differentiate individuals with Autism Spectrum Disorder from others. To do this, the GARS-3

was given to students who were diagnosed with other disabilities as well as to typically

developing students. After assessing these students, the GARS-3 was able to successfully

identify and distinguish the students diagnosed with ASD from others without this diagnosis.

Gilliam used the predictive subset of criterion validity to assess the outcome of the ASD

diagnosis. He found that the GARS-3 was on par with other assessments that test for ASD. The

last form of validity that was tested was content validity. This test was used to make sure that

the GARS-3 tested for every trait that is associated with ASD. To do this, the GARS-3 was

compared to the Diagnostic and Statistical Manual of Mental Disorders Fifth Edition (DSM-5)

and the Autism Society (2012) to see if the assessment fit their criterion of ASD (Gilliam 2014).

This comparison proved that the GARS-3 falls within the acceptable range, which means its test

items are a good fit to measure for ASD (Gilliam 2014).

Journal Reviews

Journal Review #1:

In Karren C. Benjamins review she points out strengths as well as weaknesses that occur

in the GARS-3. Improvements were made to the third edition that include: a new definition of

Autism Spectrum Disorder that correlates with the DSM-5, 42 new items on the rating scale,

and new normative data to reflect the current U.S census (Benjamin 2014). She also points out

that the instructional objectives for students within the Autism Spectrum Disorder booklet is
helpful. Although there are many strengths of this assessment, Karen C. Benjamin also found

several weaknesses. One big problem with this assessment according to Benjamin is that the

examiner has to observe a student up to 6 hours if they can not answer one of the items on the

rating scale. Another problem with this assessment was that it was normed completely on

students with Autism Spectrum Disorder, and on white and black students ages 3-19. This

causes a problem because it does not allow for individuals being tested to be compared to

others without the Autism diagnosis. Also, it hinders the assessment of individuals ages 20-22

as well as students of other ethnicities and races (Benjamin 2014).

Journal Review #2:

This Gilliam Autism Rating Scale-2 (GARS-2) was an assessment used to identify Autism

Spectrum Disorder in students until the GARS-3 was released. Janine M. Montgomery, Brendan

Newton, and Christiane Smith reviewed the GARS-2 in 2006 by highlighting strengths and

weaknesses of the assessment. Of these weaknesses, they found that the GARS-2 lacked

norming with older individuals (ages 19-22). This can be seen as a problem because the test

claims that it is effective in identifying ASD in individuals up to 22. If it was not normed on these

individuals, than the validity is questioned. Another problem found with the GARS-2 is that if

one examiner is not able to rate an item on the test, other examiners are then supposed to fill

in the item, and then an average score would be taken. The problem found with this is that

there is no reasoning as to why this needs to happen in the examiners manual (Montgomery,

Newton, Smith 2006).

When looking at the GARS-3, improvement was not necessarily made from the previous

assessment. In the GARS-3, norming was still not completed on older individuals, so the test
might not be the best option on assessing older students. When it came to the test items, the

GARS-3 did change the protocols when an item could not be answered, but the new way is also

not the most efficient due to the fact that the examiner has to observe the student for 6 hours.

It is recommended that in future versions of the Gilliam Autism Rating Score that norming be

completed on older individuals, as well as a new system for incomplete testing items.
 References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders

(5th ed.). Arlington, VA: American Psychiatric Publishing.

Benjamin C. Karren (2014). A test review: Gilliam, J.E. (2014). Gilliam Autism Rating Scale-Third

Edition (GARS-3). Austin, TX: PRO-ED. Journal of Psychoeducational Assessment.

Gilliam, J. E. (2014). Gilliam Autism Rating Scale (3rd ed.). Austin, TX: Pro-Ed.

Montgomery, J. M., Newton, B., & Smith, C. (2008). Test Reviews: Gilliam, J. (2006). "GARS-2:

Gilliam Autism Rating Scale-Second Edition." Austin, TX: PRO-ED. Journal Of

Psychoeducational Assessment, 26(4), 395-401.