Running head: TECHNICAL QUALITY

U05a1 Evaluation of Technical Quality

Ariana Apgar
Psy7610, Section 11
Summer, 2018
Capella University
TECHNICAL QUALITY 2

Introduction

The GARS-3 (Gilliam Autism Rating Scale - 3rd edition) is an examination that is

commonly used to identify a diagnosis of autism spectrum disorder (ASD) and its overall rigor on

a specific character being (Gilliam, 2014). This specific test is designed especially for individuals

between the ages of three years of age to twenty-two years of age (Gilliam, 2014). Furthermore,

there is a summation of exactly fifty-six items that are assessed and evaluated on a 4-pointed scale,

specifically known as ordinal (Gilliam, 2014). These items are beneath noted sub scales that have

set out with a goal to measure social communication, maladaptive speech, emotional response,

social interaction, repetitive behaviors as well as restrictive behaviors (Gilliam, 2014). Unlike other

assessments used in the platform of diagnosing an individual with possible ASD, the Gilliam

Autism Rating Scales tend to work very closely with the DSM domain to provide excessive input

and advanced instructional objectives in the event a differential diagnosis is present and/or needed.

This type of assessment also serves to recognize the severity of autism that the individual may have

(high functioning, low functioning). Furthermore, generality is also seen with this assessment as

the GARS-3 can be facilitated by clinicians, parents, or teachers after observing the individual. In

addition, I believe it is incredibly important to note that all the articles analyzed for this unit five

evaluation aren’t inherently relative to the Gilliam Autism Rating Scale 3rd edition (GARS-3) and

that articles involving the Gilliam Autism Rating Scale 1st edition (GARS) and Gilliam Autism

Rating Scales 2nd edition (GARS-2) assessments are presently included within this evaluation as

well. This is in fact due to such limited reviews and evaluations of the GARS-3 presently available

for use, which is in my opinion directly related to the fact that the third edition of this assessment is

under five-years of age at this present-day time.

TECHNICAL QUALITY 3

Evaluation of Technical Quality Article Reviews

Finke, E. H., Wilkinson, K. M., & Hickerson, B. D. (2016). Social Referencing Gaze Behavior

During a Videogame Task: Eye Tracking Evidence from Children with and without

ASD. Journal of Autism and Developmental Disorders,47(2). doi:10.1007/s10803-016-

2968-1

Social referencing gaze behavior among children with and without autism, regarding the

use and participation in video gaming was examined within this following study conducted.

Within the GARS-3 is a sub section that involves social interaction. This sub section covers

specific social components involving communication response, social response, emotional

response, and overall interaction. Within this study was a sum of 19 participants; 11 children with

autism spectrum disorder, 8 atypical children. Participants with an existing diagnosis of ASD had

already been diagnosed previously, this was confirmed in conjunction with the use of test-retest

approaches. The purpose of this study was to determine if the children on the spectrum would

fixate only on video game actual play or if they participated in referencing the other elements that

were taking place on the screen. Other elements of the screen included game stats, video game

streamers, and dialogue boxes. The results of this study concluded to show that the individuals

with autism spectrum disorder did not have a notable difference in comparison to the individuals

in the study that did not have autism (Finke et al, 2017).

Hashim, H., Yussof, H., Hanapiah, F. A., Shamsuddin, S., Ismail, L., & Malik, N. A.

(2013). Robot-Assisted to Elicit Behaviors for Autism Screening. Applied

Mechanics and Materials,393. doi:10.4028/www.scientific.net/amm.393.567

This 2013 study by Hashim et al. evaluates a robotic assistant’s, also known as a humanoid,

capacity in assisting with evaluating sub-scales in the Gilliam Autism Rating Scale 2 nd edition.
TECHNICAL QUALITY 4

For this specific study, 3 areas beneath the social interaction sub scale were chosen. The

humanoid known as “Nao” was organized and arranged with capabilities specific to social

interaction in order to assess the participant’s individual social response feedback. The researches

in the study stated that this examination has the possibility of being a very strong tool to screen

for autism, however, the researchers do believe it has its very limitations (Hashim et al, 2013).

One limitation noted was “rater bias” which has a reasonably high probability when evaluating

the GARS-2 reliability and validity. This is most likely due to the grave possibility that

interpretations by teachers and/or parents may have imprecision. In addition, the exactness of this

examination is not as secure as when a particular person is on the higher end of performance and

functioning. This is also noted when the individual’s symptoms begin to coincide with other

developmental and/or intellectual disabilities and disorders. The results of the research

determined that a humanoid has the required capability of assisting in supplying significant and

relevant aid but should not replace that of a behavior specialist conducting specific behavioral

techniques and associated therapies.

Lordo, D. N., Bertolin, M., Sudikoff, E. L., Keith, C., Braddock, B., & Kaufman, D. A.

(2016). Parents Perceive Improvements in Socio-emotional Functioning in Adolescents

with ASD Following Social Skills Treatment. Journal of Autism and Developmental

Disorders,47(1). doi:10.1007/s10803-016-2969-0

This study conducted an assessment of social skills intervention efficacy on specific competencies

in children with higher functioning ASD. Participants involved within this study included twenty-

nine children between the ages of eleven through seventeen, sixteen of whom had been diagnosed

with ASD prior and thirteen of whom who were developing on schedule atypically (Lordo et al,

2016). Used in this study was the GARS-3 to establish a confirmation of the absence and/or

existence of autism spectrum disorder among participants. Test-retest reliability approaches were

used in this instance. An autism spectrum disorder index within this study is noted at .93. The
TECHNICAL QUALITY 5

researcher’s made note that all subjects with autism met within these criteria. The GARS-3 was

used in the diagnosis of participants. It is further stated that the ASD item index that particularly

involved emotional and social response feedback was also used to examine data and conclude on

results of this study. It was indicated within the GARS-3 that the interventions used in this study

were shown to have efficacy in significantly reducing behaviors that were maladaptive prone as

well as symptoms within multiple areas.

O’Handley, R. D., & Allen, K. D. (2017). An evaluation of the production effects of video self-

modeling. Research in Developmental Disabilities,71. doi:10.1016/j.ridd.2017.09.012

The following study by O’Handley et al. (2017) involves the examination of an

individual’s overall task production of 3 common activities, on a day to day basis. In this

individual’s Gilliam Autism Rating Scale 3rd edition examination, the participants parent allotted

him a ranking of one hundred and twelve, which indicates that the participant was indeed likely

to have an autism diagnosis. The goal of this study was to see if video self modeling (VSM)

could possibly raise the overall achievement of a targeted task without the necessity of prompt

assistance. The tasks were compiled of activities involving cleaning the restroom, cleaning the

living area floors with the use of a vacuum, and folding clothes. A task analysis approach was

used by the individuals parent who recorded the tasks that were performed by the participant. To

ensure treatment integrity in this case inter rate reliability was used. In this exact case, the

researcher went by the task analysis provided and evaluated the video for overall generalization.

Data was then analyzed and activated into a research design known as multiple baseline. All 3

tasks were measured prior to and within the intervention phase. Test-retest approaches were used

to assess validity by having every assessment on each task measured at two different points in

time. The conclusion of the overall data collected, and results analyzed were that video self-

modeling enhanced the overall rate of success of each task significantly, across all 3 tasks that

were targeted (O’Handley et al, 2017).

TECHNICAL QUALITY 6

Rossi, M. R., Vladescu, J. C., Reeve, K. F., & Gross, A. C. (2017). Teaching Safety Responding to

Children with Autism Spectrum Disorder. Education and Treatment of Children,40(2).

doi:10.1353/etc.2017.0009

This 2017 study by Rossi et al. was conducted to evaluate training effects of behavioral

skills on minors who are diagnosed with ASD. Parents of the participants used the GARS-3 in

order to provide more detailed data about specific behavior characteristics and perspectives of

each of their children. Parent’s then assisted in scoring a rating that would determine if their

child had a high probability of ASD. It should be noted that in this study the autism index

surpassed a score of .90 (Rossi et al, 2017). The researchers noted a limitation within this study

is that it was conducted and completed by the participants parents, who are not behavioral

specialists. This limitation may have resulted in an unfair bias given the parents’ relationship to

the child and/or inexperience in the exact field of study.

South, M., Williams, B. J., McMahon, W. M., Owley, T., Filipek, P. A., Shernoff, E., Ozonoff,

S. (2002). Utility of the Gilliam autism rating scale in research and clinical populations.

Journal of Autism and Developmental Disorders, 32(6), 593-9.

This study evaluated the validity and reliability of the Gilliam Autism Rating Scale 1 st

edition (GARS) and the ability it has in order to properly diagnose children with ASD. This

study included one hundred and nine children chosen from various places among five sample

sizes (South et al, 2002). It was noted in the study that the participants had received a previous

diagnosis of ASD from the DSM-5 official manual. The Gilliam Autism Rating Scale 1 st edition

has a quotient verdict of SD = 15, M = 100 and when diagnosing the individuals of this study a

quotient verdict of SD = 13.92 and M = 90.10 (South et al, 2002). A score of ninety on the 1 st

edition of the GARS signifies a high probability of autism and the results showed more than half

of the individuals involved in this study had autism spectrum disorder. Two forms of testing
TECHNICAL QUALITY 7

reliability were used in this study known as alternate form and test-retest. Furthermore, the

GARS examination received a reliability coefficient of .48 regarding this study.

Volker, M. A., Dua, E. H., Lopata, C., Thomeer, M. L., Toomey, J. A., Smerbeck, A. M., Lee, G.

K. (2016). Factor Structure, Internal Consistency, and Screening Sensitivity of the GARS-

2 in a Developmental Disabilities Sample. Autism Research and Treatment.

doi:10.1155/2016/8243079

The following study examines the GARS-2 examination specifically on its reliability and validity

in the screening of developmental and intellectual disabilities. It should be noted that other

methods for assist in the assessment of ASD and the exact rigor of the individual’s diagnosis are

quite restricted to be completed in the use of a mental health professional only. However, with

the GARS-2 examination both parents and/or teachers can use this type of examination in both

assessing and assisting in the ID of ASD. Test-retest approaches were chosen to be used in order

to conduct this study due to the two hundred and forty participants being previously screened and

diagnosed with ASD (Volker et al, 2016). The specific range of age for each participant of this

study was between the years of three and twenty-one years of age with a overall mean of 9.5

years of age. For this study, the GARS-2 was able to rule out approximately 81% of the

participated who did not have autism and was able to accurately diagnose 65% of the participants

who had a diagnosis of autism (Volker et al, 2016).

Conclusion

In conclusion to this assignment, it is determined that the test reliability of the Gilliam

Autism Rating Scale, specifically the 3rd edition, is notably inconclusive based upon the journal

articles I have reviewed for this technical evaluation. As stated previously, the Gilliam Autism

Rating Scale 3rd edition is a fairly new test (publication within the last five years) and direct

research of the evaluation of said exam is relatively limited in nature. Many of the studies found

were shown to have frequently used the approach of test retest to confirm the validity and
TECHNICAL QUALITY 8

reliability of such assessment, and the majority of the studies maintained consistency with the

individual’s previous diagnoses. The validity of the articles regarding the GARS 1st through 3rd

editions appeared to be constant with diagnosis of the DSM-5. The limitations made aware in a

few of the studies varied from the severity of symptoms, the age of the individual, as well as the

experience of the raters themselves. Depending on the exact range and training could indeed

place an effect on the accuracy of the GARS-3 assessment. It is further determined after such

research, that the GARS-3 is quite suitable for the initial screening and diagnosing of autism

spectrum disorder. Especially when evaluating autism spectrum disorder, the GARS-3 is

suggestive to indeed cover the areas essential in order to successfully evaluate autism spectrum

disorder, however, it is insufficient to be used as a final result for intervention and treatment.
TECHNICAL QUALITY 9

Reference

Finke, E. H., Wilkinson, K. M., & Hickerson, B. D. (2016). Social Referencing Gaze Behavior

During a Videogame Task: Eye Tracking Evidence from Children With and Without

ASD. Journal of Autism and Developmental Disorders,47(2). doi:10.1007/s10803-016-

2968-1

Gilliam, James E. (2014). “Gilliam Autism Rating Scale – Third Edition”. Mental

Measurements Yearbook with Tests in Print”, EBSCOhost

Hashim, H., Yussof, H., Hanapiah, F. A., Shamsuddin, S., Ismail, L., & Malik, N. A.

(2013). Robot-Assisted to Elicit Behaviors for Autism Screening. Applied

Mechanics and Materials,393. doi:10.4028/www.scientific.net/amm.393.567

Lordo, D. N., Bertolin, M., Sudikoff, E. L., Keith, C., Braddock, B., & Kaufman, D. A.

(2016). Parents Perceive Improvements in Socio-emotional Functioning in Adolescents

with ASD Following Social Skills Treatment. Journal of Autism and Developmental

Disorders,47(1). doi:10.1007/s10803-016-2969-0

O’Handley, R. D., & Allen, K. D. (2017). An evaluation of the production effects of video self-

modeling. Research in Developmental Disabilities,71. doi:10.1016/j.ridd.2017.09.012

Rossi, M. R., Vladescu, J. C., Reeve, K. F., & Gross, A. C. (2017). Teaching Safety Responding to

Children with Autism Spectrum Disorder. Education and Treatment of Children,40(2).

doi:10.1353/etc.2017.0009

South, M., Williams, B. J., McMahon, W. M., Owley, T., Filipek, P. A., Shernoff, E., Ozonoff,

S. (2002). Utility of the Gilliam autism rating scale in research and clinical populations.

Journal of Autism and Developmental Disorders, 32(6), 593-9.

Volker, M. A., Dua, E. H., Lopata, C., Thomeer, M. L., Toomey, J. A., Smerbeck, A. M., Lee, G.

K. (2016). Factor Structure, Internal Consistency, and Screening Sensitivity of the GARS-

2 in a Developmental Disabilities Sample. Autism Research and Treatment.

doi:10.1155/2016/8243079
TECHNICAL QUALITY 10