Journal of Psychoeducational Assessment

Predictors of Social Skills 29(2) 114124

2011 SAGE Publications
Reprints and permission: http://www.
for Preschool Children sagepub.com/journalsPermissions.nav
DOI: 10.1177/0734282910378478

at Risk for ADHD: http://jpa.sagepub.com

The Relationship
Between Direct and
Indirect Measurements

Lisa B. Thomas1, Edward S. Shapiro1,

George J. DuPaul1, J. Gary Lutz1, and Lee Kern1

Abstract
The relationship between direct and indirect measurements of social skills and social problem
behaviors for preschool children at risk for attention deficit hyperactivity disorder (ADHD)
was examined. Participants included 137 preschool children, aged 3 to 5 years, at risk for
ADHD, who were participating in a larger study examining the effects of early intervention for
young children. Teachers rated the social skills and social problems of the participants. Direct
observation data of participants were also collected at preschool during free play. Results
support previous research on social skills assessment and suggest that indirect and direct
measures may not be measuring the same aspect of social skills. Thus, a variety of evaluation
tools are necessary to comprehensively assess the social skills of preschool children with social
challenges.

Keywords
Social Skills, Preschool Children, ADHD
Social skills deficits are increasingly viewed as a central characteristic of attention deficit hyper-
activity disorder (ADHD). As one of the most commonly diagnosed behavioral disorders, ADHD
affects approximately 3% to 5% of all school-aged children in the United States (American Psy-
chiatric Association, 2000). According to Barkley (2006), approximately 50% of children diag-
nosed with ADHD experience significant problems in their social functioning and relationships
with peers. This may be particularly salient among preschool-aged children who exhibit symp-
toms of ADHD. In an investigation comparing preschool children with and without ADHD,
DuPaul, McGoey, Eckert, and VanBrakle (2001) found that young children with ADHD were
observed to exhibit a significantly higher frequency of negative social behavior, especially in
free play settings, and were reported to have more behavior problems and greater social skills
deficits compared with their typical counterparts.

1
Lehigh University, Bethlehem, PA, USA

Corresponding Author:
Lisa B. Thomas, College of Education, Lehigh University, Iacocca Hall, 111 Research Drive,
Bethlehem, PA 18015-4792, USA
Email: lbs2@lehigh.edu

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

Thomas et al. 115

Social skills deficits have been linked to social adjustment difficulties (Bagwell, Schmidt,
Newcomb, & Bukowski, 2001), problems with social participation in peer groups (McElwain,
Olson, & Volling, 2002), poor peer relationships and peer rejection (Mrug, Hoza, & Gerdes, 2001),
negative school adjustment (Buhs, Ladd, & Herald, 2006), academic performance problems (Flook,
Repetti, & Ullman, 2005), and maladjustment, delinquency, and childhood psychopathology
(Wheeler & Carlson, 1994). Social skills deficits and peer rejection have also been associated with
poor outcomes in adulthood (Bagwell et al., 2001), including increased risk for substance abuse,
family conflict, and vocational disadvantage (Semrud-Clikeman & Schafer, 2000). The negative
consequences of peer rejection in childhood have been shown to be one of the strongest predictors
of both concurrent and future maladjustment (Hodgens, Cole, & Boldizar, 2000).
To prevent or remediate maladjustment, it is critical to have valid and reliable tools for identify-
ing children at risk for social problems. A variety of direct and indirect measures have been devel-
oped and used to assess childrens social skills. Direct methods examine behavior at the actual time
of its occurrence and include observations and self-monitoring. In particular, observations during
naturalistic environments can provide qualitative and/or quantitative (e.g., temporal and/or topo-
graphical) information about specific behaviors and ecological factors in the childs social environ-
ment. In contrast, indirect approaches assess behavior at a time or place detached from its actual
occurrence, and include self- and informant reports obtained through interviews, checklists, or rat-
ing scales. Rating scales provide information about an informants judgment of the degree to which
an item reflects a childs specific behavior. Rating scales are relatively easy to administer and are
efficient in terms of time and cost. Many of these measures are norm referenced, permitting com-
parisons with peers of similar age and gender. Parent and teacher reports on rating scales are the
most common indirect method used for identification of problem behavior in preschool children
(Burns et al., 2000). Nevertheless, rating scales have been criticized for subjectivity and reporter
bias (Demaray, Elting, & Schaefer, 2003). Contrary to rating scales, direct observations are not
reliant on memory, judgment, or perception of other people (Gresham, 1982). However, systematic
direct observations are less frequently used to assess social skills compared with rating scales
because of time, training, and financial constraints (Hintze, Volpe, & Shapiro, 2002).
An effective assessment of a childs social skills should include a multimethod approach
using both direct and indirect methods (Odom, McConnell, & Brown, 2008). Research on the
relationship between rating scales and direct observations has yielded low to moderate correla-
tions (Cost & Simpson, 2004), suggesting that smaller correlations between direct observations
and informant reports may be related to different perspectives and/or opportunities for observa-
tion, such that both are needed to provide a complete picture of a childs social and behavioral
functioning (McEvoy, Estrem, Rodriguez, & Olson, 2003). The low correlations between indi-
rect and direct methods raise questions as to whether indirect and direct methods are measuring
the same construct of social skills.
Although previous research has substantiated that multimethod assessment is best practice, no
prior studies have examined the relationship of information derived from indirect and direct
methods of social skills for children with ADHD. Therefore, the goal of this study was to exam-
ine the relationship between direct and indirect method predictors of social skills for preschool
children at risk for ADHD. Specifically, this study asked the following research questions:

Research Question 1: Do direct observations of social play, solitary play, and aggression
provide unique variance in predicting teacher-rated social skills for preschool children
at risk for ADHD?
Research Question 2: Do teacher ratings of social skills, social problems, and aggression
provide unique variance in predicting direct observations of social play for preschool
children at risk for ADHD?

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

116 Journal of Psychoeducational Assessment 29(2)

Table 1. Child Demographic and Diagnostic Characteristics (n = 137)

Characteristics n Percentage
Age (years)
3 32 23.4
4 68 49.6
5 37 27.0
Gender
Male 107 78.1
Female 30 21.9
Ethnicity
Caucasian 94 68.6
Latino/Hispanic 18 13.1
African American 4 2.9
Other (e.g., biracial) 18 13.1
DISC DSM-IV Diagnostic Criteria
ADHD: inattentive type 12 8.8
ADHD: hyperactiveimpulsive type 36 26.3
ADHD: combined type 87 63.5
Oppositional defiant disorder 101 73.7
Note: ADHD = attention deficit hyperactivity disorder; DISC = Diagnostic Interview Schedule for Children; DSM-IV =
Diagnostic and Statistical Manual of Mental Disorders, fourth edition.

Method
Participants
Participants were drawn from a larger study investigating the effects of a multicomponent treat-
ment for young children at risk for ADHD (Kern et al., 2007). The participants were referred to
the larger study over a 4-year period by parents, teachers, or pediatricians with 137 children
meeting the inclusion criteria. To be included, children had to be between 3 and 5 years of age at
the time of referral and enrolled in a day care or preschool setting at least 2 days per week. Par-
ticipants had to have a T-score 65 on at least one of the three ADHD indexes of both the
Connerss Parent Rating ScaleRevised (CPRS-R; Conners, 1997) and Connerss Teacher Rat-
ing ScaleRevised (CTRS-R; Conners, 1997), as well as meet criteria for one of the three sub-
types of ADHD on the Diagnostic Interview Schedule for Children (DISC; Shaffer, Fisher,
Lucas, Dulcan, & Schwab-Stone, 2000), completed by the parent. Children who exhibited cogni-
tive deficits, symptoms of neurological damage, symptoms of an autistic spectrum disorder or a
pervasive development disorder, or met diagnostic criteria for conduct disorder were excluded
from participation. In addition, children who had a Childrens Global Assessment Scale (C-GAS;
Shaffer et al., 1983) score greater than 80 were excluded for absence of impairment.
Descriptive information on the sample of children for this study is provided in Table 1. The
majority of the participants were 4 years old, Caucasian, male, and had a Diagnostic and Statisti-
cal Manual of Mental Disorders, fourth edition, text revision (DSM-IV-TR; American Psychiat-
ric Association, 2000) diagnosis of ADHDcombined type with comorbid oppositional defiant
disorder. Participants had an average C-GAS score of 47 (range = 35-68; SD = 4.48), reflecting
a moderate degree of impairment in functioning. The majority of parents of the participants were
married (n = 92), had completed some college education (n = 45 mothers; n = 31 fathers), and
worked full-time (n = 63 mothers; n = 103 fathers).

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

Thomas et al. 117

Indirect Measures
Social Skills Rating SystemTeacher Form, Elementary Level: Social Skills scale (SSRS-T:
Social Skills scale). The SSRS-T (Gresham & Elliott, 1990) broadly measures social skills, prob-
lem behaviors, and academic competence, as they relate to the school setting. The elementary
level, rather than preschool form, was chosen for the larger study to allow consistent comparisons
over time. The preschool and elementary versions are highly compatible; they consist of the same
subscales and types of questions. The Social Skills scale measures how often teachers observe
the child to display aspects of cooperation, assertion, and self-control (Gresham & Elliott, 1990).
The reliability and validity of the SSRS-T is well established (Gresham & Elliott, 1990). Items
are rated on a 3-point Likert-type scale (0 = never, 1 = sometimes, and 2 = very often). The base-
line raw score for the Social Skills scale was used as a dependent and independent variable for
this investigation.
Connerss Teacher Rating ScaleRevised, Long Form: Social Problems scale (CTRS-R: Social Problems
scale). The CTRS-R (Conners, 1997) measures behaviors commonly associated with ADHD, as
well as other behavior problems commonly found in children. The Social Problems scale mea-
sures the degree to which the teacher observes the child to: have trouble making and keeping
friends, have few friends, have low self-esteem and self-confidence, and feel socially detached
from peers (Conners, 1997). The reliability and validity of the CTRS-R is well-established
(Conners, 1997). Items are rated on a 4-point Likert-type scale (0 = not true at all, 1 = just a little
true, 2 = pretty much true, and 3 = very much true). The baseline raw score for the Social Prob-
lems scale was used as an independent variable for this investigation.
Teachers Report Form/5-18: Aggressive Behavior scale (TRF: Aggressive Behavior scale). The TRF
(Achenbach, 2001) is a general measure of child and adolescent emotional and behavioral prob-
lems. The Aggressive Behavior scale measures the degree to which the teacher observes the child
to be noncompliant and engage in verbally and physically aggressive behaviors (Achenbach,
2001). The reliability and validity of the TRF is well established (Achenbach, 2001). Items on
the TRF are rated on a 3-point Likert-type scale (0 = not true, 1 = sometimes true, and 2 = often
true). The baseline raw score for the Aggressive Behavior scale was used as an independent vari-
able for this investigation.

Direct Measures
Early Screening Profile Social Observation Code (ESP SOC). The SOC (Walker, Severson, & Feil,
1995) records a childs positive and negative social interactions that occur during a 15-second
interval for 20 minutes of free play activities. Several studies have supported the validity and
efficacy of the ESP SOC (see Feil, Severson, & Walker, 1998). The percentage of observed
intervals scores for the Parallel Play and Positive Social Engagement codes were collapsed to
form the larger category Aggregated Social Play and served as a dependent and independent
variable for this investigation. The percentage of observed intervals score for the Solitary Play
code served as an independent variable for this investigation. The percentage of observed inter-
vals scores for the Negative Verbal, Negative Physical, and Disruptive Behavior codes were
collapsed to form the larger category Aggregated Aggression and served as an independent vari-
able for this investigation. Operational definitions for these behaviors are provided in the ESP
SOC manual (see Walker et al., 1995).

Procedures
As part of the larger study, teachers were sent questionnaire packets that included the rating
scales used in this investigation; compensation in the amount of $50.00 was provided for their

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

118 Journal of Psychoeducational Assessment 29(2)

Table 2. Summary of Pearson Correlations for Variables Predicting SSRS-T: Social Skills and ESP SOC:
Aggregated Social Play (n = 105)
Variable 1 2 3 4 5 6
1. SSRS-T: Social Skills 1.000 -.514** -.513** -.166 .204* -.145
2.CTRS-R: Social Problems 1.000 .339** .096 -.106 -.023
3. TRF: Aggressive Behavior 1.000 .054 -.071 .279**
4. ESP SOC: Aggregated Social Play 1.000 -.754** -.033
5.ESP SOC: Solitary Play 1.000 -.172
6.ESP SOC: Aggregated Aggression 1.000
Note: SSRS-T = Social Skills Rating SystemTeacher Form: Social Skills scale; CTRS-R = Connerss Teacher Rating
ScaleRevised: Social Problems scale; TRF = Teachers Report Form: Aggressive Behavior scale; ESP SOC = Early
Screening Profile Social Observation Code: Aggregated Social Play, Solitary Play, and Aggregated Aggression variables.
*p < .05. **p < .01.

participation. Trained graduate student data collectors observed participants within their school
environments; a single baseline observation during 20 minutes of unstructured activities was
included in the data analysis for each participant. Data collectors were blind to the purpose and
hypotheses of the study. All data collectors achieved a minimum of 90% proficiency across three
consecutive viewings of training video tapes. Interobserver agreement was calculated on 30% of
the direct observations. The average kappa coefficient across all observed behaviors was .85
(range = .81-.91). The average agreement for occurrence across all observed behaviors was 85%
(range = 81% to 90%). The average agreement for nonoccurrence across all observed behaviors
was 94% (range = 85% to 100%).

Results
Analysis of Demographic Variables
A total of 26 one-way analyses of variance (ANOVAs) were calculated with the demographic
variables, 13 with each dependent variable. A Bonferroni adjustment of the p values (p = .05)
was performed resulting in an adjusted level of p < .01 to determine statistical significance. No
significant demographic variables were found.

Analysis of Indirect and Direct Measures

Correlations are reported in Table 2. There was a strong significant negative correlation between
the ESP SOC: Aggregated Social Play and Solitary Play variables (r = -.754, p < .01) and a mod-
erate significant negative correlation between the SSRS-T: Social Skills scale and both the CTRS-
R: Social Problems scale (r = -.514, p < .01) and the TRF: Aggressive Behavior scale (r = -.513,
p < .01). There were weak significant positive correlations between the CTRS-R: Social Problems
scale and the TRF: Aggressive Behavior scale (r = .339, p < .01), between the TRF: Aggressive
Behavior scale and ESP SOC: Aggregated Aggression variable (r = .279, p < .01), and between
the SSRS-T: Social Skills scale and the ESP SOC: Solitary Play variable (r = .204, p < .05).
Four hierarchical multiple regression analyses were conducted to examine the relationship
between direct and indirect methods in predicting the social skills of preschool children at risk
for ADHD. Multicollinearity diagnostics revealed acceptable tolerance values (range = .389-.970)
for all variables (Brace, Kemp, & Snelgar, 2006).
The first and second regression analyses assessed the ability of the indirect (CTRS-R: Social
Problems and TRF: Aggressive Behavior) and direct (ESP SOC: Aggregated Social Play,

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

Thomas et al. 119

Table 3. Summary of the First Hierarchical Regression Analysis for Variables Predicting SSRS-T: Social
Skills Scale (n = 105)
Variable B SE B b
Step 1
CTRS-R: Social Problems -.778 .166 -.384*
TRF: Aggressive Behavior -.337 .072 -.383*
Step 2
CTRS-R: Social Problems -.761 .168 -.375*
TRF: Aggressive Behavior -.324 .076 -.368*
ESP SOC: Aggregated Social Play -.013 .061 -.026
ESP SOC: Solitary Play .050 .054 .114
ESP SOC: Aggregated Aggression -.037 .097 -.032
Note: SSRS-T = Social Skills Rating SystemTeacher Form: Social Skills scale; CTRS-R = Connerss Teacher Rating
ScaleRevised: Social Problems Scale; TRF = Teachers Report Form: Aggressive Behavior scale; ESP SOC = Early
Screening Profile Social Observation Code: Aggregated Social Play, Solitary Play, and Aggregated Aggression variables.
R2 = .394 for Step 1; DR2 = .020 for Step 2.
*p < .05.

Table 4. Summary of the Second Hierarchical Regression Analysis for Variables Predicting SSRS-T:
Social Skills Scale (n = 105)
Variable B SE B b
Step 1
ESP SOC: Aggregated Social Play -.037 .076 -.074
ESP SOC: Solitary Play .055 .068 .127
ESP SOC: Aggregated Aggression -.143 .116 -.126
Step 2
ESP SOC: Aggregated Social Play -.013 .061 -.026
ESP SOC: Solitary Play .050 .054 .114
ESP SOC: Aggregated Aggression -.037 .097 -.032
CTRS-R: Social Problems -.761 .168 -.375*
TRF: Aggressive Behavior -.324 .076 -.368*
Note: SSRS-T = Social Skills Rating SystemTeacher Form: Social Skills scale; ESP SOC = Early Screening Profile Social
Observation Code: Aggregated Social Play, Solitary Play, and Aggregated Aggression variables; CTRS-R = Connerss
Teacher Rating ScaleRevised: Social Problems Scale; TRF = Teachers Report Form: Aggressive Behavior scale.
R2 = .056 for Step 1; DR2 = .357 for Step 2.
*p < .05.

Solitary Play, and Aggregated Aggression) methods to predict the SSRS-T: Social Skills scale.
As shown in Table 3, the indirect methods explained 39.4% of the variance in predicting the
SSRS-T: Social Skills scale. After entry of the direct methods the total variance explained by the
model as a whole was 41.4%, F(5, 99) = 13.97, p < .001, resulting in a nonsignificant R2 change, F
change (3, 99) = 1.11, p = .347. In the final model, only the two indirect methods were statisti-
cally significant, with the CTRS-R: Social Problems scale (b = -.38, p < .001) and TRF: Aggres-
sive Behavior scale (b = -.37, p < .001) reporting similar beta values. As shown in Table 4, the
second analysis confirms these findings. The direct methods explained 5.6% of the variance in
predicting the SSRS-T: Social Skills scale. After entry of the indirect methods, the total variance
explained by the model as a whole was 41.4%, F(5, 99) = 13.97, p < .001, resulting in a significant
R2 change, F change(2, 99) = 30.15, p < .001. In the final model, only the two indirect methods
were statistically significant.

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

120 Journal of Psychoeducational Assessment 29(2)

Table 5. Summary of the Third Hierarchical Regression Analysis for Variables Predicting ESP SOC:
Aggregated Social Play Variable (n = 105)
Variable B SE B b
Step 1
ESP SOC: Solitary Play -.688 .056 -.783*
ESP SOC: Aggregated Aggression -.383 .146 -.168*
Step 2
ESP SOC: Solitary Play -.686 .058 -.781*
ESP SOC: Aggregated Aggression -.418 .155 -.183*
SSRS-T: Social Skills -.035 .167 -.018
CTRS-R: Social Problems -.065 .307 -.016
TRF: Aggressive Behavior .081 .137 .046
Note: ESP SOC = Early Screening Profile Social Observation Code: Aggregated Social Play, Solitary Play, and
Aggregated Aggression variables; SSRS-T = Social Skills Rating SystemTeacher Form: Social Skills scale; CTRS-R =
Connerss Teacher Rating ScaleRevised: Social Problems scale; TRF = Teachers Report Form: Aggressive Behavior
scale. R2 = .596 for Step 1; DR2 = .002 for Step 2.
*p < .05.

Table 6. Summary of the Fourth Hierarchical Regression Analysis for Variables Predicting ESP SOC:
Aggregated Social Play Variable (n = 105)
Variable B SE B b
Step 1
SSRS-T: Social Skills -.359 .252 -.179
CTRS-R: Social Problems .076 .467 .019
TRF: Aggressive Behavior -.079 .203 -.045
Step 2
SSRS-T: Social Skills -.035 .167 -.018
CTRS-R: Social Problems -.065 .307 -.016
TRF: Aggressive Behavior .081 .137 .046
ESP SOC: Solitary Play -.686 .058 -.781*
ESP SOC: Aggregated Aggression -.418 .155 -.183*
Note: ESP SOC = Early Screening Profile Social Observation Code: Aggregated Social Play variable; SSRS-T = Social
Skills Rating SystemTeacher Form: Social Skills scale; CTRS-R = Connerss Teacher Rating ScaleRevised: Social
Problems scale; TRF = Teachers Report Form: Aggressive Behavior scale; ESP SOC = Early Screening Profile Social
Observation Code: Solitary Play and Aggregated Aggression variables. R2 = .029 for Step 1; DR2 = .569 for Step 2.
*p < .05.

The third and fourth regression analyses assessed the ability of the direct (ESP SOC: Solitary
Play and Aggregated Aggression) and indirect (SSRS-T: Social Skills, CTRS-R: Social Prob-
lems, and TRF: Aggressive Behavior) methods to predict the ESP SOC: Aggregated Social Play
variable. As shown in Table 5, the direct methods explained 59.6% of the variance in predicting
the ESP SOC: Aggregated Social Play variable. After entry of the indirect methods the total vari-
ance explained by the model as a whole was 59.8%, F(5, 99) = 29.51, p < .001, resulting in a
nonsignificant R2 change, F change(3, 99) = .20, p = .895. In the final model, only the two direct
methods were statistically significant, with the ESP SOC: Solitary Play variable reporting a higher
beta value (b = -.78, p < .001) than the ESP SOC: Aggregated Aggression variable (b = -.18,
p = .008). As shown in Table 6, the fourth analysis confirms these findings. The indirect methods
explained 2.9% of the variance in predicting the ESP SOC: Aggregated Social Play variable.
After entry of the direct methods, the total variance explained by the model as a whole was

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

Thomas et al. 121

59.8%, F(5, 99) = 29.51, p < .001, resulting in a significant R2 change, F change(2, 99) = 70.20,
p < .001. In the final model, only the two direct methods were statistically significant.

Discussion
The purpose of this study was to examine the relative contribution of direct and indirect methods
in predicting the social skills of preschool children at risk for ADHD. Correlational analysis
yielded several significant findings. Lower levels of observed social play were associated with
higher levels of observed solitary play during free play activities. This was not surprising given
that social play and solitary play are incompatible behaviors. As expected, lower levels of teacher-
rated social problems and aggressive behaviors were associated with higher levels of teacher-
rated social skills. Commensurate with the work of McEvoy et al. (2003), a significant positive
correlation was observed between teacher ratings and direct observations of aggressive behavior.
Interestingly, there was a weak significant positive correlation between teacher ratings of social
skills and direct observations of solitary play. One might speculate that students who engage in
high levels of solitary play may appear to be socially skilled to teachers who may mistake the
quiet and personal nature of play with good behavior.
Four hierarchical multiple regression analyses were conducted to examine whether direct
observations of social play, solitary play, and aggression provided unique variance in predicting
teacher-rated social skills for preschool children at risk for ADHD (first and second regression
analyses) and whether teacher ratings of social skills, social problems, and aggression provided
unique variance in predicting direct observations of social play for preschool children at risk for
ADHD (third and fourth regression analyses). In the current investigation, indirect outcome mea-
sure did not strongly account for observed incidence of aggression, solitary play, and social play,
and the direct outcome measure did not strongly account for teachers perceptions of a childs
social skills, social problems, and aggression. In particular, teachers ratings of social problems
and aggression were more powerful predictors of social functioning as measured by an indirect
outcome measure (SSRS-T: Social Skills scale) than direct observations of incidents of social
play, solitary play, and aggression. Likewise, direct observations of aggression and solitary play
were more powerful predictors of social functioning as measured by a direct outcome measure
(ESP SOC: Aggregated Social Play variable) than indirect measures of teachers ratings of social
skills, social problems, and aggression. This finding is important, because it suggests that indi-
rect and direct methods may not be measuring the same aspects of social skills for this popula-
tion. Therefore, assessment results may vary as a function of the technique employed. Because each
of the two assessment methods may provide differing results, adding a direct observation to the
commonly used rating scales would yield a more complete and discerning evaluation of social
skills. This is supported by the work of Odom et al. (2008) who suggest that direct observations
and rating scales have unique method variance; therefore, the combination of the two methods is
essential for a comprehensive assessment of a childs social skills.

Limitations and Directions for Future Research

There are several limitations to this study. Both the indirect predictor and outcome variables
were measured by teacher informant report and came from related sources (i.e., rating scales).
Similarly, both the direct predictor and outcome variables were measured by independent observ-
ers and came from the same source (i.e., ESP SOC). The shared method and source variance may
have enhanced the probability of finding significant relationships (Campbell & Fiske, 1959).
Future research should incorporate multiple sources and methods of information, including
informant reports from several sources (e.g., teachers, parents, peers, and self), as well as direct

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

122 Journal of Psychoeducational Assessment 29(2)

observations across times and settings. Examination of differences in the construct of social
skills measured by indirect and direct methods is warranted. Alternatively, future researchers
may want to examine the relationship between Direct Behavior Rating (DBR) and indirect and
direct measures of social skills. DBR is a hybrid method, which combines the characteristics of
behavior rating scales and direct observations to measure social skills and social problem
(Chafouleas, Christ, Riley-Tillman, Briesch, & Chanese, 2007) and might bridge the apparent
differences found between direct and indirect methods in the assessment of social skills.
An additional limitation was related to the particular population used in this study. Partici-
pants were predominantly Caucasian and the outcomes for preschool children at risk for ADHD
may not be applicable to school-age children or other clinical samples. Future research may seek
to improve generalizability by sampling from a more diverse population and/or different clinical
samples. Furthermore, it may be helpful to follow these children for several years to assess
whether social skills problems continue or whether social functioning improves, as there may be
other factors influencing the results.

Conclusions
Overall, the findings of this study are consistent with previous research on social skills assess-
ment. A comprehensive assessment of social competence should be based on a multimethod model
that draws information from a variety of sources, including behavior rating scales and direct obser-
vations. As suggested by this studys findings and previous research practitioners should base
their assessment decisions on the assumption that information from different sources (i.e., rating
scales and direct observations) contributes unique variance to our understanding of a childs
social competence. When aiming to form a comprehensive description of the social competence
of a young child, the assessment should be based on multiple components that capture informa-
tion from all necessary sources, assuming that each source contributes both a shared and unique
variance to the description. A single source assessment will most likely not present a full and
complete description of a young childs social competence.

Authors Note
The study was completed in partially meeting the requirements of the masters degree thesis equivalent at
Lehigh University.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interests with respect to the authorship and/or publication of
this article.

Funding
The authors disclosed receipt of the following financial support for the research and/or authorship of this
article:
Preparation of this manuscript was supported by National Institute of Mental Health grant R01-61563.

References
Achenbach, T. M. (2001). Teachers Report Form/5-18. Burlington: University of Vermont, Research
Center for Children, Youth, and Families.
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.,
text revision). Washington, DC: Author.
Bagwell, C. L., Schmidt, M. E., Newcomb, A. F., & Bukowski, W. M. (2001). Friendship and peer rejection
as predictors of adult adjustment. New Directions for Child and Adolescent Development, 91, 25-49.

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

Thomas et al. 123

Barkley, R. A. (2006). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment
(3rd ed.). New York, NY: Guilford Press.
Brace, N., Kemp, R., & Snelgar, R. (2006). SPSS for psychologists: A guide to data analysis using SPSS for
Windows, Versions 12 and 13 (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.
Buhs, E. S., Ladd, G. W., & Herald, S. L. (2006). Peer exclusion and victimization: Processes that medi-
ate the relation between peer group rejection and childrens classroom engagement and achievement.
Journal of Educational Psychology, 98, 1-13.
Burns, G. L., Walsh, J. A., Patterson, D. R., Holte, C. S., Sommers-Flanagan, R., & Parker, C. M. (2000).
Attention deficit and disruptive behavior disorder symptoms. European Journal of Psychological
Assessments, 17, 22-35.
Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-
multimethod matrix. Psychological Bulletin, 56, 81-105.
Chafouleas, S. M., Christ, T. J., Riley-Tillman, T. C., Briesch, A. M., & Chanese, J. A. M. (2007). Gener-
alizability and dependability of direct behavior ratings to assess social behavior of preschoolers. School
Psychology Review, 36, 63-79.
Conners, C. K. (1997). Conners rating scales manual: Instruments for use with children and adolescents.
North Tonawanda, NY: Multi-Health Systems.
Cost, H. C., & Simpson, R. G. (2004). A behavioral assessment investigation: Direct observations and
behavior ratings. Research in the Schools, 11(2), 51-60.
Demaray, M. K., Elting, J., & Schaefer, K. (2003). Assessment of attention-deficit/hyperactivity disorder
(ADHD): A comparative evaluation of five, commonly used, published rating scales. Psychology in the
Schools, 40, 341-361.
DuPaul, G. J., McGoey, K. E., Eckert, T. L., & VanBrakle, J. (2001). Preschool children with attention-
deficit/hyperactivity disorder: Impairment in behavioral, social, and school functioning. Journal of the
American Academy of Child and Adolescent Psychiatry, 40, 508-515.
Feil, E. G., Severson, H. H., & Walker, H. M. (1998). Screening for emotional and behavioral delays: The
early screening project. Journal of Early Intervention, 21, 252-266.
Flook, L., Repetti, R. L., & Ullman, J. B. (2005). Classroom social experiences as predictors of academic
performance. Developmental Psychology, 41, 319-327.
Gresham, F. M. (1982). Social skills: Principles, procedures, and practices. Des Moines: Iowa Department
of Public Instruction School Psychological Services.
Gresham, F. M., & Elliott, S. N. (1990). Social skills rating system. Circle Pines, MN: American Guidance
Service.
Hintze, J. M., Volpe, R. J., & Shapiro, E. S. (2002). Best practices in the systematic direct observation of stu-
dent behavior. In A. Thomas & J. Grimes (Eds.), Best practices in school psychology IV (pp. 993-1006).
Bethesda, MD: National Association of School Psychologists.
Hodgens, J. B., Cole, J., & Boldizar, J. (2000). Peer-based differences among boys with ADHD. Journal of
Clinical Child and Adolescent Psychology, 29, 443-452.
Kern, L., DuPaul, G. J., Volpe, R. J., Sokol, N. G., Lutz, J. G., Arbolino, L. A., . . ., VanBrakle, J. D. (2007).
Multisetting assessment-based intervention for young children at-risk for attention deficit hyperactivity
disorder: Initial effects on academic and behavioral functioning. School Psychology Review, 36, 237-255.
McElwain, N. L., Olson, S. L., & Volling, B. L. (2002). Concurrent and longitudinal associations among
preschool boys conflict management, disruptive behavior, and peer rejection. Early Education and
Development, 13, 245-263.
McEvoy, M. A., Estrem, T. L., Rodriguez, M. C., & Olson, M. L. (2003). Assessing relational and physi-
cal aggression among preschool children: Intermethod agreement. Topics in Early Childhood Special
Education, 23, 53-63.
Mrug, S., Hoza, B., & Gerdes, A. C. (2001). Children with attention-deficit/hyperactivity disorder: Peer
relationships and peer-oriented interventions. In D. W. Nangle & A. C. Erdley (Eds.), The role of friend-
ship in psychosocial adjustment (pp. 51-77). San Francisco, CA: Jossey-Bass.

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015

124 Journal of Psychoeducational Assessment 29(2)

Odom, S. L., McConnell, S. R., & Brown, W. H. (2008). Social competence of young children: Con-
ceptualization, assessment, and influences. In. W. H. Brown, S. L. Odom, & S. R. McConnell (Eds.),
Social competence of young children: Risk, disability, and intervention (pp. 3-30). Baltimore, MD:
Paul H. Brookes.
Semrud-Clikeman, M., & Schafer, V. (2000). Social and emotional competence in children with ADHD
and/or learning disabilities. Journal of Psychotherapy in Independent Practice, 1(4), 3-19.
Shaffer, D., Fisher, P., Lucas, C. P., Dulcan, M. K., & Schwab-Stone, M. E. (2000). NIMH Diagnostic
Interview Schedule of Children, Version IV (NIMH DISC-IV): Description, differences from previous
versions and reliability of some common diagnoses. Journal of the American Academy of Child and
Adolescent Psychiatry, 39, 28-38.
Shaffer D., Gould, M. S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., & Aluwahlia, S. (1983). A chil-
drens global assessment scale (CGAS). Archives of General Psychiatry, 40, 1228-1231.
Walker, H. M., Severson, H. H., & Feil, E. G. (1995). Early screening project: A proven child find process.
Longmont, CO: Sopris West.
Wheeler, J., & Carlson, C. L. (1994). The social functioning of children with ADHD with hyperactivity
and ADD without hyperactivity: A comparison of their peer relations and social deficits. Journal of
Emotional and Behavioral Disorders, 2, 2-12.

Downloaded from jpa.sagepub.com at Universidad de Valencia on July 17, 2015