Standards Performance Continuum: Development and Validation of a Measure of Effective Pedagogy Author(s): R. William Doherty, R.

Soleste Hilberg, Georgia Epaloose, Roland G. Tharp Reviewed work(s): Source: The Journal of Educational Research, Vol. 96, No. 2 (Nov. - Dec., 2002), pp. 78-89 Published by: Taylor & Francis, Ltd. Stable URL: http://www.jstor.org/stable/27542417 . Accessed: 23/04/2012 23:23
Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org.

Taylor & Francis, Ltd. is collaborating with JSTOR to digitize, preserve and extend access to The Journal of Educational Research.

http://www.jstor.org

Standards Development Measure

R. WILLIAM
University

Performance and of Effective Validation

Continuum: of a

Pedagogy
GEORGIA EPALOOSE
Zuni Public School District, ROLAND G. THARP
New Mexico of California, University Santa Cruz

DOHERTY
Santa Cruz

R. SOLESTE HILBERG
of California,

ABSTRACT
idation measure for Effective as an teachers' of for the

The authors present the development

Standards The Performance teacher Continuum of the assessing performance authors created on the and

and val
a

(SPC), Standards to serve between effective and was

Pedagogy. for instrument use of these

the SPC

research standards

relationship instructional

for teacher ness, as a guide professional development, tool for education reform. The goal of this research struct and a quantitative instrument The of that readily interpretable. studies evidence provide standards

as a

a quantitative instrument that is easily and practical, and whose data scored, readily interpretable, provide valid interpretations of the constructs measured on Standards for Educational and Psy (Joint Committee of the American Educational Research chological Testing the American Psychological Association, and Association, in Education, the National Council on Measurement 1999). to construct Standards for Effective Pedagogy to Gallimore and Tharp (1992), According learning occurs best when (a) collaboration between students and teachers is accompanied (b) instructional by discussion; to students' prior connected activities are meaningfully experience
and occurs

was

to con

is practical, scored, easily are discussed, and 3 concurrent interrater reliability, the validity

and criterion-related validity, validity supporting of interpretations of data gathered with the SPC. Key words: education reform, Standards assessment

Performance

Con

tinuum,

teacher

performance

and knowledge;
within the

and (c) instruction

zone of proximal

is dialogic
develop

learner's

Estrada, Dalton, and Yamauchi (2000) identified Tharp, five teaching standards to effectively teach students at risk of educational failure because of cultural, linguistic, or economic risk factors. There is growing evidence of the
effectiveness dards. intact For of classroom implementation of study to these stan example, were classes in a quasi-experimental randomly assigned in which or

(1978, p. 86) as "the dif (ZPD), defined by Vygotsky level as deter ference between the actual developmental mined by independent problem solving and the level of potential development as determined through problem solv ing under adult guidance or in collaboration with more ment
capable peers."

treatment

conditions, experimental Hilberg, Tharp, and DeGeest students in middle (2000) found that Native American school mathematics classes taught with the standards had and learned more improved attitudes toward mathematics
mathematics, as evidenced by higher scores on a posttest

On the basis of these three propositions, leagues (2000) formulated five pedagogy
first and standard, students joint working productive together on activity, a common

Tharp and col standards. The

involves product teachers or goal

with opportunities
standard, language

to converse about their work. The second

and literacy development, involves

administered 2 weeks after the intervention. Estrada (2000) found that greater use of the standards in literacy instruction for English-language learners was associated with higher there is a broad foundation reading performance. Although of research literature in support of the Standards for Effec tive Pedagogy (Tharp et al., 2000), to date there is no mea
sure available to assess their use in classroom instruction.

in the language and literacy of competence developing instruction and in the academic disciplines throughout all instructional activities. The third standard, contextualiza
tion, situates new academic content in contexts familiar to

students
from

to connect
school, activities,

it to prior knowledge
or community. uses complex The tasks

or experience
fourth standard, the requiring

home,

challenging

This article presents the development of the Standards Per formance Continuum (SPC), a 5-point rubric that provides
a quantitative measure for observing and assessing teaching

Address Research ifornia,

performance

of these standards. The goal of this research 78

to R. William Center Doherty, for correspondence on Education, & Excellence, University of Cal Diversity Santa Cruz, CA 95064. (E-mail: wdoherty@cats.ucsc.edu)

November/December

2002 [Vol. 96(No. 2)]

79 sonal and social relevance, and relationship to prior knowl edge and experience) result in better learning and recall than do strategies focused on more superficial characteristics important, a small (Baddeley; Craik & Lockhart). More amount of conceptual processing produces more learning than does a large amount of more superficial processing (Craik & Tulving). Each of the pedagogy standards proposed by Tharp and his colleagues promotes conceptual elabora
tion, and, consequently, assists learning.

to achieve an academic application of content knowledge is a The fifth standard, instructional conversation, goal.
planned, goal-directed conversation between a teacher and

a small group of students. Theoretical The

learning

Foundation

of the SPC on the sociocultural

novices collaborate and

SPC was
occurs best

founded
when

tenet that
converse

with more experienced and more knowledgeable others on a shared task (Vygotsky, 1978). From this perspective, learn ing and development are inherently social, and the construc tion of knowledge and meaning is situated within a socially created context. Teaching is assisting student performance with the goal of increasing that which students can do unas sisted by the teacher; learning represents movement through the ZPD in the direction of increased competence and auton 1988). omy (Tharp et al, 2000; Tharp & Gallimore, From a cognitive perspective, learning is the process of (Badde associating new information with prior knowledge ley, 1990). For learning to occur, and for new information to
be retained in memory, learners must engage in some sort of

Dimensions

of the SPC

The first step in developing the SPC was to conceptualize each of the standards being the dimensions underlying assessed. Table 1 contains the critical distinctions of each dimension used to define theoretically equivalent changes in levels of enactment of each of the standards described below. Joint productive activity (JPA). In joint productive activ
ities, achieve participants a common collaborate goal. During to generate a product collab or such goal-directed

oration, participants with varying

tise interact; exchange information;

levels of skill and exper

and share their views,

elaboration of the new information (Dansereau, 1988; Slavin, 1996;Wittrock, 1986). Elaboration refers to the internal pro of new and prior knowledge in ameaningful context cessing (Baddeley; Craik & Lockhart,
Furthermore, tual elaborative of new characteristics

1972; Craik & Tulving,

that focus (i.e., on its meaning,

1975).
per

strategies information

the concep

or thinking rationales, and problem-solving perceptions, The dimension underlying JPA is collaboration, strategies. and the continuum ranges from students working on indi vidual products to students and teacher collaborating on a joint product (see Table 1). The teacher's role in JPA is to language, skills, participate as a full collaborator; model

Table

1.?Dimensions

Underlying

the Standards

for Effective

Pedagogy

Standard

Dimension

Not

enacted

Partially Students product. collaborate

enacted

Fully

enacted

JPA

Collaboration

Students work products.

on individual

on a joint

The teacher and a small group of on a joint students collaborate product. The teacher provides systematic assistance in activities that gen erate extended language use and of content vocabu development lary.

LLD

Language

use

Instruction talk.

is dominated

by teacher

for Activities provide opportunities students' sustained reading, writing, or speaking.

CTX

Connected learning New abstract, information is provided in an manner. disconnected are incidentally connected Activities to students' prior knowledge and experience. The teacher integrates new infor mation with what students already know from home, school, or com munity.

CA

Cognitive complexity Activities rely on repetition, recall, and convergent thinking to generate declarative or procedural knowledge. Activities concepts. are connected to academic stan The teacher uses challenging dards and feedback to advance to more student understanding levels. complex The teacher and a small group of students converse about their and rationales views, judgments, in relation to an academic topic to achieve an instructional goal.

IC

Student-teacher dialogue Recitation whole-class dominates. script, lecture, or instruction pre The teacher and a small group of students converse on an academic topic.

Note. JPA = joint productive activity; LLD = language and literacy development; CTX = contextualization; CA = challenging activities; IC = instructional conversation.

80 The Journal of Educational Research

and and assess and

problem-solving student

strategies; performance. in many

responsive

ly assist Although

similar

respects

to some

cooperative

learning strategies, not all cooperative learning rises to the level of joint productive activity. The teacher's role inmany forms of cooperative learning is to move from group to group, checking student work and providing advice and assistance (Cohen & Lotan, 1997; Davidson, 1997; Heller, to Tharp and col Keith, & Anderson, 1997). According leagues (2000), however, to most effectively assist student the teacher must be a full collaborator with performance, students in the activity. For assistance to be effective, it must be understandable to the students, responsive to their needs, correct, and sufficiently elaborated to allow them the to correct misconceptions (Webb, Troper, & opportunity Fall, 1995). In addition, students must have and use the to apply the assistance in solving problems opportunity (Vedder, 1988).
Language and literacy development. Language proficien

Instruction that is situated in problems and issues from familiar contexts provides opportunities for connecting stu to more abstract levels of dents' informal understandings between the understanding. Even incidental connections
formal cepts concepts can make of the the situation learning new information and more everyday relevant con and

and, consequently, more likely to be attended meaningful, to (Cacioppo & Petty, 1982). Integrating new information in familiar contexts facilitates the organization of the new information into long-term memory by priming, or making more available, associated knowledge (Collins & Loftus, 1975). The increased availability of associated knowledge results in a more conceptual form of processing (Craik & Lockhart, 1972; Craik & Tulving, 1975), which, in turn, improves retention and facilitates recall (Baddeley, 1990). Challenging activities. In addition to teaching course con tent, challenging
such as analyzing,

activities

teach higher order thinking skills

evaluating, generalizing,

categorizing,

cy is correlated highly with overall school achievement. The of effective and appropriate language use is to learning, thinking, problem solving (Ash of worth, 1979; Collier, 1995), and the co-construction 1992). The dimension (Wells & Chang-Wells, knowledge underlying the development of language and literacy is lan development fundamental
guage use, and the continuum ranges from instruction dom

synthesizing and summarizing, exploring patterns, determin ing causal relationships, and problem solving (Tharp & Gal limore, 1988). Challenging activities require (a) appropriate leveling of tasks so that all students are stretched to grow within their ZPDs, (b) opportunities to use and apply new information, and (c) a balance between challenge and assis
tance to advance student understanding to more complex lev

inated by teacher talk to instruction in which academic lan guage use by students predominates. The critical role of the teacher is to assist student language development through questioning, rephrasing, and modeling. Language provides the raw material for cognition and is the interface through which prior knowledge and experience, the foundation for new learning, are accessed (Rueda, 1999). Purposeful discourse influences such mental functions as attention, logical memory, and the formation of concepts (Wertsch, 1985), and helps learners reorganize material in
new ways, resolve inconsistencies, and fill in gaps in under

els (Tharp et al., 2000). The dimension underlying challeng ing activities is cognitive complexity. The critical role of the
teacher is to assess and assist student understanding.

standards and feedback on performance are Challenging essential elements of challenging activities. Standards, or (Locke, Shaw, Saari, & Latham, goals, enhance motivation 1981), and when both standards and feedback are present,
self-evaluative performance mechanisms to greater motivate accordance learners with the to modify standards their (Ban

dura & Cervone,

1983; Bandura & Schunk, 1981). Self-eval uation, according to Schraw and Sperling-Dennison (1994), involves planning of (goal setting and the allocation
resources prior to learning); information management strate

standings
elaborate

(Webb et al.,
conceptualizations

1995). Discourse
and strengthens

generates more
connections

gies (organizing, elaborating,

monitoring (assessment of

summarizing);
one's learning or

comprehension
use of strate

between new information and prior learning (Wittrock, 1990). Most important, language is best learned through use and purposeful conversation (Tharp et al., meaningful 2000). The most effective
generates language use

instruction, therefore, is thatwhich

requires that students gain suffi

and

cient mastery

of language to complete
Connecting

the learning task.

content to

Contextualization.

educational

students' personal
iar, everyday knowledge enable

lives and providing

about which students to make

instruction
students sense of new

in famil
prior informa

gies); and evaluation (analyzing learning and strategies after the learning episode). This regulation of cognition typifies (Brown, 1987; Flavell, 1987), generating not metacognition only declarative knowledge (knowledge about self and strate gies) and procedural knowledge (knowledge of how to use but also promoting the development of condition strategies) al knowledge?knowledge of when and why to use strategies (Schraw & Sperling-Dennison,
Instructional teacher-student conversation. academic

contexts

have

1994).
The amount are and two of quality the most of

tion and to construct new knowledge.

lying contextualization is connected

The dimension
learning, and

under
the con

interactions

tinuum ranges from instruction in which new information

presented in an abstract, atomistic, and decontextualized

is

manner
students

to instruction
know.

edge and experience.

already

that is integrated with prior knowl The teacher's task is to build on what

important factors in student learning (Wang, Haertel, & instructional 1994). The dimension underlying Walberg, is student-teacher conversation and the (IC) dialogue, dimension ranges from informal, nonacademic discourse in
whole-class versation settings between the con to a goal-directed, inclusive fully teacher and a small of students. group

November/December

2002 [Vol. 96(No. 2)]

81

The critical task of the teacher is to elicit and extend student

talk on academic, personal, social, and cultural experience

Procedures

in relation to an academic topic (Tharp & Gallimore, 1988, 1991; Yamauchi & Tharp, 1995). to Tharp and his colleagues According (Tharp & Gal limore, 1991; Tharp & Yamauchi, 1994), student-teacher dialogue is the most effective means to assist the develop ment of basic thinking skills and the ability to form,
express, and exchange ideas. Fully inclusive, small-group

Prior to data collection, we used archived videotapes of classroom instruction to train four researchers familiar with
the standards. When trainees' ratings reached a predeter

ICs are stressed because they increase the participation of all students, including the more passive learners (Menke & Pressley, 1994). As such, ICs enable the teacher to (a) assist all students' conceptual elaboration of new content; (b) con
textualize instruction in the learner's experience base

mined level of agreement (r .95) with the lead researcher, formal data collection began with the use of live (N = 34) and videotaped (N = 32) observations of the teachers in the pairs of sample. Over four phases of data collection, observers made SPC ratings, with different pairs in each
phase. After each phase, we assessed interrater agreement

and modified
levels in an

cell criteria to (a) clarify distinctions

attempt to increase agreement between

between
coders

and (b) broaden the SPC's applicability

educational contexts and grade levels.

to a greater range of

(Ochs, 1982); and (c) individualize instruction dents' varied ability levels, assess and assist maintain appropriate cognitive challenge 1991; Tharp et al, 2000; Tharp & Gallimore,

tomatch

stu

learning, and (Goldenberg, 1988).

Results Ratings made with the final version of the SPC (Phase 4) rank exhibited high intercoder agreement; Spearman's order coefficients ranged from .86 to .98 on the subscales and .96 for the SPC total score, found by summing across the five subscales. Kendall's Ws ranged from .80 to .88 for
subscales, and .96 for the total score.

STUDY 1
After conceptualizing the dimensions the underlying standards, we defined five levels of enactment (values standard is not ranged from 0 to 4): (a) not observed?the of the standard are imple present; (b) emerging?elements standard is partially imple mented; (c) developing?the standard is fully implemented; mented; (d) enacting?the and (e) integrating?at least three standards are implement ed simultaneously in an instructional activity. Table 2 con tains the final version of the measure, with cell criteria defining enactment of each standard at each of these five levels. The next step in developing the SPC was to assess interrater reliability. Observations were conducted by pairs
of trained The observers, procedures and and intercoder results are agreement presented was below. assessed.

To determine whether the reliability of SPC scores varied between live and videotaped observations, we compared interrater agreement using Fisher's z transformations of the correlation coefficients. Because Phase 4 consisted exclu sively of live observations, we based comparisons on data from the first three phases. Even with these less reliable ver sions of the SPC, interrater agreement for ratings on live
observations, = r(17) = .67, and videotaped observations,

= .79 r(31) (bothps

.76, p .54, two-tailed.

< .01), did not differ significantly,

z=

STUDY 2
Method
Participants

The next step in the development of the SPC was to gen erate estimates of the concurrent validity of its ratings. Con current validity is estimated by the degree to which scores
on an instrument the are same related constructs to scores on other instruments at the same measuring administered

The sample consisted of 24 public school teachers from amiddle school in an American Indian pueblo inNew Mex
ico. temic The school reform was effort engaged with in a 5-year university-based collaborative researchers sys

time (Fraenkel & Wallen, 1993). To establish concurrent we used the SPC in conjunction with instruments validity,
measuring similar pedagogical constructs, and we estimat

focused on the Standards for Effective Pedagogy. Students' standardized test scores at this school ranked 89th, 88th, and 87th out of 89 schools in the state for six, seventh, and eighth grades, respectively. Students were 99.7% Native 91% received free or reduced-price lunches. American; at the middle school was exhaustive and included Sampling all teachers. The sample was 79% (n = 19) Euro-American = and 21% Native American 5) teachers, 54% men and (n 46% women. Teachers' ages ranged from 22 to 69 years, and years of teaching experience ranged from 1 to 41. Grade levels ranged from sixth to eighth, and courses
spanned the spectrum of subject areas.

ed it by the correlation
those of the other measures.

between

ratings of the SPC and

Method Participants Participants were 42 teachers from 13 public elementary schools in a south Florida school district. All schools were in a longitudinal study comparing the effec participating
tiveness of state-mandated comprehensive school reform

Table

2.?Standards

Performance

Continuum

(Rubric

for Observing

Classroom

Enactments

of CREDE's

Standards

for Effective

Pedagogy)

Variable General definition

Not

observed is

Emerging One or more elements standard are enacted. of the

Developing The teacher designs and enacts activities that demon strate a partial enactment of the standard. The

Enacting teacher designs and enacts a activities that demonstrate complete enactment of the standard. The teacher and a small group of students collaborate on a joint product. The

The standard not observed.

teacher assists in a strate skillf

tiple standa The

Joint productive activity Teacher producing and students together

Joint productive activity is not observed.

Students are seated with a part ner or group, AND (a) collabo rate or assist one another, OR (b) are instructed in how to work in groups, OR (c) contribute indi vidual work, not requiring col laboration, to a joint product. teacher (a) explicitly models appropriate language; OR (b) students engage in brief, repeti tive, or drill-like reading, writ ing, or speaking activities; OR (c) students engage in social talk while working. is The teacher (a) includes some aspect of students' everyday in instruction, OR (b) experience connects classroom activities by theme or builds on the current The

teacher and students collaborate on a joint prod uct in a whole-class setting, OR students collaborate on a joint product in pairs or small groups. teacher provides struc tured opportunities for academic language develop ment in sustained reading, writing, or speaking The activities.

The

teacher collaborates activities t ful integrat

dards simu

and literacy Language development Developing language and literacy across the curriculum

and Language literacy development is not observed.

The

teacher designs and enacts activities that gener and expression of content vocabu development lary, AND assists student and language expression instructional ate language development through question ing, rephrasing, or modeling.

The

teache assists in activities t ful integrat standards

Contextualization Making meaning? school connecting students' lives

Contextualization not observed. to

The

unit of instruction, OR (c) includes parents or community members in activities or instruc tion. Challenging Teaching thinking activities complex activity Challenging is not observed. The teacher (a) accommodates students' varied ability levels, OR (b) connects student com ments to content concepts, OR (c) sets and presents standards for student performance, OR (d) provides students with feedback on their performance. The teacher (a) responds to stu dent talk in ways that are com fortable for students, OR (b) uses questioning, listening, or rephrasing to elicit student talk, OR (c) converses with students on a nonacademic topic.

teacher makes incidental between stu connections dents' prior experience/ from home, knowledge and school, or community the new activity/information.

The

teacher integrates the new with what activity/information students already know from home, school, or community.

The

teache assists in c ties that de integration

simultaneou

The teacher designs and enacts activities that connect instructional activities academic content OR advances standing levels. to

The teacher designs and enacts activities that are connected to academic content; assists and uses challenging standards to advance student understanding to more complex levels; AND pro vides students with their performance. feedback on

The teache assists in c that demons

student under to more complex

tion of mult taneously.

Instructional conversation Teaching through conversation

Instructional conversation observed.

is not

The teacher converses with a small group of students on an academic topic AND elicits student talk with questioning, listening, or rephrasing.

The teacher designs and enacts an instructional conversation (IC) with a clear academic goal; lis tens carefully to assess and assist student understanding; AND students on their questions views, judgments, or rationales. All students are included in the IC, AND student talk occurs at higher rates than teacher talk.

The

teache assists in tions that d

integration simultaneou

Note. CREDE = Center for Research on Education, Diversity & Excellence.

November/December models.

2002 [Vol. 96(No. 2)]

83 intercoder agreement 1996; Ross and sound estimates et al., 1991, 1997). of validity

The district served students from a broad range of ethnic, cultural, and language groups residing in both urban and suburban neighborhoods with a wide range of socio economic status. Overall, 17% of the students in the district were immigrants and 33% were designated as having limit

(Ross

& Smith,

Procedures

ed English proficiency. Forty-three percent of the teachers in this sample were European American (n = 18), 33% were = Latino (n = 14), and 24% were African American (n 10). Grade levels observed ranged from first to fifth. In the 13 schools in the study, 44% of the students were Latino, 33% 18% European American, 4% Asian, and African American, less than 1% Native American; 25% were designated as having
or

In the validation
vations. There were

stage, we collected
three procedural

data from live obser

differences between

the developmental
paired observations

(Study
were not

1) and validation
conducted. Instead,

stages. First,
data were

gathered whereas sampling

teachers at the

separately
school

Second, by two trained observers. in Study 1 was exhaustive, including all

site, classroom observations at the 13

limited English
lunch.

proficiency,

and 13% received

free

reduced-price

Measures

Schedule (TROS). The TROS & Anderson, Lindvall, 1990) uses (Waxman, Wang, dichotomous and polytomous items to record information Teacher Roles Observation
on classroom instruction. settings student, at (i.e., the Domains whole teacher's class, desk, assessed small include group, with desk, (a) an or instructional individual

elementary schools in Study 2 were arranged by a site coor dinator. Third, for comparisons with the validation mea sures, SPC, TROS, and COM ratings were made conjointly, and two sets of SPC ratings were generated in each obser vation. For comparisons with the TROS, one set of SPC data consisted of multiple ratings of the observation period following the procedures prescribed for the TROS; for com parisons with the COM, the second set of SPC data consist ed of a single set of scores for the entire observation period. Results Comparisons between the SPC, TROS, and COM revealed a significant positive correlation between SPC total scores and the summed scores on the TROS Purpose of Interaction = .44 and .33, and Nature of Interaction items, respec rs(176) tively, ps < .001. SPC subscales were more strongly related to the Purpose of Interaction items than the Nature of Inter items (see Table 3). For finer grained comparisons with the Purpose of Interaction items, we combined individ action ual TROS variables for these constructed to form composite variables. The items subscales were selected a priori on a theoretical basis. For comparisons with language and literacy

student's

while floating); (b) the purpose and nature of the teacher's interactions; and (c) instructional content. We used only the TROS sections that assessed the purpose and nature of interactions (Purpose of Interaction and Nature of Interac tion sections, respectively) for comparisons with the SPC. The Purpose of Interaction section consists of 25 dichoto mous items (see Appendix) that denote a range of instruc
tional actions such as focusing on the task's content,

process,
work;

or procedures
or correcting

to be followed;
student

checking

student
or behav

praising

performance

ior; and providing corrective feedback on English language use. The Nature of Interaction section consists of 8 dichoto mous items that indicate the use of questioning, listening, explaining, commenting, cueing/prompting, demonstrating, and other behavior by the teacher. A teacher may modeling, be coded as performing multiple actions from each section during a 30-s assessment conducted every 5 min throughout each observation period. Ratings made with the TROS were found to have high interrater agreement in most studies (Waxman, Huang, & Padr?n, 1995). Classroom Observation Measure (COM). The COM (Ross & Smith, 1996) uses a variety of data types to mea
classroom characteristics, teacher and student behav

development, we combined Purpose of Interaction Items 17, 24, and 25; for challenging activities, Items 4-11, 16, 18, 20, 22, and 23; for instructional conversation, Items 13 and 14; and for joint productive activity, Item 15. No items corre
sponded made. to contextualization, Correlations ranged therefore from .43 to no .64 comparison for all was compar

sure

found a significant relationship between COM and SPC total scores, r(176) = .54, p < .001. Moderate positive relationships were found between the COM and all SPC sub isons. We scales except
assess;

iors, and instructional strategies. In this study, we used only the overall observation schedule (Part 4), which assesses
classroom of 20 organization Likert-type scales and teaching (none, strategies some, and consists items. extensive)

contextualization,
ranged from

which
.37 to

the COM
.52.

does not

correlations

Intercorrelations

(Kendall's tau-b) between SPC subscales ranged from -.01 to .54 (see Table 3); subscale correlations with SPC total scores
ranged from .42 to .57.

Teaching
using

strategies

include

sustained

writing

activities;
stu

technologies;

cooperative/collaborative

learning;

dent discussion

and self-assessment; questioning; providing and tutoring (by feedback; using alternative assessments; teacher, peer, or aide). The unit of analysis is the entire observational period. COM ratings were found to have high

STUDY 3
Criterion-related,
which an instrument's

or predictive,
scores are

validity
related

is the degree
criteria

to

to external

84

The Journal of Educational Research

Table 3.?Correlations Between Scores on the Standards and Classroom Measure Observation (COM)

Performance

Continuum

(SPC), Teacher

Roles

Observation

Schedule

(TROS),

Standards Measure

for effective

pedagogy

JPA

LLD

CTX

CA

IC

SPC

total

TROSa Purpose of interaction subscales .49 .24 Purpose r r2 Nature of interaction total .22 .05 of interaction total .13 .02 COMb r total .52 .27 .50 .25 .05 .00 .48 .23 .37 .14 .54 .29 .30 .09 .18 .03 .31 .10 .29 .33 .11 .35 .12 .24 .06 .40 .16 .46 .21 .44 .19 .43 .18 NA NA .57 .32 .64 .41 NA NA

SPC subscale

intercorrelations .37 .05 .34 .24 .42

JPA LLD CTX CA IC

Total

.19 .17 .54 .48

.26 -.01 .42

.22 .57

.42

Note. JPA = joint productive activity; LLD = language and literacy development; CTX = contextualization; CA = challenging activities; IC = instructional conversation. = TROS purpose section has no items related to contextualization. Nonsignificant correlations are italicized. NA dn= 177. bn = 50.

to measure the attribute of interest (Kerlinger, Criterion-related 1986). validity is fundamental to estab the validity of interpretations of SPC data. To esti lishing the relationship mate predictive validity, we examined between teachers' use of the standards during language arts instruction and student achievement. The hypothesis in this study was that higher SPC ratings would be associated with believed
higher dardized spelling, normal tests and curve of equivalent comprehension, achievement. scores on end-of-year stan reading, language,

(31%), 101 were grade (31%).

Measures

in fourth grade (38%), and 83 were

in fifth

vocabulary

rated as either or limited Eng English speaking, fully English proficient, lish proficient. However, because of a highly transient stu dent population, those data were available for only 151 stu dents in this sample, of which 19% were English speaking, All students in the school district were 11% fully English proficient, and 70% limited English pro ficient. To serve the needs of students with varying levels of the school provided three modes of English proficiency, instruction: English only, structured English instruction, and bilingual; 28%, 18%, and 54%, respectively. With 43% of students' English language proficiency (ELP) data miss
ing and a Spearman's rank-order correlation coefficient of

Method Participants Participants were 15 teachers (2 men, 13 women) and 266 students (137 boys, 129 girls) in a public elementary
school in central California. Situated in a rural area, the

.80 (df=
tion, we

150; p < .001) between ELP and mode

used mode of instruction as the most

of instruc
reasonable

low-income served a community of predominantly families: 90% of students in the school were His lunches, 68% panic, 78% received free or reduced-price and 38% were from were limited English proficient, migrant worker families. The school ranked in the second decile statewide on standardized test scores the previous year. Seven teachers in the sample taught third grade, and eight of them taught combined fourth- and fifth-grade class es. Teachers' years of experience ranged from 1 to 26 (M = 6.01, SD = 5.42). Of the students, 82 were in third grade school Hispanic

indicator of students' English

Procedures

proficiency.

SPC data were

the Two made course 45-min of of one

gathered
semester of

through live observations

by a pair of arts were trained language instruction separated

over
were by

observers.

observations each teacher.

Observations

approximately

7 weeks. After observations were completed,

November/December intercoder
total scores

2002 [Vol. 96(No. 2)] was assessed:

85 W on SPC

agreement
was .91.

Kendall's

tively

skewed.

We

used

a square-root

transformation

to make

Results in this study were six indicators The outcome measures estimated by year-end Stanford of student achievement scores: overall subtest Test-9 Achievement (SAT-9) curve equivalent achievement [NCE]), (overall normal
comprehension, language, reading, spelling, and vocabu

the distribution more normal (Tabachnick & Fidell, 1989). To estimate the SPC s contribution to the prediction of student achievement, above that afforded by items such as
grade level, language ability, or teacher experience, we ran

hierarchical regression analyses on each of the dependent variables. In the first step, teacher experience was entered in In the second step, grade and mode were the model. entered. SPC total scores were entered in the final step. Table 5 contains the standardized regression coefficient (?) and t test of its significance for each variable at the step correlation coefficient entered, and the multiple (R), R2, of freedom, and F statistic for each model tested. degrees Overall NCE in the equation, After Step 1, with teacher experience R2 = .00, F[nc(l, 264) = .03,p = .87.After Step 2, with grade and mode added to the equation, R2 = .02, Finc(3, 262) = 1.87, p = .14. After Step 3, with SPC total scores added to the equation, R2 = .06, Finc(4, 261) = 3.97, p = .004. The addition of SPC total scores resulted in a significant increase in R2 of .04 (p = .002) above the variance account ed for by grade and mode of instruction. The effects of our independent variables on students' overall NCE gain scores typified the pattern of effects on the comprehension,
scores. As shown

lary. (We found overall NCE by averaging across all sub tests.) To control for the tendency of scores to regress to the mean on repeated measures of parallel tests (Soar, 1978), we computed estimated gain scores (EGS) for each SAT-9 subtest. We found the EGS by subtracting students' predict
ed scores on from variables the basis observed were of students' on prior each years year's The K-12 test perfor indepen teaching mance, dent scores teachers' subtest. of

experience (grade),
total

students' grade level experience), (mode), and teachers' SPC language proficiency (teacher
derived by summing across subscales and aver

scores,

aging ratings from the two observations. From preliminary analyses, we found that teachers' SPC total scores ranged from 4.00 to 16.50, with a mean of 10.11 and a standard tests (Kruskal-Wallis) deviation of 2.55. Nonparametric revealed differences in SPC total scores in third-grade class es (M = 9.09, SD = 2.42), and combined fourth- and fifth = 10.59, SD = 2.46) did not differ signifi grade classes (M = 1.71 = .43); SPC scores did differ O cantly, x2(2) = 57.14 significantly between modes of instruction, %2(2) < .001). Teachers in SEI classrooms had the highest SPC (p total scores, followed by those in English only and bilingual
classes 3.33, (Ms = 10.91, 10.08, SPC total and 9.81; SDs were = .37, 1.43, and respectively). scores negatively corre

reading,
in Table 5,

spelling,
teacher

and vocabulary
experience was

gain
not a

lated with teachers' years of experience, r(14) = -.66 (p = .02). Intercorrelations for all SPC subscales are presented in Table 4. All subscales were significantly intercorrelated, with coefficients ranging from .15 to .82. SPC total scores are ordinal data and not likely to be nor mally distributed (Harwell & Gatti, 2001). In an inspection of the scatterplot, we found that SPC total scores were posi

reliable predictor of achievement gains on any test. Grade and mode of instruction combined accounted for only a sig nificant portion of variance on the language subtest. Even on the vocabulary subtest, on which the final model was not significant, the t test of the coefficient for SPC total scores was significant (p = .05).
Our next step was to repeat the preceding analyses using

the untransformed SPC total scores. The only result that dif fered significantly was for the vocabulary subtest; although the standardized coefficient did not change, the p value
increased from .05 to .10.

The null and marginal

language and vocabulary

effects of SPC total scores on the

subtests, respectively, were other

Table

4.?Intercorrelations

of Stanford

Achievement

Test-9

Subtests

Overall Variable Comprehension Language Reading Spelling Vocabulary

NCE

Comprehension Language Reading Spelling Vocabulary Overall NCE

.25* .69* .29* .15* .65*

.33* .29* .32* .66*

.28* .63* .82* .25* .63* .69*

Note. NCE = normal curve equivalent. *p < .05.

86 The Journal of Educational Research

Table

5.?Hierarchical

Analysis

of Stanford

Achievement

Test-9

Subtests

Variable t R?
Overall NCE Teacher experience

R2

df

-.010 Grade -.025 -.143 Mode .286 -.006 Grade -.030 Mode -.134 SPCtotal .223

-0.16 -0.36 ? ? -2.32* 3.17* -0.10 -0.44 -2.17* 2.45* -0.75 ? -3.12* -1.14 0.56 -0.95 0.77 ? ? -1.60 2.85* 1.13 ? -0.11 -2.47* 3.24* -0.01 1.81? ? -0.73 1.94*

.01 .15 .24 .01 _____ .14 .20 .05 ? .21 .21 .06 .12 .21 .07

.00 1,264 .02 .06 4,261 .00 1,264 .02 .04 .00 1,264 .04 .04 .00 1,264 .02 .05 .01 1,264 .03 .07 4,261 .00 1,264 .01 .03 4,261

0.03 ?? 1.87 3,262 3.97* 0.01 1.66 3,262 2.77* 4,261 0.57 ?? 3.93* 3,262 3.02* 4,261 0.91 ? ? 1.34 3,262 3.06* 4,261 1.28 ?? 2.47 3,262 4.54* 0.00 ?? 1.25 3,262 1.89

SPC total scores Comprehension Teacher experience

Language Teacher

experience

-.046 -.214 Grade Mode -.070 .050 SPC total -.059 Grade .054 Mode -.099 .258 SPC total .069 Grade -.008 -.151 Mode SPCtotal .290

Reading Teacher

experience

Spelling Teacher

experience

? .17 .26 .00 .12 .17

Vocabulary Teacher

experience

-.001 .126 Grade -.045 Mode .177 SPC total

Instruction language Teacher

in English experience .121 -.249 Grade SPCtotal .208 .114 -.012 Grade .243 SPC total 1.34 -2.53* 2.29* 1.25 -0.12 2.61* 1.80 .02.12 1,120 .26 .32 1.57 .01 1,120 .11 .11 .26

.07 .11

4.15* 2,119 4.62* 3,118

Vocabulary Teacher

experience

.01 .07

0.79 2,119 2.82* 3,118

Instruction language Teacher

in Spanish experience -.069 -.171 Grade SPCtotal -.136 -.019 .285 Grade .045 SPCtotal -0.82 -1.75 -.87 -0.22 2.97* 0.29 .07 .16 .18 .02 .24 .24 .01 1,141 .03 2, .03 .00 1,141 .06 .06 3, 0.68 1.87 140 1.50 3,139 0.05 4.43* 2, 140 2.96* 139

Vocabulary Teacher

experience

Note. NCE normal curve equivalent. SPC = Standards Performance Continuum. Dashes grade and mode, which were entered on the same step in the analysis. *p < .05.

indicate that the data are the same for

than

predicted

and

attenuate

the

contention

that

teachers'

use of the standards fosters English language development learners. It is possible that (a) the the by English-language of Tharp et al. (2000) is not valid or that (b) the SPC ory does not validly predict student achievement. Another pos sibility is that the language of instruction is an intervening variable?the language of instruction differentially affects on tests of English student performance language and achievement. To test this last possibility, we vocabulary removed mode of instruction from the model and collapsed

this three-level categorical variable (English only, struc tured English instruction, and bilingual) into a dichotomous in English versus instruction in Span variable?instruction ish.We then ran hierarchical regression models containing
teacher scores and the transformed grade, experience, on the and vocabulary subtests language SPC total separately

for instruction in English and instruction in Spanish. In the latter category, although some instruction occurred in Eng lish, both teacher and student talk were primarily in Span ish. The results of these analyses (presented in the lower

November/December

2002 [Vol. 96(No. 2)]

87 pretations of SPC ratings. Tests of the SPC's criterion-relat ed validity found that higher use of the standards by teach ers during language arts instruction in a school serving pre learners was poor Latino English-language dominantly on with greater student achievement associated reliably tests. After accounting for the effects year-end standardized of teachers' years of experience and students' grade level and English proficiency, higher SPC total scores were asso ciated with greater achievement gains than would be pre dicted by students' SAT-9 scores from the previous year. We found this relationship on all SAT-9 subtests except lan guage, and the relationship was only marginally significant
for vocabulary. Further analyses revealed that teachers' use

portion of Table 5) show that, for students instructed in English, SPC total scores reliably predicted their achieve ment gains in both language and vocabulary. The proportion of variance accounted for by SPC total scores, above that
accounted for by teacher experience and grade, was signif

icant in both tests?an additional 4% for language and 5% for vocabulary. Conversely, for students taught in Spanish,
teachers' mance lary on use of the standards was English unrelated language to their and perfor vocabu tests that measure

achievement.

Discussion study presented the development the Standards Performance Continuum for assessing teacher performance of et al., 2000). Effective Pedagogy (Tharp
research was to construct a quantitative

This

and validation of (SPC), a measure the Standards for The goal of this
that pro

of the standards when English was the language of instruc tion reliably predicted English language and vocabulary
achievement, where whereas, was as might be instruction in Spanish, teachers' predicted, use in of classes the stan

instrument

dards was
and guage and

unrelated
With

to student gains
no outcome achievement for

in English
of this sample,

language
lan rela the

vided reliable and valid interpretations of teachers' enact ments of the standards. The contribution of the SPC to
researchers, administrators, and those engaged in profes

vocabulary.

measures

Spanish

vocabulary

sional development is the specificity and validity of the information it provides about teachers' performance of the standards that Tharp and colleagues proposed as the teach ing strategies most effective for at-risk students.
Developing a new measure requires the construction of

the instrument and a determination of its accuracy. The next step is to determine the validity of interpretations made from data gathered by the measure. The accuracy of the SPC was estimated from the degree of agreement between different observers who used the SPC to make parallel rat ings of a set of teachers. We found indications of the valid ity of interpretations of SPC scores by (a) comparing con currently made ratings of the same teachers derived from different measures of similar constructs and (b) assessing the reliability of predictions made from SPC scores on a
related criterion.

tionship between teachers' use of the standards and stu dents' Spanish vocabulary and language development remains a question for further study. These findings should not be construed as providing support for English-only instruction for all English language learners. We developed the SPC to validly measure a teacher's per formance of the Standards for Effective Pedagogy. We used a pre- and posttest design to assess the relationship between
teachers' use of the standards and student achievement.

Although the findings of the SPC's predictive validity sug gest a link between SPC scores, the pedagogy proposed by (2000), and instructional effec Tharp and his colleagues evidence of the validity of direct interpretations tiveness, from SPC scores to effective instruction will require con trolled studies.
As with most criterion-based measures, adequate training

In Study 1, the SPC exhibited high agreement between raters for live and videotaped observations of teachers across
a range of grade levels and subject areas. In our assessments

of concurrent relationships
teachers who

validity, we found theoretically meaningful between the SPC and comparison measures;
used more of the pedagogy strategies mea

of observers is essential (Frick & Semmel, 1978). Training includes (10 to 20 hr, depending on trainees' expertise) instruction on the bases and definitions of the standards, and videotapes of classroom instruction jointly rated by trainer and trainees. Training requires the SPC manual (available through the Center for Research on Education, Diversity assessed
occur prior

sured by the TROS and COM were also more likely to enact the standards at higher levels. Our comparisons of these
measures, nization of however, may have been constrained Students by must the orga be orga the classrooms observed.

& Excellence). Interrater reliability should be as part of the training which, of course, should
to data collection. Periodic retraining is recom

mended
ration of

to maintain
observer

the accuracy and to minimize

skill when observations are

deterio
performed

nized in small groups for teachers to reach the higher levels of joint productive activity and instructional conversation; small groups were present in only 1.03% of the classrooms
observed. strain variance scores on Classroom on organization and COM, was does and, not similarly con greater these the TROS measures as a result, Even with

intermittently
We lenges research fessional created of on

over an extended period of time.

the SPC to serve many roles an as for a to meet the chal for for pro education?as effectiveness, and as a tool instrument guide

improving instructional

development,

education

reform.

these

possible.

the comparisons indicate that the SPC pro considerations, vides valid information on teacher performance. Study 3 provided further support for the validity of inter

As a research instrument, the SPC is currently being used as a predictor variable in amultisite research and development project conducted by the Center for Research on Education, to extend the research on the effica Diversity & Excellence

88 The Journal of Educational Research cy of the standards in culturally and linguistically diverse educational settings. The SPC is also being used as an out
come variable to assess the effectiveness of the profession

al development activities at the core of this project. The SPC also can serve a useful role in the professional (1988) have development of teachers. Tharp and Gallimore that education reform must focus on long maintained
improving teaching. However, the classroom environment

only for linguistically and culturally diverse students, or are they equally effective for all students? These questions scratch the surface of the rich body of research merely (2000) in the goal to engendered by Tharp and colleagues transform pedagogy and improve learning for all students. NOTE
This work was supported under the Education Research and Develop ment Program, PR/Award No. R306A60001, on the Center for Research as administered & Excellence Education, (CREDE), Diversity by the Office of Educational Research and Improvement Insti (OERI), National tute on the Education of At-Risk Students (NIEARS), U.S. Department of Education (U.S. DOE). The contents, findings and opinions expressed here are those of the authors and do not necessarily or represent the positions or the U.S. DOE. policies of OERI, NIEARS,

necessary
from what

to reform education
most teachers experienced

is fundamentally
as students.

different
For teach

ers to be successful in taking on new roles and changing decades of reform efforts, practices that have withstood they will need a clear articulation of the role of the teacher and clear standards for teaching performance. The SPC pro vides such standards, as well as developmental guidelines to Putnam for teachers learning to enact them. According and Borko (2000), teachers' interactions with colleagues are primary
learned. tive and tools concepts.

REFERENCES
Ashworth, Wiley. D. P. (1979). Social interaction and consciousness. New York:

determinants
Discourse as language, For

of what
of to be

is learned and how

provides such ideas, in taking

it is
cogni

among ways

educators

interacting, successful

theories, on new

teachers

roles and changing nities to participate

cusses new teacher

their practices, they also need opportu "in a professional community that dis
materials and strategies and that sup

ports the risk taking and struggle entailed in transforming practice" (McLaughlin & Talbert, 1993, p. 15, as cited in Putnam & Borko). It is hoped that the SPC can serve as a useful tool in this discourse, and guide and assess changes
in practice.

The SPC also can play a key role in school reform. Sub
stantive, vision teachers, enduring and education effort developers, reform between requires the a common collaborative researchers, and com

professional

administrators,

munities involved. The SPC has already proved useful to the needs of each of these groups in a school/community
co-constructed education reform project focusing on the

in a Native American Standards for Effective Pedagogy school district. Researchers assisting the district used the to document the progress of the reform efforts and to SPC determine the relationship between the use of the strategies and student outcome measures. Teachers used the SPC to facilitate self-reflection and peer dialogue around pedagogy and unit planning. Professional developers used the SPC to guide and assess their efforts, to facilitate individual and
group vices consultations, and summer and as a framework Administrators for planning inser institutes. incorporated

Boston: Theory and practice. Baddeley, A. D. (1990). Human memory: Allyn & Bacon. D. Self-evaluative and self-efficacy Bandura, A., & Cervone, (1983). mechanisms the motivational effects of goal systems. Journal governing and Social Psychology, 45, 1017-1028. of Personality self-effica Bandura, A., & Schunk, D. H. (1981). Cultivating competence, self-motivation. Journal of cy, and intrinsic interest through proximal and Social Psychology, 41, 586-598. Personality executive Brown, A. L. (1987). Metacognition, control, self-regulation, and other more mysterious mechanisms. In F. E. Weinert & R. H. Kluew and understanding motivation, (Eds.), Metacognition, (pp. 65-116). Hillsdale, NJ: Erlbaum. J. T, & Petty, R. E. (1982). The need for cognition. Journal of Cacioppo, and Social Psychology, 42, 116-131. Personality Cohen, E. G., & Lotan, R. A. (1997). Working for equity in heterogeneous classrooms. New York: Teachers College Press. success for ESL students: academic Collier, V. P. (1995). Promoting second language acquisition for school. Elizabeth, NJ: Understanding to Speakers of English of Other Languages New Jersey Teachers Bilingual Educators. Collins, A. M., & Loftus, E. F. (1975). A spreading activation theory of semantic processing. Review, 82, 407-428. Psychological Craik, F. I., & Lockhart, R. S. (1972). Levels of processing: A framework for memory research. Journal of Verbal Learning and Verbal Behavior, 11, 671-684. and the retention of Craik, F. I., & Tulving, E. (1975). Depth of processing in episodic memory. Journal eral, 104, 268-294. D. F. (1988). Cooperative Dansereau, stein, E. T Goetz, & P. A. Alexander words of Experimental Psychology: Gen

the SPC with a teacher professional development portfolio evaluation and the district curriculum. The immediate ben efits of all participants in the reform project who used the SPC to meet their varying needs were clear and immedi ate?the SPC provided specific information about teaching practice that was readily available and easily interpretable. Researchers who are using the SPC should address the use of these standards important questions: Does
improve student learning, and, if so, are some standards

In C. E. Wein learning strategies. (Eds.), Learning and study strate instruction, and evaluation gies: Issues in assessment, (pp. 103-120). Press. Orlando, FL: Academic N. (1997). Small-group Davidson, learning and teaching in mathematics: A selective review of the research. In E. Dubinski, D. Matthews, & B. E. Reynolds in cooperative (Eds.), Readings learning for undergraduate of America. mathematics. DC: Mathematical Association Washington, and read Estrada, P. (2000, October). Pedagogy, professional development in six linguistically diverse classrooms. ing performance Paper present on Education & Equi ed at the UC ACCORD First Annual Conference ty: Research, Policy, & Practice, San Jose, CA. of about the nature and development J. H. (1987). Speculations Flavell, In F. E. Weinert & R. H. Kluew (Eds.), Metacognition, metacognition. and understanding motivation, (pp. 21-29). Hillsdale, NJ: Erlbaum. to design and evaluate J. R., & Wallen, N. E. (1993). How Fraenkel, research in education. New York: McGraw-Hill. Frick, T, & Semmel, M. I. (1978). Observer agreement and reliabilities of measures. Review classroom observational Research, of Educational 157-184. 48(1), Gallimore, R., & Tharp, R. G. (1992). Teaching In L. Moll and literate discourse. schooling, mind in society: Teaching, (Ed.), Vygotsky and educa

more

important

than others? Are

the standards effective

November/December

2002 [Vol. 96(No. 2)]

89
ond Language Learning. conversa instructional L. A. (1994). Effective Tharp, R. G., & Yamauchi, classrooms tions in Native American (Research Report No. 10). Wash and Sec ington, DC: National Center for Research on Cultural Diversity ond Language Learning. and effects Vedder, P. (1988). Cooperative learning: A study on processes Gronin between primary school children. Westerhaven of cooperation gen, The Netherlands: Groningen. Rijkuniversiteit in society: The development of higher psy L. S. (1978). Mind Vygotsky, S. Scribner, & E. Sou InM. Cole, V. John-Steiner, chological processes. in society: The development berman (Trans.), Mind of higher psycho Press. logical processes. Cambridge, MA: Harvard University H. J. (1994). Educational Wang, M. C, Haertel, G. D., & Walberg, in inner cities. InM. C. Wang & E. W. Gorden resilience (Eds.), Educa and prospects in inner-city America: tional resilience Challenges (pp. Hillsdale, NJ: Erlbaum. the H. C, Huang, S. L, & Padr?n, Y. N. (1995). Investigating in of poverty in inner-city middle level schools. Research pedagogy Middle Level Education, 18(2), 1-22. H. C, Wang, M. C, Lindvall, C. M., & Anderson, K. A. (1990). Waxman, technical manual roles observation Teacher schedule (revised ed.). in Human Devel Center for Research Philadelphia: Temple University, 45-72). Waxman, opment and Education. Webb, N. M., Troper, J. D., & Fall, R. (1995). Constructive activity and small groups. Journal of Educational Psychol learning in collaborative ogy, 87(3), 406-423. C. G., & Chang-Wells, G. L. (1992). Constructing Wells, knowledge as centers of inquiry and literacy. Portsmouth, together: Classrooms NH: Heinemann. J.V (1985). Culture, communication, and cognition: Vygotskian Wertsch, Press. New York: Cambridge University perspectives. In M. C. Wittrock Wittrock, M. C. (1986). Students' thought processes. New (Ed.), Handbook of research on teaching (3rd ed., pp. 297-314). York: Macmillan. of comprehension. Educa Wittrock, M. C. (1990). Generative processes tional Psychologist, 24, 345-376. conversa L. A., & Tharp, R. G. (1995). Culturally Yamauchi, compatible and Education, classrooms. tions in Native American 7, Linguistics 349-367.

and applications New tion: Instructional implications (pp. 175-205). York: Cambridge University Press. and their classroom C. (1991). Instructional conversations Goldenberg, DC: National Cen (Research Report No. 2). Washington, applications ter for Research on Cultural Diversity and Second Language Learning. ordinal data to interval Harwell, M. R., & Gatti, G. G. (2001). Rescaling data in educational research. Review of Educational Research, 77(1), 105-131. Heller, S. (1997). Teaching solving problem through cooperative grouping Part 1 :Group versus individual problem & B. E. Reynolds (Eds.), Read solving. In E. Dubinski, D. Matthews, mathematics. Washing ings in cooperative learning for undergraduate of America. Association ton, DC: Mathematical R. S., Tharp, R. G., & DeGeest, L. (2000). The efficacy of Hilberg, CREDE's standards-based Indian mathematics instruction in American classes. Equity and Excellence in Education, 33(2), 32-39. on Standards for Educational Joint Committee and Psychological Testing of the American Educational Research Association, the American Psy on Measurement in and the National Council chological Association, P., Keith, Education. and psychological (1999). Standards for educational testing. DC: American Education Research Association. Washington, F. N. (1986). Foundation research. Fort Worth, Kerlinger, of behavioral TX: Holt, Rinehart and Winston. Locke, G. J., Shaw, K. N., Saari, L. M., & Latham, G. P. (1981). Goal set 1969-1980. Bulletin, 90, ting and task performance: Psychological 125-152. M. W, & Talbert, J. E. (1993). Contexts that matter for teach the nation's edu learning: Strategic opportunities for meeting on the Context of goals. Stanford, CA: Center for Research Stanford University. School Teaching, Secondary D. J., & Pressley, M. Menke, (1994). Elaborative interrogation: Using to enhance the learning from text. Journal of Reading, "why" questions McLaughlin, ing and cational 37(8), 642-645. Western Samoa. Language in Soci Ochs, E. (1982). Talking to children in ety, 11, 77-104. and Putnam, R. T., & Borko, H. (2000). What do new views of knowledge thinking have to say about research on teacher learning? Educational Researcher, 29(1), 4-15. S. M., & Smith, L. J. (1996). Classroom Measure Observation Ross, observer's manual. for Center Temple University, Philadelphia: Research in Human Development and Education. M. M., Nunery, J., & Rich, L. Ross, S. M., Smith, L. J., Lohr, L., McNelis, TN: (1991). Final report: 1991 classroom observation study. Memphis, of Memphis in Educational Center for Research University Policy. S. M., Troutman, A., Horgan, Ross, P., Maxwell, R., & S., Laitinen, in implementing Lowther, D. (1997). The success of schools eight outcomes. of first-year evaluation restructuring designs: A synthesis School Effectiveness and School Improvement, 8(1), 95-124. Rueda, R. (1999, April). Hart and Ris ley's research on everyday childhood learners. Paper present experiences: Implications for English-language ed at the American Educational Research Association, Montreal, Canada. R. (1994). Assessing Schraw, G., & Sperling-Dennison, metacognitive awareness. Contemporary Educational 19, 460-^-75. Psychology, on cooperative Slavin, R. E. (1996). Research learning and achievement: What we know, what we need to know. Contemporary Educational Psy chology, 21(1), 43-69. in analyzing process-product in Soar, R. S. (1978). Problems relationships studies of teacher effectiveness. Journal of Education, 160(4), 96-116. B. G., & Fidell, L. S. (1989). Using multivariate statistics Tabachnick, (2nd ed.). New York: HarperCollins. S. S., & Yamauchi, L. (2000). Teaching Tharp, R. G., Estrada, P., Dalton, excellence, inclusion, and harmony. transformed: Achieving fairness, Press. Boulder, CO: Westview R. (1988). Rousing minds to life: Teaching, Tharp, R. G., & Gallimore, in social context. New York: Cambridge Uni learning, and schooling versity Press. R. (1991). The instructional conversation: Tharp, R. G., & Gallimore, Teaching and learning in social activity (Research Report No. 2). Santa Cruz, NM: National Center for Research in Cultural Diversity and Sec R., & Anderson,

APPENDIX
TROS Purpose of Interaction Items

1. Focus on the task's content. 2. Focus on the task's product. 3. Focus on the task's process. 4. Communicate the task's procedures. the task's criteria for success. 5. Communicate the difficulty of the task. 6. Determine 7. Restructure specific learning task. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. Redirect Discuss Help Check student's thinking. student's work plans/progress. student complete work on time.

student work. to student signal. Respond Show interest in student work. Show personal regard for student. students to help each other. Encourage students to succeed. Encourage extended student responses. Encourage Encourage self-management. Praise student behavior. Praise student performance. Correct student behavior. Correct student performance. Provide Correct Provide comprehensible input (gestures, language errors. corrective feedback of English. etc.).