Beruflich Dokumente
Kultur Dokumente
net/publication/225382324
CITATIONS READS
335 730
2 authors:
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Carmen Fies on 28 April 2014.
DOI: 10.1007/s10956-006-0360-1
As the frequency with which Classroom Response Systems (CRSs) are used is increasing, it
becomes more and more important to define the affordances and limitations of these tools.
Currently existing literature is largely either anecdotal or focuses on comparing CRS and
non-CRS environments that are unequal in other aspects as well. In addition, the literature
primarily describes situations in which the CRS is used to provide an individual as opposed
to a group response. This article points to the need for a concerted research effort, one that
rigorously explores conditions of use across diverse settings and pedagogies.
KEY WORDS: Response systems; educational technology; formative assessment.
101
1059-0145/06/0300-0101/0
C 2006 Springer Science+Business Media, Inc.
102 Fies and Marshall
learners of the overall distribution of selections in a 2000; Roschelle et al., 2004a,b). That is, although
classroom. students may or may not have had opportunities to
Generally, response items are of a format where talk with peers, they ultimately made individual se-
students are asked to indicate whether a statement lections. Exceptions to this pattern are the research
is true or false, or to select an option (or options) by Mestre, Gerace, Dufresne, and Leonard at the
among given choices predetermined by the instruc- University of Massachusetts, where responses are
tor. This practice is sometimes akin to the stimulus- elicited in three different modes: individual, group,
response pattern of behaviorist theory, where the and group with dissent; trials by Reay et al. (2005) at
showing of a question (stimulus) is followed by feed- Ohio State University, as well as by Boyle et al. at the
back (response) to reinforce behaviors (Naismith University of Strathclyde, GB, which include both
et al., 2004), but may also be used in other modes. group and individual response opportunities. Reay
For example, an instructor may poll the class as a et al. compared one semester of group-based voting
formative assessment, without providing any feed- with a second semester of individual voting and
back to the students, simply to make a decision found that students enjoyed group discussions, but
about the course of further instruction. CRSs may some voiced concerns about “group experts” dom-
also be used to garner information about questions inating the discussion. In addition, although there
for which there is no single correct answer, such were fewer complaints in the individual use semester,
as how well students feel they understand certain the percentage of students voting decreased. A thor-
material. ough exploration of CRS affordances and limitations
Beyond the “traditional” CRS, is the more in group-based learning environments is of in-
inclusively defined “Classroom Aggregation Tech- terest as collaborative models are emphasized in
nology for Activating and Assessing Learning national standards of mathematics and science
and Your Students’ Thinking” (CATAALYST) education and are showing to be beneficial in terms
(Roschelle et al., 2004). This terminology grouping of learning outcomes (e.g., Barron, 2000; Benckert,
not only addresses the “voting machines” described 2001; Brown et al., 1991; Cohen, 1994; Tannen,
above, but also generative models which provide op- 2004).
portunities to capitalize on and foster the strengths
of diversity in thought, learner agency, and partici-
pation within a playful exploration that expands the CLASSROOM RESPONSE
space in which learning occurs (Stroup et al., 2002, SYSTEM LITERATURE
2004, 2006). Next-generation systems such as the
TI-Navigator, or other handheld technologies, such Although aspects of CRS use are also discussed
as PDAs, go beyond the typical question format that in publications other than education research (i.e.,
requires learners to select from given options. Poulis English, 2003; Hafner, 2004; Hallett, 2005; Horowitz,
et al. (1998), on the other hand, used a system that 1988), the focus of this article is solely on reports of
has only one single button to indicate agreement. In CRS use in educational settings. All of the reports
this case, each answer option of a multiple choice included in this review were identified via litera-
question was asked separately. This system may be of ture searches through library catalogues, library
advantage for recently proposed “enhanced multiple digital search tools, as well as through bibliogra-
choice type questions” or “partially correct multiple phies of articles as they were obtained. Literature
choice” approach (Burnstein and Lederman, 2005). search terms included: classroom response system,
Here, the number of answer options is greater electronic response system, audience response
than the traditional four or five, and, in addition system, electronic voting system, polling, keypad,
to one correct option, multiple correct options and wireless, classroom communication systems. In the
partially correct answer options are possible as reading of the literature several themes became
well. apparent, such as the types of questions used, the
Almost all of the studies included in general modalities employed, or the benefits and short-
overview articles were based on purely individual comings identified by the respective authors. To
CRS response collections (Abrahamson, 1998, better understand commonalities and differences,
1999; Beatty, 2004; Burnstein and Lederman, 2003; these findings were then entered into a spread-
Naismith et al., 2004; Penuel et al., 2004; Robertson, sheet and evaluated for overarching themes and
Classroom Response Systems: A Review of the Literature 103
findings. The sections below are structured by these address aspects that were not discussed before;
categories. where articles were not reviewed before, we also
Amongst general writings about CRS use identify how they fit with the prior findings.
are reviews of CRS literature, general recom- We categorized a total of 24 publications
mendations of use, and technical descriptions and based on their primary focus: either (a) peda-
comparisons (Abrahamson, 1998, 1999; Beatty, gogical theory; or (b) implementation (Table I
2004; Burnstein and Lederman, 2003; Naismith below).
et al., 2004; Penuel et al., 2004; Robertson, 2000;
Roschelle et al., 2004a,b). As part of providing
an inventory of what is known of CRS use, these Pedagogical Theory
publications report benefits and shortcomings of
CRS supported instruction in terms of instructor This set includes CRS literature that ranges
and learner attitudes, instructor sensitivity to learner from considerations of a specific pedagogical con-
understanding, and learning outcomes. However, struct, such as the timing of feedback (Motani
in almost all instances reported, comparisons are and Garg, 2002), to literature that reaches be-
made between traditional lecture conditions and yond considerations of affordances and lim-
CRS supported classrooms that, in addition to the itations in traditional CRS to possibilities of
technology component, also adopted interactive next-generation systems (Penuel et al., 2004;
methodologies. Roschelle and Pea, 2002; Stroup et al., 2002, 2004,
A recent analysis of 26 classroom network 2005).
studies by SRI and Better Education researchers The intent of Mazur’s group at Harvard and
Jeremy Roschelle, Bill Penuel, and Louis Abraham- Dufresne’s in Massachusetts, is to make classrooms
son indicates that there is good agreement in terms more interactive and students more active partic-
of benefits of use (Roschelle et al., 2004a,b). Specif- ipants in their own learning processes (Dufresne
ically, they found indications of “greater student et al., 1996; Mazur, 1997). Both research groups
engagement (16 studies), increased student under- take advantage of student interactions, a model that
standing of complex subject matter (11), increased proves to be effective regardless of the particular
student interest and enjoyment (7), heightened sequencing or the degree of anonymity afforded.
discussion and interactivity (6), increased student In Mazur’s Peer Instruction (PI), the initial stage
awareness of individual levels of comprehension (5), consists of individual responses which are followed
and increased teacher insight into student difficulties by small group discussion and then a second re-
(4).” However, their review also indicates that the sponse; Dufresne’s model first has students discuss in
studies lacked in rigor, making it impossible to small groups and then vote either individually or in
draw strong conclusions about the technology’s groups.
effectiveness. Mazur’s PI, although not specific to CRS class-
We, too, report on commonalities in the existing rooms, benefits from the technology: Since CRSs
CRS literature and some of the studies included provide reliable aggregation of inputs and im-
here were part of the 2004 meta-analysis as well. mediate, easily interpretable output in the form
However, the basis of comparison is a different one of a histogram, this technology greatly facili-
to complement rather than to duplicate the findings tates the use of student feedback. Yet, the au-
above. Where articles were reviewed before, we thor points out “that the success of Peer In-
struction is independent of feedback method and these low cost and highly portable devices of-
therefore independent of financial or technological fer ideal support for classroom methodologies that
resources” (p. 18). Subsequent work on the effec- follow theories of distributed intelligence or dis-
tiveness of PI includes instructor surveys admin- tributed cognition (p. 151). Although not an im-
istered by the Mazur group (Crouch and Mazur, plementation study, the 2004 CATAALYST work-
2001; Fagen et al., 2002), the results of which in- shop report by Penuel et al. also fits this section
dicate that student engagement and learning out- as they note that existing learning theories and
comes are both improved in connection with PI research models may not be sufficient to explore
pedagogy. and optimize the use of these technologies (pp. iii,
Dufresne et al. (1996) developed a CRS sup- 10–11).
ported instructional sequence that is based on Kolb’s
Experiential Learning Cycle and also benefits greatly
from instantaneous feedback aggregation. Much like Implementation Studies
Mazur’s PI, the researchers rely on student–student
interactions in small group settings and alternate This section includes only studies that report on
these with whole-class evaluations; however, in con- classroom instantiations of CRS use. First a brief
trast to PI, here students do not first make individual summary of where and how data are collected: All
selections. The conceptual discussion follows a spi- but reports by two European research groups (Boyle,
raling pathway in that the discussion of one question 1999; Boyle et al., 2001; Nicol and Boyle, 2003; Poulis
tends to naturally lead to an exploration of a related et al., 1998) and one Asian account (Cue, 1998) are
question. While instructors come to class with certain based on U.S. classrooms. Most of the articles report
preplanned questions, it is ultimately the class as a findings in undergraduate physics courses; excep-
whole that determines which questions get asked and tions are two studies in medical programs (Ganger
discussed. and Jackson, 2003; Paschal, 2002), two engineer-
Stroup et al. are engaged in research on next- ing groups (Boyle, 1999; Boyle et al., 2001; Hall
generation functionalities that open the learning et al., 2002; Nicol and Boyle, 2003), and one high
space, thus broadening the possibilities for discover- school mathematics study (Davis, 2003). The report
ies in a socioconstructivist sense. The contributions by Woods and Chiu (2003) is based on classroom
of participants direct the particulars of a given experiences in both physical science and life science
classroom conversation and thus the overall picture courses. The overall bias towards studies in physical
that emerges; content and social interactions simul- science is only partially due to our teaching in the
taneously act upon each other, making this kind of same discipline; it is largely a reflection of existing
interaction highly generative by design. The research CRS literature.
group also developed a taxonomy of generativity that Only two research groups, namely the Uni-
places different types of CRSs along a continuum versity of Massachusetts at Amherst (Mestre et al.,
(Stroup et al., in press-c). At one extreme is the nom- 1996), and the University at Strathclyde in Great
inally generative “right/wrong” discussion of fairly Britain (Nicol and Boyle, 2003), compared two types
traditional teacher-centered lecture formats, where of interactive instruction, one with and one without
there is a single correct response and a single correct a CRS. Mestre et al. found that, although the use of
pathway to find it; at the other extreme is the highly the CRS is not necessary, the histogram generated
learner-centered emergent group activity, where in- by the system software, in and of itself, promotes
dividual learners become contributors to an overall discussion. In their classrooms, students often ad-
group behavior that then becomes the subject of dressed the answer option chosen by the majority
analysis. first. Nicol and Boyle indicate that students preferred
Bridging the span between CRS generations peer instructional segments over whole-class discus-
is the article by Roschelle and Pea (2002) who sions, and that they thought of the latter as easily
describe not only current conditions of CATA- confusing.
ALYST technologies, but also considerations for Implementation studies were also categorized in
the future. They use the term Wireless Inter- terms of types of data collected in order to better
net Learning Devices (WILD) to address hand- understand methodologies in use (Table II). Note
held technologies. Roschelle and Pea suggest that that two of the articles report on in-class experiences
Classroom Response Systems: A Review of the Literature 105
in general terms and did not indicate any CRS- part of the course grade dependent upon CRS
specific data collection (Boyle, 1999; Burnstein and input.
Lederman, 2001). These two publications are not in- As also stated earlier, most frequently reported
cluded in this table. benefits of CRS include perceptions of class meet-
By far the most frequently used form of ings being more interactive, more engaging, and
data collection is surveying. However, often these more enjoyable. In addition, participants note that
data are combined with other types to under- both instructors and students become more aware
stand the learning context more deeply. Interest- of the condition of the students’ understanding, ul-
ingly, reflections were used in only two of the timately leading to more responsive instruction and
studies. to better understanding. In addition to the studies
Some of the findings in these studies show included in the review of SRI and Better Educa-
good agreement, others are mentioned only once tion researchers (Roschelle et al., 2004a,b), some or
(see Appendix: Table of CRS Implementation Stud- all of these benefits were also reported by Bullock
ies). What follows is first a discussion of fairly uni- et al. (2002), Davis (2003), Fies (2005), Hall et al.
versal findings, then those that are less frequently (2002), Mestre et al. (1996), and Paschal (2002). Sev-
reported. eral articles emphasized responsive teaching prac-
As mentioned earlier, the review of CRS tices based on student feedback (Bullock et al., 2002;
literature points to overwhelmingly individual Davis, 2003; Dufresne et al., 1996; Fies, 2005; Hall
use. Even in cases where there are small group et al., 2002). Hall et al. indicate that students per-
discussions, these are followed up with individual ceive instructors who teach in a responsive manner as
response collections. These responses frequently “caring.”
are counted toward course grades, using a “stag- Additional findings, although less universal, are
gered” point system where students may still receive of equal importance in making pedagogical deci-
partial credit for incorrect selections (Burnstein sions. Several studies included student ratings of
and Lederman, 2001; Crouch and Mazur, 2001; small group discussion components. Nicol and Boyle
Cue, 1998; Paschal, 2002; Woods and Chiu, 2003). (2003) and Fies (2005) found that students preferred
Although the intent is to motivate participation by small group discussions over whole group discus-
offering a reward for trying, this practice may lead sions, arguing that whole class discussions became
to difficulties in interpreting data: Amongst the most confusing more easily. Reay et al. (2005) report that
commonly stated benefits of CRS use are improved students ranked discussions as the most important
attendance and participation, which may to some part of the CRS interaction. However, when given
degree be attributable to the practice of making a choice between individual “voting” and group
106 Fies and Marshall
choice, students preferred to use the CRS in individ- others using combinations of instrumentation, but
ual mode. few integrating data from more than two sources.
The systems’ ability to provide anonymous par- As Penuel et al. (2004) note, it is necessary to
ticipation with private accountability is a critical expand current research models to gain a better
feature of CRSs (Davis, 2003; Fies, 2005; Nicol understanding of the technology’s affordances and
and Boyle, 2003). Students are free to provide in- limitations.
put without the fear of possible public humiliation,
and without having to worry about more vocal stu-
dents dominating the discussion. As Davis (2003)
and Fies (2005) further report, this “empowering” IMPLICATIONS FOR FUTURE RESEARCH
ability of the CRS is limited to the period of use
and does not continue to be effective in later, non- CRSs are well established in educational set-
CRS aided sessions. Both Dufresne et al. (1996) and tings, and their use will likely continue to in-
Nicol and Boyle (2003) hint at the uncertainty of ef- crease. However, rather than a “magic bullet” to
fect size of this technology by stating that improve- educational woes, these systems are merely tools
ments cannot be ascribed to the CRS alone, but which can be used in any number of ways. It is
are also due to changes in the overall pedagogical time to move beyond anecdotes and beyond tra-
approach. ditional classroom pedagogies. It is time to de-
fine what it is that a CRS can add to a learning
environment.
SUMMARY Missing from current CRS research reports
are:
Present in the CRS literature are considerations
1. Tightly controlled comparisons in which the
of the general learning literature, specifically con-
only difference is the use, or lack of use, of a
structs of learning models. There is great agreement
CRS.
that CRSs promote learning when coupled with
2. CRS use in connection with diverse pedagog-
appropriate pedagogical methodologies. While the
ical approaches:
focus is on engaging and interactive practices, the
a. Group-based methodologies that are
specifics of CRS use are mainly studied in connec-
combined with group-based CRS
tion with individual student response modalities
use.
and for conditions where instructors can trace who
b. Varying degrees of anonymity in re-
responded how to a given question. Although there
sponse collection.
are exceptions, such as implementations of PI, the
c. CRS use for purely formative as-
efficacy of CRS group use is not well defined. At
sessment modalities that scaffold
the same time, instructors report that overhead
learning.
needs connected to individual use are cumbersome
3. CRS use in connection with diverse popula-
and may lead to abandonment of the technology.
tions and content areas:
Likewise reports on the efficacy of using CRSs in
a. Same content area, but different popu-
group mode vary; some studies report that students
lations.
preferred the individual mode whereas others
b. Same population, but different content
report value in the complete anonymity of group
areas.
use.
4. Finally, it is of particular note is that the
The literature also indicates that CRS-supported
current literature base contains conflict-
environments lead to greater learning gains than
ing reports of the efficacy of using CRSs
traditional learning environments. However, since
in individual mode versus group mode.
the comparison is between traditional and in-
Given the emphasis on collaborative
teractive engagement models, it is impossible
work in the National Science Educa-
to assess the effectiveness of the technology
tion Standards and elsewhere, the ef-
itself.
fects of group mode use merit further
Reported research relies heavily on surveys and
study.
test results, some based on one single instrument,
APPENDIX
Robertson, L. J. (2000). Twelve tips for using a computerised first century models and modeling. New York: Lawrence
interactive audience response system. Medical Teacher 22: Erlbaum.
237–239. Stroup, W. M., Ares, N. M., and Hurford, A. C. (2005). A dialectic
Roschelle, J., Abrahamson, L. A., and Penuel, W. R. (2004a, April analysis of generativity: Issues of network supported design in
16). DRAFT Integrating Classroom Network Technology and mathematics and science. Journal of Mathematical Thinking
Learning Theory to Improve Classroom Science Learning: A and Learning 7(3): 181–206.
Literature Synthesis. Paper presented at the Annual Meeting Stroup, W. M., Ares, N., and Hurford, A. (2004, October). A
of the American Educational Research Association, San taxonomy of generative activity design supported by next-
Diego, CA. generation classroom networks. Proceedings of the 28th
Roschelle, J., and Pea, R. (2002). A walk on the WILD side: Annual Conference of Psychology in Mathematics Education
How wireless handhelds may change computer-supported North America.
collaborative learning. International Journal of Cognition and Stroup, W. M., Kaput, J. J., Ares, N. M., Wilensky, U., Hegedus,
Technology 1: 145–168. S., Roschelle, J., et al. (2002). The Nature and Future of
Roschelle, J., Penuel, W. R., and Abrahamson, A. L. (2004b). Classroom Connectivity: The Dialectics of Mathematics in the
Classroom Response and Communication Systems: Research Social Space. Paper presented at the 24th Annual Meeting of
Review and Theory. Paper presented at the Annual Meeting the North American Chapter of the International Group for
of the American Educational Research Association, San the Psychology of Mathematics Education, Athens, GA.
Diego, CA. Tannen, D. (2004). Conversational styles. In Bloom, L. Z., White,
Roschelle, J., Penuel, W. R., and Abrahamson, A. L. (2004c). The E. M., and Borrowman, S. (Eds.), Inquiry: Questioning,
networked classroom. Educational Leadership 61: 50–54. Reading, Writing, Prentice-Hall, Upper Saddle River, NJ, pp.
Stroup, W. M., Ares, N., and Lesh, R. A. (2006). Diversity by 203–208.
design: The what, why and how of generativity in next- Woods, H. A., and Chiu, C. (2003). Wireless Response Tech-
generation classroom networks. In R. A. Lesh, E. Hamilton, nology in College Classrooms. Retrieved August 18,
and J. J. kaput (Eds.), Foundations of the Future: Twenty- 2003.