This article was downloaded by: University of Bath

On: 23 Mar 2017

Access details: subscription number 13109
Publisher:Routledge
Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: 5 Howick Place, London SW1P 1WG, UK

Handbook of Self-Regulation of Learning and Performance

Barry J. Zimmerman, Dale H. Schunk

Use of Hypermedia to Assess and Convey Self-Regulated Learning

Publication details
https://www.routledgehandbooks.com/doi/10.4324/9780203839010.ch7
Roger Azevedo, Amy Johnson, Amber Chauncey, Arthur Graesser
Published online on: 08 Mar 2011

How to cite :- Roger Azevedo, Amy Johnson, Amber Chauncey, Arthur Graesser. 08 Mar
2011 ,Use of Hypermedia to Assess and Convey Self-Regulated Learning from: Handbook of Self-
Regulation of Learning and Performance Routledge.
Accessed on: 23 Mar 2017
https://www.routledgehandbooks.com/doi/10.4324/9780203839010.ch7

PLEASE SCROLL DOWN FOR DOCUMENT

Full terms and conditions of use: https://www.routledgehandbooks.com/legal-notices/terms.

This Document PDF may be used for research, teaching and private study purposes. Any substantial or systematic reproductions,
re-distribution, re-selling, loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden.

The publisher does not give any warranty express or implied or make any representation that the contents will be complete or
accurate or up to date. The publisher shall not be liable for an loss, actions, claims, proceedings, demand or costs or damages
whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material.
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7

7
Use of Hypermedia to Assess and
Convey Self-Regulated Learning

Roger Azevedo
McGill University, Montreal, Canada

Amy Johnson, Amber Chauncey, Arthur Graesser

University of Memphis

The ubiquity of hypermedia learning environments poses several challenges for researchers
and educators. Learning with these non-linear, mutli-representational, open-ended learning
environments typically involves the use of numerous metacognitive and self-regulatory processes
such as planning, knowledge activation, metacognitive monitoring and regulation, and reflection
(Azevedo, 2008, 2009; Greene & Azevedo, 2009; Moos & Azevedo, 2008; Schraw, 2007;
Veenman, 2007; Winne & Hadwin, 2008; Zimmerman, 2008). Unfortunately, learners do not
always regulate these processes when learning with hypermedia. As such, learners failure to
regulate limits the instructional effectiveness of these environments to enhance learning about
complex and challenging topics. In this chapter we will describe how we have used hypermedia
environments to assess learners self-regulated learning (SRL). More specifically, we have used
empirical data and associated theoretical SRL frameworks to design MetaTutor. MetaTutor is
a hypermedia-based multi-agent learning environment that was designed to model, trace, and
foster learners SRL in the context of learning about several human biological systems. The
goal was to build an intelligent learning environment capable of fostering learners SRL during
learning in laboratory and classroom contexts.
This chapter is structured into the following sections: (a) an overview of SRL with
hypermedia; (b) a theoretical overview that treats SRL as an event, with dynamically unfolding
processes that are deployed during learning; (c) a brief review of research on SRL as an event with
hypermedia; (d) a description of the MetaTutor environment and current findings, including the
various embedded instructional elements designed to assess and support various SRL processes;
(e) implications for educational practice; and (f) future directions.

SELF-REGULATED LEARNING WITH HYPERMEDIA

The complex nature of SRL can be illustrated by providing an example of learning with hypermedia.
Imagine that a student is asked to learn about the human digestive system with an open-ended
102
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 103
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
hypermedia learning environment. The environment contains several hundred static diagrams,
hundreds of paragraphs containing thousands of words with corresponding static diagrams and
video clips (all of which may be organized in some fashion, similar to sections and sub-sections
of book chapters), and hundreds of hyperlinks allowing the students to navigate freely throughout
the environment. One could imagine that this self-regulated learner would analyze the learning
situation, set meaningful sub-goals, and determine which strategies to use based on the task
conditions. The student may also generate motivational beliefs based on prior experience with the
topic and learning environment, success with similar tasks, contextual constraints (e.g., provision
of scaffolding and feedback by the hypermedia environment or human or artificial agent), and
contextual demands (e.g., a time limit for completion of the task). During the course of learning,
the student may assess whether particular strategies are effective in meeting his learning sub-
goals, evaluate his emerging understanding of the topic, and make the necessary adjustments
regarding his knowledge, behavior, effort, and other aspects of the learning context. The hope
is that there will be adaptive adjustments, based on continuous metacognitive monitoring and
control related to the standards for the particular learning task, and that these adjustments will
facilitate decisions regarding when, how, and what to regulate (Pintrich, 2000; Schunk, 2001;
Winne, 2005; Winne & Hadwin, 2008; Zimmerman, 2008). Sometime after the learning session,
he may make several cognitive, motivational, and behavioral attributions that affect subsequent
learning (Pintrich, 2000; Schunk, 2001).
This scenario represents an idealistic approach to self-regulating ones learning with
hypermedia. Unfortunately, the typical learner does not engage in these complex adaptive
cognitive and metacognitive processes during hypermedia learning (see Azevedo & Witherspoon,
2009). As such, the instructional potential of these environments is severely limited. One possible
solution is to use hypermedia as a research tool with which to study SRL and use the data
collected from this research to build hypermedia learning environments designed to support and
foster students SRL.

SELF-REGULATED LEARNING: THEORETICAL FRAMEWORK

Self-regulated learning (SRL) is an important learning phenomena that has led to a recent
rise in psychological and educational research (Azevedo, 2008; Azevedo, Johnson, Chauncey,
& Burkett, 2010; Boekaerts, Pintrich, & Zeidner, 2000; Hacker, Dunlosky, & Graesser, 2009;
Paris & Paris, 2001; Schunk & Zimmerman, 2008; Winne, 2005; Zimmerman, 2008). SRL is
an active, constructive process whereby learners set learning goals and then attempt to monitor,
regulate, and control their cognitive and metacognitive processes in the service of those goals.
We acknowledge that SRL also includes other key processes such as motivation, affect, and
emotion; however, we limit our research to the underlying cognitive and metacognitive processes
during learning about complex science. The focus of SRL research over the last three decades
has been on academic learning and achievement, with researchers exploring the means by which
students regulate their cognition, metacognition, motivation, and task engagement (see Schunk &
Zimmerman, 2008; Wigfield, Eccles, Schiefele, Roeser, & Davis-Kean, 2006).
While most theories, models, and frameworks of SRL tend to agree on some common basic
assumptions (e.g., students are actively constructing knowledge, contextual factors mediate ones
ability to regulate aspects of learning), they also differ in their views of several fundamental
issues regarding the nature of SRL (e.g., aptitude vs. event, role of various contextual agents,
number and types of processes, specificity of the underlying internal and external mechanisms,
explanatory adequacy; see Schunk & Zimmerman, 2001, this volume). These discrepancies pose
 104 ROGER AZEVEDO, AMY JOHNSON, AMBER CHAUNCEY, AND ARTHUR GRAESSER
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
challenges for those interested in using hypermedia as a research tool to understand and measure
regulatory processes during learning (Azevedo, 2009; Azevedo, Moos, Johnson, & Chauncey,
2010; Greene & Azevedo, 2009; Moos & Azevedo, 2008).
There are numerous theories of SRL that differ in sometimes subtle, but often significant
ways (see Boekaerts et al., 2000; Dunlosky & Bjork, 2008; Hacker et al., 2009; Metcalfe,
2009; Metcalfe & Dunlosky, 2008; Schunk & Zimmerman, 2008 for extensive reviews). For
example, Winne and Hadwins (2008) model provides phases, processes, and an emphasis on
metacognitive monitoring and control as the hubs of SRL. This model has been empirically
tested in several complex educational situations (Azevedo, Moos, Greene, Winters, & Cromley,
2008; Hadwin, Winne, & Stockley, 2001; Jamieson-Noel & Winne, 2003) and makes predictions
regarding the linear and iterative nature and temporal deployment of SRL processes which fit
perfectly with our current research on hypermedia (Azevedo, 2008, 2009; Azevedo, Johnson
et al., 2010; Witherspoon, Azevedo, & DMello, 2008). Winne and Hadwin posit that learning
occurs in four basic phases: task definition, goal setting and planning, studying tactics, and
adaptations to metacognition. This SRL model also differs from others in that it hypothesizes
that an information processing-influenced set of processes occurs within each phase. Using the
acronym COPES, Winne and Hadwin describe each phase in terms of the interaction of a persons
conditions, operations, products, evaluations, and standards. All of the terms represent kinds of
information that a person uses or generates during learning. It is within this COPES architecture
that the work of each phase is completed. Thus, the model complements other SRL models by
introducing a more complex description of the processes underlying each phase.
While there is no predefined or typical cycle, most learning involves re-cycling through the
cognitive architecture until a clear definition of the task has been created. The next phase produces
learning goals and the best plan to achieve them, which leads to the enactment of strategies to
begin learning. The products of learning (e.g., understanding the function of the mitral valve) are
compared against standards that include the overall accuracy of the product, the learners beliefs
about what needs to be learned, and other factors such as efficacy and time restraints. If the
product does not fit the standard adequately, then further learning operations are initiated, perhaps
with changes to conditions such as setting aside more time for studying. Finally, after the main
learning process, students reflect on the task and may make more long-term alterations to the
strategies that make up SRL, such as the addition or deletion of conditions or operations, as well
as changes to the ways conditions cue operations (Winne, 2001). The output (performance) is the
result of recursive processes that may opportunistically alter conditions, standards, operations,
and products as needed in order to meet the current goal. In sum, this complex model lead to
several assumptions that have and continue to guide the design of MetaTutor.

ASSESSING SELF-REGULATED LEARNING WITH HYPERMEDIA

Researchers from various fields have a growing interest in examining the use of computer-based
learning environments such as hypermedia to enhance learning (Aleven, McLaren, Roll, &
Koedinger, 2010; Azevedo, 2009; Azevedo & Aleven, 2010; Biswas, Leelawong, Schwartz, &
Teachable Agents Group, 2005; Graesser, McNamara, & VanLehn, 2005; Koedinger & Corbett,
2006; Schwartz et al., 2009; Quintana, Zhang, & Krajcik, 2005; White, Frederiksen, & Collins,
2009; Winne & Nesbit, 2009). In this section, we present a synthesis of our research on the use
of hypermedia as a research tool to assess SRL over the last 10 years. We have been interested in
how adaptive and fixed forms of scaffolding impact learners SRL. We broadly define scaffolding
in this context as any tool, strategy, or guide that can assist a learner in his or her regulation.
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 105
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
Our adaptive scaffolding is provided by a human tutor and adjusts (during task performance)
to current task conditions to meet students learning needs. Our fixed scaffolding consists of
static questions that are intended to scaffold students learning by providing them with a set
of optimal sub-goals for their learning session. The research program addresses the following
questions: (a) Do different scaffolding conditions influence students ability to shift to more
sophisticated mental models of complex science topics? (b) Do different scaffolding conditions
lead students to gain significantly more declarative and inferential knowledge of science topics?
(c) How do different scaffolding conditions influence students ability to regulate their learning
of science topics with hypermedia? (d) What is the role of external regulating agents (i.e., human
tutors, pedagogical agents, classroom teachers, and peers) in students SRL of science topics with
hypermedia? (e) Are there developmental differences in college and high school students ability
to self-regulate their learning of science with hypermedia?
In general, our results indicate that adolescents and college students do not self-regulate
while learning in a hypermedia environment and this results in low conceptual shifts in their
understanding of complex science topics, as measured by essays that require an understanding
of an underlying causal mental model (Azevedo, Cromley, Winters, Moos, & Greene, 2005;
Greene & Azevedo, 2009). These learners tend to demonstrate shallow, rather than deep
knowledge gains, as measured by declarative knowledge tests (Azevedo et al., 2008; Greene,
Moos, Azevedo, & Winters, 2008). Adolescents conceptual and declarative knowledge can be
more positively impacted when they are paired with human tutors who facilitate the various SRL
processes during hypermedia learning. We have empirically tested the effectiveness of various
adaptive human scaffolding conditions (i.e., those that deal with learners self-regulatory needs
in real time) which have led to some promising results. Our results demonstrate that learning
challenging science topics with hypermedia can be facilitated if students are provided with
adaptive human scaffolding that addresses both the content of the domain and the processes of
SRL (see Azevedo, 2008, pp. 136137). This type of sophisticated scaffolding is effective in
facilitating learning, as indicated by medium to large effect sizes (range of d = 0.5 to d = 1.7) on
several measures of declarative, procedural, and inferential knowledge involving mental models.
In contrast, providing students with either no scaffolding or fixed scaffolds (i.e., a list of domain-
specific sub-goals) tends to lead to negligible shifts in their mental models and only small gains
in declarative knowledge.
We have also collected thousands of hours of concurrent verbal protocols to track these SRL
processes. Students in different scaffolding conditions deploy qualitatively and quantitatively
different key SRL processes, and there is a clear association among these scaffolding conditions,
mental model shifts, and declarative knowledge gains. To date, we have investigated 38 different
regulatory processes related to planning, monitoring, learning strategies, methods of handling
task difficulties and demands, and interest (based on Azevedo, 2008; Azevedo & Witherspoon,
2009; Azevedo, Johnson et al., 2010). These studies have shown some interesting developmental
differences. Compared to college students, high school students tend to use fewer and less-
sophisticated self-regulatory processes to regulate their learning with hypermedia. Specifically,
they fail to plan throughout the learning session, create sub-goals, monitor aspects of the learning
environment (e.g., content evaluation, CE), or evaluate their own cognitive processes (e.g.,
feeling of knowing, FOK) and emerging understanding (e.g., judgment of learning, JOL). They
also fail to deploy more sophisticated key learning strategies such as coordinating informational
sources, summarizing, and making inferences. Instead, they tend to use less effective learning
strategies such as copying information verbatim from the learning environment to their notes.
In our experiments, learners are informed at the onset of the learning session that they will be
unable to use their notes while taking the posttest. With this in mind, it is a particularly inefficient
 106 ROGER AZEVEDO, AMY JOHNSON, AMBER CHAUNCEY, AND ARTHUR GRAESSER
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
use of time for learners to take copious notes. The data also indicate that key self-regulatory
processes related to planning, metacognitive monitoring, and learning strategies are not used
during the integration process with multiple representations during hypermedia learning. This
leads to declarative knowledge gains but failure to show qualitative mental model shifts related
to understanding these complex topics.
Students in the fixed scaffolding conditions tend to regulate learning by monitoring aspects of
the hypermedia learning environment (rather than their own cognition), and use more ineffective
learning strategies such as copying information. In contrast, adaptive scaffolding by a human
tutor leads students to regulate their learning by activating prior knowledge and creating sub-
goals; monitoring their cognitive system by using FOK and JOL; using effective strategies such
as summarizing, making inferences, drawing, and engaging in knowledge elaboration; and, not
surprisingly, engaging in an inordinate amount of help-seeking from the human tutor (Azevedo
et al., 2005; Greene & Azevedo, 2009). These data have been used to inform the design of
MetaTutor. Specifically, we designed MetaTutor with the assumption that learners need guidance
from an external regulating agent. The external regulating agents in MetaTutor, the animated
pedagogical agents, provide scaffolding to assist learners in the effective use of SRL during
learning with hypermedia.

METATUTOR: A HYPERMEDIA ENVIRONMENT DESIGNED TO ASSESS

AND CONVEY SRL

MetaTutor is a hypermedia learning environment that is designed to detect, model, trace, and
foster students self-regulated learning about human body systems such as the circulatory,
digestive, and nervous systems (Azevedo et al., 2009). Theoretically, it is based on a general
premise of SRL as an event and on cognitive models of SRL (Pintrich, 2000; Schunk, 2005; Winne
& Hadwin, 2008; Zimmerman, 2008). The underlying assumption of MetaTutor is that students
should regulate key cognitive and metacognitive processes in order to learn about complex and
challenging science topics. The design of MetaTutor is based on our extensive research (see
previous section; Azevedo, 2008; Azevedo & Witherspoon, 2009; Azevedo, Johnson, et al., 2010)
which has demonstrated that providing students with adaptive human scaffolding that addresses
both domain knowledge and processes of SRL enhances students learning of complex science
topics with hypermedia. Overall, our research has identified key self-regulatory processes that
are indicative of students learning about these complex science topics (Azevedo & Witherspoon,
2009). More specifically, they include several processes related to planning, metacognitive
monitoring, learning strategies, and methods of handling task difficulties and demands.
Overall, there are several phases to using MetaTutor to train students on SRL processes and
to learn about the various human body systems: (a) a facility that models key SRL processes; (b)
a discrimination task where learners choose between good and poor use of these processes; (c)
a detection task where learners watch video clips of human agents engaging in similar learning
tasks, stop the video whenever they see an SRL process used, and select the relevant process from
a list; and (d) the actual learning environment used to learn about the biological system.
The interface of the actual learning environment includes a window dedicated to declaring
the learning goal, set by either the experimenter or teacher. For example, Your task is to learn all
you can about the circulatory system. Make sure you know about its components, how they work
together, and how they support the healthy functioning of the human body. This goal is associated
with the sub-goals window where the learner can generate several sub-goals for the learning
session. A list of topics and sub-topics is presented on the left side of the screen, while the
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 107
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
actual science content (including the text, static and dynamic representations of information) is
presented in the center of the interface. The main communication dialogue window (between the
learner and the environment) is displayed directly below the content window. The pedagogical
agents are displayed on the top right-hand corner of the interface. In this case, Mary the Monitor
is available to assist the learners through the process of evaluating their understanding of the
content. Below the agent window is a list of SRL processes that the learner can use throughout
the session. Learners can choose to use any SRL process they would like at any point during the
learning task by selecting it from the list. The purpose of having learners select the processes is to
enhance metacognitive awareness of their use of SRL processes during learning and to facilitate
the environments ability to trace, model, and foster learning. In addition to learner-initiated self-
regulation, the agent can prompt learners to engage in planning, monitoring, or strategy use under
appropriate conditions traced by MetaTutor.
One objective of the MetaTutor project has been to examine the effectiveness of pedagogical
agents as external regulatory agents to detect, trace, model, and foster students self-regulatory
processes. MetaTutor is in its infancy, so the algorithms to guide feedback to the student are
developed, but not yet fully tested. The broad scope of SRL appeals to educational researchers
who seek to understand how students become adept and independent in their educational pursuits.
Whether SRL is viewed as a set of skills that can be taught explicitly or as developmental
processes that emerge with experience, pedagogical agents have the potential to provide students
of all ages with information that will help them become strategic, motivated, and independent
learners. However, several theoretical and empirical issues need further research before practical
classroom applications can be put forth (Graesser, Jeon, & Dufty, 2008; Lajoie & Azevedo,
2006). These include: How do students regulate their own learning when using a computer-
based learning environment (CBLE) to learn about complex science topics; which processes
associated with self-regulation and co-regulation do students and pedagogical agents use during
collaborative learning with a CBLE; how can pedagogical agents be designed and used in CBLEs
to support SRL; and how can pedagogical agents be used as external regulating agents that model
specific self-regulatory processes and challenge students to use and develop their own?
The challenge will be to provide feedback on the relevance of the content, on the accuracy
of responses to embedded questions, and on the appropriateness of the strategies used by the
student. Current machine learning methods for detecting students evolving mental models of
the circulatory system are being tested and implemented (Rus et al., in press). In addition, we
are testing specific macro- and micro-adaptive tutoring methods based on detailed system traces
of learners navigational paths through the MetaTutor system (Witherspoon, Azevedo, Cai,
Rus, & Lintean, 2009). In the next section, we present data collected from an initial study using
MetaTutor.

USING METATUTOR TO CONVEY AND ASSESS SRL

In this section we present a synthesis of results that illustrate how MetaTutor can be used to
convey the importance of SRL to students, followed by a presentation of data that demonstrate
its effectiveness in augmenting learners SRL. Thus far, we have collected data on 78 college
and high school students with a non-adaptive version of MetaTutor. The non-adaptive MetaTutor
conveys SRL through several training modules that are based on the socio-cognitive model of SRL
(Schunk, 2001; Zimmerman, 2006) and that use several pedagogical agents to convey the various
phases of training (similar to Biswas et al., 2005; Schwartz et al., 2009; White et al., 2009). Our
ultimate goal is to develop an adaptive MetaTutor that facilitates learners SRL during learning,
 108 ROGER AZEVEDO, AMY JOHNSON, AMBER CHAUNCEY, AND ARTHUR GRAESSER
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
using adaptive prompts for several SRL processes that we have found essential to effective SRL.
These adaptive prompts will be launched under specific conditions of the learning task (e.g.,
learners prior and current domain knowledge, current sub-goal, learning time, time on page).
It is important to highlight a few issues: (a) the design of MetaTutor is based on our previous
findings and models of SRL (presented earlier); (b) MetaTutor was used to train students to
use 13 key SRL processes during learning; (c) data demonstrate the importance of converging
various on-line trace methodologies to examine students use of SRL processes; and (d) potential
challenges still need to be resolved prior to creating an adaptive version of MetaTutor that can be
used by researchers and teachers.
The SRL training provided by the non-adaptive version of MetaTutor involves having
students learn about 13 different SRL processes that have been found to be essential in learning
about science topics. These key processes can be classified into (a) three planning processes
(creating plans and sub-goals, and prior knowledge activation), (b) four monitoring processes
(JOL, FOK, content evaluation, and monitoring progress towards goals), and (c) six learning
strategies (coordinating informational sources, summarizing, re-reading, drawing, taking notes,
and making inferences). The training regimen involves 1.5 hours over 2 days and includes the
following:

pedagogical agents transmit metacognitive (declarative) knowledge of these processes to

the students,
students observe several digitized video clips of human agents modeling these processes
(in the context of using them for another biology topic),
students are administered discrimination tasks in which they have to decide whether the
examples illustrate correct or incorrect uses of these same SRL processes, and
students are provided with two longer videos which they must periodically stop to indicate
the exact SRL processes that the agents are using to learn.

Following the training regimen, they then proceed to learn about the circulatory system with a
non-adaptive version of MetaTutor. During the training and learning interactions, MetaTutor
captures log-file data whereas the experimenters collect other process data via concurrent think-
aloud protocols, video data, and learning outcomes. What follows is a synthesis of the current
data which explored whether MetaTutor, as a multi-agent learning environment, is effective in
enhancing students SRL process and learning outcomes.
The SRL training focused on three separate, yet critical components: modeling,
discrimination, and detection. During the modeling phase, learners spent more time acquiring
metacognitive (declarative) knowledge about planning and metacognitive processes than learning
strategies. Those who underwent SRL training with MetaTutor showed a significant increase in
the amount of metacognitive knowledge gained from pretest to posttest (from 40% to 70%) with
respect to their declarative knowledge associated with each of the 13 SRL processes. As for the
discrimination phase, students spent more time on this phase than on the other two phases of
training. The internalization of these processes apparently takes some time and may be difficult for
students to achieve. The detection phase takes sufficient time for students to accurately detect and
label each of the SRL processes used by human agents in two long videos. A deeper analysis of
these data indicates that students spend greater amount of time on a few more subtle and difficult
SRL processes, namely prior knowledge activation, planning, JOL, FOK, content evaluation, and
summarization. On the discrimination task, they tend to take increased time discriminating sub-
goals, drawing, summarizing, and executing JOLs than any other SRL process.
The extensive log-file data from these same learners provided additional evidence regarding
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 109
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
the SRL processes deployed during learning with MetaTutor. On average, students in both
conditions tended to spend approximately 2 minutes on each page. They also made good
decisions by differentially allocating the amount of time spent on each of the seven sub-topics
related to the circulatory system, with greater time spent on more relevant sub-topics. However,
many learners view only 75% of the available pages and less than 50% of images on any of these
pages (associated with the seven sub-topics on the circulatory system).
The concurrent think-aloud protocols with the non-adaptive version of MetaTutor show
some very important results regarding the use of SRL processes following training. Our current
research using think-alouds indicates that SRL processes should be tracked at both the macro-
level and the micro-level to give the most insight into participants self-regulatory behavior.
Assessing macro-level SRL involves examining participants use of broad classes of SRL, such
as planning, monitoring, and learning strategies, while assessing micro-level SRL involves
examining participants use of individual processes within each class, such as content evaluation
(a monitoring process) or knowledge elaboration (a learning strategy). In this section, we present
various analyses based on the coded concurrent think-aloud data from the same 78 participants.
These data are presented at different levels of granularity (macro and micro) from proportion
scores across each SRL class (e.g., monitoring) and the processes within each class (e.g., FOK and
JOL as part of monitoring) to the temporal dynamics of two participants to illustrate fluctuations
in the deployment of SRL processes. Overall, the data indicate that learning strategies were
deployed most often than any other SRL process, accounting for 72% of all SRL processes
deployed by those in the SRL training group compared to 80% of those in the control group.
These were followed by metacognitive judgments (21% of all SRL processes deployed in the SRL
training group compared to 14% in the control condition) (see Figure 7.1). On average, learners
used approximately two learning strategies every minute and made a metacognitive judgment
approximately once every 4 minutes while using the non-adaptive version of MetaTutor.
Figure 7.2 illustrates the fluctuation of macro-level SRL processes over time, showing average
frequencies of three classes of SRL over a 60-minute learning session. To examine trends and

80

70

60

50

40
Control
30 SRL Training

20

10

0
Planning Monitoring Learning Handling Motivation
Strategies Tasks and
Demands
FIGURE 7.1 Proportion of use of SRL processes by condition during a MetaTutor study
 110 ROGER AZEVEDO, AMY JOHNSON, AMBER CHAUNCEY, AND ARTHUR GRAESSER
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7

40
35
30
25
20
15
10 Control

5 SRL Training

0
0-20 minutes

20-40 minutes

40-60 minutes

0-20 minutes

20-40 minutes

40-60 minutes

0-20 minutes

20-40 minutes

40-60 minutes
Planning Monitoring Learning Strategies

FIGURE 7.2 Mean frequency use of SRL processes by condition during a 60-minute MetaTutor study

changes in SRL over time, the 60-minute sessions were divided into three 20-minute segments,
as indicated by the x-axis. The y-axis indicates average frequency of the three classes of SRL:
planning, monitoring, and learning strategies. On average, students in both conditions deployed
more learning strategies than any other SRL process; a trend observed throughout the learning
session with about one to two learning strategies used every minute. Monitoring processes are the
second most often used SRL processes, although they appear to be used far less frequently than
learning strategies (averaging less than one per minute). Despite the low frequency of monitoring
processes, it is still a step in the right direction to see learning strategies and monitoring occurring
most frequently during the session because these two classes are assumed to be the central hubs
in Winnes model of SRL (see Winne & Hadwin, 2008). Processes related to planning occur
least frequently of all processes (on average, once in 10 minutes). Also, learners in the training
condition used more SRL processes related to planning, monitoring, and learning strategies than
those in the control condition.
A closer examination of the same data by SRL processes within each class is even more
revealing. For example, those in the control condition are learning about the biology content by
previewing the diagrams and text on each hypermedia page, taking notes, re-reading content,
providing correct summaries, and coordinating informational sources. By contrast, those in the
SRL training condition are taking notes, previewing, re-reading, correctly summarizing, making
FOK and JOL judgments, and activating prior knowledge. While there are some differences
between groups, the issue of frequency of use of several of these SRL processes is still
problematic. Quantitatively, these results show the deployment of relatively few SRL processes
during learning with hypermedia while qualitatively they also show the relative underuse of
effective SRL processes.
Another advantage of converging concurrent think-aloud data with time-stamped video data
is that we can calculate the mean time spent on each SRL process during learning. Data show that
the duration for learning strategies ranges from 230 seconds. For example, an instance of taking
notes lasts an average of 20 seconds while drawing lasts an average of 30 seconds. However,
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7

Prior Knowledge Activation

Planning
Recycle Goal in Working Memory
Sub-Goals
Planning Time and Effort Planning
Content Evaluation Positive
Content Evaluation Negative
Evaluate Adequacy of Content Positive
Evaluate Adequacy of Content Negative
Feeling of Knowing Positive
Feeling of Knowing Negative
Judgment of Learning Positive
Judgment of Learning Negative
Monitoring Progress Toward Goals
Monitoring Use of Strategies
Self Question
Self Test
Monitoring Time Monitoring
Coordination of Informational Sources
Draw
Hypothesize
Correct Inference
Incorrect Inference
Knowledge Elaboration
Memorization
Mnemonics
Paraphrase
Preview
Read Notes
Re-Read
Learning Correct Summary
Incorrect Summary
Strategies Take Notes
Help Seeking Behavior
TDD Task Difficulty Statement
Positive Affect
Negative Affect
Motivation Positive Interest
Negative Interest
Knowledge Acquistion

0 5 10 15 20 25 30
Time (in mins)

FIGURE 7.3a Trace data of the first 30 minutes of a low-performer from a MetaTutor study
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7

Prior Knowledge Activation

Planning
Recycle Goal in Working Memory
Sub-Goals
Planning Time and Effort Planning
Content Evaluation Positive
Content Evaluation Negative
Evaluate Adequacy of Content Positive
Evaluate Adequacy of Content Negative
Feeling of Knowing Positive
Feeling of Knowing Negative
Judgment of Learning Positive
Judgment of Learning Negative
Monitoring Progress Toward Goals
Monitoring Use of Strategies
Self Question
Self Test
Monitoring Time Monitoring
Coordination of Informational Sources
Draw
Hypothesize
Correct Inference
Incorrect Inference
Knowledge Elaboration
Memorization
Mnemonics
Paraphrase
Preview
Read Notes
Re-Read
Learning Correct Summary
Incorrect Summary
Strategies Take Notes
Help Seeking Behavior
TDD Task Difficulty Statement
Positive Affect
Negative Affect
Motivation Positive Interest
Negative Interest
Knowledge Acquistion

0 5 10 15 20 25 30
Time (in mins)

FIGURE 7.3b Trace data of the first 30 minutes of a high-performer from a MetaTutor study
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 113
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
previewing content only takes an average of two seconds. As for metacognitive monitoring,
these processes range on average from three to nine seconds. Metacognitive judgments such
as FOK, JOL, and content evaluation tend to last an average of three seconds while monitoring
progress towards goals takes about nine seconds. The duration of SRL processes is important in
determining their impact on learning.
The comparative deployment of SRL processes by low- and high-performing students
is illustrated in Figure 7.3a and 7.3b. The trace data are critical in examining the deployment
and fluctuation of processes during learning at a micro-level. The x-axis represents the time in
minutes whereas the y-axis represents the SRL class and corresponding SRL processes. The
gradated background represents each of the seven topics in MetaTutor. The dots at the bottom
of each figure (labeled knowledge acquisition) show when a student is reading content from
MetaTutor. Figure 7.3a illustrates the first 30 minutes of trace data belonging to a low-performing
student (i.e., low learning gains from pretest to posttest). The figure shows that this student spent
much time acquiring content knowledge and deployed a total of six learning strategies during the
first 30 minutes. In contrast, the high-performing student depicted in Figure 7.3b was much more
variable and deployed many more SRL processes associated with various SRL classes, inspected
more topics in the environment, and spent less time reading content presented in MetaTutor. This
type of data is important in understanding the cyclical nature of SRL (Aleven, McLaren, Roll, &
Koedinger, in press; Azevedo, 2009; Azevedo, Johnson et al., 2010; Biswas et al., 2005: Pieschl,
2009; Winne & Hadwin, 2008; Zimmerman, 2008).
Overall, these data show the complex nature of the SRL processes during learning with
MetaTutor. We have used quantitative and qualitative methods to converge process and product
data to understand the nature of learning outcomes and the deployment of SRL processes. The
data will be used to design an adaptive version of MetaTutor that is capable of providing the
adaptive scaffolding necessary to foster students learning and use of key SRL processes. The
next section will address the challenges in building an adaptive MetaTutor system designed to
detect, trace, model, and foster SRL about complex science topics.

INSTRUCTIONAL IMPLICATIONS AND CHALLENGES FOR USING METATUTOR TO

ASSESS AND CONVEY SRL

In this section we highlight some general and specific instructional implications regarding
the use of MetaTutor (and similar hypermedia-based learning environments) for educational
purposes. Table 7.1 presents a brief overview of some important issues that relate to the role
of the instructional context, learners self-regulatory processes, and the learning system (e.g.,
MetaTutor). In this section we focus on specific challenges in designing an intelligent version of
MetaTutor, based on the models of SRL and our data previously presented.

Major challenges. Our results have implications for the design of the adaptive MetaTu-
tor environment, intended to foster students learning of complex and challenging science top-
ics. Given the effectiveness of adaptive scaffolding conditions in fostering students mental
model shifts, it would make sense to have MetaTutor emulate the regulatory behaviors of the
human tutors. To facilitate students understanding of challenging science topics, the system
would need to dynamically modify its scaffolding methods to foster the students self-regulatory
behavior during learning. Similar approaches have been used in traditional intelligent tutoring
systems (e.g., Koedinger & Corbett, 2006; VanLehn et al., 2007; Woolf, 2009). We recognize
the technological limitations in assessing students learning of conceptually rich, ill-structured
 114 ROGER AZEVEDO, AMY JOHNSON, AMBER CHAUNCEY, AND ARTHUR GRAESSER
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
TABLE 7.1
Instructional Issues Related to the Use of MetaTutor.

Issue Variables of Relevant Issues and Questions

Interest
Learning Goal The provision of challenging learning goal(s) facilitate(s) self-regulated learning.
Learning goals allow learners to keep track of their progress and establish a standard/
criterion for performance.
Who provides the learning goal(s)? Student, teacher, peer, learning system?
Instructional Are additional instructional resources needed to facilitate goal attainment?
Resources Are students capable of engaging in the help-seeking behavior necessary to select
which instructional resources are likely to facilitate goal attainment and enhance
learning?
Learning See MetaTutor below.
Systems
External What is the role of external regulating agents? Do they provide cognitive and meta-
Regulating cognitive strategies? Do they play different roles (e.g., scaffolding, modeling, etc.)?
Agents (Human Is their role to monitor and model students emerging understanding, facilitate,
or Artificial) monitor and regulate their emotional states during learning? Facilitate knowledge
Learning Context

acquisition, provide feedback, scaffold learning, etc.?

Are these human (e.g., teachers, tutors, peers) or artificial agents (e.g., pedagogical
agents)?
Is the support afforded by these agents constant during learning, does it fade over
time, or does it fluctuate during learning? Or is based on the learners behavior?
When do these agents intervene? Is it based on predetermined time intervals, or is it
based on a dynamic model of the learners behavior, learning history, development of
self-regulatory competence, etc.?
How do they demonstrate their interventions (e.g., unidirectional verbal statements,
conversation, gesturing, etc.)?
Assessment What types of assessments are included in the learning context? Are they presented
and Feedback off-line (e.g., paper-and-pencil) by any of the external agents or are they embedded in
Systems the learning environment?
What are the goals of the various assessment methods? Are they to assess knowledge
acquisition, problem solving, SRL processes, competence in SRL skills, etc.?
What types of feedback mechanisms exist and are they part of the learning context or
the learning system?
What is the timing, level, nature, complexity, and delivery system associated with each
type of feedback mechanism?
Prior What are students levels of prior knowledge? What impact will it have on a learners
Learner (Cognitive, Metacognitive, Motivational, and

Knowledge ability to self-regulate?

Knowledge of What self-regulatory strategies are students knowledgeable about? How much practice
Strategies have they had in successfully using them?
Knowledge of How familiar are students with the task they are being asked to complete? Also, are
the Task they familiar with the various aspects of the context and learning system they are
Affective) Processes

being asked to use?

Metacognitive Do students have the necessary declarative, procedural, and conditional knowledge
Knowledge and necessary to regulate their learning? Will the learning system offer opportunities for
Skills learning about these complex processes? Will the environment provide opportunities
for students to practice and receive feedback about these opportunities?
Motivational What are students self-efficacy, interest, task value, self-efficacy, and goal
Factors and orientation(s) and how do they influence their ability to self-regulate?
Orientations*
Affective States Are students able to monitor and regulate their emotional states during learning?
and Emotion
Regulation*
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 115
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7

Issue Variables of Relevant Issues and Questions

Interest
Non-Linear What is the exact structure of the hypermedia learning environment? Is it linear,
Structure hierarchical, etc.?
Multiple What kinds of multiple representations are available in the hypermedia environment?
Representations How many types of representations exist? Are they associated with each other (to
facilitate integration) or are they embedded in some random fashion (potentially
causing extraneous cognitive load)?
Are the representations static (e.g., diagram), dynamic (e.g., animation), or both?
Are students allowed to construct their own representations? If so, are they used to
assess emerging understanding? Or are they just artifacts that may show the evolution
of students understanding, problem solving, learning, etc.? Or is the purpose for
learners to off-load their representations to increase working memory?
Context Space What is the size of the hypermedia context space (e.g., dozens of pages with a few
diagrams or hundreds of pages with thousands of words and hundreds of multiple
Learning System (MetaTutor)

representations)?
Levels of Are there different levels of learner control? Is the system purely learner-controlled
Learner Control and therefore relying on the learners ability to self-regulate? Or does the system
provide adaptive scaffolding and feedback through complex AI algorithms that
provide externally regulated learning (and therefore support self-regulated learning)?
Levels of What is the role of regulating agents? Do they provide cognitive and metacognitive
Scaffolding strategies? Do they play different roles (e.g., scaffolding, modeling, etc.)?
Is their role to monitor or model students emerging understanding, facilitate
knowledge acquisition, provide feedback, scaffold learning, etc.?
Are these pedagogical agents?
Do the levels of scaffolding remain constant during learning, fade over time, or
fluctuate during learning?
When do these agents intervene? How do they demonstrate their interventions (e.g.,
verbal, conversation, dialogue system, mixed-initiative)?
Assessment What types of assessment are included in the learning system?
and Feedback What are the goals of the various assessment methods? Are they to assess knowledge
Systems acquisition, problem solving, SRL processes, competence in SRL skills, etc.?
What types of feedback mechanisms exist and are they part of the learning context or
the learning system?
What is the timing, level, nature, and delivery system associated with each type of
feedback mechanism?

* Note: We recognize the importance of motivation and emotions during complex learning. However, we do not have any data that
pertain specifically to these processes during hypermedia learning.

topics (e.g., Azevedo, 2008; Jacobson, 2008). For example, accurately assessing qualitative shifts
in students mental models remains a challenge. Also, the ability to design a system that can
accurately detect, track, and model students SRL processes will remain a challenge for years
to come (Azevedo, 2008, 2009; Azevedo, Moos, et al., 2010; Jacobson, 2008). Current compu-
tational methods from AI (artificial intelligence) and educational data mining (e.g., Baker, in
press; Leelawong & Biswas, 2008; Rus, Lintean, & Azevedo, 2009) may offer solutions to the
tracking of mental model shifts by using a combination of embedded testing, frequent quizzing
about sections of the content, and probing for comprehension. Similarly, the use of software and
physiological sensors will be key in detecting, tracking, and modeling the deployment of SRL
processes and will provide data for how to provide adaptive scaffolding to foster students SRL
skills and content learning.
As for SRL processes, our data show that learners tend not to use key SRL processes and
that those who do use them do so infrequently and inconsistently. Some learners do use key
 116 ROGER AZEVEDO, AMY JOHNSON, AMBER CHAUNCEY, AND ARTHUR GRAESSER
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
metacognitive processes, but these processes tend to last only a short period of time (up to 9
seconds1). The major challenge for an adaptive MetaTutor is for it to be sensitive enough to
detect the deployment of these processes and to classify the processes (in real time) and then
provide the appropriate feedback mechanism and scaffolding methods vis--vis the students
history with MetaTutor. All this information would then have to be fed to the systems student
and instructional modules to make decisions regarding macro- and micro-level scaffolding
and tailoring feedback messages to the learner. For example, macro-level scaffolding would
involve helping students establish a learning plan, set appropriate sub-goals, and monitor their
progress toward these sub-goals. Micro-level scaffolding would be more complex and would
involve scaffolding students use of individual SRL processes such as assessing their emerging
understanding of the material, making inferences, and integrating text and diagrams while
they learn. Associated concerns include keeping a running model of the deployment of SRL
processes (including the level of granularity, frequency of use, and valence associated with each
process) and evolving understanding of the content and other learning measures. This history
is necessary if we are to make inferences about the quality of students evolving mental models
and the quality of the SRL processes.
Two of the most important aspects of fostering SRL are the capacity to both scaffold effective
SRL and provide timely and appropriate feedback. In this section we focus on two specific and
important modules for an adaptive MetaTutor which provide these critical components.

Scaffolding module. Scaffolding is an important step in facilitating students conceptual

understanding of a topic and the deployment of SRL processes (Azevedo & Hadwin, 2005; Pea,
2004; Puntambekar & Hbscher, 2005). A critical aspect is the agents ability to provide different
types of scaffolding that depend on the students current level of conceptual understanding and
take into account the amount of time left in a learning session. Another critical aspect relates to
learners navigation paths and whether they have skipped relevant pages and diagrams related to
either their current sub-goal or the overall learning goal for the session. In addition, we need to
factor in how much scaffolding a student may have already received and whether it was effective
in facilitating their understanding of the content. The proposed adaptive MetaTutor may start by
providing generic scaffolding that binds specific content to specific SRL processes (e.g., intro to
any section of content is prompted by a scaffold to preview, skim the content, evaluate the content
vis--vis the current sub-goal, and then determine whether to pursue or abandon the content) ver-
sus fine-grained scaffolding that is time-sensitive and fosters qualitative changes in conceptual
understanding. This approach fits with extensive research on human and computerized tutoring
(Azevedo et al., 2008; Chi, Siler, & Jeong, 2004; Graesser et al., 2008). One of the challenges
for the adaptive MetaTutor will be to design graduated scaffolding methods that fluctuate from
external regulation (i.e., a student observes as the agent assumes instructional control and models
a particular strategy or metacognitive process to demonstrate its effectiveness) to fading all sup-
port once the student has demonstrated an understanding of the content. Our current data on SRL
processes show that learners are making FOK, JOL, and CE more often than any other metacog-
nitive judgment. However, they are deploying these processes very infrequently. Thus, agents
could be designed to prompt students to explicitly engage in these key metacognitive processes
more frequently during learning. Another level of scaffolding would involve coupling particular
metacognitive processes with optimal learning strategies. For example, if students articulate that
they do not understand a certain paragraph (i.e., JOL), then a prompt to re-read is ideal. In con-
trast, students who report that they understand a paragraph (i.e., JOL+) should be prompted to
continue reading the subsequent paragraph or inspect the corresponding diagram.
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 117
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
Feedback module. Feedback is a critical component in learning (Koedinger & Corbett,
2006; Shute, 2008; VanLehn et al., 2007). The issues around feedback include the timing and
type of feedback. Timing is important because feedback should be provided soon after one makes
an incorrect inference or incorrectly summarizes text or a diagram. The type of feedback is
related to whether the agents provide simple knowledge of results after a correct answer, infer-
ence, etc., or elaborative feedback, which is difficult to create because it requires knowing about
the students learning history and therefore relies heavily on an accurate student model. For ex-
ample, a key objective of MetaTutor is to determine which and how many learner variables must
be traced for the system to accurately infer the students needs for different types of feedback
(Shute & Zapata-Rivera, 2008). We emphasize that feedback will also be provided for content
understanding and use of SRL processes. The data might show that a student is using ineffective
strategies, in which case the agent might provide feedback by alerting the student to use a better
learning strategy (e.g., summarizing a complex biological pathway instead of copying it verba-
tim). In sum, we have provided theoretically based and empirically guided evidence regarding the
use of hypermedia to assess and convey self-regulated learning.

FUTURE DIRECTIONS

Researchers continue to examine the role of key cognitive and metacognitive processes related
to SRL with hypermedia environments such as MetaTutor. We have demonstated the advantage
of converging data from various on-line trace methodologies (based on concurrent think-aloud
protocols, audio, video, and log-file data) to assess the deployment of self-regulatory processes
during learning. The data provide an in-depth view of the set of processes and how they are used
(or not) to regulate learning.
Despite the current theoretical, methodological, and educational advances, there are
outstanding issues that still need to be addressed by researchers and educators. First, we need
to create a process model that integrates the architecture provided by the cognitive and socio-
cognitive models of SRL with our extensive macro- and micro-level data. This is an important
step in establishing a common (testable) model that makes predictions about the role of various
cognitive and metacognitive processes and their impact on learners ability to regulate their
learning. This model can also be used as the basis for SRL training and for determining the
levels of feedback, scaffolding, and adaptivity that an adaptive MetaTutor system should provide
to learners. Second, we need to use techniques and methodologies from allied fields to capture
fluctuations in motivation (see Pekrun, 2009) and affect (see Koole, 2009) and to incorporate
this evidence in existing models of SRL. Third, we should use a combination of self-report and
on-line methods to assess learners calibration processes and also to use this information to
help learners understand how their self-perceptions may actually differ from their on-line self-
regulatory behavior. Additional work with pedagogical agents is needed to examine the various
roles they can play in modeling, scaffolding, and co-regulating learners cognitive, metacognitive,
motivational, and affective processes (see Graesser et al., 2008). Fourth, we need visualization
tools that allow both researchers and teachers to examine learners SRL profiles, which are ideal
for generating testable hypotheses and for teaching. Lastly, the future of advanced learning
technologies to assess and convey SRL is boundless, but must be tackled in a scientifically based
manner (Greene & Azevedo, 2010).
 118 ROGER AZEVEDO, AMY JOHNSON, AMBER CHAUNCEY, AND ARTHUR GRAESSER
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
ACKNOWLEDGMENTS

The research presented in this chapter has been supported by funding from the National Science
Foundation (Early Career Grant DRL 0133346, DRL 0633918, DRL 0731828, HCC 0841835)
awarded to the first author. The authors would also like to thank M. Lintean, Z. Cai, V. Rus, and
D. McNamara for the design and development of MetaTutor. The authors thank C. Burkett, A.
Fike, and M. Cox for data collection, transcription, and scoring on the MetaTutor project.

NOTES

1. The duration of the process is not necessarily indicative of its importance in making adaptations during
learning or of the quality of the process.

REFERENCES

Aleven, V., McLaren, B., Roll, I., & Koedinger, K. (2010). Automated, unobtrusive, action-by-action
assessment of self-regulation during learning with an intelligent tutoring system. Educational
Psychologist, 45, 224233.
Azevedo, R. (2008). The role of self-regulation in learning about science with hypermedia. In D. Robinson
& G. Schraw (Eds.), Recent innovations in educational technology that facilitate student learning (pp.
127156). Charlotte, NC: Information Age.
Azevedo, R. (2009). Theoretical, methodological, and analytical challenges in the research on metacognition
and self-regulation: A commentary. Metacognition & Learning, 4, 8795.
Azevedo, R., & Aleven, V. (Eds.). (2010). International handbook of metacognition and learning
technologies. Amsterdam, The Netherlands: Springer.
Azevedo, R., Cromley, J. G., Winters, F. I., Moos, D. C., & Greene, J. A. (2005). Adaptive human scaffolding
facilitates adolescents self-regulated learning with hypermedia. Instructional Science, 33, 381412.
Azevedo, R., & Hadwin, A. F. (2005). Scaffolding self-regulated learning and metacognition: Implications
for the design of computer-based scaffolds. Instructional Science [Special Issue on Scaffolding Self-
Regulated Learning and Metacognition: Implications for the Design of Computer-Based Scaffolds],
33, 367379.
Azevedo, R., Johnson, A., Chauncey, A., & Burkett, C. (2010). Self-regulated learning with MetaTutor:
Advancing the science of learning with MetaCognitive tools. In M. Khine & I. Saleh (Eds.), New
science of learning: Computers, cognition, and collaboration in education (pp. 225247). Amsterdam:
Springer.
Azevedo, R., Moos, D. C., Greene, J. A., Winters, F. I., & Cromley, J. G. (2008). Why is externally-
facilitated regulated learning more effective than self-regulated learning with hypermedia? Educational
Technology, Research and Development, 56, 4572.
Azevedo, R., Moos, D., Johnson, A., & Chauncey, A. (2010). Measuring cognitive and metacognitive
regulatory processes used during hypermedia learning: Issues and challenges. Educational Psychologist,
45(4), 210223.
Azevedo, R., & Witherspoon, A. M. (2009). Self-regulated use of hypermedia. In D.J.Hacker, J. Dunlosky,
& A. C.Graesser (Eds.), Handbook of metacognition in education (pp. 319339). New York: Routledge.
Azevedo, R., Witherspoon, A., Graesser, A., McNamara, D., Chauncey, A., Siler, E., Cai, Z., Rus, V., &
Lintean, M. (2009). MetaTutor: Analyzing self-regulated learning in a tutoring system for biology. In
Proceedings of the International Conference on Artificial Intelligence in Education. IOS Press.
Baker, R. S. J. d. (in press). Data mining for education. In B. McGaw, P. Peterson, & E. Baker (Eds.),
International encyclopedia of education (3rd ed.). Oxford, U.K.: Elsevier.
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 119
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
Biswas, G., Leelawong, K., Schwartz, D., & the Teachable Agents Group at Vanderbilt. (2005). Learning
by teaching: A new agent paradigm for educational software. Applied Artificial Intelligence, 19, 363
392.
Boekaerts, M., Pintrich, P., & Zeidner, M. (2000). Handbook of self-regulation. San Diego, CA: Academic
Press.
Chi, M. T. H., Siler, S. A., & Jeong, H. (2004). Can tutors monitor students understanding accurately?
Cognition and Instruction, 22, 363387.
Dunlosky, J., & Bjork, R. (Eds.). (2008). Handbook of metamemory and memory. New York: Taylor &
Francis.
Graesser, A. C., Jeon, M., & Dufty, D. (2008). Agent technologies designed to facilitate interactive
knowledge construction. Discourse Processes, 45, 298322.
Graesser, A. C., McNamara, D., & VanLehn, K. (2005). Scaffolding deep comprehension strategies through
Point&Query, AutoTutor and iSTART. Educational Psychologist, 40, 225234.
Greene, J. A., & Azevedo, R. (2009). A macro-level analysis of SRL processes and their relations to
the acquisition of a sophisticated mental model of a complex system. Contemporary Educational
Psychology, 34(1), 1829.
Greene, J. A. & Azevedo, R., (2010). Introduction: The measurement of learners self-regulated cognitive
and metacognitive processes while using computer-based learning environments. Educational
Psychologist, 45, 203209.
Greene, J. A., Moos, D. C., Azevedo, R., & Winters, F. I. (2008). Exploring differences between gifted
and grade-level students use of self-regulatory learning processes with hypermedia. Computers &
Education, 50, 10691083.
Hacker, D. J., Dunlosky, J., & Graesser, A. C. (Eds.). (2009). Handbook of metacognition in education. New
York: Routledge.
Hadwin, A., Winne, P., & Stockley, D. (2001). Context moderates students self-reports about how they
study. Journal of Educational Psychology, 93, 477487.
Jacobson, M. (2008). A design framework for educational hypermedia systems: Theory, research,
and learning emerging scientific conceptual perspectives. Educational Technology Research &
Development, 56, 528.
Jamieson-Noel, D. L., & Winne, P. H. (2003). Comparing self-reports to traces of studying behavior as
representations of students studying and achievement. German Journal of Educational Psychology,
17, 159171.
Koedinger, K. R., & Corbett, A. T. (2006). Cognitive tutors: Technology bringing learning science to the
classroom. In K. Sawyer (Ed.), The Cambridge handbook of the learning sciences (pp. 6178). New
York; Cambridge University Press.
Koole, S. (2009). The psychology of emotion regulation. Cognition and Emotion, 23, 441.
Lajoie, S. P., & Azevedo, R. (2006). Teaching and learning in technology-rich environments. In P. Alexander
& P. Winne (Eds.), Handbook of educational psychology (2nd ed., pp. 803821). Mahwah, NJ: Erlbaum.
Leelawong, K., & Biswas, G. (2008). Designing learning by teaching agents: The Bettys Brain System.
International Journal of Artificial Intelligence in Education, 18(3), 181208.
Metcalfe, J. (2009). Metacognitive judgments and control of study. Current Directions in Psychological
Science, 18, 159163.
Metcalfe, J., & Dunlosky, J. (2008). Metamemory. In H. Roediger (Ed.), Cognitive psychology of memory
(Vol. 2, pp. 349362). Oxford, U.K.: Elsevier.
Moos, D. C., & Azevedo, R. (2008). Monitoring, planning, and self-efficacy during learning with
hypermedia: The impact of conceptual scaffolds. Computers in Human Behavior, 24(4), 16861706.
Paris, S. G., & Paris, A. H. (2001). Classroom applications of research on self-regulated learning. Educational
Psychologist, 36(2), 89101.
Pea, R. D. (2004). The social and technological dimensions of scaffolding and related theoretical concepts
for learning, education, and human activity. Journal of the Learning Sciences, 13(3), 423451.
Pekrun, R. (2009). Global and local perspectives on human affect: Implications of the control-value theory
of achievement emotions. In M. Wosnitza, S. A. Karabenick, A. Afklides, & P. Nenniger (Eds.),
 120 ROGER AZEVEDO, AMY JOHNSON, AMBER CHAUNCEY, AND ARTHUR GRAESSER
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
Contemporary motivation research: From global to local perspectives (pp. 97115). Toronto, Canada:
Hogrefe.
Pieschl, S. (2009). Metacognitive calibrationAn extended conceptualization and potential applications.
Metacognition and Learning, 4, 331.
Pintrich, P. R. (2000). The role of goal orientation in self-regulated learning. In M. Boekaerts, P.Pintrich,
& M. Zeidner (Eds.), Handbook of self-regulation (pp. 451502). San Diego, CA: Academic Press.
Puntambekar, S., & Hbscher, R. (2005). Tools for scaffolding students in a complex learning environment:
What have we gained and what have we missed? Educational Psychologist, 40, 112.
Quintana, C., Zhang, M., & Krajcik, J. (2005). A framework for supporting metacognitive aspects of online
inquiry through software-based scaffolding. Educational Psychologist, 40, 235244.
Rus, V., Lintean, M., & Azevedo, R. (2009, July). Automatic detection of student mental models during
prior knowledge activation in MetaTutor. Paper presented at the Annual Meeting of the Educational
Data Mining Conference, Cordoba, Spain.
Rus, V., Lintean, M. Cai, Z., Johnson, A, Graesser, A., & Azevedo. R. (in press). Computational aspects of
the intelligent tutoring systems MetaTutor. In P. McCarthy & C. Boonthum (Eds.), Applied natural
language processing and content analysis: Identification, investigation, and resolution. Hershey, PA:
IGI Global.
Schraw, G. (2007). The use of computer-based environments for understanding and improving self-
regulation. Metacognition and Learning, 2(23), 169176.
Schunk, D. (2001). Social cognitive theory of self-regulated learning. In B. Zimmerman & D. Schunk
(Eds.), Self-regulated learning and academic achievement: Theoretical perspectives (pp. 125152).
Mahwah, NJ: Erlbaum.
Schunk, D. (2005). Self-regulated learning: The educational legacy of Paul R. Pintrich. Educational
Psychologist, 40(2), 8594.
Schunk, D., & Zimmerman, B. (2001). Self-regulated learning and academic achievement: Theoretical
perspectives. Mahwah, NJ: Erlbaum.
Schunk, D., & Zimmerman, B. (2008). Motivation and self-regulated learning: Theory, research, and
applications. Mahwah, NJ: Erlbaum.
Schwartz, D. L., Chase, C., Chin, D. B., Oppezzo, M., Kwong, H., Okita, S., et al. (2009). Interactive
metacognition: Monitoring and regulating a teachable agent. In D. J. Hacker, J. Dunlosky, & A.
C.Graesser (Eds.), Handbook of metacognition in education (pp. 340358). New York: Routledge.
Shute, V. (2008). Focus on formative feedback. Review of Educational Research, 7, 153189.
Shute, V. J., & Zapata-Rivera, D. (2008). Adaptive technologies. In J. M. Spector, D. Merrill, J. van
Merrinboer, & M. Driscoll (Eds.), Handbook of research on educational communications and
technology (3rd ed., pp. 277294). New York: Erlbaum.
VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rose, C. P. (2007). When are tutorial
dialogues more effective than reading? Cognitive Science, 31(1), 362.
Veenman, M. (2007). The assessment and instruction of self-regulation in computer-based environments: A
discussion. Metacognition and Learning, 2, 177183.
White, B., Frederiksen, J., & Collins, A. (2009). The interplay of scientific inquiry and metacognition: More
than a marriage of convenience. In D.J.Hacker, J. Dunlosky, & A.C.Graesser (Eds.), Handbook of
metacognition in education (pp. 175205). New York: Routledge.
Wigfield, A., Eccles, J., Schiefele, U., Roeser, R., & Davis-Kean, P. (2006). Development of achievement
motivation. In W. Damon, R. Lerner, & N. Eisenberg (Eds.), Handbook of child psychology (6th ed.,
Vol. 3, pp. 9331002). New York: Wiley.
Winne, P. H. (2001). Self-regulated learning viewed from models of information processing. In B.
Zimmerman & D. Schunk (Eds.), Self-regulated learning and academic achievement: Theoretical
perspectives (pp. 153189). Mahwah, NJ: Erlbaum.
Winne, P. (2005). Key issues on modeling and applying research on self-regulated learning. Applied
Psychology: An International Review, 54(2), 232238.
Winne, P., & Hadwin, A. (2008). The weave of motivation and self-regulated learning. In D.Schunk & B.
Zimmerman (Eds.), Motivation and self-regulated learning: Theory, research, and applications (pp.
297314). Mahwah, NJ: Erlbaum.
 7. HYPERMEDIA TO ASSESS AND CONVEY SELF-REGULATED LEARNING 121
Downloaded By: University of Bath At: 17:27 23 Mar 2017; For: 9780203839010, chapter7, 10.4324/9780203839010.ch7
Winne, P. H., & Nesbit, J. C. (2009). Supporting self-regulated learning with cognitive tools. In D.J.Hacker,
J. Dunlosky, & A. C.Graesser (Eds.), Handbook of metacognition in education (pp. 259277). New
York: Routledge.
Witherspoon, A. M., Azevedo, R., Cai, Z., Rus, V., & Lintean, M. (2009, July). Learners exploratory
behavior within MetaTutor. Paper presented at the Biennial Meeting on Artificial Intelligence and
Education, Brighton, U.K.
Witherspoon, A., Azevedo, R., & DMello, S. (2008). The dynamics of self-regulatory processes within
self- and externally-regulated learning episodes. In B. Woolf, E.Aimeur, R.Nkambou, & S. Lajoie
(Eds.), Proceedings of the International Conference on Intelligent Tutoring Systems: Lecture Notes in
Computer Science (LNCS 5091) (pp. 260269). Berlin: Springer.
Woolf, B. (2009). Building intelligent interactive tutors: Student-centered strategies for revolutionizing
e-learning. Amsterdam: Elsevier.
Zimmerman, B. (2006). Development and adaptation of expertise: The role of self-regulatory processes and
beliefs. In K. Ericsson, N. Charness, P. Feltovich, & R. Hoffman (Eds.), The Cambridge handbook of
expertise and expert performance (pp. 705722). New York: Cambridge University Press.
Zimmerman, B. (2008). Investigating self-regulation and motivation: Historical background, methodological
developments, and future prospects. American Educational Research Journal, 45(1), 166183.