Instructional Science 26: 299316, 1998.

c 1998 Kluwer Academic Publishers. Printed in the Netherlands.

299

Facilitating dynamic pedagogical decision making:

PEPE and GTE
BARBARA WASSON
Department of Information Science, University of Bergen, N-5020, Bergen, Norway

Abstract. The Generic Tutoring Environment (GTE) is centred around a generic instructional
knowledge base that facilitates dynamic pedagogical decision making. PEPE is a computational framework for representing the pedagogical knowledge required to dynamically tailor
instruction to an individual student. Both the GTE model and the PEPE framework form the
basis for an instructional component that orchestrates the coursewares interaction with the
student. In this paper, the knowledge content of the two is examined and the role of each in
instructional planning is described. In general, it is concluded that PEPE and GTE complement
one another and a merging of their ideas would be a worthwhile endeavour.
Key words: dynamic pedagogical decision making, instructional planning, content planning,
delivery planning, instructional models

There has been a growing interest among artificial intelligence and education
(AIEd) researchers in developing computational models of the instructional
process for use in intelligent courseware.1 Such instructional models facilitate
dynamic pedagogical decision making by enabling instructional decisions to
be tailored to an individual student at runtime (i.e. while the student is interacting with the courseware). Both GTE (Van Marcke, 1990, 1992a,b) and PEPE
(Wasson, 1990, 1992) have addressed dynamic pedagogical decision making
by providing an instructional model (GTE) and a computational framework
(PEPE) respectively, for representing the pedagogical knowledge required to
dynamically react to the current instructional situation and tailor instruction
for a particular student. They differ in that they focus on different aspects of
the decision making process. Pedagogical decision making is concerned with
both the content (e.g., what topic to focus on) and the delivery (e.g., how to
present the topic) of instruction (Wasson, 1990; Dijkstra et al., 1992). Given
this distinction it can be said that GTE focuses mainly on delivery issues
while PEPE concentrates on content issues.
In this paper, my reactions to GTE, and its presentation in this issue, will
focus on two aspects: 1) the knowledge content of the instructional model
(not its representational architecture); and 2) the role of GTE in instructional

300
planning. I will not, however, address the courseware development/authoring aspects or the epistemological claims. (See Murray (1998, this issue)
and Spector (1998, this issue) for discussions of these respectively.) Before
addressing these specific issues, a general reaction to GTE and its presentation in this issue is given. Later I will also reflect on how PEPE and GTE
complement one another.
GTE represents a significant contribution to AIEd, and in particular there
are four aspects I would like to highlight. First, the explicit distinction made
between the domain perspective and the content perspective (the same distinction is made in PEPE) is very important. The pedagogical view of a domain
(Wasson, 1990) is different from the problem-solving view of a domain and
it is important to include this in intelligent courseware. Second, even though
GTE did not focus on student modelling, it uses a student model that combines
an overlay of the content marked with one of three values (not-known,
reasonably-well-known, or well-known) with a student attribute model that
includes motivation, learning style, control and prior-knowledge variables.
These attribute values play a significant role in the selection of teaching
methods. Third, GTE has, from the beginning, recognized the importance of
supporting the development of intelligent courseware and has been sensitive
to providing means for courseware authors to tailor the instructional knowledge base to their liking. The provision of an authoring environment is crucial
for moving intelligent courseware from the research lab to educational and
industrial uses. Finally, GTEs instructional model is a major contribution to
the area of delivery planning. This item will be discussed later in the paper.
The presentation of GTE in this issue leaves several questions with the
reader. For example, there are three areas for which important details are
not clear. First, what is the teachers knowledge reflected in the instructional
model and where did you obtain it? Second, how exactly is a course authored?
Careful reading enables one to observe that there is an intricate relationship
between a topic object, domain entities and domain objects, but how does the
author go about developing these? Third, are there any instructional goals in
GTE? If so, how are they planned for? The first and third items are addressed in
Sections 1 and 2, respectively. Section two also contains a short comparative
analysis of PEPE and GTE giving a two-fold contribution: a) showing how
they can be used in complement, and b) by identifying the opportunities for
enhancing either or both separately.
1. Pedagogical knowledge for dynamic pedagogical decision making
Both the GTE and PEPE research projects use a knowledge-based approach
and aim to represent the knowledge required for pedagogical decision making.

301
While Van Marcke (1998, this issue) gives great detail on the implementation aspects of GTE, what is lacking is a description of what pedagogical
knowledge is represented and from where it has been gleaned. Van Marcke
hints at this in the statement, We recognized that the model that underlies
the instructional decision making in intelligent courseware must necessarily
reflect teachers instructional knowledge, i.e. the knowledge that allows
teachers to flexibly adapt their teaching strategy according to the situation.
and in his epistemological claims it is teachers instructional knowledge and
expertise that is modelled. While PEPE makes no claims about epistemological issues and modelling teaching expertise, we claim that PEPE is sufficient
for representing the knowledge required to make aspects of the pedagogical
content decisions made by competent teachers. We make a distinction between
modelling a teachers competence and performance. That is, we distinguish
between how the system and how a teacher makes pedagogical decisions we
want the system to have the competence of a teacher, but not perform like a
teacher. This echoes the competence/performance distinction which was useful in transformational linguistics (Chomsky, 1965). While the significance of
this distinction may be subtle, it is important in contrast to GTEs approach.
It is not PEPEs intention to model human expertise,2 rather it is to provide a
framework for developing an instructional component which represents the
knowledge required, and the mechanisms for reasoning about this knowledge
for tailoring instructional interactions to an individual student. Furthermore,
we rely on more than mere literature on teachers expertise, and draw on ISD
(Instructional Systems Design), instructional psychology, cognitive psychology and artificial intelligence in education (AIEd) research to give us insight
into the pedagogical knowledge base required to make dynamic instructional
decisions. Section 1.1 summarizes where this knowledge can be found in the
literature, and Section 1.2 discusses the types of knowledge represented in
GTE and PEPE.
1.1. Tapping the research literature
Research in the areas of instructional science, instructional psychology, cognitive science, cognitive psychology and AIEd all have something to say, albeit
not always directly, about the pedagogical knowledge required for effective
instruction.3 Developers of intelligent courseware must capture and explicitly
represent aspects of this complex knowledge in order to facilitate dynamic
pedagogical decision making (Wasson, 1990; Van Marcke, 1992; Dijkstra et
al., 1992). Unfortunately this is not an easy task. The process of instruction
is a complex and cognitively demanding task (Peterson et al., 1989) and
the knowledge base on which pedagogical decisions are made is extensive.
Some researchers examine the instructional process (i.e. what teachers do)

302
while others focus on the knowledge that instructors (e.g., teachers, tutors,
coaches, etc.) employ when making pedagogical decisions. Recently, studies
which focus on teachers and learners cognitions during instruction have
been reported. For example, research on teachers cognitions (e.g., Peterson,
1988) focuses on how teachers are effective and how they facilitate
students learning and achievement. Research on teachers knowledge (e.g.,
Elbaz, 1983; Shulman, 1986; Wilson, Shulman & Richert, 1987; Grossman,
Wilson & Shulman, 1989; Leinhardt & Greeno, 1986; Peterson et al., 1989)
suggests that the effective teacher has knowledge of the subject matter, has a
pedagogical view of the subject matter, has knowledge of student errors and
misconceptions, has knowledge of plans and strategies that students use to
solve problems (in general and in a particular domain), has insight into how
a students knowledge has evolved, is aware of learning theories and theories
of knowing, has knowledge of instructional techniques and when to use them,
has knowledge of how to interact with the student and knows when to control
the teacher-student interaction and when to relinquish control.
While such research is certainly applicable for inspiring the design of
an instructional component,4 it should not be the only research tapped. As
the role of the teacher changes with the introduction of technology (e.g., to a
mediator, tutor, modeller, etc.) and with advances in our understanding of how
students learn (e.g., learning theories), what teaching is (theories of teaching),
how teaching and learning interact (theories of learning from instruction),
how instructional material should be designed (instructional design theories),
their knowledge base is apt to change. Such research should not be ignored.
For example, traditional instructional design (ID) theories (such as Gagne &
Briggs, 1974; Landa, 1983; Merrill, 1983, 1987; Reigeluth, 1983; Scandura,
1983) provide worthwhile guidelines for the selection, sequencing and
delivery (or production) of content that can be incorporated into the instructional component (Jones, 1988; Merrill, Li & Jones, 1990). Many of these
traditional ID theories (traditional in that they have their roots in behaviourist
theories of learning) have been revamped to take into account cognitive
theories of learning5 and have been tailored for designing computer-based
learning environments (e.g., Merrill, 1988; Gagne et al., 1991). Recently,
studies have emerged that look at what expert instructional designers do when
creating instructional courseware (e.g., Perez & Neiderman, 1992; Rowland,
1992).
Intelligent tutoring system (ITS), education and cognitive science
researchers have carried out empirical studies in order to collect data about
students misunderstandings in particular domains (e.g., BUGGY, subtraction bugs, Burton & Brown, 1978, 1982; PROUST, catalogues buggy Pascal
plans, Johnson et al., 1983; Sopher et al., 1985; SCENT, LISP language errors

303
and deviations, Huang, 1987 and anological errors in LISP, Escott, 1988).
Some attention (e.g., Woolf & McDonald, 1984a,b; McKendree, Reiser &
Anderson, 1984; Littman, Pinto & Soloway, 1985, 1986; Littman & Soloway,
1986; Douglas, 1988, 1991; Fox, 1991) has been focused on carrying out
empirical studies of human tutors in the hopes of producing a principled
theory of tutoring. Douglas (1988, 1991) investigates how tutors detect and
repair their own tutorial failures and urges ITS designers to design control
strategies that are opportunistic as well as planned in order to facilitate these
repair strategies. Fox (1991) explores the interactional and cognitive processes
involved in correction and tutor assistance in tutoring sessions and concludes
that tutorial interaction is a collaborative production which has implications
for the design of ITSs the ITS must allow for learner participation in the
tutoring session by asking questions, indicating states of confusion, lack of
understanding, asking for correction, etc.
Cognitive psychologists Shuell and Winne each propose cognitive theories
of instruction that emphasize the centrality of the learners activity. Shuell
(1992) identifies twelve learning functions for psychological processes that
must be engaged by either the instructional agent or the student for effective
learning to occur. It is these learning functions that link learning theory
with instructional practices, and he gives suggestions for how these can be
incorporated into instructional computing systems.6 Winne (1992) outlines
a cognitive-oriented theory of instruction that provides the foundation for
the instructional features of an instructional computing system. ITS research
projects have been carried out with an attempt to incorporate a learning
theory (e.g., Anderson, 1983; Anderson, Boyle & Reiser, 1985) and a theory
of diagnostic testing (e.g., Lesgold, 1988, 1989).
Cognitive science researchers investigate how the learner structures and
interprets new knowledge in light of their existing knowledge (e.g., in mathematics, Carpenter & Moser, 1984). Instruction tailored to helping the learner
with this structuring and interpreting, is referred to by Ohlsson (1991) as a
third-order theory of teaching. He eloquently explains that by theory of teaching, he means a theory that tells what teaching is, and not what teachers do.
Ohlsson (1991) posits, and rightly so, that different conceptions of teaching
imply different knowledge requirements. In his interesting review of knowledge requirements for teaching, he outlines three theories of teaching that are
relevant for intelligent courseware and describes the different pedagogical
decisions to be made by the instructor, and thus, the different knowledge
requirements implied by each theory.7 The first-order theory has a traditional view of teaching as the communication of knowledge (subject matter).
Second-order theory has the current view of teaching as the remediation of
incomplete or incorrect mental representations (i.e. learners errors). Third-

304
Table 1. A summary of Ohlssons (1991) three theories of teaching.
Traditional

Current

Future

What topic
(subject matter knowledge)
Expert representation
(knowledge of experts mental representation)
What deficiency
(knowledge of various subject matter perversions & how to do cognitive
diagnosis)
Possible productive learning sequences
(knowledge about how learning happens (i.e.,
learning mechanisms))
Under what circumstances those productive
learning sequences occur
(knowledge about what trigger conditions activate the appropriate learning mechanisms)
How to teach it
(knowledge of
effective ways
of teaching the
subject matter)

How to remedy the

deficiency
(knowledge of how to
remediate the possible deficiencies)

How to bring those circumstances about

(knowledge about how to act in order to trigger
those learning mechanisms)

order theory has a future view of teaching as the facilitation of knowledge

construction by arranging situations where it is likely that productive learning
sequences will occur. Due to space limitations, the instructional decisions that
each theory implies, and the knowledge required to support these decisions
are summarized in Table 1.
Given this plethora of knowledge sources for designing a dynamic pedagogical decision making component, the next section describes which aspects
of this knowledge has been gleaned for use in PEPE and GTE.
1.2. What PEPE and GTE represent
PEPE (Wasson, 1990, 1992) is a competence-based computational framework
for a content planner that views pedagogical decision making as a planning
problem. Working in a one-on-one tutoring paradigm, PEPE incorporates the
following types of pedagogical knowledge: (a) the concepts to be learned,
and the prerequisite and other structural relationships between concepts (e.g.,
part of and isa); (b) the various abilities learners can have in using concepts
such as knowing the definition of a concept, being able to analyze behaviour
or synthesize a solution using a concept; (c) the typical misconceptions of the

305
domain; and (d) the pedagogical rules that represent a pedagogical philosophy
for learning the domain. This knowledge is used in conjunction with a model
of the students current knowledge state to dynamically map out a content
plan that is tailored to an individual student. Relating what PEPE represents
to the discussion above, the concepts to be learned are the pedagogical view
of the subject matter, that is they represent a teaching view of the domain,
and the pedagogical relationships between them (e.g., prerequisite, subconcept, is a specialization or generalization of another concept). The abilities
in using the concepts represent the learning outcomes such as those used in
instructional design literature. For demonstration purposes within the domain
of LISP programming in PEPE, for example, Blooms (1956) taxonomy8
was simplified to three types of abilities: the ability to understand a fact
(corresponds to Blooms knowledge and comprehension levels), the ability
to perform analysis (Blooms analysis), and the ability to perform synthesis
(Blooms synthesis). These were chosen because a student learning to program
should be able to define a concept (i.e. fact), should be able to analyze how
a concept works (i.e. analysis) given a piece of code and finally, should
be able to write a piece of code that requires synthesis (i.e. synthesis) of
the concept. PEPEs representation of misconceptions was based on assignment data collected from students taking a third year programming course at
the University of Saskatchewan and on empirical studies of analogies used
when solving LISP programming problems (Escott, 1988). The pedagogical
rules encoded in PEPE prototype represent a one-on-one tutoring strategy
where sequencing and selection of content was based on general pedagogical
principles and instructional design principles found in the literature. Wasson
(1990) also demonstrated how the strategies of Socratic dialogue and cognitive apprenticeship could be modelled within the PEPE framework.
In the description of the instructional knowledge base given by Van Marcke,
it is difficult to pinpoint exactly what knowledge is represented. Early in the
paper Van Marcke lists examples of what good teachers know, including:
typical examples; analogies that can be used to clarify a concept; teaching
mechanisms that are appropriate for a particular student; and which combination of these teaching mechanisms are good for the student body as a whole.
Aspects of these examples can be found in the instructional knowledge base
(e.g., examples, analogies). When describing the GTE formalism it is said that
there are three perspectives: instructional tasks, that is what an instructor does
(e.g., give-example, clarify-concept); instructional methods, that is ways to
perform the instructional tasks (e.g., clarify-with-analogy); and instructional
objects that are instantiations of the domain knowledge (e.g., a specific
question that can be asked of the student). This means that GTE represents
activities that an instructor does, and the methods to perform the activities.

306
What is unclear is what decision making process is modelled. Is the decision of
how to decide what instructional task to perform next modelled explicitly and
can it be edited by an author? To be more specific, are GTEs dynamic method
selection mechanisms modelled after teachers decision making strategies?
How are particular conditions chosen? For example, how is the value for
number-of-wrong-answers-exceeded to be true determined? Do the instructional plans that are embedded in the methods definition, resemble teachers
plans?
Chen (1995) presents a methodology for characterizing computer-based
learning environments which focuses on the cognitive, pedagogical and interactive features. The methodology distinguishes between basic pedagogical
strategies (e.g., setting goals, providing instructions, demos, explanations,
illustrations and working spaces, presenting tasks, asking questions, providing
evaluation and feedback) and advanced pedagogical strategies (e.g., modelling, coaching, reflection, articulation, scaffolding and fading, exploration)
each of which is made up of a pattern of basic pedagogical strategies. For
example, modelling comprises instruction, explanation and demonstration. It
would be interesting to see how GTEs instructional knowledge base covers
these pedagogical strategies identified in Chens methodology, and also to see
how it addresses the knowledge requirements put forth in Ohlssons (1991)
three views on teaching.

2. Instructional planning: Supporting pedagogical decision making

In the context of a computer-based learning environment, an instructional
interaction can be defined as any form of interaction between the student
and the instructional material, or the computer-mediated interaction between
collaborating students, or between the student and a teacher. The instructional
component of a computer-based learning environment is responsible for determining what to do next at each point in an instructional interaction, hence it is
in control of the systems behaviour. One popular approach to designing the
instructional component is to view pedagogical decision making as a planning
problem. Studies of the instructional process from the educational perspective
(e.g., Ohlsson, 1986; Leinhardt & Greeno, 1986; Putman, 1987; McArthur et
al., 1988), and numerous approaches to the creation of a pedagogical component for an ITS (e.g., Peachey & McCalla, 1986; Macmillan & Sleeman,
1987; Derry, Hawkes & Ziegler, 1988; Winkels, Breuker & Sandberg, 1988;
Woolf et al., 1988; Murray, 1989, 1990a,b; Wasson, 1990, 1992; Littman,
1990; Vassileva, 1990, 1995), all endorse a planning approach to managing
instructional interactions with the student.

307
Instructional planning, the application of AI planning techniques in the
domain of instruction is a major venture. Instructional planners must be able
to deal with uncertain, unstable and incomplete planning knowledge. The
world they reason about is unobservable and dynamic with the participants
(i.e. the system and the student and perhaps a teacher) having multiple goals
that are often conflicting. Instructional planning is the process of mapping out
a global sequence of instructional goals and actions that enables the system
to provide consistency, coherence, and continuity throughout an instructional
session (Wasson, 1990) and enables this global sequence to be interspersed
with local goals generated when instructional opportunities arise (e.g., a misconception is detected, a question is asked by the student, session time is
running out, a particular cognitive tool becomes relevant, fading instructional
support is required, a collaborative partner is recognized, etc.). This interleaving of global and opportunistic planning takes into account the interactive,
opportunistic, situated character of teaching (Goodyear, 1991a). In our definition, it should also be stressed that instructional planning is used to SUPPORT
instructional interaction, NOT to define it. The goals and plans are not for the
student,9 rather the are to indicate to the system what behaviour is required
given the current state of the learning environment (i.e. to enable the intelligent courseware to decide how to structure its activation to meet the needs of
the student).
Wasson (1990, 1992) distinguishes between content and delivery planning.
From an educational perspective, the separation between content and delivery
refers to the distinction between subject matter and the formats in which it
can be presented (Ohlsson, 1986). From a computational perspective, this
conceptual distinction enables a detailed investigation into the pedagogical
knowledge and reasoning required to generate content goals and content
plans while separating out delivery issues such as discourse planning or interface design. Thus, content planning entails generating, ordering and selecting
which content goals should be the focus of instruction given the current state
of the student, and monitoring the execution of the content plan in order
to determine when to replan, or generate a new plan. Delivery planning is
concerned with choosing the actual activities and instructional interactions
that help the learner achieve the goals (e.g., Mohan, Greer & Jones, 1992;
Merrill, Li & Jones, 1992). The majority of instructional planners developed
to date have focused on aspects of delivery. The systems essentially start with
an already chosen content and concentrate on determining how to deliver it
translating the already chosen content into a pedagogically effective form for
presentation to the student. For example, selecting a demonstration or remediation strategy (e.g., Woolf et al., 1988; Woolf & MacDonald, 1984a,b),
generating a question or task (e.g., McArthur et al., 1988), planning discourse

308
interaction (e.g., Winkels et al., 1988), sequencing instructions with demonstrations and problems and test items, etc., and generation and selection of a
particular delivery strategy (e.g., GTE) are all delivery planning issues. The
selection of the content, or topic, for the demonstration, explanation, question, etc. is a content planning issue. Aspects of content planning (although
the authors themselves may not refer to their work as content planning) are
addressed by Peachey & McCalla (1986) this work is the direct progenitor
for the PEPE approach as well as by Macmillan & Sleeman (1987), de
Castro, Sanchez & Verdejo (1988), Murray (1989, 1990a,b), Eggan & Lesgold
(1992), Major & Reichelt (1992). A comprehensive survey of how the notion
of curriculum in ITS has evolved from rigid course graphs to flexible, dynamic
content planning systems can be found in McCalla (1992). PEPE, however,
was the first attempt to weed out delivery issues and focus on content issues
alone. The next section presents PEPE as a content planner and GTE as a
delivery planner and shows how they complement one another.
2.1. Content and delivery planning: PEPE and GTE
PEPE,10 see Figure 1, is a competence-based content planning framework that
allows the designer of an ITS to incorporate many kinds of pedagogical knowledge and to encode a variety of pedagogical styles into an ITS. The concept
knowledge base (CKB) identifies the concepts to be learned and the relationships (e.g., prerequisite, subconcept, specialization, etc.) between them. It
constitutes the tutoring systems conceptual understanding of a domain. That
is, the CKB abstracts, into concepts, the domain knowledge the student is to
learn (i.e. the CKB does not constitute the domain knowledge, but rather it is
the goal structure for learning it). The misconception knowledge base (MKB)
enumerates common misconceptions that occur in the domain and relates
them to concepts in the CKB. The student model (SM) keeps track of the
systems beliefs about the students understanding (and misunderstanding) of
the domain. The SM used by PEPE is a modified overlay model where each
entry corresponds to a concept in the CKB and an ability to use that concept. The student history (SH) maintains the history of concepts the system
believes the student has learned, and the plan history (PH) records the goals
and plans that have been generated by PEPE to date, and whether or not they
were successful.
In the PEPE approach to content planning there are three phases: goal
generation, plan generation and plan monitoring. The content planning rules
(CPR) encode these three phases of content planning. First, a content goal
relevant to the students current knowledge state, the goal structure of the
domain to be learned, and the pedagogical principles for maneuvering through
the goal structure, is generated. Once the current goal has been set, a content

309

Figure 1. PEPEs conceptual architecture.

plan is generated based on the history of the students learning, the current
knowledge state of the student, and the pedagogical principles which express
how to engage the student in achieving the content goal. Finally, the execution
of the content plan (by a delivery planner) is monitored in order to determine
when the current content plan has been carried out successfully, needs to be
interrupted, or has failed and replanning must take place.
Content planning rules (CPR) consist of two types: pedagogical planning rules which encode pedagogical principles, and control rules which
manage the interactions of the pedagogical planning rules. The pedagogical
philosophy embedded in the pedagogical planning rules (PPR) are captured
in goal generation rules, plan generation rules and plan monitoring rules
which correspond to the three phases of content planning. An example of
the PPR can be found in Table 2. The rules shown here encode a tutor-style
instructional strategy. The control rules manage the planning cycle and thus
determine when each of the phases is carried out. They consist of meta-rules
that describe the interactions of the rules within each phase of the planning
cycle as well as between the phases. In addition to the CKB, and SM, the content planning rules synthesize information provided in three other knowledge
bases in order to generate a content plan.

310
Table 2. Examples of pedagogical planning rules in PEPE.
1.





2.





3.




Goal generation
Goal preparation rules (e.g. achieve facts before analysis)
Subgoal generation rules (e.g. when achieving analysis of a concept achieve analysis
of one of its specializations)
Subgoal selection rules
meta selection rules (e.g. use concept prerequisite rules before pedagogical
preference rules)
concept prerequisite rules (e.g. do iteration before recursion)
pedagogical preference rules (e.g. prefer cdr recursion before car recursion
can be domain dependent preferences of an individual teacher or domain
expert)
Plan generation
Operator assignment rules (e.g. if the student knows concept x then review it)
Modifer assignment rules (e.g. when to plan to achieve fact, analysis or synthesis of
a concept)
Plan subgoal generation rules (e.g. if planning to achive analysis of x then focus on
analysis of x and then have the student acquire x)
Plan monitoring
Blocking rules (e.g. if the executor has tried twice in a row to satisfy a goal without
success, assign the goal block-1 status)
Plan adaptation rules (e.g. if the currently planned for goal is not achieved and the
current plant has been carried out, then the plan has failed)

In PEPE, a content plan is derived to achieve a content goal. Each node in the
content plan consists of an operator, a modifier and a content. The operator
encodes the notion of planning learning events (or learning functions,
Shuell, 1992) by indicating to the delivery planner the type of cognitive
processing in which to engage the student. In Figure 1, a plan for the content
goal have the student learn to analyse recursion has been generated using
three operators: achieve, focus and acquire. Achieve means have the student
learn, focus refers to setting up an expectation for learning, and acquire
means set up a learning situation where the student can acquire the new
information.11 The content of a plan node is the concept from the current
content goal (recursion in Figure 1). The plan nodes modifier indicates which
ability (e.g., f-fact; a-analysis; s-synthesis adapted from Bloom (1956) as
described in Section 1.2) is appropriate, and it too is taken from the current
content goal. The nodes of the content plan are linked by either substep links
that indicate a hierarchical organization, or by sequence links indicating an
execution order. If a plan node has subnodes then execution of the node is
carried out by executing the subnodes.
GTE focuses mostly on delivery issues, although aspects of content planning can be found. The navigate library, for example, is a blending of content

311
(select tasks) and delivery (teach task) tasks and addresses navigation through
the topic structure keeping prior knowledge and prerequisite links in view and
using the learner attribute control to influence the selection of the methods.For
example, select tasks such as select-exercise, select-domain-object, selectcase, etc. deal with delivery of a topic to a student, while sequencing of topics
by author predefined order, or learner selection are content issues. It is similar
with the coaching library although it deals to a lesser degree with content
issues only determine-whether-to-intervene and select-topic are content
tasks. The elaborate and exercises libraries, on the other hand, address only
delivery issues.
While GTE has been presented here as a delivery planner because of its
numerous tasks and methods that deal with delivery issues, what is unclear
in the presentation of GTE in this issue, is how a course is managed and
controlled (i.e. what instructional planning is at a high level). Van Marcke
(1998, this issue) says that there are instructional plans inherent in the structure
of the instructional knowledge base, but what are the instructional goals? How
can there be planning without goals? The entire PEPE approach is based on
generation of a current content goal and a content plan for carrying out that
goal. The generation and selection of content goals is based on the dynamic
nature of the instructional interactions and continually updated understanding
of the students knowledge of what is to be learned, the process with which the
learning has taken place and a history of how the instruction has been given.
The generation and selection process ensures continuity and coherence in
the instructional session. How is continuity and coherence ensured in GTE?
For example, in PEPE the content planning rules deal with suspending a
plan in order to remediate some content and then deals with resuming the
original plan if it is still appropriate, or generating a new plan if needed. It
also deals with replanning if a plan becomes no longer valid because there is
new evidence that a student knows the intended content. It would be helpful
to know how goals and plans emerge in GTE.
3. Summary and conclusions
GTE and PEPE have the common goal of capturing and representing the
pedagogical knowledge needed for dynamic pedagogical decision making.
GTE provides an instructional knowledge base which has two functions:
to guide a courseware author, and to dynamically manage the instructional
dialogue with a student. PEPE offers a computational framework for explicit
representation of pedagogical knowledge and the processes for reasoning
about it to make pedagogical content decisions. Using the content-delivery
distinction (Ohlsson, 1988; Wasson, 1990, 1992; Dijkstra et al., 1992), GTE

312
has been conceptualized as a delivery planner while PEPE is a content planner.
A brief example of how the two complement one another has been given. A
deeper comparison is not possible because the planning issues have not been
highlighted in the presentation of GTE in this issue and also because GTE is a
fully developed system that has been utilized in several domains while PEPE
is only a prototype to illustrate the utility of a content planning approach. It
does, however, seem that a merging of the ideas of PEPE with GTE would
be a fruitful adventure.

Acknowledgements
The author would like to thank Mary Helander for her valuable comments.
This paper was written while the author was a visiting professor at the Department of Computer and Information Science, Linkoping University, Sweden.
Financial support was provided by NorFA (Nordic Research Development
Academy) grant number 95.30.183-O.

Notes
1. For example, see the collection of papers on instructional models in computer-based
learning environments (Dijkstra et al., 1992) and on teaching knowledge and intelligent
tutoring (Goodyear, 1991b).
2. We recognize that intelligent courseware (i.e., learning materials) is only a small part of
the Learning Environment other elements include: teachers, monitors, tools, information sources, fellow learners, school/university/workplace/self-learning, the socio-cultural
niche, other learning material, and designers and developers (e.g., researchers, instructional
designers, teachers themselves, etc.) of the learning material (Wasson, 1997).
3. This is a short, and updated, summary of a literature review in Wasson (1990).
4. See Goodyear (1991ab) for reflections on using research on teaching to guide the design
of intelligent courseware.
5. See Lowyck & Elen (1993) for a summary and a discussion of whether this is even
possible.
6. In fact, a preliminary investigation into how these learning functions can be incorporated
into PEPE is given in Shuell (1992), and Mohan (1991; Mohan et al., 1992) implements
the learning functions in IDP, an instructional delivery planner that cooperates with PEPE
to achieve instructional goals by delivering the content plan generated by PEPE.
7. Although Ohlssons interpretation of theories of teaching parallels the evolution of theories
constructivisim), it is interesting to note
of learning (i.e., behaviourism cognitivism
that the knowledge requirements for a higher-order theory do not replace those of a
lower-order theory, but augment them (i.e., the third-order theory requires the knowledge
base of the first- and second-order theories.
8. Blooms taxonomy is only one representation of cognitive outcomes. Gagne (1985), for
example, maps learning outcomes into the following dimensions: verbal information,
facts, concepts, rules, problem-solving, cognitive strategy and attitudes/motor skills. Shute
(1995) distinguishes between symbolic knowledge, procedural skill and conceptual knowledge.

313
9. One aspect of instructional planning could be to set explicit learning goals for the student,
to figure out what the students learning goals are, or even to help the student set reasonable
learning goals.
10. A prototype based on the PEPE framework has been implemented and tested within the
domain of recursive LISP programming.
11. Note that there is no description of how to help the student acquire the material, that is a
delivery issue.

References
Anderson, J. (1983). The Architecture of Cognition. Cambridge: Harvard University Press.
Anderson, J., Boyle, C.F. & Reiser, B.J. (1985). Intelligent tutoring systems. Science 28(4698):
456462.
Bloom, B.S. (1956). Taxonomy of Educational Objectives, Handbook I: Cognitive Domain.
New York: David McKay, Company Inc.
Burton, R.R. & Brown, J.S. (1978). Diagnostic models for procedural bugs in basic mathematical skills. Cognitive Science 2: 155192.
Carpenter, T.P. & Moser, J.M. (1984). The acquisition of addition and subtraction concepts in
grades one through three. Journal for Research in Mathematics Education 15: 179202.
Chen, M. (1995). A methodology for characterizing computer-based learning environments.
Instructional Science 23: 183220.
Chomsky, N. (1965). Aspects of the Theory of Syntax. Cambridge, MA: MIT Press.
de Castro, M.I., Sanchez, A. & Verdejo Maillo, M.F. (1988). Building a programming tutor
by dynamic planning: Case studies and a proposal. Proceedings of Intelligent Tutoring
Systems (ITS-88) (pp. 230237). Montreal.
Derry, S.J., Hawkes, L.W. & Ziegler, U. (1988). A plan-based opportunistic architecture for
intelligent tutoring. Proceedings of Intelligent Tutoring Systems (ITS-88) (pp. 116123).
Montreal.
Dijkstra, S., Krammer, H.P.M. & van Merrienboer, J.J.G., eds. (1992). Instructional Models
in Computer-Based Learning Environments. NATO ASI Series F, Vol. 104. New York:
Springer-Verlag.
Douglas, S.A. (1991). Tutoring as interaction: Detecting and repairing tutoring failures, in P.
Goodyear, ed., Teaching Knowledge and Intelligent Tutoring (pp. 123148). Norwood, NJ:
Ablex.
Douglas, S.A. (1988). Detecting and repairing tutoring failures. CIS-TR-88-09, Department of
Computer and Information Science, University of Oregon, Eugene, Oregon.
Eggan, G.M. & Lesgold, A.M. (1992). Modeling requirements for intelligent training systems,
in S.A. Dijkstra, H.P.M. Krammer & J.J.G. van Merrienboer, eds., Instructional Models
in Computer-Based Learning Environments. NATO ASI Series F, Vol. 104 (pp. 97111).
New York: Springer-Verlag.
Elbaz, F. (1983). Teacher Thinking: A Study of Practical Knowledge. New York: Nichols.
Escott, J.A. (1988). Problem-solving by analogy in novice programming, M.Sc thesis, Department of Computational Science, University of Saskatchewan, Saskatoon, Saskatchewan.
Fox, B.A. (1991). Cognitive and interactional aspects of correction in tutoring, in P. Goodyear,
ed., Teaching Knowledge and Intelligent Tutoring (pp. 149172). Norwood, NJ: Ablex.
Gagne, R.M. (1985). The Conditions of Learning and Theory of Instruction. New York: Holt,
Rinehart & Winston.
Gagne, R.M. & Briggs, L.J. (1974). Principles of Instructional Design (2nd ed.). New York:
Holt, Rinehart & Winston.
Gagne, R.M., Wager, W. & Rojas, A. (1991). Planning and authoring computer-assisted instruction lessons, in L.J. Biggs, K.L. Gustfson & M. Tillman, eds., Instructional Design: Princi-

314
ples and Applications (2nd Edition) (pp. 211226). Englewood Cliffs, NJ: Educational
Technology Publications.
Goodyear, P. (1991a). Research on teaching and the design of intelligent tutoring systems, in
P. Goodyear, ed., Teaching Knowledge and Intelligent Tutoring (pp. 323). Norwood, NJ:
Ablex.
Goodyear, P., ed. (1991b) Teaching Knowledge and Intelligent Tutoring. Norwood, NJ: Ablex.
Grossman, P.L., Wilson, S.M. & Shulman, L.S. (1989). Teachers of substance: Subject matter
knowledge for teaching, in M.C. Reynolds, ed., Knowledge Base for the Beginning Teacher
(pp. 2326). Toronto: Pergamon Press.
Huang, X. (1987). Finding language errors and program equivalence in automated programming advisor. M.Sc. Thesis, ARIES Research Report 87-3, Department of Computer
Science, University of Saskatchewan, Saskatoon, Saskatchewan.
Jones, M. (1988). Instructional systems need instructional theory: Comments on atruism. NATO
Advanced Research Workshop on New Directions in Educational Technology, Cranfield,
England.
Johnson, W.L., Soloway, E., Cutler, B. & Draper, S. (1983). Bug catalogue I. Technical Report
#286, Department of Computer Science, Yale University, New Haven, CT.
Landa, L.N. (1983). The Algo-heuristic theory of instruction, in C.M. Reigeluth, ed., Instructional Design Theories and Models: An Overview of their Current Status (pp. 163211).
New Jersey: Lawrence Erlbaum Associates.
Leinhardt, G. & Greeno, J.G. (1986). The cognitive skill of teaching. Journal of Educational
Psychology 78(2): 7595.
Lesgold, A. (1988). Toward a theory of curriculum for use in designing intelligent systems, in
H. Mandl & A. Lesgold, eds., Learning Issues for Intelligent Tutoring Systems (pp. 114
137). New York: Springer-Verlag.
Lesgold, A. (1989). Toward intelligent systems for testing, in L. Resnick, ed., Knowing,
Learning and Instruction: Essays in Honor of Robert Glaser. Hillsdale, NJ: Lawrence
Erlbaum Associates.
Littman, D. (1990). Strategies for tutoring multiple bugs. Ph.D. Thesis, Yale University, New
Haven, Conn.
Littman, D., Pinto., J. & Soloway, E. (1985). Observations on tutorial expertise. Proceedings
of IEEE Conference on Expert Systems in Government, Washington, DC.
Littman, D., Pinto., J. & Soloway, E. (1986). An analysis of tutorial reasoning about programming bugs. Proceedings of the National Conference on Artificial Intelligence, Pittsburgh,
Penn., 335338.
Littman, D. & Soloway, E. (1986). Towards an empirically-based process model for a machine
programming tutor. Proceedings of the IEEE International Conference on Systems, Man
and Cybernetics. Atlanta, GA.
Lowyck, J. & Elen, J. (1993). Transitions in the theoretical foundation of instructional design,
in T.M. Duffy, J. Lowyck & D.H. Jonassen, eds., Designing Environments for Constructive
Learning (pp. 213231). NATO ASI Series F. Heidelberg: Springer-Verlag.
Macmillan, S.A. & Sleeman, D.H. (1987). An architecture for a self-improving instructional
planner for intelligent tutoring systems. Computational Intelligence 3(1).
Major, N. & Reichelt, H. (1992). COCA: A shell for intelligent tutoring systems, in C. Frasson,
G. Gauthier & G.I. McCalla, eds., Proceedings of the Second International Conference on
Intelligent Tutoring Systems (ITS 92) (pp. 6673). Montreal.
McArthur, D., Stasz, C., Hotta, J., Peter, O. & Burdorf, C. (1988). Skill-oriented sequencing
in an intelligent tutor for basic algebra. Instructional Science 17: 281307.
McKendree, J., Reiser, B. & Anderson, J. (1984). Tutorial goals and strategies in the instruction
of programming skills. Proceedings of the Sixth Annual Conference of the Cognitive
Society, 252254.
Merrill, M.D. (1983). Component display theory, in C.M. Reigeluth, ed., Instructional Design
Theories and Models: An Overview of their Current Status (pp. 279333). New Jersey:
Lawrence Erlbaum Associates.

315
Merrill, M.D. (1987). The new component design theory: Instructional design for courseware
authoring. Instructional Science 16: 1934.
Merrill, M.D. (1988). Applying component display theory to the design of courseware, in D.
Jonassen, ed., Instructional Designs for Microcomputer Courseware (pp. 6196).
Merrill, M.D., Li, Z. & Jones, M.K. (1990). Limitations of first generation instructional design.
Educational Technology 30: 711.
Mohan, P. (1991). Adapting instruction to individual learners: Delivery planning for intelligent
tutoring systems. M.Sc. Thesis, Department of Computational Science, University of
Saskatchewan, Saskatoon, Canada.
Mohan, P., Greer, J. & Jones, M. (1992). Enhancing the instructional capability of intelligent
tutoring systems, in C. Frasson, G. Gauthier & G.I. McCalla, eds., Proceedings of the
Second International Conference on Intelligent Tutoring Systems (ITS 92) (pp. 532538).
Montreal.
Murray, T. (1998). Instructional Science. This issue.
Murray, W.R. (1989). Control for intelligent tutoring systems: A blackboard-base-d dynamic
instructional planner. Proceedings of the Fourth International Conference on AI and Education. Amsterdam, 150169.
Murray, W.R.(1990a). A blackboard-based dynamic instructional planner. Research Report
No. R-6376, Artificial Intelligence Center, FMC Corporation, Santa Clara, CA.
Murray, W.R. (1990b). A blackboard-based dynamic instructional planner. Proceedings of the
Eighth National Conference on Artificial Intelligence. Boston, 434441.
Ohlsson, S. (1986). Some principles of intelligent tutoring. Instructional Science 14: 293326.
Also, in R. Lawler & M. Yasdani, eds., AI and Education: Learning Environments and
Intelligent Tutoring Systems. New Jersey: Ablex Publishing, 1987.
Ohlsson, S. (1991). Knowledge requirements for teaching: The case of fractions, in P.
Goodyear, ed., Teaching Knowledge and Intelligent Tutoring (pp. 2559). Norwood, NJ:
Ablex.
Peachey, D.R. & McCalla, G.I. (1986). Using planning techniques in intelligent tutoring
systems. International Journal of Man Machine Studies 24: 7788.
Peterson, P.L. (1988). Teachers and students cognitional knowledge for classroom teaching
and learning. Educational Researcher 17(5): 514.
Peterson, P.L., Fennema, E., Carpenter, T.P. & Loef, M. (1989). Teachers pedagogical content
beliefs in Mathematics. Cognition and Instruction 6(1): 140.
Perez, R.S. & Neiderman, E.C. (1992). Modeling the expert training developer, in R.J. Seidel
& P. Chatelier, eds., Advanced Training Technologies Applied to Training Design. New
York: Plenum Press.
Putman, R.T. (1987). Structuring and adjusting content for students: A study of live and
simulated tutoring of addition. American Educational Research Journal 24(1): 1348.
Reigeluth, C.M., ed. (1983). Instructional Design Theories and Models: An Overview of Their
Current Status. New Jersey: Lawrence Erlbaum Associates.
Rowland, G. (1992). What do instructional designers actually do? An initial investigation of
expert practice. Performance Improvement Quarterly 5(2): 6586.
Scandura, J.M. (1983). Instructional strategies based on the structural learning theory, in C.M.
Reigeluth, ed., Instructional Design Theories and Models: An Overview of their Current
Status (pp. 213246). New Jersey: Lawrence Erlbaum Associates.
Shuell, T.J. (1992). Designing instructional computing systems for meaningful learning, in M.
Jones & P. Winne, eds., Adaptive Learning Environments. NATO ASI Series F, Vol. 85
(pp. 1954). New York: Springer-Verlag.
Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational
Researcher 15(2): 414.
Shute, V.J. (1995). SMART: Student modeling approach for responsive tutoring. User Modeling
and User-Adapted Interaction 5: 144.

316
Sopher, J.C., Pope, E., Lipman, M., Sack, W., Freiman, S., Littman, D., Johnson, L. & Soloway,
E. (1985). Bug Catalogue: II, III, IV. Research Report #386, Department of Computer
Science, Yale University, New Haven, CT.
Spector, M.J. (1998). The role of epistemology in instructional design. Instructional Science:
This issue.
Van Marcke, K. (1990). A Generic tutoring environment. Proceedings of the 9th European
Conference on Artificial Intelligence, 655660.
Van Marcke, K. (1992a). A generic task model for instruction, in S.A. Dijkstra, H.P.M Krammer
& J.J.G. van Merrienboer, eds., Instructional Models in Computer-based Learning Environments. NATO ASI Series F, Vol. 104 (pp. 171194). New York: Springer-Verlag.
Van Marcke, K. (1992b). Instructional expertise, in C. Frasson, G. Gautier & G.I. McCalla, eds.,
Intelligent Tutoring Systems: At the Crossroads of Artificial Intelligence and Education.
Norwood, NJ: Ablex.
Van Marcke, K. (1998). GTE: An epistemological approach to instructional modelling. Instructional Science: This issue.
Vassileva, J. (1990). An architecture and methodology for creating a domain-independent
plan-based intelligent tutoring system. Educational & Training Technologies International
27(4): 386397.
Vassileva, J. (1995). Reactive instructional planning to support interacting teaching strategies.
Proceedings of the 7-th World Conference on AI and Education (pp. 334342). Washington:
AACE.
Wasson (Brecht), B.J. (1990). Determining the Focus of Instruction: Content planning for
intelligent tutoring systems, Ph.D dissertation, Research Report 90-5, Department of
Computational Science, University of Saskatchewan, Canada.
Wasson, B. (1992) PEPE: A computational framework for a content planner, in S.A. Dijkstra,
H.P.M Krammer & J.J.G. van Merrienboer, eds., Instructional Models in Computer-Based
Learning Environments. NATO ASI Series F, Vol. 104 (pp. 153170). New York: SpringerVerlag.
Wasson, B. (1997). Advanced educational technologies: The learning environment. Computers
in Human Behaviour 13(4): 571594.
Wilson, S.M., Shulman, L.S. & Richert, A.E. (1987). 150 different ways of knowing:
Representations of knowledge in teaching, in J. Calderhead, ed., Exploring Teachers
Thinking (pp. 104124). London: Cassell.
Winkels, R., Breuker, J. & Sandberg, J. (1988). Didactic discourse in intelligent help systems.
Proceedings of Intelligent Tutoring Systems (ITS 88) (pp. 279285). Montreal.
Winne, P.H. (1992). State-of-the-art instructional computing systems that afford instruction
and bootstrap research, in M. Jones & P. Winne, eds., Adaptive Learning Environments.
NATO ASI Series F, Vol. 85 (pp. 349380). New York: Springer-Verlag.
Woolf, B. & McDonald, D.D. (1984a). Context dependent transitions in tutoring discourse.
Proceedings of the Fifth Annual Conference of the American Association for Artificial
Intelligence (AAAI) (pp. 335361).
Woolf, B. & McDonald, D.D. (1984b). Building a computer tutor: Design issues. IEEE
Computer 17(9): 6173.
Woolf, B., Murray, T., Suthers, D. & Schultz, K. (1988). Knowledge primitives for tutoring
systems. Proceedings of Intelligent Tutoring Systems (ITS 88) (pp. 491498). Montreal.