Behavioural Processes 152 (2018) 18–25

Contents lists available at ScienceDirect

Behavioural Processes
journal homepage: www.elsevier.com/locate/behavproc

Algorithmic analysis of relational learning processes in instructional T

technology: Some implications for basic, translational, and applied research
⁎
William J. McIlvane , Joanne B. Kledaras, Christophe J. Gerard, Lorin Wilde, David Smelson
University of Massachusetts Medical School, Worcester, MA, 01655, United States

A R T I C LE I N FO A B S T R A C T

Keywords: A few noteworthy exceptions notwithstanding, quantitative analyses of relational learning are most often simple
Stimulus control descriptive measures of study outcomes. For example, studies of stimulus equivalence have made much progress
Learning by exclusion using measures such as percentage consistent with equivalence relations, discrimination ratio, and response
Stimulus equivalence latency. Although procedures may have ad hoc variations, they remain fairly similar across studies. Comparison
Computerized algorithmic learning supports
studies of training variables that lead to different outcomes are few. Yet to be developed are tools designed
specifically for dynamic and/or parametric analyses of relational learning processes. This paper will focus on
recent studies to develop (1) quality computer-based programmed instruction for supporting relational learning
in children with autism spectrum disorders and intellectual disabilities and (2) formal algorithms that permit
ongoing, dynamic assessment of learner performance and procedure changes to optimize instructional efficacy
and efficiency. Because these algorithms have a strong basis in evidence and in theories of stimulus control, they
may have utility also for basic and translational research. We present an overview of the research program,
details of algorithm features, and summary results that illustrate their possible benefits. It also presents argu-
ments that such algorithm development may encourage parametric research, help in integrating new research
findings, and support in-depth quantitative analyses of stimulus control processes in relational learning. Such
algorithms may also serve to model control of basic behavioral processes that is important to the design of
effective programmed instruction for human learners with and without functional disabilities.

1. Introduction
This article discusses recent developments in the area of automated
instruction. It focuses specifically on new technology that has potential
for integrating stimulus control research with research on quantitative
analyses of behavior that concern the Society for Quantitative Analysis of
Behavior (SQAB). In making this contribution, we acknowledge that
none of the authors has much expertise in advanced quantitative ana-
lysis. That acknowledged, some of us have worked on certain problems
in this area, for example, studies of choice processes and behavioral
momentum (e.g. Dube and McIlvane, 2006). We were drawn to such
work by findings of our stimulus control research and indicated direc-
tions of our long-term program of research on relational learning (cf.
McIlvane et al., 2011). Our quantitative studies have been fruitful en-
ough for us to incorporate certain quantitative concepts in our theory
building efforts. For example, the McIlvane and Dube (2003) paper on
stimulus control topography coherence theory acknowledged its con-
ceptual debt to the work of prominent members of SQAB, most notably
that represented in the Davison and Nevin (1999) analysis.
The primary inspiration for the overall research program at the
Shriver Center came from the foundational work of B. F. Skinner and
Fred Keller (Skinner, 1968; Keller, 1968) as summarized in widely
known works such as The Technology of Teaching (1968) and Goodbye
Teacher (1968). The goal of such work was to develop and apply be-
havior analytic principles to create systematic instructional technolo-
gies that could be applied in applications with a wide range of learners.
In that vein, Murray Sidman and his group at the Massachusetts General
Hospital (MGH) initiated our program when they conducted among the
earliest instances of translational behavior analysis (McIlvane, 2009). In
their work, Sidman and his colleagues showed that functionally non-
verbal individuals and other people with profound neurological dis-
orders could be effectively taught and evaluated using stimulus control
shaping procedures deriving from studies of pigeons (Sidman and
Stoddard, 1966 and Terrace, 1963a,b, respectively).
One direction of the Sidman MGH research program and its suc-
cessors at the Shriver Center was to develop automated methods for
providing extensive support to people who needed an unusual level of
help to learn. That work yielded impressive results in many areas, most

⁎
Corresponding author.
E-mail address: william.mcilvane@umassmed.edu (W.J. McIlvane).

https://doi.org/10.1016/j.beproc.2018.03.001
Received 10 January 2018; Received in revised form 1 March 2018; Accepted 1 March 2018
Available online 12 March 2018
0376-6357/ © 2018 Elsevier B.V. All rights reserved.
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

notably in teaching simple discrimination (Sidman and Stoddard, 1966) by contrasting such relations with those learned previously, (2) is more
and conditional discrimination (Stoddard, 1982) performances to effective than mere differential reinforcement (e.g., Ferrari et al., 1993)
people with severe intellectual disabilities who had been deemed pre- and (3) does not rely on graduated stimulus changes as in stimulus
viously to be unable to learn them. More broadly, the vast majority control shaping procedures. Developed by McIlvane and Stoddard
foundational studies in behavior analytic stimulus control research on (1981), it is based on stimulus control analyses of the underlying be-
relational learning have been conducted using automated apparatus havioral processes (Dixon, 1981), analyses of “fast mapping” in early
(e.g., Sidman, 1971; McIlvane and Stoddard, 1981). Notably, grant child word learning (Carey and Bartlett, 1978), and cross-cultural ob-
proposals of that era often mentioned potential applications of that servations by Kagan (1981) of near-universality in toddler-aged chil-
work to development of more effective teaching machines. dren.
Given the many successes of the work just described, there was LBE involves (1) matching-to-sample procedures combining defined
great optimism that automated technologies and even completely au- (i.e. already learned) matching relations with undefined (not yet
tomated instructional environments would inevitably emerge. In ret- learned) matching relations (e.g., sample:comparison pairs
rospect, however, that work went against trends in instructional design “Spoon”:SPOON [defined] vs. “Spatula”:SPATULA [undefined] for a
that criticized then-available teaching machines specifically and care- toddler-aged child), (2) providing various contextual supports for
fully programmed instructional procedures generally as inappropriate learning the latter relations, (3) assessing quality of learning during and
methodologies for teaching broad classes of learners. As Benjamin after training, and (4) managing introduction of new relations, while
(1988) has recounted, the successes of classroom teaching machines in (5) minimizing and often virtually eliminating errors during training for
the Skinnerian tradition proved short-lived and eventually failed, thus most children. These features maintain attention to relevant aspects of
undercutting arguments that automated instruction was an area of fu- stimulus relations to be learned and motivate the child via the positive
ture promise. We think it likely that such failure was due in part to the feedback associated with ongoing success.
hardware used in automated instruction prior to the 1980s. Most such Table 1 presents an illustrative subset of matching-to-sample (MTS)
hardware was aesthetically unattractive, limited and inflexible in its trials from an LBE session designed to teach a new relation involving an
programming capabilities, and typically too expensive to replicate on a as-yet undefined dictated word from Set A and an as-yet undefined
large scale. picture that corresponds to it from Set B in the context of an MTS
Beginning in the mid-1980s, powerful graphics-oriented computers procedure. In this illustration, the contingencies of reinforcement have
began to offer alternative hardware options that promised to render already defined three comparisons stimuli (B1–B3) in relation to cor-
automated instruction increasingly attractive along many dimensions, responding sample stimuli (A1–A3) (Table 1, Lines 1–3; positive and
including cost. The Shriver group was a very early leader in use of such negative comparisons are indicated by + and −, respectively). Un-
technology to advance its stimulus control research program. That defined comparison stimulus B4 is introduced in a display with two
program focused specifically on developing the potential of Apple defined comparison stimuli and sample that corresponds to one of the
Macintosh hardware and software (the early stages of that apparatus latter (Table 1, Line 4). On this trial, selection of the positive compar-
development were summarized by Dube and McIlvane, 1989). One ison controls for indiscriminate selections of undefined stimuli. The key
product was a highly programmable authoring system (MTS, Dube and trial type of an LBE procedure is shown in Table 1, Line 6; the sample is
Hiris, 1999) for implementing a variety of simple and conditional dis- A4 and B4 is the positive comparison, to be selected by exclusion of the
crimination procedures supported by a number of stimulus control previously defined comparison stimuli.
shaping procedures. The authoring system permitted assessment of The LBE procedure has two types of tests for learning outcomes
learner performance and highly flexible program branching via a mini- (Table 1, Lines 9 and 11). On the first type, the sample is A4 and the
command language developed for that specific purpose. While clearly comparison display includes now defined B4 and an as-yet undefined
an advance in the field of automated instructional programming, MTS comparison, B5. This trial type controls for indiscriminate selection of
exists today only as laboratory software. It requires training and ex- recently introduced comparison stimuli. The second test trial type
perience to use it effectively, and dissemination outside the behavioral presents an undefined sample stimulus (A5) with the same comparison
research laboratory has not been practically achievable. display. It assesses whether the learner can now exclude the recently
A major purpose of this paper is to describe some next-generation defined comparison (i.e., B4). If the learner reliably selects the positive
software research and development in the Shriver program that has comparison stimuli on these two trial types, such performances indicate
been aimed specifically at development of more powerful, more user- that the specific A4:B4 relation has been acquired.
friendly procedures for accomplishing many of the functional outcomes
that the MTS program was designed to achieve. We will describe gen- 3. LBE Computerized Algorithmic Steering Logic (CASL)
eral procedures of this program and illustrative applications that il-
lustrate the capabilities of the software. With this context established, Recent Shriver research has sought to optimize the LBE method,
we will discuss how this approach might be further enriched by in- specifically incorporating an intelligent computerized algorithmic steering
corporating methods and perspectives that have been developed by logic for implementing the procedure. The main purpose of the CASL
members of SQAB and others pursuing quantitative analyses of beha-
vior. Table 1
Of note, while the matching-to-sample procedure embeds within it Subset of trials from a typical LBE teaching session.
concurrent schedules of reinforcement (cf. Jones and Elliffe, 2013),
Trial Type Sample Comparison Stimuli
potential schedule interactions of the type done in matching law re-
search are only rarely considered in stimulus control analyses. Beha- 1 Baseline A1 B1+ B2- B3-
vioral momentum principles are similarly understudied in stimulus 2 Baseline A2 B3- B1- B2+
3 Baseline A3 B2- B3+ B1-
control research. We hope to call attention to these and other possibi-
4 “Control” A2 B1- B4- B2+
lities for research to address an obvious gap in the literature. 5 Baseline A3 B3+ B2- B1-
6 “Exclusion” A4 B2- B1- B4+
2. Learning by exclusion 7 Baseline A1 B2- B3- B1+
8 Baseline A3 B1- B3+ B2-
9 “Ex. Test1” A4 B4+ B2- B5-
A primary target of our development effort has been refinement of
10 Baseline A2 B2+ B1- B3-
the learning by exclusion (LBE) method, a potentially errorless teaching 11 “Ex. Test2” A5 B4- B5+ B3-
method that (1) promotes learning of new stimulus-stimulus relations

19
W.J. McIlvane et al.
has been to systematize implementation of the various components of
the LBE procedure and eliminate one of its drawbacks–the requirement
that the user make a large number of initial and ongoing informed
decisions about the structuring of trial types within a session. As a
practical matter, such decisions require considering a great amount of
procedural detail, sometimes within a very short time window.
Moreover, managing those decisions while considering differences in
learning outcomes of individuals adds further complexity (i.e., mere
template methods may prove inefficient).
Critical decisions include: (1) the number of exclusion trials and
control trials programmed; (2) the number and nature of baseline MTS
trials that provide procedural context for LBE teaching; (3) the number
and nature of trials that test learning outcomes; (4) the rate of in-
troducing new relations via LBE; (5) the proportion of relations recently
acquired via LBE that are maintained in the baseline; (6) the pro-
grammed schedules of reinforcement for each of the trial types; (7) the
method for handling any persistent errors on learning outcome trials of
individual relations, and (8) the quality of balancing of the many type
of trials presented (cf. Gerard et al., 2014).
More subtle decisions in the design of LBE steering logics include (9)
dynamic variations in the proportion of LBE trials relative to baseline
trials: increases result in more practice with LBE trial types and coun-
terintuitive decreases in LBE trials render them relatively more novel, a
feature that may help in acquisition (Zeaman, 1976). Another decision,
also perhaps counterintuitive, derives from analyses of variables that
promote equivalence relations (Lionello-DeNolf, 2009), specifically
(10) the inclusion of interspersed identity matching trials that may
encourage development of sample-positive comparison controlling re-
lations (cf. McIlvane, 2013). Further details of and rationale for the
structure of the LBE CASL can be found in McIlvane et al. (2016).
We wish that we were able to report that the extant literature
contains sufficient data to guide researchers and teachers in selecting
values for each of the parameters just listed. Regrettably, we know of no
such parametric studies. Much of the work by the Shriver group has
focused on longer-term studies of minimally verbal children with neu-
rodevelopmental disorders–a population whose behavioral deficits
virtually preclude parametric studies for reasons of logistics and cost.
Opportunistically, we have explored variations within parameters 1–3,
5–6, and 9 that seemed to have good effect with some learners, but
controlled studies are lacking. There are good logical arguments and/or
supporting data for the remaining parameters, but none within the LBE
framework specifically.
In pointing to this current gap in the literature, we should empha-
size that a number of these parameters are not specific to LBE proce-
dures but apply also other widely studied procedures (e.g., basic dis-
crimination learning, stimulus equivalence). We believe that the virtual
lack of parametric studies of these variables is due in part to some lack
of recognition of their possible importance and perhaps more to the lack
of tools that render such studies practical. Early in the course of de-
veloping our LBE CASLs, we recognized that adaptations of them could
potentially bypass many logistical challenges with other stimulus con-
trol procedures.
4. Exemplary applications of LBE CASLs
Shriver researchers have sought to develop programs for teaching
stimulus-stimulus relations in which the stimuli are not physically
identical. Two types are needed: (1) programs for establishing so-called
feature (McIlvane et al., 1993) or perceptual (Fields, 2009) equivalence
classes and (2) arbitrary equivalence classes (Sidman, 1994). The former
type involves relations between/among stimuli that share defining
physical features (i.e., stimuli that resemble each other in some way).
The latter classes do not share defining features (e.g., pictures and
corresponding printed words) and are often termed symbolic MTS. What
follows will summarize validation studies that evaluated efficacy of LBE
CASLs in teaching children with intellectual disabilities (ID).
20
Behavioural Processes 152 (2018) 18–25
4.1. Application 1: teaching prerequisites for picture-aided communication
4.1.1. Three-dimensional (3D): two-dimensional (2D) stimulus-stimulus
relations
During typical development, children come very early to recognize
that 2D images (photos, pictures, drawings, movies, etc.) represent their
3D counterparts and are functionally equivalent in certain respects
(e.g., both may be named with the same label, pointing to a picture of a
desired item may be functionally equivalent to pointing to the item
itself in a request, etc.). Most children exhibit this capability by the
second year of life (Daehler et al., 1976). When a child has a neuro-
developmental disorder (e.g., autism spectrum disorder, Down Syn-
drome, etc.), however, development of 3D2D equivalence may be de-
layed and may even fail to develop (Carr and Felce, 2008). When
development of this pivotal capability does not occur, this is a major
obstacle for individualized intervention by behavior therapists, speech-
language pathologists, and special educators. Indeed, most approaches
in the augmentative/alternative communication (AAC) field (e.g., the
Picture Exchange Communication System [PECS; Bondy and Frost,
1994) depend ultimately on establishing or established 3D2D feature/
perceptual classes and functional equivalence.
In a recently concluded project, our group developed a broadly
applicable LBE CASL to teach 3D2D visual symbol matching as a pre-
requisite for AAC communication by nonverbal children with severe
IDD (McIlvane et al., 2016). Fig. 1 illustrates the matching task,
showing that we began with food items and corresponding pictures.
During the initial teaching, thirteen nonverbal/minimally verbal
participants learned a three-sample, three-comparison matching-to-
sample. The procedure introduced a new object:picture relation on
control and exclusion trials (18 each in a 72-trial session). Overall, the
first LBE session performance was virtually errorless for most partici-
pants (i.e. including all baseline, exclusion, and control trials). Next, the
new relation was added to the baseline and reviewed in three sub-
sequent baseline sessions in which all sample:comparison relations
were presented in equal proportions. This same approach was re-
plicated twice more, adding two new object:picture relations to the
baseline (see McIlvane et al., 2016 for additional details of procedure
and illustrative individual data).
Summarizing our findings, the 3D2D LBE CASL yielded LBE learning
outcomes better than the best published 3D2D data thus far reported
(Carr and Felce, 2008). Fig. 2 is adapted from McIlvane et al. (2016). It
shows that the magnitude of difference in accuracy on exclusion (left)
and outcome (right) trials with the LBE CASL and the Carr and Felce
data (re-analyzed to account for differences in performance attributable
to “chance” on 3- vs. 2-choice MTS tasks). Note that CASL above-chance
accuracy was significantly greater and inter-subject variability (black
range bars) was much less than those reported in the 2008 paper.
4.1.2. Teaching the first instances of reliable spoken word: picture matching
In these auditory-visual MTS studies, we combined the LBE CASL
just described with a progressive structuring procedure that used au-
ditory sample stimuli that were progressively more similar acoustically.
Briefly, we developed a stimulus series that began with maximally
discriminable auditory stimuli and progressed to minimally discrimin-
able ones. The stimulus series was scaled by detailed analyses of the
distinctive acoustical features of the stimuli within the series.
For program validation purposes, extended case studies were de-
signed to teach the first instances of reliable auditory-visual conditional
discrimination to two nonverbal children with autism spectrum dis-
orders. The results of this study were extremely promising. In training
extending over about two months, both children acquired 12 auditory-
visual relations with maintained accuracy that exceeded 99% in both
cases. Results with one child were startling indeed. This child acquired
the 12 new relations while making only one error in the many hundreds
of trials that comprised the overall teaching program. The other child
made only six errors during that program.
W.J. McIlvane et al.
Fig. 1. Illustrative display one a matching-to-sample trial presented in the LBE CASL to teach 3D (object) to 2D (picture) functional equivalence.
Fig. 2. Simplified presentation of data reported originally by McIlvane et al.
(2016), (Table 2). The histograms compare mean accuracy and accuracy
variability on exclusion trials (left histogram) and learning outcome trials (right
histogram) of children’s performance given training with our LBE CASL (left
histogram pair) and data reanalyzed from Carr and Felce (2008) (right histo-
gram pair).
As the time course and achieved accuracy may suggest, this teaching
was conducted extremely carefully due to the very low levels of func-
tioning of the children studied. We do not know–but strongly sus-
pect–that these excellent results were due to (1) the exceptionally
careful selection of auditory stimuli via distinctive feature analysis to
maximize discriminability and (2) the high level of teaching support
incorporated in the LBE CASL. We consider these to be very impressive
preliminary findings that establish the rationale for conducting
21
Behavioural Processes 152 (2018) 18–25
controlled comparison studies with standard-of-care procedures for
teaching relations of this nature.
4.2. Application 2: teaching Reading readiness skills to learners with ID
Benefitting from nearly 40 years of research since Sidman (1977),
our group was well-positioned to develop an evidence-based reading
readiness curriculum that could teach more children with ID more skills
more efficiently than ever before. We recently completed a feasibility
study aimed at developing and validating a CASL to systematize LBE
implementation with this subject matter; details of its procedures and
preliminary results were published (Kledaras et al., 2016).
Earlier research had shown that the LBE procedure has produced
highly efficient learning of reading prerequisites in children with long
histories of school failure (e.g., de Rose et al., 1996; de Souza et al.,
2009). The present work focused on preschool children with ID and/or
other developmental disorders associated with deficits with this subject
matter. The aim was to give such children their best chances to (1)
acquire prerequisite reading readiness skills at or near the typical age
and (2) develop functional reading skills in subsequent special educa-
tion. Fig. 3 exemplifies the LBE method for teaching relations involving
upper case (UC) and corresponding lower-case (LC) letters of the al-
phabet, a critical behavioral prerequisite for skilled reading. It pre-
supposes that the learner has mastered with identity matching to
sample (IDMTS) with UC and LC letters, thus showing that the letters
are discriminated visually from one another.
The upper left panel shows two trials from the IDMTS baseline with
UC letters. The panel below it shows two trials from an UC:LC MTS task
in which the LC letters differ in size but not form (i.e., members of
feature/perceptual classes). LBE procedures are shown in the right panels
in which physically dissimilar sample and comparison stimuli are in-
troduced in combinations with familiar ones. The first trial in the upper
right panel shows a task that requires the learner to match LC-a with
UC-A without prior training to do so. S/he can do that by excluding LC-
x, previously correct only in relation to UC-X. The trial beside it is a
control trial. After a history of reinforced LBE teaching trials, tasks in
the lower panel assess whether the new UC:LC matching relation has
been learned. These tasks require the learner to select yet another as-yet
undefined comparison (LC-g in this case) in relation to another as-yet
undefined sample (UC-G) while maintaining accurate matching of the
W.J. McIlvane et al. Behavioural Processes 152 (2018) 18–25

Fig. 3. Illustrative trial displays from LBE procedures used to teach children with neurodevelopmental disabilities to relate upper case with lower case letters of the
Latin alphabet.

recently defined UC:LC relation (A:a in this example).
Work thus far has systematically replicated procedures from Sidman
et al. (1974). We sought to teach children to relate (1) UC letters with
LC letters and (2) both letter sets with their corresponding dictated
names. Letter selection was guided by literature on letter similarity and
discriminability (e.g., Gilmore et al. 1979) to establish a set of UC:LC
pairs that ranged from highly similar to dissimilar. Participants were 12
children aged three to five years. Teaching was implemented in the 7-
step sequence shown in Table 2 (see Kledaras et al., 2016 for additional
details of procedure and their rationale).
Training time available was up to 5–6 hours per child. In that time,
ten children learned all targeted UC:LC relations to criterion (> 90%
correct selections) and nine completed the entire program, meeting
criterion on all relations tested including equivalence relations. Trial
accuracy scores were typically high during all seven steps of the
teaching sequence and inter-participant variability was typically low
except during the first step of the sequence. During Steps 6 and 7, ac-
curacy on teaching and test trials was uniformly high, inter-participant
variability was low, and equivalence class formation was robust. Thus,
the procedures were comparably effective with possibly familiar and
clearly unfamiliar letters.
Fig. 4 shows data for selected individual children. Participant N in
Panel A represents the outcome for most children. Error rate was low
and children met criterion on all relations.
Panel B shows results with Participant G, our youngest child. His
Step 4 accuracy was 81% – well above chance but his progress was a bit
slower than that of other children. With added direct training on au-
ditory-visual relations, however, he met criterion rapidly. Notably both
this child and one other who had no prior exposure to relational tasks
with letters of the alphabet met criterion on all relations.
Panel C shows that Participant S was similar to Participant G
through Step 4, and we treated him similarly. Unlike Participant G,
direct auditory-visual training did not immediately bring in criterion
performance with Participant S and one other child who showed this
acquisition pattern before we had to conclude data collection. That
acknowledged, both children had made substantial progress and
achieved near-criterion scores in their final sessions. We are confident
that both would have met criterion had we been able to conduct a few
more sessions.
5. Concluding comments on contributions and potential
implications
We think that the three application studies summarized here make a
good case for further research and development of algorithmic teaching
with LBE and stimulus equivalence procedures. We note also that CASL
development for other stimulus control teaching programs is underway
(e.g., for remediating so-called overselective attention in individuals
with neurodevelopmental disabilities (Farber et al., 2017)) and has had
comparably impressive successes. These studies together lead us to
suggest that automated algorithmic supports will prove to be increas-
ingly important tools for both computer-implemented teaching apps

Table 2
Summary of procedures that taught Upper Case (UC) and Lower Case (LC) matching relations and tested for equivalence among them in Kledaras et al. (2016).
Program Step Procedures Purpose

1 Two-comparison identity matching to sample (IDMTS): colors, pictures & UC letters C, P,
X
2 Three-comparison IDMTS: UC C, P, X, V, S, X; feature matching: C:c, P:p, X:x, V:v, S:s, X:x
3 Three-comparison IDMTS: UC C, P, X, V, S, X
4 Three-comparison IDMTS and arbitrary matching (Arb-MTS): G:g, A:a, R:r, D:d, E:e, and
H:h, introduced successively
5 Three-comparison auditory-visual MTS with UC and LC letters used in Step 4 and
corresponding dictated letter names
6 Three-comparison auditory-visual MTS with script letters (G, D, a, r) and selected UC and
LC letters: LC-g, LC-d, UC-A, and UC-R
7 Three-comparison auditory-visual MTS with script letters (G, D, a, r) and selected UC and
LC letters: UC-D, UC-D, LC-a, and LC-r
Verify reliable IDMTS
Verify IDMTS with six letters; verify feature matching with similar
UC and LC stimuli
Verify IDMTS with six letters to be used in teaching arbitrary UC:LC
relations
LBE teaching of arbitrary visual-visual symbolic relations
LBE teaching of arbitrary auditory-visual symbolic relations
Control procedure to assess possible extra-experimental learning of
UC and/or LC letter names
Tests for emergent relations based on stimulus equivalence

22
W.J. McIlvane et al.
Fig. 4. Individual data for three children representing the small performance variability summarized in Fig. 4. The upper graphs represent children who were
reported to have very little knowledge of the alphabet (AK), those who had no prior exposure to the alphabet, and those who had made little or no progress in their
preschool programs despite exposure to procedures that had successfully taught other children. The lower graphs show individual analyses of cumulative training
time for each of the seven steps of the teaching program.
and in computer-managed face-to-face instruction (as exemplified in
our 3D2D CASL study) for persons with severe neurodevelopmental
disabilities. If so, then the ambitious objectives of the early Sidman
MGH research initiative may ultimately be achieved.
Mindful as we are of the history of “overselling” and associated
failures in the area of automated programmed instruction, however, we
need to emphasize that the work here represents a preliminary de-
monstration of what may be possible rather than a mature, general-
izable behavior analytic instructional technology ready for widespread
dissemination. Even within the narrow applications we described and
considering our successes in teaching children with severe learning
challenges, there remains a need for fairly challenging randomized
controlled trials comparing our CASLs with other methods (e.g. verbal
instruction with point prompts, stimulus control shaping methodologies
(McIlvane and Dube, 1992), etc.) in order to conclude that the CASLs
represent superior instructional technology.
Potential contributions to development of behavior analytic sci-
ence
One objective in developing the LBE CASL was to develop a tool that
could be used to conduct parametric studies of learning by exclusion
and other stimulus control procedures such as those used in stimulus
equivalence research. In an earlier section, we listed 11 parameters that
could be considered in designing a fairly simple LBE teaching proce-
dure. We emphasize these parameters based on our history in devel-
oping and disseminating the LBE method. Surveying the LBE literature
as a whole, however, one sees very little attention to the range of
parameters that may affect learning outcomes. We think even the cur-
rent LBE CASL could be helpful in addressing this gap in the literature.
We note also that most of these parameters and a number of others
are potentially important in designing procedures to assess and promote
23
Behavioural Processes 152 (2018) 18–25
stimulus equivalence and other relational discriminations (e.g., gen-
eralized identity and oddity) in humans with or without developmental
limitations and in comparative cognition research with nonhumans. If
the values of such parameters are left to vary across studies, then one
would expect variability in outcomes–and such is the case in the present
literature (e.g., Lionello-DeNolf, 2009). We think that creating CASLs
that require attention to setting and reporting the values of these
parameters may help to minimize variability in across relational
learning studies and failures to replicate findings with research parti-
cipants and procedures that seem superficially comparable. Also, CASLs
that allow one to conveniently set values of parameters of interest
would encourage parametric research to define the contribution of
those parameter across their range.
Regarding parameters of interest to members of SQAB, we note that
our present CASLs include only the most basic attention to the inter-
actions between/among concurrent schedules of reinforcement that are
inherent in conditional discriminations. This limitation represents
common practice in much other relational learning research in which
interactions among programmed schedules are virtually ignored (e.g.,
as when potentially emergent equivalence relations are tested in ex-
tinction). Input from advances in quantitative analyses of behavior may
not only help to further augment the effectiveness and efficiency of the
CASLs but also advance the integration of principles and procedures of
quantitative analyses of choice, behavioral momentum, and other SQAB
foci with those of research on relational stimulus control, thus building
upon the initial efforts of Davison and Nevin (1999) and others pur-
suing a similarly comprehensive analytical scope.
One small example from our own work can be found in a paper by
Dube and McIlvane (2002a). Briefly, we found that discrimination re-
versal learning taught by a stimulus control shaping procedure was
more efficient if the original discrimination was taught with inter-
mittent vs. continuous reinforcement–a finding that is predictable from
W.J. McIlvane et al.
behavioral momentum analyses. As part of that same program, Dube
and McIlvane (2002b) found differential sensitivity to reinforce rate
and magnitude in individuals with neurodevelopmental disabilities that
had clear implications for behavior analyses of learning outcomes in
this population. Our own quantitative program has barely scratched the
surface, however, of possible applications of SQAB perspectives in sti-
mulus control research.
Real possibilities for developing research programs this area have
been recognized also by prominent SQAB members. As examples,
Randy Grace opened the 2003 SQAB meeting in part by articulating
what he viewed as fundamentally important objective: to “build a
bridge between behavioral research with nonhumans and human re-
search in cognition and decision making.” Dianne McCarthy’s (1991)
chapter on behavioral detection theory expressed similar thoughts.
Future versions of CASLs developed by our group or others could
explore quantitative variables that are rarely if ever studied in stimulus
control work. One obvious example derived from our 2002 study would
be to study the influence of intermittent reinforcement schedules on the
baseline trials that provide context for exclusion and control trials that
teach new relations in the LBE procedure: Would intermittent re-
inforcement on the former enhance accuracy on LBE learning outcome
tests and perhaps subsequent retention tests for the subset of children
who make errors on learning outcomes tests? Other research might
address more subtle interactions among the concurrent schedules
within matching-to-sample procedures, for example, perhaps devising
optimal schedule of reinforcement for promoting discrimination
learning of individual research participants or coming to understand
persistent conditional discrimination learning failures in seemingly
well-structured applied behavior analytic work.
To conclude, we believe that work of the type reported here has
potential for integrating major lines of research that have tended to go
forward mostly in isolation and perhaps miss opportunities to develop
more comprehensive analyses of behavior. One can certainly take ad-
ditional steps in that direction through traditional academic activities
(i.e., review articles, integrative chapter and books, etc.). However, we
believe that an even more tangible contribution would be creating
useful tools that incorporate empirical research across sub-disciplines of
behavior analysis. To the extent that such tools can also have clinical
and education applications that render them useful to other disciplines,
that contribution should also lead to both additional recognition of and
resources for behavior analytic science. In addition, training programs
that incorporate such a focus would require interdisciplinary colla-
borations that would likely prepare graduates for attractive careers
outside traditional academic environments.
As to how such activity might go forward, we think that there are
good reasons to implement evolving CASLs in open source formats (cf.
https://en.wikipedia.org/wiki/Open-source_software) that would allow
(1) open examinations of specific details of implementation and (2)
field-wide contributions that could be open to peer review and re-
cognition as scholarly contributions. Some process of this nature seems
necessary because it could help to distribute the considerable costs of
software development that are difficult practically to offset within
current research grants and contract structures in the U. S. and else-
where. While we understand the major challenges entailed in our
proposal, we think that potential benefits are worth the needed in-
vestment of time, energy, and money.
Disclosures and Conflicts of Interest
None of the authors have financial interests beyond past and/or
current grant supports listed above. There are no conflicts of interest to
report.
Acknowledgements
We are grateful to the Society for Quantitative Analysis of Behavior
24
Behavioural Processes 152 (2018) 18–25
(SQAB) for the invitation to contribute to this special section of
Behavioral Processes. The research program described in this article has
had long-term support from the U. S. National Institutes of Health, most
recently in Grants HD04147, HD25995, DC10365, and MH90272.
Manuscript preparation was supported by Grant AA026751 and an
Intergovernmental Personnel Agreement with the United States
Veterans Administration. All of these funding sources were investigator-
initiated, and none of the funding agencies were involved in formula-
tion, design, or execution of the supported projects. Contact Dr.
McIlvane for additional information concerning the EAHBB Research in
Progress article cited in the Application 2 section (william.mcilvane@
umassmed.edu).
References
Benjamin, L.T., 1988. A history of teaching machines. Am. Psychol. 43, 703–712.
Bondy, A., Frost, L., 1994. The picture exchange communication system. Focus Autism 9,
1–19.
Carey, S., Bartlett, E., 1978. Acquiring a single new word. Papers and Reports on Child
Language Development 15. pp. 17–29.
Carr, D., Felce, J., 2008. Teaching picture-to-object relations in picture-based requesting
by children with autism: a comparison between error prevention and error correction
teaching procedures. J. Intell. Disabil. Res. 52, 309–317.
Daehler, M.W., Permutter, M., Myers, N.A., 1976. Equivalence of pictures and objects for
very young children. Child Dev. 47, 96–102.
Davison, P., Nevin, J.A., 1999. Stimuli, reinforcers, and behavior: an integration. J. Exp.
Anal. Behav. 71, 439–482.
de Rose, J.C.C., Souza, D.G., Hanna, E.S., 1996. Teaching reading and spelling: stimulus
equivalence and exclusion. J. Appl. Behav. Anal. 29, 451–469.
de Souza, D.G., de Rose, J.C., Faleiros, T.C., Bortoloti, R., Hanna, E.S., McIlvane, W.J.,
2009. Teaching generative reading via recombination of minimal textual units: a
legacy of verbal behavior to children in Brazil. Int. J. Psychol. Psychol. Ther. 9,
19–44.
Dixon, L.S., 1981. A functional analysis of photo-object matching skills of severely re-
tarded adolescents. J. Appl. Behav. Anal. 14, 465–478.
Dube, W.V., Hiris, E.J., 1999. MTS v 11.6.7 [Computer Software Documentation]. E. K.
Shriver Center, Waltham, MA.
Dube, W.V., McIlvane, W.J., 1989. Adapting a microcomputer for behavioral evaluation
of mentally retarded individuals. In: In: Mulick, J., Antonak, R. (Eds.), Transitions in
Mental Retardation, vol. 4 Ablex, Norwood, NJ pp. 104–127.
Dube, W.V., McIlvane, W.J., 2002a. Reinforcer rate and stimulus control in discrimination
reversal learning. Psychol. Rec. 52, 405–416.
Dube, W.V., McIlvane, W.J., 2002b. Quantitative assessments of sensitivity to reinforce-
ment contingencies in mental retardation. Am. J. Ment. Retard. 107, 136–145.
Dube, W.V., McIlvane, W.J., 2006. Behavior-analytic experimental strategies and moti-
vational processes in persons with mental retardation. In: Switzky, H., Hickson, L.,
Schalock, R. (Eds.), International Review of Research in Mental Retardation, vol. 31:
Mental Retardation, Personality, and Motivational Systems. Academic Press, San
Diego, CA pp. 261–288.
Farber, R.S., Dickson, C.A., Dube, W.V., 2017. Reducing overselective stimulus control
with differential observing responses. J. Appl. Behav. Anal. 50, 87–105.
Ferrari, C., de Rose, J.C., McIlvane, W.J., 1993. Exclusion vs. selection training of audi-
tory-visual conditional relations. J. Exp. Child Psychol. 56, 49–63.
Fields, L., 2009. The synthesis of complex categories from perceptual and equivalence
classes: effects of training and testing parameters. Eur. J. Behav. Anal. 10, 119–142.
Gerard, C.J., Mackay, H.A., Thompson, B., McIlvane, W.J., 2014. Rapid generation of
balanced trial distributions for matching-to-sample procedures: a technical note. J.
Exp. Anal. Behav. 110, 171–178.
Gilmore, G.C., Hersh, H., Caramazza, A., Griffin, J., 1979. Multidimensional letter simi-
larity derived from recognition errors. Percept. Psychophys. 25, 425–431.
Jones, B.M., Elliffe, D.M., 2013. Matching-to-sample performance is better analyzed in
terms of a four-term contingency than in terms of a three-term contingency. J. Exp.
Anal. Behav. 100, 5–26.
Kagan, J., 1981. The Second Year: The Emergence of Self Awareness. Harvard University
Press, USA.
Keller, F.S., 1968. Goodbye, teacher. J. Appl. Behav. Anal. 1, 78–89.
Kledaras, J.B., Gerard, C.J., Twyman, J.S., McIlvane, W.J., 2016. Teaching some basic
prerequisites for reading in 2015: algorithmic learning by exclusion. Exp. Anal. Hum.
Behav. Bull. 30, 23–27.
Lionello-DeNolf, K.M., 2009. The search for symmetry: 25 years in review. Learn. Behav.
37, 188–203.
McCarthy, D.C., 1991. Behavioral detection theory: Some implications for applied human
research. In: Commons, M.L., Nevin, J.A., Davison, M.C. (Eds.), Signal Detection:
Mechanisms, Models and Applications. Erlbaum, Hillsdale, NJ pp. 239–255.
McIlvane, W.J., 2009. Translational behavior analysis: from laboratory science in sti-
mulus control to intervention with persons with neurodevelopmental disabilities.
Behav. Anal. 32, 273–280.
McIlvane, W.J., 2013. Simple and complex discrimination learning. In: Madden, G.J.
(Ed.), APA Handbook of Behavior Analysis. American Psychological Association,
Washington, DC pp. 129–163.
McIlvane, W.J., Dube, W.V., 1992. Stimulus control shaping and stimulus control
W.J. McIlvane et al.
topographies. Behav. Anal. 15, 89–94.
McIlvane, W.J., Dube, W.V., 2003. Stimulus control topography coherence theory:
foundations and extensions. Behav. Anal. 26, 195–213.
McIlvane, W.J., Dube, W.V., Green, G., Serna, R.W., 1993. Programming conceptual and
communication skill development: A methodological stimulus class analysis. In:
Kaiser, A.P., Gray, D.B. (Eds.), Understanding Children’s Language. Brookes,
Baltimore, MD.
McIlvane, W.J., Dube, W.V., Lionello-DeNolf, K.M., Serna, R.W., Barros, R.S., Galvão,
O.F., 2011. Some current dimensions of translational behavior analysis: From la-
boratory research to intervention for persons with autism spectrum disorders. In:
Mayville, E., Mulick, J. (Eds.), Behavioral Foundations of Effective Autism Treatment.
Sloan Publishing, Cornwall-on-Hudson, NY pp. 155–181.
McIlvane, W.J., Gerard, C.J., Kledaras, J.B., Mackay, H.M., Lionello-DeNolf, K.M., 2016.
Teaching stimulus-stimulus relations to nonverbal individuals: reflections on tech-
nology and future directions. Eur. J. Behav. Anal. 17, 49–68.
McIlvane, W.J., Stoddard, L.T., 1981. Acquisition of matching-to-sample performances in
severe retardation: learning by exclusion. J. Ment. Defic. Res. 25, 33–48.
Sidman, M., 1971. Reading and auditory-visual equivalences. J. Speech Hear. Res. 14,
5–13.
Sidman, M., 1977. Teaching some basic prerequisites for reading. In: Mittler, P. (Ed.),
25
Behavioural Processes 152 (2018) 18–25
Research to Practice in Mental Retardation, vol. II. University Park Press, Baltimore,
MD pp. 353-360.
Sidman, M., 1994. Equivalence Relations and Behavior: A Research Story. Authors
Cooperative, Boston.
Sidman, M., Cresson Jr., O., Willson-Morris, M., 1974. Acquisition of matching to sample
via mediated transfer. J. Exp. Anal. Behav. 22, 261–273.
Sidman, M., Stoddard, L.T., 1966. Programming perception and learning for retarded
children. In: In: Ellis, N.R. (Ed.), International Review of Research in Mental
Retardation, vol. 2 Academic Press, New York pp. 151–208.
Skinner, B.F., 1968. The Technology of Teaching. Appleton-Century-Crofts, New York.
Stoddard, L.T., 1982. An investigation of automated methods for teaching severely re-
tarded individuals. In: Ellis, N.R. (Ed.), International Review of Research in Mental
Retardation. Academic Press, New York pp. 163–207.
Terrace, H.S., 1963a. Discrimination learning with and without “errors”. J. Exp. Anal.
Behav. 6, 1–27.
Terrace, H.S., 1963b. Errorless transfer of a discrimination across two continua. J. Exp.
Anal. Behav. 6, 223–232.
Zeaman, D., 1976. The ubiquity of novelty-familiarity (habituation) effects. In: Tighe,
T.J., Leaton, R.N. (Eds.), Habituation: Perspectives from Child Development, Animal
Behavior, and Neurophysiology. Erlbaum, Hillsdale, NJ pp. 297–320.