Improving Math Learning

Through Intelligent Tutoring and

Basic Skills Training
Ivon ARROYOa, Beverly Park WOOLFa, James M. ROYERb, Minghui TAIc, Sara ENGLISHc
a Department of Computer Science, b Department of Psychology
c School of Education
University of Massachusetts Amherst

So many factors affect learning with ITS

Software
Design

Teaching
Strategies

Student
Modeling

Memory

Implementation
Learning and
Achievement

Meta-cognitive

Basic
Cognitive

Social

Spatial
Affective

Italics: Students baggage to the learning situation

2

Motivation
Problem solving takes place in a cognitive system constrained by
a limited capacity of working memory
Working Memory Capacity When Solving a Math Problem

Basic Math

Strategy

Doing Math is like speaking a language.

If you are fluent, you will concentrate better on the message.
Math Fluency
Accuracy and Speed at Easy Math, e.g. addition, division
Math Fluency helps predict performance at state-wide tests
Training Math Fluency improves performance at tests
Royer&Tronsky (1998)
Royer et al. (1999)

Hypotheses
Do students learn more with a Math ITS when training basic
math facts retrieval (MFR) from memory?

Targeted hypotheses:
Does Math Fluency Training help students w/ IEPs learn more?
>> Students w/ Learning disabilities are seconds slower
Royer&Tronsky (1998)

Does Math Fluency Training help Females learn more?

>> Females milliseconds slower and helped explain gender gap in SAT-M
Royer (1999)

Back to studying multiplication tables!

4

Training Speed as a Game against yourself

Math Success Software, LATAS Lab, Psychology Dept UMASS. James Royer.

Training with sound

Assessment: speak answer

Click item.
Incorrect
Hear
answer.
Highlighted

Progress Charts
Accuracy

Speed

Say answer out loud

Hit bar.
Hear answer back.
R-L click to code R/W

See a new dot

for each round
5

As a supplement to Wayang Outpost

Multimedia Adaptive Tutoring System for Standardized-Tests Math Problems

As a supplement to Wayang Outpost

Multimedia Adaptive Tutoring System for Standardized-Tests Math Problems

Experiment Design
225 middle school students in a public high school
Six 7th grades, and six 8th grades
12 different classes of ~19 students.

Four groups, 3 classes each , assigned to either:

Wayang

Math Fluency Training

Yes
No
Yes

Wayang-MFR

Wayang-NoMFR
788

No

NoWayang-MFR
778

788

noWayang-NoMFR
778

Two 7th and one 8th grade, or two 8th and one 7th.
8

Procedure

Four days of study, one hour periods, within a 7-8 days.

Math test on 1st and last day, tests A and B counterbalanced
Math Facts Retrieval (MFR) posttest the last day
MFR Groups trained first 3 days.
Wayang Groups worked on tutor mostly on days 2-3.
NoWayang groups had either class instruction, or other web sites
Math Facts Retrieval Training
StandShort Wayang Math ITS
ardized
Math
Other math
webRetrieval
sites (NLVM,
Wayang
Fluency
Math
Math
Facts
Training
adventures)
Pretest
Math Class or no time in between

Detailed
Standardized
Math
Fluency
Math
Posttest
9

MFR Posttest with Math Fluency Software

Assessment: speak answer

No Training Phase

Progress Charts
Accuracy

Speed

Say answer out loud

Hit bar.
Hear correct answer back.
R-L click to code R/W

We did care
about speed
10

Expected Results
Realistically.

Some improvement on stand. math test due to Wayang

Even though it was our first experience with middle school students

Hesitant about students improving fluency

Previous MFR Training success was achieved after longer exposure

Taking away time from Wayang for fluency training

might not be beneficial for performance at standardized posttest

However, low achievement students really need MFR

Could be a hit for IEP students
Probably not better for females than males
11

Math Fluency Training Worked

True Means for Post-tutor Speed at Fluency (MFR) Posttest

ANCOVA for Math Fluency Posttest, Math Fluency Pretest as covariate:

MFR Training Main Effect: F(197, 1)=13.9, p<.001
Wayang Main Effect: F(197,1)=8.6, p<.023

12

Wayang helped improve Fluency

True Means for Post-tutor Speed at Fluency (MFR) Posttest

Wayang
groups

No Wayang
groups

ANCOVA for Math Fluency Posttest, Math Fluency Pretest as covariate:

MFR Training Main Effect: F(197, 1)=13.9, p<.001
Wayang Main Effect: F(197,1)=8.6, p<.023

13

Results on Standard Math Test

True Means and SD for the standardized math pretest and posttest

ANCOVA for Posttest with Pretest as Covariate, MFR, Wayang fixed factors:
Wayang F(222,1)=3.8, p=.05; Wayang x MFR (F(222,1)=7.9, p=.005 14
Post-hoc Contrasts: Wayang-MFR > all others

Differential Impact on Easy/Hard Items?

Easy/Hard is determined by item accuracy at pretest time

Easier Test Items

Harder Test Items

Divided the tests in half: easy items and hard items

Results on Standard Math Test (Easy Items)

True Means and SD for EASY items of the standardized pretest and posttest

ANCOVA for Easy Posttest with Easy Pretest as covariate; MFR,Wayang fixed factors:
Wayang F(222,1)=10.6, p=.001; WayangxMFR F(222,1)=5.1, p=.025
Post-hoc Contrasts: Wayang > no-Wayang? Yes.
Wayang-MFR > Wayang-NoMFR? No.

16

Results on Standard Math Test (Hard Items)

True Means and SD for HARD items of the standardized pretest and posttest

ANCOVA for Easy Posttest with Easy Pretest as covariate; MFR,Wayang fixed factors:
Wayang F(222,1)=6.8, p=.01; WayangxMFR F(222,1)=6.8, p=.009
Post-Hoc Contrasts: Wayang > no-Wayang? Yes.
17
Wayang-MFR > Wayang-NoMFR? Yes.

Differential Effects?
For specific groups of students

On Learning Disability Students?

Effects on IEP/LowAchievement students were not significant

On Female students?
Gender effects were not significant

Benefit of Fluency Training on math tests is for everyone

Or maybe need more exposure time to see differential effects

Math Fluency improved for everyone

Or maybe need more exposure time to see differential effects
18

Conclusions
Immediate Conclusions:
Math ITS that also train Math Fluency can teach better
* Freeing up memory resources
* Especially in hard math problems

A math ITS can help improve Math Fluency

* The benefit goes both ways

Far reaching implications:

Too easy problems are not necessarily inappropriate
* As long as the goal changes to train speed.

Addressing Basic Skills and Other Student Baggage

* Brought to the learning situation can lead to higher learning
19

Future Work
Lengthening the time of the Intervention
Creating a Math Fluency module within the Wayang ITS
Extending the idea of Math Fluency
Square root Fluency?
Equation solving Fluency?

20

Improving Math Learning

Through Intelligent Tutoring and
Basic Skills Training
Ivon ARROYOa, Beverly Park WOOLFa, James M. ROYERb, Minghui TAIc, Sara ENGLISHc
a Department of Computer Science, b Department of Psychology
c School of Education
University of Massachusetts Amherst

21