Running head: WORD PROBLEM RESEARCH 1

What Does Research Say About Word Problem Solving Strategies Today?

A Meta-analysis.

Carlos Lavin

George Mason University

Running head: WORD PROBLEM RESEARCH 2

Abstract

Mathematics achievement continues to be a problem of interest in the United States (NAEP,

2005). In 2012, Zhang and Xin published a meta-analysis summarizing word problem

interventions since 1999 and thus, extending the work of Xin and Jitendra (1999). Due to the

continuing performance gap between students with disabilities and their non-disabled peers, this

meta-analysis extends the work previously conducted by Zhang and Xin (2012), and Xin and

Jitendra (1999) on word problem solving interventions for students with disabilities or at risk for

disabilities. Group design studies and single case studies are included in this analysis using

standardized mean change to measure group design effectiveness, and percentage of non-

overlapping data (PND) for single case studies. Results and implications for schools and the

research community will be discussed.

Running head: WORD PROBLEM RESEARCH 3

Problem

In 2015, The United States scored in 31st place out of 35 members of the Organization for

Economic Cooperation and Development (OECD) on the Programme for International Student

Assessment (PISA) (OECD, 2016). Also in 2015, only 40% of students in 4th grade and 33% of

students in 8th grade in the United States performed at or above the Proficient level on National

Assessment of Educational Progress (NAEP) assessments. Although scores have increased since

1990, they dropped by 4 points from 2013 (NAEP, 2015). Students with math disabilities (MD)

score comparatively lower on these assessments than their peers without disabilities. Although

40% of students in 4th grade and 33% in 4th grade were proficient, only 14% of MD students in

4th grade and 7% iof MD students in 8th grade scored at the proficient level (NAEP, 2015; Yan

Ping Xin & Jitendra, 1999; Zhang & Xin, 2012).

Achievement in mathematics in the United States is often assessed and measured by

standardized tests such as the NAEP, and these tests rely heavily on word problems (Driver &

Powell, 2017). Because of this, it is clear that it is of significant importance to provide quality

word problem solving instruction to students with MD (Peltier & Vannest, 2016; Powell, 2011;

Yan Ping Xin & Jitendra, 1999; Zhang & Xin, 2012). In fact, in 2009, the What Works

Clearinghouse (WWC) (Gersten et al., 2009) recommended that interventions at the elementary

level should include word problem solving instruction, and that this instruction should be explicit

and systematic, based on findings from a research synthesis on math interventions for struggling

students (Gersten et al., 2009; Jitendra, Dupuis, et al., 2013).

Previous Literature

Meta-analysis is a statistical technique for coding empirical studies and provides a

summary of findings and characteristics typically presented in research reports (Lipsey &
Running head: WORD PROBLEM RESEARCH 4

Wilson, 2001; Zhang & Xin, 2012). “Unlike more traditional narrative and vote-count research

synthesis, meta-analysis has the potential to compare the magnitude of different treatment effects

across studies to address more macroscopic research questions or high order interactions on a

specified topic” (Xin & Jitendra, 1999). There have been several key meta-analysis and reviews

of math published since 1999 (Dennis et al., 2016; Powell, 2011; Xin & Jitendra, 1999; Zhang &

Xin, 2012). However, only three of them attend to word problems as a field of study (Powell,

2011; Xin & Jitendra, 1999; Zhang & Xin, 2012), and only two are meta-analysis (Xin &

Jitendra, 1999; Zhang & Xin, 2012).

In 1999, Xin and Jitendra conducted a thorough review of the literature dating back to

1960. The search looked at studies published and not published from 1986 to 1996 on

Educational Resources Information Clearinghouse (ERIC), and Psychological Abstracts

(PsychLIT); from 1960 to 1996 in the Dissertation Abstract International (DAI); as well as a

hand search of issues in particular journals (i.e., Exceptional Children, Exceptionality, Journal of

Learning Disabilities, Learning Disabilities Research & Practice, Learning Disability Quarterly,

Remedial and Special Education, and The Journal of Special Education) for studies that met

their criteria: (a) Participants were identified as having a disability or at risk for one; (b) study

researched word problem interventions; (c) it included a baseline or pre/post treatment data; (d)

the study included enough quantitative information so effect sized could be calculated; and (e)

the study was in English in a peer reviewed journal or was an unpublished report, such as a

doctoral dissertation (Xin & Jitendra, 1999).

As a result of this review, Xin and Jitendra (1999) conducted a meta-analysis on 25

studies. Fourteen studies were group-design, and 12 studies were single case subject (one study

was counted in both categories). From the total number of studies, 15 studies were published and
Running head: WORD PROBLEM RESEARCH 5

peer reviewed, while the other 10 were unpublished dissertations. Their results indicate that there

is a positive correlation between the length of the intervention and the positive outcomes both for

group studies and single case studies(Xin & Jitendra, 1999). Also, the results indicated that

individual instruction is more beneficial than group instruction. It is important to remark that the

authors found that from the unpublished studies, only 20% reported assessment treatment

integrity vs. 67% of the published studies (Xin & Jitendra, 1999).

In 2012, Zhang and Xin published a meta-analysis continuing the previous work

previously done by Xin and Jitendra (1999) in their meta-analysis. In this meta-analysis, the

authors examine results of word problem interventions for students with learning disabilities

(LD) and students with learning problems (LP). The criteria for inclusion mirrored Xin’s and

Jitendra’s study (1999), focusing on 4 characteristics: the studies must have “(a) included K–12

students with LP in mathematics; (b) used an experimental, quasiexperimental, or single-subject

design; (c) examined effects of an instructional intervention; and (d) included word-problem

solving as one of the dependent measures”(Zhang & Xin, 2012). In the decade between meta-

analyses, several policy changes took place in the field of special education (i.e., No Child Left

Behind (NCLB), 2002; Individuals with Disabilities Act (IDEA), 2004; and the Response to

Intervention Model(RtI), 2004) (Zhang & Xin, 2012). Also, the field of mathematics went

through some reforms as well, such as emphasis on real word problem solving, and algebra

readiness in the elementary grades curriculum (Zhang & Xin, 2012). Part of this analysis was

conducted using the previously mentioned themes to guide the research questions.

The final sample was 29 group studies and 10 single case studies. Although the results

were disaggregated by theme (i.e., RtI, NCLB, IDEA 2004), their findings suggest that the use of

explicit instruction on the problem structure was very effective. This was different from the
Running head: WORD PROBLEM RESEARCH 6

previous meta-analysis since there had only been one study of this kind before 1999 (Zhang &

Xin, 2012). Zhang and Xin (2012) also noted that the effect sizes from standardized high stakes

tests, were significantly lower than those measured by tests developed specifically for the

intervention (Zhang & Xin, 2012). Finally, due to the implementation of Response to

Intervention, the study (Zhang & Xin, 2012) found that there was an increased number of studies

related to students at risk or with LP. Students are considered at risk when they score below the

25th percentile on standardized measures (Driver & Powell, 2017; Fuchs et al., 2008). This trend

was concurrent with the practical trends in education at the time (Zhang & Xin, 2012).

The remaining meta-analysis (Dennis et al., 2016), and review (Powell, 2011) are

included in this review as a reference of other work done in the field. In their meta-analysis,

Dennis et al., did not address type of word problem interventions for their review; and only

included group studies leaving out of their analysis single case design studies (Dennis et al.,

2016).

Powell (2012) presents a review of literature focusing only solving word problems with

schemas. Powell’s literature review (2011) looks at schema interventions (SI) in elementary

grades from 1996 to 2009. By only including SI studies in the elementary grades, her literature

review may have missed several studies that are relevant (Jitendra et al., 2009).

Population Shift

Word problem solving is still as relevant as it was 20 years ago for our most fragile

students (Driver & Powell, 2017; Fuchs et al., 2010; Jitendra, Dupuis, et al., 2013; Jitendra,

Rodriguez, et al., 2013; Kong & Orosco, 2015; Powell & Fuchs, 2015). Although the

composition of these populations has changed over the years mirroring the United States

demographic change(Ennis, Rios-Vargas, & Albert, 2011), the gap between students at risk for
Running head: WORD PROBLEM RESEARCH 7

MD or students with LD has remained constant(Driver & Powell, 2017; Jitendra, Dupuis, et al.,

2013; Kong & Orosco, 2015; Murphey, 2014). There has been a marked change in the research

done in the last eight years looking at struggling students through a cultural lens. (Driver &

Powell, 2017; Jitendra, Dupuis, et al., 2013; Orosco, 2014; Shumate, Campbell-Whatley, & Lo,

2012). This shift comes as a response to the continuing academic gap between white students and

their non-white peers (Driver & Powell, 2017; Kong & Orosco, 2015; Murphey, 2014) .

Research Questions

In the current meta-analysis, we extend the previous work accomplished by Xin and

Jitendra (1999), and Zhang and Xin (2012); and just like Shang and Xin (2012), we explore

changes in school climate that may influence the kind of studies that are taking place; as well as

the different trends in current research practices. The specific research questions (RQ) we are

looking to investigate are:

1. RQ1: What type of intervention studies (e.g., single case, group design, RCT) are being

done with students at risk for MD on the topic of word problem solving?

2. RQ2: Compared to interventions coded in previous meta-analyses, what kind of

interventions are being done with students at risk for MD on the topic of word problem

solving?

3. RQ3: Is there a relationship between the intervention effectiveness and the student’s cultural

background?

4. What are the similarities between the most effective interventions coded? (If using single

case 90% PND, if group control ES. > .61)

5. RQ5: Are there more studies on LD students or on students at risk for MD compared to the

meta-analysis done by Zhang and Xin (2012)?

Running head: WORD PROBLEM RESEARCH 8
Running head: WORD PROBLEM RESEARCH 9

Sources

Dennis, M. S., Sharp, E., Chovanes, J., Thomas, A., Burns, R. M., Custer, B., & Park, J. (2016).

A meta-analysis of empirical research on teaching students with mathematics learning

difficulties. Learning Disabilities Research & Practice, 31, 156–168.

https://doi.org/10.1111/ldrp.12107

Driver, M. K., & Powell, S. R. (2017). Culturally and linguistically responsive schema

intervention: Improving word problem solving for English language learners with

mathematics difficulty. Learning Disability Quarterly, 40, 41–53.

Ennis, S., Rios-Vargas, M., & Albert, N. (2011). The Hispanic population: 2010 (2010 Census

Brief No. C2010BR–04) (p. 16).

Fuchs, L. S., Seethaler, P. M., Powell, S. R., Fuchs, D., Hamlett, C. L., & Fletcher, J. M. (2008).

Effects of preventative tutoring on the mathematical problem solving of third-grade

students with math and reading difficulties. Exceptional Children, 74, 155–173.

Fuchs, L. S., Zumeta, R. O., Schumacher, R. F., Powell, S. R., Seethaler, P. M., Hamlett, C. L.,

& Fuchs, D. (2010). The effects of schema-broadening instruction on second graders’

word-problem performance and their ability to represent word problems with algebraic

equations: A randomized control study. Elementary School Journal, 110, 440–463.

Jitendra, A. K., Dupuis, D. N., Rodriguez, M. C., Zaslofsky, A. F., Slater, S., Cozine-Corroy, K.,

& Church, C. (2013). A randomized controlled trial of the impact of schema-based

instruction on mathematical outcomes for third-grade students with mathematics

difficulties. Elementary School Journal, 114, 252–276.

Jitendra, A. K., Rodriguez, M., Kanive, R., Huang, J.-P., Church, C., Corroy, K. A., &

Zaslofsky, A. (2013). Impact of small-group tutoring interventions on the mathematical

Running head: WORD PROBLEM RESEARCH 10

problem solving and achievement of third-grade students with mathematics difficulties.

Learning Disability Quarterly, 36, 21–35. https://doi.org/10.1177/0731948712457561

Jitendra, A. K., Star, J. R., Starosta, K., Leh, J. M., Sood, S., Caskie, G., … Mack, T. R. (2009).

Improving seventh grade students’ learning of ratio and proportion: The role of schema-

based instruction. Contemporary Educational Psychology, 34, 250–264.

https://doi.org/10.1016/j.cedpsych.2009.06.001

Kong, J. E., & Orosco, M. J. (2015). Word-problem-solving strategy for minority students at risk

for math difficulties. Learning Disability Quarterly, 39, 171–181.

https://doi.org/10.1177/0731948715607347

Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, Calif: Sage

Publications.

Murphey, D. (2014). The academic achievement of English language learners: Data for the U.S.

and each of the states. Child Trends.

NAEP - 2015 Mathematics & Reading - Mathematics - National achievement level results.

(2015). Retrieved October 30, 2017, from

https://www.nationsreportcard.gov/reading_math_2015/#mathematics/acl?grade=4

OECD. (2016). Country note, Key findings from PISA 2015 for the United States. Retrieved from

http://www.oecd.org/pisa/pisa-2015-United-States.pdf

Orosco, M. J. (2014). Word problem strategy for Latino English language learners at risk for

math disabilities. Learning Disability Quarterly, 37, 45–53.

https://doi.org/10.1177/0731948713504206
Running head: WORD PROBLEM RESEARCH 11

Peltier, C., & Vannest, K. J. (2016). Schema-based instruction and the mathematical problem-

solving performance of two students with emotional and behavioral disorders. Education

and Treatment of Children, 39,521-544.

Powell, S. R. (2011). Solving word problems using schemas: A review of the literature. Learning

Disabilities Research & Practice, 26, 94–108.

Powell, S. R., & Fuchs, L. S. (2015). Intensive intervention in mathematics. Learning

Disabilities Research & Practice (Wiley-Blackwell), 30, 182–192.

https://doi.org/10.1111/ldrp.12087

Shumate, L., Campbell-Whatley, G. D., & Lo, Y. (2012). Infusing Culturally Responsive

Instruction to improve mathematics performance of Latino rtudents with specific learning

disabilities. Exceptionality, 20, 39–57. https://doi.org/10.1080/09362835.2012.640905

Xin Y. P., & Jitendra, A. K. (1999). The effects of instruction in solving mathematical word

problems for students with learning problems: a meta-analysis. Journal of Special

Education, 32, 207.

Zhang, D., & Xin, Y. P. (2012). A follow-Up meta-analysis for word-problem-solving

Interventions for students with mathematics difficulties. Journal of Educational Research,

105, 303–318. https://doi.org/10.1080/00220671.2011.627397