12th International Congress on Mathematical Education

Program Name XX-YY-zz (pp. abcde-fghij)

8 July – 15 July, 2012, COEX, Seoul, Korea (This part is for LOC use only. Please do not change this part.)

PROBLEM SOLVING PROCESSES OF FIFTH GRADERS – AN

ANALYSIS OF PROBLEM SOLVING TYPES
Benjamin Rott
Leibniz University of Hanover
rott@idmp.uni-hannover.de

This paper reports on an exploratory study investigating (mathematical) problem solving processes
of fifth graders (ages 10 to 12) from German secondary schools. The videos of 110 processes of 45
different pupils working on 5 tasks were coded using an adapted version of the protocol analysis
framework from Schoenfeld (1985). The data shows a significant correlation between the pupils'
problem solving behavior (characterized by Schoenfeld's episodes) and their success (or failure).

Keywords: Mathematical Problem Solving; Process Analyses; Problem Solving Types

Please note that this article is a major update to Rott (2011) – so there might be some
repetitions regarding the “Design of the Study” and “Methodology” paragraphs (but not the
“Results” section).

BACKGROUND
Problem solving is important in every day life, when one is confronted with situations where
the solution path is not immediately obvious (cf. OECD 2003), as well as in mathematics,
because "what mathematics really consists of is problems and solutions." (Halmos 1980, p.
519) It is widely accepted, that solving problems is important for the learning of mathematics,
thus it is part of many school curricula, e.g. in the United States and in Germany (cf. NCTM
2000; KMK 2003).
Though the importance of problem solving is widely aknowledged, the term “problem
solving” has different meanings ranging from working on routine tasks to solving puzzling or
difficult situations (cf. Schoenfeld 1992, p. 10 ff.). I refer to the latter interpretation,
accounting a “problem” as a task for which no obvious solution method is available (cf.
Mayer & Wittrock 2006, p. 287). Problem solving is cognitive and, that is important to note,
personal (cf. ibid.). One person's problem can be a routine task for another person who knows
an algorithm or a solution method to resolve the situation. Therefore, research on problem
solving should examine processes instead of only tasks and/or their solutions.
Most frameworks regarding the problem solving process “attribute some relationship to
Pólya's problem-solving stages” (Fernandez et al. 1994, p. 196): (1) understanding the
problem, (2) devising a plan, (3) carrying out the plan, and (4) looking back (Pólya 1945).
More modern frameworks – like those of Fernandez et al. (1994), Mason et al. (1982), and
Schoenfeld (1985 ch. 4) – break with the linear nature of Pólya's steps, allowing cycles and
abcde
Rott

steps backwards, and they highlight the importance of metacognitive and self-regulatory
behavior; apart from that, they stay close to the original. In addition to his partition of the
problem solving process into phases, Pólya (1945) proposes assistance for problem solvers by
presenting questions and guidelines regarding heuristic and self-regulatory content.
At present, there is a lack of research regarding the problem solving abilities and processes of
younger children (cf. Heinze 2007, p. 15), thus we started the project presented below to
address (among others) the following research questions:
• How can problem solving and routine processes be distinguished from each other?
• Is there a correlation between pupil's behavior (their decision making) in problem
solving processes and their success in solving the problem?

DESIGN OF THE STUDY

From November 2008 till June 2010, the first four terms of our support and research program
MALU1 took place, which is the basis for my research. Each term, a new group of 10 – 16 (45
altogether) interested fifth graders (ages 10 to 12) from secondary schools in Hanover,
Northern Germany, was formed. These pupils came to our university once a week and the
sessions usually followed the following pattern: After some initial games and tasks, the
students worked in pairs on 1 – 3 mathematical problems for about 40 minutes and were
videotaped thereby. They eventually presented their results to the whole group.
The pupils worked on the problems without interruptions or hints from the observers, because
we wanted to study their uninfluenced problem solving attempts. We decided not to use an
interview or a think-aloud method, so as not to interrupt the students' mental processes. To get
an insight into their thoughts, we let the children work in pairs to interpret their
communication in addition to their actions.

Table 1: Overview of the selected tasks

name of the task mathematical area number of pupils working this task

Beverage Coasters Geometry 32

Marco’s Number Series Arithmetic / Number Theory 32

Squares on a Chessboard Combinatorics 19

Seven Gates Arithmetic 15

The “Super Car” Arithmetic / Algebra 12

sum 110

1
Mathematik AG an der Leibniz Universität means to Mathematics Working Group at Leibniz University.

Abcde+3 ICME-12, 2012

 Rott

Overall, we used more than 30 different tasks, selected to represent a wide range of
mathematical areas and to allow the use of different heuristics. We chose five of those tasks
(and all related processes) for the analyses that are presented here (see Table 1 and Appendix
1 for the task formulations).

METHODOLOGY
Product Coding: To determine the pupils' success in problem solving, their work results
were graded in four categories: (1) No access, when the pupils did not work on the task
meaningful. (2) Basic access, when they solved (parts of) the problem but the solution had
notable flaws. (3) Advanced access, when they solved the problem for the most part. And (4)
full access, when the pupils solved the task properly and presented appropriate reasons.
This grading system was customized for each task with examples for each category (see
Appendix 2 for examples). Then, all the products were rated independently by the author and
a research assistant. We agreed in almost all cases and discussed the few products with
different ratings, reaching consensus every time. Those discussions led to better defined
categories. It is important to note, that the two members of a pair could – and sometimes did –
gain diverse ratings, when their written results differed.
Process Coding: The pupils' behavior was coded using the framework for the analysis of
videotaped problem solving sessions presented by Schoenfeld (1985, ch. 9).2 He intended to
“identify major turning points in a solution. This is done by parsing a protocol into
macroscopic chunks called episodes […].” (ibid., p. 314) An episode is “a period of time
during which an individual or a problem-solving group is engaged in one large task [...] or a
closely related body of tasks in the service of the same goal [...].” (ibid., p. 292) Schoenfeld
(1992, p. 189) continues: “We found […] that the episodes fell rather naturally into one of six
categories:”
(1) Reading or rereading the problem.
(2) Analyzing the problem (in a coherent and structured way).
(3) Exploring aspects of the problem (in a much less structured way than in Analysis).
(4) Planning all or part of a solution.
(5) Implementing a plan.
(6) Verifying a solution.
We adopted this framework for our study with the following modifications: Firstly, we added
new categories of episodes, because our fifth graders – unlike the university students
Schoenfeld observed – showed plenty of non-task related behavior. For example, there were

2
Though this framework is older than 25 years, it was the most promising one of several frameworks I tried to apply to
our data. It also helps answering the question, whether fifth graders use Pólya-like steps in their problem solving
processes and whether they profit from it (cf. Rott 2011).

ICME-12, 2012 abcde+2

Rott

periods of time during which our pupils talked about TV series or recreational activities that
were coded as acts of digression. But these new types of episodes are not important for the
results presented in this paper, because I'll focus on the task-related episodes, which are (2) –
(6) of Schoenfeld's list.
Secondly, we operationalized Schoenfeld's episode descriptions, because in the beginning we
had some difficulties in coding reliably; especially figuring out the differences between
Analysis and Exploration was hard (as predicted in Schoenfeld 1992, p. 194). We noticed that
Schoenfeld’s (1985, ch. 9) empirical framework refers to his theoretical model of the problem
solving process (ibid., ch. 4) which is based on Pólya’s (1945) list of questions and
guidelines. By assuming an analogy between Schoenfeld's framework and Pólya's work, we
were able to sharpen the descriptions in our coding manual. Applying Pólya’s questions to the
problem solving processes helped us deciding whether to code Analysis or Exploration (see
Figure 1 for a summary and an excerpt of Pólya’s questions and advices).

Figure 1: Analogy between Schoenfeld’s episodes and Pólya’s steps

The coding of the videotapes was done independently by research assistants and the author.
We applied the “percentage of agreement” approach to compute the interrater-reliability as
described in the TIMSS 1999 video study (cf. Jacobs et al. 2003, p. 99 ff.) for randomly
chosen videos, gaining more than PA = 0.7 for the parsing into episodes and more than PA =
0.85 for the characterizing of the episode types. But, more importantly, every process was
coded by at least two raters. Whenever those codes did not coincide (which they did most of
the time), we attained agreement by recoding together (cf. Schoenfeld 1992, p. 194).
This coding was done separately for the children, even though they worked in pairs, because
it was not uncommon for them to work on different aspects of the tasks sitting next to each
other, thus gaining different episode-codes. To avoid an imbalanced weighting of the data, all
processes were counted independently, even if the two pupils of a pair worked together all the
time and got the same codes.

Abcde+3 ICME-12, 2012

 Rott

RESULTS
As stated above, I'm concentrating on the five task-related episode types Analysis,
Exploration, Planning, Implementation, and Verification. The pupils' behavior will be
characterized by the presence or absence of those types of episodes. Due to space limitations
further approaches have to be left out of this paper (see the “Conclusions”).
Routine vs. Problem Solving Processes
As stated above, it is impossible to attribute “routine” or “problem”3 to a task formulation,
because this characteristic depends on the solver and the related process. While extreme cases
might be easy to distinguish, it is not always easy to separate “routine” from “problem”
solving processes. Theoretical considerations, using the Schoenfeld coding as a tool, lead to
the following results: The coding of Exploration should be a sure sign of a “problem” solving
activity, because of heuristic usage and the absence of an algorithm to solve the task. In
contrast, processes with only Implementation or Planning-Implementation episodes are good
candidates for “routine” processes (cf. Schoenfeld 1985, p. 301). It becomes difficult, when
(planning-) implementing behavior is preceded by an Analysis and/or succeeded by a
Verification, because sometimes it is required to understand a task before one can remember a
suitable solution scheme and even solutions obtained by an algorithm should be checked and
verified. On the other hand, a very structured approach to solve a task with no known
algorithm could also lead to a coding of Analysis and (Planning-) Implementation without
any other episode types in the process. Therefore, there is no well-defined theoretical
assignment of problem solving types regarding the presence and absence of Schoenfeld
episodes to the process attributes “routine” or “problem”.
In the context of validating the process codes, we also discussed the performances of the
pupils. Did they show signs of knowing an appropriate algorithm or did they even mention to
know one? These interpretations of the empirical data confirm the theoretical thoughts
insofar, that every process with at least one Exploration episode was considered to be a
“problem” process. We assigned the “Super Car” task (see Appendix 1) with the reasonable
assumption, that every fifth grader knows a scheme to solve it. And indeed, all but one of the
12 pupils working this task showed (planning-) implementing behavior with no approach to
an Exploration and no need for an Analysis; two of those finished their work on this task with
a Verification. The interpretations of the other tasks (especially the “Number Series”
problem) unfortunately confirm, that there are a few videos with Analysis and
Planning-Implementation but without Exploration, that we graded as “problem” processes.
Thus, the empirical result after more than 100 coded processes is: Processes with an
Exploration can be considered as “problem” while processes with only Implementation or
Planning-Implementation can be regarded as “routine” without need to further look at the
regarded videos thoroughly. But processes with all other possible combinations of episodes

3
In the literature, the terms task and problem are often poorly distinguished. In this paragraph, to avoid confusion, the task
and process attributes “problem” and “routine” are labelled by quotation marks.

ICME-12, 2012 abcde+2

Rott

(without an Exploration) should be analyzed qualitatively in order to decide on the attribute

“routine” or “problem”.
There was an interesting observation analyzing the “Squares on a Chessboard” task: 10 out of
19 pupils didn't really understand the problem formulation and solved the “routine” task
“How many 1x1 squares are on a chessboard?” just by counting up to 64. These processes,
corresponding to the theoretical assumption, just contained Planning and Implementation
episodes.
In the following analysis, I focus only on the “problem” processes.
Wild Goose Chases
One of Schoenfeld's major results, obtained with his video analysis framework, was the
importance of metacognitive and self-regulatory activities in problem solving processes.
Problem solvers who missed out on such activities often engaged in a behavior that
Schoenfeld called “wild goose chase” and that he described this way:
“Approximately 60% of the protocols were of the type [...], where the students read the
problem, picked a solution direction (often with little analysis or rationalization), and then
pursued that approach until they ran out of time. In contrast, successful solution attempts
came in a variety of shapes and sizes – but they consistently contained a significant amount
of self-regulatory activity, which could clearly be seen as contributing to the problem
solvers' success.” (Schoenfeld 1992, p. 195)
Similar observations were true for our fifth graders; several of them, who did not show any
signs of structured actions or process evaluations, were not able to solve the tasks. For
example, some children working on the Beverage Coasters task started to measure side
lengths of the quadrilaterals and to compute the circumferences of those shapes, getting stuck
to their first idea without reflecting on their actions.
So, to test if these observations bear up a statistical check-up and to apply Schoenfeld’s result
to the MALU data, we had to operationalize the problem solving type “wild goose chase” as
Schoenfeld gave no real definition for it. A process is considered to be a “wild goose chase”,
if it consists of only Exploration or Analysis & Exploration episodes, whereas processes that
are not of this type mostly contain planning and/or verifying activities.
To check if this kind of behavior in the processes is interrelated with success or failure of the
related products, I used a chi-square test.4 The null hypothesis is “no correlation between the
problem solving type 'wild goose chase' and (no) success in the product”. Because of the
small data base, I had to subsume the product categories by twos, to “no & basic approach” as
well as “advanced and full access”.
The entries in Table 2 consist of the observed numbers, the expected numbers (calculated by
the marginal totals) are added in brackets. The entries in the main diagonal are apparently
above the expected values. The test (χ² = 20.095) shows a significant correlation (p < 0.01,
even Yates corrected) between the problem solvers' behavior and their success.

4
Because of the nominal character of the process categories, no (Pearson or Spearman) correlation could be calculated.

Abcde+3 ICME-12, 2012

 Rott
Table 2: Contingency table – process behavior and product success

Process / product categories no & basic access advanced & full access sum

wild goose chase 27 (15.6) 5 (16.4) 32

miscellaneous 16 (27.4) 40 (28.6) 56

sum 43 45 88

χ² = 25.378 p = 0.0000005 Yates-p = 0.000001

The analyses of individual tasks and their processes also show interesting results (see Rott
2011), but only the combination of multiple tasks leads to statistically reliable results. Please
note, that the “routine processes” of the “Super Car” and “Squares on a Chessboard” tasks
(see above) were excluded from this computation.

DISCUSSION AND CONCLUSIONS

The results show, that parsing videotapes of problem solving attempts into episodes with the
adapted version of Schoenfeld's protocol analysis framework can be helpful in analyzing
those processes. This framework can assist in deciding whether a task is a problem or a
routine task for it's solver by an analysis of the related processes. Such a distinction between
“routine” or “problem” can be helpful for researchers or for teachers in schools choosing the
right tasks to aid their students developing mathematical skills.
The chi-square test shows a significant interrelation between the behavior in a problem
solving process and its outcome: For “Wild goose chases”, that is picking a solution approach
without considering alternatives or evaluating the progress, will most likely lead to a failure
in a problem solving attempt. This result is in agreement with Schoenfeld's (1985; 1992)
findings, but was previously unknown for fifth graders. In the long run, this could mean that
being aware of problem solving structures and using Pólya-like steps (analyzing the task,
devising a plan, carrying it out and verifying the solution afterward) is helpful for children
working on a problem. This could imply training programs to enhance the problem solving
competencies of pupils.
In addition to the results presented in this paper, further analyses of our data have been and
will be carried out, regarding the order in which the episodes appear in and their duration as
well as the non-task related episodes.
I also intend to look for heuristic elements and elements of self-regulation in the processes of
our pupils; both seem to be important factors when it comes to understanding problem solving
behavior and both seem to be highly related to success or failure working on problems (cf.
Schoenfeld 1985).

ICME-12, 2012 abcde+2

Rott

References
Bruder, Regina (2003). Learning Methods and Techniques on How to Solve Problems. URL
(25.08.2011): http://www.math-learning.com/files/learnmethod.pdf
Fernandez, Maria L.; Hadaway, Nelda & Wilson, James W. (1994). Problem Solving:
Managing It All. In The Mathematics Teacher, Vol. 87, No. 3, p. 195 – 199.
Halmos, Paul R. (1980). The Heart Of Mathematics. In The American Mathematical
Monthly, Vol. 87, No. 7 (Aug. - Sep., 1980), p. 519 – 524.
Heinze, Aiso (2007). Problemlösen im mathematischen und außermathematischen Kontext.
Modelle und Unterrichtskonzepte aus kognitionstheoretischer Perspektive. In Journal für
Mathematikdidaktik (JMD), 28(1), p. 3 – 30.
Jacobs, Jennifer; Garnier, Helen; Gallimore, Ronald; Hollingsworth, Hilary; Givvin, Karen
B.; Rust, Keith et al. (2003). Third International Mathematics and Science Study 1999
Video Study Technical Report. Volume 1: Mathematics. Washington: National Center for
Education Statistics. Institute of Education Statistics, U. S. Department of Education.
KMK (2003). Kultusministerkonferenz (Ed.). Bildungsstandards im Fach Mathematik für
den mittleren Schulabschluss. Bonn: KMK.
Mason, John; Burton, Leone; & Stacey, Kaye (1982 / 2010). Thinking Mathematically.
Dorchester: Pearson Education Limited. Second Edition.
Mayer, Richard E. & Wittrock, Merlin C. (2006). Problem Solving. In Alexander, P. A. &
Winne, P. H. (Ed.): Handbook of Educational Psychology. Routledge, p. 287 – 304.
NCTM (2000). National Council of Teachers of Mathematics (Ed.). Principles and Standards
for School Mathematics. Reston: NCTM.
OECD (2003). Organisation for Economic Co-Operation and Development (Ed.). The PISA
2003 Framework – Mathematics, Reading, Science and Problem Solving Knowledge and
Skills. URL (12.12.2010): http://www.oecd.org/dataoecd/46/14/33694881.pdf
Pólya, George (1945). How to Solve it. Princeton, NJ: Princeton University Press.
Rott, Benjamin (2011). Problem Solving Processes of Fifth Graders: an Analysis. In Ubuz, B.
(Hrsg.): Proceedings of the 35th Conference of the International Group for the Psychology
of Mathematics Education, Vol. 4. Ankara, Turkey, p. 65 – 72.
Schoenfeld, Alan H. (1985). Mathematical Problem Solving. Orlando, Florida: Academic
Press, Inc.
Schoenfeld, Alan H. (1992). On Paradigms and Methods: What do you do when the ones you
know don't do what you want them to? Issues in the Analysis of data in the form of
videotapes. In The Journal of the learning of sciences, 2(2), p. 179–214

Abcde+3 ICME-12, 2012

 Rott

Appendix 1 – the tasks

Beverage Coasters
The two pictured squares depict coasters. They are placed so, that the corner of
one coaster lies in the center of the other.
Examine the size of the area covered by both coasters.
Idea: Schoenfeld (1985, p. 77)

Marco’s Number Series

Marco wants to arrange the numbers from 1 to 15 into the caskets so that the sum of every
adjoining pair is a square number:

For instance, if there are the numbers 10, 6, 3 in three consecutive caskets, the 6 adds up to a
square number with its left (10+6=16) and its right neighbor (6+3=9).
How could Marco fill-up his 15 caskets?
Source: Fürther Mathematikolympiade, 2005/06, 1. round (www.fuemo.de)

Squares on a Chessboard
Peter loves playing chess. He likes playing chess so much that he keeps thinking
about it even when he isn’t playing. Recently he asked himself how many
squares there are on a chessboard. Try to answer Peter’s question!
Idea: Mason, Burton, & Stacey (2010, p. 17)

Seven Gates
A man picks up apples. On his way into town he has to go through seven gates. There is a
guardian at each gate who claims half of his apples and one apple extra. In the end the man
has just one apple left. How many apples did he have first?
Source: Bruder (2003, p. 12)

The “Super Car”

The new 3 liter car is there! It uses only 3 liters of gas per 100 kilometers. The fuel tank of the
car holds 42 liters.
How far can you travel with one tankful?
Idea: German Mathematics schoolbook for grade 5

ICME-12, 2012 abcde+2

Rott

Appendix 2 – sample operationalization of the product categories

Beverage Coasters:
(1) No access – no (meaningful) work on the task; e.g. calculating the circumference of the
sought-after quadrilateral instead of determining the size of the area.
(2) Basic access – approximate calculation of the area (without relating it to the area of the
squares) or stating the conjecture "one fourth" for one or both special case/s without
considering more general positions of the squares.
(3) Advanced access – stating the conjecture "one fourth for every location of the two
squares" without an (correct) explanation for the general case.
(4) Full access – presenting correct reasons for the general conjecture; e.g. working with
rotational symmetry or congruent triangles.

Marco’s Number Series:

(1) No access – no (meaningful) work on the task; e.g. listing numbers randomly.
(2) Basic access – creating a list with at least some square numbers in adjoining caskets; e.g.
“1, 15, 2, 14, 3, 13, …”.
(3) Advanced access – presenting a list with numbers that add up to a square number with
every neighbor; this list doen’t need to contain all 15 numbers but should exceed the given
example (“10, 6, 3”) by at least two numbers.
(4) Full access – presenting the correct list of 15 numbers; ideally also noting that 8 and/or 9
have to be the starting / ending numbers.

Squares on a Chessboard:
(1) No access – no (meaningful) work on the task; e.g. just counting 64 (small) squares.
(2) Basic access – being aware that there are larger squares than 1x1; e.g. counting 85 squares
(the number of all squares that cover the whole chessboard without overlapping).
(3) Advanced access – presenting all sizes of possible squares, from 1x1 to 8x8, but
miscounting their number.
(4) Full access – presenting the correct number of squares: 204.

Abcde+3 ICME-12, 2012