Mathematics Education Research Journal 2002, Vol. 14, No.

1, 16-36

Assessment in Calculus in the Presence of Graphics

Calculators
Patricia A. Forster and Ute Mueller
Edith Cowan University

In this paper we explore the extent and nature of students’ calculator usage as
determined from examination scripts in the Western Australian Calculus Tertiary
Entrance Examination. Errors made and understanding called upon are discussed
for seven questions. The inquiry highlights that skills associated with graphical
interpretation need to be the subject of instruction, and that an awareness of the
differing cognitive demands of graphical interpretation is needed when setting
assessment items.

Graphics calculators have been assumed equipment in Western Australian

Tertiary Entrance Examinations (TEE) for mathematics, chemistry, and physics
since 1998. This paper reports our findings on the extent and nature of students’ use
of graphics calculators in the 1999 Calculus TEE. We discuss results for seven
questions from the 20-question paper. A major part of the inquiry is to consider the
types of mathematical understanding called upon in calculator-assisted solutions.
Implications of the examination outcomes for teaching and assessment are
discussed.
The significance of the paper is that it reports examination outcomes for a
large cohort in an upper-secondary context, where graphics calculators were
required. Others (e.g., Anderson, Bloom, Mueller, & Pedler, 1997; Brown &
Neilson, 2001; Jones & McCrae, 1996; Kemp, Kissane, & Bradley, 1997) discuss
directions for assessment with the technology and analyse assessment questions,
but few empirical studies focus on student performance in formal examinations.
Exceptions are the studies in tertiary settings by Berger (1998) and Boers and Jones
(1994). Other papers from the study of which this paper is part include a critical
scrutiny of the Calculus TEE questions for 1996-1999 (Mueller & Forster, 2000),
investigation of gender-related effects 1995-2000 (Forster & Mueller, in press) and
qualitative and quantitative analyses of students’ performance in the 1998
Calculus TEE (Forster & Mueller, 2001).

Interpretative Framework

Mathematical Understanding
Various types of mathematical understanding are distinguished in the
literature. In our analysis we rely on the categories operational understanding
and structural understanding (Sfard, 1991). Operational understanding entails
conceiving a mathematical notion to be the result of a sequence of “processes,
algorithms and actions” (p. 4). Structural understanding is indicated if a notion is
referred to as an object or “static structure” (p. 4). Typically, operational
understanding precedes structural understanding. However, development might
be in the reverse order, that is structural to operational, especially for geometry
Assessment in Calculus in the Presence of Graphics Calculators 17

(Forster & Taylor, 2000; Sfard, 1991).

Others classify understanding in similar ways. For example, Hiebert and
Carpenter (1992) use the terms procedural and conceptual. Nesher (1986)
differentiates between algorithmic performance and mathematical understand-
ing. Gray and Tall (1994) refer to understanding processes and understanding
concepts. In addition, Gray and Tall (1994) distinguish processes from procedures.
For them, a procedure “is a specific algorithm for implementing a process” (p.
117). Thus, processes such as addition have meaning even when they are not
enacted. We draw on this distinction in the analysis.
To elaborate the above definitions, we consider the example of limits.
Evidence of operational understanding of the limit lim f(x ) could be the
x →a

evaluation of a given function f for values close to a specified value of a and on

either side of a, using the table of values on a graphics calculator. The table of
values procedure results in a value for the limit, provided that limit exists.
Structural understanding is said to be present when the student recognises limits
or limiting values as having meaning separate from the processes that generate
them: general properties of the category limit can be investigated, as can “the
various relations between its representatives” (Sfard, 1991, p. 20). A next stage of
development could involve a new category, the derivative, emerging from the
limit concept. The derivative of a function at a point could be understood
operationally as the limit of the slopes of secant lines, and a graphics calculator
could be used to explore numerically and visually this concept. Over time, the
derivative (or derivative function) might be understood as a structure in its own
right and its properties discovered.
Characteristically, successful mathematics students can interpret
mathematical notation in terms of operations and structures, whichever is
appropriate to the task at hand, and they recognise that a mathematical object
can be represented in multiple ways (Gray & Tall, 1994). However, many students
do not proceed beyond operational understanding. They progress through the
curriculum with pseudostructural thinking (Sfard & Linchevski, 1994). So, they
learn and apply algorithms and, therefore, operate on mathematical objects, but
do not understand the nature of those objects. As a result, their ability to apply
their knowledge in unusual or unfamiliar contexts is limited.
There is empirical evidence that graphical approaches on a graphics
calculator can facilitate both types of understanding of the function concept (e.g.,
Hollar & Norwood, 1999), and the use of numerical calculation capabilities can
advance structural understanding. For instance, equations can be entered in the
solve facility on some calculators, values for all variables except one entered, and
the calculator will produce a value for the non-specified variable. The
expressions that are entered can be conceived in terms of the processes shown (for
example, addition) or, because the solution procedures are hidden, the input-
output tasks can foster understanding about the relationships between the
variables, that is, structural understanding (Forster & Taylor, 2000; Tall &
Thomas, 1991).
18 Forster & Mueller

Assessment, Understanding, and Graphics Calculators

Issues of assessment in the presence of graphics calculators include
accessibility of calculators. In Western Australia graphics calculators have been
required for mathematics tertiary entrance examinations since 1998. Non-
symbolic graphics calculators and the Hewlett Packard HP38G and HP39G with
limited symbolic capabilities are approved for examination purposes. In 1998 and
1999, students were asked to write the brand of their calculator on their
examination scripts and access was universal for this cohort (Alguire & Forster,
1999).
Other assessment issues are the types of understanding that should be tested,
and the scope for use of the calculators. Senk, Beckmann, and Thompson (1997)
conducted a major inquiry into test items used in 19 high-school mathematics
(precalculus) classrooms where graphics calculators and other computer
technologies were used for learning. They developed a classification scheme for
the characteristics of test items and found high emphasis on the ability to apply
algorithms and low calls on technology usage in the test items that they
collected. We modified the scheme to suit calculus (Mueller & Forster, 2000) and
used it to code the questions in the Calculus TEE for 1996-1999. Our analysis
showed that the percentages of part questions that tested algorithmic
competence were 70% in 1996, 80% in 1997, 67% in 1998 and 65% in 1999. The high
percentages reflected the findings of Senk et al. (1997), but changes were evident
in 1998-1999 upon the introduction of the calculators. In the curriculum component,
Applications of Calculus, there was a move towards interpretative questions
where graphics calculator usage was an option. Diagrams, including calculator-
generated graphs, played a greater role in Functions and Limits than previously,
and fewer marks were allocated to algebraic procedures than in 1996-1997.
Complex Numbers was the curriculum component most affected by the introduction
of calculators. Questions needed a greater number of steps to reach an answer,
called on more reasoning, and fewer questions relied on the application of an
algorithm.
Our inquiry (Mueller & Forster, 2000) showed also that there was less
opportunity for graphics calculator usage in the 1998-1999 Calculus papers than
in 1996-1997, had the technology been allowed. This difference is explained by
the presence in 1996-1997 of procedural questions that are trivialised by the
technology.

Examination Performance in Calculus

Boers and Jones (1994) found in the final examination of a first university
calculus course that “the calculator was underutilized by most students” (p. 491).
In particular, students showed a preference for analytic methods for limits, even
though graphical and numeric approaches on the calculator had been
emphasised in lectures. For a limit that was difficult to determine analytically,
those students who chose a calculator method made errors that indicated a naïve
interpretation of the relevant calculator graph, or data/formula entry mistakes.
When asked to graph a rational function, many students did not include critical
features on their hand-drawn graphs, which Boers and Jones (1994) explain as
Assessment in Calculus in the Presence of Graphics Calculators 19

failure to integrate algebraic information in the question with the graphical

display on the calculator. We observed similar under-utilisation of the
technology for limits and problems with graphing in the 1998 Calculus TEE
(Forster & Mueller, 2001). Graphing rational functions with their asymptotes and
point discontinuities seems a source of major difficulty. Ward (1997) also recorded
that students make errors in these areas.
Berger (1998), in the setting of first year university calculus, investigated two
possible effects of graphics calculator usage: effects on ability when working
with technology (due to human working memory being freed from calculation) and
effects on conceptual understanding that result from using it. The first effect was
identified, but students’ examination and test answers (completed without the
technology) indicated that calculator use over one year did not result in students
approaching their mathematics in significantly different ways or in gaining
deeper understanding of calculus concepts. Berger attributed these outcomes to the
technology being used during the calculus course only as an add-on tool to support
and verify traditional analytic methods.
Thus, sites of error in calculator-based solutions in calculus and constraining
effects of instruction with the technology have been identified. On the other
hand, passing calculation to the technology can be beneficial, and inquiries in
non-calculus settings have shown that if calculator approaches are a focus of
instruction then insightful and different understandings can result (e.g., Hollar &
Norwood, 1999; Schwarz & Hershkowitz, 1999).

Overview of the Research

Three research questions guided our inquiry into the 1999 Calculus TEE.
• How might students have utilised their graphics calculators in
answering examination questions and what errors did they make that
were associated with calculator usage?
• What types of understanding are called upon in graphics-calculator
assisted solutions to the examination questions?
• What are the distributions and mean scores for students using graphics
calculator approaches and students using traditional methods on the
examination questions where graphics calculator use was viable?
Accordingly, prior to the examination, we selected for inquiry seven
examination questions that were either graphics calculator active, where use of
the tool is necessary or greatly simplifies the solution, or neutral, where use of
the tool is a viable option (Senk et al., 1997). We collected four types of data:
interviews with students; marks for all candidates; details of students’ answers
recorded by markers; and our own observations of students’ written answers, made
in our roles as examiners and markers.
Ten students were interviewed in the three days following the examination.
The purpose of the interviews was to ascertain how the students had answered
the examination questions, specifically, how they had used their graphics
calculators. Three students from each of two schools and four from another were
interviewed. The students were selected by their teachers on the basis of being
20 Forster & Mueller

communicative. Their school assessment grades ranged from A to C (D is the

lowest grade). The examination paper was the focus of discussion in the
interviews. The interview data were used in answering the first and second
research questions.
The Curriculum Council supplied us with the marks for each candidate for
each examination question, and with summary statistics, including the
coefficients of correlation between the total and each question. In addition, marks
were collected and made available to us for the two part questions that we
anticipated would attract the widest calculator usage (Questions 6d and 13c).
This data contributed to answering the third research question.
The Curriculum Council also supported us in obtaining from markers the part
marks for seven questions, and markers’ opinions on the nature of students’
methods (traditional or calculator based) that they recorded by ticking columns
on a proforma. Graphics calculator usage was optional for the seven questions.
The data was collected for 20% of the scripts from the first marking round only
(scripts are marked twice) and participation in the collection was voluntary.
Nine out of 24 markers volunteered to be involved. All markers were experienced
teachers and the result was data for 195 scripts, from a total of 1937 scripts. This
data was used in answering the third research question.
The fourth source of data was our own observations, which we recorded while
marking 240 scripts. We made notes and collected data on (a) the seven questions
selected for the research so that we had information on these for 435 candidates
in all, and (b) on an additional question, that emerged as interesting during the
marking. Graphics calculator usage turned out to be difficult to discern for two of
the seven questions and we did not pursue analysis of the outcomes on them. In
summary, in the results section we discuss one question for which we had marks
for the whole population (Question 6d, N = 1937) and five questions for which we
had qualitative data and part marks (Questions 7, 12, 13, 14 and 20; n = 435), and
Question 8 (n = 240).
Our method was to conduct a qualitative analysis of students’ interview data
and our own notes. Statistical analysis of the sample data followed, to determine
how many students adopted traditional methods and how many seemed to use
calculator approaches. Limitations were that the scripts from which data were
collected were not randomly selected and determination of the methods students
used relied on subjective judgement. However, we ascertained that our sample (n =
435) is representative for the population via a chi-squared goodness of fit test.
This test showed that the total raw examination scores of the sample had the
same distribution as the population, χ 2 (13, N = 435) = 8.56. We present a summary
of students’ marks and a description of the nature of their responses for each
question, and an analysis of types of understanding called upon, for graphical
solutions in particular. Mean scores for the population are based on numbers of
students who attempted the questions. For the sample data, means are calculated
for students whose methods were recorded. Some students’ methods were not
recorded as being calculator-based or traditional because they were difficult to
discern, and some students did not attempt all questions, resulting in discrepancies
between the sample size and the number of students accounted for in the summary
tables.
Assessment in Calculus in the Presence of Graphics Calculators 21

Results and Discussion

Graphics Calculator Evaluation of a Definite Integral

π /6
∫ 1 − 2 sin x dx .
2
Question 6(d): Find the integral
0
Solution: 0.47 (2 d.p.) [3 marks]
The integral in Question 6d cannot be obtained analytically so students
needed to use their graphics calculators. Table 1 is a summary of students’ scores.
Results are included for Questions 6a-c, which required evaluation of indefinite
integrals and where graphics calculator use was not possible or not practical,
depending on the model of calculator. The percentage mean scores are the
percentage of the total marks available for the questions (e.g., 52% = 1.56 out of
3).

Table 1
Evaluation of Indefinite Integrals (Question 6a-c) and a Definite Integral
(Question 6d)

6a-c 6d
No. of students attempting the questions (N=1937) 1892 1774
Mean score 6.65 (60%) 1.56 (52%)
No. who scored zero 29 814

Altogether, nearly 50% of candidates did not attempt Question 6(d), or scored
zero for it. Explanations that emerged in the interviews were that students:
• did not recognise the need to use their calculators;
• did not know how to evaluate the integral on their calculator, which
included not knowing the syntax; and/or
• did not have time to return to the question after missing it out or
abandoning it.
Some scripts showed answers that were consistent with calculators set in
degree mode rather than the required radian mode. In summary, the results
highlight the importance of appreciating and understanding calculator
functionality, and show that half the candidature did not display the necessary
knowledge.

Identification of Maximum Stationary Point and Evaluation of Area

3cosx
Question 7: Suppose that f( x ) = , −π ≤ x≤ π .
2 + sin x
(a) Determine exactly the two zeroes r 1 and r 2 of f.
(b) Calculate the exact co-ordinates of the maximum stationary point.
(c) Determine the area of the region above the x-axis bounded by the x axis and the
graph of f.
22 Forster & Mueller

Solution:
(a) x = ± π /2 [2 marks]
(b) Max stationary point = ( −π / 6, 3 ) [5 marks]
π/ 2
(c) ∫ −π /2 3 cos x /(2 + sin x) dx = 3 ⋅ 30 (2 d.p.) [3 marks]

For Questions 7b and c markers were asked to indicate traditional method

(working shown) or calculator approach (the answer only), together with part
marks, and whether exact values were given in 7b. The results are summarised in
Table 2. Data were not gathered for 7a as the zeroes of the cosine function in the
interval [–π, π] would be subject to recall by many students, therefore, calculator
usage would be impossible to discern on the basis of no working.

Table 2
Determining a Maximum Turning Point (Question 7b) and Area (Question 7c)

7b 7c
Traditional Graphics Traditional Graphics
calculator calculator
Number of students 311 99 73 300
(n = 435)
Mean score 4.1(81%) 3.8(76%) 1.9(63%) 2.5(84%)
Number who scored zero 8 5 14 17
% who gave exact values 89% 74% - -
Note. The χ2 –value for goodness of fit between sample and population data is χ2 (8, N = 435) = 6.45.

For 7b, about three-quarters of scripts in the sample showed an analytic

(traditional) method. Interviewed students said a reason for the choice was the
requirement of exact values. Some said that they combined the analytic method
with graphics calculator use for determination of the co-ordinates of the
stationary point, or used the calculator to check the coordinates. Calculator use
involved:
• entering into the calculator the function, together with the derivative
command, setting the expression equal to zero, and solving the equation in
a numerical solve facility; and
• graphing the function (which could be done from the start) then using the
built-in capabilities of the calculator to obtain the co-ordinates of the
turning point (see Figure 1).
Derivatives that were not simplified in written answers and sketches of the
function indicated also a combination of methods.
An error with answers obtained analytically was to locate the maximum
point at x = π/6 instead of at x = –π/6. However, the percentage of students who
failed to convert decimal approximations to exact values was greater for the
group who relied solely on a calculator solution, which explains the lower mean
mark. The interviewed students who did the conversion said they recognised the
Assessment in Calculus in the Presence of Graphics Calculators 23

decimal was –π/6 or converted it using the routine of dividing by π to obtain the
1/6; and they recognised the second coordinate was √3.

Figure 1. Graphics calculator display of f(x) = (3 cos x)/(2 + sin x).

In 7c, a high majority of students in the sample used numerical integration on

their calculators, which resulted in the greater mean score. An error was to
integrate f(x) over the interval [-π, π] instead of [–π/2, π/2]. The use of f(x) is
encouraged by the calculators for it simplifies calculation of areas between the
curve and x axis. In isolated cases, answers were consistent with the calculator set
to degrees instead of radians.
From a theoretical viewpoint, the graph encourages and calls on structural
thinking: the structure of a function is displayed as an integrated whole (Sfard,
1991). Furthermore, the correct stationary point on the graph is easily discerned,
because of the literal association that the relative maximum function value is at
the highest visible point in the given domain. Thus, use of a graph can help
avoid errors in locating the maximum point but necessitates converting the
decimal coordinates. The graph could also assist choosing the limits of
integration for 7c, and was used for this purpose by most of the interviewed
students. One of the interviewed students said that she used the area function on
the plot screen of the HP38G, but she found it impossible to place the cursor on the
exact limits of integration. She had chosen an inappropriate scale. The benefit of
having the structure of the area visible on the graph was compromised by her
failure to enact a correct procedure for setting up a trigonometric scale.
Students performed relatively well on the question as whole, judging by the
mean score of 7.76 out of 10 (n = 1842, n = 1937).

Evaluation of an Integral for Volume

Question 8: A wood-turner turns a piece of wood to make a curved bowl with a
solid cylindrical base. The base has a radius of 1 unit and thickness 0.1 unit. The
curved part of the bowl can be described mathematically as the solid obtained by
rotating about the x-axis the region in the first quadrant bounded by the y-axis, y =
cos x and y = sin x. For both parts of the bowl one unit represents 9 cm.
(a) Write an integral representing the volume of the curved part of the bowl.
(b) What is the volume of wood in the finished bowl?

Solution:
π /4
(a) π ∫ 0 (cos 2 x − sin 2 x) dx [4 marks]

(b) Volume of the curved part = π/2 or 1.57 (2 d.p.)

24 Forster & Mueller

Total volume = (π/2 + πx1 2 x0.1) x 9 3 cm3 = 1374 cm3 [4 marks]

For Question 8, we recorded the nature of students’ answers for the 240 scripts
that we marked and the results are summarised in Table 3. Incorrect answers for
the integral with a calculator evaluation (inferred from no working shown)
caught our attention and led to the data collection.

Table 3
Evaluation of an Integral for Volume (Question 8)

Traditional Graphics calculator

Part (a) Part (a) Part (a) Part (a)
Correct Incorrect Correct Incorrect
Number of students 23 31 43 65
(n = 240)
Number with correct 20 14 31 38
evaluation

The proportion of the graphics calculator group with Part (a) correct who
evaluated the integral correctly was 72%. Further, of the students in the sample
who chose to use their graphics calculator to evaluate the integral that they had
obtained, only 64% succeeded in obtaining a value consistent with their integral.
Entry and/or syntax errors are salient.
The mean mark for the question as a whole for the cohort was low (3.78 out of
8, n = 1728).

Solution of a Non-linear Equation

Question 12: The difference between high and low water tide levels in one of the
ports along the northwest coast of Western Australia is 6 metres. At the entrance of
2
the port from the ocean, the level x of water is given by d x = − 1 x where t is
dt 2 4
measured in hours from the time of the low tide.
(a) Calculate the amplitude and the period of the motion of the tides.
(b) Write x as a function of t.
(c) A ship can enter or leave the port as long as there is at least one metre of water
above the low tide mark. If the low tide takes place at 10:15 am, what is the latest
time a ship can leave the port on that day?

Solution:
(a) amplitude = 3m, period = 4π hours [3 marks]
(b) x = –3cos(t/2) is one possible answer. [3 marks]
(c) –3cos(t/2) = –2, t = 10·8842, Time = 10:15 am + 653·05mins = 9:08 pm [5 marks]
Sample data were generated for 12c and are summarised in Table 4.
Calculator use was inferred from no working or the provision of a graph only.
Except for checking in 12b that would be impossible to infer, the other part
questions were calculator inactive, so were not considered in the data collection.
Assessment in Calculus in the Presence of Graphics Calculators 25

Table 4
Solving the Simple Harmonic Motion Equation (Question 12c)

Traditional Graphics calculator

Number of students (n = 435) 151 123
Mean score 1.9(64%) 2.2(75%)
Number who scored zero 28 21
Note. The χ2 –value for goodness of fit between population and sample data is χ2 (12, N=435) = 5.90 .

Students showed a slight preference for a traditional, inverse cosine

approach for 12c. An error was to give the first positive value for cos-1(t/2) = 2/3,
which did not allow for the tide context. Calculator alternatives, as given in the
interviews, were to determine the time from the points of intersection of the
graphs f(t) = –3cos(t/2) and g(t) = –2, and to solve the equation –3cos(t/2) = –2 in a
solve facility. An error in calculator-based solutions was also to give the first
positive value but it was less prevalent, leading to the higher mean score.
From an analytical viewpoint, use of the graph facilitates structural
interpretation of the cosine function: the periodic structure is displayed and
resembles the periodic nature of the tide. Multiple intersection points are visible,
which encourages consideration of more than one solution. Furthermore,
interpretation that accommodates the context requires reading across the graph
and taking in the detail. The depth of water at selected points needs to be judged,
and that the ship leaves on the falling tide established: a dynamic
interpretation that, in Sfard’s (1991) terms, relies on operational understanding.
Indeed, interpretation of the graph requires switching between structural and
operational interpretations.
Restricting the domain to 10:15 am to 12 midnight on the same day, that is, to
[0, 13.75], simplifies the interpretation (see Figure 2). Some students provided a
graph with the domain [0, 13.75] in their written solutions, and producing it first
on the calculator obviates prior consideration of the period/number of cycles
when drawing the graph, which is necessary when producing graphs by hand. If,
as interviewed students mentioned, the time of departure is obtained with the
point-of-intersection function on the technology, then calculation to obtain the
second point of intersection from the first is not necessary. Hence, a calculator-
based graphical method, where structures are visible and procedures are
automated, assists in several ways the determination of the correct (second
positive) solution. Use of a graph explains students’ greater success with a
calculator solution more than use of the solve facility, which requires students to
enter an approximate time of departure.
26 Forster & Mueller

Figure 2. Graph of f(t) = –3cos(0.5t) representing the tide, and g(t) = –2.

The mean score on Question 13 overall was 5.72 out of 11, (n=1811). The
question was a good discriminator for examination purposes, with a correlation
coefficient of 0.71 between students’ scores on the question and their total
examination scores. That is, students who did well on the question tended to do
well in the examination and the sample data indicate that choosing to use a
graphics calculator as first option might have contributed to their success.

Sketching a Graph

Question 13: If f (x) = x + 3x − 10 ,

2

x +x −6
2

(a) state the domain of f,

(b) evaluate lim f (x ) ,
x→ 2
(c) sketch the graph of f showing the intercepts, asymptotes and any other
distinguishing features.
f( x)

Solution:
(a) all reals ≠ -3 or 2 [2 marks]
(b) 7/5 [2 marks]
5/3
(c) Graph [6 marks] ( 2, 7/5 )
x
-5 -3
1

Markers were asked to indicate whether students used traditional methods or

appeared to use their graphics calculator (as indicated by an answer only for
question 13b and a graph only for 13c). They were also asked to indicate if
students’ graphs for 13c included the vertical asymptote, horizontal asymptote
and point discontinuity. A summary of results is given in Tables 5 and 6. Data
were not collected on 13a. While a graphics calculator could have been used to
find the roots of a quadratic function, it would have been difficult to discern such
usage.
Assessment in Calculus in the Presence of Graphics Calculators 27

Table 5
Finding a Limit (Question 13b) and Sketching a Graph (Question 13c)

13b 13c
Traditional Graphics Traditional Graphics
calculator calculator
No. of students (n = 435) 223 160 63 363
Mean score 1.8(90%) 1.9(93%) 4.4(73%) 4.5(74%)
Number who scored zero 19 10 1 1
Note. The χ2 –value for goodness of fit between sample and population data for questions 13a+13b is 1.12
(4 df) and for question 13c is χ2 = 2.11 (5 df)

Table 6
Features on the Graph of a Rational Function (Question 13c)

Number who included the feature Horizontal Vertical Point

Asympt. Asympt. Discontinuity
Traditional approach (n=63) 50 (80%) 61(97%) 25(40%)
Calculator approach (n=363) 272(75%) 352(97%) 183(50%)

Question 13b was answered correctly by a large proportion of students,

yielding the high mean scores. The traditional approach for the limit was
usually factorisation but a few students opted for L’Hôpital’s rule, which is not
specified in the Calculus TEE syllabus. Calculator methods used by the
interviewed students for determining the limit (or for checking their analytic
solutions) were entering the function then:
• accessing function values near 2 in the table of values,
• substituting values in run mode (Casio 9850+),
• graphing the function, running the cursor near x = 2 and reading the
function values on the graph screen.
For 13c, a high proportion of students in the sample showed no working so
appeared to rely on their graphics calculators to produce the graph. Irrespective
of method, fewer students included the horizontal asymptote than included the
vertical asymptote, and the point discontinuity was the feature that was most
commonly omitted. The results associated with calculator use reflect the findings
of Boers and Jones (1994), Ward (1997), and the outcomes for the 1998 Calculus TEE
(Forster & Mueller, 2001), although a greater percentage of students included the
horizontal asymptote in the 1999 TEE than in 1998 (≈50% of scripts sighted in
1998 showed the asymptote).
Strategies used by interviewed students to assist interpretation of the
calculator graph were:
• discerning the discontinuities from the factorised form of the function
obtained in 13a;
28 Forster & Mueller

• being alert to the vertical asymptote and point discontinuity from

answers to 13a and b;
• mentally substituting ±x into the function;
• using the table of values—with x = 0 to evaluate the y intercept, with
x = 2 to check the discontinuity, and with large positive and negative
numbers to locate the horizontal asymptote;
• on the graph, using the root finding facility, observing the break in the
graph at x = 2, and using the trace facility to establish that the graph
approached y = 1.
Thus, interpretation of the graph was assisted by determination of the
structure from the algebraic form of the function; and by mental calculation,
dynamic reading of the graph and procedures in the table of values. Some of the
interviewed students used one of these approaches to check the results of another.
Bringing to bear the structural and operational understandings mediated
favourably interpretation of the graph and omission to do so explains the non-
identification of key features. The discretisation of the calculator graph means
features are not always visible and necessitates that students predict them from
the function expression. Other errors attributable to discretisation and to blind
copying of a calculator graph were stopping the branches of the curve without
indicating asymptotic behaviour, and joining the branches of the curve. In copying
the graph it is also important to clearly identify the features that have been
noticed. Some students drew the branches of the curve as approximately
horizontal but did not clearly identify the y = 1 asymptote and were penalised.
The mean mark for the population for parts 13a-b was 3.36 out of 4, n = 1981,
and for 13c was 4.47 out of 6, n = 1907.

Generation and Interpretation of a Graph

Question 14: A particle is moving along a straight line that2 runs in an east-west
t +1
direction. Its position function s(t) at time t is given by s(t) = 4 .
t +1
(a) Determine the velocity function of the particle.
(b) The particle is moving in an easterly direction when the velocity is positive. Use
the graph of the velocity function to decide when the particle is moving in a westerly
direction.
(c) Use the graph of the velocity function to determine the maximum speed of the
particle and when it is attained.
(d) Calculate the position of the partic le at the time when the maximum speed is
attained.

Solution:
(a) s = t + 1 , v = (t +1)2t − (t +1)4t = 2t − 2t − 4t [3 marks]
2 4 2 3 5 3

t +1
4 4
(t + 1)
2 4
(t + 1)
2

(b) t > 0.64 [2 marks]

(c) t = 1.095 (3 d.p.), speed = 1.045 (3 d.p.) [2 marks]
(d) s = 0.90 (2 d.p.) [1 mark]
Part marks for questions 14a-c were recorded for the sample of students (see
Table 7) and whether students drew a graph in support of their answers.
Assessment in Calculus in the Presence of Graphics Calculators 29

Table 7
Finding a Derivative (Question 14a) and Interpreting a Graph (Question 14b-c)

14a 14b 14c

Students attempting the question (n = 435) 419 380 368
Mean score 2.6 (88%) 1.3(63%) 1.2(59%)
Number of students scoring zero 16 102 122
Note. The χ2 –value for goodness of fit between sample and population data is χ2 (7, N = 435) = 1.41 .

Students were more successful in question 14a, which involved differentiation

(without technology), than in 14b and 14c. Written answers for 14b and c lacked
algebraic working, suggesting that students drew the required graph using their
calculators (see Figure 3). The graph would have been difficult to establish
analytically, so that graphical approaches on a calculator were effectively
forced. Mistakes in the calculation of the derivative were followed through in
the marking, so they did not impact greatly on the results for 14b and 14c. The
high number of students in the sample scoring zero and the low average marks
associated with these part questions can therefore be attributed largely to
difficulties with interpreting the questions and with obtaining an adequate
graphical display and interpreting it.

Figure 3. Graph of the velocity function.

Student errors in 14b were giving the answer as:

• t < 0.64, which is consistent with knowing that the direction of travel
changes at v = 0, but incorrectly assumes that westerly travel involves
moving to the left on the graph;
• 0.64 < t < 1, which correctly recognises that the section below the
horizontal axis represents travel in the westerly direction but incorrectly
assumes that the relative minimum turning point is where the particle
turns round;
• t = 0.64, which implies a misinterpretation of the question.
Recognising that the root of the velocity function was needed was less
problematic than determining the appropriate interval. Interviewed students
indicated that they used built-in capabilities of their calculators to determine
the root.
In relation to the maximum speed for 14c errors were:
30 Forster & Mueller

• to give maximum velocity (the highest point on the graph) for maximum
speed,
• to give the speed at the minimum point but write it incorrectly as a
negative quantity, and
• to omit the time.
Interviewed students again indicated that they used built-in capabilities on
their calculators to determine the time and speed.
From a theoretical viewpoint, the graph allows the functional relationship
to be seen as a whole, but its structure does not bear any likeness to the motion it
signifies. Therefore, literal interpretation of the structure is inappropriate (that
west is left, and the particle turns at the turning point), yet, such interpretation
explains the mis-identification of the interval in 14b. Checking the negativity of
velocity function values at a few positions, in other words, exploring the detail of
the graph, which calls on operational understanding, could increase the
likelihood of a correct choice of the interval. Students that we interviewed
typically just “knew” movement was west below the x axis. Literal interpretation
also explains the answer of maximum velocity in 14c.
The mean score for the question for the entire population was 5.50 out of 8, n =
1891.

Testing for Continuity and Graphical Solution of a Non-linear

Equation
2
Question 20: The function f is defined by f (t) = ∫0 sin(tx) dx .
1 − cos(2t)
 for t ≠ 0
(a) Show that f(t) =  t
0 fort = 0

(b) Determine lim f(t) , justifying your answer.
t →0

(c) Is f continuous at t = 0? Justify your answer.

(d) Sketch the graph of f.
(e) What is the least integer value of K such that all solutions of the equation
f(t) = 0.25 are contained in the interval [0, K]?
(f) How many values of t are there wit h f(t)=0.25?

Solutions: For economy of space, solutions for (a) and (b) are omitted. For (b) the
limit is zero and methods students adopted are described below.
(c) f(t) = 0 = lim f(t) ∴ f(t) is continuous a t = 0 [2 marks]
t →0

(d) Graph [4 marks]

(e) K = 8 [1 mark]
(f) 6 solutions [2 marks] f(t )
2
(1·17, 1·45)
1

t
π
-1

-2
Assessment in Calculus in the Presence of Graphics Calculators 31

Markers recorded part marks for question 20b and the method (traditional,
table of values, or a graph); and part marks for 20d and 20f, which required the
use of the calculator due to the non-routine nature of the function (see Table 8).
Few students conclusively established the value of the piecewise-defined
function at x = 0 and errors from 20a and 20b flowed into 20c. Of the students whose
method was recorded for the limit in 20b, about two thirds drew on standard
approaches for limits including application of L’Hôpital’s rule and the
properties of trigonometric limits. Some interviewed students noted that the
value of the limit was a known fact. The alternatives were a table of values or
graph, which potentially were both graphics calculator assisted. Mean scores
were noticeably higher for those students who used these approaches. However,
errors were:
• some students did not provide sufficiently many values in their table to
adequately establish the limiting behaviour, an omission also noted for
the 1998 Calculus TEE (Forster & Mueller, 2001),
• students correctly stated the limit of the function as t approached zero
from above and below zero, but incorrectly included f(0) = 0 in their
justification,
• isolated instances of graphs with asymptotic behaviour at t = 0, which
was consistent with students keying the function 1− cos( 2t)/ t into their
calculators.

Table 8
Evaluating a Limit (20b), Sketching a Graph (20d) and Interpreting a Graph (20f)

20b 20d 20f

Traditional Table Graph
No. of students (n = 435) 182 58 35 312 271
Mean score 1.7(58%) 2.5(83%) 2.1(70%) 2.3(58%) 0.8(40%)
Note. The χ2 –value for goodness of fit between sample and population data is χ2 (16, N = 435) = 13.15 .

The earlier discussion on limits in the interpretative framework is pertinent

here. The correct application of traditional symbolic procedures does not
necessarily indicate operational understanding of the limit concept, that is,
understanding of limiting behaviour. Too few values in a table implies, at best,
weak operational understanding, as does the inclusion of f (0) = 0 with the
graphical approach: a sufficient procedure with the graph is gauging function
values on either side of t = 0. Besides being unnecessary, stating f (0) = 0 indicates
misinterpretation of the structure of the function. The function (1 − cos 2t)/ t has a
point discontinuity at t = 0. This is another instance where predicting features of
the graph from the algebraic expression is desirable because the discontinuity
would not necessarily be evident on calculator graphs. Misconstruing that f(0)
exists affected answers for 20c.
In 20d, calculator graphing was effectively forced due to the nature of the
32 Forster & Mueller

function. Graphs typically lacked coordinates of a point to set the scales and
lacked scales on the axes, which is attributable to there being no scales on some
calculator graphs. In 20e, the interviewed students deduced the interval by:
• using the extremum function on the graph to find the relative maxima
values,
• tracing along the curve with the cursor and relying on the function value
outputs,
• graphing f(x) = 0.25 and using the point of intersection capability to check
the allowable t values.
A widespread error in 20e in written solutions was failure to round to an
integer. Only 58% of candidates in the sample answered 20f and less than half of
them gave 6 for the answer. The errors and omissions explain the low mean scores
for questions 20d and 20f. The mean result for the question as a whole was a low
7.05 out 16, n = 1749. Question 20 was the last question in the examination paper
and, judging by interview comments, many students were pressed for time.
In summary, we have described how students might have utilised their
graphics calculators in answering seven examination questions, the associated
mean scores, errors that were made, and some aspects of the understanding called
upon. We finish this section of the paper with an overview of the apparent
extent of calculator use and success associated with the use. For the evaluation of
an integral where calculator-use was necessary, results indicate that a significant
number of students did not recognise to use the tool or did not use it correctly
(Question 6). Only 45% of all candidates scored full marks. Where calculator-use
was optional for integrals, the high majority of students in the sample chose it
and, in Question 7, 70% of them scored full marks. In the other (Question 8), 64%
correctly used the calculator to evaluate the integral that they had. The
complexity of the expressions in Questions 6 and 8 is relevant to the lower results.
The mean marks achieved with hand and calculator evaluations do not indicate
a clear pattern of inferior or superior performance with either method (see Tables
2 and 3).
Use of graphs, tables of values, and the solve function on the calculators is
often not distinguishable in written answers, so results for them are considered
together and summarised in Table 9. Use of the calculators was forced in questions
14b and c, and 20d and f because of the complex functions.
Where the use of calculators was optional, the majority of students in the
sample appeared to rely on them when a graph was required (Question 13).
Otherwise, less than half the written answers were consistent with a calculator
method, although students might have used the technology for checking. The
particularly small number of students choosing the calculator in Question 7 was
attributable to exact values being stipulated, and they scored, on average, lower
than students who presented analytic working.
Assessment in Calculus in the Presence of Graphics Calculators 33

Table 9
Results by Question when Choosing Graphs, Table of Values or the Solve Facility

Application No. of students (n = 435) Mean Score %

Graphics Traditional Graphics Traditional
calculator calculator
7b Obtaining the co-ordinates of a 99 311 76 81
maximum stationary point.
12c Solving a trigonometric equation 123 151 75 64
involving the tide.
13b Evaluating a limit. 160 223 93 90

13c Graphing a rational function. 363 63 74 73

14b Solving for times when a particle 380 n.a. 63 n.a.

was travelling West.
14c Identifying maximum speed. 368 n.a. 59 n.a.

20b Evaluating a limit. 93 182 78 58

20d Graphing a trigonometric 312 n.a. 58 n.a.

function.
20f Finding the no. of solutions to 271 n.a. 40 n.a.
simultaneous equations.

The difference in percentage scores is highest in favour of calculator users on

the questions that asked for a departure time (Question 12c) and a trigonometric
limit (Question 20b). The difference in marks was not substantial for Question 12c
(0.3 marks out of 5), but contributed to Question 12 as a whole being one of the best
discriminators on the examination. The tabular approach to the limit in Question
20b resulted in the greatest advantage (see Table 8) and students using the table
scored on average 0.8 marks out of 3 more than the traditional group.
The lowest mean scores recorded for calculator users were on Questions 20d
and f, but students’ running out of time might have affected performance on them
more adversely than on other questions. The relatively low scores on Questions
14b and c are explained by inappropriate literal interpretation of the graph.
However, misinterpretation of graphs of derivative functions is known to be
widespread (e.g., Hale, 2000) and is not limited to calculator-produced graphs.

Conclusion
In this paper we have described students’ performance in relation to graphics
calculator use on selected questions from the 1999 Calculus TEE. Based on the
methods that the interviewed students articulated, we have identified how
graphs can assist problem-solving because they display the structure of functions.
For example, a periodic structure (Question 12) and the existence of turning points
(in Questions 7, 12, 14 and 20). Importantly, though, domains and scales need to be
carefully selected so that features are displayed. Optimal selection can involve
34 Forster & Mueller

restricting the domain to values specified in the question (Question 12), and a
trigonometric rather than decimal scale (Question 7). Even so, some features
might not be visible, the structure displayed might be inconsistent with
conventional graphing, and the structure might bear little resemblance to the
phenomena it is representing. Examples are, respectively, point discontinuities
(Questions 13 and 20), branches of a curve stopping on asymptotes (Question 13),
and spatial relationships on a velocity time graph (Question 14). Therefore,
exploratory work is recommended.
We established that exploration meant prediction of graphical features from
the function expression and previous part questions; reading along the branches of
a graph and judging the function values; and accessing the table of values to
determine or check function values. Thus, ideally, structural understanding (of
functions as wholes), operational understanding (of the ways function values
depend on x) and procedures for accessing function values are brought to bear in
interpreting the graphs; and contexts need to be accommodated.
Specifically, operational understanding was elicited and incomplete
understanding revealed by some (relatively few) students in their calculator-
based determination and justification of a limit (Question 20). Students’ scores
indicate the demands of graphical interpretation were particularly high with
the velocity function (Question 14), but difficulty with interpreting graphs of a
derivative function is not limited to calculator graphs.
Once the graphical features that are relevant to an examination question
have been determined, students can use the automated calculation facilities on
the calculator to produce numerical answers. The automation is compensation for
the demanding work of graphical interpretation. The interviewed students
indicated use of automated facilities for roots (Question 14), coordinates of
turning points (Questions 7, 14 and 20), and coordinates of points of intersection
(Questions 12 and 20). The table of values and trace facility were also accessed.
A final step in the graphical solution is transcription of the graph or the
numerical values from the graph, onto the examination script. Attention needed
to be given (and was not always) to clearly identifying graphical features
(Question 13); to compensating for limitations of the calculator graph (by, for
example, not joining branches across a vertical asymptote) (Question 13); to
providing scales on the axes (Question 20); and to converting decimals from the
calculator to exact values (Question 7).
Implications for teaching are that procedures associated with graphical
solutions need to be the subject of instruction. These include procedures for (a)
setting up the calculator for an adequate graph, (b) enhancing graphical
interpretation, (c) obtaining numerical outputs and (d) ensuring written answers
are adequate. In particular, the moving between the different sources of
information while interpreting the graph might not fall into our conventional
definition of algorithmic, but is a learnt skill that needs to be explicitly
addressed in class. In other words, different operational and structural
interpretations of graphs and other symbolic forms need to be encountered in class
and students encouraged to integrate them. As well, operational views need to be
revisited so that pseudo-structural thinking (Sfard & Linchevski, 1994), the
application of procedures without knowing the true nature of the concept to
Assessment in Calculus in the Presence of Graphics Calculators 35

which they relate, is avoided.

If a calculator solution is attempted it is also important to know the syntax
required in relation to entering function expressions (Question 20) and more so in
relation to complex integral expressions (Questions 6 and 8). Our inquiry has
uncovered this as a problematic area for some students. Boers and Jones (1994) also
found incorrect entry a problem. Other sites for error were not having the
calculator set to the appropriate angle mode (Questions 6 and 7), and use of the
solve facility (Questions 7 and 12). A limitation of the numerical solve is that
the solution obtained is nearest to an estimate that the student enters. A visual
check with the linked graph is recommended, or use of the graph from the start.
These aspects of calculator use also have implications for teaching.
Implications of our inquiry for assessment are that an awareness of the
demands of graphical interpretation and of likely misinterpretation needs to be
brought to the setting of questions. In addition, a balance between opportunities
for visual, empirical approaches and analytic methods needs to be built into
examination papers. We say this in view of the increased role for diagrams and
graphs in the Calculus TEE since the introduction of graphics calculators (Mueller
& Forster, 2000), and in view of indications of superior performance by girls on
questions requiring analytic solutions and by boys on questions requiring complex
graphical interpretation (Forster & Mueller, in press). A question for research is
what balance is desirable, and an assessment issue is why do examination
questions ask for exact values if use of the calculators is expected and valued?
The subjective nature of the sample data leads us to be cautious in reaching
firm conclusions on the extent of calculator usage. However, in relation to optional
use, when a question required a graph (Question 13) or the evaluation of an
integral (Questions 7 and 8), the high majority of students opted for calculator
use. In other instances when calculator usage was optional, less than half the
students in the sample seemed to choose it, but for those that did there seemed to
be most advantage in using the table of values for the evaluation of a difficult
limit (Question 20).
Our analysis has illustrated the changed nature of TEE Calculus in Western
Australia following the introduction of graphics calculators and has developed
the theme that students need to have strategies to check graphical, empirical
solutions. It also seems time to clarify what it is that we want Year 12 Calculus
students to know in the presence of the technology.

References
Alguire, H., & Forster, P. A. (1999). Promoting mathematical understanding through the use of
graphics calculators. Proceedings of the Australian Curriculum, Assessment and
Certification Authorities Conference (pp. 131-144). Perth: Curriculum Council..
Anderson, M., Bloom, L., Mueller, U., & Pedler, P. (1997). Graphics calculators: Some
implications for course content and examination. Paper presented at the third Asian
Technology Conference in Mathematics. Available: http://www.runet.edu/~atcm/
atcm97.html
Berger, M. (1998). Graphics calculators: An interpretative framework. For the Learning of
Mathematics, 18 (2), 13-20.
Boers, M. A. M., & Jones, P. L. (1994). Students’ use of graphics calculators under
examination conditions. International Journal of Mathematics Education in Science and
Technology, 25 (4), 491-516.
Brown, R., & Neilson, B. (2001). What algebra is required in “high stakes” system wide
36 Forster & Mueller

assessment? A comparison of three systems. In H. Chick, K. Stacey, J. Vincent & J. Vincent

(Eds.), The future of teaching and learning of algebra, Proceedings of the 12th ICME Study
Conference (pp. 128-135). Melbourne: University of Melbourne..
Forster, P. A., & Mueller, U. (2001). Outcomes and implications of students’ use of graphics
calculators in the public examination of calculus. International Journal of Mathematical
Education in Science and Technology, 32 (1), 37-52.
Forster, P. A., & Mueller, U. (in press). What effect does the introduction of graphics
calculators have on the performance of boys and girls in assessment of tertiary entrance
calculus? International Journal of Mathematical Education in Science and Technology.
Forster, P. A., & Taylor, P. C. (2000). A multiple perspective analysis of learning in the
presence of technology. Educational Studies in Mathematics, 42 (1), 35-59.
Gray, E. M., & Tall, D. O. (1994). Duality, ambiguity, and flexibility: A “proceptual” view of
simple arithmetic. Journal for Research in Mathematics Education, 25 , 116-146.
Hale, P. (2000). Kinematics and graphs: Students’ difficulties and CBLs. Mathematics Teacher,
93 (5), 414-417.
Hiebert, J., & Carpenter, T. P. (1992). Learning and teaching with understanding. In D. A.
Grouws (Ed.), Handbook of research in mathematics teaching and learning (pp. 65-97).
New York: Macmillan.
Hollar, J. C., & Norwood, K. (1999). The effects of a graphing-approach intermediate algebra
curriculum on students’ understanding of function. Journal for Research in Mathematics
Education, 30 (2), 220-226.
Jones, P., & McCrae, B. (1996). Assessing the impact of graphics calculators on mathematics
examinations. In P. Clarkson (Ed.), Technology in mathematics education (Proceedings of
the 19th annual conference of the Mathematics Education Research Group of
Australasia, pp. 306-313). Melbourne: MERGA.
Kemp, M., Kissane, B., & Bradley, J. (1996). Graphics calculators use in examinations:
Accident or design? Australian Senior Mathematics Journal, 10 (1), 33-50.
Mueller, U., & Forster, P. A. (2000). A comparative analysis of the 1996-1999 Calculus TEE
papers. In J. Bana & A. Chapman (Eds.), Mathematics education beyond 2000 (Proceedings
of the twenty-third annual conference of the Mathematics Education Group of
Australasia, Perth, pp. 465-473). Sydney: MERGA.
Nesher, P. (1986). Are mathematical understanding and algorithmic performance related? For
the Learning of Mathematics, 6(3), 2-9.
Schwarz, B. B., & Hershkowitz, R. (1999). Prototypes: Brakes or levers in learning the
function concept? The role of computer tools. Journal for Research in Mathematics
Education, 30 (4), 362-389.
Senk, S. L. Beckmann, C. E., Thompson, D. R. (1997). Assessment and grading in high school
mathematics classrooms. Journal for Research in Mathematics Education, 28 (2), 187-215.
Sfard, A. (1991). On the dual nature of mathematical conceptions: Reflections on processes
and objects as different sides of the same coin. Educational Studies in Mathematics, 22, 1-
36.
Sfard, A., & Linchevski, L. (1994). The gains and pitfalls of reification – the case of algebra.
Educational Studies in Mathematics, 26 , 191-228.
Tall, D. O., & Thomas, M. O. J. (1991). Encouraging versatile thinking in algebra using the
computer. Educational Studies in Mathematics, 22 , 125-147.
Ward, R. A. (1997). An investigation of scaling issues and graphing-calculator associated
misconceptions among high school students. (Doctoral dissertation, University of
Virginia, 1997). Dissertations Abstracts International, 58/06, 1.

Authors
Patricia A. Forster, Faculty of Community Services, Education and Social Sciences, Edith
Cowan University, 2 Bradford Street, Mt Lawley, Western Australia 6050. E-mail:
<forster@iinet.net.au>

Ute Mueller, School of Engineering and Mathematics, Edith Cowan University, 100
Joondalup Drive, Joondalup, Western Australia 6027. E-mail: <u.mueller@ecu.edu.au>