Innovations in Education and Teaching International, 2014

Vol. 51, No. 4, 400410, http://dx.doi.org/10.1080/14703297.2013.785252

Grading rubrics: hoopla or help?

Rebecca J. Howell*

Department of Criminal Justice, Charleston Southern University, Charleston, SC, USA

The purpose of the study was to offer some quantitative, multivariate evidence
concerning the impact of grading rubric use on academic outcome among Amer-
ican higher education students. Using a pre-post, quasi-experimental research
design, cross-sectional data were derived from undergraduates enrolled in an
elective during spring and fall 2009 at an American research university. The
control group, who completed an assignment without a grading rubric, scored
signicantly lower, on average, than the treatment group who completed the
same assignment with the use of the rubric. Regression results indicated that the
grading rubric constituted a signicant predictor of assignment outcome, but
exerted a less powerful effect than baseline course knowledge and grade point
average. Suggestions for future research are provided.
Keywords: grading rubric; outcome evaluation; learning assessment

Introduction
For the past 20 years, standards of teaching and learning in academia have been the
focus of concern and reform in a number of industrialised countries worldwide,
such as the UK, (Dearing, 1997; Woodward, 2002), Australia (Australian Universi-
ties Quality Agency, 2009; James, 2003; Sadler, 2009), and New Zealand (Brown,
Glasswell, & Harland, 2004; Crooks, 2002; Lee & Lee, 2001). Within the past
10 years, an international conversation has also emerged, with cross-national
comparisons of assessment standards and dialogue about the global implications of
the quality of education of single nations (Harris, 2009; Nusche, 2008; Organisation
for Economic Cooperation & Development, 2009; Yorke, 1999).
The concern about student assessment and accountability in higher education
in the USA has trended similarly to that of similar industrialised countries, but, in
America, this concern emerged for slightly different reasons, began slightly later in
time, and was slower in gaining ground (Allen & Tanner, 2006). Beginning in the
mid-1980s, American higher education began to shift from an emphasis on the
traditional paradigm of testing knowledge and teacher-centred learning to a
paradigm characterised by active, student-centred learning and purposive assessment
(Cox & Richlin, 1993; Wiggins, 1998). In contrast to simply regurgitating factual
information, the modern-day American student is expected to work toward solving
open-ended real world problems and tasks, by actively demonstrating higher-order
cognitive competencies, such as reective and critical thinking (Brown, Bull, &
Pendlebury, 1997).

*Email: rhowell@csuniv.edu

2013 Taylor & Francis

 Innovations in Education and Teaching International 401

It was within this re-direction in pedagogy, and in partial response to the

accountability pressure exerted by the public and state and federal governments, that
evidence-based learning assessment in the USA was born (Dochy, Gijbels, &
Segers, 2006). In Americas contemporary academia, evidence of the permanence of
assessment abounds. There has been federal talk of national standardised testing in
higher education (Arensen, 2006), some US state legislatures have enacted assess-
ment-based performance standards for institutions in making budget decisions
(Wellman, 2001), and each of the six US regional accrediting associations have
established assessment-related standards of effectiveness.
Regardless of nation or institution, course instructors employ a variety of strate-
gies to enhance student learning. One commonly used mode of assessment involves
evaluating open-ended, academic work (such as essays and papers) with the guid-
ance of grading rubrics, or marking schemes. In its most basic form, a grading rub-
ric is a matrix that provides levels of achievement for a set of criteria (Allen &
Tanner, 2006; Kuisma, 1999). The purpose of a grading rubric is threefold: formu-
late standards for achievement, provide an objective way to grade work, and make
expectations clear to students, particularly when used as a guide while completing
the task at hand (Sadler, 2009). Rubrics tend to either assess a specic task or a
more general type of performance, and they are either holistic (i.e. assess overall
quality) or analytic (evaluate multiple aspects of performance) in nature (Jonsson &
Svingby, 2007).

Extant research
A relatively large literature base exists on particular aspects of rubric use, with
much attention paid to the philosophical rationale of use; design, development, and
implementation; and construct validity (Hafner & Hafner, 2003; Stemmack,
Konheim-Kalkstein, Manor, Massey & Schmitz, 2009; Thaler, Kazemi, & Huscher,
2009). With respect to the effects of rubric use on academic result/outcome,
however, the research base is smaller in size (Price, 2005). Studies from the UK
(Dearing, 1997; Hornby, 2003; Yorke, Bridges, & Woolf, 2000) and Australia
(James & McInnis, 2001; Krause, Hartley, James, & McInnis, 2005; Smith &
Coombe, 2006) suggest that grading rubric use increases academic performance.
Unfortunately, much less quantitative, multivariate research has been conducted in
the USA (Rowntree, 1987), despite the increasing popularity of grading rubrics in
American higher education (Jonsson & Svingby, 2007). Of the limited American
research that has been conducted, it appears that rubric use engenders promising
academic benets, by increasing academic outcome (Dahm, Newell, & Newell,
2004; Moriarty, 2006). Other research indicates specic aspects of performance may
be enhanced, including critical thinking and problem-solving (Sadler & Good,
2006; Schafer, Swanson, Bene, & Newberry, 2001). Some studies suggest that
students and teachers perceive that rubrics clarify expectations and make the
grading standards most explicit (Bissell & Lemons, 2006; Dahm et al., 2004; Fred-
eriksen & Collins, 1989).
The results from Jonsson and Svingbys (2007) qualitative meta-analysis, one
of the most comprehensive reviews conducted on the rubrics literature to date,
underscores the need for further quantitative, multivariate research. They found
that only 10 of the 75 studies published within the past 40 years investigated the
impact of rubrics on student improvement, and the bulk of these studies did not
402 R.J. Howell

investigate student academic result/outcome. In addition, not only were the bulk
of these studies conducted using non-US samples, but the mixed ndings that
were yielded precluded the ability to draw any conclusions about student
improvement related to the use of rubrics from this material (Jonsson & Svingby,
2007, p. 139).
Taken together, a paucity of multivariate, quantitative studies exist on the utility
of grading rubric use for positively impacting academic outcome, particularly
studies utilising data from American samples of higher education students. Latter
studies are needed, as an increased priority has been placed on standardised
methods of assessment, including rubric use. The current study aimed to address
this gap in the literature, by using data from students attending an American
research university.

Method
A quasi-experimental, prepost evaluation was carried out within a lower-level,
undergraduate elective, Juvenile Delinquency, which was offered during spring and
fall 2009 at a large research university. The university is located in a large-size,
southeastern US college town that serves as a bedroom community for a major
city within relatively short commuting distance. The researcher served as the
instructor of the course. This campus-wide elective constituted a social/behavioural
course that satised a requirement of the general educational curriculum. Criminal
justice majors and minors, as well as sociology minors, could complete the course
and apply the credit hours toward respective programmes.
Students were required to complete 10 assignments in the course. Assignment
#6 constituted the test instrument. Worth a total of 20 points (or 4% of the nal
course grade), this assignment was completed by 36 students enrolled in the course
during spring 2009. This version of the assignment did not contain a grading rubric;
students simply were directed to follow the assignment directions and they were
informed that the assignment was worth a total of 20 points. Prior to the fall 2009
semester, a holistic, task-specic grading rubric was developed and added to the
end of the assignment. Presented in Table 1, this rubric contained four levels of
performance. The inclusion of this grading rubric was the only change made to the
assignment. During fall 2009, the assignment, which now contained the grading
rubric, was distributed and completed by 40 students.
During spring and fall 2009, 53 and 51 students were enrolled in the course,
respectively. Considering these enrolment numbers in conjunction with the number
of students who completed the assignment, the completion rate for students in the
spring 2009 course was 68% (n = 36), while 78% (n = 40) of students enrolled
during fall 2009 completed the assignment. No students from either class were
aware that the assignment would be or had been redesigned and no students
enrolled in the course during the spring took the course in the fall. Students from
both semesters had one week to complete the assignment, they completed it during
the same time period (ninth week) of the semester, and the instructor graded all
assignments from both semesters.
The research was approved by the universitys Institutional Review Board
(IRB) for the Protection of Human Subjects. The study was conducted after the
fall 2009 semester ended and the researcher was not in contact with nor had
any way to contact the students in the two courses to gain their informed
 Innovations in Education and Teaching International 403

Table 1. Grading rubric.

Performance Point
level Criteria Comments breakdown
Exemplary Well-written and clear 20 pts
Identies the major factors found in social
disorganisation social bond and social learn-
ing theory and articulates how these factors
work together in explaining the initiation
and continuation of gang delinquency
Illustrates each of the theoretical elements
that are discussed, by drawing upon exam-
ples from Gang Wars

Accomplished Relatively well-written and/or fairly clear 19 pts

Identies the major factors found in social
disorganisation social bond and social learn-
ing theory and articulates how these factors
work together in explaining the initiation
and continuation of gang delinquency
Illustrates each of the theoretical elements
that are discussed, by drawing upon exam-
ples from Gang Wars

Developing Moderately well-written and/or somewhat 1718 pts

clear
Identies some of the major factors found
in social disorganisation social bond, and
social learning theory and articulates, in
part, how these factors work together in
explaining the initiation and continuation of
gang delinquency
Illustrates, in part, each of the theoretical
elements that are discussed, by drawing
upon examples from Gang Wars

Beginning Poorly written and/or unclear 116 pts

Inaccurate theoretical application that identi-
es some/none of the major factors found in
social disorganisation social bond, and
social learning theory; articulates inaccu-
rately how these major factors explain the
initiation and/or continuation of gang delin-
quency, and/or fails to illustrate each of the
theoretical elements that are discussed, by
drawing upon examples from Gang Wars

consent to use the data they provided during the normal course of the classs
educational process. Consequently, the researcher submitted to the IRB and had
approved a request for the waiver of informed consent and a further application
to use the students grades and grade point average data in a deidentied, aggre-
gate manner in the research.
404 R.J. Howell

Measures
A total of seven variables were examined. The dependent variable, Assignment, was
a continuous variable that was measured on a 100-point scale and indicated how
well a given student performed on Assignment #6.
Treatment, a dichotomous predictor constituted the independent variable of inter-
est. The control group (0) constituted those students who were enrolled in the
spring 2009 semester and who completed the assignment without the aid of the
rubric. The treatment group (1) comprised students from the fall 2009 course who
completed the assignment with the use of the grading rubric.
Research nds that academic outcome may be impacted positively by higher
year in college (Cejda, Kaylor, & Rewey, 1998); baseline course-specic
knowledge, which may also be linked to college major (Halloun & Hestenes,
1985); and sex, with females more apt to perform better academically than males
(Hyde, Fennema, & Lamon, 1990; Pomerantz, Altermatt, & Saxon, 2002). Other
research indicates a positive, linear relationship between overall academic perfor-
mance and outcome at the course-level (Kleijn, Ploeg, & Topman, 1994; Potolsky,
Cohen, & Saylor, 2003; Sansgiry, Bhosle, & Sail, 2006; Suda, Franks, McKibbin,
Wang, & Smith, 2008). In light of these associations, the effects of ve possible
explanatory factors were controlled: College Year, CJ Major, Prior Grade Point
Average (GPA), Pre-Test, and Sex. College Year (1 freshmen, 2 sophomore, 3
junior, 4 senior) was treated as a continuous variables, while Sex (0 female, 1
male) was measured dichotomously. The dichotomous variable, CJ Major, was mea-
sured with 0 signifying non-criminal justice major and 1 denoting criminal jus-
tice major. As a continuous variable, Prior GPA was operationalised as a given
students cumulative grade point average prior to completing the juvenile delin-
quency course. Finally, Pre-Test, which was measured on a 100-point scale, consti-
tuted students scores on the pre-test instrument for the course. Each semester, the
instructor administered the same non-graded, pre-test instrument to students on the
rst day of class. The purpose of this instrument was to assess, at base-line, what
course-specic knowledge students possessed upon starting the class.
Based on the extant research (see, e.g. Jonsson & Svingby, 2007), the variables
were used to test the following hypothesis: Grading rubric use exerts an important,
positive impact on academic outcome. To test this hypothesis, two major types of
analyses were conducted: an independent samples t-test and ordinary least squares
regression (OLS), a multivariate statistical technique. As opposed to logistic regres-
sion, which is utilised in instances where the dependent variable is measured as a
discrete variable, OLS was chosen because it requires that the outcome be measured
as a continuous variable (Anderson, 2008).

Results
A t-test for independent samples was rst calculated, in order to determine whether
the treatment and control groups differed signicantly, on average, with respect to
the dependent variable, Assignment. This statistical technique provided a simple test
of the differences in the means scores on the assignment for the control and treat-
ment groups. Results indicated that the treatment groups mean (86.77%) on the
assignment was signicantly higher than the control groups mean (79.55%)
(t = 2.186, p < .05). To test the hypothesised expectation that the grading rubric
 Innovations in Education and Teaching International 405

enhanced performance on the assignment, attention turned to analysing the multivar-

iate regression model.
To prepare the data for multivariate analysis, the control and treatment groups
were collapsed into one total sample (N = 76). Descriptive statistics and bivariate
correlations then were calculated, and then the OLS model was developed. There
were no missing data on any of the independent or dependent variables. Variable
diagnostics indicated that the values of the independent variables across both groups
were normally distributed.
The total can be described as follows. Across both semesters (N = 76), scores on
the assignment ranged from a low of 35% to a high of 100%, with an average of
83.36%. More than half (52.6%, n = 40) of the sample completed the assignment
with the aid of the grading rubric. Across both courses, on average, students were
female, in their junior year of college, enrolled in a non-criminal justice programme
of study, possessed a grade point average of 2.77 prior to completing the course,
and scored a 52.41% on the course pre-test.
Bivariate correlations (Table 2) were evaluated in an effort to identify problem-
atic collinearity between any two measures. Bivariate collinearity was not a
problem, as all coefcients were smaller than .500 (George & Mallery, 2001). Three
of the ve signicant correlations centred on the assignment. As compared to their
counterparts, the students who performed better on the assignment tended to be
those who were provided the grading rubric (p < .05), had a high GPA (p < .01) and
did well on the pre-test (p < .01). Students from the control group tended to be
lower-classmen as opposed to upper classmen (p < .01). Finally, across both
semesters, students who had relatively high GPAs also tended to have relatively
high pre-test scores (p < .01).
Table 3 presents the ndings from the OLS regression model that tested the
hypothesis that grading rubric use exerts an important, positive impact on academic
outcome. Together, the variables in the model explain 33.8% of the variance in
Assignment, a proportion considered by conventional social science and educational
research standards to be moderate in size (Hensen, Hull, & Williams, 2010; Ozer,
1985). As hypothesised, use of the grading rubric, as measured by Treatment,
exerted a positive and signicant impact on Assignment (p < .05). Two other vari-
ables exerted signicant effects at the .05 alpha level of signicance: Prior GPA and
Pre-test. An examination of the standardised regression coefcients (Beta) enabled a
comparison of the relative importance of the predictors, whereby the size and signi-
cance level of each signicant (p < .05) standardised coefcient were compared
(Bring, 1994; Rodgers & Nicewander, 1988). This comparison indicated that

Table 2. Bivariate correlations (N = 76).

Variable Assignment Treatment College year CJ major Prior GPA Pre-test Sex
Assignment
Treatment .246
College year .056 .306
CJ major .209 .037 .119
Prior GPA .441 .090 .079 .151
Pre-test .476 .113 .079 .196 .485
Sex .110 .038 .133 .192 .213 .034
Note: p < .10;
p < .05.
406 R.J. Howell

Table 3. OLS results (N = 76).

Predictor b SE Beta Sig.

Treatment 6.153 3.060 .210 .048
College year 1.397 1.729 .085 .422
CJ major 3.746 3.079 .126 .228
Prior GPA 4.665 2.274 .238 .044
Pre-test .377 .141 .304 .010
Sex 1.666 3.203 .054 .605
R2 = .338 < .01
Note: b = unstandardised coefcient; SE = standard error; and Beta = standardised coefcient.

although the grading rubric played an important role in students assignment

outcome, the effects posed by Pre-test and Prior GPA were larger in magnitude, with
priority ascribed to Prior GPA. Hence, in order of importance, students Prior GPA
had the greatest impact in effecting change in Assignment, followed by Pre-test and
the grading rubric.

Conclusion
Student assessment remains an important issue in academia. One way to evaluate
academic outcome at the course-level is to grade work using a standardised rubric.
The use of grading rubrics among faculty is widely cited in the literature,
particularly in Australia and the UK (Krause et al., 2005); however, there is little
quantitative, multivariate research within American higher education. Quantitative
investigations using data from American samples have the potential to add to the
knowledge base in this area.
The multivariate, quasi-experimental ndings reported here favour the continued
use of grading rubrics on college and university campuses in the US and world-
wide, but with some caution. Although the grading rubric was helpful in positively
impacting academic outcome, the results underscore the fact that the use of grading
rubrics in higher education is not a silver bullet solution for bolstering academic
performance among undergraduates. Baseline course-specic knowledge and overall
academic performance prior to course start both had a stronger impact on academic
performance than the grading rubric. This nding is consistent with the literature
consensus that academic achievement (as measured by grade point average) is a
signicant predictor of course-specic academic outcomes (Potolsky et al., 2003).
Hence, with respect to educational practice, the ndings underscore the need for
instructors to use grading rubrics for course-specic assignments, yet continue to
stress to undergraduates the preeminent importance that general academic achieve-
ment has for success in current and future classes.
Suggestions for future research largely stem from the three major study limits.
The rst limitation involves intra-rater reliability. There are three major explanations
for variability between a control and treatment group in academic outcome: actual
group differences in outcome, group differences in the graders judgment, and group
differences in expected tasks (Brown et al., 1997). The latter issue is not a likely
alternative explanation for the rubric-outcome association observed in the study,
since the same assignment and directions were distributed to both the treatment and
 Innovations in Education and Teaching International 407

control group; the only difference was that the treatment groups assignment
contained the grading rubric.
In contrast, group differences in grading judgment (i.e. grading bias) remain
a plausible alternative explanation for the positive rubric-outcome association; it
was beyond the scope of the study to rule out this validity threat. Although this
limitation poses some concern for the validity of the rubric-outcome relationship
observed, the impact of this threat on the ndings likely is minimal. Seven
research projects reviewed by Jonsson and Svingby (2007) investigated the intra-
rater reliability of rubric scoring. Nearly all of these studies reported Cronbachs
alpha values that exceeded the commonly accepted value of .70. So, while it
was likely that intra-rater reliability within the present study was not exceedingly
low, and the instructor made it a point to grade the assignments in a similar
fashion across both semesters, this threat remains plausible due to the simple
fact that it was not objectively ruled out. Future research in this area might
assess the impact of this threat on the validity of ndings.
Second, it was beyond the scope of the research to examine systematic differ-
ences between students who did and did not complete the course assignment.
Future research should attempt to determine the nature, if any, and degree of
difference between these two groups.
Finally, the study did not employ a true experimental research design, and
although the amount of the explained variance (R2 = .338) in the outcome is typical
for an educational research study (Good & Fletcher, 2006), more than half of the
variation in the scores on the assignment was not explained by the variables
investigated. Since it is probable that factors other than those examined made a
contribution to explaining assignment outcome, the ndings should be interpreted
with some caution.
A potential fruitful line of future research involves accounting for particular
mediating variables. One such factor may be expectation clarity, as students can
nd written descriptions of criteria on grading rubrics and standards of excel-
lence difcult to understand (Carless, 2006; ODonovan, Price, & Rust, 2004).
Given this nding, it is suggested that future research substantiate empirically
the mediation pathway (i.e. grading rubric use increased expectation clarity
increased academic performance) that individual studies imply may exist.
In a similar vein, the grading/marking experience of the marker may
constitute a moderating variable that interacts with grading rubric use in
impacting the academic outcome of students. Not only have a few studies found
a mismatch between assessment criteria and the grades assigned to student work,
but instructors experienced in grading have been found to view themselves as
such (Norton et al., 2004; Price, 2005). It may be that, in consequence, these
experienced graders may become more intuitive than conscious when grading,
thereby not relying and adhering to a given grading rubric as closely as an
instructor with less grading experience (Ecclestone, 2001). Indeed, instructors
who are new to grading have been found to pay closer attention to grading and
the tools (e.g. grading rubrics) used to guide their grading decisions (Price,
2005).

Acknowledgement
This research was supported by the 2009 Active and Collaborative Learning Grant from the
Division of Academic Affairs, University of Alabama.
408 R.J. Howell

Note on contributor
Rebecca J. Howell is assistant professor of criminal justice at Charleston Southern
University. Her expertise includes student learning assessment, criminological theory testing,
and juvenile delinquency.

References
Allen, D., & Tanner, K. (2006). Rubrics: Tools for making learning goals and evaluation
criteria explicit for both teachers and learners. Life Sciences Education, 5, 197203.
Anderson, R. (2008). Modern methods for robust regression. Thousand Oaks, CA: Sage.
Arensen, K. W. (2006, February 9). Panel explores standard tests for colleges. NewYork
Times. Retrieved June 27, 2011, from http://www.nytimes.com
Australian Universities Quality Agency. (2009). Setting and monitoring academic standards
for Australian higher education. Melbourne: Author.
Bissell, A. N., & Lemons, P. P. (2006). A new method for assessing critical thinking in the
classroom. Bioscience, 56, 6672.
Bring, J. (1994). How to standardize regression coefcients. The American Statistician, 48,
209213.
Brown, G., Bull, J., & Pendlebury, M. (1997). Assessing student learning in higher educa-
tion. London: Routledge.
Brown, G. T., Glasswell, K., & Harland, D. (2004). Accuracy in the scoring of writing:
Studies of reliability and validity using a New Zealand writing assessment system.
Assessing Writing, 9, 105121.
Carless, D. (2006). Differing perceptions in the feedback process. Studies in Higher Educa-
tion, 31, 219233.
Cejda, B. D., Kaylor, A. J., & Rewey, K. L. (1998). Transfer shock in an academic
discipline: The relationship between students major and their academic performance.
Community College Review, 26, 113.
Cox, M. D., & Richlin, L. (1993). Emerging trends in college teaching for the 21st Century:
A message from the Editors. Journal on Excellence in College Teaching, 4, 17.
Crooks, T. J. (2002). Educational assessment in New Zealand schools. Assessment in Educa-
tion: Principles, Policy & Practice, 9, 237253.
Dahm, K. D., Newell, J. A., & Newell, H. L. (2004). Rubric development for assessment of
undergraduate research: Evaluating multidisciplinary team projects. Chemical Engineer-
ing Education, 38, 6873.
Dearing, R. (1997). Higher education in a learning society. London: National Committee of
Enquiry into Higher Education.
Dochy, F., Gijbels, D., & Segers, M. (2006). Learning and the emerging new assessment
culture. In L. Verschaffel, F. Dochy, M. Boekaerts, & S. Vosniadou (Eds.), Instructional
psychology: Past, present, and future trends (pp. 191206). Oxford: Elsevier.
Ecclestone, K. (2001). I know a 2:1 when I see it: Understanding criteria for degree classi-
cations in franchised university programmes. Journal of Further and Higher Education,
25, 301313.
Frederiksen, J. R., & Collins, A. (1989). A systems approach to educational testing. Educa-
tional Researcher, 18, 2732.
George, D., & Mallery, P. (2001). SPSS for Windows step by step: A simple guide and
reference (3rd ed.). Boston, MA: Allyn and Bacon.
Good, R., & Fletcher, H. J. (2006). Reporting explained variance. Journal of Research in
Science Teaching, 18, 17.
Hafner, J. C., & Hafner, P. M. (2003). Quantitative analysis of the rubric as an assessment
tool: An empirical study of student peer-group rating. International Journal of Science
Education, 25, 15091528.
Halloun, I., & Hestenes, D. (1985). The initial knowledge state of college physics students.
American Journal of Physics, 53, 10431055.
Harris, K. L. (2009). International trends in establishing standards of academic achievement
in higher education. Melbourne: Australian Universities Quality Agency.
 Innovations in Education and Teaching International 409

Hensen, R. K., Hull, D. M., & Williams, C. S. (2010). Methodology in our education
research culture: Toward a stronger collective quantitative prociency. Educational
Researcher, 39, 229240.
Hornby, W. (2003). Assessing using grade-related criteria: A single currency for universities?
Assessment & Evaluation in Higher Education, 28, 435453.
Hyde, J. S., Fennema, E., & Lamon, S. J. (1990). Gender differences in mathematics perfor-
mance: A meta-analysis. Psychological Bulletin, 107, 139155.
James, R. (2003). Academic standards and the assessment of student learning: Some current
issues in Australian higher education. Tertiary Education and Management, 9, 187198.
James, R., & McInnis, C. (2001). Standards oil tertiary debate. The Australian Higher
Education, 15, 28.
Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and
educational consequences. Educational Research Review, 2, 130144.
Kleijn, W., Ploeg, H., & Topman, R. (1994). Cognition, study habits, test anxiety, and
academic performance. Psychological Reports, 75, 12191226.
Krause, K., Hartley, R., James, R., & McInnis, C. (2005). The rst year experience in
Australian universities: Findings from a decade of national studies. Canberra: Common-
wealth of Australia.
Kuisma, R. (1999). Criteria referenced marking of written assignments. Assessment & Evalu-
ation in Higher Education, 24, 2739.
Lee, H., & Lee, G. (2001). The National Certicate of Educational Achievement (NCEA):
Fragile-handle with care. New Zealand Annual Review of Education, 10, 538.
Moriarty, L. J. (2006). Investing in quality: The current state of assessment in criminal
justice programs. Justice Quarterly, 20, 409427.
Norton, L., Harrington, K., Elander, J., Sineld, S., Reddy, P., Pitt, E., & Aiyegbayo, O.
(2004, September). Supporting diversity and inclusivity through writing workshops.
Paper presented at the International Improving Student Learning Symposium, Birming-
ham, England.
Nusche, D. (2008). Assessment of learning outcomes in higher education: A comparative
review of selected practices. Paris: Organisation for Economic Cooperation and Develop-
ment.
ODonovan, B., Price, M., & Rust, C. (2004). Know what I mean? Enhancing student under-
standing of assessment standards and criteria. Teaching in Higher Education, 9, 325335.
Organisation for Economic Cooperation and Development. (2009). International assessment
of higher education learning outcomes (AHELO) feasibility study. Paris: Author.
Ozer, D. J. (1985). Correlation and the coefcient of determination. Psychological Bulletin,
97, 307315.
Pomerantz, E. M., Altermatt, E. R., & Saxon, J. L. (2002). Making the grade but feeling
distressed: Gender differences in academic performance and internal distress. Journal of
Educational Psychology, 94, 396404.
Potolsky, A., Cohen, J., & Saylor, C. (2003). Academic performance of nursing students: Do
prerequisite grades and tutoring make a difference? Nursing Education Perspectives, 24,
246250.
Price, M. (2005). Assessment standards: The role of communities of practice and the
scholarship of assessment. Assessment & Evaluation in Higher Education, 30, 215230.
Rodgers, J. L., & Nicewander, A. (1988). Thirteen ways to look at the correlation coef-
cient. The American Statistician, 42, 5966.
Rowntree, D. (1987). Assessing students: How shall we know them? (2nd ed.). London:
Kogan Page.
Sadler, D. R. (2009). Indeterminacy in the use of preset criteria for assessment and grading.
Assessment & Evaluation in Higher Education, 34, 159179.
Sadler, P. M., & Good, E. (2006). The impact of self- and peer-grading on student learning.
Educational Assessment, 11, 131.
Sansgiry, S., Bhosle, M., & Sail, K. (2006). Factors that affect academic performance among
pharmacy students. American Journal of Pharmacy Education, 70, 104116.
Schafer, W. D., Swanson, G., Bene, N., & Newberry, G. (2001). Effects of teacher knowl-
edge of rubrics on student achievement in four content areas. Applied Measurement in
Education, 14, 151170.
410 R.J. Howell

Smith, E., & Coombe, K. (2006). Quality and qualms in the marking of university assign-
ments by sessional staff: An exploratory study. Higher Education, 51, 4569.
Stemmack, M. A., Konheim-Kalkstein, Y. L., Manor, J. E., Massey, A. R., & Schmitz, J. P.
(2009). An assessment of reliability and validity of a rubric for grading APA-style intro-
ductions. Teaching of Psychology, 36, 102107.
Suda, K., Franks, A., McKibbin, T., Wang, J., & Smith, E. (2008, July). Differences in stu-
dent performance based on student and lecture location when using distance learning
technology. Paper presented at the annual meeting of the American Association of Col-
leges of Pharmacy, Chicago.
Thaler, N., Kazemi, E., & Huscher, C. (2009). Developing a rubric to assess student learning
outcomes using a class assignment. Teaching of Psychology, 36, 113116.
Wellman, J. V. (2001). Assessing state accountability systems. Change, 33, 4652.
Wiggins, G. (1998). Educative assessment: Designing assessments to inform and improve
student performance. San Francisco, CA: Jossey-Bass.
Woodward, W. (2002, May 20). Universities in crisis. The Guardian, pp. 45.
Yorke, M. (1999). Benchmarking academic standards in the UK. Tertiary Education and
Management, 5, 7994.
Yorke, M. P., Bridges, P., & Woolf, H. (2000). Mark distributions and marking practices in
UK higher education: Some challenging issues. Active Learning in Higher Education, 1,
727.
Copyright of Innovations in Education & Teaching International is the property of Routledge
and its content may not be copied or emailed to multiple sites or posted to a listserv without
the copyright holder's express written permission. However, users may print, download, or
email articles for individual use.