Sie sind auf Seite 1von 21

A Five-Dimensional Framework for Authentic Assessment

Author(s): Judith T. M. Gulikers, Theo J. Bastiaens, Paul A. Kirschner


Source: Educational Technology Research and Development, Vol. 52, No. 3 (2004), pp. 67-86
Published by: Springer
Stable URL: http://www.jstor.org/stable/30220391
Accessed: 03/09/2010 12:54

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at
http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless
you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you
may use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at
http://www.jstor.org/action/showPublisher?publisherCode=springer.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed
page of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of
content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms
of scholarship. For more information about JSTOR, please contact support@jstor.org.

Springer is collaborating with JSTOR to digitize, preserve and extend access to Educational Technology
Research and Development.

http://www.jstor.org
A Five-DimensionalFramework for
Authentic Assessment

JudithT.M.Gulikers
TheoJ. Bastioens
PaulA. Kirschner

Authenticityis an importantelementof new D It is widely acknowledged that in order to


modesofassessment.Theproblemis thatwhat meet the goals of education,a constructivealign-
authenticassessment really is, is unspecified. ment between instruction,learning and assess-
In this article, wefirst review the literatureon ment (ILA) is necessary (Biggs, 1996).
authenticity ofassessments, along with a Traditional frontal classroom instruction for
five-dimensionalframeworkfordesigning learningfacts,assessed throughshort-answeror
authenticassessments with professional multiple-choicetests, is an example of such an
practiceas the starting point. Then, we present alignment.TheILA-practicesin this kind of edu-
the results ofa qualitativestudy to determine cation can be characterized as instructional
if theframeworkis complete,and what the approach-knowledge transmission; learning
relative importanceof thefive dimensions is in approach-rote memorization;and assessment
the perceptionsof students and teachersofa procedure-standardized testing (Birenbaum,
vocationalcollegefor nursing. We discuss 2003). This approach to assessment is also
implicationsfor theframework,along with known as the testing culture (Birenbaum&
importantissues that need to be considered Dochy, 1996) and consists primarily of
when designing authenticassessments. decontextualized, psychometrically designed
items in a choice-response format to test for
knowledge and low-level cognitiveskill acquisi-
tion. The tests are primarilyused in a summa-
tive way to differentiatebetween students and
rankthem accordingto theirachievement.How-
ever, the alignment compatible with present-
day educational goals has changed over the
years. Currenteducationalgoals focus more on
the development of competent students and
future employees than on simple knowledge
acquisition.The ILA-practicesthat characterize
these goals areinstructional-approach-focused
on learning and competence development;
learning-approach-reflective-active knowl-
edge construction;and assessment-procedure-
contextualized,interpretative,and performance
assessment (Birenbaum,2003).Here, the goal of
assessment is the acquisition of higher-order
thinking processes and competenciesinstead of
factualknowledge and basic skills. The function
of the assessment changes from being summa-

Vol. 52, No. 3, 2004, pp. 67-86 ISSN1042-1629


ETR&D, 67
68 ETR&D,
Vol.52, No. 3

tive to also serving a formativegoal of promot- assessmentthis meansthat (a)tasksmust appro-


ing and enhancing student learning.This view priately reflect the competencythat needs to be
requires alternative assessments because stan- assessed, (b) the content of an assessment
dardized, multiple-choicetests are not suitable involves authentic tasks that representreal-life
for this (Birenbaum & Dochy, 1996; Segers, problems of the knowledge domain assessed,
Dochy, & Cascallar, 2003). Birenbaum and and (c) the thinkingprocessesthat expertsuse to
Dochy (1996) characterizedalternative assess- solve the problemin reallife arealso requiredby
ments as follows: Studentshave a responsibility the assessment task (Gielen et al., 2003). Based
for their own learning;they reflect,collaborate, on these criteria, authentic competency-based
and conduct a continuous dialogue with the assessmentshave a higher constructvalidity for
teacher. Assessment involves interesting real- measuring competencies than so-called objec-
life or authentic tasks and contexts as well as tive or traditionaltests have.
multiple assessment moments and methods to
Consequentialvaliditydescribes the intended
reach a profile score for determining student
and unintended effectsof assessmenton instruc-
learningor development.Increasingthe authen-
tion or teaching(Biggs,1996)and student learn-
ticity of an assessmentis expectedto have a pos-
itive influence on student learning and ing (Dochy & McDowell, 1998). As stated,
motivation (eg., Herrington & Herrington, Biggs's (1996)theory of constructivealignment
1998).Authenticity,however, is only a vaguely stressesthateffectiveeducationrequiresinstruc-
describeddimensionof assessment,becauseit is tion, learning,and assessmentto be compatible.
thought to be a familiar and generally known If students perceive a mismatch between the
concept that needs no explicit defining messages of the instructionand the assessment,
(Petraglia,1998).This articlefocuses on defining a positive impacton studentlearningis unlikely
authenticity in competency-based assessment, (Segers,Dierick,& Dochy, 2001).This impact of
without ignoring the importanceof other char- assessmenton instructionand on student learn-
acteristicsof alternativeassessment.
ing is corroborated by researchers as
Based on an extensive literaturestudy, a the- Frederiksen(1984, "The Real Test Bias"),Pro-
oretical framework consisting of five dimen- dromou (1995,"BackwashEffect"),Gibbs(1992,
sions of assessmentthat can vary in theirdegree "TailWagsthe Dog"),and Sambelland McDow-
of authenticity is presented. After the descrip- ell (1998, "Hidden Curriculum").Fredericksen
tion of this framework,the results of a qualita-
and Prodromouimplied that tests have a strong
tive study are discussed. This study explored
influence on what is taught, because teachers
whether the frameworkis a complete descrip-
teach to the test, even though the test might
tion of authenticityor is missing importantele-
ments, and what the relative importanceof the focus on things the teacherdoes not find most
dimensionsis in the perceptionsof studentsand important. Gibbs emphasized that student
teachersat a nursing college. learningis largely dependent on the assessment
and on student perceptions of the assessment
requirements.Sambelland McDowell held that
the effects of instruction and assessment on
The Importance of Authentic
Competency-Based Assessment learning are largely based on teacher and stu-
dent perceptions of the curriculum,which can
deviate fromthe actualintentionsof the curricu-
The two most important reasons for using
lum. All four ideas supportthe propositionthat
authenticcompetency-basedassessmentsare (a)
their constructvalidity and (b) their impact on learning and assessment are two sides of the
student learning, also called consequential same coin, and that they stronglyinfluenceeach
validity (Gielen, Dochy, & Dierick,2003). Con- other.To changestudentlearningin the direction
struct validity of an assessment is related to of competency development, authenticcompe-
whether an assessmentmeasureswhat it is sup- tency-based instruction aligned to authentic
posed to measure. With respect to competency competency-basedassessmentis needed.
A FIVE-DIMENSIONAL
FRAMEWORK
FORAUTHENTIC
ASSESSMENT 69

DefiningAuthenticAssessment resemblanceto the criterionsituation.This idea


is extended and specified by the theoretical
frameworkthat describesthatan assessmentcan
The question is thus, What is authenticity?Dif-
ferentresearchershave differentopinions about resemblea criterionsituationalong a numberof
dimensions.
authenticity.Some see authenticassessmentas a
synonym for performance assessment (Hart, Complicatingmattersis the fact that authen-
1994; Torrance,1995), while others argue that ticity is subjective(Honebein,Duffy & Fishman,
authenticassessmentputs a specialemphasison 1993; Huang, 2002; Petraglia, 1998) and is
the realistic value of the task and the context dependent on perceptions. This implies that
(Herrington & Herrington, 1998). Reeves and what studentsperceiveas authenticis not neces-
Okey (1996)pointed out that the crucialdiffer- sarily the same as what teachersand assessment
ence between performance assessment and developers see as authentic.If these perceptions
authentic assessment is the degree of fidelity of do indeed differ,then the fact that teachersusu-
the task and the conditionsunderwhich the per- ally develop authenticassessmentsaccordingto
formance would normally occur. Authentic their own view causes a problem:Although we
assessmentfocuses on high fidelity,whereasthis may do our best to develop authentic assess-
is not as important an issue in performance ments, this may all be for nothing if the learner
assessment. These distinctionsbetween perfor- does not perceive them as such. This process,
mance and authentic assessment indicate that known as preauthentication(Huang, 2002;
every authentic assessment is performance Petraglia,1998),can be interpretedeitheras that
assessment,but not vice versa (Meyer,1992) it is impossible to design an authentic assess-
ment, or that it is very important to carefully
Saveryand Duffy (1995)defined authenticity
examine the experiences of the users of the
of an assessment as the similaritybetween the
authenticassessments,beforedesigning authen-
cognitive demands-the thinking required-of
tic assessments(Nicaise,Gibney& Crane,2000).
the assessmentand the cognitivedemandsin the
criterion situation on which the assessment is We chose the latterinterpretation.
based. A criterionsituationreflectsor simulates This discussion about authentic assessment
a real-lifesituationthat could confrontstudents and validity shows that:
in their internship or future professional life. 1. In light of the constructivealignmenttheory
Darling-Hammond and Snyder (2000) argued (Biggs, 1996)authenticassessment should be
that dealing only with the thinking requiredis aligned to authentic instructionin order to
too narrow. In their view, students need to positively influencestudentlearning.
develop competenciesbecausereallife demands 2. Authentic assessment requires students to
the ability to integrate and coordinate knowl- demonstrate relevant competenciesthrough
edge, skills, and attitudes, and the capacity to a significant, meaningful, and worthwhile
apply them in new situations(VanMerrienboer, accomplishment (Resnick, 1987; Wiggins,
1997). Birenbaum (1996) further specified the 1993).
competency concept by emphasizing that stu-
dents need to develop not only cognitivecompe- 3. Authenticityis subjective,which makes stu-
dent perceptions important for authentic
tencies such as problem solving and critical
assessmentto influencelearning.
thinking, but also meta-cognitivecompetencies
such as reflection,and social competenciessuch These three elements led to the following gen-
as communicationand collaboration. eral framework (Figure 1) for the place of
The definition of authenticassessmentused in authenticassessmentin educationalpractices.
this study is: an assessment requiringstudents The concept of authenticachievement,as we
to use the same competencies,or combinations use it here, requiresa note of explanation.This
of knowledge, skills, and attitudes, that they article deals with authenticassessment in gen-
need to apply in the criterionsituationin profes- eral, regardlessof the level or field of endeavor.
sional life. The level of authenticityof an assess- This does not mean that we dismiss the concept
ment is thus defined by its degree of of authentic academicachievement (Newmann,
70 Vol. 52, No. 3
ETR&D,

Figure1 D General framework.

authentic authentic
instructon assessment

perceptionof
authenticity

winthina
student authenticlearning transfer sueS
winthina
student

authentic
achievement

1997),but ratherthat we see it as a specific sub- ment were distinguished: (a) the assessment
set within a specific field of endeavor, namely task, (b) the physical context, (c) the social con-
becoming an academic. In this we concur with text, (d) the assessment result or form, and (e)
Brown,Collinsand Duguid (1989)who, too, saw the assessment criteria.These dimensions can
authenticachievementto be morethanauthentic vary in their level of authenticity(i.e., they are
academicachievement. continuums).It is a misconceptionto think that
The following section discusses five dimen- something is either authentic or not authentic
sions (a theoreticalframework)that can vary in (Cronin, 1993; Newmann & Wehlage, 1993),
their degree of authenticityin determiningthe because the degree of authenticityis not solely a
authenticity of an assessment. The purpose of characteristicof the assessmentchosen;it needs
this frameworkis to shed light on in the concept to be defined in relationto the criterionsituaiton
of assessmentauthenticityand to provide guide- derivedfromprofessionalparctice.Forexample:
lines for implementing authenticity elements carryingout an assessmentin a teamis authentic
into competency-basedassessment. onlyif the chosen assessmenttask is also carried
out in a team in real life. The main point of the
framework is that each of the five dimensions
TOWARD
A FIVE-DIMENSIONAL can resemblethe criterionsituationto a varying
FRAMEWORKFORAUTHENTIC degree, thereby increasing or decreasing the
ASSESSMENT authenticityof the assessment.
Because authentic assessment should be
To define authenticassessment,we carriedout a aligned to authentic instruction (Biggs, 1996;
review of literatureon authenticassessment,on Van Merrienboer,1997),the five dimensionsof a
authenticity and assessmentin general,and on framework for authentic assessment are also
student perceptions of (authentic) assessment applicableto authenticinstruction.Even though
elements. Five dimensions of authenticassess- the focus of this article is on authentic assess-
A FIVE-DIMENSIONAL
FRAMEWORK
FORAUTHENTIC
ASSESSMENT 71

ment, an interpretationof the five dimensions in a conceptualizationof these five aspects as


for authenticinstructionis included in this arti- dimensions that can vary in their degree of
cle to show how the same dimensions can be authenticity.
used to create an alignment between authentic
instruction and authentic assessment. The
dimensions and the underlying elements of Task. An authentic task is a problem task that
authentic instruction as presented in Figure 2 confronts students with activities that are also
and Figure 3 do the same for authenticassess- carriedout in professionalpractice.The factthat
ment. an authentic task is crucial for an authentic
assessment is undisputed (Herrington &
As the figuresshow, learningand assessment
Herrington, 1998; Newmann, 1997; Wiggins,
tasks are a lot alike. This is logical, because the
1993), but different researchersstress different
learningtask stimulatesstudents to develop the elements of an authentic task. Our framework
competencies that professionals have and the defines an authentictaskas a task that resembles
assessment task asks students to demonstrate
the criteriontask with respectto the integration
these same competencies without additional
of knowledge, skills, and attitudes,its complex-
support (Van Merrianboer, 1997). Schnitzer
ity, and its ownership(see Kirschner,Martens,&
(1993) stressed that for authenticassessment to
Strijbos, 2004). Furthermore,the users of the
be effective, students need the opportunity to
assessment task should perceive the task,
practicewith the form of assessmentbefore it is
including above elements,as representative,rel-
used as an assessment. This implies that the
evant, and meaningful.
learning task must resemble the assessment
task, only with different underlying goals. An authenticassessmentrequiresstudentsto
Learningtasks are for learning,and assessment integrateknowledge, skills, and attitudesas pro-
tasks are for evaluating student levels of learn- fessionals do (Van Merrienboer,1997).Further-
ing in order to improve (formative),or in order more, the assessment task should resemblethe
to make decisions (summative).These models complexityof the criteriontask (Petraglia,1998;
show how a five-dimensional framework can Uhlenbeck,2002).This does not mean that every
deal with a (conceptual) alignment between assessment task should be very complex. Even
authenticinstructionand assessment.The inter- though most authentic problems are complex,
pretationand validation of the five dimensions involving multidisciplinarity,ill-structuredness,
for authentic assessment will be further and having multiple possible solutions
explained and examined in the rest of this arti- (Herrington & Herrington, 1998; Kirschner,
cle. 2002;Wiggins, 1993),real-lifeproblemscan also
be simple, well structured with one correct
answer, and requiringonly one discipline (Cro-
nin, 1993).The same need for resemblanceholds
An Argumentationforthe Five for ownershipof the task and of the process of
Dimensionsof AuthenticAssessment
developing a solution. Ownership for students
in the assessmenttask should resemblethe own-
As stated, there is confusion and there exist ershipforprofessionalsin the criteriontask.Sav-
many differences of opinions about what ery and Duffy (1995) argued that giving
authenticityof assessment really is, and which students ownership of the task and the process
assessment elements are importantfor authen- to develop a solution is crucialfor engaging stu-
ticity. To try to bring some clarity to this situa- dents in authenticlearningand problemsolving.
tion, the literaturewas reviewed to explicatethe On the other hand, in real life, assignmentsare
different ideas about authenticity. Many sub- often imposed by employers, and professionals
concepts and synonyms came to light, which often use standardtools and proceduresto solve
were conceptually analyzed and divided into a problem,both decreasingthe amount of own-
categories, resulting in five main aspects of ership for the employer.Therefore,the theoreti-
authenticity.The notion of authenticityas a con- cal framework argues that in order to make
tinuum (Newmann & Wehlage, 1993) resulted studentscompetentin dealing with professional
72 Vol. 52, No. 3
ETR&D,

Figure2 OI Five-dimensionalmodel for authentic instruction.

g
0)
u~ c
0
u,
r, C
O)
o-~*~g
'-(-'5o
r
)
0O a 0
0-

0> O
~~E5
(I 0P' OcL'u

0-
00
0
0)~E
V -
))
+o
a! 2 0)
L
mQ)[n
L
cnQ,

t+r 0 CL oB
c k
o
co O3~
+e ~ Q:
(b

I+ +' ;5 a, g~e~y
o
r~9
~cEl~
CeL

t E "
80
04"-
0.-I
K ~ *c

IO2
QO_
ix::
0 " Co
4>7J
Oc.-
~00
Cu'-
0
C0
m' a C
-0a E

~C
0,

0)'
7
0.0

cLo
0,

0 -~
0
~C:b

L
o0i
I CO)
.c =
O>a0 3

CL
EC.
C.). o
CE~o 4-- CL
0

0) u
S>0..-

00~

i>Cu'-'0
>0)0
2E
A FIVE-DIMENSIONAL
FRAMEWORK
FORAUTHENTIC
ASSESSMENT 73

Figure3 O Five-dimensionalmodel for authentic assessment.

E ~c

O
t... q)5C
C' Q~I L X ,

C.,Q 0 Uv G
E '

a c'
tO4 0)O- CpQ
2. U2 O
74 Vol. 52, No. 3
ETR&D,

problems, the assessment task should resemble contain relevant as well as irrelevantinforma-
the complexityand ownershiplevels of the real- tion (Herrington& Oliver),should resemblethe
life criterionsituation. resourcesavailablein the criterionsituation.For
Up to this point, task authenticityappearsto example,Resnick(1987)arguedthatmost school
be a fairly objectivedimension. This objectivity tests involve memorywork, while out-of-school
is confounded by Sambell, McDowell, and activitiesareoftenintimatelyengaged with tools
Brown (1997),who showed that it is crucialthat and resources (calculators,tables, standards),
studentsperceive a task as relevant,that (a) they making such school tests less authentic.Segers
see the link to a situation in the real world or et al. (1999)argued that it would be inauthentic
working situation;or (b)they regardit as a valu- to deprivestudentsof resources,becauseprofes-
able transferable skill. McDowell (1995) also sionals do rely on resources.Anotherimportant
stressedthat students should see a link between characteristiccrucialfor providing an authentic
the assessment task and their personalinterests physical contextis the timestudents are given to
before they perceive the task as meaningful. perform the assessment task (Wiggins, 1989).
Clearly,perceived relevanceor meaningfulness Tests are normally administeredin a restricted
will differfrom student to student and will pos- period of time, for example two hours, com-
sibly even change as students become more pletely devoted to the test. In real life, profes-
experienced. sional activities often involve more time
scatteredover days or, on the contrary,require
fast and immediate reaction in a split second.
Physicalcontext. Where we are, often if not Wiggins (1989) said that an authentic assess-
always, determineshow we do something,and ment should not rely on unrealisticand arbitrary
often the realplace is dirtier(literallyand figura-
time constraints.In sum, the level of authenticity
tively) than safe learning environments.Think, of the physical context is defined by the resem-
for example, of an assessmentfor auto mechan- blance of these elements to the criterionsitua-
ics for the military.The capacity of a soldier to tion.
find the problemin a nonfunctioningjeep canbe
assessed in a clean garage,with all the conceiv-
ably needed equipment available,but a future Socialcontext. Not only the physicalcontext,but
physical environment may possibly involve a also the social context,influencesthe authentic-
war zone, inclement weather conditions, less ity of the assessment. In real life, working
space,and less equipment.Eventhough the task together is often the rule ratherthan the excep-
itself is authentic,it can be questioned whether tion, and Resnick(1987)emphasized that learn-
assessing students in a clean and safe environ- ing and performingout of school mostly takes
ment really assesses their ability to wisely use place in a social system. Therefore,a model for
theircompetenciesin real-lifesituations. authenticassessmentshould considersocialpro-
The physical context of an authenticassess- cesses that arepresentin real-lifecontexts.What
ment should reflect the way knowledge, skills, is really importantin an authenticassessmentis
and attitudes will be used in professionalprac- that the social processes of the assessment
tice (Brown et al., 1989; Herrington & Oliver, resemble the social processes in an equivalent
2000).Fidelityis often used in the contextof com- situationin reality.At this point, this framework
puter simulations,which describehow closely a disagrees with literature on authentic assess-
simulation imitates reality (Alessi, 1988). ment that defines collaborationas a characteris-
Authentic assessment often deals with high- tic of authenticity (e.g., Herrington &
fidelity contexts. The presentation of material Herrington,1998).Ourframeworkarguesthat if
and the amount of detail presented in the con- the real situation demands collaboration,the
text are importantaspects of the degree of fidel- assessment should also involve collaboration,
ity. Likewise, an important element of the but if the situationis normallyhandled individ-
authenticity of the physical context is that the ually, the assessment should be individual.
number and kinds of resources available When the assessment requires collaboration,
(Segers,Dochy, & De Corte,1999),which mostly processes such as social interaction, positive
A FIVE-DIMENSIONAL
FRAMEWORK
FORAUTHENTIC
ASSESSMENT 75

interdependencyand individual accountability realistic outcome, explicating characteristicsor


need to be taken into account (Slavin, 1989). requirements of the product, performance,or
When, however, the assessment is individual, solutions that students need to create.Further-
the social contextshould stimulatesome kind of more, criteriaand standardsshould concernthe
competitionbetween learners. developmentof relevantprofessionalcompeten-
cies and should be based on criteriaused in the
real-lifesituation(Darling-Hammond&Snyder,
Assessmentresultorform. An assessmentinvolves 2000).
an assessment assignment (in a certainphysical
and social context) that leads to an assessment Besidesbasing the criteriaon the criterionsit-
uation in real life, criteriaof an authenticassess-
result, which is then evaluated against certain
assessment criteria(Moerkerke,Doorten, & de ment can also be based on the interpretationof
the otherfour dimensionsof the framework.For
Roode, 1999).The assessmentresultis relatedto
the kind and amountof output of the assessment example,if the physical context determinesthat
an authentic assessment of a competency
task, independent of the content of the assess-
ment. In the framework,an authenticresult or requires five hours, a criterionshould be that
formis characterizedby four elements.It should students need to produce the assessment result
within five hours. On the other hand, criteria
be a an (a) quality product or performancethat
students can be asked to produce in real life based on professionalpracticecan also guide the
(Wiggins, 1989). This product or performance interpretationof the other four dimensions of
authenticassessment.In otherwords, the frame-
should be a (b) demonstrationthatpermitsmak-
work argues for a reciprocal relationship
ing valid inferencesabout the underlying com-
between the criteriondimension and the other
petencies (Darling-Hammond& Snyder, 2000).
four dimensions.
Since the demonstrationof relevant competen-
cies is often not possible in one single test, an
authentic assessment should involve a (c) full
array of tasks and multiple indicatorsof learn- Some Considerations
ing in orderto come to fairconclusions(Darling-
Hammond & Snyder, 2000). Uhlenbeck (2002) What does all of this mean when teachers or
showed that a combinationof differentassess- instructionaldesigners try to develop authentic
ment methods adequately covered the whole assessments?Whatdo they need to consider?
range of professionalteachingbehavior.Finally, The first considerationdeals with predictive
students should (d) present their work to other
validity. If the educational goal of developing
people, either orally or in written form,because competent employees is pursued, then increas-
it is important that they defend their work to
ing the authenticity of an assessment will be
ensure that their apparent mastery is genuine valuable. More authenticityis likely to increase
(Wiggins,1989). the predictivevalidity of the assessmentbecause
of the resemblancebetween the assessmentand
Criteriaandstandards.Criteriaare those charac- real professionalpractice.However, one should
teristicsof the assessmentresultthat are valued; not throw the baby out with the bath water.
standardsare the level of performanceexpected Objectivetests are still very useful for certain
from various grades and ages of students (Arter purposesas high-stakessummativeassessments
& Spandel, 1992). Setting criteria and making on individual achievement, where predicting
them explicitand transparentto learnersbefore- student abilityto functioncompetentlyin future
hand is important in authentic assessment, professionalpracticeis not the purpose.
because this guides learning (Sluijsmans,2002) Anotherconsiderationin designing authentic
and, after all, in real life, employees usually assessmentis thatwe should not lose sight of the
know on what criteriatheir performanceswill educational level of the learners. Lower-level
be judged. This implies that authentic assess- learnersmay not be able to deal with the authen-
ment requires criterion-referencedjudgment. ticityof a real,complex,professionalsituation.If
Moreover,some criteriashould be related to a they are forced to do this, it may result in cogni-
76 Vol.52, No. 3
ETR&D,

tive overload and, in turn, have a negative the authenticity dimensions differed between
impact on learning (Sweller,Van Merrienboer, students and teachers and between students
& Paas, 1998). As a result, a criterionsituation with different amounts of practicaland educa-
will often need to be an abstractionof real pro- tional experience.The differencesand similari-
fessional practice in order to be attainable for ties along a limited number of dimensions can
students at a certaineducationallevel. The ques- give insight into what is crucialfor defining and
tion that immediatelycomes to mind in this con- designing authenticassessments.
text is How do you create an authentic
assessmentfor studentswho arenot preparedto
function as beginning professionals? The METHOD
answer is that the authenticityof an assessment
should be defined by its degree of resemblance Participants
to the criterionsituation(i.e.,an abstractionfrom
professionalpractice)and not necessarilyto real Students and teachers from a nursing college
professional practice. Van Merrianboer(1997) took part in this study. One session of the study
argued that an abstractionof real professional involved only teachers, one session involved
practice (i.e., the criterionsituation)can still be
sophomorestudents (secondyear), and one ses-
authentic as long as the abstracted situation
sion involved senior students (fourthyear). The
requiresstudents to performthe whole compe- student groups could be furtherdivided into a
tency as an integratedwhole of constituentcom-
group of students studying nursing in a voca-
petencies. The abstraction results from tional training program (VTP)where they are
simplifying contextual factors that complicate primarily in school and make use of short
the performanceof the whole competency.
internships,and a group that studied nursing in
A third consideration also sheds a light on a block release program (BRP)where learning
the question stated in the previous sections, and working are integrated on an almost daily
namely the subjectivityof authenticity.The per- basis. This resulted in five groups of partici-
ception of what authenticityis may change as a pants: (a) 8 sophomore VTPstudents (M age =
resultof educationallevel, personalinterest,age, 18.5 years), (b) 8 sophomore BRP students (M
or amount of practicalexperiencewith profes- age = 20.9 years), (c) 8 senior VTP students (M
sional practice (Honebein et al., 1993). This age = 19.7 years), (d) 4 senior BRPstudents (M
implies that the five dimensionsthat are argued age = 31.4 years), and (e) 11 teachers (M age =
in the framework for authenticassessment are 42.8 years). The numberof participantsper ses-
not absolute but, rather,variable.It is possible sion was limited because of the practicalpossi-
that assessing professional competence of stu- bilities of the group support system used in this
dents in their final year of study, when they study.
have often served internshipsand have a better
idea of professional practice, requires more
authenticityof the physical context than when Materials
assessing first year students, who usually or
often have little practicalexperience.Designers An electronicgroup supportsystem (GSS)at the
must take changing student perspectives into
Open Universityof the Netherlandswas used as
accountwhen designing authenticassessment. researchtool. A GSSis a computer-basedinfor-
The qualitativestudy describedin the rest of mation processing system designed to facilitate
this articlehas two main goals. First,it explores group decision making. It is centered on group
whether our five-dimensionalframeworkcom- productivity through idea generation, prefer-
pletely describesauthenticityor whetherimpor- ence, and opinion exchange of people involved
tant elements may be missing. Second, it in a common task in a sharedenvironment.The
explores the relative importance of the five GSS allows collaborativeand individual activi-
dimensions. A subgoal of this study was to ties such as brainstorming,idea generation,sort-
explore if the perceptionof (the importanceof) ing, rating, and clustering via computer
A FIVE-DIMENSIONAL
FRAMEWORK
FORAUTHENTIC
ASSESSMENT 77

communication. To prevent participants(espe- because of the differencesin their studies, they


cially students) from feeling inhibited in would have differentperceptionsof what deter-
expressing their ideas and opinions, the GSS mines authenticity.VTPstudents,BRPstudents,
was a good optionbecauseit is completelyanon- and teachers were randomly divided in two
ymous. Furthermore,it was a practicaland valu- halves, one thatreceivedCasesABCDin the pre-
able method because it made it possible to test and EFGH in the posttest, and one that
collecta lot of informationin a structuredway in receivedthe cases in the reverseorder.
a shortperiod of time. After the initial rating of the case descrip-
To examine the relative importance of the tions, the participants were appraised of the
five dimensions,fourcase descriptionsof assess- purposeof the study. In orderto createa specific
ments thatvariedin theiramountof authenticity frame of mind, a very general descriptionwas
based on the five dimensions of the model were given of the term authenticity(i.e., true to life).
designed. They described competencies from TheGSSpartof the study consistedof four activ-
the nursingcompetencyprofile,which were val- ities. The first activity requiredthe participants
idated by two employees of the nursing college. to enter into the system their own statements
To check the influence of the GSS session itself that described authenticity of an assessment.
on the perceptions of the authenticity of the This was a free brainstorm, and participants
cases, the descriptionswere used in a pre- and a were encouraged to generate as many state-
posttest. To do this, a second set of differentbut ments as possible. Statements were anony-
comparable case descriptions was designed, mously entered into the GSS,where it was also
which resultedin two sets of four cases. CasesA possible to respond to statementsmade by oth-
and E were completely authenticexcept for the ers. After this electronicbrainstorm,the contri-
task; Cases B and F were completely authentic butions were discussed in orderto clarifythem.
except for the physical context;Cases C and G This was recordedfor lateruse and analysis.
were completely authenticexcept for the result The second activity requiredrespondents to
or form; and Cases D and H were completely specify (voting is a featureof a GSS)the 10 most
authentic(see Appendix for a full descriptionof important statements for designing authentic
a completelyauthenticcase description). assessments that were generated during the
brainstorm.The purpose of this activity was to
Procedure determinewhich elements the participantsper-
ceived as being especiallyimportantfor authen-
All participantshad access to a GSScomputer. tic assessment. After completing these two
During a two-hour session, participantscarried activities, a prototype five-dimensionalframe-
out both individual and collaborativeactivities. work for authenticassessmentwas presentedas
a framework for assessing professionalbehav-
At the beginning and end of the GSSsession,
ior. The five dimensions were explained to the
participantswere presented four case descrip-
participants in an attempt to create mutual
tions (ABCDor EFGH).In six paired compari-
understandingaboutthe meaning of the dimen-
sons (4 x 3/2), they chose the case that they sions. Thefive dimensionswere characterizedas
considered to be a more authentic assessment.
follows:
Thisactivitywas meantto determinethe relative
importance of the different dimensions of 1. Task:Whatdo you have to do?
authenticassessmentin the eyes of the different 2. Physicalcontext:Wheredo you have to do it?
groups of participants. A second underlying 3. Socialcontext:Withwhom do you have to do
purpose of this activitywas to bringparticipants it?
in a specific referenceframe for the rest of the 4. Result or form: What has to come out of it?
session, and to focus their thinking toward
Whatis the resultof your efforts?
authenticityof assessmentinsteadof assessment
in general. 5. Criteria:How does what you have done have
A distinction was made between VTP stu- to be evaluatedor judged?
dents and BRP students; it was possible that The third and fourth activities consisted of
78 ETR&D,
Vol.52, No. 3

paired comparisons to determine the relative The paired comparison data of the five
importance of the dimensions. Activity three dimensions, that is, the number of times that a
consisted of 10 paired comparisonsof the five dimension in the paired comparisonswas rated
dimensions (5 x 4/2). Participantshad to choose as more important than another dimension,
the dimensions of the frameworkthat they per- were talliedper participantgroup. The absolute
ceived to be more important for authentic scores were then translatedinto rankings. The
assessment.The fourth activitywas the same as paired comparisons of the case descriptions
the activity at the beginning of the experiment: were analyzedin the same way.
The participantswere again required to carry
out pairedcomparisonsof case descriptionsthat
varied in theiramountof authenticityaccording RESULTS
to the five-dimensionalframework.Eachgroup
receivedthe counterbalancedset of case descrip- In general, the task, the result or form, and the
tions to those compared at the beginning of the criteria were rated as most important for the
experiment. authenticity of the assessment. The social con-
text was clearlyconsideredto be least important
for authenticity,and the importanceof the phys-
Analysis
ical contextwas stronglydiscussed.
A characteristicof the GSS is that the answers,
statements, choices, and so forth, of each indi-
vidual participantare anonymous. This means The RelativeImportance of the Five
that scores per participantwere not available, Dimensions:PairedComparisons
which precluded the possibility of carryingout
statisticaltests. On the otherhand, the anonym- Thepairedcomparisonsof the dimensionsand of
ity inhibitedsocially acceptedansweringbehav- the case descriptionsgave insightinto the relative
ior, and has been shown to stimulateresponsein importanceof the five dimensionsfor designing
idea generation and increase the reliability of authenticassessments.The comparisonsof the
answers.The data, thus, were qualitativelyana- dimensionsresultedin five rankings(sophomore
lyzed. The tapes of the discussions were tran- VTPstudents,sophomoreBRPstudents,teachers,
scribed. Both discussion statements and the senior VTP students, and senior BRP students)
statements keyed in during the brainstorms from 1 to 5. The pairedcomparisonsof the case
were analyzed to discern which of the five descriptionswere analyzedfor the same groups,
dimensions of the framework they fit. State- but were measuredin pre- and posttests,which
ments that did not fit were classifiedas other. resultedin ten rankingsfrom1 to 4.

Table 1 D Rankingof dimensionsby group.

Physical Social Result


Task context context orform Criteria

SophomoreVTPstudents 2.0 4.5 4.5 1.0 3.0


SophomoreBRPstudents 1.0 3.5 5.0 3.5 2.0
Teachers 1.0 4.0 5.0 2.0 3.0
SeniorVTPstudents 2.0 5.0 3.5 3.5 1.0
Senior BRP students 2.0 4.0 5.0 1.0 3.0

Total 8.0 21.0 23.0 11.0 12.0

Note. 1 = most important,5 = least important


A FIVE-DIMENSIONAL
FRAMEWORK
FORAUTHENTIC
ASSESSMENT 79

Table1 shows rankingsper group of the five dimensions was perceived as most authentic
dimensions, based on their perceived impor- (score 1) by all, except the senior BRPstudents
tancein providing authenticityto an assessment on the posttest (score2.5). The other three kinds
(1 = most important, 5 = least important).Table 1 of cases showed an interestingpattern.The case
shows that all groups perceived the task as that was authenticexcept for the task received
important (score 1 or 2), and all groups except mostly a score of 2, which meant that this case
the senior VTP students (score 3.5), perceived was perceived as relatively authentic,which in
the social contextas the leastimportant.Further- turn meant that the task (which was not authen-
more, the result or form and criteriondimen- tic in this case) was notperceivedas very impor-
sions received more than average importance, tant in designing an authenticassessment.This
whereas all groups perceived the physical con- is contraryto the findings of the paired compar-
text as relativelyunimportant(scoreabout4). In isons of the dimensions in which the task was
short, independent of the group (see totals in perceived as very important in providing
Table 1), the task was perceived as most impor- authenticityto an assessment.Finally,the partic-
tant,followed by the resultor formand criterion ipant groups disagreedaboutthe authenticityof
dimensions;the physical context and especially the remaining two kinds of cases. All
the social contextlagged farbehind.
sophomore students ranked the case that was
The results of the paired comparisonsof the authenticexpectfor the resultas 4, meaningthat
case descriptions,in pre-and posttests,also gave they perceivedthis case to be the least authentic.
insight into the relative importance of the In otherwords, they perceivedthe resultor form
dimensions. Table2 shows rankingsper group dimension as most importantfor designing an
of the four case descriptions. authentic assessment. Teachers, on the other
A 1 meant that this case was perceived as the hand, rankedthe case that was authenticexcept
most authenticcase descriptionand a 4 referred for physical context as the least authenticcase
to the least authentic case description. An (score 4), which meant that teachers perceived
importantfinding, for the framework,was that physical context to be most important in
the case that described a completely authentic designing an authentic assessment. Senior stu-
assessment based on the presence of all five dents did not appear to differentiate,meaning

Table 2 O Rankingof case descriptionsby group.


All authentic All authentic
All authentic exceptfor the exceptfor the
exceptfor thetask physicalcontext resultorform All authentic

Sophomore VTP, pretest 2.0 3.0 4.0 1.0


SophomoreBRP,pretest 2.0 3.0 4.0 1.0
SophomoreVTP,posttest 3.0 2.0 4.0 1.0
Sophomore BRP, posttest 2.0 3.0 4.0 1.0

Teacherspretest 3.0 4.0 2.0 1.0


Teachers
posttest 2.0 4.0 3.0 1.0

SeniorVTPpretest 2.0 3.5 3.5 1.0


SeniorBRPpretest 2.0 3.5 3.5 1.0
SeniorVTPposttest 2.0 3.5 3.5 1.0
SeniorBRPposttest 1.0 4.0 2.5 2.5

Note.1 = mostauthentic,4 = leastauthentic


80 Vol.52, No. 3
ETR&D,

that they perceived the cases with no authentic should be real professionalpracticeor a simula-
physical context or with no authenticresult or tion in school.
form as equally inauthentic(score3.5). To sum, A closerlook at the contentof the brainstorm
the findings of the paired comparisonsof the
statements gave the impression that teachers
case descriptionsindicatedthat when all of the
and seniors agreed more with each other and
dimensions in the frameworkare present in a
with the idea of the framework,than with the
case, the case was unequivocally seen as most
sophomorestudents,especiallywhen it came to
authentic.In addition,thereappearto be contra- task and result or form dimensions. Teachers
dictory results with respect to task authenticity and seniors agreed with the frameworkthat an
compared to the results of the paired compari- authentictask requiredan integrationof profes-
sons of the dimensions. Finally, when evaluat-
sional knowledge, skills, and attitudes,and they
ing assessment cases, teachers and students
acknowledged that the task should resemble
appear to differ with respect to the importance real-life complexity. On the other hand,
of the authenticity of physical context versus
sophomore students were preoccupied with
resultauthenticity.
knowledge testing,they had problemspicturing
the idea of integratedtesting, and were primar-
ily concerned with making assessment easier
Completeness and Relative and clearer (e.g., "assignments should be less
Importance:What Do ParticipantsSay? vague, not more than one answershould be pos-
sible") instead of simulating real-world com-
Table 3 shows that all dimensions received plexity.In the resultor formdimension,teachers
attention in the brainstormand discussion ses- and seniors agreed that more assessment
sions. Furthermore,these results corroborated moments and methods should be combinedfor
the earlier findings, in that social context a fairerand more authenticpictureof students'
receivedthe least attentionin all groups.Besides professional competence. Sophomores did not
the five dimensions, almost all subelements of discuss the resultor form dimensionmuch;they
the dimensions, described in the framework, only mentioned that reshaping currenttests in
were reviewed. the form of cases would make them more realis-
Based on the number of statementsand the tic. In other words, every kind of assessment
could be made more authenticby adding realis-
ratios of the statementscomparedto each other,
tic information.
as shown in Table3, sophomoresplace primary
interest on task, followed by physical context. A specificationof the otherstatements (see
Seniors and teachers place equal emphasis on Table 4) showed, first, that all groups made
task and result. Teachers differ from all stu- statementsemphasizingthe alignmentbetween
dents, regardless of level, with respect to the instructionand assessment,and between school
emphasis on physical context.Teachersdevoted and real-lifepractice.This is in agreementwith
a lot of time to discussing the requiredfidelity the theoreticalideas behind the frameworkfor
level of the physical context in an effective authentic assessment. Second, Table 4 shows
authentic assessment. Especially emphasized that issues concerningthe assessorof an authen-
was the questionof whetherthe physicalcontext tic assessment, and organizational or pre-

Table 3 O Numberof statements per dimensionof each group,

Physical Social Result


Task context context orForm Criteria Other

Sophomore students 24 19 6 7 13 45
Senior students 34 21 9 36 12 26
Teachers 16 39 5 19 21 56
A FIVE-DIMENSIONAL
FRAMEWORK
FORAUTHENTIC
ASSESSMENT 81

Table4 O Variablesin the other category, per group.

students Seniorstudents
Sophomore Teachers

Generalstatementsapplicableto all five dimensions 6 1 2


Instruction 28 7 5
Alignmentinstruction-assessment 2 3 3
Alignmentschool-practice 6 3 3
Assessor 3 3 6
Organizationorpreconditions - - 7
Influenceon the learningprocess - - 4
Not defined or nonsense - 9 26

conditional issues, should be taken into account A combination of the results of the GSS activ-
in a framework for authentic assessment. Issues ities led to the conclusion that task, result or
related to the assessor dealt with the realization form, and criteria were perceived as very impor-
that people from professional practice should be tant for authentic assessment. Physical context
involved in defining and using criteria and stan- was most important in the eyes of teachers.
dards. Organizational issues involved state- Social context was perceived as the least impor-
ments about conditions that should be met before tant dimension.
authentic assessment can be implemented in Furthermore, not all groups perceived the
school. For example, teachers talked about plac- dimensions and elements in the same way.
ing students in professional practice sooner and Teachers and seniors mostly agreed with each
more often for the purpose of assessing them in other and with the theoretical framework; how-
this professional context. Finally, Table 4 shows ever, sophomores often deviated from the other
that sophomores took the opportunity to talk groups. There were no differences between VTP
and complain about the instruction. Although and BRP students.
instruction was not being evaluated (i.e., it was
about assessment), 28 statements dealt with
what was taught and not with what was
DISCUSSION
assessed. Seniors were more focused, and
teacher statements were spread over different
other variables and the 26 statement of the not To reiterate: The two questions with which we
defined variable included mostly jokes or ques- began were (a) Is the framework complete? (b)
tions they asked each other. Do students differ from teachers with respect to
what they perceive as important for authentic-
ity? Both of these questions shed light on possi-
ble guidelines for designing authentic
CONCLUSION assessments.
The answer to Question 1 appears to be yes.
Overall, the five-dimensional framework gave a The five dimensions appear to adequately
good description of what dimensions and ele- define authenticity, as demonstrated by both the
ments should be taken into account in an brainstorming and the high ranking of those
authentic assessment; the participants discussed cases that were authentic on all five dimensions.
all dimensions and almost all elements The adequacy of the framework is corroborated
described in the framework. However, elements by the finding that during the brainstorming,
concerning the assessor and organization issues most subelements of the dimensions as de-
should be added to complete the framework, as scribed by the framework were seen as impor-
these elements turned out to be important to all tant when designing authentic assessment. The
participant groups. paired comparisons showed some subtle differ-
82 Vol. 52, No. 3
ETR&D,

ences in the importanceof the five dimensions their perception of authenticity,some interest-
for providing authenticity.While the task, the ing findings came to light. The most differences
result or form, and the criterion dimensions were found between sophomores and teachers,
turnedout to be very importantfor authenticity, while seniors agreed with teachersmore often.
the physical context and especially the social Moreover, the perceptions of teachers and
context dimensions were perceived as less seniors agreed more with the ideas of the theo-
important.Social context is unequivocallyper- retical framework.Possibly, the perceptions of
ceived as the least important dimension of older students have changed during their col-
authenticity. All groups stressed the need for lege careeras a result of having had experience
individual testing, although both students and with professional practice; the perceptions of
teachersstressed that most nursing activitiesin sophomores-who have less practical experi-
real life are collaborative.Teachers explained ence-seemed to be based primarily on their
that "assessing in groups is a soft spot, we just previous experiences with assessment, which
don't know how to assess students together, explainedthe focus on knowledge and in-school
because at the end we want to be sure that every testing. In other words, it appears that
individual student is competent."It should not sophomore students have differentconceptions
be concluded, based on these findings, that and possibly misconceptionsof realprofessional
social context is not important for authentic practiceand, thus, authenticityof assessment.
assessment, but if choices have to be made in Furthermore, the brainstorming and the
designing an authentic assessment, social con- paired comparisons of the case descriptions
text is probablythe first dimensionto leave out. showed differencesbetween teachers and stu-
The findings on importanceof task are some- dents in the perception of physical context.
times contradictory.Although the brainstorm- Teachersfocused on the importanceof increas-
ing and the paired comparisons of the ing the authenticityof physical contextby plac-
dimensions show that task was perceived as ing the assessment in professional practice,
very importantby all, the pairedcomparisonsof whereas students, especially sophomores,
the cases made task seem less important.It is mostly focused on in-school testing with, for
possible, thus, that although the respondents example,simulatedpatientsand realisticequip-
consider task (as an abstractedconcept) to be ment.
most important,they arenot ableto identify(i.e., Finally, all groups agreed on the relative
they do not perceive)an authentictask. A possi- unimportanceof the social context and on the
ble explanationfor this is that the all-authentic- importance of using criteriathat resemble the
except-for-the-task case resembles current criteriaused in realprofessionalpractice.Teach-
assessment practices. Because previous experi- ers and students agree that, at this point, the cri-
ences are found to strongly influence percep- teriaused in school differtoo much fromcriteria
tions (Birenbaum,2003),the familiarityof these used in professionalinstitutes, and that school
cases may have influenced the paired compari- criteriaare often unknown or misinterpretedby
sons of the cases. If this is the case, the paired assessorsat the professionalinstitutes.
comparisonsof the five dimensionswere proba-
bly a more objectivemeasureof the importance
of the five dimensions. FutureImplications

Finally, it might be the case that assessor- The findings of the study allow for some critical
related issues would complete the framework.
questions and guidelines concerningthe design
Thiscould be done by adding a sixth dimensions of authentic assessment. First, student percep-
called "the assessor," or by adding the issues tions should be consideredin designingeffective
concerning who should use and develop authenticassessments.The qualitativeresults of
authenticcriteriaand standardsas subelements
this study showed that students, especially at
to the criteriondimension.
the beginning of theirstudy and with little prac-
With respect to Question 2, concerning the tical experience, have different conceptions
differences between students and teachers in (possibly misconceptions)of what authenticity
FRAMEWORK
A FIVE-DIMENSIONAL FORAUTHENTIC
ASSESSMENT 83

meansthando older,moreexperiencedstudents Finally,as stated at the beginning of this arti-


and teachers.Forauthenticassessmentto work, cle, authenticityis only one of the elements and
two options need to be considered:either (a) the quality criteria of competency-based (alterna-
assessment should meet the expectationsof the tive) assessment (Birenbaum& Dochy, 1996;
sophomores,for example,by stickingto explicit Dierick,Dochy, & Van de Watering,2001).Mak-
knowledge testing in the name of authentic ing decisions aboutimplementingauthenticele-
assessment,which is likely to confirmunwanted ments in an assessmentshould be consideredin
learning behavior; or (b) explicit attention the broadercontextof qualitycriteriafor assess-
should be given to changingstudentperceptions ment (i.e., reliabilityor generalizability),and in
and, thereby,opening the possibilitiesto change the contextof otherassessmentgoals (i.e.,timeli-
their learning behavior toward professional ness, affordability,and accountability).How-
competency development, when implementing ever, a thorough discussion of these other
authenticassessment. assessment goals and criteria is beyond the
Second, we might be able to save precious scope of this article.
time and money in the design, developmentand The argumentationof the theoreticalframe-
work and the qualitativestudy gave some inter-
implementation of authentic assessment with
respect to the physical context and the creation esting impulses to further theoretical and
of social contexts. Researchshould examine if practical research concerning authentic assess-
ments and student perceptions,and especially
assessing students in a real professionalcontext
has additionalvalue for students,or if assessing the focus on vocational college is interesting,
in an (electronic)simulationin school is authen- because most assessment research is done in
tic enough as long as students are confronted higher education. All participantsin this study
with an authentic task, result or form, and cri- agreedthatinstructionand assessmentin school
teria. Simulation in school, virtual or not, is should be aligned with each other and that
probably easier and less expensive to imple- developing educationthat focuses on the devel-
ment, and, therefore,warrantscarefulconsider- opment of competenciesand takes professional
ation. practiceas a startingpoint, requiresassessments
that are also competency based and based on
The exploratorynatureof this study, without
professionalpractice.In otherwords, it requires
the possibilityof quantitativestatisticalanalyses authenticassessment. O
owing to the natureof the GSS,makes firm con-
clusions impossible. However, the electronic
GSSefficientlydelivereda lot of qualitativedata JudithT. M. Gulikers[judith.gulikers@ou.nl], TheoJ.
in a short period of time. What the data of this Bastiaens,and PaulA. Kirschnerarewith the
EducationalTechnologyExpertiseCenterat the
study do show is that authenticityis definitely a
OpenUniversityof the Netherlands,P.O.Box 2960,
multifacetedconcept, and that a number of the 6401DLHeerlen,TheNetherlands.
facets (dimensions)appearto be of more impor- The authorswould like to thankMarijkeBijnensfor
tance than others. This can have far-reaching her help in organizingthe participationof teachers
implicationsfor educationaldesign. and studentsin the GSS.Theywould also like to
thankDr. RobertSchuwerfor his assistancein setting
Theactualeffectivenessof this frameworkfor up and carryingout the GSSsessions.
designing authentic assessments, however,
should be examined by evaluating the influ-
ences of differentkinds and levels of authentic- REFERENCES
ity of assessment on student learning and
motivation. Becauseimplementing authenticity Alessi, S. M. (1988).Fidelity in the design of instruc-
elements in assessment requires a lot of time, tionalsimulations.JournalofComputer-Based Instruc-
tion,15(2),40-47.
money, and energy (Martens, Bastiaens, &
Arter,J.A., & Spandel,V. (1992).An NCMEinstruc-
Gulikers,2002),researchshould examinewhich
tionalmodule on:Using portfolioof studentwork in
elements of the frameworkare crucialfor affect- instruction and assessment. EducationalMeasure-
ing student learning in the direction of the ment:IssuesandPractice,11(1),36-45.
developmentof professionalcompetencies. Biggs,J. (1996).Enhancingteachingthroughconstruc-
84 Vol. 52, No. 3
ETR&D,

tive alignment.HigherEducation, 32,347-364. Constructivismand the design of learningenviron-


Birenbaum,M. (1996).Assessment 2000:Towards a ments:Contextand authenticactivitiesfor learning.
pluralisticapproachto assessment.InM.Birenbaum In T. M. Duffy, J.Lowyck,& D. H. Jonassen(Eds.),
& F. J.R. C. Dochy (Eds.),Alternativesin assessmentof Desingingenvironments for constructivelearning(pp.
achievements, learningprocessesand priorknowledge 88-108).Berlin:Springer-Verslag.
(pp. 3-29). Boston,MA:KluwerAcademicPublish- Huang, H.M. (2002). Towards constructivism for
ers. adultlearnersin onlinelearningenvironments.Brit-
Birenbaum,M. (2003).New insightsinto learningand ishJournalof Educational Technology, 33, 27-37.
teachingand theirimplicationsforassessment.InM. Kirschner,P. A. (2002).Threeworlds of CSCL:Canwe
Segers, F. Dochy, & E. Cascallar(Eds.), Optimising support CSCL?Heerlen: Open University of the
new modesof assessment: In searchof qualityandstan- Netherlands.
dards (pp. 13-36). Dordrecht, The Netherlands: Kirschner,P. A., Martens,R. L.,& Strijbos,J.W. (2004).
KluwerAcademicPublishers. CSCL in higher education? A framework for
Birenbaum,M., & Dochy,F. J.R. C. (1996).Alternatives designing multiple collaborativeenvironments.In
in assessmentof achievements, learningprocessesand P. Dillenbourg (Series Ed.) & J.W. Strijbos,P.A.
priorknowledge. Boston,MA:KluwerAcademicPub- Kirschner&R. L.Martens(Vol.Eds.),Computer-sup-
lishers. portedcollaborative learning:Vol. 3. Whatwe know
Brown,J.S., Collins,A., & Duguid, P. (1989).Situated aboutCSCL_. Andimplementing it in highereducation
cognition and the culture of learning. Educational (pp. 3-30). Boston,MA:KluwerAcademicPublish-
Researcher, 18(1),32-42. ers
Cronin, J.F. (1993). Four misconceptions about Martens,R., Bastians,Th., & Gulikers,J. (2002).Leren
authenticlearning.Educational Leadership,50(7),78- met computergebaseerde authentieke taken:
80. motivatie, gedrag en resultaten van studenten
Darling-Hammond,L., & Snyder,J. (2000).Authentic [Learningwith computer-basedauthentictasks:stu-
assessment in teaching in context. Teachingand dent motivation,behaviorand results].Pedagogische
Teacher Education,16,523-545. Studien,79(6),469-482.
Dierick,S., Dochy, F., & Van de Watering,G. (2001). McDowell,L. (1995).The impactof innovativeassess-
Assessment in het hoger onderwijs: Over de ment on student learning.Innovationsin Education
implicaties van nieuwe toetsvormen voor de andTrainingInternational, 32(4),302-313.
edumetrie.[Assessmentin highereducation:About Meyer, C. (1992). What's the difference between
the implications of new assessment forms for authentic and performanceassesment?Educational
edumetrics]Tijdschrift voorHogerOnderwijs, 19(1),2- Leadership,49(8),39-40.
18. Moerkerke,G., Doorten,M., & de Roode,F. A. (1999).
Dochy, F. J.R. C., & McDowell,L. (1998).Assessment Constructievan toetsenvoorcompetentiegericht curric-
as a tool for learning.Studiesin Educational Evalua- ula [Construction of assessments for competency-based
tion,23(4),279-298. curricula](OTEC report1999/W02).Heerlen, The
Frederiksen,N. (1984).The realtest bias, influencesof Netherlands:Open UniversiteitNederland,Educa-
testing and teachingon learning.American Psycholo- tionalTechnologyExpertiseCenter.
gist, 39(3),193-202. Newmann, F.M. (1997). Authentic assessment in
Gibbs,G. (1992).Improvingthequalityof studentlearn- social studies: Standards and examples. In G.D.
ing.Bristol,UK:Technicaland EducationalServices. Phye (Ed.).Handbook of classroomassessment: Learn-
Gielen,S.,Dochy,F.,&Dierick,S. (2003).Theinfluence ing, achievement,and adjustment(pp. 359-380). San
of assessmenton learning.In M. Segers,F. Dochy,& Diego, CA:AcademicPress.
E. Cascallar(Eds.), Optimisingnew modesof assess- Newmann, F. M., & Wehlage,G. G. (1993).Five stan-
ment:In searchof qualityand standards(pp. 37-54). dards for authenticinstruction.Educational Leader-
Dordrecht, The Netherlands: Kluwer Academic ship,50(7),8-12.
Publishers. Nicaise,M.,Gibney,T.,&Crane,M. (2000).Towardan
Hart, D. (1994).Authenticassessment:A handbook for understandingof authenticlearning:Student per-
education.Menlo Park, CA: Addison-Wesley Pub- ceptionsof an authenticclassroom.Journalof Science
lishingCompany. Education andTechnology, 9, 79-94.
Herrington, J., & Herrington,A. (1998). Authentic Petraglia,J. (1998).Realityby design:Therhetoricand
assessment and multimedia:How university stu- technologyof authenticityin education.Mahwah,NJ:
dents respond to a model of authenticassessment. LawrenceErlbaumAssociatesPublishers.
Higher EducationalResearch& Development,17(3), Prodromou,L. (1995).Thebackwasheffect:Fromtest-
305-322. ing to teaching.ELTJournal,49(1),13-25.
Herrington,J., & Oliver, R. (2000).An instructional Reeves, T. C., & Okey,J.R. (1996).Alternativeassess-
design framework for authenticlearning environ- ment for constructivistlearning environments.In
ments. EducationalTechnology Research and Develop- B.G. Wilson (Ed.). Constructivistlearningenviron-
ment,48(3),23-48. ments:Casestudiesin instructionaldesign(pp. 191-
Honebein, P. C., Duffy, T. M., & Fishman,B.J. (1993). 202).EnglewoodCliffs,NJ:EducationalTechnology
A FIVE-DIMENSIONAL
FRAMEWORK
FORAUTHENTIC
ASSESSMENT 85

Publications. Research, 2, 191-213.


Resnick, L. B. (1987). Learning in school and out. Edu- Slavin, R. E. (1989).Researchon cooperativelearning:
cationalLeadership,16(9), 13-20. An internationalperspective.Journalof Educational
Sambell, K., & McDowell, L. (1998). The construction Research, 33,231-243.
of the hidden curriculum: Messages and meanings
Sluijsmans,D. (2002).Studentinvolvementin assessment:
in the assessment of student learning. Assessmentand
thetrainingofpeerassessmentskills.Unpublisheddoc-
Evaluationin Higher Education,23(4), 391-402. toral dissertation,Open University of the Nether-
Sambell,K.,McDowell,L., & Brown,S. (1997).Butis it
lands,Heerlen,TheNetherlands.
fair?:An exploratorystudy of studentperceptionsof
the consequentialvalidity of assessments.Studiesin Sweller,J.,Van Merrianboer, J.J.G., & Paas,F. (1998).
Educational Evaluation,23(4),349-371. Cognitive architecture and instructional design.
Savery,J.,& Duffy, T. (1995).Problembased learning: Educational PsychologyReview,10(3),251-296.
An instructionalmodel and its constructivistframe- Torrance,H. (1995). Evaluatingauthenticassessment.
work. EducationalTechnology, 35, 31-38. Buckingham,UK:OpenUniversityPress.
Schnitzer,S. (1993).Designing and authenticassess- Uhlenbeck,A. (2002).Thedevelopment of an assessment
ment. EducationalLeadership, 50(7),32-35. procedure for beginningteachersof Englishas aforeign
Segers, M., Dierick, S., & Dochy, F. (2001).Quality language.Unpublished doctoral dissertation,Uni-
standardsfor new modes of assessment.An explor- versity of Leiden,Leiden,TheNetherlands.
atory study of the consequentialvalidity of the VanMerriinboer,J.J. G. (1997).Trainingcomplexcogni-
OverAlltest. European JournalofPsychology of Educa- tiveskills:Afour-component instructionaldesignmodel
tion,16(4),569-586.
for technicaltraining.Englewood Cliffs, NJ:Educa-
Segers,M.,Dochy,F.,&Cascallar,E. (2003).Optimising tionalTechnologyPublications
newmodesofassessment: In searchofqualitiesandstan-
dards.Dordrecht, The Netherlands:Kluwer Aca- Wiggins, G. (1989).Teachingto the (authentic)test.
demicPublishers. Educational 46(7),41-47.
Leadership,
Segers, M., Dochy, F., & De Corte,E. (1999).Assess- Wiggins, G. P. (1993). Assessingstudentperformance:
ment practicesand students'knowledge profilesin Exploringthepurposeand limitsof testing.San Fran-
a problem-basedcurriculum.LearningEnvironments cisco,CA:Jossey-Bass/Pfeiffer.

See Appendix,
overleaf
86 Vol.52, No. 3
ETR&D,

Appendix D Authentic case description:Authenticon five dimensions.


You'reworkingas an internin a nursinghome. Duringyour internshipyou have to show thatyou are
capableof providingbasiccareto geriatricpatientswith differentkinds of problems.You have to be able to
help them with theirown personalcare,such as washing themselves,gettingdressed,and combingtheir
hair.In school you learnedwhat basic careis, the carethatyou have to be able to provideto differentkinds of
geriatricpatientsin a nursinghome, how you should do this, and what you should takeinto accountwhile
doing this (e.g., the differentkinds of disabilitiesor dysfunctions,privacyissues, and rules).In otherwords,
you know which criteriawill be used to judge your competencein providingthis basiccareto the geriatric
patients.Tojudge whetheryou arecompetent,your mentorat the nursinghome observesyou while you are
takingcareof threepatients,eachwith differentproblems.The firstpatientis sufferingfromdementia,but is
physicallyin good health,the second is paraplegic,and the thirdis obese.You are allowed to takecareof
these patientson your own, but you can also ask colleaguesfor assistanceif you thinkthe careis too tryingor
difficultto carryout alone.You arerequiredto show thatyou aresensitiveto the differentdisabilitiesof the
patients,and thatyou takethis into accountin your actions.Furthermore,you have to be able to
communicatewell with the patients,explainto them what is going to happen,and answertheirquestions.
Yourmentorwill write a reporton your performancein each of the threecases,and finallydraw a conclusion
aboutyour competencein providingpersonalcareto differentkinds of geriatricpatients.
Authentictask:
Providepersonalcareto geriatricpatients(washing,gettingdressed,combing)while takinginto account
theirdisabilities,dysfunctions,and privacyrules and issues.
Authenticphysicalcontext:
Performingthe tasksduringan internshipat a nursinghome for geriatricpatients.
Authenticsocialcontext:
Allowing the studentto takecareof the patientsalone,but also allowingthe studentto ask for assistanceif
the taskis too tryingor difficultto carryout alone.
Authenticresult:
Judgingthe competenceof the studentby observingthe studentdemonstratingcompetenceby performing
the whole task.Competenceis judgedbased on multiple(three)performancesin differentcontexts(different
patients).
Authenticcriteria:
Judgingthe relevantprofessionalcompetencyin relationto differentaspectsof the completecompetency
(washing,dressing,combing,but also communicating,takingdisabilitiesand dysfunctionsinto account,
takingprivacyrules into account).Criteriaareknown to the students-they learnedthem in schoolbefore
beginningthe internship-and arebased on the basiccarethatnurseshave to providein reallife.