Beruflich Dokumente
Kultur Dokumente
Understand the arguments that traditional assessments are not fit for
purpose in assessing authentic, active learning
Think about this…If you teach a student to learn how to drive a car by practicing in a real car, would you in you
professional opinion feel that after a theoretical, multiple choice test, that the assessment allowed you to make a
judgement about the roadworthiness of the student?
If we are indeed teaching skills then isn’t it logical that the assessment should allow students to demonstrate mastery of
those skills?
This is why some educators believe that traditional assessment (examination and multiple choice testing etc) are
ineffective for use in a problem-based curriculum
Assessment
Background
It will come as no surprise that the literature pertaining to assessment in problem-based learning is wide-ranging, varied,
and often conflicting. However, there are common themes that occur:
• Proponents of PBL who suggest that traditional methods of assessment that are focused on knowledge acquisition
are entirely inappropriate, and misleading for evaluating student progression in a PBL program.
• Problem Based Learning pedagogies are based on the principles of learning to learn, and therefore traditional forms
of assessment, i.e. those that evaluate rote-learned knowledge and comprehension (i.e. multiple choice tests, fill in
the blanks, essays) are nothing short of ineffective.
• They argue that following an ‘authentic learning activity’ with a traditional pencil and paper test that evaluates
content knowledge is a misalignment that actually disturbs student learning because it sends out mixed messages
about teacher expectations.
learning to learn
Assessment
Background
Zinc is Zn on the periodic table…
Authentic Assessment, an alternative to the traditional ‘content-based assessments’ was developed in line with social
constructivist theories with an emphasis on students using higher order thinking to apply the skills, attitudes and
knowledge they have learned during the program of learning. In this model, assessment:
• Places great emphasis on there being ‘multiple learning indicators’, to provide insight into the holistic development of
the leaner,
• Provides the student with opportunities for expressing and demonstrating their learning in different ways
• Is considered before planning the lesson activities in a model called backwards planning (or reverse planning)
Assessment
Authentic Assessment
When designing an authentic assessment, Olfos & Zulantay (2007) suggest that you should be very clear on the purpose.
Your authentic assessment should:
• Provide opportunities to monitor the learner’s progress using a variety of methods, such as observation records,
interviews, and evidence gathering
Frey et al (2012) conducted a review of 109 scholarly articles on what constitutes authentic assessments for children in
grades K-12 and proposed three criteria:
1. Context of the Assessment – the assessment should be a realistic activity; performance-based and cognitively
complex
2. Role of the student - a defence (or justification) of the solution or product is required, assessment should be
formative and students should collaborate with each other
3. Scoring – scoring criteria should be known or student developed, scoring should involve multiple indicators (or
portfolios are used), performance expectation is mastery.
Assessment
Authentic Assessment
For the purposes of planning and authentic assessment, we can draw from assessment models that split assessment into
three areas; Content, Process, Outcomes.
Sale (2000) proposes that the three domains of knowing, thinking and doing underpin the PBL model! Sale (2000)
Assessment
Authentic Assessment
“In this model of learning (Knowing-Thinking-Doing) knowledge is recognized as an essential component, however the
process of learning is actually driven by the learner’s participation in real world activities, and importantly, the thinking
that translates knowledge into competent performance”. Sale (2000)
Teachers are very well-acquainted with assessing ‘what learners know’. This is typically done using summative
assessments such as multiple choice, essay-based questions or closed-ended short answer questions.
Such types of assessment enable teachers to quantify gains in knowledge and make judgments about a student’s
progress and attainment. However, as discussed previously, such types of assessment provide little insight into the
effectiveness of the development of a range of other skills and competencies that PBL is specifically employed to
promote.
While it is important to acknowledge that the acquisition of knowledge is an essential component of PBL, and should be
assessed – and indeed is accounted for as part of a content-process-output framework, we dedicate the following
sections to exploring potential strategies for assessing authentic learning (how are learners thinking, and what can
students do to illustrate their learning/thinking) which may be less well-documented that traditional summative
assessments.
Assessment
Assessing Content
In order to be able to assess how students think, it is important that you are able to first identify the kinds of thinking
you would like to teach when planning your assessment.
Educational research suggests that Problem-Solving is comprised of a whole array of cognitive skills and functions. In a
simplified model, Creative thinking (divergent thinking) would be at one end of the cognitive spectrum and Critical
Thinking (convergent) would be at the other.
Metacognition is also recognised as essential for problem-solving and it is situated somewhere in the middle of the
continuum of thinking.
The spectrum does not indicate good thinking or bad thinking, rather it indicates the different cognitive functions
involved in these types of thinking.
Assessment
Assessing Process
Divergent thinking (or creative thinking) entails a range of strategies that generates multiple possible creative solutions.
Often there will be so many solutions proffered that some of them will be unworkable, unaffordable, unachievable and
so on. This does not matter for in this kind of thinking, ideas are best generated spontaneously and unhindered in an
‘anything goes’, judgement-free environment.
Convergent (critical) thinking, conversely does involve judgement, reason and logic. It is employed in problem solving to
bring together different ideas from different disciplines or stakeholders to determine a single best solution. Therefore,
convergent thinking often follows divergent thinking during problem solving activities, when of course the desirable
outcome is one single solution.
Assessment
Assessing Content
The diagram below shows the spectrum of thinking and provides examples of the kinds of cognitive functions involved in
each of the aspects of problem solving.
Generating novelty or
Revising one's own thinking,
originality in possible Evaluating the relative
descions and strategies
options worth of options
Assessment
Assessing Process
In real-world problem solving, experts will turn to a well-practiced set of methods or strategies when they encounter a
new challenge. These strategies are very much based on the kinds of thinking described in Figure 6, but for teachers
new to PBL or to teaching thinking skills, the ability to identify the types of thinking embedded in these strategies and
practices is not easy. However, Sale (2000) suggests there are three important aspects to evaluating students’ thinking.
These are:
As you are planning the assessment and teaching activities be clear about the kinds of thinking you would like learners
to develop and exhibit e.g. analysis, weighing/evaluating, interpreting information, generating novel ideas.
In authentic assessment, curriculum planning begins with the learning objectives and assessment in a model referred
to backwards planning.
The instructional design comes much later once the ideas about what a student should be learning, and how we’ll
know when they are, have been identified.
Therefore the objectives you have for students (e.g. I would like to focus on developing critical thinking, reasoning, and
argument and justification skills) will drive the design of the assessment and then the design of the learning activities.
Assessment
Assessing Process
In assessing this domain of learning, ‘process’, teachers seek to gather evidence on how students approach a problem,
which thinking strategies they are drawing upon to solve a problem and how they evidence, provide argument for and
justify their choices. Here there are a number of ways to gather evidence on thinking, including through:
• Portfolio
• Formal observation
• Structured questioning
• Reflective feedback
Assessment
Assessing Process
There are many ways to structure a portfolio, and many teachers feel that it can become unwieldly, adding to their
workload. However, in PBL the onus of assessment is not always on the teacher.
Within the portfolio there may well be pieces that have been assessed by the teacher, but there should also be pieces
that have been peer assessed, as well as those that have been self-assessed.
In many models of PBL an external examiner is invited to assist in some of the assessment. This may be a teacher from
another class or school or an ‘expert’ in the field (for example a local business person with experience relevant to the
problem). This additional form of evaluation can help provide a valuable source of feedback and evidence.
Whoever is involved in collating the portfolio, the pieces should be purposefully selecting to demonstrate progression
towards mastery of some competency – deciding the purpose and the competency that you are illustrating is crucial to
building an effective portfolio.
Assessment
Assessing Process
Portfolio assessment involves developing a purposefully-selected suite of documented evidence over time.
The inclusion of time in this definition is crucial, as it depicts a style of learning where the learning activities are not just
a means to an end, but provide opportunities of learning, assessment and reflection in themselves.
The portfolio should be comprised of a range of evidence from the PBL activities which could include student notes,
lab write ups journal entries; peer reviews; artwork, diagrams, charts, graphs; annotated photographs, multimedia
presentation; group reports; rough drafts and polished writing, self-assessments, reflections among others.
1. Growth Portfolios
to show growth or change over time
to help develop process skills such as self-evaluation and goal-setting
to identify strengths and weaknesses
to track the development of one more products/performances
2. Showcase Portfolios
to showcase end-of-year/semester accomplishments
to prepare a sample of best work for employment or college admission
to showcase student perceptions of favorite, best or most important work
to communicate a student's current aptitudes to future teachers
3. Evaluation Portfolios
to document achievement for grading purposes
to document progress towards standards
to place students appropriately
Assessment
Observation
In some studies, the term process is synonymous with ‘behaviour’. Unlike products which can be measured and
evaluated after the fact, behaviours are no longer visible once they have been exhibited.
This concept of students’ expressing process (or thinking), through a behaviour adds an interesting dimension to the
suite of assessment tools, to include observation.
Teachers are naturally attuned to observing students and making evaluative assessments in this way. However, formally
adding observation to the assessment strategies (perhaps through the use of observation schedules etc), provides yet
another source of rich information on student thinking.
Assessment
Observation
This particular method can benefit from having an external examiner. The external examiner can be used as an
independent assessor, or more usefully as a second assessor. You can use grading and feedback from the external as
benchmark for your own observation:
• Did the external assessor have generally more positive or negative reflections on their observations?
• Reflecting upon your own observations, were there particular students who had more negative feedback in your
observations when compared with the observer?
An important aspect of problem based learning is reflection and metacognition. These may sound like tricky things to
assess, but gathering evidence on these is just as possible as assessing other types of skills.
One of the most common modes employed is asking students to keep a ‘reflective journal’. Not all students will initially
know how to reflect, so you may have to structure this to begin with, perhaps by setting two or three reflective
questions for them to consider. The aim, however, is for you to be able to withdraw the scaffolding as the students
become increasingly independent.
It is also important that you provide time for this to happen on a regular basis, and you may wish to set aside a
dedicated period of time towards the end of each week to allow learner to reflect upon their own progress and that of
the group. If students are undertaking PBL as part of a digital skills initiative, this could also be done as part of a v-log,
a blog or web forum.
Assessment
Reflection
Additionally, as part of the PBL scaffolding, many facilitators encourage groups to self-manage and self-report by
collectively assessing the group’s performance each week.
Group targets can very easily be derived from the ‘rule setting’ or ‘group dynamics’ discussions that you have before
you embark upon your first PBL adventure. These can have variety of forms and functions for example:
• Team evaluations, where each member of the team evaluates the dynamics of the team as a whole
• Peer evaluations, each team member evaluates the contributions of his/her teammates and
• Self-evaluations, when each team member documents and evaluates his own contributions to the team
Assessment
Reflection
Team, peer and self-assessments can be quantitative (scoring checklist) or qualitative (reflective writing) or as
questionnaires targeting specific dimensions of teamwork.
Regardless of which format, these reflections can be used as an indicator of metacognition for the purpose of charting
progress, and/or can be selected to form a part of a wider portfolio of progress. Providing groups with weekly
checklists serves a multitude of functions including:
• Reminds students of the rules they agreed to abide by, and help maintain the group
• Helps students practice reflection, dialogue, discussion and reaching a group consensus
• Can identify problems within the group before they come to a head
• Charts progress on a regular basis, providing you with a valuable source of evidence
Assessment
Structured Questioning
In the PBL process, teachers facilitate learning by providing scaffolding to support students in their learning. This is
frequently done through asking questions, the purpose of which, is to highlight and model successful thinking
strategies.
When the purpose is for assessment of process, rather than modelling, the types of questions might be quite different.
The questions to be asked as part of assessment should be well-considered, planned in advance and should be
designed to provide evidence of thinking.
Again, as with any backward planning assessment, you should already know what kinds of thinking you are assessing
and design your questions to provide cues that will give the students opportunities to exhibit these.
The use of Socratic Seminars is beginning to be trialled in some classrooms. Here particular types of question are used
(literal question - fact based; interpretive questions -based on student’s interpretation; applied questions - based on
experience and ability to apply knowledge). These are a particular strategy that can be used equally well as a stand-
alone activity, or as a framework for asking structured questions.
Assessment
Output
As part of the knowing-thinking-doing model of assessment, the domain of output examines what a learner can do,
show or produce.
The emphasis is on gathering evidence that indicates a level of mastery in a particular area.
There are two well-documented forms of assessment that can enable teachers to assess mastery
• Portfolio Assessment
• Performance Assessment
Assessment
Output
Portfolio Assessment has been well-detailed in the previous section. Needless to say, it is as useful in assessing this
domain as it is in assessing student process (or thinking). In fact, the ‘showcase’ style of portfolio tends to more
typically associated with output (products) than with process.
Again, it is necessary to stress the long-term, continuous element of this kind of assessment which enables the teacher
to build up a holistic picture of progress throughout the PBL programme.
Assessment
Output
Performance Assessment aims to test students' ability to apply acquired knowledge and skills in a variety of authentic
contexts and work collaboratively to solve complex problems.
• Different types of performance assessments can be undertaken throughout the programme to provide a suite of
evaluations for different competencies, or
• Individual performance outputs can be selected as exemplary pieces to be archived in a portfolio of learning.
There are a number of different ways to assess Performance. We summarise these from work undertaken by McDonald
(2005) in a table on the following slides. These assessment types were largely devised for use in the discipline of
medicine, and as such offer tried and tested methods which could be adapted for use in your classroom. We also
supplement the table with some practices already used in K-12 education, indicated by an * in the table.
Assessment
Output
The types of assessment in the table were largely devised for use in the discipline of medicine, and as such offer tried
and tested methods which could be adapted for use in your classroom.
Case-based assessment Students are presented with a real life scenario to solve, or a This is best used when there is a real external client (such as a
real client to manage. local librarian, green grocer, park ranger etc. The students can
then be assessed on the client briefs they draw up and how
closely they managed to produce solutions that actually fitted
the client’s needs.
Group presentations Students can be asked to take on a role or work within a context These are difficult to mark, because the person presenting isn’t
or scenario. These are often used to report progress in PBL, necessarily the person who has done the work, there for it may
present findings, or foe debriefing. be best to stay away from grading individuals, and instead
providing group grade, with formative feedback for individuals.
Portfolios The purpose of the portfolio has to be clear from the start (are These can be unwieldy if not managed well and may be difficult
we gathering evidence on thinking, on progress or just for to mark. Double marking on some items (Student and teacher,
showcasing) Portfolios that include reflection and criticality from student and peer, teacher and external) may help provide some
the student often provide the best sources of evidence. Creating sense of moderation.
the portfolio can also be an assessment task in itself, where
students are required to draw out conclusions and synthesise
the materials contained in the portfolio.
Assessment
Output
Triple jump Largely used in professional disciplines such as medicine. Students are rated by the examiner on problem-solving skills,
(i) Individuals are presented with a problem and expected to self-directed learning skills and on their knowledge of the
discuss the problem and their learning needs with an oral problem area.
examiner. The method is time and resource intensive, though can be
(ii) Students then research material and work on a solution replicated with student mentors or group work.
(iii) Students then discuss their findings with the examiner
Individual presentations Students are asked to present the components of work they As long as the criteria are clear before the students prepare
have researched for their contribution to the overall solution or their presentation, this can be assessed in any way at all.
management of the problem scenario
tripartite assessment i. The group submits a report for which they receive a mark. These three components are added together to form the overall
ii. The individual submits the piece of work they researched. individual mark. The advantage of this is that it does not
iii. The individual writes an account of the group process that is privilege some students who do less work and an individual
linked to the theory of group work. student will be responsible for gaining two-thirds of the marks
and therefore most students perceive this kind of grading as
being fair.
patchwork texts Students build up text in course work over a number of weeks. This can be assessed using rubrics or standard methods for
Each component of work is shared with other students and they assessing texts.
are expected to use different styles, such as a commentary on a
lecture, a personal account, a book review to build up a tapestry
of different textual components.
Assessment
Output
Exhibition The group work on a product that can be exhibited (a These can be assessed in any number of ways by teachers,
poster, a painting, a physical artefact, an architectural students, as long as teachers and students are clear on the
model, a concept design etc). These products can be assessment criteria.
exhibited in a hall in school or in a place of significance such
as a local community centre etc Visitors may also be permitted to provide feedback on the
products and the product information at the exhibit.
Additionally, students may be asked to prepare an individual
piece of work, or produce a written piece to support the
submission of the physical artefact. For example, they may
be asked to write the companion ‘information panel’ that
often is situated on the wall beside an exhibit, or submit a
piece for the exhibition programme so visitors have
information to hand about the exhibits.
Report or essay Written communication is an important skill for students to This can be assessed using rubrics or standard methods for
acquire. Requiring written reports allows students to assessing texts.
practice this form of communication, particularly if the word
allowance is short it can promote succinct, critical pieces of
work. In keeping with the ideals of authentic assessment
written reports should have a ‘real’ purpose for example, an
Executive Summary for assessment, accompanied by a
portfolio of supplementary back-up evidence which could
be sampled rather than marked in full, a letter to the local
council providing arguments for or against a particular local
issue, or an environmental impact assessment report etc.
Assessment
Output
While there is much discussion on the ways in which teachers might approach assessment in PBL, there are a number
of common themes that have arisen in the literature reviewed here so far. Specifically, these are that assessment
should be:
• Continuous – assessment should not be undertaken only at the end of the programme
• Varied – it should assess process (how), product (Doing) and knowledge (what)
With respect to this last point, much has been written on the need to provide clear assessment criteria. Much of the
literature around assessment, suggests that it should be Criterion-Referenced as opposed to Norm-Referenced.
Assessment
Output
Norm-referenced tests - are those traditionally used in schools and are designed to compare a student’s performance
against that of other students.
Criterion-referenced assessments - are designed to enable teachers to compare students’ performance against a set
of agreed standards for the task. There is some consensus that criterion-based standards are required in an authentic
learning environment, like that of PBL but this does not suggest that this type of standard does not have drawbacks of
its own.
To aid the process of designing clear, transparent and criterion-referenced assessments, it is suggested that teachers
develop an assessment rubric.
A rubric - indicates the competencies to be achieved, but additionally provides a sliding scale to indicate the ways in
which learners can demonstrate different levels of mastery of each competency
Assessment
Output
An important part of enabling learners to understand expectations is to share the assessment criteria with them at the
outset.
In a PBL programme there are a variety of ways you can do this, but it has been suggested that in order to help
learners become self-directed, you may wish to allow the learners to participate in shaping some of the indicators for
themselves.
This may not be possible with all criteria (it may well be that you allow them to do this for one competency), or indeed
for all learners (it may be particularly difficult with young learners).
Furthermore, it may be more useful to trial this as a developmental strategy at the beginning of a new unit once
learners have tried PBL a few times.
Assessment
Assessment Within PBL
It is suggested that there are implicit factors in assessment, specifically, how much emphasis or weight you will apply to
the different assessment strategies you employ, that may mean that even when learners are involved in creating the
rubric, they may not fully understand how they will be applied. Therefore it is important to consider, and make clear:
• What percentage of the student’s total project grade will be based on the group’s performance vs. individual components?
• Will feedback from external clients also be incorporated into your assessment of the group’s work?
• If so, what sorts of feedback will you solicit: feedback on product (e.g., Does it work? Is it a good solution/design?),
• Feedback on process (e.g., Did the group communicate effectively with the client? Did it meet deadlines?), or both?
Assessment
Assessment Within PBL
The more complex the assessment, the more judgment is required from markers, and this will be especially true if you
choose to invite external assessors into your classroom. The advice on keeping your assessment simple, reliable and
an assured standard of quality include:
• When grading, involve others such as host supervisors and marking teams, and students themselves as self- or peer
assessors
Assessment
Assessment Within PBL
As discussed at the beginning of this section, there are no clear cut rules to assessing PBL. Here we have tried to
provide what we believe is guidance on effective ways to assess the three important elements of PBL; Content
(knowledge acquisition), Process (How learners think), Product (What learners do to meet an outcome).
There are many other ways to approach assessment, and we suggest that you continue to conduct some research for
yourself. In particular, discussing your assessment practices with other teachers will be a helpful source of best practice
guidance.
Assessment
Output
1. In order to prepare young people for learning, life and work in a digital age, it has been suggested that we need to
reimagine our classroom practices and wholly embrace pedagogies that make learning through experiences and
the acquisition of Higher Order Thinking skills a priority. Reflecting upon your current practice, which activities do
you feel are helping to develop high demand skills like problem solving, creative thinking and analysis?
2. Considering the constructivist perspective on successful learning environments (i.e. those that foster active learner
engagement, critical thinking, problem solving, student-centered, experiential learning, discussion and reflection)
what elements of your classroom environment and practice work well and where might you consider making
changes?
3. An important element underpinning both constructivism and 21 century learning is collaboration. How well does
your current practice and learning environment actively encourage this? How capable are your learners in
managing and negotiating group dynamics to enable successful collaboration?