See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/8463131

Portfolio assessment: Practice teachers' early experience

Article  in  Nurse Education Today · August 2004

DOI: 10.1016/j.nedt.2004.04.005 · Source: PubMed

CITATIONS READS

29 716

2 authors, including:

Walid El Ansari
University of Gloucestershire
271 PUBLICATIONS   4,418 CITATIONS   

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Collaboration between the municipalities of Varde, Esbjerg and University of Southern Dk View project

Occupational Health View project

All content following this page was uploaded by Walid El Ansari on 06 November 2017.

The user has requested enhancement of the downloaded file.

Nurse Education Today (2004) 24, 388–401
Nurse
Education
Today
intl.elsevierhealth.com/journals/nedt

Portfolio assessment: practice teachers’

early experience
William Spencea,*, Walid El-Ansarib,1

a
Oxford Brookes University, Isis Education Centre, Roosevelt Drive, Oxford OX3 7JX, UK
b
Oxford Brookes University, Sandringham House, Heritage Gate, Sandy Lane West, Oxford OX4 6LB, UK

KEYWORDS Summary Experience was recognised to be a vital source of learning as long ago as
Practice assessment;

1762 [Emile, Everyman, London, 1993] and reflection on practice experience may be
Portfolio; one way forward in addressing nursing’s anxieties concerning the practice theory
Specialist practice; gap. However, despite the acceptance that subjectivity in the process seems
Learning outcome inevitable and potentially important, little is understood of the practitioner’s
evidence experience of practice assessment. Two questionnaires sought the views of
specialist community nursing practitioner (SCNP) programme (United Kingdom
Central Council for Nursing, Midwifery and Health Visiting (UKCC) 2001) practice
teachers (PTs) on the introduction of the portfolio approach to practice assessment.
These were distributed to 62 and 76 PTs and the response rates were 32% and 50%,
respectively. Responses of those PTs from the three specialisms participating in the
piloting of the portfolio approach were compared with those using an existing
approach. An action research method was adopted which attempted to use
established theory to explain the challenges presented by the introduction of this
approach and ultimately to raise the PT group’s awareness of assessment issues.
Both qualitative and quantitative data were collected and the findings support the
use of the portfolio approach to practice assessment. The PT experience of portfolio
use was found to be a largely positive one. PTs reported the utility of the portfolio in
prompting student self-evaluation of learning. Concerns were expressed by PTs
around the quality of portfolio evidence although many felt that it had promoted
students’ reflection on practice. Inter-PT reliability in practice assessment was
identified as a topic for PT continuing professional development. Many sources of
evidence, including patient feedback, were used by PTs in their assessment of
students although PTs using the portfolio approach used less first-hand experience
of students’ practice in their assessments of competence, relying more on written
evidence, than those PTs using a traditional approach to practice assessment.
c 2004 Elsevier Ltd. All rights reserved.

Introduction
*
Corresponding author. Tel.: +44-1865-226518; fax: +44-1865-
223998.
E-mail addresses: wspence@brookes.ac.uk (W. Spence),
Professional practice is marked by its particular-
walid.ansari@brookes.ac.uk (W. El-Ansari). istic, ideographic and reinterpretive nature
1
Tel.: +44-1865-488110; fax: +44-1865-488126. (Ramsden, 1992). It is characterised by a context


0260-6917/$ - see front matter c 2004 Elsevier Ltd. All rights reserved.
doi:10.1016/j.nedt.2004.04.005
Portfolio assessment: practice teachers’ early experience
of ‘uncertainty, disorder and indeterminacy’
(Scho €n, 1983, p. 15), where the application of
theory is less than straightforward. It is devel-
oped through participation in a significant setting
(Illich, 1971) and the learning that underpins this
is likely to be self-discovered and self-appropri-
ated (Rogers, 1983). Rousseau (1993) believed
experience and emotion to be the ‘real teachers’
and he assigned to educators, as a fundamental
principle, the work of inspiring interest and
teaching learning methods. Professional practice
assessment issues have preoccupied nurse edu-
cators for many years and yet continue to pose
some problems (Brown, 2000). Although portfolios
have been considered central to nurse education
at all levels (Glen and Hight, 1992), there is no
unified understanding of the term in the litera-
ture or among practitioners. The recognition of
these factors in SCNP contributed to the adoption
here of the portfolio approach in practice as-
sessment. Portfolios may be seen as collections
‘of evidence that learning has taken place’
(Snaddden and Thomas, 1998, p.192) and in this
paper the portfolio evidence relates directly to
each of the UKCC (2001), SCNP themed learning
outcomes (Box 1). Portfolios enable the gathering
of evidence and reflection upon events relating
to practice (English National Board for Nursing,
Midwifery and Health Visiting (ENB), 2000) and
may be better established in formative than
summative assessment (Runciman, 1990). Portfo-
lio compilation could promote the deep and ac-
tive approach to learning described by Fransson
(1977), where learner activity and involvement
are high. Redfern et al. (2002) listed the many
benefits of portfolio use: sensitivity to students’
varying performance levels, the facilitation of
student progression to heightened competence
and the enhancement of students’ self-aware-
ness. Portfolio use is consistent with the com-
petency approach, e.g., Neary (2000), whose
‘pedagogical potency’ has long been acknowl-
edged in nurse education (Whittington and Boore,
1988, p. 134).
There is some ambiguity in the literature
around the definition of the term ‘competence’
(Ashworth and Morrison, 1991) and Gerrish et al.
(1997a, p.29) described the notion of competence
as ‘confusing and contentious’. Absence of com-
petence has recently been identified as occurring
where ‘knowledge, skill or judgement’ is found to
be lacking in addition, potentially, to the pres-
ence of ‘negative attitude’ (NMC, 2003). This
description adds little to the clarification of this
concept. However, the problems with the under-
standing of competence have not deterred re-
389
searchers from advocating a national minimum
nursing competence standard and assessment tool
(Norman et al., 2002). More pragmatically, the
UKCC (1999, p.35) viewed competence as a de-
scription of the ‘skills and ability to practise
safely and effectively without the need for direct
supervision’. This definition is particularly apt for
those undertaking SCNP programmes and will be
used in this report.
Study setting
This study was implemented at a university in the
south of England over the 2000/2001 academic
year. At the time of the study seven SCNP pro-
grammes were offered at degree level. Nursing
practice was afforded 50% of student effort on the
course and around 600 h were devoted to practice
across the one year full-time, or up to three years
part time equivalent, course.
One year before the study took place the
teaching team planned the implementation of the
portfolio approach to the assessment of practice.
The portfolio approach was then piloted over the
following academic year (2000/2001) across three
specialist practitioner pathways: community
mental health nursing (CMHN), community nursing
in the home/district nursing (CNH/DN) and public
health nursing/health visiting (PHN/HV). The ex-
perience of the PTs on these pathways was
compared with that of school nursing (SN), gen-
eral practice nursing (GPN), community learning
disabilities nursing (CLD) and community chil-
dren’s nursing (CCN) PTs. The assessment strat-
egy for the pilot programmes required that
students complete the portfolio to evidence the
attainment of each individual practice outcome
relating to SCNP themes (Box 1).
The practice assessment steering group com-
prised PTs and lecturers and was formed in 1999
by the first author (W.S.) to develop the teaching
team’s approach to practice assessment. PTs
were employed by the National Health Service
(NHS) as community nursing practitioners with
responsibility for mentoring SCNP students and
lecturers included four university employees and
two lecturer practitioners. Steering group mem-
bers met regularly for one year preceding the
study, to develop the assessment of practice
across the seven pathways. The steering group
oversaw the development of the portfolios and
co-ordinated the pilot specialism work groups in
their development of the documentation includ-
ing learning outcome attainment guidance. Each
390 W. Spence, W. El-Ansari

Box 1. SCNP learning outcomes: themes and examples

UKCC (2001) SCNP themes and concepts of learning outcomes
Clinical practice
Assess health, health-related and nursing needs of patients or clients, their families and other carers.
Facilitate learning in relation to identified health needs for patients, clients and carers.
Support and empower patients, clients and their carers to influence decisions concerning their care.
Care and programme management
Supervise and manage clinical practice to ensure safe and effective holistic research-based care.
Initiate and contribute to strategies designed to promote and improve health and prevent disease.
Recognise ethical and legal issues and take appropriate action.
Clinical practice development
Lead and direct the professional team clinically and monitor quality assured standards of care.
Identify individual potential through effective appraisal and advise on educational opportunities.
Provide preceptorship, mentorship, counselling, clinical supervision and an educational environment.
Clinical practice leadership
Create an environment where practice development is fostered, evaluated and disseminated.
Initiate and lead practice developments to enhance the nursing contribution and quality of care.
Explore and implement strategies for quality assurance and quality audit.
Specialist pathway-specific learning outcomes
(CMHN-specific) Initiate health promotion and education to facilitate well being.
(CNH/DN-specific) Assess, plan, provide and evaluate specialist clinical nursing care.
(PHN/HV-specific) Undertake diagnostic, health screening and surveillance and therapeutic
techniques.
(Source: UKCC, 2001).

Box 2. Portfolio guidance: acceptable evidence

Examples of data collection, e.g., caseload/community profile.
Notes of meetings attended.
A formally presented reflective account of a critical practice incident.
Review of individual patient’s needs and care.
Citation of published literature to justify particular aspects of care.
Extracts from a reflective diary relating to practice events.
Video and/or audio recordings of practice sessions/events/supervision or excerpts from these
recordings (participant permission required).
Action plans relating to practice experiences enabling the attainment of learning outcomes.
Records of supervision with supervisor.
Observations and witness statements.
Analysis of practice using critical incident technique.
Short written reports relating to practice.

portfolio retained the UKCC (2001) SCNP learning Study aims

outcomes in their entirety and these numbered
between 31 and 39. Over the first two terms of
the 2000/2001 academic year, each student was
required to evidence the attainment of between
23 and 26 of these. The pilot group portfolios
shared common introductory sections and all
promoted the same range of acceptable evidence
including the use of reflection on practice to
demonstrate outcome attainment (Box 2).
The aims of this study were to:
1. Determine the PTs’ early experience of the
portfolio approach to practice assessment in-
cluding its use in: student self-evaluation,
guiding the compilation of evidence and the
development of the student’s professional
practice.
Portfolio assessment: practice teachers’ early experience
2. Compare experience of practice assessment
between participating pilot PTs using the portfo-
lio approach and those using an established
approach across a number of variables including
the sources of evidence used by the PT in judg-
ing competence.
3. Determine the PTs’ view of personal continuing
professional development needs with respect
to practice assessment.
4. Feed back and inform the development of the
portfolio approach.
Method
An action research (Hart and Bond, 1995) mode was
employed in this study, where the first author
(W.S.) was the leader of one of the pathways in-
cluded in the study. Action research has been re-
cognised as an appropriate methodology, which
resonates with the aims of education and the
portfolio approach (Glen and Hight, 1992) and was
used here to inform the ongoing development of
educational practice. The cyclical process of action
research involves planning, acting, observing and
reflecting (Mayer, 1993), and it may be seen as a
way of generating understanding of a social system
in attempting to change it (Lewin, 1946). The in-
volvement of participants in this study addressed
an important ethical concern for this methodology.
The near democratic representation of participants
in the practice assessment steering group sup-
ported the meaningful relation of ensuing changes
to educational practice in the PTs’ work. The re-
search method adopted was designed to facilitate
the observation stage of action research, which
was preceded by the teaching team’s work in
Table 1 Questionnaire structure
Item category
Demographic factors
Perception of personal PT role support requirement
Utility of portfolio guidelines and format
Inter PT reliability
Student’s self-assessment
Nature of student evidence
Student’s consideration of the contextual factors of critical incidents
Further information on experience of practice assessment
Number of learning outcomes
Independent reflective learning
Student’s engagement in portfolio compilation
Experience of documentation
Total
Number of asterisks indicates number of questions eliciting qualitative data.
391
planning and implementing the portfolio approach
to practice assessment. Such participation has the
potential to contribute to the empowerment of
professionals through the generation of policy in a
transparent manner that is endorsed by the
majority.
The research tools comprised two postal ques-
tionnaires – questionnaire A (23 items) and ques-
tionnaire B (18 items). Questionnaires were used in
this study to reflect the participants’ strength of
view and opinion (Black, 1999) and the Likert scale
chosen to measure aspects of each concept with a
focus on the present (Oppenheim, 1992). Both
questionnaires were developed by the first author
(W.S.) to address the PTs’ experience of portfolio
use in relation to issues identified by the steering
group (Table 1). The questionnaires were informed
by the work of Phillips et al. (2000a,b), Redfern
et al. (2002) and Ashworth and Morrison (1991) in
relation to the strengths and weaknesses of port-
folio use, rating scale use and reliability and va-
lidity of assessment methods respectively. The
pilot stage of questionnaire design was omitted as
the items identified for inclusion arose directly
from the practice of those closely involved in the
project and the design of the questionnaire
deemed to be uncomplicated. The second ques-
tionnaire built on the data of the first and enabled
comparison of pilot and non-pilot groups. Ques-
tionnaire B also sought to qualitatively explore
many of the issues arising from the first question-
naire (Table 1).
The questionnaires elicited quantitative and
qualitative data, including demographic data (Ta-
ble 1). Qualitative inquiry explores social or human
problems, assumes that knowledge is within the
meanings that people make of their experiences
Number of questions
Questionnaire A Questionnaire B
2 2*
1* 3**
4 3
0 4
3 2**
2 2*
0 1*
2 1*
1 0
3 0
4 0
1 0
23 18
392
and reports the detailed views of informants.
Qualitative data collection here was consistent
with the tendentious assumptions of action re-
search (Creswell, 1998).
In questionnaire A, one item was open-ended in
order to generate qualitative data. Quantitatively,
this questionnaire comprised 18 items (employing
5-point Likert scales) to determine respondents’
agreement or disagreement to a series of state-
ments. The items sought PT opinion on a range of
issues pertaining to the implementation of the
portfolio approach as well as the guidance for
completion offered to the student and the PT. In
questionnaire B, eight questions generated quali-
tative data while another nine items were in Likert
scale format (Table 1 and Fig. 1).
Several aspects of the portfolio approach were
explored:
(a) Practitioners’ doubts about their assessment
competence have been previously noted (Phil-
lips et al., 2000a) which, in addition to their as-
sumed limited experience of this approach, led
the steering group to consider the best way of
supporting PTs in this role.
(b) Black and Wolfe (1990) questioned the amount
and level of evidence required by portfolio as-
sessment. They reported no clear guidance on
the utility of specifying a word limit. Although
a word limit was not stipulated in the pilot or
non-pilot conditions, the PTs’ opinion on this
was sought by one question.
(c) Elbow and Belanoff (1997) have expressed
doubts about inter-rater reliability relating to
Figure 1 PT experience of portfolio use.
W. Spence, W. El-Ansari
portfolio assessment. They acknowledged the
portfolio’s high validity and low reliability and
several items across both questionnaires ad-
dressed reliability.
(d) Knowles (1980) believed that students must de-
velop their own criteria for monitoring personal
learning. He held that the diagnosis of personal
learning is an underlying assumption of the
portfolio approach, as is the self-directed na-
ture of the student (Knowles, 1975). Accord-
ingly, data on self-assessment were requested
in this study.
(e) The steering group held anecdotal evidence of
the positive association between self-assess-
ment and persuasive evidence of outcome at-
tainment. This area of assessment was
explored further in several items of question-
naire B.
(f) Members of the steering group believed that re-
flective understanding would develop through
the discussion of portfolio evidence with PTs
(Phillips et al., 1994) and would promote the stu-
dents’ active engagement in the learning pro-
cess. Several questionnaire items explored the
students’ use of reflection with regard to their
awareness of: personal progression, acquisition
of practice competence and self-evaluation.
Questionnaires A and B were distributed to 62
and 76 PTs, respectively, and the response rates
were similarly 32.3% and 50% and data from both of
these questionnaires are presented here. The rea-
sons for the increase in response rate over these
two questionnaires are not clear.
Portfolio assessment: practice teachers’ early experience
Data analysis
As the responses to most questions were not nor-
mally distributed, the Mann–Whitney U test was
used to study the underlying population distribu-
tions (Gibbons and Chakraborti, 1992). Samples’
responses to the questions with Likert scale results
were compared across groups using the
Mann–Whitney U test. The percentage of respon-
dents reporting positively was compared using the
t-test for equality of means (Black, 1999). Quali-
tative data generated from the open-ended ques-
tionnaire items were coded and emerging
categories were identified and stored on an elec-
tronic data filing system, developed for ease of
retrieval and analysed. Manifest content analysis
(Fox, 1982) resulted in the identification of several
major categories and relevant written evidence of
these was noted accordingly. Verbatim quotations
were chosen and included here to represent these
major themes (Fetterman, 1989).
Quantitative findings
Pilot group students were given extensive guidance
on the nature of required evidence including in-
teractive classroom sessions on portfolio comple-
tion, and the majority of PTs involved in the
assessment of this group’s practice were qualified
in practice teaching (Table 2). PTs in two of the
pilot groups were required to be PT qualified. The
non-pilot group comprised CCN, GPN, CLDN and SN
PTs, and used an established approach to practice
assessment, which prompted minimal evidence
gathering and included little guidance on the at-
tainment of the learning outcomes and mecha-
Table 2 Questionnaire and respondent details
Dimension Details
Questionnaire (No. of items) A (23 items)
Questionnaire completion End of term 1 (Nov. 2000)
Pilot status Pilot group
Respondent specialism and CMHN
number (%) PT qualified
CNH/DN
PHN/HV
Overall % PT qualified 95%
Sample size 62
CMHN, community mental health nurse; CCN, community children’s nurse; CNH/DN, community nursing in the home/district
nurse; CLDN, community learning disabilities nurse; PHN/HN, public health nurse/health visitor; GPN, general practice nurse; SN,
school nurse.
a
Number and percentage of respondents with PT qualification unknown.
393
nisms for evidencing this, other than a limited
prompt for the inclusion of reflective practice ac-
counts. The majority of the respondent PTs held
the practice teaching qualification (Table 2).
Questionnaire A (pilot group)
The majority of respondents reported student
documentation of evidence to have begun before
the 10-week term’s midpoint. With regard to the
assessment of the portfolio, 40% of respondents
agreed that the portfolio should be percentage
marked – only a pass/fail grading was used at the
time of the research – and the remainder neither
agreed nor disagreed that it should be percentage
marked. Most (70%) PTs believed that the portfolio
offered an optimum level of guidance for students
to demonstrate the attainment of learning out-
comes, while 30% neither agreed nor disagreed that
this was the case. About half of respondents felt
that portfolio guidance represented degree level
practice. Almost all PTs commented on the positive
nature of the portfolio experience, although three
quarters of the sample felt that the UKCC learning
outcomes in the portfolio should have been con-
flated. Almost all PTs reported that the students’
self-evaluation on the Benner (1984), novice to
expert, spectrum was employed in a way that
enhanced the learners’ awareness of personal
competence. Similarly, 85% of informants reported
that the portfolio approach encouraged students to
take an optimum level of control in personal learn-
ing. More than half of PTs (60%) felt that the port-
folio had simplified the assessment of competence
and many believed that it had promoted the stu-
dents’ reflection on practice (80%). Three quarters
of the sample reported that the compiled portfolio
B (18 items)
End of term 2 (Apr. 2001)
Pilot group Non-pilot group
0 (0%) CMHN 2 (29%) CCN 0 (0%)
7 (100%) CNH/DN 13 (100%) CLDN 0 (0%)
12 (100%) PHN/HV 13 (100%) GPNa
SNa
a
85%
62 14
394
evidence reflected PTs first-hand experience of the
respective student’s practice competency.
Questionnaire B (pilot and non-pilot group)
Fig. 1 depicts details of PT experience, with re-
spect to various facets of the portfolio, where low
scores indicate agreement with each of the nine
statements. Slightly less than half the respondents
believed that the student’s evidence should have
been subjected to a word limit, while almost half
the sample agreed that different standards had
been adopted by PTs in what constituted good ev-
idence of learning outcome attainment. Almost
half (46%) of pilot respondents disagreed that the
newly introduced practice assessment document
offered overly prescriptive guidance on outcome
attainment and more than half would have appre-
ciated further guidance on the nature of adequate
evidence. Almost three quarters of the pilot sam-
ple believed that the Benner scale, included in only
the portfolio documentation, promoted their re-
spective students’ awareness of personal progres-
sion in the acquisition of practice competence.
Around 75% of pilot respondents believed that the
evidence compiled by the student accurately re-
flected their experience of the student’s level of
practice competency. Only 16% of all PTs believed
that they were able to check personal standards for
student evidence against that of other PTs. Less
than one-third of PTs believed that the standard of
evidence required by them was ‘similar to that of
other PTs’.
Figure 2 PT assessment of student competence: sources of evidence and their contribution to assessment.
W. Spence, W. El-Ansari
PTs were asked to provide a breakdown of the
percentage contribution that a number of sources
made to their assessment of students (Fig. 2). Fig. 2
depicts the various sources of evidence used by PTs
in assessing students and their respective contri-
bution to assessment. The distribution of responses
across these sources may reflect, among other
things, nursing’s apprenticeship history where first-
hand experience of the student’s practice featured
highly. The figure also suggests that supervision was
well used to evidence competence and the reliance
on this was, on average, only slightly less than first-
hand experience of practice. Of the mean 4%
(Fig. 2) contribution made by PT-specified catego-
ries, the bulk of this related to patients’ opinion of
the students’ performance, where six of the seven
categories related to patient involvement in the
assessment process, e.g., ‘feedback from pa-
tients’, ‘client evidence’, ‘patients’ comments’,
‘role play/case studies [and] client feedback’ and
‘client feedback’.
Questionnaire items where significant differ-
ences were noted between pilot and non-pilot re-
spondents, together with respective means, are
presented in Table 3. Pilot PTs used significantly
less first-hand experience of the students’ practice
in their assessments and used significantly more
written portfolio evidence in this process.
Other interesting but statistically insignificant
results were noted. On one hand, non-pilot PTs
reported an almost 4% higher mean contribution of
evidence arising from discussing practice issues
with the student. On the other, however, pilot
group PTs reported an almost 3% higher mean
Portfolio assessment: practice teachers’ early experience 395

Table 3 Sources of evidence: pilot and non-pilot groups

Source of evidence Contribution to student assessment (mean %) P value
Pilot Non-pilot
PT use of first-hand experience of the student’s 28.8 43.7 0.02
practice in the assessment process
PT use of written evidence compiled in the 25.2 11.25 0.03
portfolio in the assessment process

Table 4 UKCC learning outcomes reported as ‘most challenging’ by PTs

Brief description of learning outcomesa % of total citations
Quality and development
Provide accurate and rigorously collated health data through health profiles
Identify individual potential through effective appraisal systems 62%
Initiate and lead practice developments
Undertake audit review and quality assurance activities
Implement strategies for staff appraisal, quality assurance and audit
Prescribing, promoting and standards
Prescribe from a nursing formulary 15%
Initiate health promotion in individuals, groups and communities
Lead the professional team to ensure quality assured standards of care
Practice issues
Assess and manage care needs in a range of settings
Act independently within a multi-disciplinary/multi-agency context 13%
Recognise ethical and legal issues which have implications for practice and take action
Create environment in which practice development is fostered, evaluated and disseminated
Public health and research
Plan, provide and evaluate skilled nursing care in differing environments
Stimulate an awareness of health and care needs at both individual and structural levels
Identify and select from a range of health and social agencies, those which will assist and
improve the care of individuals, groups and communities
Identify evolving health care needs and hazardous situations and take appropriate action 10%
Empower people to take appropriate action to influence health policies
Identify, apply and disseminate research findings relating to specialist nursing practice
a
UKCC (2001).

contribution of evidence arising from other prac-
titioners’ first-hand experience of the student’s
practice. Of all citations of sources of evidence
used in assessment, other than those prompted,
77% were attributable to pilot PTs compared with
8% to non-pilot PTs.
Further comparisons revealed that PTs who dis-
agreed that they were able to check their stan-
dards, for student evidence, against that of other
PTs were more likely to disagree that the standard
of student evidence required by them was similar
to that of other PTs. Unsurprisingly, the corollary
of this result was also found, where those who
agreed that they were able to check their stan-
dards also tended to agree that their standards
were similar to other PTs. Those PTs who tended to
agree that they were able to check their standards
also tended to disagree that they required further
guidance on the required breadth and depth of
student evidence.
The distribution of the 61 citations of learning
outcomes that PTs reported as posing ‘most chal-
lenge’ to students is depicted in Table 4. A range of
learning outcomes (18) was identified by PTs as
most challenging where 68% of PTs described at
least one of these outcomes in this way. Between
five and 11 PTs reported each of the learning
outcomes that constituted 62% of total citations
(Table 4).
Qualitative results
PT opinion was sought on a number of topics:
students’ interest and skills in self-evaluation,
practice and continuing professional develop-
396
ment, wider implications of critical practice in-
cidents, guidance on evidence quality and general
portfolio experience. There was evidence that
the use of the Benner scale supported students’
interest and skills development in self-evaluation,
where some PTs felt that the scale promoted the
students’ professional development more widely.
The value of considering inter-PT assessment re-
liability for continuing professional development
work was highlighted, as was the role of the
teaching team in facilitating this. The PT expe-
rience of portfolio use over two terms was gen-
erally a positive one where the positive affect on
student self-evaluation and professional practice
were made explicit.
Students’ interest in self-evaluation
PTs commented on the extent to which the port-
folio’s Benner scale had successfully developed the
student’s interest in self-evaluation and promoted
collaboration between student and PT:
Use of Benner scale assisted student to recognise existing
skills as well as identifying deficiencies.
Self-assessment well used by student. Also, a good basis
for reflective discussions from [PT] perspective, [the stu-
dent] was well motivated from the start.
Used Benner scale to develop self-evaluation skills and
consequently practice, very well. Often reflected on in
supervision.
Joint activity in first week of practice placement.
Other benefits identified were the extent to
which the scale had aided: the focusing of stu-
dents’ work, identification of learning needs and
the highlighting of the knowledge gaps at various
stages of learning outcome attainment.
Very well. We discussed it three times per term, marking
on each scale where they thought they were, how much
they had moved on. It focussed the student.
Positive identifying existing skills and knowledge and gaps
– I felt very informed – student able to clarify learning
needs.
Conversely, some informants felt limited success
with the scale in developing the learners’ interest
in self-evaluation:
We didn’t [use self-rating on the Benner scale]. We iden-
tified individual needs and progressed in practice and in
discussion to meet them.
As the tendency for students to underestimate
their skills was a recurring theme, PTs were often
involved in promoting the student’s more accurate
self-appraisal of skills:
I feel students still underestimate where they are and un-
der acknowledge where they have reached. They need to
W. Spence, W. El-Ansari
either do this with another or discuss with [PT]/tutor or in
a group situation.
Students’ self-evaluation skills
Participants acknowledged the constructive use of
the scale in the development of the students’ self-
evaluation skills over the two terms. Scale use
promoted: the sensitivity of self-assessment, evi-
dence of student progression and collaboration
between student and PT:
It encouraged these [self-evaluation skills] as we dis-
cussed why the point on the scale – what made it so.
Made her evaluate her skills beginning and end of term
and [the student] was able to see development and con-
fidence in her practice.
Student found helpful but initially was reluctant to self-
assess. Appeared to feel [PT] should make decisions.
Has developed self-evaluation skills with supervision in
using Benner’s scale.
Practice and continuing professional
development
The respondents suggested that the scale had the
potential to be used to develop personal practice,
however, there was some doubt about the extent
to which students had demonstrated a causal link
between personal practice development and the
scale’s use:
Excellent tool to encourage self-awareness. Catalyst for
professional development.
Not really in an explicit way but perhaps inevitably in an
implicit way.
With respect to PT personal development needs,
which might be met over the following term and
academic year (2001/2002), many expressed a wish
to explore the rigour and accuracy of student as-
sessment and inter-specialism PT reliability in the
assessment of practice and the nature of accept-
able evidence:
Agreement of standards of evidence, agreement of how
[PT] evaluates evidence.
More guidance re. what fitted each competency - some
are vague and need to be creatively interpreted.
When considering the wider implications of
practice incidents with regard to the relevant
social, psychological, political, community or
population diversity issues, the instrumental role
of the PT in realising the potential for reflection
and supervision to meet these objectives was
raised:
Portfolio assessment: practice teachers’ early experience
By discussion on reflection – looking at appropriate re-
flective models.
I feel that this is a key aspect of the reflection process be-
tween student and [PT] and can/should be facilitated by
the [PT].
Guidance on quality of evidence
The form of guidance requested by PTs, in relation
to their assessment of the adequacy of evidence
submitted, included the opportunity to develop
dialogue with peers and/or university staff:
In an ideal world it would be good to meet up with other
[PTs] to discuss the portfolio.
Some discussion about how [PTs] and students in differ-
ent pathways demonstrate evidence.
The wish to have access to previously completed
student evidence as a basis for this discussion was
also made clear:
Examples from portfolios that evidence a range of evi-
dence from students, accompanied by commentary from
university staff and [PTs].
Practice assessment over two terms
The improved guidance offered by the portfolio
approach in assessing practice over 20 weeks
was highlighted as was the need to share good
practice:
Guidelines and Benner scales have made assessment eas-
ier/more clarified. Will assist new [PTs] tremendously, as
I found competencies very daunting in my training year/
part 2 [of PT preparation programme].
Practice assessment is the core skill of the [PT]. Shared
experiences with other [PTs] and the academic staff is
key. Plus clear guidelines, examples, debate, exchange
with others.
However participants were critical of the UKCC
(2001) learning outcomes:
Very repetitive UKCC learning outcomes. It felt very rigid
and tended to dominate the placement. I would like to
see more emphasis on ‘innovation’ and student identifica-
tion of learning needs. It felt very task focussed.
Discussion
The professionalising form of action research em-
ployed here aimed to effect professional culture
change and improvement in professional practice
(Hart and Bond, 1995). There is recognition of va-
lidity and reliability limitations in assessment of
397
practice (Gerrish et al., 1997a), and the study
participants echoed such sentiments. The portfolio
approach described in this paper has gone some
way towards improving the rigour of the assess-
ment of students’ practice by PTs. The study has
several limitations. The small sample sizes and
response rates limit the validity and generalisabil-
ity of results. Also, findings relate only to the first
cycle of action research and this should be borne in
mind when considering the practice recommenda-
tions, which may be underdeveloped.
The first aim of the study was to determine the
PTs’ early experience of the portfolio approach to
practice assessment. The findings suggest that
where self-evaluation of learning is prompted,
improved self-assessment might reasonably be ex-
pected. Self-evaluation is consistent with an active
learning style (Fransson, 1977), which in turn is
linked to deep level information processing con-
ducive to improved understanding and recall
(Marton, 1978). Although the Benner (1984) scale
has yet to be thoroughly evaluated, it may promote
the participation in and identification of the
learning experiences outlined in the Steinaker and
Bell (1979) experiential taxonomy. These include:
exposure to, participation in, identification, inter-
nalisation and dissemination of the learning expe-
rience. In this study the enthusiasm for Benner’s
model (1984) was high given the challenges of op-
erationalising this model with respect to clinical
and communication skills (Nicol et al., 1996). The
students’ self-rating on the Benner scale was in-
tended to increase their skill in the identification of
learning needs in relation to each learning out-
come, if this skill building is often taken for gran-
ted in portfolio construction (Knowles, 1975).
One limitation of Benner’s model (1984) is its
lack of detail on the meaning of its stages in
practice (Nicol et al., 1996). As the scale offers
no explicit criteria for clinical skills assessment
(Brown, 2000), its usefulness is limited. In this
study the scale was not adopted for summative
assessment and it was formatively used by stu-
dents and PTs to provide feedback on perfor-
mance.
The introduction of the portfolio approach to
practice assessment was well received by PTs.
Nevertheless the high number of learning outcomes
and the repetitive nature of these led some PTs to
call for a reduction in their number. The increased
reliance of portfolio using PTs on written evidence,
their satisfaction with portfolio guidance offered,
and belief that the portfolio had promoted stu-
dents’ optimal level of control in personal learning
may be seen to demonstrate their confidence in its
effectiveness in guiding the compilation of evi-
 398
Box 3. Benner scale (After Benner, 1984)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Novice Advanced beginner Competent Proficient Expert
Performance is guided by Has prior experience of the Has a long range view of Has holistic understanding Designs and implements a
rules and objectives. situation. practice. of the situation. process for resolving
situations.
Lacks a strategy for Can identify overall impor- Develops a strategy for Coherent strategy for Highly developed, organised
practice. tant aspects of a situation. practice. practice developed. and integrated strategy for
practice.
Whole situation is not Unable to sort out priorities Practice informed by Adopts a problem-solving Decision-making based on
understood. in a situation. analysis and problems approach in each situation. sound analysis of the
solving. situation.
No experience of the Compares, contrasts and Identifies own practice Prioritises between Intuitive grasp of the
situation. discriminates aspects of goals and formulates competing variables. situation.
the situation. plans for achieving these.
Inability to discuss reasons Beginning to develop a Can manage many aspects Theories and well devel- Innovative and creative.
for practice. strategy for practice. within a situation oped perspectives guide
performance.
Following assessment of learning outcome: date . . . . . . . . . . . . . . . . . .
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Novice Advanced beginner Competent Proficient Expert
At the end of the programme: date . . . . . . . . . . . . . . . . . .

W. Spence, W. El-Ansari
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Novice Advanced beginner Competent Proficient Expert
Portfolio assessment: practice teachers’ early experience
dence. The findings suggest that the portfolio ap-
proach has the potential to promote reflective
practice.
The expectation that student specialist practi-
tioners might be able to achieve in 32 weeks the
outcomes identified as most challenging – in ad-
dition to the others – is an ambitious one (Table 4).
The challenge posed by appraisal and audit may
reflect the low priority that the NHS has tradi-
tionally afforded these activities although the de-
gree of challenge posed was not assessed here.
Similarly, that posed by the collation of health data
may reflect the traditionally low emphasis in
nursing on public health information and practice
(Table 4).
Secondly, the study aimed to compare the ex-
periences of portfolio using PTs with those using an
established approach across a number of variables.
The findings highlight anxieties relating to the reli-
ability of portfolio assessment, perhaps reflecting
the enormous scope for subjectivity in this process
(Ashworth and Morrison, 1991). Frequently with the
use of assessment instruments the rigour of their
construction comes into question (Chambers, 1998;
Glen and Hight, 1992; Gerrish et al., 1997a). The
challenges include the doubts about: the differen-
tiation of academic levels (Gerrish et al., 1997b),
validity in ambiguous and complex environments
(Phillips et al., 2000b), access to cognitive pro-
cessing via portfolio use (Mitchell, 1993) and inter-
rater reliability of portfolio use (Pitts et al., 1999).
In this study the doubts expressed about the per-
centage marking of practice, via the portfolio, re-
flected steering group concern about the validity
and reliability of portfolio assessment. Given that
50% of student effort was devoted to practice,
marking may offer a higher level of incentive for
students to develop their work in this area if reli-
ability and validity were better established.
The portfolio has promoted reflection on prac-
tice, which was a programme aim. Although there
was support for limiting the number of words used
in evidence compilation, portfolio evidence made a
substantial mean contribution to the PT judgement
of competence and the contribution of patient
opinion to this was considerable. This is consistent
with the NHS Plan’s (DoH, 2000) aim of increasing
the salience of the patient experience for health
care professionals. PTs who employed portfolios,
used their first-hand experience of student practice
significantly less in the student assessment process
than those using the established approach and this
may have reflected the former’s significantly
greater reliance on written evidence (Table 3).
Although the approach may have proved more
time efficient for some PTs, this does not neces-
399
sarily indicate more rigorous assessment or better
practice experiences for the students. The in-
creased reliance of portfolio using PTs on: other
practitioners’ experience of student performance,
the discussion of practice issues and use of other
sources of evidence in the assessment process may
reflect the increased responsibility afforded the
student in this process and the increased confi-
dence of the pilot PTs in the student’s ability to
exercise this in an independent fashion. It may also
reflect PT confidence in the structure offered by
the portfolio to guide student learning. The use of
consumer feedback by the PT as a source of evi-
dence is encouraging and in line with the current
NHS focus on the patient’s experience (DoH, 2000).
Lastly the study aimed to determine the PTs’
view of personal continuing professional develop-
ment needs regarding practice assessment. Doubts
about inter-PT reliability in the assessment process
were clear and have previously been expressed by
Orchard (1992). This has subsequently become a
focus of the continuing professional development
sessions offered to PTs by university staff. How-
ever, PTs who were able to check their standards
with other PTs reported less need for further
guidance which has implications for establishing PT
support systems in practice that aim to increase
inter-PT reliability. Since the completion of the
research the teaching team has continued to de-
velop the portfolio in line with the study’s findings.
Developments have included the continuous revi-
sion of outcome attainment guidance, the use of
questionnaires to elicit patient and informal carer
feedback on student performance, adoption of the
portfolio approach by all pathways involved in the
study and portfolio prompts to enable a more de-
tailed breakdown of practice assessment evidence
used by PTs.
Conclusion
The portfolio approach to practice assessment in
nursing is supported by the adult learning principles
of Knowles (1975). Shortly after the introduction of
the portfolio approach, the study findings revealed
a greater level of positive evaluation of it than
expected.
Portfolio prompts supported the students’ self-
evaluation of learning, and in particular the Benner
scale proved useful despite its acknowledged limi-
tations. The incorporation of patients’ evaluations
of student performance and supervision discussion
in assessment may reflect the professional maturity
of practice teaching where these factors might
traditionally have been afforded much less impor-
400
tance in student evaluation. The complexity of
many of the UKCC (2001) specified learning out-
comes has implications for the rigour of their as-
sessment across course centres.
Pilot PTs significantly lower reliance on first-
hand experience of the student’s practice and
significantly more on written evidence of compe-
tence may be part of a maturational process in
nursing. The study has prompted continuing work
on inter PT reliability in portfolio assessment. The
inclusion of consumers in the evaluation of stu-
dents was identified by the study and this repre-
sents a growing area of professional interest.
Some support for the portfolio’s potential as a
valid assessment mechanism was evident although
respondents expressed concerns for inter-rater
reliability. Further work relating to the rigour of
portfolio assessment in this context is required,
and further scrutiny and substantiation are called
for, given the many benefits that this approach has
to offer professional preparation. The results of
this study offered encouragement to develop this
assessment mechanism further.
Recommendations
1. Student involvement: The portfolio approach
has increased the level of student engagement
with self-evaluation of learning. However, the
provision of numerous prompts to support
self-evaluation in the portfolio documentation
is imperative, e.g., Benner scale, clinical su-
pervision recording schedule, clinical supervi-
see and supervisor skill self-rating. Secondly,
widening the range of acceptable evidence
and student autonomy in the identification
of, and engagement with, practice, contributes
to increasing the students’ reflective style.
2. Guidance on learning outcome attainment:
Guidelines need to be collectively generated
and informed by the PTs and lecturing staff.
3. Practice/theory split: Given the multi-faceted
nature of the learning outcomes, the require-
ment that the student experience is comprised
of an equal 50% practice and 50% theory may
limit course developers and an increase in the
practice portion of these programmes is recom-
mended.
4. Inter-PTA reliability: The ongoing consideration
of reliability in practice assessment across PTs in
the continuing professional development of this
group is recommended.
5. Practice assessment group: The establishment
of a group of lecturing and PT staff to oversee
W. Spence, W. El-Ansari
practice assessment has highlighted the salient
and dynamic nature of the practice component
of the programme and is recommended.
6. Consumer involvement: The incorporation of
consumer evaluation of health services has de-
veloped slowly over recent years. The use of pa-
tient and informal carer feedback in the
assessment of students is recommended and this
may form part of the student’s portfolio of evi-
dence of learning outcome attainment.
Acknowledgements
The participation of practice teachers in this re-
search is gratefully acknowledged as is Dr. Diane
Pearson’s assistance in reading and commenting on
the paper.
References
Ashworth, P., Morrison, P., 1991. Problems of competence-
based nurse education. Nurse Education Today 11, 256–260.
Benner, P., 1984. From Novice to Expert: Excellence and Power
in Clinical Nursing Practice. Addison-Wesley, Menlo Park, CA.
Black, T., 1999. Doing Quantitative Research in the Social
Sciences. Sage, London.
Black, H., Wolfe, A., 1990. Knowledge and Competence: Current
Issues in Training and Education. Careers and Occupational
Heath Information Centre, London.
Brown, N., 2000. What are the criteria that mentors use to make
judgements on the clinical performance of student mental
health nurses. An exploratory study of the formal written
communication at the end of clinical nursing practice
modules. Journal of Psychiatric and Mental Health Nursing
(7), 407–416.
Chambers, M.A., 1998. Some issues in the assessment of clinical
practice: a review of the literature. Journal of Clinical
Nursing (7), 201–208.
Creswell, J.W., 1998. Qualitative Inquiry and Research Design:
Choosing Among Five Traditions. Sage, London.
Department of Health, 2000. The NHS Plan: A Plan for Investment
a Plan for Reform Cm 4818-I. The Stationery Office, London.
Elbow, P., Belanoff, P., 1997. Reflections on an explosion:
portfolios in the ’90s and beyond. In: Yancey, K.B., Weiser, I.
(Eds.), Situating Portfolios: Four Perspectives. Logan Utah
State UP, Utah.
English National Board for Nursing Midwifery and Health Visiting,
2000. Practice Assessment: An Evaluation of Practice at
Diploma, Degree and Postgraduate Level in Pre- and Post-
registration Nursing and Midwifery Education. ENB, London.
Fetterman, D.M., 1989. Ethnography: Step by Step. Applied
Social Research Methods Series, vol. 17. Sage, Newbury Park,
CA.
Fox, D.T., 1982. Fundamentals of Research in Nursing. Appleton
Century Crofts, Connecticut.
Fransson, A., 1977. On qualitative differences in learning:
effects of motivation and test anxiety on process and
outcome. British Journal of Educational Psychology (47),
244–257.
Gerrish, K., McManus, M., Ashworth, P., 1997a. Levels of
Achievement: A Review of the Assessment of Practice.
 Portfolio assessment: practice teachers’ early experience
English National Board for Nursing Midwifery and Health
Visiting, London.
Gerrish, K., McManus, M., Ashworth, P., 1997b. The Assessment
of Practice at Diploma Degree and Postgraduate Levels in
Nursing and Midwifery Education: Literature Review and
Documentary Analysis. English National Board for Nursing
Midwifery and Health Visiting, London.
Gibbons, J.D., Chakraborti, S., 1992. Nonparametric Statistical
Inference, third ed. Dekker, London.
Glen, S., Hight, N., 1992. Portfolios: an ‘affective’ assessment
strategy? Nurse Education Today (12), 416–423.
Hart, E., Bond, M., 1995. Action Research for Health and Social
Care: A Guide to Practice. Open University Press, Bucking-
ham.
Illich, I.D., 1971. Deschooling Society. Caldar and Boyars, London.
Knowles, M.S., 1975. Self Directed Learning. Cambridge Pub-
lishing, New York.
Knowles, M.S., 1980. The Modern Practice of Adult Education:
From Pedagogy to Andragogy. Follet, Chicago.
Lewin, K., 1946. Action research and minority problems. In:
Lewin, G.W. (Ed.), Resolving Social Conflicts: Selected
Papers on Group Dynamics by Kurt Lewin. Harper and
brothers, New York.
Marton, F., 1978. Describing Conceptions of the World About Us.
Research Report from The Institute of Education. University
of Gothenburg, Gothenburg.
Mayer, M., 1993. Action research, history and images of science.
Educational Action Research 1 (2), 317–319.
Mitchell, L., 1993. The Contribution of S/NVQS. De Montfort
University, Leicester.
Neary, M., 2000. Teaching, Assessing and Evaluation for Clinical
Competence: A Practical Guide for Practitioners and Teach-
ers. Nelson Thornes, Cheltenham.
Nicol, M.J., Fox-Hiley, A., Bavin, C.J., Sheng, R., 1996.
Assessment of clinical and communication skills: operation-
alizing Benner’s model. Nurse Education Today (16),
175–176.
Norman, I.J., Watson, R., Murrells, T., Calman, L., Redfern, S.,
2002. The validity and reliability of methods to assess the
competence to practise of pre-registration nursing and
midwifery students. International Journal of Nursing Studies
(39), 133–145.
Nursing and Midwifery Council, 2003. NMC agrees definition of
lack of competence: NMC press statement issues on
21.1.2003. Available from: http://www.nmc-uk.org/nmc/
main/pressStatements/nmcAgreesDefinitionOfLackOfCom-
petence.html. Retrieved, 20.11.2003 at 20.13 h.
Orchard, C., 1992. Factors that interfere with clinical judge-
ments of students’ performance. Journal of Nursing Educa-
tion (31), 309–313.
View publication stats
401
Oppenheim, A.N., 1992. Questionnaire design, interviewing and
attitude measurement second ed. Lonond, Pinter.
Phillips, T., Bedford, H., Robinson, J., Schostak, J.F., 1994.
Education Dialogue and Assessment: Creating Partnership for
Improving Practice. English National Board for Nursing
Midwifery and Health Visiting, London.
Phillips, T., Schostak, J., Tyler, J., 2000a. Practice Assess-
ment in Nursing and Midwifery: Doing it For Real. English
National Board for Nursing Midwifery and Health Visiting,
London.
Phillips, T., Schostak, J., Tyler, J., Allen, L., 2000b. Practice
and Assessment: An Evaluation of the Assessment of Practice
at Diploma Degree and Postgraduate Level in Pre- and Post-
Registration Nursing and Midwifery Education. The English
National Board for Nursing Midwifery and Health Visiting,
London.
Pitts, J., Coles, C., Thomas, P., 1999. Educational portfolios in
the assessment of general practice trainers: reliability of
assessors. Medical Education (33), 515–520.
Ramsden, P., 1992. Learning to Teach in Higher Education.
Routledge, London.
Redfern, S., Norman, I., Calman, L., Watson, R., Murrells, T.,
2002. Assessing competence to practise in nursing: a review
of the literature. Research Papers in Education 17 (1),
51–77.
Rogers, C.R., 1983. Freedom to Learn For The 80’s. Merrill,
Columbus, OH.


Rousseau, J., 1993. Emile. Everyman, London.
Runciman, P., 1990. Competence-Based Education and the
Assessment and Accreditation of Work Based Learning in
the Context of Project 2000 Programmes of Nurse Education:
A Literature Review. National Board for Nursing Midwifery
and Health Visiting for Scotland, Edinburgh.
Scho€n, D., 1983. The Reflective Practitioner. Basic Books, New
York.
Snadden, D., Thomas, M., 1998. The use of portfolio learning in
medical education. Medical Teacher (20), 192–199.
Steinaker, N., Bell, M., 1979. The Experiential Taxonomy: A New
Approach to Teaching and Learning. Academic Press, New
York.
United Kingdom Central Council for Nursing, Midwifery and
Health Visiting, 1999. Fitness for Practice. UKCC,
London.
United Kingdom Central Council for Nursing, Midwifery and
Health Visiting, 2001. Standards for Specialist Education and
Practice. UKCC, London.
Whittington, D., Boore, J., 1988. Competence in nursing.
In: Ellis, R (Ed.), Professional Competence and Quality
Assurance in the Caring Professions. Chapman & Hall,
London.