Russell, J. D. & Butcher, C. (1999).

Using portfolios in educational technology

courses. Journal of Technology and Teacher Education, 7(4), 279-289.
ABSTRACT
This article describes the evolution of student evaluations. Traditionally a major, if
not the sole, method of student evaluation was a paper-and-pencil test. In the past
some instructors assigned a number of small-scale, disconnected practice exercises.
Later they offered a menu of projects of a larger scale. Today the trend is to
encourage students to compile portfolios of professional quality work, organized
around major themes. The nature of portfolios and their advantages and limitations
will be explored. The use of portfolios in two different educational technology
courses will be described. "In many schools, students are being evaluated on the
basis of portfolios that document what they can do in language arts, science, social
studies, and other skill areas. Portfolios often include such items as studentproduced books, videos, and audio-visual presentations," (Heinich, Molenda,
Russell, & Smaldino, 1999, p. 20)
EVOLUTION OF PROJECTS
Educators frustrated with standardized testing and conventional paper-and-pencil
assignments are having students demonstrate their achievements by compiling
portfolios of their work. Many educators believe that preparation of portfolios gives
a truer, more rounded view of an individual's strengths and weaknesses. "Further,
portfolio assessment is consistent with the constructivist philosophy, which
emphasizes that what is important is the knowledge that students themselves
construct," (Heinich et al., 1999, p. 20).
With increased attention on multidisciplinary inquiry, open-ended explorations,
and more complex learning tasks, the focus of evaluation has shifted from
traditional student achievement measures (e.g., paper-and-pencil tests) to case
studies of student work (e.g., portfolio outcomes) (Seely, 1994).
One example of this shift is the evolution of projects in media utilization courses.
Twenty-five years ago many instructors required every student to do the same
small-scale projects--a bulletin board, an audio tape, one dry mount, one
permanent rubber cement mount, one temporary rubber cement mount, and so
forth. Later instructors provided a list of projects from which students could choose.
Now many instructors are using portfolios--a collection of projects often related to a

single topic and a single audience. Portfolios should include student comments and
reflections about their projects.
WHAT IS A PORTFOLIO?
Paulson, Paulson, and Meyer (1991) describe a portfolio as, "A purposeful,
integrated collection of student work showing student effort, progress, or
achievement in one or more areas. The collection is guided by performance
standards and includes evidence of students' self-reflection and participation in
setting the focus, selecting contents, and judging merit" (p. 295).
COMPONENTS OF A PORTFOLIO
The idea of portfolio assessment, then, is to measure students' achievements by
their ability to create tangible products exemplifying their talents. Portfolios may
contain the following components:
* written documents such as poems, stories, lesson plans, or research papers;
* media presentations, such as slide sets or photo essays;
* audio recordings of debates, panel discussions, or oral presentations;
* video recordings of the student's athletic, musical, or dancing skills;
* computer spreadsheets of student data; and
* multimedia projects incorporating print, data, graphics, and moving images
(Heinich et al., 1999).
ADVANTAGES OF PORTFOLIOS
There are many advantages for both students and instructors when using
portfolios. Table 1 lists several of the advantages to using portfolios.
LIMITATIONS OF PORTFOLIOS
Even though there are many advantages of using portfolios, they also have
limitations. Table 2 lists most of these limitations:
APPLICATIONS
Portfolios have been used for peer review, to show progress and experiences, to
demonstrate abilities and experiences, and during employment interviews. Many
educational institutions are using portfolios to document professional development
of both preservice and inservice teachers (Cole, Ryan, & Kirk, 1995). Other uses
include showing competencies, focusing on learning, reflecting on experiences, and
assessing accomplishments.

We have used portfolios in an instructional media course for preservice and

inservice teachers and in a technology for teachers course. Fortunately the classes
enrolled fewer than 30 students and the staff included two instructors (a professor
and a graduate student) that facilitated grading of portfolios. For large-enrollment
courses it may be necessary to cut back on the magnitude of the portfolios or to
increase the number of evaluators.
MEDIA UTILIZATION COURSE
Several years ago the first author began using a portfolio approach to evaluation
rather than written quizzes and a final exam in the media utilization course at
Purdue University. A co-instructor, Peg Ertmer, suggested the idea and proved you
can "teach an old dog new tricks!" After trying portfolios for one semester, paperand-pencil quizzes and unrelated projects for portfolios were a thing of the past.
How a portfolio approach to evaluation is being used rather than written quizzes
and a final exam in a media utilization course will be described. Students select,
modify, design, and produce a collection of media and methods to teach a topic of
their choice to an audience of their choice. At the end of the course, the complete
portfolio is also evaluated in terms of how well the portfolio "hangs together" as an
integrated product.
The topics tend to be broad ones like aviation, biology, calculus, dinosaurs,
engineering, forestry, home economics, industrial technology, and so forth.
Portfolios encourage the integration of more than one content or subject matter
area (Tippens, 1998). The audiences may be specified by grade level (fifth graders,
middle school students), employment levels (assembly line workers, upper level
managers) or age level (senior citizens, preschool children) with whom the students
are familiar. Allowing students to select their topic and audience adds to the interest
and individualistic factors by insuring the projects for inclusion in their student
portfolios are meaningful and useful to the student's current situation. This also
reduces student resistance to portfolio creation by making the time spent
worthwhile because they are creating materials that can be used in their
professional life.
Students meet individually with the instructor(s) to discuss their topic and
audience as well as their course goals. This meeting assists in guiding the students
to create materials that are meaningful to them and not just to "please" the
instructor but with the focus on what would be useful to the student either currently
or in the near future.

Select. Students use the Appraisal Checklists in either paper or electronic form
included in Instructional media and technologies for learning (6th ed.) to preview
different media/materials for possible inclusion in their portfolios. Each preview is
accompanied by a brief explanation (one or two paragraphs) on the effectiveness of
that medium in teaching the portfolio topic that the student has chosen. If the
material was judged appropriate to be included, the student specifies how it would
be used.
Modify. Students also modify existing media/materials. These materials are
selected based upon congruence with the portfolio topic and audience. For example,
a student could add slides of a local environment to a commercial slide set or use
an existing set of visuals but change or omit some of the labels or captions.
Students are cautioned to be sure their handling and use of materials does not
violate copyright laws and restrictions that are described in the textbook.
Produce. As another part of the course portfolio, each student produces different
pieces of media for the chosen topic and audience. They are required to incorporate
different types of media which encourages them to expand on their current media
production skills. For example, they cannot produce two audiotapes, but they could
produce an audiotape, a bulletin board, a set of overhead transparencies, and a
videotape. The choice of media formats is up to each student, but they must to be
varied. By requiring students to include a variety of projects, the completed
portfolios will be more representative of their overall abilities rather than just their
skills in one area.
Where appropriate, the projects are evaluated using the corresponding Appraisal
Checklist from the textbook. This serves three purposes, it guides instructor
evaluations by giving them a grading rubric. For the student, the rubric (Appraisal
Checklist) gives them practice in using concepts covered in class and allows them to
"grade their projects" in advance using the same rubric.
Design. The student again chooses the topic and audience. The recommended
length of the lesson is from 30 minutes to one hour. The student can use the
printed template in the textbook or the computer template in "The Classroom Link"
as a guide. Each student designs a lesson using the ASSURE Model described in the
textbook. The design must include:
* Analyze Learners--including the general characteristics, entry competencies,
and learning styles of the students/trainees.
* State Objectives--describing the learning outcomes and are written in the
Audience, Behavior, Conditions, and Degree (ABCD) format.

* Select Methods, Media, and Materials--providing a rationale as to why the

methods, media, and materials have been selected, modified, or produced.
* Utilize Materials--describing how the materials and media will be used. The
student describes how she/he will preview the materials, prepare the materials,
prepare the environment, prepare the learners, and provide the learning experience
(the 5 Ps of utilization).
* Require Learner Participation--describing the activities which will get the
learners involved, particularly through practice with feedback.
* Evaluate and Revise--indicating how the objectives will be measured and
including evaluation of learner achievement, media, methods, the instructor's role,
and the overall process. To ASSURE quality instruction, it is important to evaluate
and revise the experience, if necessary, before future utilization.
EVALUATION OF STUDENT PORTFOLIOS
The individual components (select, modify, produce, and design) are evaluated
as they are completed. Students are encouraged to make weekly progress and not
to put everything off to the end of the course. This spreads the instructor's
evaluation time over the course of the semester instead of requiring enormous
amounts of time at the end of the semester. It also gives students frequent, timely
feedback to assist in their developmental process and understanding over the
semester.
At the end of the course, the complete portfolio is evaluated in terms of how well
the portfolio "hangs together" as an integrated product. How well do the pieces fit
together to support a user's understanding of the selected topic? Is the portfolio
well organized, making it easy to locate specific items? Is the work neatly arranged
and attractively presented? A well-organized portfolio is helpful in demonstrating
student's skills and abilities to current and/or future employers.
TECHNOLOGY COURSE
In an instructional technology course that integrates computers, media, and
instructional design at Florida State University, portfolios are used as the major part
of the course evaluation. This course typically enrolls 30 students. The portfolio
format was based upon the work of a co-instructor Terri Buckner. The textbook for
the course is Instructional Technology for Teaching and Learning (Newby, Stepich,
Lehman, & Russell, 1996).
Goals. Students state their goals for the course. Students in the class include

undergraduate education majors and graduate students who are employees of the
Florida Department of Education. Students then meet with an instructor to discuss
and review their goals. The goals can be modified and adjusted during the course.
This is one of the advantages of portfolios--students can determine what they want
to learn and how they will demonstrate their knowledge and skills.
Artifacts. Materials (lesson plans, work samples, materials developed,
evaluations, position papers) are used to illustrate progress toward the stated
goals. The nature of the goals determines how the artifacts are organized in the
portfolio. Each section includes an index listing the artifacts in that section. The
artifacts pose a potential problem--the quantity of materials in a portfolio may be
overwhelming. It is a question how to display and store the materials. The final
portfolios are very bulky and difficult for the instructor to carry from the classroom
to his/her office.
Caption. A short written description is included with each artifact stating what
the artifact represents in the portfolio. The purpose of each caption is to convince a
reader (and the instructors) that learning has occurred. Each caption/description is
a part of that argument. It is recommended that descriptive captions be between
50 and 250 words. The caption "forces" the student to identify the relevance of
each artifact to current study or future work.
Rubrics. Each student develops the rubrics (explanation of evaluation criteria) for
assessing her/his portfolio. Each section of the portfolio includes a scoring rubric
used to "quantify" the student's learning. For example, a student might want to
assess each section on a 5-point scale. Each scale point has a description of what
would constitute a rating at that level. The student provides a description of each
scale point. The rubrics developed by the students may not reflect the relative
importance of the tasks, products, or other components being evaluated.
Self evaluation. Students complete a self reflection comparing their results with
their intentions. Students evaluate their own portfolios twice during the course
(near the middle and end of the semester). Each evaluation includes the strengths
and weaknesses of the goal statements, the artifacts included, and a self-reflection
on their progress in learning about technology, teaching, and learning. These selfevaluations reflect what was learned to date and what still needs to be explored.
The end-of-course evaluation addresses how the student plans to proceed after the
course. The self-evaluation may not be an accurate assessment of the value of the
portfolio. Some students over estimate the quality of their portfolios while others
under estimate the quality.

Peer review. Twice during the course, two classmates review each portfolio using
the rubrics developed by the student who produced the portfolio. These evaluations
occur at approximately the one-third and two-thirds point in the semester. The
purpose of peer reviews is to help students look at their own learning from a
different perspective. The reviewers are given the goal statement, the artifacts, the
captions, and scoring rubrics. This is not a grading session, but an opportunity for
students to share what they have done and to receive feedback from peers for the
purpose of improving the learning experience. Copies of the written review are
given to the students who authored the portfolio, the student reviewer, and the
instructors. Reviewers look for a match among the goal statements, the artifacts,
and captions produced. If the scoring rubrics are not understandable, the student is
given an opportunity to revise them before the end of the course. Both the portfolio
developer and the reviewer learn from the experience. Peer feedback includes an
oral report (debriefing) between the two students followed up by a written report
that becomes part of each student's portfolio. As with self-evaluation, the peerevaluation may not be an accurate assessment of the value of the portfolio. Some
peers overestimate the quality of portfolios while others underestimate the quality.
Instructor Review. The instructor review procedure is identical to the peer review
except it is graded. The instructor evaluations occur midway through the course
and at the end of the course. The instructor review provides an opportunity for indepth evaluation of student products and processes. It provides the advantage of
assessing student learning both for formative evaluation and revision of the
instructional activities in the course as well as summative evaluation to determine
student grades.
RESEARCH FINDINGS
Wade and Yarbrough (1996) found important implications for instructors when
using portfolios in reflective teacher education. Our experiences with using
portfolios in these educational technology courses support their findings. To
enhance student learning with portfolios, Wade and Yarbrough (1996) recommend
the following:
1. Focus attention on students' initial understanding of the portfolio process, its
purpose in the course, and its role in enhancing reflection. Share examples of
student portfolios in class.
2. Encourage student ownership, individual expression, and making connections
between assignments and outside of class interest. Focus early in the semester on

potential uses for the portfolio.

3. Provide structure in the forms of some required portfolio assignments, due
dates, specific times for in-class sharing, and constructive feedback from both other
students and the instructor(s).
4. Evaluate the portfolio process and use of the portfolios by students.
Instructors need systematic feedback from students about potential uses and
meaningfulness of the portfolios (pp. 77-78).
SUMMARY
Portfolios have been used for peer review, to show progress and experiences,
and during employment interviews. Other uses include showing competencies,
focusing on learning, reflecting on experiences, and assessing accomplishments.
Some colleges and universities use portfolios as a graduation requirement.
Portfolios can make learning more interesting. They tend to be individualistic and
can allow each student to determine what they want to learn during a course. As a
perspective over time during a course or curriculum, portfolios pull together a lot of
information and artifacts. These materials lend themselves to formative evaluation
and revision. Portfolios have the added advantages of allowing students to organize
their knowledge, skills, and materials, to develop an in-depth understanding of the
content, and to show peers and professionals what they have learned and can do.
Instructors who have used portfolios point out that they may require more time
from the student and the instructor. Another concern was lack of precision in
evaluating portfolios. Since portfolios are individualized, not all students may learn
the skills taught in a course. The relative emphasis on process versus content was
mentioned. An over emphasis on the electronic aspects of portfolios was seen as a
potential limitation. A final drawback was the quantity of materials in a portfolio
and how to display and store these materials.
ADDED MATERIAL
JAMES D. RUSSELL AND CHERYL BUTCHER
Purdue University 1442 Liberal Arts and Education Building West Lafayette, IN
47907-1442, USA jrussell@purdue.edu
Table 1 Portfolio Advantages
Interest
Individualistic

makes learning more interesting

allows each student to determine what they want to learn
and how they will demonstrate their knowledge and skills

Relevance

should be designed to be highly relevant to current study

or future work

Developmental

provides a perspective over time during a course or

curriculum

Variety

includes a lot of information and artifacts

Organization

allows students to organize their knowledge, skills, and

materials

Understanding

leads to an in-depth understanding of the content

Demonstration

shows peers and professionals what they have learned

and can do

Evaluation

provides a method for students and instructors to assess

learning, for students to reflect on their work and abilities,
and for formative evaluation and revision of instructional
activities

Table 2 Portfolio Limitations

Time

requires more time from the students and the instructors

than other evaluation approaches

Resistance by students

are not appreciated and understood by some students;

they perceive them as more work and may not see the
benefits
Research

lack of research evidence in value of portfolios

Evaluation

may not reflect most important aspects of task, product,

or course and "grade" influenced by person doing the
evaluation

Content vs. process

some students and teachers may focus on the content

to
the exclusion of the process
Over emphasis on
technology

there may be an over emphasis on the

electronic aspects of portfolios to the exclusion to simpler
ways to demonstrate progress

Amount of Material

the quantity of materials in a portfolio may be

overwhelm
ing; how to display and store these materials becomes
problematic

Representative

may not represent work or ability of students

Teacher-Centered

may be designed to "please" the teacher rather than to

meet the needs of the student

REFERENCES
Cole, D.J., Ryan, C.W., & Kirk, F. (1995). Portfolios across the curriculum and
beyond. Thousand Oaks, CA: Corwin Press.
Heinich, R., Molenda, M., Russell, J.D., & Smaldino, S. (1999). Instructional
media and technologies for learning (6th ed.). Englewood Cliffs, NJ: Merrill/Prentice
Hall.
Kimeldorf, M. (1994). A teacher's guide to creating portfolios. Minneapolis, MN:
Free Spirit Publishing.
Newby, T.J., Stepich, D.A., Lehman, J.D., & Russell, J.D. (1996). Instructional
technology for teaching and learning. Englewood Cliffs, NJ: Merrill/Prentice Hall.
Paulson, F.L., Paulson, P.R., & Meyer, C. (1991). What makes a portfolio a
portfolio? Educational Leadership, 48(5), 60-63.
Seeley, A.E. (1994) Portfolio assessment. Westminister, CA: Teacher Created
Materials.
Tippens, D. Using portfolios in integrating academic and vocational curricula, The
Delta Kappa Gamma Bulletin, 64(3), 48-53, 1998.
Wade, R.C. & Yarbrough, D.B. (1996). Portfolios: A tool for reflective thinking in
teacher education? Teaching & Teacher Education, 12(1), 63-79.