Sie sind auf Seite 1von 10

Running head: LMS EVALUATION RUBRIC

1"

LMS Evaluation Rubric

Megan Dodworth, Quentin Flokstra, Gurpreet Koonar, D'Alice Marsh

ETEC 565A Section 64A

University of British Columbia

Dr. Natasha Boskic

September 26, 2014

LMS EVALUATION RUBRIC

2"

Précis

Our team’s given task was to produce an evaluation tool to aid in the selection of the

most suitable Learning Management System (LMS) for UBC’s Undergraduate Medical

Program. The medical program is in need of an online education platform to support

distance education at varying rural locations throughout British Columbia. The selected

LMS must aid in the clinical assessments of medical students in Year 3 Rural Family

Practice Clerkships. Third year students will be videotaped demonstrating clinical skills

and need to be able to receive instructor and peer feedback. Therefore, the LMS must

include video capabilities, which includes the uploading of multiple clips and reliable

playback, as well as discussion forums and multimedia objects. Moreover, the video

capabilities need to include the ability to provide detailed feedback from both instructors

and peers. Furthermore, the students should be able to access other educational artifacts

and tools that will be available on the LMS.

Rational

Our group created the following rubric by drawing and merging information from Bates

and Poole (2003) in A Framework for Selecting Technology, and Panettieri (2007) in

Addition by subtraction.We utilized Bates & Poole’s SECTIONS model, as well as their

questions for consideration, to guide in the creation of the categories and subcategories

that would be most significant in this particular scenario. The SECTIONS model was

originally created to support distance education, for technology to reach students who

cannot attend conventional class (Bates & Poole, 2003). As the medical students will be

carrying out their clerkships at varying location, we thought this to be the most

appropriate fit as communication across the sites needs to be effective. As this rubric was

LMS EVALUATION RUBRIC

3"

created, factors such as the heavy workload of medical students, privacy, and

technological skills were all considered. Panettieri’s Addition by subtraction article

mentions categories of system enhancements such as website structure, interactions with

students and ease of use which were also explored in our rubric.

LMS EVALUATION RUBRIC

!

4!

Learning(Management(System((LMS)(Evaluation(Rubric( – ( UBC’s(Undergraduate(Medical(Program((UMP)(Advisory(Committee (

( LMS(Model: ( Evaluator: ( Date: ( Criteria ( Continuum ( ( ( Category! Subcategory!
(
LMS(Model: (
Evaluator: (
Date: (
Criteria (
Continuum (
(
(
Category!
Subcategory!
3( – ( Exemplary(
2( – ( Adequately(
Acceptable (
1( – ( Limited (
0( – ( Inadequate(
Comments(
Score (
(
(
(
(
(
(
(
(
Students(
Student(
Reliable and fast
bandwidth without
limitations
Reliable internet
Demographics(
connect; medium
Access to internet,
however connection
not always reliable;
No access to
internet
!
!
&(Access (
bandwidth speed
!
!
slow bandwidth
speed
!
!
Privacy((
Secure, password
Password protected
Does not require a
!
!
!
protected server
Password protected
but not encrypted
!
password
!
!
!
!
Restricted access to
students & teachers
within the class;
Users must be
granted permission
for access
Restricted access to
registered users
(students, teachers)
Access to all LMS
users regardless of
program
Public access to all
postings
!
!
!
!
!
!
!
!
!
!
Ease(of(Use! !
Computer(&(
!
Information(
(for!Students!&!
Literacy(
Familiar platform
for year three
students to use
Familiar platform
with some new
features to learn
The platform is
difficult to
understand
Platform is
complex and
difficult to learn
!
!
!
Faculty)!
!
!
!
Orientation (
Tutorial is provided,
is quick and
efficient
Tutorial is easy to
follow but requires
some time
Tutorial is too long
and complex
There is no tutorial
provided
!
!
!
!

LMS EVALUATION RUBRIC

!

5!

Reli ability ( Tech support Tech support is No tech support ! ! ! available
Reli ability (
Tech support
Tech support is
No tech support
!
!
!
available 24/7
available during
!
peak hours
Tech support is not
reliable and
available in times of
!
!
need
!
!
Server is consistent
Server is fairly
consistent
Server is consistent
only for limited
times
Server is
!
!
inconsistent
Server operates via
high bandwidth
Server operates via
medium bandwidth
Server operates via
low bandwidth
Server operates
with little or no
bandwidth
!
!
Scheduled
maintenance
occurring at slow
periods of usage
Scheduled
maintenance does
not interfere with
work periods
Scheduled
maintenance is often
disrupting learning
Scheduled
maintenance does
not allow
for students to log
in most of the time
!
!
Interface(
Design(
Ability to create
videos and embed
!
and/or link to other
Ability to create
video but not share
to other sites
Ability to hyperlink
video only
Ability to use
video recording
provided by LMS
!
!
video sharing sites
!
!
!
only; no ability to
link to other video
sites
!
No restriction on
duration and length
of videoclips
Length of videoclips
are an appropriate
length
Limited upload
length for videos
Length of
!
!
videoclips is
restricted
single-use log in to
access all tools
use of single
username for only
some tools, others
require multiple
login
use of single
username but
requires multiple
logins to access all
tools
Multiple user
names and logins
required
!
!

LMS EVALUATION RUBRIC

!

6!

( ( ! ! Cost ( Items(of( Purchase of LMS Purchase of the LMS system
(
(
!
!
Cost (
Items(of(
Purchase of LMS
Purchase of the
LMS system and
design is within the
department’s
technology budget
Purchase of the
LMS system and
design is slightly
higher the
department’s
!
!
Expenditure(
system and design
is substantially
lower than the
department’s
technology budget
!
technology budget
Purchase of the
LMS system and
design does greatly
exceeds the
department’s
technology budget
!
!
!
All software
upgrades are
included in the
initial purchase cost
Limited software
upgrades are
available at no
additional costs,
All software
upgrades are
available at an
additional costs
Software upgrades
are not available
with the LMS
system
!
!
!
other software
upgrades are
available at
additional costs
!
!
!
!
!
Additional features
and widgets are
provided at with no
additional charges
Basic additional
features are
provided at no cost,
superior additional
features and widgets
Additional features
and widgets are
available at an
additional cost
Additional features
and widgets are
not available for
purchase
!
!
!
!
are required to be
purchased
!
!
Drivers(of(
!
!
Costs (
Ongoing cost ratio
per student is
substantially lower
than the
department’s
technology budget
Ongoing cost ratio
per student is within
the department’s
technology budget
Ongoing cost ratio
per students is
slightly higher the
department’s
technology budget
Ongoing cost ratio
per student greatly
exceeds the
department’s
technology budget
!
!
!
!

LMS EVALUATION RUBRIC

!

7!

( ( ! ! Teaching(&( Content(&( Comprehensive ! ! Learning( Skills( video editor; student is
(
(
!
!
Teaching(&(
Content(&(
Comprehensive
!
!
Learning(
Skills(
video editor;
student is able to
add video
commentary for
explanations of
decisions made
Adequate video
editor; student is
able to add some
video commentary
for explanations and
decisions
Limited video
editor; student is not
able to add video
commentary for
explanations and
decisions
No video editor;
student is not able
to add any video
commentary
!
!
!
!
Student(
Assessment(
Strategies(
Easily view and
receive peer and
teacher feedback in
text, audio, visual
Able to view and
receive peer and
teacher feedback in
only some forms
No opportunity to
view and receive
peer and teacher
feedback
!
!
!
forms
Restricted and
limited options to
view and receive
peer and teacher
feedback
!
!
!
!
!
Annotated, tagged
feedback within
video
Tagged feedback to
discuss performance
Audio or text
feedback regarding
performance
No discussion of
performance
!
!
!
!
!
!
Video conference
between student and
teacher to discuss
performance
Video response
delivered to the
Feedback is
delivered orally or
No conference is
held
!
!
student to discuss
performance
via text, conference
is not an option
!
!
!
!
!
!
!
!
!
!
!
!
Interactivity(
Interaction(&(
Interactivity((
Provides the ability
for synchronous and
asynchronous
discussion
platforms;
scheduling is
flexible
Provides the ability
for asynchronous
discussion platforms
among peers;
scheduling is
flexible
Provides the ability
for asynchronous
discussion platforms
among peers;
limited to scheduled
times for viewing
Does not provide
the ability for
discussion
platforms among
peers.
!
!
!
and posting content
!
!

LMS EVALUATION RUBRIC

!

8!

Interaction Provides an effective opportunity to actively engage and interact with Provides a limited opportunity
Interaction
Provides an
effective
opportunity to
actively engage and
interact with
Provides a limited
opportunity to
actively engage in
two-way feedback
and dialogue
between the
instructor and the
student
Provides a restricted
opportunity for
instructor to provide
feedback to student
regarding learning
materials
No opportunity to
actively engage
and interact with
the instructor
regarding learning
materials
!
!
Between
Learners and
Learning
Materials
instructor regarding
!
learning materials
between the
instructor and
multiple-students in
a single dialogue
!
!
!
!
Learning as a
Social Activity
Effective
opportunity to send
Limited opportunity
to send and receive
!
and receive peer to
peer feedback
No opportunity to
provide peer
feedback
!
!
peer feedback
synchronously and
asynchronously
synchronously and
asynchronously
Restricted
opportunity to send
and receive peer
feedback
asynchronously
!
!
!
!
Opportunity to
receive ongoing
descriptive
instructor feedback
on content produced
by students
Limited opportunity
two-way (instructor
to student; student
to instructor) to
receive instructor
feedback on content
produced by
Restricted one-way
(instructor to
student) opportunity
to receive instructor
feedback on content
produced by
students
No opportunity to
receive instructor
feedback on
content produced
by students
!
!
!
!
students
!
!
Quality of
!
!
Interaction
Ability to interact
through multimedia
Limited ability to
interact with peers
!
platforms; such as
and instructor
Restricted ability to
interact with peers
and instructor
through text-based
discussion platforms
No ability to
interact with peers
or instructor
video, text, and
audio discussion
platforms
through use of text
and audio
discussion platforms
!
!
!
(
!
!
!

LMS EVALUATION RUBRIC

!

9!

Organizational( ! University will ! ! Issues( provide detailed technical support University is able to
Organizational(
!
University will
!
!
Issues(
provide detailed
technical support
University is able to
provide technical
support
University is able to
provide limited
technical support
!
University unable
to provide
additional
technical support
University servers
capable of
supporting LMS
University servers
able to support LMS
University servers
somewhat able to
support a new LMS
University servers
unable to support
LMS and/or server
is hosted off-site
!
!
(
!
!
!
!
!
!
!
Novelty(
!
Provides an
effective learning
experience while
being appealing
modern to users
Provides a good
learning
environment with
some modern and
new tools
Provides a limited
learning
environment with a
mixture of outdated
and new tools
Provides a limited
learning
environment using
many outmoded
tools/widgets
!
!
New technologies
motivate the
learners experience
New technologies
enhance the learners
experience
Limited new
technologies which
do not enhance
No new
!
!
technologies are
included
!
!
learning
!
Has several
multimedia options !
Has some
Has limited
!
!
multimedia options
multimedia options
Does not have
multimedia
!
!
options !
!
!
!
!
!
!
!
!
Speed(
!
LMS model
adaptable to the fast
evolving medical
field !
LMS model
somewhat adaptable
to evolving medical
field
LMS model is
limited in capability
to evolve with
medical field
LMS model is
rigid and unable to
evolve with
medical field
!
!
!
!
LMS keeps up with
technological
innovation and
changes
LMS can keep up
with technological
changes but requires
major update
LMS has limited
options for updates
and/or upgrades
LMS comes with
rigid set of
tools/options;
cannot be updated
or upgraded
!
!
!
!
!
!
!
!
Overall(Score (
!
!

Running head: LMS EVALUATION RUBRIC

10#

References

Bates, A.W. & Poole, G. (2003). “A framework for selecting and using

technology.” Effective Teaching with Technology (p. 75- 108). San Francisco:

Jossey-Bass. Retrieved from

https://connect.ubc.ca/bbcswebdav/courses/SIS.UBC.ETEC.565A.65B.2013W2.2

5841/ETEC565A%20%28new%29/module01/Bates-Poole-Chapter4.pdf

Panettieri, J. (2007). Addition by subtraction. University Business, August, 58-62.

Retrieved from

http://www.universitybusiness.com/viewarticle.aspx?articleid=845

#