Evaluation of the use of Webassign in Chemistry Courses

Submitted to the Glenbard West Chemistry Department

Submitted by Arne Hendricksen
2
Summary
Webassign is an online system in which students log in to access and complete
homework assignments !eachers upload various assignments for students As
"uestions are completed# the students are given feedback on the correctness of their
answers Webassign keeps track of the number of correct and incorrect answers for
each assignment !eachers can control when the assignments are available and when
they are due# they can also control how many attempts a student has before a "uestion
is officially marked wrong $n addition# Webassign has a forum system in which
students can interact and discuss the homework problems Each assignment has
randomi%ed numbers so no two students have the same homework problem &or this
evaluation# Webassign was used in a high school chemistry course
Webassign has a number of goals in it use !he first goal is to increase
homework completion among students $t does so by allowing students to keep track of
their work in a central online system !he system can also send email reminders as to
when assignments are due !he second goal is to increase test scores among
students 'y increasing the amount of homework completed# Webassign increases a
student(s interaction with content and# potentially# improves test scores !he third goal
of Webassign is to increase the time available for teachers to plan and instruct students
'y grading homework automatically# Webassign frees up teacher time to plan for
lessons 'y eliminating the need to discuss homework in class or to pass it back to
students# webassign gives an instructor more time in class to interact with students and
teach
!he purpose of the evaluation is to determine if Webassign is meeting its
intended purposes in chemistry courses &irst# it will determine if homework completion
rates have increased since its use Secondly# it will determine if test scores have
increased since its use &inally# the evaluation will determine if teachers have gained
time in class and during plan periods since the use of webassign
!he evaluation took place in three ma)or phases !he first phase was data
collection During this phase the evaluator collected data to determine the average
chemistry test scores from *+,* and *+,- to determine if the use of Webassign causes
tests scores to improve !he ne.t part of the data collection phase re"uired the
evaluator to collect the homework completion rates for chemistry homework
assignments from *+,* and *+,- from each chemistry teacher to determine if
Webassign had caused an increase in homework completion Data was chosen from
*+,* because Webassign had not yet been implemented at Glenbard West *+,- was
the first year of its use# where the evaluator could actually collect data $n the ne.t
phase# the evaluator administered a survey to all chemistry students to determine if
students felt Webassign helped them complete homework and increase their test
3
scores &inally# the evaluator interviewed chemistry instructors to determine if the use of
Webassign had increased the amount of time available to teach and plan during the
school day
!he evaluation found that webassign has no positive impact on homework
completion rates/ in fact it may cause a decrease in year over year rates $n terms of
test scores# there was not a significant difference between *+,* and *+,- test scores/
therefore Webassign had no impact on chemistry test scores 0verall# students had a
fairly negative view of webassign &inally# teachers reported that Webassign did give
them more time in class# but had no impact on plan time
Description of Program Evaluated
Webassign is an online homework system used by Glenbard West High School
chemistry students and teachers to administer and answer homework "uestions
!eachers create assignments and answer keys and submit them to Webassign
Students log in and receive their assignments# which contain randomi%ed# uni"ue
numbers !hey answer the assignments and submit the answers for scoring# which
they receive immediately $f incorrect# they receive -12 more attempts to submit a
different answer After the attempts are used# the "uestion is marked incorrect !he
teacher can then access the scores online to use as they see fit Students can access
a forum hosted by Webassign to confer and work together on the homework "uestions#
but because the numbers are randomi%ed# they cannot cheat
Webassign is used throughout the entire year for the all regular chemistry
classes and all chemistry students Chemistry students are sophomores that have
completed algebra and biology !he program was first introduced in the fall of *+,- and
will continue its use into the spring of *+,2
Program Objectives
Webassign does not occur in a vacuum/ it was chosen by the chemistry instructors
at Glenbard West to complete a number of ob)ectives !hese are the ob)ectives that
were chosen to be assessed during the evaluation process
!o increase homework completion rates
!o increase chemistry test scores
!o increase teacher instruction time
!o increase teacher plan time
Program Components
!o complete Webassign assignments students must have access to a computer
and the internet !he high school has desktop computers that have internet access that
4
are available before# during and after school hours !o log in to Webassign# students
must have been registered by their instructor 0nce they log in to Webassign# they
must either purchase an access card from the high school or use a credit card to
purchase a code online# which costs 3,+4+ &rom there# they access an assignment
when it is posted by their teacher and they must complete it by the due date set Each
"uestion answered correctly in less than the allowed attempts is marked correct !he
teacher can then access student scores online and use them for grading purposes
Evaluation method
Participants
!he evaluator met with the chemistry teachers at Glenbard West to determine
the ob)ectives of Webassign !he meeting also established what the team hoped to
learn from the evaluation $t was determined that the results of the evaluation would be
reported to the chemistry team
!he participants for the evaluation were the chemistry teachers at Glenbard
West !hey were chosen because they used the Webassign system with their courses
!here are only four teachers who used the system# besides the evaluator !hey were
interviewed to determine how Webassign had impacted their instructional and plan time
Each chemistry student at Glenbard West also participated in the evaluation because
they use the Webassign system to complete their homework assignments All students
were chosen to take part in a survey to collect as much date as possible on their opinion
of Webassign $n order to eliminate a potential source of error or bias# all students were
used
Procedure:
During the fall of *+,- Webassign was used normally by chemistry students and
teachers to administer and complete homework !he evaluation process was carried
out while Webassign was in use during the semester
!he process began when surveys were given to each chemistry student at
Glenbard West High School to determine their use of Webassign and their feeling
towards Webassign !he survey was given in class using electronic devices or
computers !he results of these surveys were sent to the evaluator using a Google
Docs account
$n the second phase of the evaluation# the evaluator collected the homework
completion rates from the fall of *+,* chemistry classes at Glenbard west !his was
done by having instructors e.port e.cel files containing all homework grades for each
chemistry student from the fall of *+,* and having them send it to the evaluator !he
5
evaluator then calculated the ratio of completed assignments to total assignments for
each teacher and then overall !hen# the evaluator had the chemistry teachers send
out an e.cel file e.port of the *+,- homework scores to perform the same analysis to
find the homework completion rate for the fall *+,-
5e.t the evaluator had the chemistry teachers at Glenbard West create an e.cel
gradebook e.port of test scores from the fall of *+,* and the fall *+,- !he evaluator
calculated the average for each test during the fall based on each teacher6s data for
*+,* and *+,-
&inally# the evaluator created a list of "uestions in# order the interview the
chemistry team !hese "uestions pertained to the teams perceptions on how the use of
Webassign impacted their time to plan and time in class for instruction !he evaluator
met with each teacher to administer the "uestions and listen to and recorded the results
for each "uestion
Data Sources
!he first factor measured was the homework completion rate of chemistry
students in Glenbard West high school during the fall of *+,* and the fall of *+,- !his
was measured by using teacher gradebook data from each chemistry teacher !he
evaluator took the total number for homework assignments completed by all students
and divided it by the total number for homework assignments possible for all students to
determine the percent of homework assignments competed# or the homework
completion rate !his calculation was done for both the fall of *+,* when Webassign
was not used and the fall of *+,- when Webassign was used
!he second factor measured was chemistry test scores !o collect this
information# each chemistry teacher sent the evaluator an e.cel file of the test portion of
the teachers gradebook for both the fall of *+,* and the fall of *+,- &or each year the
evaluator calculated the average test score for each chemistry test for all students
taking chemistry !his was done for both the fall of *+,* and *+,-
!he third factor was student satisfaction with webassign !his was measured
using a google form survey that asked ,+ "uestions measuring student satisfaction with
the webassign process !he survey was given in class by each teacher using class
desktop computer or personal electronic devices !he "uestions were listed as a
7ichert scale where students were asked to answer in terms of aggreeance from
strongly agree to strongly disagree !he results were sent directly to the evaluator
using Google Docs !he results were collected using a Google &orm document A list
of the survey "uestions can be found in appendi. A
6
&inally# teacher(s perception of plan time and instructional time was measured
using an interview process !he evaluator sat down with each teacher to ask "uestion
pertaining to their perceptions of how Webassign has impacted their time to teach in
class and plan outside of class !he answers from each teacher were then recorded by
the evaluator A list of the "uestions asked in the survey can be found in appendi. '
Results
Test Scores
During the evaluation process# chemistry students took four tests for which the
chemistry teachers submitted tests scores to the evaluator !he teachers also
submitted student test scores from the previous year for the same four tests !here
were 2*4 students who took the four tests during *+,* !he evaluator calculated the
average of each test for *+,* and those averages were# 89:# 824:# 8-4: and ;+:
for tests ,12 respectively &or the -2< students who took the tests during *+,- when
Webassign was implemented# the evaluator found that the averages for the same tests
were 89:# <94:# 8--: and ;+: respectively
!est =nit ,
>atter
=nit *
Energy
=nit -
5uclear
=nit 2
Compounds
Average
score
*+,*
89: 824: 8-4: ;+:
Average
score
*+,-
89: <94: 8--: ;+:
ome!or" completion rate
!he evaluator collected e.cel files from each chemistry teacher from the fall of
*+,- and the fall of *+,- !hen# he divided the total number of completed homework
assignments# but the total number of assignments possible for all students !he result
is the homework completion rates for the fall of *+,* and the fall of *+,- $n *+,*#
chemistry students completed 8-2: of their homework assignments and in the fall of
*+,- when Webassign was implemented they completed 8*4: of homework
assignments
?ear
@ate of
Homework
Completion
*+,* 8-2:
*+,- 8*4:
Student survey results
7
A Google Docs survey was given in class to students using desktop computers
and personal electronic devices to determine students( feelings towards Webassign $n
total -2< responses# representing the entire body of chemistry students# were collected
!he survey determined that most# 89:# of students could log in to webassign $t was
found that ;*: of the students had purchased and entered their access codes to
webassign 0nly 2+: of the students felt that Webassign actually helped them learn
chemistry 89: of the students polled liked the fact that they knew when their
homework "uestions were right or wrong when they were submitted 0nly *-: of the
students surveyed used the homework forums to collaborate and ask for help when they
needed it >ost students# <*:# were able to log in to webassign without any technical
issues 4+: of the students preferred Webassign to bookwork and -;: preferred
bookwork to Webassign# while ,-: did not have an opinion 0nly ;: of the students
felt that the "uestions in webassign were irrelevant to what was being covered in class
<9: of the students polled used mobile devices such as phones or iAads to access
their Webassign homework &inally# <+: of the students polled knew how to e.tend the
due date on their Webassign to complete their homework after the due date
Buestion :
strongly
agree
:
agree
: no
opinion
:
disagree
:
strongly
disagree
$ can log into webassign 4< *- 2 * ,4
$ have purchased and entered a
webassign code
<9 ,- * + ,4
Webassign helps me learn
chemistry
,9 *, -, ,+ ,9
$ like knowing if $ get homework
"uestions right or wrong right
away
42 *4 ,* < 2
$ use the homework forums to
get help on "uestions $ don(t
understand
,- ,+ *- *9 *4
$ am able to login in to
webassign without technical
issues
-- *9 ,- ,- ,*
$ prefer webassign to bookwork -4 ,4 ,- ,* *4
!he "uestions in webassign are
relevant to what we are learning
in class
-4 -- *4 + ;
$ use my phone or other mobile
device to access webassign
-; -, 2 ,+ ,8
$ know how to re"uest an
e.tension on webassign
-- *8 ,8 ,* ,*
Teacher #ntervie! Results
8
Each chemistry teacher was asked a series of si. "uestions by the evaluator and
was given time to provide an oral response# which the evaluator recorded !he first
"uestion asked if the use of webassign increased the teaching time available during
class All four teachers responded yes to this "uestion !hey cited that they did not
need to review answers or hand back homework# which added a few e.tra minutes a
week to instruction time
!he second "uestion asked if Webassign increased the amount of prep time they
had !hree of the four teachers responded yes to this "uestion as well !hey cited the
fact that they did not have to grade the homework# so they had e.tra time to plan other
activities !he fourth instructor who said her plan time was not increased cited that she
was responsible for posting the entire chemistry team6s homework assignments# so that
task took some of her plan time She did mention that the time for Webassign was
comparable to grading time for non1 Webassign homework
!he third "uestion asked if the use of Webassign increased the overall amount of
work they do in a day All four teachers responded no to this "uestion !hree of which
said that the use of Webassign actually decreased their workload during the day
!he fourth "uestion asked how they think Webassign will impact the amount of
work they do in the future All four teachers responded that the use of Webassign will
decrease the amount of work they have to do in the future Each teacher cited the fact
that as they become more familiar with the system and as they implement more
assignments/ they will have to do less prep work in the future
!he fifth "uestion asked teachers if they felt competent in their use of Webassign
as an instructor 0nly one teacher responded yes !his was the same instructor that
was responsible for posting the assignments for the group !his instructor had also had
four years of e.perience using Webassign in another course !he other three teachers
stated that they needed more time to interact with Webassign and training as well
!he final "uestion asked if the teachers thought they would benefit from a
Webassign training session All four teachers responded yes !hey mentioned that
they did not yet feel completely comfortable using the system !hey also said that they
did not know how to use some of the key features of the system
Discussion:
!he purpose of the evaluation process was to determine the impact that Webassign
was having on chemistry courses at Glenbard West High School !he purpose of the
evaluation was to assess the effectiveness so the program in completing its ob)ectives
which wereC
9
!o increase chemistry test scores
!o increase homework completion rates
!o increase teacher instruction time
!o increase teacher plan time
!he completion of these ob)ectives by the implementation of Webassign cannot be
confirmed at this point $n fact# the results show that homework completion rates went
down slightly after the implementation of the Webassign program During the same
time chemistry test scores were seen to remain constant Anecdotal teacher interviews
have shown that the use of Webassign has created a perception of more teaching time
and in most cases# an increase in planning time &inally# a student survey revealed
mi.ed student opinions of webassign as a tool to improve learning and homework
completion
!he results of collecting and averaging student tests scores from the fall of *+,* and
comparing them to the average scores from the fall of *+,- show that the use of
Webassign has not led to an increase in student test scores $n *+,* the students
averaged 89:# 824:# 8-4: and ;+: respectively on their first four tests $n *+,- on
the same four tests the averaged 89:# <94:# 8--: and ;+: respectively &or each
test the average either remained the same or went down from *+,* to *+,- !his
indicates that use of Webassign cannot be linked to an increase in student test scores
Homework completion rates for chemistry students in the fall of *+,* were 8-2:
when Webassign was not yet introduced !he fall of *+,- saw homework completion
rates drop to 8*4: !his dip in homework rates means that Webassign cannot be
attributed to causing an increase in homework completion rates for students
Anecdotal evidence through teacher interview indicates that the use of
Webassign gives chemistry teachers the perception that they have more time in class
for instruction and also# more time for planning $t should be noted that this was not a
unanimous perception for the increase of plan time 0ne of the four teachers
interviewed cited that her plan was unaffected by the use of Webassign !here is no
"uantitative data that suggests teachers actually have more time during class or during
their plan period
While the student survey does not specifically set out to prove any ob)ective of
the evaluation# it does help to reveal student attitudes# which may in turn provide an
insight into why the ob)ectives have been mostly unmet 0ne "uestion of the survey
found that ,8: of students were unable to log in to or had chosen not to log into
Webassign at the time of the survey Another "uestion indicates that *4: of the
students had technical difficulties when trying to log into Webassign &inally# -8: of
students prefer the older method# using a te.tbook and paper# to answer their
10
homework "uestions to the new online Webassign system !he results of these
"uestions did not reveal the completion or failure to meet ob)ectives# but rather they give
strong insights to how and why ob)ectives are or are not met
!he evaluation has shown that teachers perceive that they have more time to
teach and more time to plan activities due to the use of Webassign !his is a definite
positive result but it cannot be truly verified until data is collected to reveal how many
minutes are saved each day due to the use of Webassign !he student survey revealed
that students( inability or lack of interest to use technology may be tied to the lack of
improvement in homework completion in Webassign ,8: of students could not# or
chose not# to login in to Webassign and another *4: claimed that they had technical
difficulties at time or another !he combination of these two facts may have had an
impact on the number of students that were able to complete their homework# which in
turn would bring down homework rates
!he evaluation has determined that the use of Webassign has not allowed
chemistry teachers to reach their goals of improved homework completion and test
scores $t has# although# led to a perception# and# maybe a reality# of more plan and
teaching time $nterviews have indicated that some teachers are committing more time
to Webassign than others !he lack of improvement in homework completion may be
tied to a number of factors that could be improved upon in the future !he first is
student access to computers and computer skills !he student survey revealed that
,8: of students never logged into Webassign and another *4: had technical issues $f
these numbers could be decreased# than it is possible that homework completion rates
could rise above the 8-2: rate from *+,* 'y increasing the number of students
attempting their homework# it is also possible that students could perform better on
future tests
$t is recommended that Webassign is retained as the method of homework for
chemistry students for at least one more entire school year !his is recommended
because there are a number of improvements that the chemistry could make to improve
the functionality of Webassign and in turn# help it to reach its ob)ectives as a program
&rist# the chemistry team would benefit from training in Webassign !his could be done
by completing a tutorial series found on the Webassign website or by hiring a
Webassign consultant !his would give each teacher the skills to work "uickly and
efficiently with Webassign in order to save time and also# remove some burden from the
instructor who was committing more time and effort to the program
$t is also recommend that the chemistry team create a day of classroom
instruction to ensure that each student has created a Webassign account and knows
how to login to it !his would potentially remove the hesitance to )oin Webassign and
11
alleviate any technical issues students had with logging in Doing so would potentially
increase homework completion rates and eventually# test scores for chemistry
Project cost
!he pro)ect cost incorporates the time the evaluator worked on the evaluation#
which was the e"uivalent of ,4 total ; hour work days# at the rate of 34++ per day !he
driving consisted of *+ miles per day at the standard rate of 3+4<4 per mile !he
materials used for this report were the print and production costs for 4 total reports at a
cost of 3-++++
$tem Amount Cost !otal cost
Aer diem ,4 34++++ 384++++
mileage -++ 3+4<4 3,<94+
Arinting
and
production
, 3-++++ 3-++++
!otal 389<94+
$ppendi% $:
Student Survey
12
$ can log into webassign D, E strongly agree *E agree -E no opinion 2E disagree 4E
strongly disagree
,
*
-
2
4
$ have purchased and entered a webassign code D, E strongly agree *E agree -E no
opinion 2E disagree 4E strongly disagree
,
*
-
2
4
Webassign helps me learn chemistry D, E strongly agree *E agree -E no opinion 2E
disagree 4E strongly disagree
,
*
-
2
4
13
$ like knowing if $ get homework problems right or wrong right away D, E strongly agree
*E agree -E no opinion 2E disagree 4E strongly disagree
,
*
-
2
4
$ use the homework forums to get help on "uestions $ don6t understand D, E strongly
agree *E agree -E no opinion 2E disagree 4E strongly disagree
,
*
-
2
4
$ am able to login to webassign without technical issues D, E strongly agree *E agree
-E no opinion 2E disagree 4E strongly disagree
,
*
-
2
4
$ prefer webassign to bookwork D, E strongly agree *E agree -E no opinion 2E disagree
4E strongly disagree
,
*
-
14
2
4
!he "uestions in webassign are relevant to what we are learning in class D, E
strongly agree *E agree -E no opinion 2E disagree 4E strongly disagree
,
*
-
2
4
$ use my phone or other mobile device to access webassign D, E strongly agree *E
agree -E no opinion 2E disagree 4E strongly disagree
,
*
-
2
4
$ know how to re"uest an e.tension on webassign D, E strongly agree *E agree -E no
opinion 2E disagree 4E strongly disagree
,
*
-
2
4
15
$ppendi% &
Teacher #ntervie! 'uestions
,F Has webassign increased the time you have to teach in classG
*F Has webassign impacted the time you have available to plan during the dayG
-F $n terms of workload# has webassign increased the amount of work you do
during the dayG
2F How do you think webassign will impact the amount of work you do in the
futureG
4F Do you think you are competent in the use of webassign as an instructorG
<F Could you benefit form a training session in how to teach with webassignG