Learning and Performance Quarterly, 1(4), 2013 24

Web-based Learning Management System Considerations for Higher Education

Chih-Hung Chung, University of North Texas
Laura A. Pasquini, University of North Texas
Chang E. Koh, University of North Texas






Abstract

Coinciding with the development and growth of the Internet, there has been a dramatic
increase in the application of the Learning Management Systems (LMS) in higher
education. University and college campuses should consider evaluating each LMS to
ensure that the system meets the requirements and demands of the institution. Therefore,
the purpose of this study is to present a model which incorporates the concepts and
findings from research on LMS application in higher education. The alternative model
was modified based on the Technology Acceptance Model (TAM). In addition, five
categories of LMS features for higher education are discussed including: (1) transmitting
course content; (2) evaluating students; (3) evaluating courses and instructors; (4)
creating class discussions; and (5) creating computer-based instruction. This study
reviews and discusses prior research and provides several recommendations including a
model of development and design of an LMS for future implementation in higher
educational environments.

Keywords: Learning Management System, Technology Acceptance Model, Human Effect,
Human Computer Interface.
Learning and Performance Quarterly, 1(4), 2013 25

Web-based Learning Management System Considerations for Higher Education

Introduction
With the development of learning technologies, learning management systems (LMS)
have become an important component in the education field. Most universities and colleges in
the United States (U.S.) have adopted an LMS to support instructor teaching activities and
student learning processes. One of the most important features of an LMS is to provide an
environment for learning and teaching without the restrictions of time or distance (Epping,
2010). Since the increased development of the Internet, the LMS concept has been broadly
applied at various higher education institutions around the world. LMSs helps instructors and
learners discuss the course content by posting and responding to each other, maintaining student
learning tracks, and managing learning activities in an online environment (Falvo & J ohnson,
2007). Currently, LMSs improve instructor teaching and student performance across various
fields of study (Boggs, Shore, & Shore, 2004). Since Blackboard Corporation introduced
Blackboard in 1997 (Falvo & J ohnson, 2007); over 60 countries have adopted the Blackboard
LMS. As technology continues to develop, Blackboard faces increasing competition from LMS
rivals, such as Desire2learn and Moodle. These learning management systems present
convenient functions and innovative course management approaches. For instance, Moodle, an
open-source course management system, offers a flexible management environment for users,
and it can quickly add or modify available extension models (Kumar, Gankotiya, & Dutta, 2011).
In addition, Unal and Unal (2011) suggested that Moodle would be an effective alternative LMS
to Blackboard and other LMSs because of its flexibility and open-source resource. These
instructor functions and course management approaches urge users to choose a flexible LMS
system.
As a result of increased competition and advanced technology, higher education
institutions now have a wide variety of options to manage their learning curriculum. Each LMS
offers specific functions and management approaches, so choosing the appropriate system
becomes an important concern for educational institutions. In addition, an LMS does not offer
enough finalized functions to satisfy the demands of the institutions. As a result, institutions must
spend valuable time and effort comparing each LMS system individually to ensure that the one
chosen meets their demands. Although LMSs have become increasingly popular, several
drawbacks and limitations exist. There is a lack of social interaction. Social interaction
encourages high learner motivation, which has the potential to improve users teaching and
learning performance (Chou & Chou, 2011). In addition, Huffman and Huffman (2012)
suggested that students utilizing the appropriate technological tool improved learning
performance. This paper addresses the question, What types of designs should LMSs utilize for
higher education?
The purpose of this manuscript is to provide an alternative model of a LMS design and
development by reviewing important findings from research on LMS in higher education
learning environments. In addition, the authors reviewed five categories of LMS features for
higher education to help contribute to the design and structure of effective and efficient learning
management systems. After discussing the relevant research, this study offers a model for LMS
design for higher education institutions based on a variation of Davis (1989) Technology
Acceptance Model (TAM). Due to the widespread use of Blackboard as an LMS in higher
education, this system will be utilized as a baseline to discuss design applications required for
higher education learning curriculum.
Learning and Performance Quarterly, 1(4), 2013 26

Theoretical Framework
The theory, used as a framework, in this study was the Technology Acceptance Model
(TAM) developed by Davis (1989). It was adopted to explain the acceptance and use of
technology in LMSs by end users. This theory posited that the perceived usefulness and ease of
use would best determine an individual's intention to utilize a particular technology. In addition,
five categories of LMS features for higher education, proposed by Malikowski, Thompson and
Theis (2007), are discussed as the major features in LMSs. As applied to this study, the TAM
theory and research model guided the literature review and the discussion of the findings as they
pertain to the development and design of LMSs.

Literature Review
This section overviews and categorizes the literature related to learning management
system (LMS) into several parts, including the definition of LMS, the Human-Computer
Interaction (HCI), the human effect on the online environment and five categories of LMS
features with HCI, the Technology Acceptance Model (TAM), and cross-cultural issues in the
online environment. Each topic will provide a modified framework of LMS design and
development.

Definition of LMS
Learning Management Systems (LMSs) have been widely used in higher education due
to their many advantages including flexible learning times and unlimited distance education
(Hamuy & Galaz, 2009). Research on the teaching, retention, and learner usability for learning
management systems has been mounting steadily for a number of years (Coates, J ames &
Baldwin, 2005; Ozkan, Koseler & Baykal, 2009; de Porto Alegre Muniz et al, 2012; Alhazmi &
Rahman, 2012). Various definitions of learning management systems have been proposed over
the course of years of research; however, Simonson (2007) provided a practical definition of
LMS:
Course management systems, also called learning management systems or virtual
learning environments, are software systems designed to assist in the management of
educational courses for students, especially by helping teachers and learners with course
administration. The systems can often track the learners progress. While usually thought
of as primarily tools for distance education, they are also used to support the face-to-face
classroom (p. vii).

Definition of Learner Performance
Tsao et al. (2007) define learner performance as the knowledge and skills learned,
partnered with the learning attitudes they display. Learners in higher education are the user of the
learning management systems and should be considered during evaluation of the platforms.
Further, learner performance is articulated as the knowledge and skills obtained during lessons
with the resulting attitude and behavior displayed by the learner Lee (2002). This would result in
the learning success and meet the learning objectives set by the course curriculum.

Definition of Culture
Culture can be defined in many different ways; however specifically, for the online
environment, Hofstede (1998) defined culture as, [T]he collective programming of the human
mind that distinguishes the members of one human group from those of another. Culture in this
Learning and Performance Quarterly, 1(4), 2013 27

sense is a system of collectively held values (p. 478). The authors adopted Hofstedes (1998)
definition of culture to study the impact of culture on LMS design.

Overview of Learning Management Systems (LMSs)
An LMS is a particular type of software system designed and promoted for instructors
and learners to utilize in teaching and learning activities. Typically LMSs offer various tools,
such as course content organization and presentation, communication tools, student assessment
tools, grade book tools, and other functions supporting classroom learning and teaching
performance (Morgan, 2003). The 2009 report from the American Society for Training and
Development (ASTD) revealed that ninety-one percent of ASTD respondents in their
organizations utilized LMSs, and it also identified the challenges of using an LMS.
Customization requirements and content integration need to be overcome in the learning
management system (Ellis, 2009). To improve LMS implementations, organizations need to
consider the virtual learning environments to improve learning outcomes, ensure accessibility,
and engage learners in user-friendly virtual learning environments (ON24 Inc., 2012). Since the
field of Information Technology (IT) is constantly growing, various LMSs and technologies to
support learning management systems have been developed. The challenge included evaluations
of benefits and functional factors for effective consideration for web-based learning management
systems that will be effective for both the instructor and learner in higher education.
In order to offer effective LMS evaluation methods, many researchers have proposed
several methods to compare differences between each LMS. Lins (2010) analytic hierarchy
process proposed several key issues for evaluating the LMS, such as system quality, information
quality, reliability, and attractiveness. With regards to key HCI issues, Oztekin, Kong, and Uysal
(2010) utilized a criticality metric analysis to evaluate a learning management system and
presented the Uselearn assessment model to explain the relationship between each factor.
Usability emerged as a major component to evaluate the quality of an LMS; in addition, HCI has
been proposed that it was useful for improving the usability of computer systems (Schmidt,
1997). For these reasons, HCI plays an important role in developing and designing LMSs.

Cross-cultural Factors in the Online Environment
In recent years, cross-cultural learning research has received considerable attention. Lee
and Croker (2006) explored the related research as it pertains to the cross-cultural environment.
Their study evaluated the influences of expatriate characteristics, the complexities of task
assignments, and observed cross-cultural differences and they proposed that the training of
expatriates was one of the most vital elements for successful business globalization (Lee &
Croker, 2006). In contrast, little research has been done on cross-cultural issues in the online
environment, which is quite relevant since these issues also contribute to enhancement of the
students learning performance (Cameron & Limberger, 2004). A growing number of Asian
students have enrolled in online courses at universities across the U.S. (Ku & Lohr, 2003) and
management of the online cultural issues should be considered. Ku and Lohrs (2003) study
addressed important claims regarding whether the Chinese culture influences Chinese students
learning behaviors when they enrolled in online courses in the U.S., particularly during their first
year. Ku and Lohrs (2003) proposed that Chinese students believe online courses made them
feel isolated from other students and rarely allowed any interaction with classmates; however,
American students did not feel isolated. To address this concern, researchers proposed
suggestions for designing online courses for international students as follows: (1) Increase the
Learning and Performance Quarterly, 1(4), 2013 28

self-confidence and motivation of students early on; (2) offer the opportunity for students to
work in small groups to increase their interaction; (3) maintain a self-paced and self-directed
design of the online learning environment; (4) provide various opportunities for reading and
writing; and (5) encourage student meetings with group members and instructors (Ku & Lohr,
2003). Yang, Olesova and Richardson (2010) systemically reviewed the impact of culture and
social interaction within the online environment. The results of their study showed that the
diversity of culture had a significant influence on HCI in the web design. Therefore, cross-
cultural issues are an important factor for LMS design and development.

The Human Effect on the Online Environment
According to a study by Southwell, Anghelcera, Himelboima, and J ones (2007), a variety
of human factors, such as user control and experience, influenced user-learning patterns as they
used the LMS. Hence, the LMS should consider the human effect to enhance instructor and
learner experiences and satisfaction (Chrysostomou, Chen, & Liu, 2009; Kim, Kwon, & Cho,
2011). A variety of research adopted assumption-driven, statistical techniques to analyze the data
in order to present useful information in regards to the human effect in the LMS; however,
Chrysostomou, Chen, and Liu (2009) found the scope of the results was restricted due to
statistical hypotheses and they adopted data mining analysis techniques to investigate the
influential factors as consumers used the LMS. The results of this research proposed four
clusters: (1) users like a single window layout with a static button, no use of icons and menus,
and colors with the effect's scheme; (2) users like to use multiple window layouts with dynamic
buttons, use of icons and menus, and standard color scheme format; (3) users like to utilize a
single window layout with a static button, use of icons, but no menus, and multiple color
schemes; and (4) users like to utilize various windows with dynamic buttons, no use of icons and
menus, and colors with the effect's scheme (Chrysostomou, Chen, & Liu, 2009).
The influential factors resulting in the identified four clusters, suggested a particular color
scheme was most popular, which was also compatible with cognitive theory (Swelleer, Van
Merrienboer, & Paas, 1998). With various human effects, such as experience and habits, users
expressed different results in HCI. In addition, computer novices preferred to use particular
window layouts and a static button, whereas computer experts preferred to use multiple window
layouts and drop-box menus. According to research by Beckers and Schmidt (2003), computer
anxiety and the avoidance of using technology makes users feel a lack of control. Moreover, the
less time people are exposed to computers, the less they feel in control. A study by
Chrysostomou et al. (2009) revealed that different experiences using the computer influences
users teaching and learning performance. LMS developers should consider the impact of use
and users length of time in a web-based learning environment when designing features for
course content delivery.

Five Categories of Learning Management Systems Features
Malikowski, Thompson, and Theis (2007) proposed an LMS research model, which
includes the following five categories with HCI: (1) transmitting course content; (2) evaluating
students; (3) evaluating course and instructors; (4) creating class discussions; and (5) creating
computer-based instruction. According to their study, transmitting course content was the most
important function utilized by instructors. For instance, instructors utilized an LMS to announce
important events, such as midterm exams, articles for reading, course information, syllabi, and
assigned tasks. In contrast, certain LMS features were not utilized by instructors, specifically
Learning and Performance Quarterly, 1(4), 2013 29

those who were not familiar with the computer-based applications or available interaction
options with their learners (Malikowski et al., 2007). Utilizing more of the available LMS
features, such as the underused synchronous discussions, would allow instructors to achieve their
teaching goals and improve the learning performance of their students (Malikowski et al., 2007).
Additionally, Kibaru and Dickson-Deane (2010) and Niedlsen (2003) proposed a usable
online environment that could enhance web users performance. Abdalla (2007) adopted the
TAM framework to evaluate the effectiveness of Blackboard and suggested that the convenience
of technology enhanced the effectiveness of the LMS. In other words, instructors would be able
to reduce the time to manage and set-up the course to improve their teaching experience, as well
as the students learning experience (Inversini, Botturi, & Triacca, 2006). Nevertheless, previous
research suggested that the development of an LMS might suffer by failing to devote a sufficient
amount of attention to the overall design (Tsang, Kwan, & Fox, 2007). Integrating various
appealing options might result in too many LMS features which could potentially confuse
learners and stifle the LMS potential to help support the course objectives (Kidney & Puckett,
2003). Due to this fact, the development of an LMS should be systematically analyzed and
designed to promote effectiveness and efficiency of the users learning performance.

HCI in LMS
A decade of HCI research has offered beneficial information on how users perform and
think about the system. Research in this area provides important insight for technology usability
and consideration of the user for the design element of HCI (De Lera, Fernandez, & Valverde,
2010). According to the International Organization for Standardization, usability means that
users can effectively use a tool to finish a task with satisfaction (ISO 9241-11, 1998). The better
HCI that LMS offers users, the greater satisfaction users will have with their systems. Hollender,
Hofmann, Deneke, and Schmitz (2010) also discussed the significant influence of HCI on online
learning. Although usability can improve the learning experience for students, Tselios, Avouris,
and Komis (2008) pointed out that learning technology should not solely support the efficient
execution of a task. By simply improving usability, one might have a negative impact on the
learning experience in specic cases because implementing a task efficiently might preclude
crucial learning processes. Therefore, a balanced design of HCI is one of the key components in
the design and development of LMS.
In order to develop and design a higher quality LMS, the five HCI categories, (1)
transmitting course content; (2) evaluating students; (3) evaluating course and instructors; (4)
creating class discussions; and (5) creating computer-based instruction, should be considered. As
evident from the literature review, HCI can influence the effectiveness and ease of use. Paul
(1989) presented a simple definition of HCI, which is studying the interaction between
computers and humans. In addition to enhancing the usability of an LMS, HCI plays an
important role in attaining the goal of improving user performance (Sung & Mayer, 2012).
Moreover, each of these five categories aid in the understanding and improvement of quality
LMS design. Therefore, under the premise that HCI will positively impact the usability of LMS,
the authors will evaluate and develop five major categories of LMS characteristics with HCI as
functions of learning management systems. In our modified model of the LMS design, HCI
included five categories to help designers to develop an effective and efficient framework.



Learning and Performance Quarterly, 1(4), 2013 30

Technology Acceptance Model (TAM)
Davis (1989) proposed the Technology Acceptance Model (TAM) to predict the
phenomenon of a new technology adopted within a group. TAM emphasizes the acceptance and
use of technology as explained by a users internal beliefs, attitudes, and intentions (Turner,
Kitchenham, Brereton, Charters, & Budgen, 2010). Since TAM was proposed, various related
research explored the topic of technology acceptance and design. This research provides the
foundation of understanding and identifying a users technology acceptance and use of
technology for various disciplines. In the field of online learning, TAM has been applied to and
considered in a number of e-learning acceptance research (umak, Hericko, & Punik, 2011).
According to the study by umak et al., (2011), TAM can effectively explain the acceptance of
online learning. Consequently, this paper adopted TAM to help develop our LMS design
framework.
An Alternative Model of LMS Design and Development

We adopted two frameworks to address the design and development of the learning
management system in higher education.

Transmitting Course Content Category
The transmitting course content category includes three major features of an LMS: course
content, announcements, and the grade book. Although the first two features are valuable, the
grade book was unsuitable for complex grading or evaluation systems. Most LMSs do not offer
effective grade books for instructors. Instructors adopt other techniques and tools to process
more complex evaluations (Malikowski et al., 2007). This category was not as effective as the
other categories in the LMS; therefore, LMS designers should consider improving these features
and providing a more suitable design when building a new LMS. In other words, it would be
beneficial to determine which educational theories should be applied by considering this
category first. Although most of the LMSs offered features designed to support presentations and
provide guidance, in practice, the components were inadequate (Gilbert & Moore, 1998; Kidney
& Puckett, 2003; Ozkan, Koseler & Baykal, 2009; de Porto Alegre Muniz et al, 2012; Alhazmi &
Rahman, 2012).
In a study by Kumar, Gankotiya, & Dutta (2011), a comparison between Blackboard and
Moodle was discussed in detail. This research classified fifty-two comprehensive features into
six major categories to compare the two LMSs. Through the development of technology,
Blackboard and Moodle present social and interactive features for users, such as personal blogs
and Wimba classrooms (synchronous online teaching platforms). For example, Moodle
incorporated social media tools (e.g., blogs) to support the diversity of the teaching and learning
environment and allows students to upload their files on their blog to share with other users.
Moodle also permits the instructor to share and copy the entire course with other users, or export
the course shared by other instructors within the LMS. These functions can aid instructors in
maintaining and developing courses.
Blackboard provides options to include social media applications and other
personalization tools. This personalization feature allows students to create their personal home
page within Blackboard. Although the two LMSs have applied social media tools, these features
are not completely integrated. Recently, researchers have proposed several studies regarding
social media education, specifically utilizing social media to improve students learning
performance (Gerlich, Browning, & Westermann, 2010). Sing and Mayer (2012) also concluded
Learning and Performance Quarterly, 1(4), 2013 31

that the navigational and signaling aids in the LMS had a positive correlation between usability
and recall tests. Therefore, this category was adopted in our model of the LMS development and
design.

Creating a Discussion Category
In creating a discussion category, asynchronous discussions had higher approval ratings
than synchronous discussions because participants have more time to read and respond to
questions. In contrast, synchronous discussions in the LMSs were difficult for users since the
messages could be quickly scrolled away from the users view, and many users needed more
time to focus on reading and responding to the quick discussions. In spite of this difficulty,
synchronous discussions have certain benefits that could help participants feel a stronger sense of
social interaction. Therefore, designing a feature that creates longer discussion formats should be
considered as a viable option to provide users the opportunity to participate in discussions. A
better functioning synchronous discussion feature design should decrease the difficulties in
participating in synchronous discussions. Comparing Moodle and Blackboard, Kumar et al.
(2011) asserted that both of LMSs offered asynchronous and synchronous discussion functions;
however, social media tools offer a better interacting approach for users (Dabbagh & Kitsanta,
2012). Traditional interaction in LMSs are inadequate for current users and the more interactive
features of LMS need to include more varied engagement approaches.

Evaluating Students Category
The most commonly used LMS tool in the evaluation and assessment category is the test
generator, which helps instructors create course assessments and allows students to submit the
test via the drop box function. The test generator allows instructors to create various types of
assessments, such as multiple choice, multi-select, matching, ordering, arithmetic tests, long and
short answers, fill in the blank, and true/false. In addition, the drop box function is easy to use for
the student and easy to access for the instructor. A good test generator should include a flexible
approach to create various evaluation methods; however, both variations of the LMS, Blackboard
and Moodle, are limited in their assessment design.
Fritz (2011) also emphasized the importance of interaction in the evaluation process,
which would allow the learner to check and monitor their academic activities and improve
student-learning performance. In contrast, simply delivering a grade without the ability to
interact with the student would fail to enhance student participation, as well as their learning
experience.

Evaluating Courses and Instructors Category
In the evaluating course and instructor category, the LMS allows students to evaluate
both the course and the instructor at the end of the semester. Course evaluation serves a very
important function in education because it helps instructors understand if they need to modify the
course content, as well as their teaching approaches to improve the overall learning and teaching
experience (Sims, Dobbs & Hand, 2002). Although this feature provides more information to
institutions, it has seldom been utilized to evaluate courses by universities because most have
their own evaluating systems (Malikowski, Thompson, & Theis, 2007). This feature of an LMS
might not be broadly applied since colleges and universities have established evaluating systems
to collect survey data. For example, the University of North Texas uses the SETE and
Blackboard learning management system separately, thereby increasing maintenance costs.
Learning and Performance Quarterly, 1(4), 2013 32

Developing and designing a new system needs to take this into consideration and offer
alternatives which universities might be quicker to adopt. The implementation of an LMS,
including a completed survey system, would enable higher education institutions to save money
in both maintenance and development fees.

Creating Computer-Based Instruction
The use of computer-based instruction (CBI) is a feature that has been designed to help
instructors transmit content for years. With the developed technology, CBI could make various
options and features available in an LMS for instructors to choose from (Malikowski, Thompson,
& Theis, 2007). Today, several technique tools could effectively and efficiently help instructors
to create teaching content, e.g., Prezi, SlideRocket, Flickr, YouTube, Sliceshare, and Knovio.
These social media tools could enhance various teaching presentations and course curriculum.
Therefore, the LMS should be flexible and offer an adaptable and creative in its CBI to improve
the online learning experience.
Considering the literature review and consideration for the online learning frameworks,
Figure 1 demonstrates a cohesive model of an LMS design and development. The HCI should
consider two major factors, cross-cultural impact and the human effect, in designing an effective
learning management system. Moreover, the designs of HCI, with regards to the five categories,
will influence the perceived ease of use and usefulness in an LMS. Therefore, we propose a new
model for LMS development and design.


Figure 1. The modified model of development and design of LMS
Learning and Performance Quarterly, 1(4), 2013 33

Discussion
We proposed how to consider cross-cultural impacts, the human effect, and HCI in the
development for an LMS design. In this section, the characteristics of the alternative model and
the implications, the interaction between HCI, the cross-cultural impact, and human effect, and
limitations of the alternative model will be discussed.

Characteristics of the Alternative Model
The online environment provides non-linearity and hypermedia forms for the LMS users.
Individuals can have access to information and knowledge without many restrictions due to the
flexibility of online learning. Through interaction in the LMS, learners would be able to quickly
share and gain information from other users. However, an unsuitable design of an LMS feature
could result in negative impact on learning performance. For instance, learners might spend more
time searching the information rather than sharing ideas or course material. In contrast, a suitable
LMS design helps users minimize the barriers of utilizing the LMS to improve the learning
experience. Improving the HCI, including five categories, helps the users perceived usefulness
and ease of use would improve the intention to use the LMS (Venkatesh, Thong, & Xu, 2012).
Regarding the cross-cultural impact, the LMS design becomes more complex because the cross-
cultural impact influences how individuals adapt to use a LMS. Moreover, human effect is also
one of important factors. This is because peoples control and experience for using computer
systems would influence the usability of LMS (Ku & Lohr, 2003). The well-designed HCI
improve users perceived usefulness and perceived ease of use. Furthermore, users intention to
use LMS presents positive results, and LMS improves students learning performance.

The Interaction between HCI, the Cross-cultural Impact, and the Human Effect
The online environment helps learners to exchange information at any time; however, the
interaction between HCI, the cross-cultural impact, and human effect is complicates this learning
exchange. With the cross-cultural impact and the human effect, learners have different attitudes
toward using certain features of LMSs. For example, Chinese students (collectivism culture)
reported they felt isolated from others when using the discussion board. In contrast, American
students (individualistic culture) did not feel isolated (Ku & Lohr, 2003). In addition, the human
effect factor affects users perception of usefulness (Womble, 2007). The designer should be able
to improve uses perceived usefulness by providing appropriate system design features (David,
1993).

Implications and Limitations of the Alternative Model
This LMS design and development study attempted to explain the human effect and the
cross-cultural impact that can be emphasized in an LMS design. Moreover, the alternative model
indicated the complexities of the interaction between these factors and the influence on perceived
usefulness, perceived ease of use, and intention to use LMSs. For future studies, this alternative
model provides an LMS design and development along with the cultural impact and human
effect. Future research should explore how to design suitable LMSs to increase the learning
performance in the LMS. From a practical aspect, this model helps instructional designers,
instructors, and educational developers understand the cross-cultural impact and the human
effect process of the LMS to consider the impact learning environments have on users.


Learning and Performance Quarterly, 1(4), 2013 34

Conclusions
The development and design framework of a learning management system is an important
foundation of learning performance and the instructional experience in higher education. The
result of this paper clearly identifies the human effect and cross-cultural factors influence on
LMS development and design. Based on the TAM theory and the course management model
framework provided by Malikowski, Thompson, and Theis (2007), the LMS development and
design needs to consider the described five categories and two factors. In this review of LMSs
for higher education institutions, much remains to be done, but we anticipate that the study will
generate crucial findings in the field of LMS research.
Future studies would be encouraged to test and improve upon this LMS alternative model
design. We intend to continue pursuing this line of investigation in a series of studies and
investigate how instructors and learners successfully utilize more LMS tools to enhance their
performance. By testing the LMS framework design, we will be able to generate more specic
recommendations about LMS design. With the development of LMS research, these studies can
have the potential to aid LMS developers and instructional designers who support instructors and
learners, and consider how to effectively use technologies to support effective learning
objectives in course design.




















Learning and Performance Quarterly, 1(4), 2013 35

References
Abdalla, I. (2007). Evaluating effectiveness of e-blackboard system using TAM framework: A
structural analysis approach. AACE Journal, 15(3), 279-287.
Alhazmi, A. K., & Rahman, A. A. (2012). Why LMS failed to support student learning in higher
education institutions, E-Learning, E-Management and E-Services (IS3e), 2012 IEEE
Symposium,1-5.
doi: 10.1109/IS3e.2012.6414943
Beckers, J . J ., & Schmidt, H. G. (2003). Computer experience and computer anxiety. Computers
in Human Behavior, 19, 785797. doi:10.1016/S0747-5632(03)00005-0.
Boggs, S., Shore, M., & Shore, J . A. (2004). Using e-learning platforms for mastery learning in
developmental mathematics courses. Mathematics and Computer Education, 38(2), 213-
220.
Cameron, H., & Limberger, J . (2004). Online learning: Cross-culture development in time poor
environments. Journal of European Industrial Training, 28, 429-439. Retrieved from
ABI/INFORM Global. (Document ID: 646658641).
Chou, A., & Chou, D. (2011). Course management systems and blended learning: An innovative
learning approach. Decision Sciences Journal of Innovative, 9, 463484.
doi:10.1111/j.1540-4609.2011.00325.x
Chrysostomou, K., Chen, S. Y., & Liu, X. (2009). Investigation of users preferences in
interactive multimedia learning systems: a data mining approach. Interactive Learning
Environments, 17(2), 151-163.
Dabbagh, N., & Kitsanta, A. (2012). Personal learning environments, social media, and self-
regulated learning: A natural formula for connecting formal and informal learning.
Internet and Higher Education, 15(1), 3-8. doi:10.1016/j.iheduc.2011.06.00
Davis, F. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information
technology, MIS Quarterly, 13(3), 319340.
Davis, F. (1993). User acceptance of information technology: system characteristics, user
perceptions and behavioral impacts. International Journal of Man-Machine Studies, 38(3),
475487. Retrieved from http://deepblue.lib.umich.edu/handle/2027.42/30954
De Lera, E., Fernandez, C., & Valverde, L. (2010). The emotional gap in virtual online
environments. In Z. Abas et al. (Eds.), Proceedings of Global Learn Asia Pacific 2010
67-70. AACE.
de Porto Alegre Muniz, M., de Moraes, A., J acobs, K., & Soares, M.M. (2012). Usability issues
in Learning Management Systems (LMS). Work, 41, 832-837.
Ellis, R. K. (2009). Learning management systems 2009. Retrieved from
http://www.astd.org/LC/2009/0509_LMS2009.htm
Epping, R. J . (2010). Innovative use of Blackboardto assess laboratory skills. Journal of
Learning Design. 3(3). 32-36.
Falvo, D. A., & Ben F., & J ohnson, B. F. (2007). The use of learning management systems in the
United States. TechTrends. 51(2). 40-45.
Fritz, J . (2011). Classroom walls that talk: Using online course activity data of successful
students to raise self-awareness of underperforming peers. Internet and Higher
Education, 14, 8997. doi:10.1016/j.iheduc.2010.07.007
Gerlich, R. N., Browning, L., & Westermann, L. (2010). The social media affinity scale:
Implications for education. Contemporary Issues in Education Research, 3(11), 35-41.
Learning and Performance Quarterly, 1(4), 2013 36

Gilbert, L., & Moore, D. (1998). Building interactivity in web courses: Tool for social and
instructional interaction. Educational Technology, 38, 29-35.
Hamuy, E. & Galaz, M. (2010). Information versus communication in course management
system participation. Computers & Education, 54, 169177.
Hollender, N., Hofmann, C, Deneke, M., & Schmitz, B. (2010). Integrating cognitive load theory
and concepts of humancomputer interaction. Computers in Human Behavior, 26, 1278-
1288.
Huffman, H. W., & Huffman, A. H. (2012). Beyond basic study skills: The use of technology for
success in college. Computers in Human Behavior, 28. 583-590. doi:
10.1016/j.chb.2011.11.004
Inversini, A., Botturi, L., & Triacca, L. (2006). Evaluating LMS usability for enhanced e-
learning experience. Paper presented at the EDMEDIA Conference, Orlando, Florida.
ISO (1998). ISO 9241-11 Ergonomic requirements for office work with visual display terminals
(VDTs) Part 11: Guidance on usability. Retrieved from
http://www.iso.org/iso/catalogue_detail.htm?csnumber=16883
Kibaru, F., & Dickson-Deane, C. (2010). Model for training usability evaluators of e-learning.
Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare,
and Higher Education 2010 (pp. 517-522). Chesapeake, VA: AACE. Retrieved from
http://www.editlib.org/p/35596.
Kidney, G., & Puckett, E. (2003). Rediscovering the first principles through online learning.
Quarterly Review of Distance Education, 4, 203-212.
Kim, J ., Kwona, Y., & Cho, D. (2011). Investigating factors that influence social presence and
learning outcomes in distance higher education. Computers & Education, 57(2), 1512-
1520. doi:10.1016/j.compedu.2011.02.005
Ku, H. Y., & Lohr, L. L. (2003). A case study of Chinese students attitudes toward their first
online learning experience. Educational Technology Research and Development, 51(3),
95-102. doi: 10.1007/BF02504557
Kumar, S., Gankotiya, A. K., & Dutta, K. (2011). A comparative study of moodle with other e-
learning systems. Electronics Computer Technology (ICECT), 2011 3rd International
Conference on (Vol. 5, pp. 414418). doi:10.1109/ICECTECH.2011.5942032
Lee, L. Y., & Croker, R. (2006). A contingency model to promote the effectiveness of expatriate
training. Industrial Management + Data Systems, 106(8), 1187-1205. Retrieved from
ABI/INFORM Global. (Document ID: 1143355771).
Lin, H. F. (2010). An application of fuzzy AHP for evaluating course website quality. Computers
& Education, 54(4), 877-888. doi:10.1016/j.compedu.2009.09.017
Malikowski, S. R., Thompson, M. E., & Theis, J . G. (2007). A model for research in to course
management systems: Bridge technology & learning theory. J. Educational Computing
Research. 36(2), 149-173.
Morgan, G. (2003). Faculty use of course management systems. EDUCAUSE. Retrieved from
http://net.educause.edu/ir/library/pdf/ers0302/rs/ers0302w.pdf
ON24, Inc. (2012). Is your learning management system leaving your users dazed and confused?
[Whitepaper]. Retrieved from http://www.astd.org/Professional-Resources/ASTD-
Professional-Partners/Professional-Partner-Content/2013/02/Is-Your-Learning-
Management-System-Leaving-Your-Users-Dazed-and-Confused
Learning and Performance Quarterly, 1(4), 2013 37

Ozkan, S., Koseler, R., & Baykal, N. (2009). Evaluating learning management
systems. Transforming Government: People, Process and Policy, 3(2), 111-130. doi:
http://dx.doi.org/10.1108/17506160910960522
Oztekin, A., Kong, Z. J ., & Uysal, O. (2010). UseLearn: A novel checklist and usability
evaluation method for eLearning systems by criticality metric analysis. International
Journal of Industrial Ergonomics, 40, 455-469. doi:10.1016/j.ergon.2010.04.001
Schmidt, C. T. (1997). The systemics of dialogism: on the prevalence of the self in HCI design.
Journal of the American Society for Information Science, 48(11), 10731082.
Simonson, M. (2007). Course management systems. Quarterly Review of Distance Education,
8(1), vii-ix.
Sims R., Dobbs, G., & Hand, T. (2002). Enhancing quality in online learning: Scaffolding
planning and design through proactive evaluation. Distance Education, 23(2), 135-148.
Southwell, B.G., Anghelceva, G., Himelboima, I., & J onesa, J . (2007). Translating user control
availability into perception: The moderating role of prior experience. Computers in
Human Behavior, 23(1), 554563.
umak, B., Hericko, M., Punik, M. (2011). A meta-analysis of e-learning technology
acceptance: The role of user types and e-learning technology types. Computers in Human
Behavior, 27, 20672077.
Sung, E., & Mayer, R. (2012). Affective impact of navigational and signaling aids to e-learning.
Computers in Human Behavior, 28, 473-483. doi: 10.1016/j.chb.2011.10.019
Sweller, J ., van Merrienboer, J . J . G., & Paas, F. G. W. C. (1998). Cognitive architecture and
instructional design. Educational Psychology Review, 10(3), 251-296.
Tsang, P., Kwan, R., & Fox, R. (2007). Enhancing learning through technology. World Scientific
Publishing Co. Hong Kong.
Tsao, J . Z., J ang, S. T., & Chen, W. C. (2007). Research of university students' learning
satisfaction of physical education- A case study on Chung Yuan Christian University.
Journal of Taipei Physical Education. 15, 322-333.
Tselios, N., Avouris, N., & Komis, V. (2008). The effective combination of hybrid usability
methods in evaluating educational applications of ICT: Issues and challenges. Education and
Information Technologies, 13(1), 5576. doi:10.1007/s10639-007-9045-5
Turner, M., Kitchenham, B., Brereton, P., Charters, S., & Budgen, D. (2010). Does the
technology acceptance model predict actual use? A systematic literature review.
Information and Software Technology, 52, 463479.
Unal, Z., & Unal, A. (2011). Evaluating and comparing the usability of web-based course
management systems. Journal of Information Technology Education, 1019-1038.
Venkatesh, V., Thong, J . Y. L., & Xu, X. (2012). Consumer acceptance and use of information
technology: Extending the unified theory of acceptance and use of technology. MIS
Quarterly, 36(1), 157178. Retrieved from
http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2002388
Womble, J . (2007). E-learning: The relationship among learner satisfaction, self-efficacy, and
usefulness. The Business Review, Cambridge, 10(1), 182189.
Yang, D., Olesova, L., & Richardson, J . C. (2010). Impact of cultural differences on students'
participation, communication, and learning in an online environment. Journal of
Educational Computing Research, 43(2), 165-182.
Zhang, C.X. (2004). Educational Psychology. Taipei: Tunghua Books Co.