Computers in Human Behavior 27 (2011) 20672077

Contents lists available at SciVerse ScienceDirect

Computers in Human Behavior

journal homepage: www.elsevier.com/locate/comphumbeh

Review

A meta-analysis of e-learning technology acceptance: The role of user types and e-learning technology types
Botjan umak , Marjan Hericko, Maja Punik
Faculty of Electrical Engineering and Computer Science, Smetanova ulica 17, University of Maribor, Slomkov trg 15, 2000 Maribor, Slovenia

a r t i c l e

i n f o

a b s t r a c t
Existing literature in the eld of e-learning technology acceptance reects a signicant number of independent studies that primarily investigate the causal relationships proposed by technology acceptance theory, such as the technology acceptance model (TAM). To synthesize the existing knowledge in the eld of e-learning technology acceptance, we have conducted a systematic literature review of 42 independent papers, mostly published in major journals. Furthermore, in order to view the research context by combining and analyzing the quantitative results of the reviewed research studies, a meta-analysis of the causal effect sizes between common TAM-related relationships was conducted. The main ndings of this study, which is the rst of its kind, are: (1) TAM is the most-used acceptance theory in e-learning acceptance research, and (2) the size of the causal effects between individual TAM-related factors depends on the type of user and the type of e-learning technology. The results of the meta-analysis demonstrated a moderating effect for user-related factors and technology-related factors for several evaluated causal paths. We have gathered proof that the perceived ease of use and the perceived usefulness tend to be the factors that can inuence the attitudes of users toward using an e-learning technology in equal measure for different user types and types of e-learning technology settings. 2011 Elsevier Ltd. All rights reserved.

Article history: Available online 30 August 2011 Keywords: E-learning Acceptance Meta-analysis Moderator analysis

Contents 1. 2. 3. 4. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Backgrounds: information technology acceptance theories. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Research methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Data analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1. Causal effect size analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2. The search for moderating variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.1. User type. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2.2. E-learning technology type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3. Meta-analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2067 2068 2068 2070 2070 2070 2071 2073 2073 2076 2076

5.

1. Introduction E-learning is a way of learning that can provide education and training with the use of information communication technologies (ICT) to anyone, anytime and anywhere. E-learning technologies are mostly used by universities and other educational organizations
Corresponding author. Tel.: +386 2 220 7378; fax: +386 2 220 7272.
E-mail addresses: Bostjan.Sumak@uni-mb.si (B. umak), Marjan.Hericko@uni mb.si (M. Hericko), Maja.Pusnik@uni-mb.si (M. Punik). 0747-5632/$ - see front matter 2011 Elsevier Ltd. All rights reserved. doi:10.1016/j.chb.2011.08.005

for providing new and innovative ways for delivering education to their students. Studies on e-learning acceptance mostly incorporate students as subjects; researchers try to explain the factors inuencing the acceptance of e-learning technology by students. From another perspective, for the successful implementation and introduction of e-learning technologies, they must be accepted and used by teachers or professors who use these technologies for providing learning materials to their students. Recently, the importance of e-learning has been rising, especially in the business sector, where companies have recognized the benets of using e-learning

2068

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077

technologies to provide cost-effective on-line learning for their employees (Chiu & Wang, 2008; Karaali, Gumussoy, & Calisir, 2011). It is therefore important to search for factors that may inuence the perceptions of employees when using a specic e-learning technology. The use of e-learning technologies must have a positive impact on users. When the user is presented with a new e-learning technology, different factors may inuence their decision on how and when they will use a particular technology. Furthermore, the weight of the impact of these factors may differ for different user types and e-learning technology types. Existing literature comprises several studies that deal with the identication of factors that inuence the behavioral intentions of users and the actual use of an e-learning technology. E-learning acceptance studies mostly use well-known and contemporary acceptance theories and approaches that have been developed and continuously improved over the previous two decades. A quick, non-systematic review has revealed that TAM is the most common ground theory in e-learning acceptance literature. Researchers mostly explain the intentions of a user towards using an e-learning technology by using or extending the TAM research model. A study that would incorporate information from existing e-learning technology acceptance studies in order to provide an objective picture of the results of research using TAM in the previous 10 years was not found. Moreover, to date, none of the research has dealt with explaining whether individual causal effects depend on the type of the user or the e-learning technology type. The main objectives of this study were to (1) systematically examine existing knowledge in the eld of e-learning acceptance, and (2) to statistically compare the size of the effects in the most common causal relationships in order to provide evidence for a moderating role of the user type and e-learning technology type. In this study, we conducted a meta-analysis of 494 causal effect sizes between different factors that were evaluated in 42 independent studies. Hedges g statistic was the metric that was used to describe the differences in the arithmetic means of individual studies. This paper is organized as follows: in the next section, most well-known technology acceptance theories are introduced. The third section describes the research methodology of our study. In the subsequent section, the results of the data analysis are given. In the last section, we present our conclusions and future work.

utilization (MPCU), the innovation diffusion theory (IDT) and the social cognitive theory (SCT). Their study resulted in a new theoretical model, called the unied theory of acceptance and the use of technology (UTAUT). Another TAM-based use model was introduced by Liaw, Chang, Hung, and Huang (2006), which they called a three-tier use model (3-TUM). According to the 3-TUM model, individual attitudes toward IT can be divided into following tiers: (1) the tier of individual experience and system quality, (2) the affective and cognitive tier, and (3) the behavioral intention tier. In IT acceptance studies, researchers are mostly interested in the relationships among constructs in which empirical data is usually statistically analyzed using structural equation modeling (SEM) and the results are presented in a set of causal relationships; each causal relationship is described with the following attributes: independent variables, dependent variables, path coefcient size (b), and signicance level (p). In e-learning acceptance literature, we can nd variations in the predicted effect sizes and signicance levels. This fact prompted our rst research question: RQ1: What is the mean causal effect size of a particular factor (usefulness, ease of use, etc.) on the behavioral intentions and attitudes of users when using an e-learning technology? E-learning acceptance studies have been applied for different elearning technologies and have been using different user types as respondents. However, what is lacking is a study that would explain how effect sizes differ with different e-learning technology types and user types. We therefore formulated our second research question: RQ2: Does the user type and/or e-learning technology type have a moderating role in causal relationships in TAM?

3. Research methodology To answer the above-stated research questions, a systematic review of existing literature was conducted in order to collect empirical data about e-learning acceptance. After the literature review, a meta-analysis was conducted to combine various results, taking into account the relative sample and effect sizes (King & He, 2006). Studies relevant to the analysis were sought based on a combination of keywords, either related to acceptance theories (TAM, TTF, UTAUT, etc.) or keywords related to e-learning technologies (e-learning, elearning, on-line learning, web learning, etc.). The search in different databases (ScienceDirect, IEEExplore, ACM, etc.) and publicly available search engines (Google, Yahoo) provided 42 studies (see Table 1 for a complete list of identied studies). To ensure that all studies would be analyzed consistently, rules about coding data about study characteristics and the results were specied. For each study, the following information was recorded:  Sample size the number of respondents included in the sample frame.  User type the type of respondents that participated in the study, such as students, teachers, professors, and employees.  E-learning technology type the type of e-learning technology for which the acceptance research was conducted.  Ground theory the ground acceptance theory that was used in the study.  Causal relationships all causal links that were evaluated in a study were recorded with the following data:  Independent variable the name of the independent variable.  Dependent variable the name of the dependent variable.  Path coefcient () the size of the path coefcient.  Signicance level the p value.

2. Backgrounds: information technology acceptance theories In information technology (IT) acceptance literature, there are different streams of research that examine how and why individuals adopt new information technology. According to Venkatesh, Morris, Davis, and Davis (2003), IT adoption research can be distinguished between (a) research that focuses on the individual acceptance of IT, in which the behavioral intentions of users, or actual use, are used as a dependent variable, and (b) research that is more focused on implementation success at the organizational level. Davis (1989) proposed a technology acceptance model (TAM) to explain a potential users behavioral intentions of using a technological innovation. TAM has become one of the most widely used technology acceptance theories. TAM is also the most commonly used theory in e-learning acceptance studies, which was borne out by our systematic review of existing literature. By conducting a meta-analysis, King and He (2006) provided evidence that TAM is a powerful and robust predictive model. However, Venkatesh et al. (2003) reviewed existing literature and empirically compared eight theoretical models: the theory of reasoned action (TRA), TAM, the motivational model (MM), the theory of planned behavior (TPB), the combined TAM and TPB model, the model of PC

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077 Table 1 Literature review in the eld of e-learning acceptance. Study Brown (2002) Lee, Cho, Gay, and Davidson (2003) Yi and Hwang (2003) Hu, Clark, and Ma (2003) Ong, Lai, and Wang (2004) Wagner and Flannery (2004) Saade and Bahli (2005) Lee, Cheung, and Chen (2005) Ong and Lai (2006) Roca, Chiu, and Martinez (2006) Pituch and Lee (2006) Liaw, Huang, and Chen (2007) Toral, Barrero, and Martnez-Torres (2007) Ngai, Poon, and Chan (2007) Zhang, Zhao, and Tan (2008) Lee (2008) Chiu and Wang (2008) van Raaij and Schepers (2008) Padilla-Melendez, Garrido-Moreno, and Del Aguila-Obra (2008) Roca and Gagne (2008) Liaw (2008) Wang and Wang (2009) Mcgill and Klobas (2009) Teo, Lee, Chai, and Wong (2009) Teo (2009) Sreb, Halvari, Gulli, and Kristiansen (2009) Lee, Yoon, and Lee (2009) Tao, Cheng, and Sun (2009) Liu, Liao, and Pratt (2009) Snchez-Franco, Martnez-Lpez, and MartnVelicia (2009) Cho, Cheng, and Lai (2009) Bourgonjon, Valcke, Soetaert, and Schellens (2010) Snchez and Hueros (2010) Sanchez-Franco (2010) Lee (2010) Liu, Chen, Sun, Wible, and Kuo (2010) Liu, Li, and Carlsson (2010) Lee, Hsieh, and Ma (2011) Karaali et al. (2011) Lin (2011) Pynoo et al. (2011) Chen (2010) Sample size 73 31 109 138/134 140 184/264 102 544 156 172 259 30 142 836 121 1107 286 40 225 166 424 268 269 250/245 475 124 214 185 102 304/376 445 858 226 431 363 436 220 357 546 256 64/41/ 55 193 User type Students Students Students Teachers/ professors Employees Employees Students Students Employees Students Students Teachers/ professors Students Students Students Students Students Employees Students Employees Students Teachers/ professors Students Teachers/ professors Teachers/ professors Teachers/ professors Students Students Students Teachers/ professors Students Students Students Students Students Students Students Employees Employees Employees Teachers/ professors Employees E-learning technology type E-learning System (WebCT) E-learning Technology/Tool (Web Based Collaboration in Virtual Learning Environment) E-learning System (BlackBoard) E-learning Technology/Tool (MS PowerPoint) E-learning System (Asynchronous System) E-learning Technology/Tool (computer-based training support tool) E-learning System Web-based Learning Technology (FaBWeb) E-learning System E-learning System E-learning System E-learning System E-Learning Technology/Tool E-learning System (WebCT) E-learning System E-learning System E-learning System E-learning System E-learning Technology/Tool (Web-based Collaboration) E-learning System E-learning System (BlackBoard) E-learning System E-learning System (WebCT) E-learning Technology/Tool E-learning Technology/Tool E-learning Technology/Tool E-learning E-learning E-learning E-learning System Technology/Tool (video games) Technology/Tool (textaudio presentation) Technology/Tool Theory TAM TAM TAM TAM TAM TAM TAM TAM TAM TAM TAM 3-TUM TAM TAM TAM TAM UTAUT TAM TAM TAM TAM TAM TTF TAM TAM TAM + SDT TAM TAM TAM TAM TAM TAM

2069

E-learning Technology/Tool E-learning Technology/Tool (video games) E-learning E-learning E-learning E-learning E-learning E-learning E-learning E-learning E-learning System (Moodle) System (WebCT) System Technology/Tool (On-line community) Technology/Tool (Mobile Learning Tool) System System System System

TAM TAM TAM + TPB + ECM TAM TAM TAM TAM TAM UTAUT TAM

E-learning System

The journal Computers & Education publishes, by far, the most e-learning acceptance papers (see Table 2). In existing e-learning technology studies, TAM is the most common theory being used as a ground theory (see Fig. 1). Researchers usually extend the basic research models with prior factors. In Fig. 2, we can see that the most common user type used in existing research is students, followed by employees and nally academics. Factors that may have an inuence on the decisions of teachers or professors to use or avoid a specic e-learning technology were examined in seven independent studies, in which four incorporated two or more independent sample frames. Table 3 summarizes the data about samples from existing e-learning acceptance studies. It is important to note that one paper may include multiple

Table 2 Distribution of e-learning acceptance research papers. Journal Computers & Education Computers in Human Behavior Information & Management International Journal of HumanComputer Studies The Electronic Journal of Information Systems in Developing Countries Knowledge-Based Systems Tsinghua Science & Technology Journal of European Industrial Training Other Count of papers (total = 42) 25 5 5 2 1 1 1 1 1

2070

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077

UTAUT 4% TTF 2% TPB 2% 3-TUM 2% Other 4%

positive causal links (NNSIG+), the number of signicant negative causal links (NSIG), the number of non-signicant negative causal links (NNSIG), the maximum signicant positive path coefcient size (SIG+MAX), the minimum signicant positive coefcient size (SIG+MIN), the average signicant positive coefcient size (SIG+AVE), the maximum signicant negative coefcient size (SIGMAX), and the minimum signicant negative coefcient size (SIGMIN). The e-learning acceptance model shown in Fig. 3 summarizes elearning technology acceptance research containing two types of causal relationships: (1) paths, originally proposed by TAM, and (2) supplementary causal links, found in existing studies. Causal links include the following coefcient sizes (bMAX, bMIN, bAVE):  bMAX (the maximum estimated path coefcient),  bMIN (the minimum path coefcient), and  bAVE (the average path coefcient).

TAM 86%
Fig. 1. Theories applied in e-learning acceptance studies.

Teach./Prof.; 193 Students; 322

Employees; 250
Fig. 2. Average sample size by user type.

samples; therefore, the cumulative number of samples is greater than the number of reviewed papers. 4. Data analysis The statistical analysis was based on 494 records about causal relationships, evaluated in the systematic literature review process. First, a descriptive analysis of causal effects was carried out. Then, the search for possible moderator variables was performed by conducting a meta-analysis on effect sizes regarding user type and e-learning technology type. 4.1. Causal effect size analysis Table 4 provides descriptive statistics about causal relationships summarized from existing studies with the following data: the independent variable (Independent), the dependent variable (Dependent), the number of occurrences in which this causal relationship was analyzed in existing studies (N), the number of significant positive causal links (NSIG+), the number of non-signicant

Existing e-learning acceptance studies mostly conrm causal links proposed by the TAM research model (see Fig. 3 and Table 4); therefore, TAM represents a good ground theory for studying the factors that inuence the decision of users on whether they will use or refuse to use a specic e-learning technology. In technology acceptance studies, researchers often search for external factors with a signicant inuence on the perceptions of users regarding the usefulness and ease of use of technology. Some of the important prior factors are listed in Table 4 and included in the technology acceptance model shown in Fig. 3. Because of limited space, only the factors that were evaluated in at least three independent e-learning acceptance studies were included. The complete list of prior factors in which existing research have conrmed either a signicant positive or signicant negative effect on perceptions of users when using e-learning technology is larger. The path coefcient sizes between TAM constructs are summarized in Table 5. All causal paths were evaluated in at least 10 independent studies, with the exception of the path BI ? U, which was evaluated in six independent studies. The summary reveals the strongest coefcient sizes for paths PU ? ATU, PU ? BI, PEOU ? PU, and BI ? U with large means and rather small standard deviations. The PEOU has the weakest inuence on BI with a mean of 0.238. The latter causal relationship, however, was excluded in the search for moderating variables, because it was analyzed in an insufcient number of studies. 4.2. The search for moderating variables Existing technology acceptance studies often compare causal effect sizes between inexperienced and experienced users. The level of experience with technology is the most studied moderator variable in technology acceptance studies (Venkatesh, 2000). In this study, however, we intended to investigate the size of the effects between individual variables in relation to the user type and elearning technology type. King and He (2006) conducted a metaanalysis in which users were classied into students, professionals and general users. In this study, users were classied into: (a) employees users who use e-learning technologies at work/home for training/learning, (b) students who use e-learning technolo-

Table 3 Summary of samples in existing e-learning acceptance studies. Respondent type Students Employees Teachers/professors Overall Samples (N) 28 10 13 51 MIN sample size 31 140 30 31 MAX sample size 1107 546 475 1107 AVE sample size 322 250 193 270 Cumulative size 8829 2293 2504 13764

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077 Table 4 Summary of causal relationships investigated in existing e-learning acceptance studies. Independent Causal links between TAM constructs PEOU PU PU PEOU ATU PEOU PU BI ATU PEOU Other causal links evaluated in e-learning acceptance studies SE SE PU SN SN SI COM FC PE CONF EXP SYSQ SYSQ ANX COMPSE SE CONF MANGSUPP FLOW EXP Dependent PU BI ATU ATU BI BI U U U U PEOU BI SAT BI PU BI BI PEOU BI PU PEOU PU SAT PEOU PU PU SAT PU BI PU N 37 28 16 15 14 14 7 6 4 4 8 6 5 5 5 4 4 3 3 3 3 3 3 3 3 3 3 3 3 3 NSIG+ 36 28 16 13 14 10 5 6 3 4 7 5 4 3 3 4 3 3 3 3 3 2 3 1 2 2 2 2 2 2 NNSIG+ 1 0 0 2 0 0 2 0 1 0 1 1 1 2 0 0 0 0 0 0 0 0 0 0 0 0 1 1 0 0 NSIG 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 NNSIG 0 0 0 0 0 4 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 1 1 0 0 1 1 SIG+MAX 0.73 0.85 0.75 0.71 0.90 0.41 0.67 0.75 0.40 0.30 0.53 0.96 0.59 0.48 0.30 0.59 0.27 0.73 0.61 0.50 0.49 0.45 0.35 0.34 0.33 0.29 0.28 0.28 0.24 0.15 SIG+MIN 0.20 0.13 0.18 0.17 0.10 0.12 0.18 0.19 0.22 0.11 0.24 0.19 0.47 0.14 0.19 0.22 0.11 0.09 0.33 0.17 0.12 0.21 0.27 0.34 0.17 0.07 0.20 0.22 0.13 0.15 SIG+AVE 0.40 0.40 0.51 0.29 0.33 0.24 0.45 0.44 0.33 0.22 0.38 0.46 0.52 0.30 0.25 0.36 0.18 0.34 0.44 0.38 0.25 0.33 0.30 0.34 0.25 0.18 0.24 0.25 0.19 0.15 SIGMAX

2071

SIGMIN

0.23

0.32

0.22

0.53

Variables in order of appearance: perceived ease of use (PEOU), perceived usefulness (PU), behavioral intentions (BI), attitude toward using (ATU), use (U), self-efcacy (SE), satisfaction (SAT), social inuence (SI), compatibility (COM), facilitating conditions (FC), performance expectancy (PE), conrmation (CONF), experience (EXP), system quality (SYSQ), anxiety (ANX), computer self-efcacy (COMPSE), management support (MANGSUPP), ow (FLOW).

Prior Factors

TAM Factors

Anxiety Confirmation Facilitating Conditions Self-Efficacy Information Quality Computer SelfEfficacy Technical Support ...

System Quality Experience Subjective Norms Managmenet Support Perceived Affective Quality Job Relevance Compatibility ...

(0.67, 0.18, 0.45)

Perceived Usefulness (PU)

(0.85, 0.13, 0.40) (0.75, 0.18, 0.51)

(0.73, 0.20, 0.40)

Attitude (0.90, 0.10, 0.33) Behavioral (0.75, 0.19, 0.44) Toward Using Intentions (BI) (ATU)
(0.40, 0.22, 0.33)

Usage (U)

(0.71, 0.17, 0.29)

Perceived Ease (0.41, 0.12, 0.24) of Use (PEOU)

(0.30, 0.11, 0.22)

Original TAM causal paths Additional paths found in studies

Fig. 3. Summary of causal links between TAM-related constructs.

gies at school, and (c) teachers/professors who use e-learning technologies for pedagogical purposes. Additionally, we wanted to investigate whether the size of causal effects depends on the type of e-learning technology being used. Therefore, we distinguished between studies that investigated factors with an impact on acceptance and use of an e-learning system, and studies about the acceptance of e-learning technologies or tools that cannot be classied as an e-learning system.

4.2.1. User type Table 6 and Figs. 4a4f show the results of an analysis of six causal relationships for employees, students and teachers/professors. In the case of the relationship PEOU ? ATU, the mean values of the causal effect size are very similar for both groups: employees and teachers/professors, where there was only one study for the category of employees. The mean value of the causal effect size for the same causal relationship was a bit larger for students. The

2072

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077

Table 5 Summary of path coefcients between TAM constructs. PEOU ? ATU Number of samples Cumulative sample size Mean Median Stand. deviation Minimum Maximum 13 4152 0.294 0.248 0.138 0.17 0.71 PU ? ATU 16 4983 0.508 0.514 0.148 0.18 0.75 ATU ? BI 14 3962 0.326 0.296 0.200 0.10 0.90 PEOU ? BI 10 2948 0.238 0.225 0.089 0.12 0.41 PU ? BI 28 7901 0.397 0.420 0.150 0.13 0.85 PEOU ? PU 36 11015 0.403 0.390 0.146 0.20 0.73 BI ? U 6 915 0.437 0.388 0.191 0.19 0.75

Table 6 Coefcient size (b) analysis by user type. PEOU ? ATU Employees Number of samples Total sample size Mean Median Stand. deviation Minimum Maximum Students Number of samples Total sample size Mean Median Stand. deviation Minimum Maximum Teachers/professors Number of samples Total sample size Mean Median Stand. deviation Minimum Maximum 1 546 0.25 / / 0.25 0.25 7 1956 0.333 0.290 0.189 0.18 0.71 5 1650 0.247 0.245 0.054 0.17 0.33 PU ? ATU 2 802 0.505 0.505 0.120 0.42 0.59 9 2531 0.514 0.600 0.194 0.18 0.75 5 1650 0.496 0.475 0.061 0.42 0.57 ATU ? BI 3 952 0.196 0.178 0.047 0.16 0.25 6 1438 0.325 0.346 0.106 0.16 0.46 5 1650 0.404 0.383 0.309 0.09 0.90 PEOU ? BI 4 819 0.310 0.310 0.091 0.21 0.41 6 2129 0.189 0.209 0.049 0.12 0.23 0 NA NA NA NA NA NA PU ? BI 7 1640 0.394 0.400 0.057 0.32 0.48 13 4197 0.364 0.419 0.150 0.13 0.60 8 1849 0.452 0.435 0.201 0.22 0.85 PEOU ? PU 7 1621 0.453 0.400 0.137 0.31 0.69 21 7146 0.368 0.324 0.151 0.20 0.73 8 2190 0.452 0.465 0.129 0.24 0.65

Fig. 4a. PEOU ? ATU.

Fig. 4b. PU ? ATU.

mean values of the causal effect size for the relationship PU ? ATU were very similar for all user type categories. However, there were differences in the effect sizes for the relationship ATU ? BI, where the mean value of the effect size was higher for teachers/professors in comparison with the mean value of the effect size for employees. The mean value of the effect size for the same relationship in the case of students lies somewhat in between. PEOU seems to have a greater inuence on behavioral intentions for employees when compared with students. Unfortunately, because of the lack of studies analyzing the relationship PEOU ? BI using professors/ teachers as a subject, no comparison could be made with the other two user types. The results of the analysis also show that in the case of the relationships PU ? BI and PEOU ? BI, the causal effect

size is signicantly larger for employees and teachers/professors than it is for students. The box-plot diagrams in Figs. 4a4f show the intervals of the effect sizes of the six analyzed causal relationships for the three user categories. When comparing the effect sizes in different causal relationships for employees and students, the effect sizes mostly overlap. A difference was found only in the case of PEOU ? BI, where the effect size was larger for employees. Overall, these results indicate that students can be used as surrogates for employees in a study that investigates the acceptance of an e-learning technology. Perceptions of employees when using an e-learning technology for learning purposes are very close to those of students.

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077

2073

e-learning technologies or tools are presented. The mean values of the causal effect size for almost all six causal links were very close for both e-learning technology categories. A difference was found in the path ATU ? BI, where the mean value of the effect size was higher for other e-learning technology/tools when comparing the mean value of the causal effect in the case of an e-learning system. 4.3. Meta-analysis The meta-analysis was performed by focusing on a set of TAMrelated causal relationships, as listed in Table 5. Since several independent studies were included, the meta-analysis was conducted on a random effects basis. The underlying assumption was that every study included in this study is taken from a population that is likely to have a different effect size to any other study included in the meta-analysis. Additionally, the heterogeneity tests Q and I2 were assessed in order to validate the use of a random effects analytic base. The Q statistic, proposed by Cohran (1954), can exhibit a poor ability to detect a true heterogeneity among studies when the meta-analysis includes a small number of studies (Huedo-Medina, Snchez-Meca, Marn-Martnez, & Botella, 2006). The I2 measures the extent of true heterogeneity, where the I2 index can be interpreted as the percentage of total variability in a set of effect sizes due to true heterogeneity the intra-study variability (HuedoMedina et al., 2006). Table 8 summarizes the estimation of the effect sizes of path coefcients together with results of heterogeneity tests. Q estimates for all path coefcients were signicant, resulting in a rejection of the null hypothesis about homogeneity for all paths. According to the I2 estimates and the classication proposed by Higgins and Thompson (2002), high heterogeneity was found for almost all paths with the exception of causal paths PEOU ? ATU and PEOU ? BI, where medium heterogeneity was estimated. The results of both heterogeneity tests conrm that the random-effects model is an appropriate basis for a meta-analysis. In order to search for moderator variables, we rst clustered our studies according to user type. Table 9 lists the estimation of the combined effect sizes (Hedges g) of six causal paths for different user types, together with condence intervals, Z-values and p-values. The effect sizes were signicant for all three user types, where the effect size was higher for students in the relationships PEOU ? ATU and PU ? ATU. However, the attitude toward using and perceived usefulness can have the biggest inuence on the intention of teachers/professors using a specic e-learning technology. When comparing the three user types, it is also more likely that professors/teachers will nd a specic e-learning technology useful for teaching purposes if it is easy to use. The perceived ease of use has a higher impact on the intention of employees using a specic e-learning technology in comparison with students. Next, we labeled studies as dealing with e-learning systems or other e-learning technologies. Table 10 shows that effect sizes are very close for almost all causal paths for both e-learning technology types. An exception was found for the relationship ATU ? BI, where the attitude of users towards using an e-learning system had a lower inuence on their intentions of using an e-learning system. A difference was also found for the relationship PEOU ? BI, where the perceived ease of use can have a greater inuence on the intention of users to use an e-learning system compared to other elearning technologies. Kampenes, Dyba, Hannay, and Sjoberg (2007) summarized the effect sizes from existing research and dened three categories for the sizes of Hedges g values. Table 11 provides an interpretation of the effect sizes estimated in this study, according to the categories proposed by Kampenes et al. (2007). The interpretation of effect size reveals that the size of the impact of the two independent variables, PU and PEOU, on ATU are

Fig. 4c. ATU ? BI.

Fig. 4d. PEOU ? BI.

Fig. 4e. PU ? BI.

Fig. 4f. PEOU ? PU.

4.2.2. E-learning technology type Next, a comparison of causal effect sizes in relation to the e-learning technology type being used was made. In Table 7 and Figs. 5a5f, an analysis of the six causal links for e-learning systems and other

2074

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077

Table 7 Coefcient size (b) analysis by e-learning technology type. PEOU ? ATU E-learning System Number of samples Total sample size Mean Median Stand. deviation Minimum Maximum Other E-learning Technology/Tool Number of samples Total sample size Mean Median Stand. deviation Minimum Maximum 4 1971 0.280 0.270 0.068 0.211 0.370 9 2181 0.299 0.245 0.164 0.174 0.707 PU ? ATU 5 2227 0.515 0.590 0.220 0.183 0.750 11 2756 0.504 0.475 0.118 0.286 0.695 ATU ? BI 2 909 0.162 0.162 0.003 0.160 0.164 12 3131 0.353 0.346 0.205 0.098 0.896 PEOU ? BI 8 2370 0.257 0.231 0.087 0.137 0.410 2 578 0.159 0.159 0.054 0.120 0.197 PU ? BI 14 4432 0.398 0.410 0.136 0.134 0.600 14 3254 0.396 0.425 0.169 0.180 0.850 PEOU ? PU 17 5661 0.408 0.390 0.161 0.220 0.732 19 5354 0.399 0.410 0.136 0.200 0.645

Fig. 5a. PEOU ? ATU.

Fig. 5d. PEOU ? BI.

Fig. 5b. PU ? ATU.

Fig. 5e. PU ? BI.

Fig. 5c. ATU ? BI.

Fig. 5f. PEOU ? PU.

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077 Table 8 Summary of the effect size of path coefcients. PEOU ? ATU Number of samples Total sample size Hedges g Standard error Variance 95% Lower limit 95% Higher limit Z p (effect size) Heterogeneity test (Q) df (Q) p (heterogeneity) I2 13 4152 0.593 0.066 0.004 0.463 0.722 8.952 <0.001 44.349 12 <0.001 72.942% PU ? ATU 16 4983 1.228 0.135 0.018 0.965 1.492 9.121 <0.001 224.246 15 <0.001 93.311% ATU ? BI 14 3962 0.795 0.138 0.019 0.524 1.065 5.754 <0.001 200.050 13 <0.001 93.502% PEOU ? BI 10 2948 0.484 0.065 0.004 0.385 0.610 7.502 <0.001 20.720 9 <0.05 56.564% PU ? BI 28 7901 0.866 0.069 0.005 0.731 1.001 12.569 <0.001 194.160 27 <0.001 86.094%

2075

PEOU ? PU 36 11,015 0.888 0.061 0.004 0.768 1.008 14.498 <0.001 275.773 35 <0.001 87.308%

Table 9 Effect sizes with moderator variable (user type). Number of studies Hedges g Standard error Variance 95% Conf. interval Low PEOU ? ATU Employees Students Teachers/professors PU ? ATU Employees Students Teachers/professors ATU ? BI Employees Students Teachers/professors PEOU ? BI Employees Students PU ? BI Employees Students Teachers/professors PEOU ? PU Employees Students Teachers/professors 1 7 5 2 9 5 3 6 5 4 6 7 13 8 7 21 8 0.516 0.708 0.502 1.176 1.296 1.135 0.361 0.667 1.237 0.665 0.396 0.819 0.792 1.077 0.934 0.798 1.041 0.089 0.129 0.051 0.266 0.263 0.077 0.069 0.104 0.374 0.102 0.045 0.054 0.103 0.187 0.096 0.078 0.120 0.008 0.017 0.003 0.071 0.069 0.006 0.005 0.011 0.140 0.010 0.002 0.003 0.011 0.035 0.009 0.006 0.014 0.342 0.455 0.402 0.655 0.781 0.984 0.226 0.464 0.504 0.465 0.308 0.713 0.590 0.710 0.747 0.646 0.806 High 0.689 0.962 0.602 1.698 1.811 1.287 0.496 0.871 1.971 0.864 0.485 0.925 0.994 1.443 1.122 0.950 1.275 5.826 5.743 9.851 4.422 4.930 14.727 5.238 6.425 3.307 6.531 8.761 15.175 7.681 5.761 9.780 10.295 8.708 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Z p

Table 10 Effect sizes with moderator variable (e-learning technology type). Number of studies Hedges g Standard error Variance 95% Conf. interval Low PEOU ? ATU E-learning system Other technology/tool PU ? ATU E-learning system Other technology/tool ATU ? BI E-learning system Other technology/tool PEOU ? BI E-learning system Other technology/tool PU ? BI E-learning system Other technology/tool PEOU ? PU E-learning system Other technology/tool 4 9 5 11 2 12 8 2 14 14 17 19 0.595 0.607 1.325 1.158 0.327 0.886 0.535 0.279 0.849 0.892 0.935 0.852 0.091 0.095 0.359 0.094 0.067 0.165 0.068 0.084 0.092 0.110 0.096 0.082 0.008 0.009 0.129 0.009 0.005 0.027 0.005 0.007 0.008 0.012 0.009 0.007 0.417 0.420 0.621 0.975 0.195 0.563 0.401 0.114 0.669 0.677 0.747 0.692 High 0.772 0.794 2.028 1.341 0.459 1.210 0.668 0.444 1.028 1.108 1.124 1.012 6.560 6.363 3.691 12.379 4.856 5.364 7.823 3.314 9.266 8.119 9.719 10.442 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.001 0.000 0.000 0.000 0.000 Z p

2076

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077

Table 11 The interpretation summary of effect sizes for moderator variables. PEOU ? ATU User type Employees Students Teachers/professors E-learning technology type E-learning system Other technology/tool Overall M M M M M M PU ? ATU L L L L L L ATU ? BI S M L S M M PEOU ? BI M M / M S M PU ? BI M M L M M M PEOU ? PU M M L M M M

Notes: S small size, M medium size, L large size.

very similar in relation to the user type and e-learning technology type. For the other four causal relationships that were investigated in this study, the causal effect size depends on the two moderating variables. For example, the perceived usefulness had a larger impact on behavioral intention. The same result was found in the case of the relationship between perceived ease of use and perceived usefulness. Attitudes toward use affected all three user categories differently: the impact was largest for teachers/professors and smallest for employees. The perceived ease of use can have a medium impact on the behavioral intentions of users. However, because of a lack of studies that include teachers/professors as subject types, a comparison between all three user types was not possible. With e-learning technology type as a moderator variable, the differences were only found for two causal relationships. Attitudes towards use can have a smaller inuence on behavioral intentions in the case of an e-learning system. However, when users are working with an e-learning system, the inuence of perceived ease of use can have a greater inuence on the behavioral intentions of users in comparison with other e-learning technologies. 5. Conclusion The purpose of our effort was to combine several independent studies about the individual acceptance and use of e-learning technologies and extract a general conclusion from existing knowledge. The main objective of the study was to search for the mean causal effect size in TAM-related causal relationships and to explain whether there are, in fact, factors that may have a moderating role in these relationships. A review of literature showed that TAM is the mostused theory in e-learning acceptance research and indicated the small presence of studies that have used UTAUT as a ground theory; therefore, future research needs to include studies that will evaluate this state-of-the art theory in the eld of e-learning acceptance. A systematic review of existing literature in the eld of e-learning acceptance comprised 42 papers, mostly published in journals, and this constituted the basis for our empirical data analysis. Based on descriptive statistics, an acceptance model with information about the minimum, maximum and average path coefcient size in TAM-related relationships was created. In existing e-learning acceptance research, TAM-related relationships are mostly supported. Therefore, we can conclude that TAM constitutes a good model for the investigation of the acceptance of e-learning technologies. Additionally, the literature review provided a set of factors that have inuence on the main TAM constructs. Some of them were included in the summarized e-learning acceptance model presented in this paper. The meta-analysis of the causal path coefcients showed that the sizes of PEOU and PU on ATU are very similar for different elearning technology types and user types as well. A large effect size was found in the relationship of PU on ATU, whereas a medium effect size was estimated in the relationship between PEOU and ATU. From this result, we can conclude that PU in general can have a

greater impact on users attitudes towards using an e-learning technology. A relationship between PEOU and PU was found only for teachers/professors. In relation to this user group, a large effect was also found in other TAM-related relationships. In relation to elearning technology type, the effect sizes were different in two causal relationships. The attitude toward using had a smaller effect on the BI of users in studies that investigated the acceptance of elearning systems in comparison with other e-learning technologies. However, in the case of an e-learning system, PEOU can have a greater impact on the intentions of users when compared with other e-learning technologies. The ndings of this study have several implications for managers. First, it was shown that students can be used as respondents in the research of e-learning acceptance when investigating the attitudes of learners towards using and their behavioral intentions for using an e-learning technology. For managers, the results of elearning acceptance research that includes students provides useful knowledge about the most important factors that inuence the acceptance of e-learning technologies when introducing such technologies to their employees. For researchers, students can be used as surrogates for employees in a study that investigates the acceptance and use of an e-learning technology. However, this nding contradicts the results of previous meta-analysis research (Kampenes et al., 2007; Peterson, 2001) that showed that the causal effect sizes derived from student subjects can differ from those derived from non-student subjects. However, the e-learning acceptance process can differ for teachers or professors who use e-learning technologies for pedagogical purposes. Therefore, managers at educational institutions must be aware that the usability and utility aspects can have a greater inuence on instructors, rather than on students. From the results of this study, it is evident that literature in the eld of e-learning acceptance and use calls for studies that would be based on acceptance theories other than TAM. Our future work will be dedicated to including forthcoming e-learning acceptance studies and to analyze causal effect sizes of factors from other acceptance theories. When available, with a larger number of elearning acceptance studies, it will be possible to analyze external factors, which are usually investigated by extending basic acceptance theoretical models (e.g. compatibility, experience, etc.). In our future research, we also want to search for other moderating variables by synthesizing results from research studies in the eld of e-learning acceptance. For example, it would be interesting to analyze whether the causal effect sizes of individual factors depend on the culture type.

References
Bourgonjon, J., Valcke, M., Soetaert, R., & Schellens, T. (2010). Students perceptions about the use of video games in the classroom. Computers & Education, 54(4), 11451156. Brown, I. T. J. (2002). Individual and technological factors affecting perceived ease of use of web-based learning technologies in a developing country. The Electronic Journal of Information Systems in Developing Countries, 2002(9), 115.

B. umak et al. / Computers in Human Behavior 27 (2011) 20672077 Chen, H.-J. (2010). Linking employees e-learning system use to their overall job outcomes: An empirical study based on the IS success model. Computers & Education, 55(4), 16281639. Chiu, C., & Wang, E. (2008). Understanding Web-based learning continuance intention: The role of subjective task value. Information & Management, 45(3), 194201. Cho, V., Cheng, T. C. E., & Lai, W. M. J. (2009). The role of perceived user-interface design in continued usage intention of self-paced e-learning tools. Computers & Education, 53(2), 216227. Cohran, W. G. (1954). The combination of estimates from different experiments. Biometrics, 10(1), 101129. Davis, Fred. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319340. Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a metaanalysis. Statistics in Medicine, 21(11), 15391558. Hu, P. J.-H., Clark, T. H. K., & Ma, W. W. (2003). Examining technology acceptance by school teachers: A longitudinal study. Information & Management, 41(2), 227241. Huedo-Medina, T. B., Snchez-Meca, J., Marn-Martnez, F., & Botella, J. (2006). Assessing heterogeneity in meta-analysis: Q statistic or I2 index? Psychological Methods, 11(2), 193206. Kampenes, V. B., Dyba, T., Hannay, J. E., & Sjoberg, D. I. K. (2007). A systematic review of effect size in software engineering experiments. Information and Software Technology, 49(1112), 10731086. Karaali, D., Gumussoy, C. A., & Calisir, F. (2011). Factors affecting the intention to use a web-based learning system among blue-collar workers in the automotive industry. Computers in Human Behavior, 27(1), 343354. King, W., & He, J. (2006). A meta-analysis of the technology acceptance model. Information & Management, 43(6), 740755. Lee, Y. (2008). The role of perceived resources in online learning adoption. Computers & Education, 50(4), 14231438. Lee, M.-C. (2010). Explaining and predicting users continuance intention toward elearning: An extension of the expectationconrmation model. Computers & Education, 54(2), 506516. Lee, M., Cheung, C., & Chen, Z. (2005). Acceptance of Internet-based learning medium: The role of extrinsic and intrinsic motivation. Information & Management, 42(8), 10951104. Lee, J.-S., Cho, H., Gay, G., & Davidson, B. (2003). Technology acceptance and social networking in distance learning. Educational Technology & Society, 6(2), 5061. Lee, Y.-H., Hsieh, Y.-C., & Ma, C.-Y. (2011). A model of organizational employees elearning systems acceptance. Knowledge-Based Systems, 24(3), 355366. Lee, B.-C., Yoon, J.-O., & Lee, I. (2009). Learners acceptance of e-learning in South Korea: Theories and results. Computers & Education, 53(4), 13201329. Liaw, S. (2008). Investigating students perceived satisfaction, behavioral intention, and effectiveness of e-learning: A case study of the Blackboard system. Computers & Education, 51(2), 864873. Liaw, S., Chang, W., Hung, W., & Huang, W. (2006). Attitudes toward search engines as a learning assisted tool: Approach of Liaw and Huangs research model. Computers in Human Behavior, 22(2), 177190. Liaw, S., Huang, H., & Chen, G. (2007). Surveying instructor and learner attitudes toward e-learning. Computers & Education, 49(4), 10661080. Lin, K.-M. (2011). e-Learning continuance intention: Moderating effects of user elearning experience. Computers & Education, 56(2), 515526. Liu, I.-F., Chen, M. C., Sun, Y. S., Wible, D., & Kuo, C.-H. (2010). Extending the TAM model to explore the factors that affect intention to use an online learning community. Computers & Education, 54(2), 600610. Liu, Y., Li, H., & Carlsson, C. (2010). Factors driving the adoption of m-learning: An empirical study. Computers & Education, 55(3), 12111219. Liu, S.-H., Liao, H.-L., & Pratt, J. A. (2009). Impact of media richness and ow on elearning technology acceptance. Computers & Education, 52(3), 599607. McGill, T., & Klobas, J. (2009). A tasktechnology t view of learning management system impact. Computers & Education, 52(2), 496508. Ngai, E., Poon, J., & Chan, Y. (2007). Empirical examination of the adoption of WebCT using TAM. Computers & Education, 48(2), 250267. Ong, C., & Lai, J. (2006). Gender differences in perceptions and relationships among dominants of e-learning acceptance. Computers in Human Behavior, 22(5), 816829.

2077

Ong, C., Lai, J.-Y., & Wang, Y.-S. (2004). Factors affecting engineers acceptance of asynchronous e-learning systems in high-tech companies. Information & Management, 41(6), 795804. Padilla-Melendez, A., Garrido-Moreno, A., & Del Aguila-Obra, A. (2008). Factors affecting e-collaboration technology use among management students. Computers & Education, 51(2), 609623. Peterson, R. A. (2001). On the use of college students in social science research: Insights from a second-order meta-analysis. The Journal of Consumer Research, 28(3), 450461. Pituch, K., & Lee, Y. (2006). The inuence of system characteristics on e-learning use. Computers & Education, 47(2), 222244. Pynoo, B., Devolder, P., Tondeur, J., van Braak, J., Duyck, W., & Duyck, P. (2011). Predicting secondary school teachers acceptance and use of a digital learning environment: A cross-sectional study. Computers in Human Behavior, 27(1), 568575. Roca, J., Chiu, C., & Martinez, F. (2006). Understanding e-learning continuance intention: An extension of the technology acceptance model. International Journal of HumanComputer Studies, 64(8), 683696. Roca, J., & Gagne, M. (2008). Understanding e-learning continuance intention in the workplace: A self-determination theory perspective. Computers in Human Behavior, 24(4), 15851604. Saade, R., & Bahli, B. (2005). The impact of cognitive absorption on perceived usefulness and perceived ease of use in on-line learning: An extension of the technology acceptance model. Information & Management, 42(2), 317327. Snchez, R. A., & Hueros, A. D. (2010). Motivational factors that inuence the acceptance of Moodle using TAM. Computers in Human Behavior, 26(6), 16321640. Sanchez-Franco, M. J. (2010). WebCT The quasimoderating effect of perceived affective quality on an extending technology acceptance model. Computers & Education, 54(1), 3746. Snchez-Franco, M. J., Martnez-Lpez, F. J., & Martn-Velicia, F. A. (2009). Exploring the impact of individualism and uncertainty avoidance in Web-based electronic learning: An empirical analysis in European higher education. Computers & Education, 52(3), 588598. Sreb, ., Halvari, H., Gulli, V. F., & Kristiansen, R. (2009). The role of selfdetermination theory in explaining teachers motivation to continue to use elearning technology. Computers & Education, 53(4), 11771187. Tao, Y.-H., Cheng, C.-J., & Sun, S.-Y. (2009). What inuences college students to continue using business simulation games? The Taiwan experience. Computers & Education, 53(3), 929939. Teo, T. (2009). Modelling technology acceptance in education: A study of preservice teachers. Computers & Education, 52(2), 302312. Teo, T., Lee, C. B., Chai, C. S., & Wong, S. L. (2009). Assessing the intention to use technology among pre-service teachers in Singapore and Malaysia: A multigroup invariance analysis of the technology acceptance model (TAM). Computers & Education, 53(3), 10001009. Toral, S., Barrero, F., & Martnez-Torres, M. (2007). Analysis of utility and use of a web-based tool for digital signal processing teaching by means of a technological acceptance model. Computers & Education, 49(4), 957975. van Raaij, E. M., & Schepers, J. J. (2008). The acceptance and use of a virtual learning environment in China. Computers & Education, 50(3), 838852. Venkatesh, V. (2000). Determinants of perceived ease of use: Integrating control, intrinsic motivation, and emotion into the technology acceptance model. Information Systems Research, 11(4), 342365. Venkatesh, V., Morris, M. G., Davis, B. D., & Davis, F. D. (2003). User acceptance of information technology: Toward a unied view. MIS Quarterly, 27(3), 425478. Wagner, G. D., & Flannery, D. D. (2004). A quantitative study of factors affecting learner acceptance of a computer-based training support tool. Journal of European Industrial Training, 28(5), 383399. Wang, W.-T., & Wang, C.-C. (2009). An empirical study of instructor adoption of web-based learning systems. Computers & Education, 53(3), 761774. Yi, M., & Hwang, Y. (2003). Predicting the use of web-based information systems: Self-efcacy, enjoyment, learning goal orientation, and the technology acceptance model. International Journal of HumanComputer Studies, 59(4), 431449. Zhang, S., Zhao, J., & Tan, W. (2008). Extending TAM for online learning systems: An intrinsic motivation perspective. Tsinghua Science & Technology, 13(3), 312317.