FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 1

Factors that Predict Learners Completion of Massive Open Online Courses (MOOCs)

ETR560

Computer Data Analysis

1
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 2

Introduction

In the last four years, Massive Open Online Courses (MOOCs) have become a major

source of learning for adult students in higher education. According to Pappano (2012), the New

York Times announced that the year of MOOCs was 2012. MOOCs provide thousands of

learning resources to millions of students around the world. MOOCs represent a new online

platform that offers courses in different subjects and includes video lessons, regular activities,

and assessments. The term Massive Open Online Course (MOOC) was first introduced by

Siemens and Downes, who were experimenting with opening their online courses at a Canadian

university. Later, Udacity, edX, and Coursera partnered with some of the most well-known

universities (Harvard, Stanford, and Massachusetts Institute of Technology) to offer MOOCs.

According to Coursera (2015), Coursera provided more than 943 MOOCs to more than

11 million students across more than 200 countries. Compared to traditional higher education

courses, MOOCs are free of charge, and students are not required to fulfill prerequisites

(McAuley, Stewart, Siemens, & Cormier, 2010). Furthermore, MOOCs are flexible in that

students can enroll and participate anywhere and anytime. These features allow people who

cannot attend regular courses, because they are working or taking care of children, for example,

to continue their education.

In spite of all these benefits and the popularity of this new education delivery method that

attracts a large number of students, most students do not complete the courses and usually drop

out a short time after beginning them (Clow, 2013). According to many studies, the average rate

of the completion of MOOCs is lower than 10% (Balch, 2013; Jordan, 2013; Kolowich, 2012).

The issue of the lower rate of MOOC completion raises questions about the factors that might

2
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 3

affect students completion of MOOCs. Further research is needed to investigate what factors

predict students completion when taking MOOCs.

Researchers have conducted several studies to investigate the completion rate of MOOCs.

Jordan (2014) conducted a quantitative study to investigate factors that influence the completion

rate and the number of students who enroll in MOOCs. For purposes of the study, the researchers

collected data about MOOCs from publicly available sources via the Internet and from students

who had participated in MOOCs. The data were analyzed using the linear regression test to

identify different factors that affect enrollment numbers and completion rates. The results of the

study indicated that the length of MOOCs significantly predicted the completion rate. According

to the results of the study, the percentage of students who completed longer courses is lower than

the percentage of those who completed short courses. The study also focused on whether

university rank affected enrollment numbers and completion rates. The study showed that there

was no significant relationship between university rank and completion rate or enrollment

numbers.

In 2013, Belanger and Thornton published an article about the first MOOCs at Duke

University, which were designed by Dr. Roger Barr. According to Belanger and Thornton

(2013), more than 12,000 students from more than 100 different countries enrolled in these

courses. Regarding the students level of education, Belanger and Thornton (2013) stated that

one-third of enrolled students held less than a four-year degree, one-third held a Bachelors or

equivalent, and one-third held an advanced degree (p. 1). According to the results of the study,

most of the students who completed MOOCs were students who held a bachelors degree or

higher. Learners who participated in this course mentioned that lack of time and insufficient

math background and skills were the main reasons that prevented them from completing the

3
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 4

courses.

In 2001, Latanich, Nonis, and Hudson conducted a quantitative study to determine the

differences and similarities between students taking distance courses and students taking face-to-

face courses. The demographic characteristics examined in this study included gender, age,

employment status, motivation, and risk-taking propensity. Latanich, et.al. (2001) stated that

gender was found to be a significant factor influencing students ability to complete distance

courses.

Another very recent study was conducted by Evans, Baker, and Dee (2016) to examine

persistence patterns (enrollment, engagement, persistence, and completion) in MOOCs. The

researchers used a large dataset that included about 2.1 million students, who were enrolled in

more than 44 MOOCs offered on the Coursera platform. The results of the study indicated that

short courses have higher rates of engagement, persistence, and completion. Moreover, the study

found that there was no significant relationship between the number of students in the class or

the number of lectures and persistence or completion rates.

Purpose of the Study

The purpose of this quantitative study is to investigate the factors that predict students

completion of MOOCs.

Research Questions

- Do the following factors (gender, level of education, number of previous MOOCs, length
of MOOCs) predict the students completion of MOOCs?
Significance of the Study

Based on the previous studies, the completion rate of MOOCs is low for a variety of

reasons. The results of this study will help providers of MOOCs, such as universities, colleges,

and institutions, to better understand what contributes to participants completion of MOOCs.

4
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 5

This study will help providers to learn more about the characteristics of individuals enrolled in

MOOCs and increase completion rates for their MOOCs. Providers can estimate which features

and kinds of design motivate students to complete MOOCs. The findings of the study will also

contribute to understanding the issue of completion of MOOCs. The study will provide a

foundation for researchers studying MOOCs. In addition, educators can use the results of this

study as a guideline when designing and teaching MOOCs.

Methodology

Source of Data

The researchers used secondary data from Dr. Reeves and Dr. Stichs study on MOOCs.

The researchers sent an email to Dr. Reeves asking for permission to use the data. The

researchers obtained permission from Dr. Reeves to use the data. The data were collected

through survey questions that were sent to MOOC participants between 2008 and 2012. The

researchers distributed their survey by contacting the main providers of MOOCs at that time

such as Coursera, Udacity, and edXand asking them to share the survey with students who

were enrolled in MOOCs. The survey includes items about participants characteristics,

experience, and motivation to take MOOCs. Also, the survey includes items about the MOOCs

features. Of the 16,100 responses received for this survey, about 34% of them were male and

30% were female (Figure 1). Regarding the participants ages, most of the participants were

between13 and 25 years old, while less than five hundred of the participants were 56 or older.

The missing data from this survey was 65%, which may have occurred because of the large

number of participants.

Data analysis

5
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 6

This section presents the statistical methods that were used in this study. The statistical

software RStudio was used to conduct logistic regression. As the predicted variable (MOOC

completion) is a binary variable that has two values (yes and no), logistic regression was

conducted to estimate the probability of a binary response based on each of the independent

variables. The dependent variable item has four responses (yes, no, dont know, and dont

remember). First of all, the researchers recorded the two responses (dont know and dont

remember) as NA in order to have nice, clean data. Then, to change the dependent variable to

factor variable, students who successfully complete MOOCs were coded as yes, or 0, and

students who did not complete the MOOCs were coded as no, or 1, using the recode and factor

command in RStudio. Based on the research questions, the independent variables or predictors

are gender, level of education, number of previous MOOCs, and length of MOOCs; the predicted

or dependent variable is completion of MOOCs. The question that asking participants about their

gender has three responses (male, female, and dont know). The response dont know was

recorded as NA using the recode command in RStudio.

The researchers hypothesized that:

H0 = the following predictors (gender, level of education, number of previous MOOCs,

length of MOOCs) do not significantly predict the students completion of MOOCs.

H1= the following factors (gender, level of education, number of previous MOOCs,

length of MOOCs) significantly predict the students completion of MOOCs.

6
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 7

Figure 1. Gender Distribution

Figure 2. Age Distribution.

7
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 8

Table 1.

Logistic regression output

Table 2.

Effect size

8
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 9

Result

A binary logistic regression was conducted to predict whether students complete or do

not complete MOOCs using gender, length of MOOC, level of education, and number of

previous MOOCs as predictors. Table 1 shows the output of the logistic regression. First of all,

the researchers looked at the difference between the null deviance and the residual deviance to

see how well this set of predictors predicted whether or not students complete MOOCs. Based on

the results shown in the table, the null deviance is 945.74 with df = 743, and the residual

deviance is 927.33 with df = 739. To obtain the difference between those values, the researchers

subtracted the values (null deviance of 945.74 minus residual deviance of 927.33 equals 18.41),

and the corresponding degrees of freedom (743 - 739 = 4). As a result, the test statistic is 18.41

with 4 degrees of freedom, which has a chi-square distribution. Using the online calculator, the

researchers computed the p-value for the chi-square difference (p < 0.001). Based on the result

(x2 = 18.41, p < .05), the null hypothesis that the following predictors (gender, level of

education, number of previous MOOCs, length of MOOCs) do not significantly predict students

completion of MOOCs is rejected.

The set of these predictors does significantly predict students completion of MOOCs.

Based on the outcome shown in Table 1, this initial model fit well and was significant. As shown

in Table 1, the number of previous MOOCs is a statistically significant positive predictor of

students completion of MOOCs (b = 0.341, p < 0.000233). Specifically, an increased number of

previous MOOCs corresponds to a higher likelihood of MOOC completion. On the other hand,

gender (b = -0.024, p < 0.877), length of MOOCs (b = -0.186, p < 0.242), and education level (b

= 0.093, p < 0.101) are not significant predictors of MOOC completion. As the number of

previous MOOCs was a statistically significant effect, the researchers computed the odds ratios

9
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 10

using RStudio to see how strongly the number of previous MOOCs affects MOOC completion.

As shown in Table 2, the effect size or odds ratio is 1.40722, which means that each unit increase

in number of previous MOOCs is associated with a 1.40 times increase in the odds of MOOC

completion.

Discussion

In the previous studies reviewed in the literature review section, the length of MOOCs

significantly predicted completion rates (Jordan, 2014). According to the result of this study, the

percentage of students who completed longer courses is lower than those who completed short

courses. A similar finding was reported by Evans, Baker, and Dee (2016). The results of that

study indicated that short courses have higher rates of engagement, persistence, and completion.

The current study reported a contradictory finding. The length of MOOCs was not a significant

predictor of MOOC completion.

Regarding the level of education, the current study indicated that completion of MOOCs

is not significantly predicted by the level of education of students. This finding is inconsistent

with what Belanger and Thornton (2013) reported. Belanger and Thornton found that most of the

students who completed MOOCs were students who held a bachelors degree or higher.

The current study found that the completion of MOOCs is not significantly predicted by

gender. However, the study conducted by Latanich, Nonis, and Hudson (2001) revealed a

different finding. The authors reported that gender is found to be a factor that significantly

influences students ability to complete distance courses.

10
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 11

Limitation

Every study has limitations, no matter how well it is conducted. This study has some

limitations that might affect the outcome. One of the limitations is the high percentage of missing

data. The data used in this study had about 65% missing data, which occurred because of

inaccurate or incomplete data. Another limitation is that this study only examined four potential

predictors of MOOC completion. Future research may consider adding more independent

variables, such as age, employment status, and motivation. The final limitation of this study is

that this study only applied the quantitative method to collect and analyze the data. As the

MOOCs topic is still considered a phenomenon, future qualitative research is suggested to

provide in-depth details which will help researchers understand why students do not complete

MOOCs.

11
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 12

References

Balch, T. (2013). About MOOC completion rates: The importance of student investment [Web

log post]. Retrieved from http://augmentedtrader.wordpress.com/2013/01/06/about-

mooc-completion-rates-the-importance-of-investment/

Belanger, Y., & Thornton, J. (2013). Bioelectricity: A quantitative approach Duke Universitys

first MOOC. Retrieved from http://hdl.handle.net/10161/6216

Coursera. (2015). Basic participation information. Retrieved from www.coursera.org

Clow, D. (2013). MOOCs and the funnel of participation. Proc. of the Third International

Conference on Learning Analytics and Knowledge, 185189. New York, NY: ACM.

doi:10.1145/2460296.2460332

Evans, B. J., Baker, R. B., & Dee, T. S. (2016). Persistence patterns in massive open online

courses (MOOCs). Journal of Higher Education, 87(2), 206242. Retrieved from

http://www.ulib.niu.edu:2329/docview/1826520781?accountid=12846

Jordan, K. (2013). MOOC completion rates: The data [Web log post]. Retrieved from

http://www.katyjordan.com/MOOCproject.html

Jordan, K. (2015). Massive open online course completion rates revisited: Assessment, length

and attrition. The International Review of Research in Open and Distributed Learning,

16(3), pppp.

Kolowich, S. (2012). The MOOC survivors: edX explores demographics of most persistent

MOOC students. Inside Higher Education. Retrieved from http://www.insidehighered

.com/news/2012/09/12/edx-explores-demographics-most- persistent-mooc-students

12
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 13

Latanich, G., Nonis, S. A., & Hudson, G. I. (2001). A profile of todays distance learners: An

investigation of demographic and individual difference variables of distance and non-

distance learners. Journal of Marketing for Higher Education, 11(3), 116.

McAuley, A., Stewart, B., Siemens, G., & Cormier, D. (2010). The MOOC model for digital

practice. Retrieved from http://www.elearnspace.org/Articles/MOOC_Final.pdf

Pappano, L. (2012). The year of the MOOC. The New York Times, pp.

Watters, A. (2012). 6.003z: A learner-created MOOC spins out of MITx [Web log post].

Retrieved from

13
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 14

Appendix A

14
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 15

15
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 16

16
FACTORS THAT PREDICT LEARNERS COMPLETION OF MOOCS 17

17