Monitoring students activities in CS courses

Vincenza Carchiolo, Alessandro Longheu, Marialaura Previti, Georgia Fichera

Dipartimento di Ingegneria Elettrica, Elettronica ed Informatica (DIEEI)
University of Catania
Catania, Italy
e-mail: vincenza.carchiolo@dieei.unict.it, alessandro.longheu@dieei.unict.it, ml.previti@gmail.com,
georgia.fichera@dieei.unict.it

Abstract— The act of learning is becoming more and more
sophisticated, thanks to several models and tools available today,
like MOOCs, personalization, social networks, Web 2.0,
gamification and others. This vast landscape leads to a huge
amount of datasets thus the Big Data paradigm is also being
adopted. Joined with Big Data is the emergency of extracting
information about students’ learning interactions and return
them to teachers (learning analytics). According to this scenario,
in this paper we introduce the e-learning platform currently used
at University of Catania. The platform also collects data about all
these students activities acting as a Big Data base for further
learning analytics.
Keywords—Social networking, complex networks, Big Data,
web application, e-learning, learning analytics
I. INTRODUCTION
The shifting of classical centralized e-learning towards the
landscape of social networks and Semantic Web is radically
changing the way students approach the learning process [1].
Indeed, in the past they simply retrieved previously stored
learning materials through a web interface, then, significant
evolutions as Massive Online Open Courses (MOOC) [2] and
personalization for tailoring learning to students' needs and
abilities [3][4] occurred.
Nowadays it is possible to exploit several collaborative
tools and approaches to promote a more active role for
learners, including standard and well-known chats, forums,
wikis, as well as the emerging Web 2.0 technologies used to
endorse pervasive knowledge [5].
Other tools and models also contribute to compose the still
called “e-learning” framework. For instance the Gamification
[6] is the use of game elements and techniques in learning (but
even other) contexts to engage students in problem solving; its
use promotes creative thinking, act as a motivational tool
through frequent rewards and allows to integrate
disadvantaged target groups. In addition, the adoption of cloud
computing as an underlying storage increase cost efficiency,
reliability and ease of access [7][8]. Last but not least,
exploitation of social network tools and models [9][10][11]
allows students to participate more actively in knowledge
creation and dissemination. Indeed, learning stakeholders (e.g.
schools and universities) can benefit from connecting with
existing and potential students through various social
networks, tracking their comments, opinions and suggestions
in turn approaching them with a more personal touch. Finally,
leveraging complex network analysis [12][13] provide better
understanding of learning mechanisms.
The plethora of possibilities and tools described so far
consequently leads to a huge amount of datasets too large for
standard database systems to handle; they are gathered from
student, teachers and educational institutions, and falls within
the umbrella of Big Data paradigm [14][15].
Joined with Big Data is the emergency of extracting
information from these data about students’ learning
interactions and return them to teachers; Learning analytics is
the cutting-edge imperative [16][17].
The main goal of learning analytics is to understand and
improve the whole learning process, its outcomes and the
environments in which it occurs; to do this, measurement,
collection, analysis and reporting of data about students and
their contexts is performed, and expertise from different
disciplines as educational data mining and predictive
modelling are combined [18]. In particular, learning analytics
can provide significant contributions to:
1. improve and assure learning quality; the intensive use of
predictive models and/or social network analysis is indeed
one of the main activities being undertaken by institutions.
Moreover, providing students with successful learner
profile promotes the right behaviors.
2. boost retention rates; it can help stakeholders to tackle high
attrition rates and identify struggling students earlier;
address this problem will prevent wasted expense for the
institutions.
3. assess and act upon differential outcomes among students;
they increase awareness about their own learning, their
progress (even in comparison with colleagues, acting as a
competitive factor), what they need to do to get
educational goals.
4. endorse the development of adaptive learning; exploiting
students data as their prior qualifications and those
accumulated during their educational progress allow to
tailor the system to their needs.
Two non-academic issues but still relevant for an effective
learning analytics must be addressed. The former is the fact
that the main motivation of its worldwide adoption comes
from an increasingly stretched university budgets hence the
need for evidence-based prioritization of spending is often
mandatory. Moreover, students also want to check that their
fees are being spent appropriately (as consumers).
The latter, but sometimes critical, is the question of
privacy, indeed the use of students data for learning analytics
is not generally allowed on a tacit approval basis, rather
explicit and personal authorization is required (this occurs in
Italy according to the legislative decree no. 196/2003). In
many cases, the organizational culture affects the acceptance
of learning analytics, therefore ethical and privacy issues are
addressed in a variety of ways, depending upon the
institutional context.
In this paper we introduce the e-learning platform
currently used at University of Catania. The platform is
mainly accessed through a web portal (named
STUDIUM.UniCT) where students can access learning
materials, as well as being involved as active players by
performing tests, answering questionnaires to detect their
moods and needs, exploiting social-based tools as described
previously. Additional modules can be easily connected to the
platform to enhance its features (e.g. Facebook and Twitter
plug-ins, gamification support and so on). The platform also
collects data about all these students activities acting as a Big
Data base for further learning analytics.
The rest of the paper is organized as follows. In section 2
we introduce a social-based description of roles and
functionalities available in our e-learning system. Section 3
introduces the system architecture, whereas section 4 presents
some information extracted from system data to show learning
analytics assessment. Finally in section 5 we present our
conclusion and further works.
II. SOCIAL NETWORK BASED MODEL
A first step in the creation of our system was the definition
of user profiles which have different access permission and
perform different activities. The profiles we decided to
implement are student, teacher and coordinator.
The student is everyone who attends, for various reasons,
university courses. The main activities that a student can
perform through the social network, after authorizing the
processing of personal data entered in the system (according to
the legislative decree 196/2003 cited in the introduction) are:
• record his presence to each lesson via mobile application;
• download the educational materials recommended them by
teachers notifications;
• read notices posted by teachers;
• at the beginning of each course, play educational game that
allow teachers to understand what are the student's basic
knowledge and suggest him to acquire other useful
knowledge for a good comprehension of teaching;
• chat with other users of the system, both students and
teachers, for educational purposes or other;
• perform the homework assigned by teachers at the end of
each didactic unit;
• write in educational forum in order to obtain information
that could not be found in other parts of the system;
• share educational materials with other students, such as
notes, theses and other documents;
• at the end of each course, fill anonymously the feedback
questionnaire concerning a teacher and its course.
Several categories of users can access the system with
teacher profile: professors, assistants and tutors. The
professors perform the teaching in the classical sense by
providing theoretical concepts to students. The assistants
conduct exercises and help students to put into practice the
concepts provided by the teacher. Finally the tutors (typically
students on the verge of complete their university studies) help
a group of freshmen assigned to him to bridge the gap between
high school and university with advice not necessarily
academic.
These users, in order to best carry out their tasks, need to
monitor student progress in order to bring as many as possible
to graduation without excessive delay. To do this, they can:
• create notices containing courses information;
• provide educational materials;
• chat with all users of the system;
• make themselves available to provide clarification on
request to students at certain times;
• read the statistics on courses attendance, performance in
games, homework scores and questionnaires results to
identify students' problems or deficiencies and then to re-
explain certain topics to the whole class or to a subset of
this through individual or group meetings.
Coordinators are the professors who, in addition to
performing the above mentioned activities, also:
• create the academic calendar, i.e. the lesson timetable, the
examinations dates, the graduation session calendars, etc.;
• supervise activities of the other teachers;
• collect the students' reports on possible improvements of
the courses.
The interactions between user categories are shown in fig.
1.
The main task of our social network is to move many of
the activities that were previously carried out by traditional
systems, often on paper or on several web pages, into a single
environment with user friendly interface that encourages the
users of system belonging to different categories to interact
with each other both with the sharing of material and with
direct communication and to stay up to date by enabling the
notification system. Students, on request, can in fact receive
an alert for publication of material or news by one or more
teachers, while teachers can set the minimum thresholds for
gaming or homework scores below which receive a warning
for the drop in performance for a specific topic.
In addition to classic teaching methods, the system allows
students to help each other through the sharing of material, the
interaction on forums and the competition in group games
through which is generated the ranking of the top 10 for each
course of study.
For example, a very popular game among students is the
academic trivial pursuit: a game similar to the famous board
game, but the categories in which students can choose the
questions are the subjects of the course of study. Questions or
exercises proposed to conquer the game board, and
consequently to win the game, are inserted progressively by
the teachers during the academic year and cover the part of the
program carried out until the week before. This encourages
students to study constantly.
In addition to user interactions (marked with continuous
lines) there is also a part of the system involved in the
collection of data (marked with the dashed lines) in order to
analyze and extract information useful to the improvement of
the system of teaching.

Fig. 1. Diagram of interactions among social network user categories.

III. ARCHITECTURE OVERVIEW Each of these modules enters the information in a database.
Its record are periodically examined by learning analytic
In our platform (fig. 3), the above described features are
engine and the results of this analysis are available for users in
divided into modules, each of which is represented by a
a summary on dashboard that allows teachers to change the
different software:
educational activities based on the results obtained.
• STUDIUM.UniCT (fig. 2), a university educational In learning analytic engine, methods of analysis [19] we
activities support portal, based on the e-learning open used are:
source platform Dokeos that has been integrated with the
• content analytics, i.e. analysis of documents posted by
information systems of the University of Catania and
students, as homework, questionnaires and shared material,
enriched with additional features. It is a tool that can be
using automatic correctors, scoring detectors and statistics
used by teachers for the publication of teaching materials,
creators;
lecture notes and related information to its teachings and
• discourse analytics, i.e. the analysis of questions and
has useful tools to support interactive communication
answers written in the forum, chat content and notes shared
between students and teachers.
in order to understand what are the students' attitudes to
• Presence detector, software consists of two Android apps,
connect them with educational experiences that are truly
one for teachers and one for students, which, using the
meaningful to them;
mobile Bluetooth, allows the teacher of the course to
• social network analytics, i.e. the analysis of the interaction
automatically compile attendance register in csv format
between nodes to identify interventions that are likely to
and send it to the platform database.
increase a network’s potential and support the learning of
• Social integration, plugin allows, if enabled by the user, to
its users, e.g. with the forced connection of disconnected
receive notifications regarding the platform directly to
students with student groups with high levels of academic
Facebook and / or Twitter user's account.
performance.
 Fig. 2. Studium Fundamental of programming course page.

IV. MONITORING STUDENTS ACTIVITIES

Our system has been tested in the Electric, Electronics and
Computer Engineering Department University of Catania,
Italy. The students were divided into three classes for each
course, so each teacher can see how his work is carried out
with respect to other classes, other courses or other (past)
academic years.
In particular, we focus on the results of the course
“Fundamental of programming”. In this course the professor
submitted a start test to check student competences, to
organize exercises and slides online. Exercises and slides are
updated continually.
Each student is monitored with online test, exercises and
survey. At the end of course, the learning analytics engine
shown in fig. 3 provides an evaluation for each course that can
be displayed on the dashboard. This evaluation consists of a
set of items each one rated with an integer numerical value
Fig. 3. Platform architecture. ranging from zero (low) to five (high), representing how much
that item is perceived by students; ratings come from the
analysis of data provided by students. Items we considered
are:
1. the overall study load (to what extent students consider to
be difficult that course);
Both graphs are available on a per-class or per-course
basis.
In fig. 6 our System show the presences trend of three
Fundamental of programming courses.

2. the quality of final exam (organization, difficulty…);

3. to what extent topics in final exams were properly
addressed by suggested learning material;
4. the overall usefulness of online available learning
materials (slides, exercises...);
5. the overall satisfaction level about the course;
6. the usefulness of final exam groundwork (as classroom
exercises performed with assistants);
7. teacher understandability, helpfulness and expertise;
Fig. 6. Presences trend.
The results of “Fundamental of programming” course are
represented in fig. 3 as the percentage of students associated
The information that can be gathered from presence is
with a given rate for each item (from 1 to 7); for instance, in
about student interest and motivation, even if sometimes a
item 4 most students found that online learning material was
significant delta in a course presence is due to some
useful for their success in the course. Conversely, the lowest
intermediate exams in other courses attended by the same
positive percentages concern item 6, therefore the teacher
students (hence they prefer to focus on the course with
should improve classroom exercise to strengthen his students’
forthcoming exams); for instance, in fig. 5 class 3 participation
skills.
has a negative slope around day 14. In fig. 7 we show the
percentages of students that passed positively the final course
exam with respect to all attendees (black bars) or with respect
to enrolled students, named “cohort” in our system (dotted
bars).

Fig. 4. Fundamental of programming course evaluation.

Another graphic representation can be shown in the

dashboard, for instance in fig. 5 a teacher decides to show only
Fig. 7. Exam results.
positive values, 4 and 4+5.
As a final example of learning analytics, in Fig. 7 the
relationship between attendance and final grade is shown. All
students with zero did not take the final exam; when the grade
ranges from 12 to 18 (minimum value to positively pass the
course final exam) the student needs more support to improve
his knowledge. In this case the professor, tutor and assistance
has to dedicate his time to recover these students. Grade from
18 to 30 (maximum) are acceptable to get credit for that
discipline. In fig. 8 in particular, most students positively
concluded the course with a final grade of 23 or 28.

Fig. 5. Fundamental of programming course positive values.


Fig. 8. Presence vs results.
V. CONCLUSIONS AND FUTURE WORKS
In this paper we described the e-learning platform
currently used at University of Catania, contextualizing it in a
landscape that includes several tools and models, as social
networking approach web 2.0 and gamification. Our platform
collects data about all students activities acting as a Big Data
base for further learning analytics we also briefly illustrated.
Further investigation is needed to make our system an
effective tool for learning analytics, including the adoption of
predictive models and the exploitation of complex network
analysis, with the aim of providing students and teachers with
several information, e.g. about performance prediction,
attrition risk detection, behavior detection and intelligent
feedback and student skill estimation.
REFERENCES
[1] T. J. McGill and J.E. Klobas, "A task–technology fit view of
learning management system impact". Computers & Education,
52(2), pp. 496–508, 2009.
[2] A. Margaryan, M. Bianco, and A. Littlejohn, "Instructional
quality of Massive Open Online Courses (MOOCs)". Computers
& Education, Volume 80, January 2015, pp. 77-83, ISSN 0360-
1315, http://dx.doi.org/10.1016/j.compedu.2014.08.005.
[3] V. Carchiolo, D. Correnti, A. Longheu, M. Malgeri, and G.
Mangioni "Exploiting trust into e-learning: adding reliability to
learning paths" Intl Journal of Technology Enhanced Learning
(IJTEL) Vol.1 No. 4 - ISSN (Online): 1753-5263 ISSN (Print):
1753-5255,UK, pp. 253-265, 2009, Inderscience publishers.
[4] V. Carchiolo, A. Longheu, M. Malgeri, "Reliable peers and
useful resources: Searching for the best personalised learning
path in a trust- and recommendation-aware environment".
Information Sciences, Volume 180, Issue 10, pp. 1893-1907, 15
May 2010, ISSN 0020-0255,
http://dx.doi.org/10.1016/j.ins.2009.12.023.
[5] M. Anshari, Y. Alas, and L. S. Guan, - "Developing online
learning resources: Big data, social networks, and cloud
computing to support pervasive knowledge" - Education and
Information Technologies, pp. 1-15, 2015
http://dx.doi.org/10.1007/s10639-015-9407-3.
[6] A. Domínguez, et al., "Gamifying learning experiences:
Practical implications and outcomes." Computers & Education,
Volume 63, pp. 380-392, April 2013, ISSN 0360-1315,
http://dx.doi.org/10.1016/j.compedu.2012.12.020.
[7] M. Bahrami and G. Riahi "E-learning Systems Based on Cloud
Computing: A Review" Proceedings of SCSE'15 , Volume 62,
pp. 352-359, 2015, ISSN 1877-0509,
http://dx.doi.org/10.1016/j.procs.2015.08.415.
[8] N. Radhakrishnan, N. Poorna Chelvan, and D. Ramkumar,
"Utilization of cloud computing in E-learning systems." Cloud
Computing Technologies, Applications and Management
(ICCCTAM), 2012 International Conference on, Dubai, pp. 208-
213, 2012. doi: 10.1109/ICCCTAM.2012.6488100
[9] C. Megele, "eABLE: embedding social media in academic
curriculum as a learning and assessment strategy to enhance
students learning and e-professionalism". Innovations in
Education and Teaching International, 52 (4). pp. 414-425.
ISSN 1470-3297, 2015.
[10] Jin Mao "Social media for learning: A mixed methods study on
high school students’ technology affordances and perspectives."
Computers in Human Behavior, Volume 33, pp. 213-223, April
2014, ISSN 0747-5632,
http://dx.doi.org/10.1016/j.chb.2014.01.002.
[11] C. Limongelli, F. Sciarrone, M. Temperini "A social network-
based teacher model to support course construction." Computers
in Human Behavior, Volume 51, Part B, pp. 1077-1085, October
2015, ISSN 0747-5632,
http://dx.doi.org/10.1016/j.chb.2015.03.038.
[12] J. P. Scott. "Social Network Analysis: A Handbook." SAGE
Publications, 2nd edition, 2000.
[13] S. Wasserman and K. Faust. "Social Network Analysis: Methods
and Applications." Cambridge University Press, 1994.
[14] Saptarshi Ray, “Big Data in Education”, Gravity, the Great
Lakes Magazine, pp. 8-10, 2013.
[15] Manolis Mavrikis, Patricia Charlton and Demetra Katsifli, “The
Potential of Learning Analytics and Big Data”,
http://www.ariadne.ac.uk/issue71/charlton-etal.
[16] K. Sin and L. Muthu "Application of big data in education data
mining and learning analytics – a literature review" - ICTACT
journal on soft computing: special issue on soft computing
models for big data, july 2015, volume: 05, issue: 04
[17] C. Gunn, J. McDonald, C. Donald, J. Milne, M. Nichols, and E.
Heinrich, "A practitioner’s guide to learning analytics". In T.
Reiners, B. R. von Konsky, D. Gibson, V. Chang, L. Irving, &
K. Clarke (Eds.), Globally connected, digitally enabled.
Proceedings ascilite 2015, pp. 672-675, 2015. Perth, Australia.
[18] R. Ferguson, "Learning analytics: drivers, developments and
challenges" International Journal of Technology Enhanced
Learning, 4(5/6) pp. 304–317, 2012.
[19] R. Ferguson and S. Buckingham Shum, "Social learning
analytics: five approaches." Proceedings of the 2nd international
conference on learning analytics and knowledge. ACM, pp. 23-
33, 2012.