30/04/2019

Self-Reflection on the Suitability of

Adaptive Self-Regulated e-Learning to

Research Cycle Conference 2014

Vocational Training Based on an
Experimental Study in Heat Pump System
Installation
Aurora Dimache, Project Manager
Assistant researcher: Lisa Winter
Department of Mechanical Engineering
1
Why This Study?

30/04/2019
• e-Learning suitable for

Research Cycle Conference 2014

third level education

• Not enough evidence of

suitability to vocational
education and training, in
particular heat pump
system installation

2
Research Question

30/04/2019
Is adaptive self-regulated e-learning a viable alternative for
practically-oriented vocational training?

Research Cycle Conference 2014

3
Objectives of the Study

30/04/2019
Research Cycle Conference 2014
Assess the influence of different learning systems on the
level of knowledge gained by the learners
(effectiveness)

Identify the users’ opinion on their learning experience,

the influence of the type of learning system on the
users’ time, effort and comfort (efficiency)

4
Measures of e-Learning Courses

30/04/2019
• Learning achievements/outcomes or the effectiveness of e-
learning
• Student satisfaction

Research Cycle Conference 2014

5
e-Learning vs. Face-to-Face

30/04/2019
• No difference in effectiveness

Research Cycle Conference 2014

• e-Learning
• Better for competences at
the bottom of Bloom’s
pyramid
• Self-regulated learning
encouraged
• Less interpersonal interaction

6
Skill-based (Personalised)
Learning

30/04/2019
Research Cycle Conference 2014
• Competencies and skills that already exist do not have to be
taught

• ⇒ Skill-based learning systems are adaptive

• Advantage:
• greater efficiency in learning
• increased amount of time saved in learning the content

7
The INNOVRET Approach

30/04/2019
• Combination of self-regulated learning and skill-based
learning – Competence-based Knowledge Space Theory
(CbKST)

Research Cycle Conference 2014

• Adaptive
system

8
Research Methodology

30/04/2019
• Empirical study organised in two parts:
• An experimental study

Research Cycle Conference 2014

• Pre-test (5 students in GMIT)
• Experiment with target group (14 installers in Galway area)
• A survey

• Think aloud protocol (or protocol analysis technique) for

triangulation purposes

9
Data Collection Methods

30/04/2019
• Non-participant overt
observation

Research Cycle Conference 2014

• Semi-structured
interview

• Questionnaires

• Screen and voice

recording
10
 The Adaptive System

11

Research Cycle Conference 2014 30/04/2019

Results – Pre-Test

30/04/2019
• The results of the knowledge tests were inconclusive due to
limited learning time
• No tests in the following phase

Research Cycle Conference 2014

12
Results – Experiment with
Target Group

30/04/2019
Research Cycle Conference 2014
The IT skills of the installers play a very
important role in the way they perceive
the system and the whole learning
experience.

13
 Results – Survey

30/04/2019
The iterative learning process (Q1), the awareness support
(Q2), and the guidance support (Q4)

Research Cycle Conference 2014

Q1 Q2 Q4
6 6 10
No. of respondents

No. of respondents
No. of respondents

8
4 4 6
4
2 2
2
0 0 0
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5

CbKST Moodle CbKST Moodle CbKST Moodle

14
Results – Survey

30/04/2019
Was the approach limiting (Q3) or stressful (Q5)?

Research Cycle Conference 2014

Q3 Q5
6 6

5 5
No. of respondents

No. of respondents
4 4

3 3

2 2

1 1

0 0
1 2 3 4 5 1 2 3 4 5

CbKST Moodle CbKST Moodle

15
Results – Survey

30/04/2019
The participants’ enjoyment (Q6) and the perceived learning
success (Q7)

Research Cycle Conference 2014

Q6 Q7
4.5 7
4 6
3.5
No. of respondents

No. of respondents
5
3
2.5 4
2 3
1.5
2
1
0.5 1
0 0
1 2 3 4 5 1 2 3 4 5

CbKST Moodle CbKST Moodle

16
Results – Survey

30/04/2019
The user interface (Q8) and the content quality (Q10)
Would the participants use the system in the future (Q9)?

Research Cycle Conference 2014

Q8 Q9 Q10
7 7 4.5
6 4
6
No. of respondents

No. of respondents
3.5
No. of respondents

5 5 3
4 4 2.5
3 2
3
1.5
2
2 1
1 0.5
1
0 0
1 2 3 0 4 5 1 2 3 4 5
1 2 3 4 5
CbKST Moodle CbKST Moodle
CbKST Moodle

17
Results – Think Aloud Protocol

30/04/2019
Navigation problems

• The users didn’t understand the menu

Research Cycle Conference 2014

• The users found inconsistencies in the design of
the learning objects
• The user with very limited IT skills wasn’t
able to go back to the Internet browser once a
learning object was open

Comprehensiveness problems

• Some of the terms used are not familiar to the

users (e.g. learning resources, learning iteration) 18
Conclusions

30/04/2019
• No conclusion can be drawn in relation to the
effectiveness of the adaptive system

Research Cycle Conference 2014

• The learning experience is very much influenced by the
IT skills of the learner – in contrast to other studies
carried out at third level education, where IT skills are
taken for granted

19
Evaluation of Research

30/04/2019
• The research methodology adopted for the study is robust
and can assure the validity of the study

Research Cycle Conference 2014

• The generalisability of the study is limited due to a
relatively low number of participants to the experiment

• This was a pilot study ⇒ more experiments are needed

20
 Thank you for your attention!

21

Research Cycle Conference 2014 30/04/2019

Graphics References

30/04/2019
• Wikimedia Commons, free images, available at
http://commons.wikimedia.org/wiki/Main_Page
• Bloom’s Learning Taxonomy, available at http://ci484-

Research Cycle Conference 2014

learning-
technologies.wikispaces.com/Blooms's+Learning+Taxono
my

22