Available online at www.sciencedirect.

com

ScienceDirect
Available online at www.sciencedirect.com
Procedia Computer Science 00 (2019) 000–000
www.elsevier.com/locate/procedia
ScienceDirect
Procedia Computer Science 157 (2019) 266–273

4th International Conference on Computer Science and Computational Intelligence 2019

(ICCSCI), 12–13 September 2019

Analysis of the Indicator's Performance to Predict Indonesian

Teacher Engagement Index (ITEI) using Artificial Neural Networks
Sucianna Ghadati Rabihaa, Sasmokob,*
a
Information System Department, Binus Online Learning, Research Interest Group in Educational Technology, Bina Nusantara University,
Jakarta 11480, Indonesia
b
Primary Teacher Education Department, Faculty Humanities, Research Interest Group in Educational Technology, Bina Nusantara University,
Jakarta 11480, Indonesia

Abstract

Research related to the Indonesian Teacher Engagement Index (ITEI) is still being developed to get better predictive accuracy. In
this study we conducted an analysis using the Neural Network approach to determine the performance of each dimension obtained
from the ITEI database. Root Mean Square Errors values generated when using variables in dimensions related to Nationality
Character are quite low, which is 0.151, with a correlation of 0.906 and execution time of 13s. In addition, the Nationality Character
Dimension obtained excellent performance predictions when combined with the Positive Education Dimension. The Square Errors
Root Mean value can be reduced to 0.102 with 8s execution time. The results of this analysis will be used as the basis for developing
ITEI applications in the future so that the resulting Teacher Profiling can be used as a basis for policies and strategies to improve
teacher performance and engagement in the school environment.
© 2019 The Authors. Published by Elsevier B.V.
© 2019 The Authors. Published by Elsevier B.V.
This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
This is an open
Peer-review access
under article underof
responsibility thethe
CCscientific
BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
committee of the 4th International Conference on Computer Science and
Peer-review
Computationalunder responsibility
Intelligence 2019. of the scientific committee of the 4th International Conference on Computer Science and
Computational Intelligence 2019
Keywords: Neural Network; Teacher Engagement; Performance;

* Corresponding author. Tel.: +62-811-8891-777.

E-mail address: sasmoko@binus.edu

1877-0509 © 2019 The Authors. Published by Elsevier B.V.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review under responsibility of the scientific committee of the 4th International Conference on Computer Science and Computational
Intelligence 2019

1877-0509 © 2019 The Authors. Published by Elsevier B.V.

This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Peer-review under responsibility of the scientific committee of the 4th International Conference on Computer Science and Computational
Intelligence 2019.
10.1016/j.procs.2019.08.166
 Sucianna Ghadati Rabiha et al. / Procedia Computer Science 157 (2019) 266–273 267
2 Rabiha et al. / Procedia Computer Science 00 (2019) 000–000

1. Introduction

To achieve high student involvement in the education process, a high level of teacher involvement is needed.
Teacher involvement is very closely related to internal teacher factors, namely, as an energetic person with a high level
of self-identification with his profession1.
The quality of education is one of the responsibilities of a teacher so that the Teacher becomes a focus for the
development of this System. Since 2015, an evaluation tool called the Indonesian Teacher Engagement Index (ITEI)2
has been developed through a grant by the Indonesian Ministry of Research and Higher Education (Kementerian Riset
dan Pendidikan Tinggi, Kemenristekdikti) partnered with the Directorate General of Teachers and Educational
Manpower (Direktorat Jenderal Guru dan Tenaga Kependidikan), the Indonesian Ministry of Education and Culture
(Kementerian Pendidikan dan Kebudayaan Republik Indonesia), and the Executive Board of Indonesian Teachers
Association (Pengurus Besar Persatuan Guru Republik Indonesia, PB-PGRI).
In the previous study, an instrument was formed in the form of a Teaching Engagement Index designed to assist
teachers in self-detection personally through self-diagnostic. So that later the results of the self-diagnosis will be further
developed systemically so that it can help the central government in the decision-making process specifically in the
education sector in certain regions3.
Neural Network is a complex nonlinear network system that refers to a computer software system that can simulate
the structure and function of a biological brain4. The neural network structure consists of a large number of processing
units similar to neurons that are determined by the basic processing unit and its interconnection methods. ANN is a
network three layer that implements supervised learning algorithms to classify data input (for example, number of
participants, region of origin, level of education, and number of values from each quiz), and produces specific output
categories (for example, pass or fail)5.
Turhan et al.6 explains the possibility of predicting the involvement of school students by using artificial neural
networks based on demographic, family and school variables In other studies related to predictive accuracy, neural
networks have very high values with an accuracy of 98.3%. Neural networks have far better results than all other
classifiers in classifying students into predictable categories of success and failure5 .
Previous research related to Neural Network showed that the ANN models developed were able to detect driving
behavior based on driver involvement in calls, SMS, and interaction with passengers with accuracy of 99.5%, 98.1%,
and 99.8% respectively7. These results indicate that the selected driving performance attributes prove effective in
detecting secondary tasks related to driving behavior.
Research related to the Teacher Engagement is still being developed to Increase the accuracy of predictions based
on data that has been collected. In this study the Neural Network is used as an approach in conducting analysis to
determine the performance of each dimension obtained from the ITEI database, so that we can see which dimensions
have a high contribution as predictors of ITEI buildings. The thing that becomes the basis of the application of Neural
Network is the ability that is owned in detecting all possible interactions that occur between predictor variables. In
addition, Neural Networks can detect completely and clearly the complex nonlinear relationships between independent
and dependent variables 8.
By knowing the performance of each dimension, the results of this analysis will be used as the basis for developing
ITEI applications in the future as a predictor that can be used by educational institutions so that the resulting Teacher
Profiling can be used as a basis for strategies and policies to improve teacher performance and engagement in the
environment school.

2. Literature Review

2.1 Indonesian Teacher Engagement Index (ITEI)

The Teacher Engagement concept was created to describe the profile of teachers in Indonesia who have a high
complexity of tasks and responsibilities. To improve the quality of education in Indonesia the role of teachers is very
much needed especially in its involvement in the world of education in accordance with the Indonesian context and
absorbing all positive aspects as a whole2.
268 Sucianna Ghadati Rabiha et al. / Procedia Computer Science 157 (2019) 266–273
Rabiha et al. / Procedia Computer Science 00 (2019) 000–000 3

Based on previous research, it has been proven that the ITEI Instrument is valid and reliable in measuring teacher
involvement in the Indonesian context. So that later it can be used as a tool to conduct an integrated assessment so that
the mapping of the conditions of teacher involvement in each individual, school, sub-district, district, province and
even nationally can be described in detail9.
The concept of the Indonesian Teacher Engagement Index (ITEI) is a depiction of the various conditions of the
teacher psychologically related to how they understand participation in building positive education, how to show good
performance, how to understand competencies that support, have national character as Indonesian Characteristics and
are able shows the involvement of nationalism leadership2.

2.2 Neural Network

Neural network is one technique that is often used in exploring educational data. The ability of a neural network is
to detect all possible interactions between predictor variables. In addition, the Neural Network can detect completely
and clearly the complex nonlinear relationships between independent and dependent variables8.
The distinctive features of the ANN model are the input layer, hidden layer, and output layer that are interconnected
and form vertices. Because of its ability to filter and send information in a controlled manner, the Neural Network is
used as a prediction model built to classify data stored in memory5. The knowledge-based ANN model was developed
into two parts, training and testing. The training process is a learning process for the database that is owned, while the
testing process is a training process for the proposed validation model using a database that was not involved in the
previous training process10. The Multi-Layer Perceptron (MLP) training algorithm is used to train neural networks.
Cross validation is done to avoid excessive model incompatibility. The initial network is examined using 1-10 hidden
nodes, because there is no heuristic way to determine the number of hidden nodes in a neural network. To measure the
accuracy of the model used the Root Mean Square Error (RMSE) and this has been applied in more than ten
configurations11.
The performance of ANN has been a concern of many researchers regarding marketing issues, including consumer
behavior. The flexibility of the Artificial Neural Network methodology and its usefulness in various fields of research
caused the application of ANN to increase12. By providing a dataset, Neural Network can study data with learning
algorithms. And through backpropagation, the Neural Network forms a mapping between the desired input and output
from the training set by changing connections in the network.

2.3 Related research

Paliwal et al.13 in his study explained that a new approach was proposed to interpret the importance of relatively
independent variables a new approach is needed in neural networks and a comparison with the heavy connection
approach. Neural networks and regression techniques are used interchangeably for prediction problems. In this study
it was found that the performance of the neural network approach was better than the other methods presented in this
study.
In many practical problems now the number of explanatory variables can be very large. Someone often learns
optimal predictors by using approaches in predicting data from each input variable, important variables taking into
account the compatibility obtained14.
In order to reduce the problem of instability inherent in the application of methods to determine the contribution of
predictor variables in the NN model, Ona et al.15 presents an approach The applied method determines the difference
in the percentage of interest set for the variable, because there is no unique subset of the most important variables the
reason is the inter-correlation variable, and this interest is limited by individual metrics. So that expert opinion is
needed to evaluate which methods can show important values that can be applied to get the expected results.
Other studies related to predictions of student performance using Artificial Neural Network Model,
Backpropagation algorithm is used as an advanced feed for training on this model. Student registration records are the
variables that will be applied to the modelling. This study shows the potential of artificial neural networks to predict
student performance. The results obtained from the testing of the overall model are 84.6%16.
 Sucianna Ghadati Rabiha et al. / Procedia Computer Science 157 (2019) 266–273 269
4 Rabiha et al. / Procedia Computer Science 00 (2019) 000–000

3. Research Method

3.1 Research Model

Fig. 1. Research Model Using Neural Network

The research model used in figure 1 is the stage carried out in the process of determining performance indicators
from each dimension. Data sets obtained from the ITEI application are then reprocessed and grouped into each
dimension. Furthermore, the data is divided into two categories, namely Training Data and Data Testing.

3.2 Data Collection Technique

In this study data collection was carried out using a simple sampling technique. Where sampling is done randomly
so as to provide equal opportunities for each element (member) of the population to be selected as members of the
sample.
Sampling is obtained from the extraction process of data obtained from the Index Statistical Value Calculation using
the ITEI Application. The ITEI application is a tool used in conducting surveys of Educational institutions. Where each
teacher must participate by answering the questions contained in the ITEI application 3. From the results of a survey
conducted randomly in 34 provinces in Indonesia, data obtained as many as 10645 records, consisting of various levels
of schools ranging from kindergarten to high school or vocational school. Where the ITEI instrument consists of 44
items, which are grouped into 6 dimensions and 22 indicators.

3.3 Data Analysis

This study uses the Neural Network approach. Based on the initial testing, the model applied in the Neural Network
method to the ITEI sample data uses the following parameters, Learning rate with a value of 0.3, momentum with a
value of 0.5, and the number of hidden layers: the number of neurons in layer 1 is similar 6 and Neurons in layer 2 are
also equal to 6. Because in these conditions the resulting RMSE value is quite good..
In this study the analysis used is based on several variables including the input layer of the proposed ANN model.
Instrument used consists of 6 dimensions and 22 indicators. Later these parameters will be grouped according to their
dimensions. The value of each dimension illustrates the perception of several indicators in the form of items related to
the index value of the ITEI application. The average index value associated with the dimensions of Positive Psychology
is shown by Dimension 1. The average index value related to the dimensions of Positive Education is shown by
Dimension 2. Furthermore, for Dimension 3 shows a picture related to the average index value in the Teacher
Perfomance dimension. For dimensions related to the Nationality dimension are dimensions 4 and 5, where the average
index value associated with the dimensions of Nationality Character is shown by dimension 4 and the average index
value related to the Nationalism dimension Leadership Engagement is shown by dimensions 5. While Dimension 6 , is
a description of the average index values related to Teacher Competence dimensions3.
270 Sucianna Ghadati Rabiha et al. / Procedia Computer Science 157 (2019) 266–273
Rabiha et al. / Procedia Computer Science 00 (2019) 000–000 5

4. Result and Discussion

4.1 Data Set

Artificial Neural Networks were trained using 4824 training data sets and 2353 test data sets obtained from the ITEI
application extraction process.

Fig. 2. Variable Mapping

Figure 2 describes the variable mapping of the ITEI Instrument. The X value is the Index value that describes the
value of the Teacher Engagement Index. The value of each dimension illustrates the perception of several indicators
in the form of items related to the index value of the ITEI application.
Figure 2 describes the variable mapping of the ITEI Instrument. Where X1 is the Index value which represents
Dimension 1, the Positive Psychology dimension. Dimension 1 consists of 3 indicators namely X7, X8, X9. Each
indicator is represented by several items. X7 consists of 2 items, namely B1, B2. X8 consists of 2 items B3, B4. And
X9 consists of 2 items B5, B6.
Then X2 is Dimension 2, namely the dimension of Positive Education. Consists of 3 indicators namely X10, X11,
X12. Each indicator is represented by several items. X10 consists of 2 items, namely B7, B8. X11 consists of 2 items
B9, B10. And X12 consists of 2 items B11, B12.
Then X3 is the 3rd Dimension, which is Teacher Performance dimension. Consists of 4 indicators namely X13,
X14, X15, X16. Each indicator is represented by several items. X13 consists of 2 items, namely B13, B14. X14 consists
of 2 items B15, B16. And X15 consists of 2 questions B17, B18. And X16 consists of 2 questions B19.20.
Then X4 is the 4th Dimension, namely the dimension of Nationality Character. Consists of 5 indicators namely
X17, X18, X19, X20, X21. Each indicator is represented by several items. X17 consists of 2 items, namely B21, B22.
X18 consists of 2 items B23, B24. And X19 consists of 2 questions B25, B26. X20 consists of 2 items B27, B28. And
X21 consists of 2 questions B29, B30.
Then X5 is Dimension 5, which is the dimension of Nationalism Leadership Engagement. Consists of 4 indicators
namely X22, X23, X24, X25. Each indicator is represented by several items. X22 consists of 2 items, namely B31,
B32. X23 consists of 2 items B33, B34. And X24 consists of 2 questions B35, B36. X25 consists of 2 items, namely
B37, B38.
Then X6 is the 6th Dimension, namely Teacher Competence dimension. Consists of 3 indicators, namely X26, X27,
X28. Each indicator is represented by several items. X26 consists of 2 items, namely B39, B40. X27 consists of 2
items B41, B42. And X28 consists of 2 questions B43, B44.
Furthermore, the Y value is the result of calculating the value of Teacher Engagement Index obtained using the
ITEI application.
 Sucianna Ghadati Rabiha et al. / Procedia Computer Science 157 (2019) 266–273 271
6 Rabiha et al. / Procedia Computer Science 00 (2019) 000–000

4.2 Performance Indicator

In this study we tested the performance of each dimension using the Neural Network approach. From the results of
the analysis carried out using the same test parameters, we can see that the Dimensions - X4 (Nationality Character)
gives a large contribution in the results of the Indonesian Teacher Engagement Index (ITEI).

Table 1. Performance indicator

Table. 1 Explain that the Root Mean Square Error (RMSE) value that is generated when using variables in the X4
dimension is quite low, which is 0.151 , with a correlation of 0.906 and execution time of 13s. This shows that the
resulting predictive value is also higher than the variables in other dimensions.
The comparison of Perfomance from the six Dimension can be seen in Figure 3.

Fig. 3. Graph of Performance Indicators from each Dimension (a) Dimension 1- X1, (b) Dimension 2 – X2, (c) Dimension 3 -X3,
(d) Dimension 4–X4, (e) Dimension 5 – X5, (f) Dimension 6 - X6
272 Sucianna Ghadati Rabiha et al. / Procedia Computer Science 157 (2019) 266–273
Rabiha et al. / Procedia Computer Science 00 (2019) 000–000 7

From Figure 3 we can see the performance of each indicator against the results of the index value prediction. Figure
3 (a) shows the effect given by the index value of Dimension 1-X1 on the results of the ITEI value prediction. Figure
3 (b) shows the effect given by the index value of Dimension 2-X2. Figure 3 (c) shows the effect given by the index
value of the 3-X3 Dimension. Figure 3 (c) shows the effect given by the index value of the 3-X3 Dimension. Figure 3
(d) shows the effect given by the index value of the 4-X4 Dimension. Figure 3 (e) shows the effect given by the index
value of the 5-X5 Dimension. And Figure 3 (f) shows the effect given by the index value of the 6-X6 Dimension.

Next, we do a test again by combining Performance from Dimensions X4 with other dimensions to see the effect
on predictive values.

Table 2. Performance from a combination of dimensions

The results obtained when combining the X4 Dimension with the other dimensions turned out to be quite good.
From Table 2, we can see that the X4 Dimension obtained excellent results in predicting when combined with X2
Dimensions (Positive Education). The RMSE value can be reduced to 0.102 with an execution time of 8s.

Fig. 4. Effect of Dimensions X2 on Dimensions X4

Figure 4 shows the effect of dimension X2 when combined with the X4 dimension in determining the Prediction
value of the Indonesian Teacher Engagement Index (ITEI). From the graph presented we can see that the Index values
produced from X2 and X4 have a significant influence on determining the prediction of the overall index value.

5. Conclusion

This paper presents a set of the same Neural Network architecture to determine the contribution of predictor
variables in the NN model. The database used comes from the ITEI application to determine the performance of each
 Sucianna Ghadati Rabiha et al. / Procedia Computer Science 157 (2019) 266–273 273
8 Rabiha et al. / Procedia Computer Science 00 (2019) 000–000

dimension used in determining the Index value. To see the performance of variables from each dimension, we compare
them using the Neural Network approach with several architectural models.
From the results of the analysis that has been done, it is found that Dimensions - X4 (Nationality Character) gives
a large contribution in the results of the Indonesian Teacher Engagement Index Prediction (ITEI). The RMSE value
that is generated when using variables in the X4 dimension is quite low, which is 0.151, with a correlation of 0.906
and execution time of 13s. In addition, the X4 Dimension obtained excellent results in predicting when combined with
X2 Dimensions (Positive Education). The RMSE value can be reduced to 0.102 with an execution time of 8s.

Acknowledgements

This work is partially supported by Ministry of Research, Technology, Higher Education of the Republic of
Indonesia and The Ministry of Education, Culture of the Republic of Indonesia and National Board Teachers
Association of the Republic of Indonesia.

References

1. Cardwell ME. Patterns of Relationships Between Teacher Engagement and Student Engagement. Educ Dr [Internet]. 2011;1–125. Available
from: http://libguides.sjfc.edu/citations.
2. Sasmoko, Doringin F, Indrianti Y, Goni AM, Ruliana P. Indonesian Teacher Engagement Index (ITEI): An Emerging Concept of Teacher
Engagement in Indonesia. IOP Conf Ser Mater Sci Eng. 2018;306(1).
3. Sasmoko, Noerlina, Indrianti Y, Permai SD, Manalu SR. APPLYING INDONESIAN TEACHER ENGAGEMENT INDEX (ITEI) APPS:
SELF-DIAGNOSTIC. ICIC Int. 2018;9(3).
4. Rajabi S. A Recommender System for the Web: Using User Profiles and Machine Learning Methods. Int J Comput Appl. 2014;96(11):975–
8887.
5. Z. Zacharis N. Predicting Student Academic Performance in Blended Learning Using Artificial Neural Networks. Int J Artif Intell Appl
[Internet]. 2016;7(5):17–29. Available from: http://www.aircconline.com/ijaia/V7N5/7516ijaia02.pdf
6. Turhan M, Erol YC, Ekici S. PREDICTING STUDENTS ’ SCHOOL ENGAGEMENT USING. Int J Adv Sci Eng Technol. 2016;4(3):159–
62.
7. Ye M, Osman OA, Ishak S, Hashemi B. Detection of driver engagement in secondary tasks from observed naturalistic driving behavior. Accid
Anal Prev. 2017;106(January):385–91.
8. Shahiri AM, Husain W, Rashid NA. A Review on Predicting Student’s Performance Using Data Mining Techniques. Procedia Comput Sci
[Internet]. 2015;72:414–22. Available from: http://dx.doi.org/10.1016/j.procs.2015.12.157
9. Sasmoko, Muqsith AM, Widhyatmoko D, Indrianti Y, Khan A. Indonesian teacher engagement index (ITEI): Decision support system for
education. 2017 5th Int Conf Cyber IT Serv Manag CITSM 2017. 2017;
10. Li H, Zhang Z, Liu Z. Application of Artificial Neural Networks for Catalysis: A Review. Catalysts. 2017;7(10):306.
11. Sahin H, Topal B. Impact of Information Technology on Business Performance: Integrated Structural Equation Modeling and Artificial Neural
Network Approach. Sci Iran. 2018;25:1272–80.
12. Wang W-C, Silva MMS, Moutinho L. Modelling Consumer Responses to Advertising Slogans through Artificial Neural Networks. Unknown
[Internet]. 2016;15(2):89–116. Available from: http://search.ebscohost.com/login.aspx?direct=true&db=eoh&AN=EP120086962&site=ehost
-live
13. Paliwal M, Kumar UA. Assessing the contribution of variables in feed forward neural network. Appl Soft Comput J [Internet].
2011;11(4):3690–6. Available from: http://dx.doi.org/10.1016/j.asoc.2011.01.040
14. Weissteiner DJ. Variable importance measures in regression and classification methods. 2018;
15. de Oña J, Garrido C. Extracting the contribution of independent variables in neural network models: A new approach to handle instability.
Neural Comput Appl. 2014;25(3–4):859–69.
16. Naser SA, Zaqout I, Ghosh MA, Atallah R, Alajrami E. Predicting Student Performance Using Artificial Neural Network: in the Faculty of
Engineering and Information Technology. Int J Hybrid Inf Technol. 2015;8(2):221–8.