Proceedings of The Fifth International Conference on E-Learning and E-Technologies in Education (ICEEE2016), Malaysia 2016

Performance Evaluation for Higher Educational Institutions within Data

Envelopment Analysis
Qutaibah Althebyan Hassan Najadat Bushra Zaareer
Jordan University of Science and Technology (JUST)
Irbid, Jordan
(qaalthebyan, najadat)@just.edu.jo, ahbushra13@cit.just.edu.jo
ABSTRACT
Nowadays, governments are increasingly investing
more money in universities and especially in the
higher education in order to improve their quality. At the same time universities are facing a
dramatic growth on the number of students especially the graduate students. Hence, measuring the
performance of such universities in order to evaluate how they react with such number increase is
vital. Hence, this paper will evaluate the performance of Jordanian higher educational institutions
and then highlights weakness points in reasons that
caused inefficient universities (if found). In order to achieve this, the Data Envelopment Analysis (DEA) is used to measure the performance of
the institutions so that inefficient ones (if found)
are labeled. Recommendations will then be drawn
to guide the inefficient institutions to improve their
performance. In this paper, we will measure the
performance of Jordanian universities as a special
case.

KEYWORDS
Data envelopment analysis, DEA, higher educational universities, decision making unit, DMU, input unit, output unit.

INTRODUCTION

Nowadays, higher education plays an important role in the development of many societies.
Such societies try to give the higher education
more concentration and importance on their
sake of achieving better high education outcomes and results. Although giving higher education more concentration and interest is vital, however, this can impose pressure on these
societies governments budgets especially when
considering countries with limited resources

ISBN: 978-1-941968-37-6 2016 SDIWC

where Jordan is one example of such countries. In such countries, the increased interest in higher education resulted in opening new
universities and educational institutions, which
as a result increase the number of students who
are willing to enroll in them and pursue their
graduate studies [1, 2]. This increased interest
can be utilized if these societies concentrate on
improving the quality of the higher education
institutions as well as improving the quality of
the graduate students which in turns positively
affects the economy of these countries. Jordan
is one of these countries who consider the human resources and their educational quality as
an asset to their economy. It is also considered
as one of the major reasons for their progress.
In light of Jordan higher education institutions
(JHEI), it offers a qualification for young people to gain jobs. Since higher education degree
is an essential prerequisite to gain career place
in Jordan institutions, more and more young
people are going to enroll in such higher education institutions. The following example is an
illustration of the above mentioned fact where
MUTAH university has a total number of students enrolled at years from 2010 to 2013 to
be as 15957, 16998, 18519 and 21051 respectively. On the other hand, the total number of
academic staff is 498, 511, 523 and 533. In
general, this leads to a more resources consumption in JHEI. As mentioned earlier, JHEI
should provide high and effective performance.
Given these points, the necessity of scientific
studies and surveys about JHEI built upon scientific methods to apply in development countries is needed.
This study is intended to decide Which inputs
and outputs should be considered to be applied
to our approach that will lead to the best results. It is also intended to decide Which fea37

Proceedings of The Fifth International Conference on E-Learning and E-Technologies in Education (ICEEE2016), Malaysia 2016

tures will produce the worst output and hence,

should be recommended to be resolved. It will
finally draw conclusions and recommendations
(based on the results) that should improve the
JHEI performance.
2

OUR FRAMEWORK

To truly measure the efficiency of higher education in any country, a Decision Making
system (considering several decision-making
units DMUs) should be considered. Any Decision Making Unit (DMU) should consider both
inputs to the higher educational system process
as well as the output to the system; so that an
efficiency measurement should be calculated
and estimated. A DMU will be considered in
our system where the DMU can be defined as
a group of people in a company or other organization concerned to make an important decision. So, to estimate the whole higher education system outcomes; several DMUs that measure different higher educational aspects and
parameters should be considered. One major
concern that should be raised here is where the
system deals with multiple input/output (homogeneous input/output) needed to evaluate
and estimate the overall performance. This,
in fact, makes the process not as trivial as one
might think. Because of all the aforementioned
facts, there is a vital need to come up with
a system that can effectively estimate the efficiency of the DMUs and hence; the higher
educational system as a whole. Therefore,
the Data Envelopment Analysis (DEA) is suggested as our approach to deal with the above
problem. DEA is a non-parametric technique
that is used to estimate and compute the performance in homogeneous DMUs. In general,
DEA handles multiple input and output variables that are used to calculate the efficiency of
a system considering different weighted units
for them. In order to use the DEA, it needs
to identify DMUs as well as the inputs and
outputs that should be applied. In this paper,
we will examine DEA efficiency that measures
universities performance in Jordan from 2010
to 2013. Universities will be defined as DMUs
and then inputs and outputs variables will be

ISBN: 978-1-941968-37-6 2016 SDIWC

specified. So, statistics comparisons will be

conducted among a set of 19 researched articles to specify the set of inputs and the set
of outputs for higher education institution data
set. Once the set of inputs and outputs are determined, we will collect the data set from different resources such as public and private universities in Jordan and the Ministry of Higher
Education and Scientific Research. The following figure 1 illustrates the whole process.

Figure 1. Our Approach Process Workflow

2.1

Data Envelopment Analysis

Data Envelopment Analysis (DEA) is usually

used as a measurement for systems efficiency.
It is usually expressed as a percentage value. It
is determined by the ratio of desired output to
the total input. Measuring efficiency is computed as it is illustrated in equation 1.
Ef f iciency = Outputs/Inputs

(1)

In case of multiple inputs and multiple outputs,

the DEA is also good choice. Hence, DEA will
measure efficiency by finding the ratio between
weighted sums of outputs to weighted sums of
inputs, which is illustrated in Equation 2 [4].
The value of efficiency is usually normalized
between 0 and 1. If the efficiency value for
a specific DMUs is found to equal 1 then this
DMU is called an efficient DMU. Otherwise,
the DMU is called inefficient.

Ef f iciency = W.SumOutputs/W.SumInputs
(2)
The DEA method can be applied to various
fields to measure the efficiency [5]. In banks
and finance fields, the authors in [6, 7, 8]
used DEA to measure efficiency. Also in
[9], health system used DEA. Furthermore,
38

Proceedings of The Fifth International Conference on E-Learning and E-Technologies in Education (ICEEE2016), Malaysia 2016

the study in [10] used DEA to measure efficiency for Brazilian team football in season
2014. In the light of our dataset, the higher educational institutions are considered as homogeneity units. As an illustration, in [2, 11, 12]
DMUs have three input variables and three outputs variables in Spain, Italy, and USA respectively. DMUs have six inputs variables and
three outputs variables in England [13]. So,
the higher educational institutions have multiple inputs and outputs. DEA was used to evaluate DMU performance [2, 11, 12, 13].

DEA determines the maximum degree of output desired with respect to given inputs variables.

2.2

3.1

Input Oriented Measures

Input-oriented model minimizes the weighted

of inputs without changing the weighted of
outputs. This model reduces the input quantities proportionally with output fixed quantities. In general, the model looks at the amount
by which inputs can be proportionally reduced,
while the outputs are fixed. Moreover, efficiency in this model is still defined as the ratio between the weighted sum of outputs and
that of inputs. As it is previously mentioned,
its value is weighted between 0 and 1. In
[17], input-oriented DEA is applied to configure how much the input variables could be reduced while maintaining the same level of output. For example, if the following two input
variables are considered: the number of academic staff and non-academic staff with the
values 120 and 90 respectively, then, the target values should be 100 and 81 respectively.
After these values, inefficient DMUs become
efficient ones.
2.3

Output Oriented Measures

Output-oriented model minimizes the weights

of outputs without changes in the inputs
weight. This model expands output quantities proportionally with input fixed quantities.
In general, the model looks at the amount by
which outputs can be proportionally increased
while the inputs are fixed. Again, efficiency
value is still limited between 0 and 1. In [14], it
is configured to evaluate a unit probable output
given its input. Additionally, Output-oriented

ISBN: 978-1-941968-37-6 2016 SDIWC

THE DEA MODEL

This section presents input/output specification, dataset collection and data resources.
Then, it clarifies our approach and describes
the design for this approach. Finally, the experiment cases are addressed with results discussions.
Input-Output Specifications

This section examines inputs/outputs used to

measure DEA efficiency for universities in
higher educational institutions in Jordan. In
general a view for DEA process is shown in
figure 2 which shows inputs and outputs relation in this process.

Figure 2. DEA Process

In this paper, two categories are classified in

order to specify input variables for DEA. Resources are categorized into human and financial resources. Human resources are defined
as a collection of instructors, students, and
human licenses or roles recourses in universities. By reviewing a total of 15 articles,
it has the number of academic/non-academic
staff, full/associate/assistant professors as illustrated in [1, 2, 3, 13, 15, 16], student contact hours [16], the number of full-time equivalent instructors, human resource [11], the number of licenses to award higher doctorate degrees, the total number of postgraduate students, number of licenses to award the PhD degrees, and total number of undergraduate students. In other words, instructors, the staff
and students play an important role in the education process in universities. Additionally,
39

Proceedings of The Fifth International Conference on E-Learning and E-Technologies in Education (ICEEE2016), Malaysia 2016

students are the target in the education process. Indeed, instructors and staff help education process (academic or research product).
Financial resources are defined as a collection
of money (equipment, building, and facilities)
income to universities DMU. By reviewing a
total of 15 articles, we classified the finical resources to be as: instructional/overhead /general/administration/equipment expenditure as
illustrated in [12, 13, 16, 17], non-current assets [1], total coststhird-party funds [2], financial resources facilities and laboratories [2],
physical investment, total depreciation and interest payable [1], budget, research grants, age
of the institution, personal/non-personal cost
[1] and research income [2]. Indeed, these
studies show that financial recourses percentage is higher than human recourses percentage;
this indicates that the financial resources lead
the education wheels in universities.
Here, we can summarize the input variables.
The first variable is the total number of academic staff. The second is the total number
of nonacademic staff. The third is the summation of the total number of post-graduate student enrolled and the total number of undergraduate student enrolled. Moreover, this variable is vital because the total number of students enrolled in universities indicates the size
of the population in the education process. As
it can be noticed from these studies the total
expenditure variable is the highest percentage
in the finance resources. Finally, it is vital to
include the total expenditure as an input variable. These resources (set of inputs) employed
to complete DEA processed in our approach.
On the output side, we categorized outputs
variables as three variables. The first variable
is the summation of a total number of documents published in Scopus and the total number of documents published in Thomson. Because of the limitations of the sources for the
number of documents published in Jordanian
universities, this study uses Scopus and Thomson library to aggregate these variables into
one variable. Moreover, the total numbers of
undergraduate/post students graduated are the
highest percentages in the academic services.

ISBN: 978-1-941968-37-6 2016 SDIWC

So, they should be added to the output variables set.

3.2

Data Collection

This section presents our data set that is used

in our approach. It addresses the data set
collected as well as data resources. In our
study, the total numbers of universities under consideration will be twenty four universities of which ten are public universities and
the remaining fourteen are private universities.
Hence, we have twenty four DMUs applied to
DEA. Now, our study is interested in collecting
JHEI (inputs/outputs variables for our DMUs
on the period of a year from 2010 to 2013.
All the data are collected from the following
sources: Ministry of Higher Education and
Scientific Research, Scopus JUST library and
Thomson Reuters Web of Science as illustrated
in table 1 where it summarizes the DMUs
alongside their sources.
Table 1. Variables with Dataset Sources Summary
Variable
NumAcStaff
NumNAcStaff
NumStdEnroll
Expenditure
Research-Thompson
Research-Scopus
NumUnGrad
NumPostGrad

Source
Ministry of Higher Education
Ministry of Higher Education
Ministry of Higher Education
Ministry of Higher Education
Thomson database-JUST
Scopus database-JUST
Ministry of Higher Education
Ministry of Higher Education

The following figure 3 illustrates our DEA

model that is applied to the above mentioned
variables (DMUs).

Figure 3. DEA Model Process

EXPERIMENTS AND RESULTS

This section shows our experiments cases

where we employed two models. On the DEA
40

Proceedings of The Fifth International Conference on E-Learning and E-Technologies in Education (ICEEE2016), Malaysia 2016

Model side, we examined two cases. The

first case (called case 1) employed the constants return to scale type within input orientation. Specifically, case1 used the 24 DMUs
with three inputs (NumAcStaff, NumNAcStaff, NumStdEnroll) and three outputs variables (Research, NumUnGrad, NumPostGrad)
for years from 2010 to 2013. On case 2, we ran
DEA for 10 DMUs (public universities) with
four input variables (NumAcStaff, NumNAcStaff, NumStdEnroll, Expenditure) and three
output variables (Research, NumUnGradand,
NumPostGrad).
4.1

Case 1 Results

Figure 4 provides a summary of case 1 setting,

which includes 24 DMUs with three inputs and
three outputs based IO-CRS.

(MUTAH, AABU and PHILADLPHIA). An

increasing group that shows an increase in efficiency score in all years. This group has six
universities (JADARA, ASU, ZUJ, ZU, JPU
and ANU) and finally an irregular group that
shows variable performance scores in different years. These universities are (YU, BAU,
AHU, TTU, GIU, MEU, AMMANU, IU, UOP
and PSUT). On the positive side from IO-CRS,
the increasing group highlights the universities
who are efficient during the studied years. So,
these universities should be encouraged to keep
their growing movements. On the other side,
decreasing and irregular groups performances
should be studied to have a strategic plan in
managing their recourses so that they can improve their performance and be efficient.
4.2

Case 2 Results

Figure 6 provides a summary of case 2 setting,

which includes 24 DMUs with three inputs and
three outputs based IO-CRS.

Figure 4. Case 1 Experiment Settings

After running case 1 experiment and evaluating the performance of all the institutions under consideration for the input and output variables summarized in figure 4, we get the following results illustrated in figure 5

Figure 6. Case 2 Experiment Settings

After running the case 2 experiment and evaluating the performance of all the institutions
under consideration for the input and output
variables summarized in figure 6, we get the
following results illustrated in figure 7

Figure 5. Case 1 IO-CRS Summary

IO-CRS provides 19 inefficient universities as

can be seen in figure 5. It can be noticed
that their performance behavior can be categorized into three groups: a decreasing group
that shows a decrease in efficiency score in
all years. This group has three universities

ISBN: 978-1-941968-37-6 2016 SDIWC

Figure 7. Case 2 IO-CRS (Inefficient DMUs Only)

Figure 6 shows a review of case 2 settings,

which include 10 DMUs (public universities
41

Proceedings of The Fifth International Conference on E-Learning and E-Technologies in Education (ICEEE2016), Malaysia 2016

in Jordan) with four inputs and three outputs,

based IO-CRS. In case 2, IO-CRS, figure 7
shows five efficient universities in all years
(JU, YU, JUST, HU and BAU). Whereas two
universities fall in the inefficient status in the
one year (MUTAH and AABU). The AHU
grows in 2012 to efficient status. The other two
universities are inefficient in all years. Moreover, the TTU decreasing score is changed
during the study years, whereas the GIU is
changed to the irregular movements. Again,
decreasing and irregular institutions change for
DMUs movements. This in fact determines
which universities should consider critical attention to their performance movements. Also,
these universities should start handling weakness points in their recourses so that they can
better utilize their resources, and hence, be
more efficient.
5

CONCLUSIONS

The paper measured the efficiency of higher

education institutions and especially the Jordanian universities. In the first step, inputs/outputs for universities were specifiedd.
Then, the DEA process was used. Starting, in
the DEA Model we employed the DEA process to measure universities performance. Final results lead to determine the best features
that can improve efficiency. Highlights were
drawn to guide universities in improving their
efficiency relying on the best features that are
expected to produce the best results.
REFERENCES
[1] M. Abbott, C. Doucouliagos, The efficiency of
Australian universities: a data envelopment analysis, Economics of Education review, Vol. 22, pp.
89-97, 28 Feb 2003.
[2] A. GarcaAracil, M. D. Palomares, Methodological problems to measure university efficiency in
relation with its geographic localization, Institute
for Innovation and Knowledge Management, INGENIO (CSIC-UPV), Spain 2008.
[3] G. Kempkes, C. Pohl, The efficiency of German
universities some evidence from nonparametric and
parametric methods, Applied Economics, Vol. 42,
pp. 2063-2079, 2010.

ISBN: 978-1-941968-37-6 2016 SDIWC

[4] S.Talluri, Data envelopment analysis: models and

extensions, Decision Line, Vol. 31, pp. 8-11, May
2000.
[5] C. Lee, C. Lin, Globalization, political institutions,
financial liberalization, and performance of the insurance industry, The North American Journal of
Economics and Finance, Vol. 36, pp. 244-66, 30 Apr
2016.
[6] S. Ohsato, M. Takahashi , Management Efficiency
in Japanese Regional Banks: A Network DEA,
Procedia-Social and Behavioral Sciences, Vol. 172,
pp. 511-8, 27 Jan 2015.
[7] R.B. Staub, G.D. e Souza, B.M. Tabak, Evolution
of bank efficiency in Brazil: A DEA approach, European journal of operational research., Vol. 202,
pp. 204-13, 1 Apr 2010.
[8] Y. Luo, G. Bi, L. Liang,Input/output indicator selection for DEA efficiency evaluation: An empirical
study of Chinese commercial banks, Expert Systems with Applications., Vol. 39, pp. 1118-23, 31
Jan 2012.
[9] M. Torres-Jimnez, C.R. Garca-Alonso, L. SalvadorCarulla, V. Fernndez-Rodrguez, Evaluation of system efficiency using the Monte Carlo DEA: The
case of small health areas, European Journal of
Operational Research, Vol. 242, pp. 525-35, 16 Apr
2015.
[10] M.C. Roboredo, L. Aizemberg, L.A. Meza, The
DEA Game Cross Efficiency Model Applied to the
Brazilian Football Championship, Procedia Computer Science, Vol. 55, pp. 758-63, 31 Dec 2015.
[11] J.E. Beasley, Determining teaching and research
efficiencies, Journal of the operational research society, pp. 441-52, 1 Apr 1995.
[12] T. Ahn, A. Charnes, W. Cooper, Some statistical and DEA evaluations of relative efficiencies of
public and private institutions of higher learning,
Socio-Economic Planning Sciences, Vol. 22, pp.
259-69, 31 Dec 1988.
[13] J. Johnes, Data envelopment analysis and its application to the measurement of efficiency in higher
education, Economics of Education Review, Vol.
25, pp. 273-88, 30 Jun 2006.
[14] S. Pascoe, J.E. Kirkley, D.F. Greboval and C.J.
Morrison, Measuring and assessing capacity in
fisheries: Issues and methods, Food & Agriculture Org., 2003. FAO Fisheries Technical Paper. No.
433/2. Rome, FAO. 2003. 130p.

42

Proceedings of The Fifth International Conference on E-Learning and E-Technologies in Education (ICEEE2016), Malaysia 2016

[15] G. Abramo, T. Cicero, CA.DAngelo, A fieldstandardized application of DEA to national-scale

research assessment of universities, Journal of Informatics, Vol. 5, pp. 618-28, 31 Oct 2011.
[16] T . Agasisti, C. Prez-Esparrells, Comparing efficiency in a cross-country perspective: the case of
Italian and Spanish state universities, Springer US.
Higher Education, 59(1). pp. 85-103, 2010.
[17] N.K. Avkiran, Investigating technical and scale
efficiencies of Australian universities through data
envelopment analysis, Socio-Economic Planning
Sciences. , Vol. 35, pp. 57-80, 31 Mar 2001.

ISBN: 978-1-941968-37-6 2016 SDIWC

43