Nakato Ruth Cit Masters Report

A MULTI CRITERIA DECISION MAKING
SUPPORT TO SOFTWARE SELECTION
BY
Nakato Ruth
Reg. No. 2005/HD18/0038U
BBA (MAK), UDBS (Mak)
E-mail: nakatodinah@yahoo.co.in +256772501880
A Dissertation Submitted to the school of Graduate Studies

in Partial Fulfilment for the Award of
Master of Information Technology Degree of
Makerere University
Option: Information Technology Management
April, 2008
this page is empty
i
Declaration
I Nakato Ruth do hereby declare that this dissertation is original and has not been published
and/or submitted for any other degree award to any other University before.
Signature...................................... Date:..........................................
Nakato Ruth
BBA (Mak)
Department of Information Technology.
Faculty of Computing and Information Technology
Makerere University
Approval
This dissertation has been submitted for Examination with the approval of the following
supervisor.
Signature........................................ Date:............................................
Dr. Tom Wanyama, PhD
Department of Software Engineering
Faculty of Computing and Information Technology
Makerere University
ii
Dedication
This dissertation is dedicated to God The Almighty who gave me the breath and life to see it
complete. And to my beloved husband Mr.Twinomugisha Didas, with whom I have shared
the most splendid thing LOVE.
iii
Acknowledgements
”If I have been able to see further than others, it is because I have stood on the shoulders
of giants.”-Sir Isaac Newton. No intellectual achievement occurs in a vacuum. All new cre-
ativity builds on the efforts that have gone before. Like Newton, I have been able to stand
on the shoulders of extremely talented people. I am forever indebted to these giants that
have contributed necessary ingredients to this work. First, my academic advisors/lecturers
for exposing me to the world of Information Technology. And second, the companies that
supported this project through provision of expertise, and data. I am also grateful for the
support of Namirembe Esther for her constructive criticism and thorough editing performed
on my report . The ideas presented here exist as a result of the trailblazing vision of my
supervisor, Dr. Tom Wanyama who gave his unconditional intellectual support, precious
time and effort to ensure that I finish successfully.
To my Spouse Mr. Twinomugisha Didas who kept the home fire burning. He had Great
patience and gave me tireless support. He sacrificied alot to see me successfuly through
and kept encouraging me. I got a lot of courage from our sons Bashabe Rodney, Agaba
Albert, and Keita Rhene Didas. Albert kept asking me Mum how can you be married to
Dad and yet you are still a student? Keita would ask me whether my teacher gives me a
star when I perform well, or whether the head master beats me when I reach late. While
they were concerned about what I was going through I kept encouranging them to mantain
the academic spirit towards attaining the same goal.
Special thanks go to the following students Esther Namirembe, Brenda Tusubira, Eilu Em-
manuel, Maria Komunte and the entire class of March in-take 2005 academic year, for having
made my academic and social life comfortable at Makerere University.
”In matters of style, swim with the current; In matters of Principle , stand like
a rock”. -Thomas Jefferson.
iv
Contents
Title Page . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
Declaration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Approval . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ii
Dedication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv
Contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii
List of Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Background to the Study . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Definition of Key Theoretical Terms . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Context of Decision Making and Software Selection . . . . . . . . . . . . . . 3
1.4 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 General Objective of the Research . . . . . . . . . . . . . . . . . . . . . . . . 5
1.6 Specific Objectives of the Research . . . . . . . . . . . . . . . . . . . . . . . 6
1.7 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
v
1.8 Significance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.9 Outline of the Dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Literature Review 8
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 Issues Related to Multi-Criteria Decision Making in Software Selection . . . 8
2.3 Multi Criteria Decision Making in Software Selection . . . . . . . . . . . . . 11
2.4 Multi Criteria Decision Making Supports for Software Selection . . . . . . . 13
2.4.1 Analytical Hierarchy Process (AHP) and Software Selection . . . . . 14
2.4.2 The Simple Multi-Attribute Rating Technique (SMART) and Software

Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.4.3 Electre and software selection . . . . . . . . . . . . . . . . . . . . . . 16
2.4.4 Using Fuzzy Multi-Criteria Decision-Making to Software Selection . 16
2.4.5 Weighted Sum Method (WSM) and Software Selection . . . . . . . . 17
2.5 Chapter Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
3 Methodology 18
3.1 Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.2 Study Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
3.3 Data Collection Method and Instrument . . . . . . . . . . . . . . . . . . . . 19
3.4 Validity of the Self Administered Questionnaire (SAQ) . . . . . . . . . . . . 19
3.5 Reliability of the SAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.6 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.7 Approach to Formulate a Multi Criteria Decision Making Support for Software
Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
vi
3.8 Chapter Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4 Data Presentation, Analysis and Interpretation of Results 23
4.1 Survey Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
4.2 Description of the Background Variables . . . . . . . . . . . . . . . . . . . . 23
4.3 Issues Related to Multi Criteria Decision Making in Software Selection . . . 25
4.4 Identification of the Criteria Used for Multi Criteria Decision Making in Soft-
ware Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
4.5 Formulation of a Multi Criteria Decision Making Support for Software Selection 33
4.5.1 Make a Team of Decision Makers . . . . . . . . . . . . . . . . . 33
4.5.2 Identify the Software Project Characteristics . . . . . . . . . . 34
4.5.3 Evaluate the Software by the AHP Decision Making Approach 34
4.5.4 Derive the Priorities for each Alternative Reflecting the Degree to
which the Alternative Satisfies the Goal . . . . . . . . . . . . . . . . 35
4.5.5 Make the Final Decision . . . . . . . . . . . . . . . . . . . . . . . . . 36
5 Discussion, Conclusion and Recommendations 40
5.1 Limitation of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.2 Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
5.3 Further Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
vii
List of Figures
4.1 A multi criteria decision making support to software selection . . . . . . . . 37
4.2 Evaluation of the software by the AHP decision making approach . . . . . . 38
viii
List of Tables
4.1 Response According to Gender . . . . . . . . . . . . . . . . . . . . . . . 24
4.2 Response According to the Age and Experience . . . . . . . . . . . . 25
4.3 Analysis of Variance of Cost According to Product Characteristics: 27
4.4 Analysis of Variance of Vendor According to Product Characteristics: 28
4.5 Analysis of Variance of Functionality According to Product Char-

acteristics: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
4.6 Analysis of Variance of Usability According to Product Character-

istics: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
4.7 Judgement scores for the importance/preference of criteria using

AHP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
ix
LIST OF ACRONYMS
DSS Decision Support System
ICT Information Communication Technology
MCDM Multi Criteria Decision Making
IT Information Technology
IS Information Systems
ROI Return on Investment
AHP Analytic Hierarchy Process
ERP Enterprise Resource Planning
DW Data Warehousing
SAQ Self Administered Questionnaire
DICTS Directorate of Information Technologies
FCIT Faculty of Computing and Information Technology
COTS Commercial Off the Shelf Software
WSM Weighted Sum Method
SMART Simple Multi-Attribute Rating Technique
x
Abstract
Decision analysis looks at the paradigm in which an individual decision maker or decision
group contemplates a choice of action in an uncertain environment. The theory of decision
analysis is designed to help the individual or decision group to make a choice among a set
of pre specified alternatives. Selecting a software is a complex process involving multiple
alternatives to choose from, multiple vendors to be considered and multiple criteria to be
evaluated. Owing to the imprecise nature of software selection, a multi criteria decision-
making support was formulated, in which make a team of decision makers, identify the
software project characteristics, evaluate software, derive the priorities for each alternative
and make the final decision are the steps. This process was based on criteria got from
literature review and refined using data from a survey of Ugandan IT managers and is the
main contribution of the research. Findings indicated that product costs, training costs,
vendor reputation , quality and ease of use are major issues considered in the decision
making process for software selection. The multi criteria decision making support provides
a framework to assist managers in analysing various software selection factors, evaluating
software alternatives, and making final software selections.
xi
Chapter 1
Introduction
This chapter covers the background, problem statement, general and specific objectives,
scope, significance and outline of the dissertation.
1.1 Background
The background is subdivided into the theoretical background, conceptual and contextual
background. The theoretical background described the importance of decision-making in
software selection hence linking the two components of our study; the theoretical background
also highlights the theory underpinning the research. The conceptual background explains
the operational terms used in the research and the contextual background explains the
situation of multi criteria decision making in relation to software selection highlighting the
problems.
1.1.1 Background to the Study
Decision making has inspired reflection of many thinkers since the ancient times. Salvatore
et al (2003) [85] says that the great philosophers for example, Aristotle, Plato, and Thomas
Aquinas, discussed the capacity of humans to decide and in some ways claimed that this
possibility is what distinguishes humans from animals. Baudry and Vincent 2002 [5] view
decision analysis as a paradigm in which an individual decision maker or decision group con-
templates a choice of action in an uncertain environment. The theory of decision analysis is
1
designed to help the individual or decision group make a choice among a set of prespecified
alternatives. The act of decision making is not a new idea in software selection. Capterra
(2005) [17] emphasises that the growth of the enterprise software market and the upcoming
of new applications make the task of identifying, evaluating, and selecting the right soft-
ware solution a tremendous challenge. The rising number of companies, including software
vendors, resellers, research firms, industry publications, consulting firms, and other service
providers only intensifies the challenge. So it is no great surprise that most enterprises en-
gaged in the software selection process fall far short of their intended goals. In principle,
software selection embeds an art of multi criteria decision making in which the best software
package has to be selected from a set of options. In the new global economy, as Hmlinen
et al (2003) [36] found , software selection decision making occurs across multiple physical
locations more frequently and it is a very important decision . The decision can seriously
affect the way the company is run. Get it wrong and the consequences of failure could be
very serious. There have been recorded instances of failed business or software implementa-
tions actually causing companies to go into liquidation.
Verville and Hallingten (2002) [100] observed that in order to be successful in any selec-
tion and evaluation there must be clearly defined business objectives that are communicated
throughout the company. Selection of solutions requires the company to examine its people,
policies, and processes as well as its technology. Jenz and Partner (2005) [44], show that
decision making is an important aspect in selection of company software. Jenz and Partner
(2005) [44] claim that it pays if the right product is selected for a task instead of wasting
valuable resources if a wrong product is selected.
1.2 Definition of Key Theoretical Terms
Multi Criteria Decision Making: According to Gulfem and Gulcin (2007) [31], Multi
criteria decision making (MCDM) refers to screening, prioritising, ranking, or selecting a set
of alternatives under usually independent, incommensurate or conflicting criteria. Accord-
ing to Salo et al, (2003) [84] Chou et al (2004) [18], MCDM concerns breaking a problem
down into its constituent parts or components, in the framework of a hierarchy or a feedback
network, and establishing importance or priority to rank the alternatives in a comprehensive
and general way to look at the problem mathematically. Hua-Yang (2004) [41] considers
MCDM to be largely concerned with the deployment of systematic methods to help address
problems characterized by incomparable objectives, multiple stakeholders and conflicting in-
2
terests. Sant(2003) [97] defines MCDM as the study of methods and procedures by which
concerns about multiple conflicting criteria can be formally incorporated into the manage-
ment planning process. In respect to the research, multi-criteria decision making is the
method and the procedure by which concerns about multiple conflicting criteria are incor-
porated into software selection.
A support: According to the free dictionary.com [42] a support is something provid-

ing immaterial or assistance to a person or cause or interest. In respect to the research,
the term support signifies a framework that aids multi criteria decision making to software
selection. Therefore a multi criteria decision-making support in this research is the method
and the procedure by which concerns about multiple conflicting criteria can be formally
incorporated into a frame work.
Software: According to the free dictionary.com [42] Software are the programs, routines,
and symbolic languages that control the functioning of the hardware and direct its operation.
Set of instructions that cause a computer to perform one or more tasks, available on line [42].
Software is often called a program or, if the set is particularly large and complex, a system.
Software in respect to this research are considered as computer application programs that
are used within organisational settings to perform different tasks.
Selection: According to free dictionary.com the term selection means choice, [42]. Soft-
ware selection in respect to this research is the process of choosing new enterprise software;
requirements are established, candidate solutions identified, and selection is made.
1.3 Context of Decision Making and Software Selec-

tion
Today software selection has become both increasingly crucial to corporate success and si-
multaneously difficult to perform, as Capterra (2005) [17] observed. Mamaghani (2002) [57],
found that with the increased number of software vendors to choose from and intense pres-
sures to keep up with the latest technologies; decisions about software selection are hard to
make. Managers in most companies resort to subjective decisions due to an inability to as-
sess the profitability of new technologies. As Williams (2002) [103] explains, the failure of a
chosen software often results from poor management and preparation of the change process.
Some of the causes have been attributed to the inability to consider the wider relationship
3
of software to the business and organisational context and how to include these issues in
the software investment considerations. Hua-Yang Lin et al (2006) [40] argue that decision
making in the field of software selection has become more complex due to a large number
of software products in the market, ongoing improvements in information technology, and
multiple and sometimes conflicting objectives.
Burmark and Thornberg (2000) [14] further assert that, the real problem can usually be
attributed to the initial stages of the project and ineffective software selection. When mea-
suring long-term value, software selection becomes far more important than software im-
plementation. The proliferation of software systems has created a difficult and complex
decision problem for enterprises or users to select and evaluate the best software products.
Such decision making is inherently difficult. Specifically, there are three main difficulties;
interactions amongst multiple decision makers, dealing with decisions under multiple crite-
ria, and involving decision makers who are physically located in different sites. One major
source of complexity in decision problems is the need to simultaneously consider different,
and sometimes conflicting, criteria. When a decision involves multiple criteria, it cannot be
made in a purely objective way, but requires subjective judgement to evaluate the trade-offs
between criteria, (Banks and Gibson 2004) [3].
Mamaghani (2002), Banks and Gibson (2004) [3] and Capterra (2005) have recently ver-
ified what many researchers have contended for a long time that, most software purchases
fail to meet the project objectives and provide long lasting benefit. Budget overruns, deploy-
ment delays, major customizations, and underutilization are not the problems themselves,
but symptoms of a problem that cause failure. Enterprise software initiatives fail to meet
their stated objectives, because of ineffective analysis and evaluation during the selection
process. Gusdorf (2006) [30] insists that software selection is complex and time-consuming,
but selecting the right enterprise software can result in market leadership and sustainable
competitive advantage, picking the wrong one can result in the loss of valuable time, and
in some cases, complete failure. Gusdorf acknowledges that selection of business software
is far from being an easy task, with more products coming on the market everyday, more
low-end products reaching up, more high-end products reaching down, and with mergers,
acquisitions and name changes, it is actually getting more confusing. In the midst of this
confusion it is tempting to look for short cuts in the software selection process. Gusdorf
argues that taking shortcuts or skipping steps in the process increases the risk of making a
wrong choice.
4
Mamaghani notes that software evaluation and selection has become an emotive issue in
some companies. Mulebeke and Zheng (2006) [63] tried to eliminate the illusion that general
knowledge in purchasing software is sufficient but that rather a multicriterion, multiattribute
strategic decision making approach that would yield better performance in the selection cri-
teria. Mulebeke and Zheng argue that those tasked with finding software or those involved
in the software selection and comparison process or software requirements specification often
base their selection on the fact that they like ”the look” of a particular software application.
But the assortment of features available in modern business software applications can be
Overwhelming. This approach is wrong according to Mulebeke and Zheng. The approach
to the process of selecting software must be measured and scientific. The features that are
required must be listed and checked to ensure that the applications being considered can
effectively provide them. Robert (2004) [76] points out that the selection phase has been
identified as a key area where problems arise.
1.4 Statement of the Problem
The preceeding discussion indicates that the process of decision making in software selection
is a complex issue, and that poor analysis and evaluation during the selection process can
cause software initiatives to fail to meet objectives. Burmark and Thornberg (2000) [14]
explain that software selection becomes far more important than software implementation.
One wonders what managers and software selection decision makers can do to make efficient
decisions for organisations while selecting software. Hua-Yang Lin (2006) confirms that
selection of the most suitable software from a set of alternatives, on the basis of many
criteria, creates a multi-criteria decision-making problem. This is why this research presents
a multi-criteria decision making support to software selection.
1.5 General Objective of the Research
The general objective of this research is to establish a multi criteria decision making support
for software selection.
5
1.6 Specific Objectives of the Research
The specific objectives of the study were the following:

1. To investigate issues related to multi criteria decision making in software selection
2. To identify criteria used for multi criteria decision making in software selection
3. To formulate a multi criteria decision making support for software selection
1.7 Scope
The research data collection used to refine the criteria, took place in Kampala covering the
following organisations: DICTS, FCIT, Makerere University Business School, department of
business computing and management science, Kyambogo University IT department, Mulago
Hospital IT department, and Ministry of Tourism IT department. The research conceptually
covers multi-criteria decision making supports in software selection.
1.8 Significance
The study provides information for managers who make decisions in selecting software with
in organisations. The multi criteria decision making support provides a framework to assist
managers in analysing various software selection factors, evaluating software alternatives, and
making final software selections. The study is a platform to the governemnt in implementing
issues related to multi-criteria decision making supports within orgainsations. The study is
also knowledge to researchers interested in decision making while selecting software.
1.9 Outline of the Dissertation
This dissertation comprises of five chapters: Chapter one details the background of the study,
the research problem, offers a justification for the study and defines the research objectives,
finally the significance of the study is explained. In the literature review (Chapter two),
6
emphasis was placed on issues related to software selection and formalisations of a software
selection support process. Different multi criteria decision making supports were reviewed as
a basis to design the multi criteria decision making support process. This chapter notes that
multi criteria decision making to software selection is an important aspect that has received
little attention and identifies the research gaps in past studies. Chapter three describes
the steps and procedures followed in order to accomplish the research. The questionnaire
method was used to investigate issues related to software selection. Data collection was
largely quantitative based on use of questionnaires, and qualitative using descriptive means
in analysis of data. Chapter four describes the survey data analysis, which enables the most
important criteria in software selection to be identified. This chapter goes on to show how
these criteria are used in the proposed support of software selection. Chapter five discusses
results in relation to objectives of this study. The criteria used in multi criteria decision
making in software selection have been identified from literature review and from the field
data that was collected, the results were used to refine the list of criteria used in decision
making in software selection process. The limitations of the study included the limited
nature of the survey, both in terms of questions and population surveyed. The support was
also not tested.
7
Chapter 2
Literature Review
2.1 Introduction
This chapter covers issues related to multi-criteria decision making in software selection,
criteria used for multi criteria decision making in software selection and multi criteria decision
making supports for software selection
2.2 Issues Related to Multi-Criteria Decision Making

in Software Selection
The need for multi-criteria decision making in software selection is well established in the
literature. According to Oyku (2005) [67] ”Computer-based information systems cover every
sphere of management. They are pointers of a ”new information age”, where information is
a key organizational resource, and where management activities become more information-
intensive. Enterprise applications are used widely in every sector and become more wide
spread everyday.” Oyku goes on to explain that the selection of the most appropriate solution
is a semi-structured decision problem. This is because traditional structured approaches
and procedures such as standard investment calculations can only help with part of the
decision making. The decision maker also has to evaluate business impact aspects. There
is no widely accepted formal procedure for this. According to Jie et al. (2005) [32] any
organizational decisions are made through evaluating a set of alternatives and then selecting
the most satisfactory one from them based on the information at hand and the perspectives
8
of decision makers. Jie et al. explain that multiple criteria are often used to evaluate the
set of alternatives where some criteria could be more important than others in selecting the
most satisfactory one.
The need to link selection criteria to fulfillment of business objectives is also discussed by
many authors. Liang and Hung (1997) [54], argue that in order to take the advantage of
using computers, the computerisation strategy should cope with the corporate goals and
business needs of delivery, quality, and cost control. From this view, as a computer system,
a selected software solution should be able to support decision-making. Mabert et al. (2001)
[56] explains that if the right solution is selected, it can be an excellent solution that will
provide a competitive advantage. The software should address the companies’ core activities
(Brewer, 2000) [12].
Cost is another important issue in software selection. Gupta and Sahay (2003) [29] observed
that the initial costs of a package are but a fraction of the true cost. Companies offer
very competitive and low basic software cost but charge high rates for additional hardware
or special equipment and annual cost. So the total cost of ownership should be put into
consideration. The overall cost of the software is essential to gain top management support
in buying the software, according to Hecht (1997) [37]. These costs includes costs per
module, total package cost, annual maintenance cost, planning and implementation cost,
installation and training cost, cost of upgrades and special hardware cost. Mabert et al.
(2000) [56] note that cost components could include software, hardware, consulting, training,
implementation team, and other costs.
Technology is another important driver in software selection. Gupta and Sahay (2003) [29]
warn that before buying any package it is essential to check its compatibility with existing
hardware, operating systems and database engines. In the Web-era these softwares must
support Internet, network and e-commerce setups. High technology support for business
integration is essential in broadening the marketing network. Chung and Chik, (2001) [19]
explain that the increased competition, along with the rapid development of new infor-
mation technologies, has forced more and more businesses to rely on information systems.
Using information technology based on computer to support decision-making can be a pow-
erful competitive weapon, especially when its adoption is aligned with corporate strategy.
Shikarpur, (1997) [89], Poon and Wagner, (2001) [72] argue that the software should be
current, so users need to check the currency of IT trends in the software product and examine
if the vendor is committed to incorporating the latest trends in IT in the product.
9
Technical architecture uncovers the fit between IS and the end user’s needs. Service and
support associated with the application is very important to the success of the partnership
between end user and application vendor, as noted by Gupta and Sahay (2003) [29]. Most
companies face technical or other problems during installation, implementation, or after
the implementation period. Integration with existing systems, customisation, and security
are the most serious problems for the companies. Therefore companies need support from
suppliers as noted by Rao, (2000) [75], both in terms of IT expertise and domain knowledge.
Customisation is another important factor in software selection, as Oyku (2005) [67] ar-
gued that companies need to adapt the available software in the market for their own use.
Avshalom (2000) [2] noted that software vendors are required to provide tools and utilities
that will allow the firm’s in house IT personnel or independent consulting firms to customise
the software.
In the vendor evaluation process, criteria such as vendor strength and/or reputation, fi-
nancial stability, and vendor’s vision are considered Verville and Hallingten, (2002) [100],
Hecht (1997) [37]. Companies should know about the reliability and how long the vendor
has been in the core software solution business and whether current users have been satisfied
with the software packages, claims Brewer(2000) [12]. Learning from past experiences, some
companies lay high stress on vendor reputation and service infrastructure when selecting
their software systems (Kumar et al., 2003) [49]. Shikarpur (1997) [89] advises that sales
references, reputation, and internationality of the vendor, could be considered as important
criteria for the selection process. It is also important that the software developer or supplier
knows the industry and is willing to implement the software for the industry according to
Rao (2000) [75].
Compatibility/integration is a critical factor for the system success. Bingi et al., (1999) [9]
stress this and it is also echoed by (Kumar et al., (2003) [49]. Compatibility with the
other systems is crucial for realising the potential benefits of software packages. Kumar et
al. (2003) [49] argue that system reliability is the second important selection criterion, and
Shikarpur, (1997) stresses that incorporating the best business practices of every area as
well as the latest trends in IT are important for the new system. In the selection process,
providing compatibility with parent/allied organisations can affect the decision process for
some companies, (Gupta and Sahay, 2003) [29] noted. According to Shikarpur, (1997) [89]
the real benefit of a software is in integration, a fully integration should be present between
the modules. Gupta and Sahay (2003) noted that if the integration cannot be achieved,
10
implementation time can extend, implementation cost can increase and finally the effec-
tiveness of the system can decrease. Software implementation is highly costly and complex
in company-wide project. According to Mabert et al. (2000) [56], implementation time
is closely correlated to the selected implementation strategy. The implementation time can
also be changed with implementation scope. On the other hand, more customisation requires
more time and more cost, (Bingi et al., 1999 [9], Gupta and Sahay, 2003) [29]. Software sys-
tems are projects that cause massive change in companies and therefore, the projects should
include a clear methodology. The methodology proposed by the software vendor should be
effective and should not include unnecessary activities for the companies.
The following five factors make software selection and evaluation difficult and complex argue
Hua-Yang Lin et al. (2006) [40]: the number of software products available in the market,
the continual advancements and improvements in information technology , the existence of
incompatibilities between various hardware and software systems , the functional dissimilar-
ities are difficult to evaluate among software packages, and the users, lack of the technical
knowledge and experience for software selection decision making. According to Mulebeke
and Zheng (2006) [63], the mushrooming of software is not because of differentiation in
functionality but as a result of the variation in vendors. There are many software programs
performing the same functions but branded different names giving the impression they are all
different yet actually they are not, this actually increases the dilemma in software evaluation
and selection.
2.3 Multi Criteria Decision Making in Software Selec-

tion
Decision making at all levels of an organization is about information collection, evaluation

and tradeoffs to analyze complex problems. More often than not, those decisions are made
by collective or individual judgment, after weighing advantages and disadvantages of policy
options, under circumstances of risk and uncertainty (Saaty and Niemira 2006) [83]. Accord-
ing to Mulebeke and Zheng (2006) [63], Software selection is not a technical procedure, but
is rather, a subjective and uncertain decision process . Selecting a suitable software system,
among many, depends on the assessment of objective, measurable criteria (e.g., acquisition
costs and training costs), as well as subjective criteria (e.g., compatibility, vendor selection
and technical factors) (Stamelos, et al., 2000 Mulebeke and Zheng, 2006). Software selection
11
decisions involve the simultaneous consideration of multiple criteria, including tangible and
intangible factors, and prioritizing these factors can be challenging.
Power (2002) [74] argues that, software selection is a sequential process that should begin
with specifying requirements and inquiring functions and tasks that managers will perform
with the software. Determination of products that meet the need can then take place. Once
comparable products are identified then the one that suits a particular situation is chosen.
Power (2002) explains that sometimes one criterion is so important in making a choice that
all other criteria take on a secondary role. If a dominant alternative does not exist and
if no one criterion dominates all others, then Power claims that approximately five major
criteria should be identified and weighted for evaluating the comparable software packages:
examining the functions that a software product can perform and how important they are to
the decision support need of targeted users, checking the cost of the package by examining
the total cost of ownership including (acquisition costs, implementation and training costs,
maintenance costs, and any annual software license costs ), ease of use, ease of installation
and operation, and performance of a product. Brewer (2000) [12] argues that affordability
is a very crucial criterion in software selection process. According to Rao, (2000) [75] the
software should have attractive prices.
According to Mohsen et al. 2005 [60] important selection characteristics include the fol-
lowing; strategic fitness or fitting the software system to vision, strategies and goals of the
organization; adaptability to existing infrastructure and ease of integration with external
systems; user-friendliness including ease of operations, ease of learning, online and offline
help; ease of in-house development and upgrading; use of newest capabilities of information
technology; processing times and minimisation of errors, bugs and breakdowns; and main-
tainability, among other factors.
Hecht,(1997) [37] presents six major criteria for selection and evaluation process includ-
ing: functionality, technical architecture, cost, service and support, ability to execute, and
vision. According to Bernroider and Koch, (2001)[8] the criteria for selection of a particu-
lar system show different priorities related to organisational size. Organisational flexibility,
extra organisational ties with customers and suppliers and internationality were important
criteria for bigger companies, while cost and adaptability are more important criteria in soft-
ware selection process for smaller companies. According to Kumar et al.(2003) [49] research
functionality, system reliability, and fit with parent/allied organisation systems are deter-
mined as the three most important criteria. Birdogan and Kemal (2005) [10] identified 17
main selection criteria for ERP selection process which included the following; consultancy,
12
methodology of the software, cost, references of the vendor, service and support, technical
criteria, functionality, vision, system reliability, fit with parent/allied organisation systems,
cross module integration, compatibility with other systems, market position of the vendor,
domain knowledge of suppliers, among others.
Oyku (2005)[67] listed the following criteria, in order of priority: customization, imple-
mentability, maintenance, real time changes, flexibility , user friendliness, cost, systems
requirements, after sales, support & training , back-up system, reporting & analysis fea-
tures, vendor credentials, integration with other software/applications, internet integration,
and financing options. Verville and Hallingten (2002) [100], determined three distinct types
of criteria for evaluation: vendor, functionality and technical, vendor evaluation criteria in-
cluded size, financial stability, reputation of vendor, functional criteria dealt with the features
of the software, and included functionalities specific to front-end interfaces, user friendliness.
Technical criteria dealt with the specifics of the systems architecture, integration, perfor-
mance, and security. Mulebeke and Zheng (2006) [63] considered the following criteria to be
the most relevant among others, performance, usability, data and data file support. Capterra
(2005) [17], suggested software guidelines and include; planning, identification, evaluation,
and selection.
2.4 Multi Criteria Decision Making Supports for Soft-

ware Selection
The methods used to identify a set of possible software candidate solutions are, always
rather subjective according to Bandor (2006) [4]. The individual or individuals performing
the evaluation have various, distinct experiences that will factor into the decision process,
either consciously or subconsciously. To have a successful software evaluation and selec-
tion, a formal process is needed to properly evaluate the software products and vendors
supplying them , according to Bandor. In this instance, the term formal means having an
established and documented process to perform the selection and evaluation activities in a
consistent, repeatable manner. The literature reviewed was limited to the application of
different multi criteria decision making methods to select a number of software applications.
These frameworks and methods are grouped into those that are similar for example Ana-
lytic Hierarchy Process (AHP) methodology, The Simple Multi-Attribute Rating Technique
(SMART), Electre, Using Fuzzy Multi-Criteria Decision-Making, Weighted Sum Method
13
(WSM). The coverage is not exhaustive, however it demonstrates the diversity of software
selection.
2.4.1 Analytical Hierarchy Process (AHP) and Software Selection
Analytical Hierarchy Process (AHP) is an approach to decision making that involves struc-
turing multiple choice criteria into a hierarchy, assessing the relative importance of these
criteria, comparing alternatives for each criterion, and determining an overall ranking of the
alternatives Ngai and Chan (2005) [64]. AHP provides a hierarchical approach for consolidat-
ing information about alternatives using pair-wise comparisons. It is evidently shown in lit-
erature that the Analytic Hierarchy Process (AHP) methodology, Ngai and Chan (2005) [64]
is an excellent technique as it provides a structure and hierarchy method, for synthesizing
software selection problems. The evidence of diversity in software system selection shows
that, the AHP can be applied to the selection of software products consistent with the
maximization of the underlying criteria and sub-criteria expectations of the decision makers.
Software selection using AHP has been applied and includes; Beck and Lin (1981) [6] and Sei-
dmann and Arbel (1983) [88] on office automation software, Seidmann and Arbel (1984) [88]
on accounting information system, by Zahedi (1985) [106] on database management system,
by Roper-Lowe and Sharp (1990) [77] on computer operating system, by Zahedi (1990) [106]
on expert system. AHP was also used by , Phillips-Wren et al. (2004) and Kim and Yoon
(1992) [34] on expert system shell , and by Min (1992) [59] on Logistics software. More still
Davis and Williams (1994) [21] used it on manufacturing simulation software, by Mohanty
and Venkataraman (1993) [61] on automated manufacturing system, by Kim and Moon
(1997) [34] on workflow management system, by Lai et al. (1999) [52] on multimedia au-
thoring system ,and by Teltumbde (2000) [95] on ERP system. Lai et al. (2002) [52] on
multimedia authoring system , and Mamaghani (2002) [57] on antivirus and content filtering
software. It was also used by Phillips-Wren et al. (2004) [71] on decision support system,
by Sarkis and Talluri (2004) [86] on supply-chain software and e-commerce communication
systems, by Jung and Choi (1999) [45] on COTS software , by Ngai and Chan (2005) [64]
on knowledge management tool , by Wei et al. (2005) on ERP system [104].
According to Golden et al. (1989), [81] Saaty T L. (1992), [83] Saaty T L. (1995) [83] the
Analytic Hierarchy Process allows users to assess the relative weight of multiple criteria
14
(or multiple alternatives against a given criterion) in an intuitive manner. Its major inno-
vation was the introduction of pairwise comparisons. As Saaty T L. (1995) [83] explains,
when quantitative ratings are unavailable, humans are still adept at recognizing whether one
criteria is more important than another.
AHP multi criteria decision making has some recognised limitations, because judgments
in AHP are relative by nature, changing the set of alternatives may change the decision
scores of all the alternatives. It has been shown that even if a new, very poor alternative is
added to a completed model, the alternatives with top scores sometimes reverse their relative
ranking. Early in the development of AHP this issue was debated in the literature (see, for
example, Harker and Vargas (1987) [35] and Dyer (1990) [23]. This approach produces real
numbers as results, so they can easily be misinterpreted as the true differences between the
alternatives rather than the relative ranking. With AHP it is still difficult to assign weights
when the number of criteria is large. Saaty (1994) [83] emphasizes that greater consistency
does not imply greater accuracy, and judgments should be altered only if compatible with
one’s understanding. Otherwise, more information may be necessary or the hierarchy may
need reexamination.
2.4.2 The Simple Multi-Attribute Rating Technique (SMART)

and Software Selection
Browse da Costa, and Ana Paula 2005 [13] used SMART multi criteria decision making
method to select a decision analysis software. In the Simple Multi-Attribute Rating Tech-
nique, as Browse da Costa and Ana Paula 2005 [13] explain, ratings of alternatives are
assigned directly, in the natural scales of the criteria (where available). In order to keep the
weighting of criteria and rating of alternatives as separate as possible, the different scales of
criteria need to be converted to a common internal scale by means of a value function. The
simplest choice of a value function is a linear function, and in most cases this is sufficient.
The advantage of this is that the ratings of alternatives are not relative, so that changing the
number of alternatives considered will not, in itself, change the decision scores of the original
alternatives. The introduction of value functions somewhat complicates the decision model-
ing process. George et al.2005 [27] also questiones the possible mistakes that happen during
the elicitation process, the trade off among the consequences of the modeling mistakes and
the mistakes generated in the elicitation process is also not clear. In this sense the modeling
mistakes are considered as the cost of the simplifications done through the hypotheses of
15
linearity of the one-dimensional utility functions and of addictivity. George et al.2005 affirm
that hypothetical judgments are unreal and they don’t represent the real preferences, and it
bores decision makers, leading them to reject the elicitation process or to accept any answer
with the intention of finishing the questionnaire faster.
2.4.3 Electre and software selection
Another option for software selection is the outranking approach using the Electre family
of methods. These methods according to Buchanan and Vanderpooten (2007) [15] rank
each candidate on each attribute and determine an outranking relationship to categorise
attributes into preferred and non-preferred Roy (1991) [78]. Comparisons are made between
candidates on each attribute, removing the issue of units and attribute types. The method
has been applied in a wide variety of applications, including expert systems by Vlahavas
et al. (1999) [101], electricity utility projects by Buchanan and Vanderpooten (2007) [15],
Subramanian and Gershon (1991) [91], Morisio and Tsoukis (1997) [62]. Although it has been
successfully used for software evaluation and selection see for example Anderson (1989) [1]
Stamelos et al. (2000) [90] and Morisio 1997 [62], this method has a limitation where there
are issues with explaining the reasoning for decisions and a complete ranking may not be
possible (Kunda 2003) [51].
2.4.4 Using Fuzzy Multi-Criteria Decision-Making to Software

Selection
Fuzzy set theory as explained by Hua-Yang Lin et al. (2006) [40] is primarily concerned with
quantifying the vagueness in human thoughts and perceptions, where linguistic terms can
be properly represented by the approximate reasoning. The importance weights of various
criteria and the rating values of the software alternatives are considered as linguistic terms,
that is, as a variable whose values are not numbers, but words or sentences in natural
language. The importance weight can be evaluated by linguistic terms such as very low,
low, medium, high and very high. see Hua-Yang Lin et al. (2006) [40]. Hua-Yang Lin et
al. (2006) [40] proposed a fuzzy multi-criteria decision-making procedure, to facilitate data
warehouse system selection.
16
2.4.5 Weighted Sum Method (WSM) and Software Selection
Much literature reviewed uses the Weighted Sum Method (WSM) to aggregate a value by
summing attribute weights multiplied by their respective values when selecting software see
for example; Williams 1992. [103], Solberg and Dahl (2001) [69], Maiden and Ncube (1998)
[70] , Valerie et al.(2004) [99]. Criticism of the WSM includes the summing of differing
types of data (e.g. cost plus memory plus quality), lack of process for determining attribute
weights and the inherent problem with the formula losing dependency information between
attributes (e.g. conflicts and co-requisites) (Valerie et al.2004).
2.5 Chapter Conclusion
Emphasis in this literature review has been on identifying the major criteria used for soft-
ware selection and how to formulate a software selection support. The criteria identified
are adopted in the instrument in Appendix B. The major criteria identified include; cost,
vendor, functionality and usability. The cost criteria examines issues like the total cost of
ownership including product costs, implementation and training costs, maintenance costs,
software subscription costs, and any annual software license costs. Usability checks issues
like ease of software administration, ease of use, automated, easy to repair, easy fault identi-
fication and minimal errors. Vendor checks issues like vendor reputation, training provided,
implementation partner, support and maintenance, consultancy services and care about the
business. The literature further emphasised functionality as an important criteria for soft-
ware selection and it examines issues like; flexibility, quality, software reliability , support
and maintenance and scalability.
Attempts to apply multi criteria decision making to software selection have been inconclusive
and it has been noted that multi criteria decision making to software selection is an important
aspect that has received little attention.
17
Chapter 3
Methodology
This chapter describes the steps and procedures that were followed in order to accomplish
the research. These included research design, study population, data collection method and
instrument, reliability and validity, data analyis.
3.1 Research Design
A survey was administered to a selected sample from a specific population identified by

the reseacher. The term survey is commonly applied to a research methodology designed
to collect data from a specific population, or a sample from that population, and typically
utilizes a questionnaire or an interview as the survey instrument Hopwood (2004) [39]. The
research design was quantitative and qualitative. Data collection was largely quantitative
basing on use of questionnaires.
3.2 Study Population
Targeted population included the heads of departments who actually purchase software be-
cause they are engaged in decision making. These included heads of department from:
Makerere University faculty of computing and information technology, Makerere University
Business School department of business computing and management science, Kyambogo Uni-
versity department of computing and information technology, Mulago Hospital information
technology department and Ministry of Tourism information technology department.
18
3.3 Data Collection Method and Instrument
The questionnaire method was used as a means of data collection because of its merits.
According to Kirakowski, (2000) [48] there are distinct advantages in using a questionnaire
including; cost effectiveness and ease to analyse data among the rest. The questionnaire
method was used to investigate issues related to software selection. The self administered
questionnaire (SAQ) in this case was the data collection instrument that was used. The
SAQ was divided into three sections: section A the background, section B software selection
criteria, and section C product characteristics in the software selection process. For the
SAQ see Appendix B. The researcher developed the questionnaire by identification of the
dependent and independent variables. The survey items in this study were developed as a
result of an analysis of previous studies, and a review of the literature. The criteria identified
which was actually adopted for the questionnaire included: cost with its issues of product
costs, implementation and training costs, maintenance costs, software subscription costs,
annual software license costs. Vendor objective with issues like vendor reputation , training
provided , implementation partner, support and maintenance, consultancy services and care
about the business. Another criteria identified and considered was usability that included
issues of ease of software administration, ease of use, automated, easy to repair, easy fault
identification and minimal errors. Functionality is another criteria that examined issues
of flexibility, quality, software reliability, support and maintenance and scalability. The
above criteria was identified as dependent variables and were categorised into cost objective,
vendor objective, functionality objective, and usability objective. The independent variable;
product characteristics included the following aspects; strategic fitness, compatibility with
existing hard ware, compatibility with existing operation software, consistency with interface
and userfriendly operations, having complete functionality, having high system reliability
(stability and recovery ability), accessibility and quality of product support, designed to
minimize implementation time, and ease of installation.
3.4 Validity of the Self Administered Questionnaire

(SAQ)
For validity the researcher had people with diverse backgrounds and viewpoints review the
survey questionnaire before it was administered, as advocated by Teijlingen (2001) [96]. This
involved: finding out if each item is clear and easily understood, finding out if respondents
19
interpreted each item in the intended way, finding out whether items have an intuitive
relationship to the study’s topic and goals, and finding out whether the researcher’s intent
behind each item is clear to colleagues knowledgeable about the subject. The pilot survey,
[see Appendix A] was distributed to the heads of departments in the faculties of physics, food
science and technology , computing and information technology of Makerere University. The
researcher used the Likerts’s scale: not influential, didn’t affect my decision, fair, influential,
very influential, in rating the experts views. The likert’s scale was recommended by experts
since this provides standardised response categorises in survey questionnaires.
Finally, the respondents were asked, whether there were any other issues that they thought
should be included in the survey. Results of the responses and questions were collected
and analyzed. At first the questionnaire included the problem issues that affected software
selection but these were dropped since the experts recommended that they were not necessary
since they were not helping the researcher to achieve the objectives. The problems were
instead recommended by the researcher to be studied by future researchers.
The final questionnaire was also reviewed and approved by the research department of Mak-
erere University and the researcher’s supervisor Dr Tom Wanyama. These procedures re-
sulted in the questionnaire used in this study see (Appendix B.)
3.5 Reliability of the SAQ
Reliability is an important aspect of questionnaire design. According to Golafshani(2003) [28],

Suskie (1996) [93], a perfectly reliable questionnaire elicits consistent responses. Although
a questionnaire is difficult to develop, it is reasonable to design a questionnaire that ap-
proaches a consistent level of response. To achieve the reliability of the research instrument
the researcher used the following prescription as reported by Kirakowski, (2000) [48]: using
precise terminology in phrasing the questions, writing the questions as simple as possible,
avoiding unwarranted assumptions about the respondents, avoiding double-barreled ques-
tions, choosing an appropriate response format, and pretesting the questionnaire see (3.4
above.)
20
3.6 Data Analysis
The data analysis consisted of examining the SAQ for correctness and completeness. Data
was edited by eliminating questionaires with inconsistencies and summarised as table. Data
was then compiled, sorted, edited, classsified , coded into a coding sheet and imported to
SPSS (statistical package for social scinces). Data was entered into data table, using SPSS,
and analysed with one way Anova by following the steps in SPSS software and Anova. In
this study a one-way Anova, or single factor Anova, was used because it tests differences
between groups that are only classified on one independent variable. The one independent
variable of this study is software project characteristics. The dependent variable was entered,
the independent variable was also entered in factor. Of all the information presented in the
Anova tables, the major interest of the researcher is focused on the significance value. If
the numbers found in this column are less than the critical value set by the experimenter,
then the effect is said to be significant. King and Minium (2003) [47], explain that the
value is usually set at .05, and this is where the researcher placed the value. Any value
less than this resulted in significant effects, while any value greater than this value resulted
in nonsignificant effects. One potential drawback to an Anova is loss of specificity: All an
F tells you is that there is a significant difference between groups, not which groups are
significantly different from each other. To test for this, a post-hoc comparison can be used
to find out which groups are significantly different from each other and which are not. Some
commonly used post-hoc comparisons are Scheffes and Tukeys. But the researcher did not
do this because of time and the scope of the study. However, the significant factors identified
from the survey were used in the next step of the research, see (chapter 4) to formulate a
multi criteria decision making support for software selection.
3.7 Approach to Formulate a Multi Criteria Decision

Making Support for Software Selection
The support was based on the steps used to illustrate the data warehouse system selection
process, proposed by Hua-Yang et al. (2006) [40]. Based on the results of the data collected,
the AHP decision making theory was partially used to come up with the support for soft-
ware selection in which appropriate criteria were specified to provide detailed guidance for
software evaluation. This involved: structuring a decision problem into a hierarchy helping
to understand and simplify the problem and the prioriotization of weights of criteria show-
21
ing the concerns and preferences of decision makers as borrowed from the AHP approach.
Therefore the research incorporated ideas of Hua-Yang et al. (2006) [40] as used in data
warehouse system selection procedure. The first two steps and the last two steps of the
procedure were incorporated with the AHP decision making theory to come up with the 5
step decision making procedure called a multi criteria decision making support to software
selection.
This chapter basically showed several approaches the researcher used to achieve the ob-
jectives. The research design was quantitative and qualitative. Data collection was largely
quantitative basing on use of questionnaires, and qualitative using descriptive means in anal-
ysis of data. The questionnaire method was used to investigate issues related to software
selection. The self administered questionnaire (SAQ) in this case was the data collection
instrument that was used. Targeted population included the heads of departments who
actually purchase software because they are engaged in decision making. For validity the
researcher had people with diverse backgrounds and viewpoints to review the survey before
it was administered. Data was entered into data table, using SPSS, data was analysed with
one way Anova. The criteria were then used in formulating a multi criteria decision making
support for software selection, combining the process suggested by of Hua-Yang et al. (2006)
[40] with the AHP approach.
22
Chapter 4
Data Presentation, Analysis and

Interpretation of Results
The chapter covers, description of background variables, identification of issues related to

multi criteria decision making in software selection, and formulation of a multi criteria deci-
sion making support for software selection.
4.1 Survey Responses
The total number of responses to the survey were 48, of which 42 were usable and represen-
tative of the intended population. Some institutions could not respond due to the limitations
enshrined in their ICT policies. The sample size was enough and the intended audience was
reached.
4.2 Description of the Background Variables
Usable data were collected from a total of 42 respondents from the different institutions.
71% were males and 29% were females as shown below in table 4.1. This is an indication
that there are more males in the IT field than females.
23
Table 4.1: Response According to Gender
Gender Frequency percentage
female 12 29
Male 30 71
Total 42 100
Table 4.2 shows that of those who responded to the study 41%, were between 20 and 29
years, 55% were between 30 and 39 years, while only two percent were between 40 and 49
years. Two percent had experience in this study, less than 1 year, 26% had 2 to 3 years
experience, 45% had 4 to 5 years, 21% had 6 to 7 years, 5% had above 8 years of experience
in this study. So according to the findings, most people in software selection are young, ie
below 40 years.
24
Table 4.2: Response According to the Age and Experience
Age Groups Frequency percentage
Less than 19 years 1 2
20-29 years 17 41
30-39 years 23 55
40-49 years 1 2
Total 42 100.00
Experience
Less than one year 1 2
2-3 years 11 26
4-5 years 19 45
6-7 years 9 21
Above 8 years 2 5
Total 42 100
4.3 Issues Related to Multi Criteria Decision Making

in Software Selection
The data were analysed by using the analysis of variance ( Anova). The groups in this case
were cost objective, vendor objective, functionality objective, and usability objective which
were also the dependent variables. These were classified on to the independent variable,
product characteristics in software selection process.
The Anova software used reports sum of squares, the degrees of freedom, the mean squares,
the F-statistic, and the significance computed between groups, within groups and as a total,
See Ferguson and Takane (2005) [25]. In Table 4.3, df is for degrees of freedom, this is the
number of independent variables. The mean square is the sums of squares divided by the
corresponding degrees of freedom. (F) or F-ratio is the ratio of the mean squares. And sig.
is the significance of the F ratio. The only columns that are critical for interpretation are
the first and the last. The others are used mainly for intermediate computational purposes
as shown in tables 4.3-4.6 below: The total amount of variability in the response is the total
sum of squares.
Table 4.3: Analysis of Variance of Cost According to Product Characteristics:
25
Findings indicate that F (9, 32)= 1.637, p<0.01 was highest at product cost. There were
significant differences in the means of cost as per training costs and mantainance costs F
(9, 32)= 1.558, p< .012 and F (9, 32)= 1.282, p<.013 respectively. All the variations of the
other types of cost according to product characteristics were significant at 0.05 level.
26
Table 4.3: Analysis of Variance of Cost According to Product Characteristics:
Software Selection Variables Sum of Squares df Mean Square F Sig.
Product costs Between Groups 2.213 9 .246 1.637 .01*
Within Groups 9.817 32 .386
Total 14.119 41
Training costs Between Groups 4.302 9 .478 1.558 .012*
Total 14.571 41
License costs Between Groups 2.477 9 .275 .914 .526
Total 12.119 41
Maintenance costs Between Groups 1.977 9 .220 1.282 .013*
Total 13.619 41
Software subscription costs Between Groups 2.910 9 .323 0.246 .284
Total 10.976 41
Cost is a common factor influencing the purchaser to choose the software as noted by Gupta
and Sahay (2003 [29], Ngai and Chan (2005) [64]. In this research cost included issues like;
product costs, license costs, training costs, maintenance and software subscription costs.
However findings indicate that only product costs, training costs and maintenance costs are
highly significant issues in the decision making during software selection.
Table 4.4: Analysis of Variance of Vendor According to Product Characteristics:
Findings indicate that F (9, 32)= 2.105, p<0.04 was highest at vendor reputation. There
were significant differences in the means of vendor as per training provided F (9, 32)= 1.397,
p< 11.
27
Table 4.4: Analysis of Variance of Vendor According to Product Characteristics:
Vendor reputation Between Groups 9.101 9 1.011 2.105 .04*
Total 24.476 41
The training provided Between Groups 3.546 9 .394 1.397 .011*
Total 22.571 41
Implementation partner Between Groups 4.930 9 .548 1.397 .436
Total 21.905 41
Support& maintenance Between Groups 9.796 9 1.088 .114 .382
Total 19.071 41
Consultancy services Between Groups 4.963 9 .551 .181 .340
Total 18.905 41
Care about the business Between Groups 2.261 9 .251 .092 .395
Total 9.619 41
The quality of vendor support and its characteristics are of major importance in the se-
lection of software, as was also found by Mulebeke and Zheng (2006) [63]. The successful
installation and maintenance of the software were also found to be critical. Of the vendor
issues considered in this research (vendor reputation, the training provided, the implemen-
tation partner, support and maintenance and consultancy services) only. Vendor reputation
and the training provided were found to be highly considered during the process of decision
making in software selection.
28
Table 4.5: Analysis of Variance of Functionality According to Product Charac-
teristics:
Findings indicate that F (9, 32)= 1.202, p<.028 was highest at quality.
29
Table 4.5: Analysis of Variance of Functionality According to Product Character-
istics:
Flexibility Between Groups 2.796 9 .311 1.202 .328
Total 11.071 41
Quality Between Groups 3.951 9 .439 1.202 .028*
Total 15.643 41
Software reliability Between Groups 2.726 9 .303 .750 .761
Total 15.643 41
Scalability Between Groups 2.594 9 .288 .472 .882
Total 22.119 41
Ngai and Chan (2003) [64] explains that functionality refers to those features that the soft-
ware performs and to how well the software can meet the user’s needs and requirements.
Therefore functionality is always considered when selecting software, such as in Gupta and
Sahay (2003) [29], Ngai and Chan (2005) [64]. The functionality issues which were con-
sidered in this reaserch were; flexibility, quality, software reliability, and scalability only.
However findings indicate that only quality is highly considered when selecting software.
30
Table 4.6 Analysis of Variance of Usability According to Product Characteristics:
Findings indicate that F (9, 32)= 1.677, p<0.11 was highest at ease of use. There were
significant differences in the means of usability as per ease of use and minimal errors: F (9,
32)= 1.677, p<0.011 and F (9, 32)= 1.162, p<.024 respectively. All the other variations of
usability according to product characteristics were not significant at 0.05 level as shown in
the table below.
31
Table 4.6: Analysis of Variance of Usability According to Product Characteristics:
Ease of S/W administration Between Groups 1.302 9 .145 .733 .676
Total 7.619 41
Ease of use Between Groups 3.519 9 .791 1.677 .011*
Total 14.286 41
Automated Between Groups 3.351 9 .372 .875 .557
Total 16.976 41
Easy to repair Between Groups 5.363 9 .596 .718 .688
Total 31.905 41
Easy fault identification Between Groups 4.192 9 .466 .450 .897
Within Groups 33.142 32 1.036
Total 37.333 41
Minimal errors Between Groups 5.802 9 .645 1.162 .024*
Total 36.286 41
Mulebeke and Zheng (2006) [63] explain that usability is about overcoming technical short
comings and interoperability issues. It can include easy identification of problems in software
and trouble shooting, and the ability to help decision-makers make decisions that solve
problems. The usability issues that affect the decision to select the software that were
considered were; ease of software administration, ease of use, automated, easy to repair,
easy fault identification and minimal errors. Findings indicate that the ease of use issue is
highly considered when selecting the software. This was also found by Oyku (2005)[67] who
concluded that an end user demands easy-to-use and easy-to-learn software. So, the criteria
would be shaped according to the users needs.
32
4.4 Identification of the Criteria Used for Multi Crite-
ria Decision Making in Software Selection
As discussed in the preceding sections, in order to identify the criteria used in decision making
for software selection the researcher visited the work of several authors who considered
different aspects in software selection. Literature shows that the criteria used in software
selection is wide, and the researcher reduced these to the components of ; cost, vendor,
functionality, and usability. The findings reported in the previous subsection indicate that
these criteria are important, although not all aspects of these components reported in the
literature were found to be significant in software selection.
4.5 Formulation of a Multi Criteria Decision Making

Support for Software Selection
The support, which can also be called a software selection process model, is illustrated in
Fig 4.1. It is based on the steps of data warehouse system selection procedure by Hua-Yang
et al. (2006) [40]. Criteria identified as most important in the data analysis were used in an
AHP decision making process to come up with the support for software selection, the steps
in the formulated multi-criteria decision making support are explained below:
4.5.1 Make a Team of Decision Makers
The first step in the support is to group the responsible team of decision makers together to
have a common voice. In this case the research suggests that a team to support the process
of decision making for software selection may include, top managers or decision makers,
executive managers, functional experts, users or their representatives. The involvement of
users greatly influences the adoption of the system because they are the ones going to use
the system hence should be involved in the decision making about which software is best
for them, (Birdogan and Kemal, 2005) [10]. The participation and support of top managers
significantly influences the success of software purchase according to (Hua-Yang Lin et al.,
2006) [40].
33
4.5.2 Identify the Software Project Characteristics
The second step in the support is to identify the software project characteristics. Many
organizations may purchase different or even same software for completely different reasons.
Decision makers must be able to tell the business software vendors exactly what they want
because their expectations may differ from the software offered on the market. Software
requirements should be clearly defined and the choice of software should be based on the
software requirements. In this case the reason for the software purchase should be identi-
fied, during the processes analysis period, the functional characteristics of required software
should be recognized (Birdogan and Kemal, 2006 [10], Mohsen et al., 2005) [60]. This will
affect problem definition, identifying and structuring objectives, measuring the achievement
of objectives, and other subsequent decision-making activities (Hua-Yang et al., 2006) [40]).
According to Franch and Carvallo (2003) [26] the absence of structured descriptions of quality
features and user quality requirements makes selecting the right software package difficult.
On the basis of the literature review reported in chapter 2, this research considered the fol-
lowing aspects to be the most important software project characteristics: strategic fitness,
compatibility with existing hardware, compatibility with existing operation software, con-
sistency with interface and userfriendly operations, having complete functionality, having
high system reliability (stability and recovery ability), accessibility and quality of product
support, designed to minimize implementation time, and ease of installation.
4.5.3 Evaluate the Software by the AHP Decision Making Ap-

proach
The third step is to evaluate the software by the AHP decision making approach which was
described in chapter 2. The first step in the AHP decision making approach is to develop
a hierarchical structure to define a single pre-defined goal, the decision criteria supporting
that goal, and potential sub-criteria supporting each criterion as shown below ( fig 4.2). All
criteria and sub-criteria eventually contribute to the goal. A list of alternatives provides the
decision points that are evaluated against this hierarchy. In this research the over all goal is
to select the best software and this is reprented as level 1. Level 2 represents the criteria ie
cost, vendor, functionality and usability, and the sub-criteria used in selecting the software,
level 3 contains the decision alternatives that affect the ultimate selection of the software ie
alternatives 1, 2, 3 see Fig. 4.2.
34
Table 4.7: Judgement scores for the importance/preference of criteria using AHP
Verbal Judgement Numerical rating
Extremely important/preferred 9
Very strongly to Extremely important/preferred 8
Very strongly important/preferred 7
Strongly to very strongly important/preferred 6
Strongly important/preferred 5
Moderately to strongly important/preferred 4
Moderately important/preferred 3
Equally to moderately important/preferred 2
Equally important/preferred 1
4.5.4 Derive the Priorities for each Alternative Reflecting the De-
gree to which the Alternative Satisfies the Goal
The fourth step is to derive the priorities for each alternative reflecting the degree to which
the alternative satisfies the goal. Once the AHP model is set up, and the priority of each
criterion and sub-criterion developed in the hierarchy, then the decision makers can evaluate
the alternatives. This involves a set of pairwise comparisons, between each alternative, eval-
uated against each criterion and sub-criterion as shown in fig 4.2. When these comparisons
are finished, each alternative will have a derived priority, representing how well that alterna-
tive satisfies the pre-defined goal. Weights are assigned to each criterion and sub-criterion.
These weights are assigned through a process called pairwise comparison. In pairwise com-
parison, each objective is compared at a peer level in terms of importance. Comparisons
can be done by verbal comparison, decision-makers compare criteria for their relative impor-
tance and alternatives for their relative preferences, using words such as equal, moderate,
strong, very strong and extreme. Thus, decision makers when using pairwise comparison
may ask questions like how important is the vendor factor to the selection of software? The
answer can be equal, moderate, etc. The verbal responses are then quantified and translated
in a score through the nine-point scale shown in Table 4.7. adapted from Ngai and Chan,
(2005) [64].
35
For each identified software alternative, one would ask, does any of the identified alternative
satisfy the goal? If the answer is yes then , such an alternative is considered a better choice.
If the response is ”no” the alternative does not satisfy the goal then the evaluation process
is repeated to determine the alternative that satisfies the goal.
4.5.5 Make the Final Decision
The fifth and the last step is to make the final decision. After comparisons have been done
and the alternative that satisfies the goal determined, the software with the highest score
will be the final choice.
36
Figure 4.1: A multi criteria decision making support to software selection
adapted from Hua-Yang et al. (2006) [40]
37
Figure 4.2: Evaluation of the software by the AHP decision making approach
38
The chapter has identified, issues related to decision making in software selection, and criteria
to be used for multi criteria decision making in software selection then formulated a multi
criteria decision making support for software selection based on AHP.
39
Chapter 5
Discussion, Conclusion and

Recommendations
This research dealt with multi criteria decision making in software selection. The first ob-
jective of the study was to investigate issues related to multi criteria decision making in
software selection. Findings from the literature review reported in chapter 2 indicated that
product costs, training costs, and maintenance costs are the issues under cost objective that
are highly significant when selecting software. Vendor selection, vendor reputation, the train-
ing provided, support and maintenance are highly considered issues when selecting software
which agreed with the findings of Farbey et al.(2006) [24]. Functionality and usability are
other important issues.
The second objective of the study was to identify criteria used for multi criteria decision
making in software selection. cost, vendor, functionality, and usability, were the main criteria
considered in a survey of relevant experts. Findings show that cost of the package including
product costs, training costs and maintenance costs, is a very important criteria that is
always considered when selecting a software package; this was also confirmed by Ngai and
Chan (2005) [64] and Mulebeke and Zheng (2006) [63]. Vendor selection including vendor
reputation, the training provided support and maintenance was also considered an important
criteria in the selection of a software, as it was in Mulebeke and Zheng (2006) [63]. Findings
show that quality issues are the most important aspect of functionality, which is also an
important criteria. Usability, including the ease of use issue, is highly considered when
selecting the software. Therefore the criteria identified in this research to be used for multi
criteria decision making in software selection, are; cost, vendor, functionality and usability.
40
The third objective was to formulate a multi criteria decision making support for software
selection. A simple, easy to use and systematic multi-criteria decision making support to
software selection, has been formulated based on the AHP decision making theory and ideas
of Hua-Yang et al. (2006) [40]. The support is to aid decision makers in the selection
of software among alternatives, with several decision criteria. The multi criteria decision
making support is comprehensive and can be applied to selection of any kind of software
solution. The multi-criteria decision making support presented in Fig. 4.2. was the main
contribution of the research, as it provides a useful guide as a structured and logical means
of synthesizing judgements for evaluating appropriate software for an organization. Where
as Hua-Yang et al. (2006) [40] based his selection procedure on the fuzzy set theory, the
support formulated in this research is based on AHP multi criteria decision making theory.
5.1 Limitation of the Study
The research was limited to multi criteria decision making support for software selection and
could not look at decision making in other areas. Besides that, the organisations that were
covered were mainly Ugandan academic, results could be different if the study was conducted,
say, in the health or banking sectors, or in other countries. The study population was limited
by resources, and even the questions were limited to software selection criteria and software
project characteristics. Further, the support was not tested because of time limitations.
Since the multi criteria decision- making support is based on AHP, this support was not
incorporated with the aid of the computer tool, Expert Choice, such that the decision-makers
could compare different scenarios and possibilities with respect to appropriate criteria and
sub-criteria through the sensitivity analysis of AHP. By implementing AHP assessments on
a computer, decision makers could automatically obtain the ranking order of alternatives.
This was not done because of the limited time given to the study.
5.2 Recommendations
Product cost is an important factor in developing countries like Uganda. Companies may
offer very competitive and low basic software cost but charge high rates for additional hard-
ware or special equipment and annual cost. So the total cost of owership should be put into
consideration when selecting software (Gupta and Sahay 2003) [29] . Vendor reputation and
41
the training provided by the vendor is vital and consideration of these factors is therefore
recommended. The multi criteria decision making presented in this thesis support provides
comprehensive evaluation criteria, but it is not exhaustive, to evaluate software that are
rapidly changing (Marwick 2001) [58]. The checklist of the criteria may have to be updated.
The evaluation criteria can be further refined so that softwares can be evaluated for many
different environments. The importance of each criterion may vary, under different require-
ments and situations, therefore decision makers are recommended to describe their desired
values, and the importance of a criterion considered for decision making in software selec-
tion. AHP is a methodology that is appropriate to use during a software selection process,
as it is a structured decision making problem, and so especially useful for complex software
systems selection. However, AHP should not be limited to software selection alone it can be
used to a wide range of decision-making problems with multi-attributes and alternatives as
recommended by Oyku (2005)[67].
5.3 Further Work
The study has related multi criteria decision-making to software selection. Other issues
related to software selection were left behind for example problems associated with software
selection. Perhaps it would be better to incorporate these problems and other issues of the
software selection process to provide better solution to decision makers. Since the multi
criteria decision- making support is based on AHP, future studies can incorporate this work
with the aid of a computer tool, such as Expert Choice, (Saaty T L.1992, [83]Saaty T L.
1995) [83] so that the decision-makers can compare different scenarios and possibilities with
respect to appropriate criteria and sub-criteria through the sensitivity analysis of AHP. The
study can also be extended to cover other social sectors like, banking sectors, and other
countries.
This chapter has been vital in putting forward the discussions and conclusions on issues
related to multi criteria decision making in software selection, findings indicated that product
costs, training costs, vendor reputation, quality and ease of use are major issues considered
in the decision making process for software selection. The criteria identified in this research
42
that are used for multi criteria decision making in software selection, include; cost, vendor,
functionality and usability. A simple, easy to use and systematic multi-criteria decision
making support to software selection, has been formulated. The support is based on the
AHP decision making theory, and ideas of Hua-Yang et al. (2006) [40] as used in data
warehouse selection procedure. The research was limited to multi criteria decision making
support for software selection and could not look at decision making in other areas in support
for software selection. The study because of the limited nature of the survey could not cover a
larger population, and even the questions were only limited to software selection criteria and
software project characteristics, many variables were not looked into for example problems
associated with software selection.
43
References
1. Anderson E. . A Heuristic for Software Evaluation and Selection, Software Practice

and Experience,, 19 (8): 707-717,
2. Avshalom, A. (2000). A new approach to ERP customisation. Retrieved on April,

4th, 2006 from http:// www.erpfans.com/erpfans/eshbel.htm
3. Banks, J. Gibson, R. (2004). Selecting simulation softwares. IIE Solutions, 29(5):

30-33.
4. Bandor, M. S.(2006). Quantitative Methods for Software Selection and Evaluation

Technical Note , 26 Acquisition Support Program.
5. Baudry, M. and Vincent N. (2002). multi criteria Decision Making using multiattribute
utility theory. Retrieved on 4th january 2007 from http:www.univ-tours.fr/ed/edss/comm
2002/baudry,pdf
6. Beck M.P. and Lin B.W.,(1981) Selection of automated office systems: A case study,
OMEGA, 9 (2): 169176
7. Belton, V. and Steward, T.J. (2002). Multiple Criteria Decision Analysis-An Integrated
Approach, Kluwer Academic Publishers: Boston.
8. Bernroider and Koch,E. S. (2001). ERP selection process in midsize and large organi-
zations. Business Process Management Journal, 7(3): 251-257.
9. Bingi, P., Sharma, M.K. and Godla, J.K. (1999). Critical issues affecting an ERP
implementation. Information Systems Management, 16 (3): 7-12.
10. Birdogan B.,Kemal C., (2005) Determining the ERP package-selecting criteria Business
Process Management Journal ,11 (1) : 75-86
11. Boloix, G. and Robillard, P.N. (1995). A software system evaluation framework. IEEE
Computer, 28 (12): 17-26.
12. Brewer, G. (2000). On the road to successful ERP. Instrumentation and Control Sys-
tems, 73(5): 49-58.
13. Browse da Costa, Ana Paula C.S.(March 1 2005) Multicriteria decision making on
selection of decision analysis software Journal of Academy of Business and Economics
retrieved on 20th november 2007 from http://www.thefreelibrary.com/
44
14. Burmark,L. Thornberg,D. (2000), Criterion for software selection for teaching and
learning, www.astd.org/virtual,community/forums/learning,tech/. Retrieved on 30th
June 2006 from http://www.emeraldinsight.com.
15. Buchanan, John; Vanderpooten, Daniel (4, July 2007) Ranking projects for an electric-
ity utility using ELECTRE III International Transactions in Operational Research,,
14,(15) : 309-323 Publisher: Blackwell Publishing
16. Carney, D.J. and Wallnau, K.C. (1998). A basis for evaluation of commercial software.
Information and Software Technology , 40 (14): 851-860.
17. Capterra(2005).The enterprise software center. Retrieved on April, 19th, 2006 from
http://www.capterra.com
18. Chou Y., Lee, C., and Chung, J. (2004). Understanding m-commerce payment systems
through the analytic hierarchy process. Journal of Business Research, 57 (12): 1423-
1430
19. Chung W. W. C. , Chik, S. K. O. (2001), ”Computerization strategy for small man-

ufacturing enterprises in Hong Kong”, International Journal of Computer Integrated
Manufacturing,, 14 (2): 141-153.
20. Davies, M A. (1994). A multi criteria decision model for managing group decisions.
Journal of the Operational Research Society , 45 (1) : 47-58.
21. Davis L. and Williams G.,(1994) Evaluation and selecting simulation software using
the analytic hierarchy process, Integrated Manufacturing Systems, 5 (1):2332
22. Cochran J.K. and Chen H.,(2005) Fuzzy multi-criteria selection of object-oriented sim-
ulation software for production system analysis, Computers and Operations Research,
32 (1): 153168
23. Dyer J. S. (1990). Remarks on the analytic hierarchy process. Management Science ,
36 (3) : 249-258
24. Farbey, B., Land, F. and Targett, D. (2006). Investments: A Study of Methods and
Practices. Butterworth-Heinemann: Oxford.
25. Ferguson, G. A., Takane, Y. (2005). Statistical Analysis in Psychology and Education,
Sixth Edition. Montral, Quebec: McGraw-Hill Ryerson Limited.
45
26. Franch, X.Carvallo P. (Jan 2003), Using Quality Models in Software Package Selection,
IEEE Software: 34-41
27. George Valiris,Panagiotis Chytas,Michael Glykas,(2005) Making decisions using the

balanced scorecard and the simple multi-attribute rating technique Performance Mea-
surement and Metrics ,6: 159-171
28. Nahid Golafshani(December 2003) Understanding Reliability and Validity in Qualita-

tive Research. The Qualitative Report, 8 (4): 597-607
29. Gupta, A. K. and Sahay, B. S. (2003) .Development of software selection criteria for
supply chain solutions Industrial Management and Data Systems, 103 (2): 97-110
30. Gusdorf, J. (Sep 11, 2006) Software Selection ProcessesAccelerating Vendor Identifi-
cation. Retrieved on 24th April 2007 from http://www.technology-evaluation.com
31. Gulfem, I. and Gulcin B. (February 2007). Using a multi-criteria decision making
approach to evaluate mobile phone alternatives. Computer Standards & Interfaces ,
29 (2): 265-274
32. Jie, L., Guangquan, Z. and Fengjie, W. (2005). Web-based multi-criteria group decision
support system with linguistic term Processing Function IEEE Intelligent Informatics
Bulletin, 5 (1).
33. Kim J. and Moon J.Y.,(1997) An AHP and survey for selecting workflow management
systems, International Journal of Intelligent Systems in Accounting, Finance,and Man-
agement 6: 141161.
34. Kim C.S.and Yoon Y.,(1992) Selection of a good expert system shell for instructional
purposes in business, Information and Management, 23 (5): 249262.
35. Harker P T. and Vargas L G (1987). The theory of ratio scale estimation: Saaty’s
analytic hierarchy process. Management Science 33(11), 1383-1403.
36. Hmlinen, R. P., Mustajoki, J. and Alanaatu, P.(2003). Smart swaps Smart choices
with the even swaps method. Helsinki University of Technology: Computer software,
Systems Analysis Laboratory, Helsinki University of Technology. Retrieved on May,
10th, 2006 from http://www.smart-swaps.hut.fi
37. Hecht, B.(1997). Choose the right ERP software. Datamation, 43 (3): 56-58.
46
38. Hluoic and Paul, R.J. (1996). Methodological approach to manufacturing simulation
software selection. Computer Integrated Manufacturing Systems, 9 (1): 49-55.
39. Hopwood, N. (2004). Research design and methods of data collection and analysis: re-
searching students conceptions in a multiple-method case study. Journal of Geography
in Higher Education, 28 (2): 347-353.
40. Hua-Yang, L., Ping-Yu, H. and Gwo-Ji, S. (2006). A fuzzy-based decision-making

procedure for data warehouse system selection. Expert Systems with Applications: An
International Journal, 32 (3) : 939-953
41. Hua-Yang, L., Ping-Yu, H. and Yung-Tai, Y. (2004). Application of the AHP in data
warehouse system selection decisions for SMEs in Taiwan. International Journal of
Management and Enterprise Development, 3 (6): 599 - 617
42. http://www.thefreedictionary.com
43. Illa, X. B., Franch, X. and Pastor, J.A. (2000). Formalising ERP selection criteria.
10th International Workshop on Software Specification and Design. Organised by
IEEE, at San Diego, Carlifornia.
44. Jenz and Partner, G. M.B. H. (2005). Research Analysis and consulting with a clear
focus on business process integration.
45. Jung H. and Choi B., (1999)Optimization models for quality and cost of modular
software systems, European Journal of Operational Research, 112 (3): 613619
46. Kimball R., L. Reeves, M. Ross and W. Thornthwaite, (1998) The data warehouse life-
cycle toolkitexpert methods for designing, developing, and deploying data warehouses,
John Wiley and Sons, NY (1998).
47. King, B. M., Minium, E. W. (2003). Statistical Reasoning in Psychology and Education,
Fourth Edition. Hoboken, New Jersey: John Wiley and Sons, Inc.
48. Kirakowski, J. (2000). Questionnaires in usability engineering a list of frequently asked

questions. 3rd Ed. Human Factors Research Group: Cork.
49. Kumar, V., Maheshwari, B. and Kumar, U. (2003). An investigation of critical manage-
ment issues in ERP implementation: empirical evidence from Canadian organization’s.
Technovation, 23: 793-807.
47
50. Lai V.S. , Trueblood R.P. and Wong B.K., (1999)Software selection: A case study of
the application of the analytic hierarchical process to the selection of a multimedia
authoring system, Information and Management, 36 (4): 221232.
51. Kunda D. STACE(2003). Social Technical Approach to COTS Software Evaluation,

Component-Based Software Quality, LNCS 2693,:64-84, Springer- Verlag Berlin Hei-
delberg,
52. Lai V.S., Wong B.K. and Cheung W., (2002) Group decision making in a multiple
criteria environment: A case using the AHP in software selection, European Journal
of Operational Research 137 (1): 134144.
53. Le Blanc, L. A. and Jelassi M.T.(1989). DSS software selection: a multiple criteria
decision methodology. Information and Management, 17 (1): 49-65.
54. Liang, T.P. and Hung, S.Y. (1997). DSS and EIS applications in Taiwan. Information
Technology and People, 10 (4): 303-315.
55. Liu, C., Wang, M. and Pang, Y.(1994). A multiple criteria linguistic decision model
(MCLDM) for human decision making.European Journal of Operational Research, 76
: 466-485.
56. Mabert, A.V., Soni, A. and Venkataramanan, M.A. (2001). Enterprise resource plan-
ning: common myths versus evolving reality, Business Horizons,: 69-76.
57. Mamaghani, F.(2002). Evaluation and selection of an antivirus and content filtering
software technical paper Information Management Computer Security, 10 (1): 28-32
58. Marwick A.D. (2001). Knowlegde Management. IBM Systems Journal, 40(4): 814-
830.
59. Min H.,(1992) Selection of software: The analytic hierarchy process, International
Journal of Physical Distribution and Logistics Management, 22 (1): 4252.
60. Mohsen ,.Z. ,Mohammad F., Sadjadi S.J.,(2006) A modular approach to ERP system
selection Information Management & Computer Security 14, (5): 485-495
61. Mohanty R.P. and Venkataraman S., (1993) Use of the analytic hierarchy process for
selecting automated manufacturing systems, International Journal of Operations and
Production Management,13 (8): 4557
48
62. Morisio, M. and Tsoukis, A. (1997). Iusware: A methodology for the evaluation and
selection of software products, IEEE Proceedings of Software Engineering 144 (3):
162-174.
63. Mulebeke, James, A. W. and Zheng, L.(2006). Analytical network process for software
selection in product development: a case study Huawei Technologies. Department of
Industrial Engineering, School of Economics and Management, Tsinghua University:
Beijing.
64. Ngai, E.W.T. and Chan, E.W.C. (2005). Evaluation of knowledge management tools
using AHP. Expert Systems with Applications, 29 (4): 889-899
65. Ondrus, J., Tung, B. and Yves, P. (2005). A multi-actor, multi-criteria approach for
technology selection when designing mobile information systems. University of Hawaii:
Manoa.
66. Ossadnik W.and Lange O.,(1999) AHP-based evaluation of AHP software, European
Journal of Operational Research118 (3):578588.
67. Oyku, . A. (2005) ERP selection using expert choice software ISAHP, 47 , held on
July 8-10 2005 Honolulu, Hawaii.
68. Williams F.. 1992. Appraisal and Evaluation of Software Products, Journal of Infor-
mation Science, Principles and Practice, 18, :121-125,
69. Solberg H. and Dahl K. M.,(November, 2001) COTS Software Evaluation and Integra-
tion Issues,Norwegian Institute of Technology and Science
70. Maiden N. A. and Ncube C. (1998) . Acquiring COTS Software Selection Require-
ments, IEEE Software, : 46-56,
71. Phillips-Wren G.E., Hahn E.D. and Forgionne G.A.,(2004) A multiple-criteria frame-
work for evaluation of decision support systems, OMEGA, 32 (4): 323332.
72. Poon P, and Wagner, C. (2001). Critical success factors revisited: success and failure
cases of information systems for senior executives. Decision Support Systems, 30:
393-418.
73. Power, D. J. ( 1997). Tips for choosing enterprise-wide DSS Software, DS*Star. The
On-Line Executive Journal for Data-Intensive Decision Support, 1 (7).
49
74. Power, D. J. (2002). Decision support systems: concepts and resources for managers.
Quorum Books.
75. Rao, S. S. (2000). Enterprise resource planning: business needs and technologies.
Industrial Management and Data Systems, 100 (2): 81-88.
76. Robert, C. K. and William, D.S. (1999). Software selection guide : 31-35.
77. Roper-Lowe G.C. and Sharp J.A.,(1990) The analytic hierarchy process and its appli-
cation to an information technology decision, The Journal of the Operational Research
Society, 41 (1): 4959
78. Roy B.(1991) The Outranking Approach and the Foundations of the ELECTRE Meth-
ods, Theory and Decision, 31 : 49-73, Kluwer Academic Publishers, Netherlands.
79. Saaty, T. L. (2004). Fundamentals of the Analytic network process-multiple networks

with benefits, costs, opportunities and risks. Systems Science and Systems Engineer-
ing, 13 (3): 348.
80. Saaty T. L. and Niemira M.P. (2006). A Framework for making a better decision,
how to make more effective site selection, store closing and other real estate decisions.
Research Review, 13 (1): 96-108
81. Golden, B. L., Harker, P. T. and Wasil E. A.(1989) The Analytic Hierarchy Process -
Applications and Studies. Springer-Verlag, New York.
82. Saaty, Thomas L.(1992) Multicriteria Decision Making - The Analytic Hierarchy Pro-
cess, Pittsburgh, RWS Publications, USA
83. Saaty T L. (1995), Decision Making For Leaders: The Analytic Hierarchy Process for
Decision in a Complex World (3rd ed.). RWS Publications, USA
84. Salo, A., Gustafsson, T. and Ramanathan, R. (2003). multi criteria methods for
technology foresight. Journal of Forecasting, 22 (2): 235-255.
85. Salvatore, G., Figueira J.E and Ehrgott, M.(2003). Multiple criteria decision analysis:
State of art surveys Kluwer Academic Publishers: Boston.
86. Sarkis J.and Talluri S., (2004) Evaluating and selecting commerce software and com-
munication systems for a supply chain, European Journal of Operational Research, 159
(2): 318329.
50
87. Seidmann A.and Arbel A.,(1983) An analytic approach for planning computerized
office systems, OMEGA, 11 (6): 607617.
88. Seidmann A. and Arbel A.,(1984) Microcomputer selection process for organizational
information management, Information and Management, 7 (4): 317329.
89. Shikarpur, D. (1997). The dilemma of buying ERP. India. Retrieved on October, 4th,
2006 from http:// www.dqindia.com/Oct159/3ij1141101.html, .
90. Stamelos, I., Vlahavas, I., Refanidis, I. and Tsoukis, A. (2000). Knowledge based
evaluation of software systems: A case study, Information and Software Technology,
42 (5): 333-345.
91. Subramanian G.H. and Gershon M.,(1991) The selection of computer-aided software
engineering tools: A multi-criteria decision making approach, Decision Sciences, 22
(5) :11091123.
92. Stylianou, A. C., Madey, G.R. and Smith, R.D. (1992). Selection criteria for expert
system shells: a socio-technical framework. Communications of the ACM, 35 (10):
30-48.
93. Suskie, L. (1996). Questionnaire survey research: What works, 2nded. Association for
Institutional Research: Tallahassee.
94. Stewart T J( 2000.) Policy decisions in the public sector: can MCDA make a difference?
In: Haimes YY, Steuer R, editors.Retrieved on 20th April 2007
95. Teltumbde A. (2000), A framework for evaluating ERP projects, International Journal
of Production Research, 38 (17): 45074520
96. Teijlingen, v., Rennie, E., Hundley, A.M., Graham, W.V. (2001). The importance
of conducting and reporting pilot studies: the example of the Scottish Births Survey.
Journal of Advanced Nursing, 34: 289-295.
97. Tom, S.(2003). Suppose you’re a sales professional representing a vendor of specialized
computer systems. 2nd edition. American Management Association : 224
98. Triantaphyllou E (2000) Multi-Criteria Decision Making Methods: A comparative

Study, Applied Optimization 44, Kluwer Academic Publishers
51
99. Valerie Maxville., Jocelyn Armarego., Chiou Peng Lam., (2004) Intelligent Compo-
nent Selection Proceedings of the 28th Annual International Computer Software and
Applications Conference (COMPSAC04) IEEE :0730-3157/04
100. Verville, J. Hallingten, A. (2002). An investigation of the decision process for selecting
an ERP software: the case of ESC. Management Decision, 40 (3): 206-216.
101. Vlahavas, I., Stamelos, Refanidis I. and Tsoukis A.,(1999) ESSE: An expert system for
software evaluation, Knowledge-Based Systems, 12 (4): 183-197.
102. Vicent, S., Lai, B., Wong, W. (2001). Group decision making in a multiple Criteria
Environment. A case using the AHP in software selection.
103. Williams, D. L.G. (2002). Evaluating and selecting simulation software using the
analytic hierarchy process. Integrated Manufacturing System, 5 (1): 23-32.
104. Wei C. Chien C., and M.J. Wang,(2005) An AHP-based approach to ERP system
selection, International Journal of Production Economics, 96 (1): 4762.
105. Zahedi F.,(1985) Database management system evaluation and selection decisions, De-
cision Sciences, 16 (1): 91116.
106. Zahedi F., (1990) A method for quantitative evaluation of expert systems, European
Journal of Operational Research, 48, (1): 136147
107. Zeleny M.(1992) An Essay into a Philosophy of MCDM: A Way of Thinking or Another
Algorithm? Computers and Operations Research, 563-566. Analytic Hierarchy Process
(AHP) Publications
52
APPENDIX A ( a pilot study)
A MULTI CRITERIA DECISION MAKING QUESTIONNAIRE TO SOFTWARE SE-

LECTION
Your name(Optional):
Date:
Name of the Organisation:
NB: the information you provide here is kept completely confidential and no information is
stored on computer media that could identify you as person.
This questionnaire is entirely for academic purpose.
SECTION A: BACKGROUND INFORMATION.
This section is meant to provide the researcher background information for statistical group-
ings. Please place a tick against any option of your choice in the boxes corresponding to the
questions where possible.
Qn no Question coding category

1. kindly tick your gender female or Male
2. What is your age range?
less than 19 years
20-29 years
30-39 years
40-49 years
above 50 years
3. How long have you worked for this organization
Less than one year
2-3 years
4-5 years
6-7 years
Above 8 years
53
Section B: About Your software selection Criteria
Cost objective.
How influential were /are the following cost objective criteria to you in the selection of
software? (Please select an option, which best represents your option by placing a tick
against the answer of your choice. In this case 1=Not influential, 2= didn’t affect my
decision, 3=Fair, 4=Influential 5=Very influential.
qn no Rating 5 4 3 2 1
4. Product cost.
5. Training Costs
6. License costs
7. Maintenance costs
8. Software subscription costs.
About Your software selection Criteria
Vendor objective.
How influential were /are the following Vendor objective criteria to you in the selection
of software? (Please select an option, which best represents your option by placing a tick
against the answer of your choice. In this case 1=Not influential, 2= didn’t affect my decision,
3=Fair, 4=Influential 5=Very influential.
9. Vendor reputation .
10. Training provided
11. Implementation partner
12. Support and maintenance
13. Consultancy services.
14. Care about the business
54
Functionality objective.
How influential were /are the following functionality objective criteria to you in the selection
15. Flexibility .
16. Quality
17. Software reliability
19. Scalability
Usability objective.
How influential were /are the following usability objective criteria to you in the selection
20. Ease of software administration.
21. Ease of use
22. Automated
23. Easy to repair
24. Easy fault identification
25. Minimal errors
55
SECTION C: About your selection process
Software project characteristics
How influential were /are the following product characteristics to you in the selection of
26. Strategic fitness ,
27. Compatibility with existing hard ware
28. Compatibility with existing operation software
29. Consistency with interface and user friendly operations
30. Having complete functionality
31. Having high system reliability,(stability and recovery ability)
32. Accessibility and quality of product support.
33. Designed to minimize implementation time
34. Ease of installation
About the problems in the software selection process
How critical were /are the following Problems encountered during the software selection
process important to you in the selection of software? (Please select an option, which best
represents your option by placing a tick against the answer of your choice. In this case
1=Not influential, 2= didn’t affect my decision, 3=Fair, 4=Influential 5=Very influential.
56
9. lack of hand book/guide book
10. Not sure whether existing operation Problems
were caused by lack of the software
12. Lack of knowledge about the system
13. No established criteria.
14. Large numbers of software Available in the market
Are there any other issues that you think should be included in the survey? PLease state
them here......................................................................................................................................................
THANK YOU.
57
APPENDIX B
A MULTI CRITERIA DECISION MAKING QUESTIONNAIRE TO SOFTWARE SE-

LECTION
Your name(Optional):
Date:
Name of the Organisation:
NB: the information you provide here is kept completely confidential and no information is
stored on computer media that could identify you as person.
This questionnaire is entirely for academic purpose.
SECTION A: BACKGROUND INFORMATION.
This section is meant to provide the researcher background information for statistical group-
ings. Please place a tick against any option of your choice in the boxes corresponding to the
questions where possible.
Qn no Question coding category

1. kindly tick your gender female or Male
2. What is your age range?
less than 19 years
20-29 years
30-39 years
40-49 years
above 50 years
3. How long have you worked for this organization
Less than one year
2-3 years
4-5 years
6-7 years
Above 8 years
58
Section B: About Your software selection Criteria
Cost objective.
How influential were /are the following cost objective criteria to you in the selection of
4. Product cost.
5. Training Costs
6. License costs
7. Maintenance costs
8. Software subscription costs.
Vendor objective.
How influential were /are the following Vendor objective criteria to you in the selection
59
9. Vendor reputation .
10. Training provided
11. Implementation partner
13. Consultancy services.
14. Care about the business
Functionality objective.
How influential were /are the following functionality objective criteria to you in the selection
15. Flexibility .
16. Quality
17. Software reliability
19. Scalability
Usability objective.
How influential were /are the following usability objective criteria to you in the selection
60
20. Ease of software administration.
21. Ease of use
22. Automated
23. Easy to repair
24. Easy fault identification
25. Minimal errors
SECTION C: About your selection process
Software project characteristics
How influential were /are the following product characteristics to you in the selection of
61
26. Strategic fitness ,
27. Compatibility with existing hard ware
28. Compatibility with existing operation software
29. Consistency with interface and user friendly operations
30. Having complete functionality
31. Having high system reliability,(stability and recovery ability)
32. Accessibility and quality of product support.
33. Designed to minimize implementation time
34. Ease of installation
THANK YOU.
62

Nakato Ruth Cit Masters Report

Hochgeladen von

Dokumentinformationen

Originalbeschreibung:

Originaltitel

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Nakato Ruth Cit Masters Report

Hochgeladen von

Copyright:

Verfügbare Formate

A MULTI CRITERIA DECISION MAKING

SUPPORT TO SOFTWARE SELECTION

A Dissertation Submitted to the school of Graduate Studies

Option: Information Technology Management

List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii

1.1.1 Background to the Study . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Definition of Key Theoretical Terms . . . . . . . . . . . . . . . . . . . . . . . 2

1.3 Context of Decision Making and Software Selection . . . . . . . . . . . . . . 3

1.4 Statement of the Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.5 General Objective of the Research . . . . . . . . . . . . . . . . . . . . . . . . 5

1.6 Specific Objectives of the Research . . . . . . . . . . . . . . . . . . . . . . . 6

1.9 Outline of the Dissertation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2.2 Issues Related to Multi-Criteria Decision Making in Software Selection . . . 8

2.3 Multi Criteria Decision Making in Software Selection . . . . . . . . . . . . . 11

2.4 Multi Criteria Decision Making Supports for Software Selection . . . . . . . 13

2.4.1 Analytical Hierarchy Process (AHP) and Software Selection . . . . . 14

2.4.2 The Simple Multi-Attribute Rating Technique (SMART) and Software

2.4.3 Electre and software selection . . . . . . . . . . . . . . . . . . . . . . 16

2.4.4 Using Fuzzy Multi-Criteria Decision-Making to Software Selection . 16

2.4.5 Weighted Sum Method (WSM) and Software Selection . . . . . . . . 17

2.5 Chapter Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.1 Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.2 Study Population . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

3.3 Data Collection Method and Instrument . . . . . . . . . . . . . . . . . . . . 19

3.4 Validity of the Self Administered Questionnaire (SAQ) . . . . . . . . . . . . 19

3.5 Reliability of the SAQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3.6 Data Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4 Data Presentation, Analysis and Interpretation of Results 23

4.1 Survey Responses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.2 Description of the Background Variables . . . . . . . . . . . . . . . . . . . . 23

4.3 Issues Related to Multi Criteria Decision Making in Software Selection . . . 25

4.5.1 Make a Team of Decision Makers . . . . . . . . . . . . . . . . . 33

4.5.2 Identify the Software Project Characteristics . . . . . . . . . . 34

4.5.3 Evaluate the Software by the AHP Decision Making Approach 34

4.5.5 Make the Final Decision . . . . . . . . . . . . . . . . . . . . . . . . . 36

4.6 Chapter Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5 Discussion, Conclusion and Recommendations 40

5.1 Limitation of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.3 Further Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.4 Chapter Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

4.1 A multi criteria decision making support to software selection . . . . . . . . 37

4.2 Evaluation of the software by the AHP decision making approach . . . . . . 38

4.1 Response According to Gender . . . . . . . . . . . . . . . . . . . . . . . 24

4.2 Response According to the Age and Experience . . . . . . . . . . . . 25

4.3 Analysis of Variance of Cost According to Product Characteristics: 27

4.4 Analysis of Variance of Vendor According to Product Characteristics: 28

4.5 Analysis of Variance of Functionality According to Product Char-

4.6 Analysis of Variance of Usability According to Product Character-

4.7 Judgement scores for the importance/preference of criteria using

DSS Decision Support System

ICT Information Communication Technology

MCDM Multi Criteria Decision Making

ROI Return on Investment

AHP Analytic Hierarchy Process

ERP Enterprise Resource Planning

SAQ Self Administered Questionnaire

DICTS Directorate of Information Technologies

FCIT Faculty of Computing and Information Technology

COTS Commercial Off the Shelf Software

WSM Weighted Sum Method

SMART Simple Multi-Attribute Rating Technique

1.1.1 Background to the Study