A Decision Support System for Evaluation of IT Investment Projects

by Ssemaluulu Paul BBA (Mak)

A Research Proposal Submitted for the study leading to a Dissertation in partial fullment of the requirements for the award of a Master of Science in Computer Science of Makerere University

Supervisor, Dr. Ddembe Williams, London Southbank University

December, 2004

LIST OF ACRONYMS

DS DSS IS IT

Decision Support Decision Support System Information System Information Technology

1 Introduction
Evaluation, that is assessing, against implicit or explicit criteria ia always being carried out to nd the value of individuals, objects, situations and outcomes every day of our lives (Legge,1984) [31]. This though does not make evaluation a trivial activity. Harrison states that decision making is an integral part of the management of any kind of organisation, and more than anything else, competence in this activity differentiates the manager from the nonmanager and, more important, the effective manager from the ineffective manager(Harrison,1987) [22]. Increased competition and the global economic aspects have forced companies to cut costs signicantly, through the empowerment of lower-level staff and the removal of middle management. These trends are increasingly supported by developments in information technology(IT).(Serafeimidis and Smithson, 1998) [42]. The existing literature(Willcocks and Lester,1994 [52]; Ballantine et al.,1995 [3]; Ward et al.,1996 [51]; Farbey et al.,1999 [18]; Serafeimidis and Smithson, 2000 [43]) identies noticeable gaps between academic theories, commercially available methodologies and actual evaluation practices promoted by organisational rules and structures, informal practices implemented by stakeholders and academic recommendations which are not used in practice. Research shows that many organisations are experiencing difculties with evaluating information systems. Research further indicates that more than half of the organisations do not apply any formal evaluation at all(Willcocks,1996 [52]; Dhillon, 2000 [12]). Dhillon concludes that 90% of all senior IS managers have no idea how to determine the value of information systems(Dhillon,2000) [12]. Many researchers have stated problems and suggested solutions related to evaluation of IT investments and IS projects (Hirscheim and smithon, 1988; [24] Orlikowski and Crash, 1994 [39]; Farbey et al.,1995 [18]; Dhillon and Backhouse,1996 [11]; Irani and Love,2001 [26]).

1.1 Denition of terms to be Used in the Research

Denition 1: A decision support tool is a computer-based system that brings together information from a variety of sources, assists in the organisation and analysis of information and facilitates 1

the evaluation of assumptions underlying the use of specic models (Williams, 2002) [53]. In this research the decision support tool is an interactive computer-based model-driven decision support system that uses data and parameters provided by decision makers to aid decision-making in analysing a situation. Although there are already other studies in IT investment evaluation, they are not customised to support an individual manager at the desktop. Others are overly complex in nature and require experts to operate them.

1.2 Background to the Study

The growing dependence of organisations on IT is viewed by many as source of uncertainty. Serafeimidis and Smithson state that it is difcult to identify and measure the potential benets and costs of an IT investment (Serafeimidis and Smithson, 1998). [42] It is also said that IT evaluation is complex and elusive (Hirscheim and Smithson,1988 [24]; Dhillon,2000 [12]). According to Dhillon(2000) [12], a phenomenal amount of money is lost because of inability of organisations to realise IS/IT benets. Figures coming from the United States suggest nearly $ 59 million in cost overruns and some $ 81 million in cancelled IS/IT projects(Johnson,1995) [27]. In Uganda, IT investments seem to have run into difculties with the latest being the Electoral Commission Voters Register System. It was hatched in 1995 but todate, its benets have not been realised. According to eGovernment for Development, a project coordinated by the University of Manchesters Institute for Development Policy and Management, a total sum of $ 22 million was spent on equipment, consultancy services and operations. It is further stated that there were no formal benets because the system was not put to use for the 2001 elections. The acrimony which arose out of the electoral process led to a number of court cases between government and opposition groups, and police surveillance of opposition leaders. Some opposition leaders ed the country as a result of these tensions, leading to greater potential for destabilisation(egov4dev.org,2002) [55]. Executives often face information overload when they have to decide whether to invest in new IT projects or when management demand an audit of a completed project. The existence of a simple and easy to use tool for evaluation would greatly reduce this information overload.

1.2.1 Difculties in Evaluation of IT Investment Projects

Various studies have highlighted the difculties associated wuth the evaluation of IT investment projects. Beach states the following characteristics make decisions more difcult: complexity, ambiguity, time elapsed, many interests involved, and unfamiliarity(Beach,1990) [5].

1. Complexity- About the complexity of IT investment projects, Symons states, Information systems themselves are complex social objects inseperable from the organisational context within which they are situated and the infrastructure supporting them, and are products of history and human agency(Symons, 1994) [49]. 2. Time elapsed- The time frame between planning, designing, building, implementing and reaping the benets of an information system is considerable. In the meantime, both the organisation and its environment have changed. 3. Many interests involved-Information systems inuence organisational boundaries and change the way of working. Costs and benets arise often in different parts of the organisation, and also surface disproportionately in different organisations. 4. Unfamiliarity-Information systems are still relatively new especialy in a developing country like Uganda. the technology is changing rapidly as is applicability.

1.3 Statement of the Problem

For many organisations, little monitoring and evaluation of IT investment projects is carried out. Various techniques that have been brought in take longer and the results are not very visible. Recent literature on IT investment projects(Dhillon,2000) [12] states that the majority of IT executives do not know how to measure the benets of IT. There is a clear need for a framework that enables all parties to communicate and exchange information on possible IT investments in a form that is readily understood. If organisations are to benet from IT investments, then a tool for identifying the costs and benets of IT is required so that managers can understand and feel fully condent in applying it.

1.4 Objectives
1.4.1 General Objective

The objective is to establish problems relating to evaluation of IT investment projects. Understanding these problems will lead to better evaluation methods and will in the long run reduce losses due to failed IT investments.

1.4.2 Specic Objectives

The specic objectives of the research programme are to:

1. Review critically literature related to evaluation of IT investment projects. 2. Design a computer-based decision support system for evaluation of IT investment projects. 3. Implement a system dynamics model inoporating the key variables. 4. Validate and test the model.

1.5 Scope
The research will use the Ministry of Finance as a case study focusing on evaluation of IT investment. The Ministry is the coordinating agency for many donor funded projects for the Government of Uganda. The researcher will limit the study to a desktop DSS due to limitations of time, cost and complexity. The researcher will use Javascript programming language which allows for easy encapsulation in object-oriented code modules.

1.6 Justication
A decision support system to help managers in evaluation of IT investment projects will have the following functions:

1. To provide the basic feedback function to managers(Smithson and Hirscheim,1998) [44]. 4

2. To support the organisational learning process(Hallikainen,1999) [21]. 3. To allow problem diagnosis, planning and reduction of uncertainty(Smithson and Hirscheim,1998) [44]. 4. To offer an estimation of the expected value of the IT investment(Hallikainen et al.,1998. [20].

This will help nancially trained and non-nancial IT managers make meaningful decision quickly.

1.7 Research Contributions

This research will make a contribution to the literature in terms of evaluating IT investment projects in a developing country like Uganda. the tool that will be designed will help managers cut down on time spent debating investment decisions, cut down on costs and reduce information overload.

2 Literature Review
Literature on evaluation of information systems has existed since the seventies(Frielink,1975; [19] Joslin,1977; [28] Borovits and Neuman,1979; [8]). Many studies have been conducted in this area(Kauffman and Weill,19890; [29] McKeen and Smith,1993 [36]; Willcocks,1992 [53]; Farbey et al.,1993 [18]; Hitt and Brynjolfsson,1994 [25]; Serafeimidis and Smithson,2000) [43], and many conferences have been devoted to this subject(the 15th International Conference on Information Systems research themeImproving productivity and adding value through information systems, and The European Conference on IT investment evaluation, Henley-onThames , 13/14 september 1994 and 11/12 July 1995). Failure of value generation of information systems is also documented(Loveman,1988 [34]; Roach,1989 [41]; Svendsen,1996 [46]). A perceived lack of effective IT benets often creates difculty in justifying future expenditure and managing the benets of IT innovations. The problem of identifying IT costs and benets is neither new nor unique to a given sector. It is a global problem experienced in all types of business sectors and organisations(Hochstarasser and Grifths, 1991) [23]. This means companies must recognise that the full benets of an IT project can only be realised as part of an overall business strategy (Bruce, 1995) [9]. It has been proven that IT managers frequently lack a full understanding of their organisations business and are often not involved in the senior management decision-making of the company (Andresen, 2000) [2]. Senior management who do understand the business are 5

usually not comfortable with the emerging information technologies. (Andresen, 2000) [2]. When considering new IT investments, senior management seldom have feedback from previous investments to provide comfort for their earlier decisions (Ballantine and Stray, 1998) [4].

2.1 IT Investment Evaluation

The essential questions are how and when to evaluate IT investments. There are three inter-related questions. How does information technology (IT) improve business performance? How do we decide the IT projects in which to invest? How do we assess the performance of systems after their implementation? The use of diffrent evaluation techniques to answer these questions varies from organisation to organisation. Research into the use of these techniques and their value to different organisations provides varying responses(Farbey et al.,1992 [16]; Willcocks and Lester, 1994; Berghout, 1996 [7]). Earl (1989), stresses that not all organisations face an identical challenge, their business sectors differ, the competitive forces they combat vary, their histories are not alike and they make different strategic choices. In addition, organisations must evaluate where in their evolution of IT developments they stand so as to ensure that they are able to make and manage the appropriate degree of strategic change.Farbey et al.,1993 [17] argue that the search for a single technique for evaluating investments in IT is fruitless. The range of circumstances that one technique would have to be applied to is so wide that no single technique is likely to be applicable. Berghout (1997) [7] concludes that a mixture of both qualitative and quantitative methods should be used.

2.2 Review of Evaluation Methods

There are numerous IT benets evaluation methods. Andresen (1999) [2] gives 30 while Berghout (1997) [7] gives 99. These may be categorised into objective methods, that is those seeking to quantify system inputs and outputs in order to attach values to them, and subjective methods, that is, those relying on the attitudes and opinions of users and system builders. The early investment evaluation methods are primarily based on nancial measures such as Return On Investment (ROI) and Net Present Value (NPV). These methods are, however, found to be inadequate when used to evaluate IT investments because of their use of only one measure (monetary value). More complex methods designed for evaluation of IT investments, have emerged such as Information Economics

(IE), Return On Management (ROM), and SESAME (Parker and Benson, 1988 [40]; Strassman, 1985 [45], Lincoln et al.,1990 [33]. These complex methods are, however, rarely used in practice for a number of reasons. First, because of little awareness of the methods by managers. Second, because of the methods large operation requirements. Third, some critical problems are still not solved in the methods (Andresen, 1999) [2].

2.3 DSS applications

There has been a growing amount of interest and research in the area of Decision support Systems as tools for supporting managers in their semi-structured or unstructured decison-making process (Eom and Lee, 1990 [14]; Eom et al.,1998) [15]). DSS have been used to assist decision makers in evaluating investments in many diverse application areas. However, existing literature reveals that they typically focus on only one out of a number of factors that should be assessed when evaluation is being carried out. The following are the common areas:-

1. Financial-DSS in this area focus on the economic evaluation of alternative investment proposals. Examples include systems for nancial risk assessment (Mareschal and Brans, 1991 [35]; Siskos et al.,1994) [47], new product development funding (Kettelhut, 1991) [30] and small business nancial planning (Chen, 1989 [30]; Sterling and Stubbleeld, 1994 [48]). 2. Commercial-DSS are used to support the alignment between an organisation, its products, and the marketplace in which these products are sold. An Examples is DSS for industry analysis (Vickers, 1992 [50]). 3. Strategic-DSS are used to support strategic decision-making regarding proposed investments. An example is a DSS for strategy formulation (Mooremann and Lochte-Holygreven, 1993 [37]). 4. Technical-DSS in this area are used to identify the optimal equilibrium between performance, reliability, and cost factors within alternative technical investment options. An example is the evaluation of different Local Area Network topologies (Ligget and Sullivan, 1992 [32]).

It can be observed that the individual manager at the desktop is still not catered for as all these systems focus on the enterprise as a whole. 7

3 Methodology
3.1 Introduction
This research proposes a research design using the case study approach. Case study research is the most common qualitative method used in information systems. (Orlikowski and Baroudi, 1991 [38]; Alavi and Carlson, 1992 [1]). A case study is an empirical enquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident (Yin, 2002) [54]. Clearly, the case study research method is particularly well-suited to IS research, since the object of our discipline is the study of information systems in organisations, and interest has shifted to organisational rather than technical issues(Benbasat et al.,1987 [6]).

3.1.1 Field Study

The study will use the following data collection techniques:-

1. Questionnaires to be given to IT decision makers in the Ministry of Finance. 2. Semi-structured Interviews with IT decison makers and other IT related personnel in the Ministry of Finance.

3.2 System Design

The Decision Support System will be designed based on the eld study results which describe the parameters and the information to be incoporated in the system. The conceptual model of the decison support system methodology includes the following subsystems: the database component, the model component, the communications component and the user interface component. Each sub-system is described as a separate entity and the relationships between the systems are dened.

DSS Design Methodology

Figure 1: The Components of DSSs

3.2.1 Database Component

The database component serves as a data bank for the DSS. It stores large quantities of data that are relevant to the class of problems for which the DSS will be designed and will provide logical structures with which the users will interact.The primary information in the database component will come from data obtained in the eld study. The database component will be designed using MySql which is compatible with a number of major database formats (e.g. FoxPro, DBase, Oracle).

3.2.2 Model Component

The model component is analogous to the database component. Its primary function is to provide independence between specic models that are used in a DSS from the applications that use them. the purpose of the model component is to transform data from the database component to information that is useful in decison-making. The easiest way of implementing this is to implement a decision algorithm.

3.2.3 Communications Component

The DSS architecture and networking design component refers to how hardware is organised, how software and data are distributed in the system, and how components of the system are integrated and connected. A major issue today is whether DSS should be available using a web browser on a company intranet or as a stand-alone. The Communications component will be designed so as to be accessible via a web browser.

3.2.4 User Interface Component

In many ways, the user component is the most important component. The user interface, both input and output will be designed using Javascript and PHP scripting languages. PHP provides very powerful querrying capabilities, while Javascript presents an easy to understand interface for engineers, accountants, and other managers. The portability of the system will be assured by virtue of combination of widely used packages

10

together with Javascript programming language, which allows encapsulation in object-oriented code modules. These can be run on any IBM compatible machine.

11

References
1. Alavi, M. and Carlson (1992). A Review of MIS Research and Disciplinary Development. Journal of Management Information Systems. 8(4):45-62. 2. Andresen, J. (2000). A Framework for Measuring IT Innovation Benets. Doctoral Dissertation. Technical University of Denmark, Copenhagen. 3. Ballantine, J., Galliers, R., and , Stray, S.J.(1995). The Use and Importance of Financial Appraisal Techniques in the IS/IT Investment Decision-Making Process-Recent UK Evidence .Project Appraisal, 10: 233-241. 4. Ballantine, J. and Stray, S. (1998).Financial Appraisal and the IS/IT Investment Decision Making Process. Journal of Information Technology 13:3-14. 5. Beach, L.R. (1990). Image Theory:Decision Making in Personal and Organizational Contexts . John Wiley and sons, ISBN 0-471-92030-5. 6. Benbasat, I., Goldstein, D.K., and Mead, M. (1987). The Case Research Strategy in Studies in information Systems. MIS Quartely . 11(3):369-386. 7. Berghout, E. (1997). Evaluation of Information System Proposals:Design of a Decision Support Method. Doctoral Dissertation. Delft University of Technology. 8. Borovits, I. and Neuman, S. (1979) Computer Systems Performance Evaluation:Criteria Mesurement, Techniques, and Costs. Lexington Books, ISBN 0-669-02802-9. 9. Bruce, F.M. (1995). Obtaining Return on Investments in InformationTechnology Projects. Journal of Information Technology. (13): 3-14. 10. Chen, K.C. (1989). Developing Decision Support Systems for Small Business Management:A Case Study. Journal of Small Business Management. 27(3):11-12. 11. Dhillon, G. and Backhouse, J. (1996). Risks in the Use of Information Technology Within Organizations. International Journal of Information Management,. 16(1): 65-74. 12. Dhillon, G. (2000). Interpreting Key Issues in IS/IT Benets Management. Proceedings of the 33rd Hawaii International Conference on System Sciences.Jan 04-07.

12

13. Earl, M.J. (1989). Management Strategies for Information Technology.Prentice Hall International (UK) Ltd. 14. Eom, S.B. and Lee (1990). A Survey of Decision Support System Applications. Interfaces. 20:65-79. 15. Eom, S.B., Lee, S.M., and Somarajan, C. (1998). A Survey of Decision Support System Applications. Journal of the Operational Research Society. 49:109-120. 16. Farbey, B., Land, F. and Targett, D. (1992). Evaluating investments in IT. Journal of Information Technology. (7):109-122. 17. Farbey, B., Land, F. and Targett, D. (1993). IT Investment-A Study of Methods and Practice. Butterworth and Heineman. 18. Farbey, B., Land, F. and Targett, D. (1995). Hard Money-Soft Outcomes. Evaluating and Managing the IT Investment , Alfred Waller Ltd, Oxon, UK. 19. Frielink, A.B., (1974). Economics of Informatics.Proceedings of the IBI-ICC International Symposium. Mainz, September 16-20, North Holland Publishing Company, Amsterdam, 1975. 20. Hallikainen, P., Heikkila, J., and Saarinen, T. (1997). Evaluation of Information Systems in Finnish Companies. Publications of Helsinki School of Economics and Business Administration, Research Report B-9. 21. Hallikainen, P.(1999). Improving the Success of Information Systems by Evaluation: A Learning Approach. Publications of Helsinki School of Economics and Business Administration, Research Report W-228. 22. Harrison, E.F.(1987) The Managerial Decison-Making Process. Houghton Mifn Company. Boston(MA). Third Edition. ISBN 0-395-4281-X. 23. Hochstrasser, B. and Grifths, C. (1991). Controlling IT Investment- Strategy and Management. Chapman and Hall, London. 24. Hirscheim, R. and Smithson, S. (1998). A critical analysis of Information Systems Evaluation. Information Systems Assesment. 1:17-37.

13

25. Hitt, L., and Brynjolfsson. (1994). The Three Faces of IT value: Theory, and Evidence. Proceedings ot the 15th International Conference on Information systems. Vancouver (Ca), December 14-17:263-277. 26. Irani, Z. and Love, P.E.D. (2001). The Propagation of Technology Management Taxonomies for Evaluating Investments in Information Systems. Journal of Management Information Systems. (17): 161-177. 27. Johnson, J. (1995). Chaos: The Dollar Drain of IT Project Failures. Application Development Trends.2(1):44-47. 28. Joslin, E.O.(1977). Computer Selection. The Technology Press, Augmented Edition. 29. Kauffman, R.J. and Weill,(1989) P. An Evaluative Framework for Research on the Performance. Effects of Information Technology Investments. Proccedings of the 10th International Conference on Information Systems (ICIS). Boston (Ma), December 4-6, 377-388. 30. Kettelhut, M.C. (1991). Using a DSS to Incoporate Expert Opinion in Strategic Product Development Funding Decisions. Information and Management. 20(5):363-371. 31. Legge, K. (1984). Evaluating Planned Organisational Change. Academic Press. London. ISBN 0-12-440980. 32. Liggett, H.R. and Sullivan, W.G. (1992). Multi-attribute Evaluation of Local Area Network Topologies. Engineering Economist. 37(2):91-114. 33. Lincoln, T, Berenbaum, R., Shorrock, D and Amos, W.J. (1990). Managing Information Systems for Prot. John Wiley and Sons. 34. Loveman, G. (1988) An assessment of the Productivity Impact of Information Technologies. Massachusettes Institute of Technology Report. MIT-Management in the 1990s, 88-054WP. 35. Mareschal, B. and Brans, J.P. (1991). Bankadviser:An Industrial Evaluation System. European journal of Operational Research. 54(3):318-324. 36. McKeen, J.D., and Smith, H.A.(1991) The value of Information Technology: a Resource View. Proceedings of the 12th International Conference on Information systems(ICIS).December 1991. 41-51.

14

37. Mooremann, J. and Lochte-Holtgreven, M. (1993). An Approach for an Integrated DSS for Strategic Planning. Decision Support Systems. 10(4):401-411. 38. Orlikowski, W.J. and Baroudi, J.J. (1991) Studying Information Technology in Organizations:Research Approaches and Assumptions. Information Systems Research 2:1-28 39. Orlikowski, W.J. and Gash, D.C. (1994)Technological Frames: Making Sense of Information Technology in Organizations. ACM Transactions on Information Systems. 12:174-207. 40. Parker, M.M. and Benson, R.J. (1998). Information Economics-Linking Business Performance to Information Technology. Prentice Hall. 41. Roach, S.S. (1989). The Case of the Missing Technology Payback. Proceedings 10th International conference on information Systems (ICIS) Boston (Ma). December 4-6. 42. Serafeimidis, V. and Smithson, S. (1998). Interpretive Information Systems Evaluation in Practice:Experience From a Case Study. Computing Science Report Series. Department of Computing, University of Surrey, CS-98-01. 43. Serafeimidis, V. and Smithson, S. (2000). Interpretive IS Evaluation in Practice:Experience From a Case Study. Journal of Information Technology. (15):93-105. 44. Smithson, S and Hirschheim, R. (1998). Analysing Information Systems Evaluation:Another look at an Old Problem. European Journal of Information Systems. 7:158-174. 45. Strassman, P.A.(1985). Information Payoff-The Transformation of Work in the Electronic age. The Free Press. 46. Svendsen, C.(1996). Investment Evaluation of Technology-Based Training Systems, Master of Science Report. Delft University of Technology, Faculty of Mathematics and Informatics. June, 12. 47. Siskos, Y., Zopounidis, C. and Pouliezos, A. (1994). An Integrated DSS for Financing Firms by an Industrial Development Bank in Greece. Decision support Systems. 12(2):151-168. 48. Sterling, J.W. and Stubbleeld, A. (1994). ADVIA:Planning and Decision Support for Smaller Business. Planning Review. 22(1):50-54.

15

49. Symons, V.J. (1994). Evaluation of Information Systems Investments: Towards Multiple Perspectives. Chapman and Hall, London. ISBN 0-412-41540-2. 50. Vickers, B. (1992). Using GDSS to Examine the Future European Automobile Industry. Futures. 24(8):789-812. 51. Ward, J., Taylor, P., and Bond,P. (1996). Evaluation and Realisation of IS/IT Benets: an Empirical Study of Current Practice. European Journal of Information Systems. 94: 214-225. 52. Willicocks, L.(1996) Investing in Information Systems: Evaluation and Management. Chapman and Hall, London. ISBN 0-412-72670-X 53. Williams, D. (2002). An Application of system Dynamics to Requirements Engineering Process Modelling, Unpublished PhD, London South Bank university. 54. Yin, R.K.(2002) Case Study Research, Design and Methods. 3rd ed Newbury Park, Sage Publications. 55. http://www.egov4dev.org/iecuganda.htm. (Accessed on 7th October 2004).

16

Project Time Frame

NO 1 2 3 4 5 6

ACTIVITY Proposal Writing and Approval Surveying Literature and Framework Development Methodology, Planning, Requirements analysis and Design Implementations Testing and Evaluation Report Consolidation and Handing in

PROPOSED START DATE DEC 2004 OCT 2004 FEB 2005 APR 2005 JUNE 2005 JULY 2005

END DATE JAN 2005 FEB 2005 MAR 2005 JUNE 2005 JUNE 2005 AUG 2005

17

BUDGET (Uganda Shillings)

QTY ITEM UNIT COST 1 Laptop 3,000,000 1 DSS Development Software 1,200,000 3 Stationery 10,000 90 Transport 2,000 2 Research Assistants 300,000 2000 Photocopying 50 1500 Printing 200 5 Binding 10,000 Sub Total 1 Contingency (10%) GRAND TOTAL

TOTAL 3,000,000 1,200,000 30,000 180,000 600,000 100,000 300,000 50,000 5,560,000 556,000 6,116,000

18