The frequent citations to IS research suggest its role in advancing science.

IS INFORMATION SYSTEMS A REFERENCE DISCIPLINE?

he notion of information systems (IS) being an applied discipline drawing upon other disciplines has been held as conventional wisdom by IS researchers for the past two decades [2]. IS has been conceived as growing from other foundational fields, such as computer science, management science, and organization science [4]. Recently, this conventional wisdom has been challenged by the new perception that IS is a reference discipline for others. Evidence supporting this new perception includes: established IS journals, settled academic conferences, distinct subject matter and perspectives in IS research, an established international society, other well-developed networking channels [2], and usage and recognition of the value of IS research [1]. In addition, IS researchers maintain that IS pervades diverse areas and has much to offer researchers in other disciplines [2]. This claim is supported by two well-known IS articles [5, 10] frequently cited by numerous publications in many disciplines. Unfortunately, no published literature has examined knowledge con-

tributions of IS research to other disciplines. Thus, to provide more objective evidence in support of this new perception, we conducted research using citation analysis to examine how frequently the articles published in major IS journals are cited by publications in other disciplines. Although citation analysis is not new in business research, most past research might better be termed reference analysis since the unit of analysis was the references in, rather than the citations of, an article [3]. Examining the references provided in a journal is useful for identifying what disciplines influence that journal, but not for identifying that journals contributions to other disciplines. Who we reference indicates where our knowledge base comes from [9]; conversely, where our research is cited provides insights into its contributions. Therefore, this study employs citation analysis to examine the citations made to the articles published in major IS journals to assess knowledge contributions of the IS discipline. Six major IS journals: Communications of the ACM (CACM), European Journal of Information Systems (EJIS), Information Systems
AND

BY Pairin Katerattanakul, Bernard Han,

114
May 2006/Vol. 49, No. 5 COMMUNICATIONS OF THE ACM

Alan Rea

psychology, philosophy, and sociology. ISJ Journal (ISJ), Information Systems Research (ISR), Information & Management (I&M), and is known for publishing research using qualitative methodology. MIS Quarterly (MISQ), were selected to highlight the contributions made by the IS disci- ISR. Published since 1990, this is the representative jourpline. The selection nal of the process was careCACM EJIS ISJ ISR I & M MISQ Total Institute for fully done by folCitations 276 125 501 433 665 4668 2668 Operations lowing the same % of Citations 5.9 2.7 10.7 9.3 14.2 100 57.2 Research and the approach employed Target articles 90 64 86 199 83 1120 598 Management in another study on Citations per article 4.46 3.07 1.95 5.83 2.18 8.01 4.17 Sciences the quality ranking (INFORMS). ISR focuses on theory, creof computing jour- Table 1. Citations made to each selected journal. nals based on citaative research, and novel use and design, tion analysis [8]. A development, and implementation of brief description information systems that involves organiPairin table zational, 1 (5/06)economical, and social ramificahere highlights general characteristics of each journal: tions. I&M. Published since 1977, this journal is CACM. This flagship publication of the aimed at knowledge dissemination of Association for Computing Machinery applied research in information systems in (ACM) was established in 1958. CACM is order to provide guidelines and insights published 12 times a year and covers a into the successful undertaking of inforrich assortment of feature articles that mation technology initiatives. I&M priexamine both technical and non-technical marily serves information professionals of subjects related to computing, information the organizations that design, develop, processing, and evolving technologies. manage, and use information systems EJIS. Published since 1992, this journal is applications. MISQ. Published since 1977, this journal aimed at both the theory and practice of information systems. Specifically, it is aimed at academic research that is critifocuses on technology development, cal to appropriate decision making in implementation, management, strategic regard to the acquisition, deployment, and use of information technology. MISQ has planning, and policymaking. EJIS publishes research articles contributed by acaa strong focus on the managerial aspect of demic researchers, and case studies/ knowledge in IT, and its impact on modreflective articles by practitioners. ern organizations and human societies. ISJ. Published since 1991, this journal From the six selected IS journals, a list of focuses on the study of information sys1,120 target articles published in these jourtems. ISJ publishes articles with a widenals from 1995 to 1998 was compiled. This range and interdisciplinary nature since it timeframe was chosen because citation data promotes integration of technology with management and other disciplines such as for some selected journals (such as EJIS and

COMMUNICATIONS OF THE ACM May 2006/Vol. 49, No. 5

115

CACM EJIS ISJ ISR I&M MISQ Total ISJ) was only available from 1995. Then, a second list of the ctn1 %2 ctn % ctn % ctn % ctn % ctn % ctn % citations made to each target arti- Acct 1 0.0 0 0.0 1 0.8 1 0.2 2 0.5 3 0.5 8 0.2 cle was compiled. To ensure a rea- Comm 16 0.6 2 0.7 1 0.8 2 0.4 4 0.9 3 0.5 28 0.6 sonable citation history for CS 1147 43.0 29 10.5 23 18.4 32 6.4 31 7.2 45 6.8 1307 28.0 2 0.1 3 1.1 0 0.0 1 0.2 2 0.5 1 0.2 9 0.2 analysis, all citation data was col- Econ 59 2.2 1 0.4 1 0.8 2 0.4 1 0.2 5 0.8 69 1.5 lected from 1997 to 2000 since Edu the modal elapsed time between Engine 211 7.9 6 2.2 0 0.0 16 3.2 22 5.1 14 2.1 269 5.8 IS article publication and citation Health 78 2.9 6 2.2 1 0.8 10 2.0 1 0.2 7 1.1 103 2.2 728 27.3 186 67.4 72 57.6 308 61.5 298 68.8 458 68.9 2050 43.9 was found to be approximately IS Library 93 3.5 1 0.4 0 0.0 5 1.0 2 0.5 5 0.8 106 2.3 two years [7]. 94 3.5 32 11.6 16 12.8 87 17.4 34 7.9 90 13.5 353 7.6 Citations were drawn from two Mngt 13 0.5 0 0.0 0 0.0 2 0.4 4 0.9 2 0.3 21 0.4 well-known citation sources: Mkt OthBus 7 0.3 0 0.0 0 0.0 1 0.2 1 0.2 2 0.3 11 0.2 Social Science Citation Index Psy 29 1.1 2 0.7 0 0.0 11 2.2 11 2.5 10 1.5 63 1.3 (SSCI) and Science Citation Index Socio 62 2.3 6 2.2 10 8.0 13 2.6 11 2.5 19 2.9 121 2.6 (SCI), which cover a broad range of publications. Altogether 4,668 Others 128 4.8 2 0.7 0 0.0 10 2.0 9 2.1 1 0.2 150 3.2 Total 2668 100 276 100 125 100 501 100 433 100 665 100 4668 100 citations were made to the 1,120 1. ctn indicates the number of citations target articles. These citations 2. The percentage is derived by dividing the column total into the row count were made from the articles pubTable 2. IS journals highest proportion (43.9%) of these citations were lished in 586 citing journals. cited by different from the publications in the IS discipline. Thus, Then, using the journal categories disciplines. provided in SSCI and SCI with approximately 56% of the collected citations are Pairin table 2 (5/06) some proper modifications, these attributed to other disciplines. Computer science 586 citing journals were further classified into 15 (28%), management (7.6%), engineering (5.8%), major disciplines. Finally, for each selected IS journal, sociology (2.6%), library and information science the numbers of citations made to its articles by the (2.3%), and medicine and health care (2.2%) account 586 citing journals was recompiled and aggregated for approximately 48% of the citations. Nearly 8% of with respect to each discipline. the citations are attributed to other disciplines (eduTable 1 reports the number of citations, the num- cation, psychology, economics, and marketing). ber of target articles, and the average citations per tarMore than half (ranging from 57.6% to 68.9%) of get article for each selected IS journal. The target the citations made to target articles published in five articles published in CACM receive more than half of selected journal: EJIS, ISJ, ISR, I&M, and MISQ, are the citations (2,668 citations or 57.2%) collected in from the articles published in IS journals. However, this study. For citations per target article, on average only 27.3% of the citations made to target articles each target article receive 4.17 citations. MISQ has published in CACM are from the articles published the highest citations per target article (8.01) followed in IS journals. In contrast, publications in computer by ISR (5.83), CACM (4.46), EJIS (3.07), I&M science journals moderately cite (ranging from 6.4% to 18.4%) the target articles published in EJIS, ISJ, (2.18), and ISJ (1.95), respectively. Table 2 reports how often each selected IS journal ISR, I&M, and MISQ, while 43% of the citations to was cited by each discipline. From a total of 4,668 CACM target articles are from computer science citations made to the target articles in this study, the journals.

WE CONDUCTED RESEARCH USING CITATION ANALYSIS TO examine how frequently the articles published in major IS journals are cited by publications in other disciplines.

116

May 2006/Vol. 49, No. 5 COMMUNICATIONS OF THE ACM

nother interesting finding reported in Table 2 is that European IS journals (such as EJIS and ISJ) are cited by fewer disciplines (only 12 and eight disciplines respectively). Possible reasons for this result include the limited availability of European IS journals in U.S. universities and the relative newness of these IS journals (EJIS and ISJ started in the early 1990s). However, a firm explanation can only be supported with additional investigations. To examine whether particular IS journals are popular to any specific disciplines, we considered the percentage that each selected IS journal contributes to each discipline (see Table 3). Considering that target articles published in CACM receive the highest proportion of citations (2,668 citations or 57.2%), it is not surprising that, among the six selected journals, CACM has the highest percentage of contribution to virtually every discipline. For CACM, contributions to each discipline range from 12.5% to 87.8%. Publications in computer science, library and information science, education, engineering, medicine and health care, and others cite CACM more frequently (87.8%, 87.7%, 85.5%, 78.4%, 75.7%, and 85.3%, respectively) than any other selected journals. Similarly, publications in communication, psychology, sociology, marketing, and other businesses cite CACM at moderately higher percentages than other selected journals. Thus, CACM is quite a popular knowledge source for a variety of disciplines. The citations in accounting and economics publications for each selected IS journal are comparably low (ranging from 0 to 3 citations). Thus, none of the selected IS journals seems to be a major knowledge source for these two disciplines. However, it is worth noting that each of these disciplines cites the selected journals less than 10 times. For publications in management, CACM, ISR, and MISQ are three major knowledge sources; whereas the other three selected journals (EJIS, ISJ, and I&M) also noticeably contribute to management publications. Finally, the percentage that IS publications cite target articles published in CACM and MISQ (35.5% and 22.3% respectively) make these two journals the main knowledge sources for IS publications. ISR and I&M (15% and 14.5% respectively) are considered significant knowledge sources for IS articles as well; whereas the European IS journals EJIS and ISJ (9.1% and 3.5% respectively), also noticeably contribute to IS publications.
CONCLUSION Any discipline striving toward scientific maturity is justifiably concerned about the utility of its knowledge and the rate at which such knowledge is dis-

CACM EJIS Accounting Comm CS Economics Education Engineering

ISJ 1 12.5 1 3.6 23 1.8 0 0.0 1 1.4 0 0.0 1 1.0 72 3.5 0 0.0 16 4.5 0 0.0 0 0.0 0 0.0 10 8.3 0 0.0 1

ISR 1 12.5 2 7.1 32 2.4 1 11.1 2 2.9 16 5.9 10 9.7 308 15.0 5 4.7 87 24.6 2 9.5 1 9.1 11 17.5 13 10.7 10 6.7 6

I & M MISQ Total 2 25.0 4 14.3 31 2.4 2 22.2 1 1.4 22 8.2 1 1.0 298 14.5 2 1.9 34 9.6 4 19.0 1 9.1 11 17.5 11 9.1 9 6.0 6 3 37.5 3 10.7 45 3.4 1 11.1 5 7.2 14 5.2 7 6.8 458 22.3 5 4.7 90 25.5 2 9.5 2 18.2 10 15.9 19 15.7 1 0.7 8 8 100 28 100 1307 100 9 100 69 100 269 100 103 100 2050 100 106 100 353 100 21 100 11 100 63 100 121 100 150 100

ctn % ctn % ctn % ctn % ctn % ctn %

1 12.5 16 57.1 1147 87.8 2 22.2 59 85.5 211 78.4 78 75.7 728 35.5 93 87.7 94 26.6 13 61.9 7 63.6 29 46.0 62 51.2 128 85.3 15

0 0.0 2 7.1 29 2.2 3 33.3 1 1.4 6 2.2 6 5.8 186 9.1 1 0.9 32 9.1 0 0.0 0 0.0 2 3.2 6 5.0 2 1.3 1

ctn Health % ctn IS % ctn Library % ctn Management % ctn Marketing % ctn OthBus % ctn Psychology % ctn Sociology % ctn Others %
# of disciplines with % 10

1. ctn indicates the number of citations 2. The percentage is derived by dividing the row total into the column count

seminated across the sci- Table 3. Popular IS journals for each disciplines. entific community [3]. Pairin The IS discipline is table no 2 (5/06) exception. This study employs citation analysis to examine the contributions of IS knowledgewhere and how often it is cited. Results from this study provide strong evidence that the IS discipline has become a reference discipline for others. That is, IS research published in IS journals is frequently cited by other disciplines, even those fields that previously served as reference disciplines for IS (such as computer science, management, and organization science). The frequent citations made from other disciplines to IS research also suggest that IS research contributes to advancing the body of scientific knowledgeas the intended purpose of publications in academic journals is to impart knowledge to others, furthering the advancement of scientific achievements [11]. We believe that further investigation of IS knowlCOMMUNICATIONS OF THE ACM May 2006/Vol. 49, No. 5

117

edge contributions is warranted since the results of this study are an average across target articles published in six selected IS journals during four-year period 19951998. In addition, an examination of the citation patterns for all major IS journals would more clearly reflect knowledge contributions within and across disciplines of IS research and would also shed more light on the maturity of IS discipline. Finally, despite its extensive use, citation analysis is not without its drawbacks. Citation data available in SSCI and SCI is by no means complete. However, these indices have the most extensive citation coverage available for more than 7,000 journals. Study results have shown that as few as 150 journals account for half of what is cited and a core of only approximately 2,000 journals account for about 85% of published articles and 95% of cited articles [6]. Thus, the citation data available in SSCI and SCI should provide a valid picture of the knowledge contributions of journals. c References
1. Barnes, S.J. Assessing the value of IS journals. Commun. ACM 48, 1 (Jan. 2005), 110112. 2. Baskerville, R. and Myers, M. Information systems as a reference discipline. MIS Quarterly 26, 1 (2002), 114. 3. Cote, J., Leong, S., and Cote, J. Assessing the influence of Journal of Consumer Research: A citation analysis. Journal of Consumer Research 18, Dec. (1991), 402410. 4. Culnan, M.J. and Swanson, E. Research in management information

systems, 1980-1984: Points of work and reference. MIS Quarterly 10, 3 (1986), 289302. 5. Davenport, T. and Short, J. The new industrial engineering: Information technology and business process redesign. Sloan Management Review 31, 4 (1990), 1127. 6. Garfield, E. The significant scientific literature appears in a small core of journals. Scientist 10, 17 (1996), 13. 7. Hamilton, S. and Ives, B. Knowledge utilization among MIS researchers. MIS Quarterly 6, 4 (1982), 6177. 8. Katerattanakul, P., Han, B., and Hong, S. Objective quality ranking of computing journals. Commun. ACM 46, 10 (Oct. 2003), 111114. 9. Leong, S.M. A Citation analysis of the Journal of Consumer Research. Journal of Consumer Research 15, (Mar. 1989), 492497. 10. Markus, M.L. Power, politics, and MIS implementation. Commun. ACM 26, 6 (June 1983), 430444. 11. Sharplin, A. and Mabry, R.H. The relative importance of journals used in management research: An alternative ranking. Human Relations 38, 2 (1985), 139149.

Pairin Katerattanakul (pairin@wmich.edu) is an associate professor in the Department of Business Information Systems, Western Michigan University, Kalamazoo, MI. Bernard Han (bernard.han@wmich.edu) is a professor in the Department of Business Information Systems, Western Michigan University, Kalamazoo, MI. Alan Rea (alan.rea@wmich.edu) is an associate professor in the Department of Business Information Systems, Western Michigan University, Kalamazoo, MI.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. 2006 ACM 00052-0782/06/0500 $5.00

118

May 2006/Vol. 49, No. 5 COMMUNICATIONS OF THE ACM