Beruflich Dokumente
Kultur Dokumente
Article information:
To cite this document: Ian Rowlands, (2002),"Journal diffusion factors: a new approach to measuring research influence", Aslib
Proceedings, Vol. 54 Iss: 2 pp. 77 - 84
Permanent link to this document:
http://dx.doi.org/10.1108/00012530210435211
Downloaded on: 13-12-2012
References: This document contains references to 13 other documents
Citations: This document has been cited by 15 other documents
To copy this document: permissions@emeraldinsight.com
This document has been downloaded 943 times since 2005. *
Ian Rowlands, (2002),"Journal diffusion factors: a new approach to measuring research influence", Aslib Proceedings, Vol. 54 Iss:
2 pp. 77 - 84
http://dx.doi.org/10.1108/00012530210435211
Ian Rowlands, (2002),"Journal diffusion factors: a new approach to measuring research influence", Aslib Proceedings, Vol. 54 Iss:
2 pp. 77 - 84
http://dx.doi.org/10.1108/00012530210435211
Access to this document was granted through an Emerald subscription provided by BROWN UNIVERSITY
For Authors:
If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service.
Information about how to choose which publication to write for and submission guidelines are available for all. Please visit
www.emeraldinsight.com/authors for more information.
About Emerald www.emeraldinsight.com
With over forty years' experience, Emerald Group Publishing is a leading independent publisher of global research with impact in
business, society, public policy and education. In total, Emerald publishes over 275 journals and more than 130 book series, as
well as an extensive range of online products and services. Emerald is both COUNTER 3 and TRANSFER compliant. The organization is
a partner of the Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive
preservation.
*Related content and download information correct at time of download.
Journal impact
Journal diffusion Journal impact factors (JIFs) are widely
factors: a new approach accepted as a simple, useful and convenient
to measuring research shorthand notation for calibrating the relative
standing of serials titles (Garfield, 1994). Their
influence publication, on an annual basis by ISI1, is
eagerly awaited by bibliometricians, research
Ian Rowlands administrators, journal publishers and
(increasingly) by academics and researchers.
The results can facilitate or hinder success in
grant competitions, influence journal
acquisitions or disposal policies, and make or
break the careers of researchers, editors and
publishers. Impact factors matter.
Impact factors naturally have their critics.
The author More thoughtful members of the
bibliometrics community have pointed to
Ian Rowlands is a Senior Lecturer, CIBER: Centre for
certain technical problems with the way that
Information Behaviour and the Evaluation of Research,
impact factors are usually calculated (see for
Department of Information Science, City University,
example Davis, 1998; Moed et al., 1996;
London, UK.
Moed and van Leeuwen, 1995; Lewison,
2001) in a spirit of constructive criticism.
Keywords Other commentators, unconvinced that a
Libraries, Research, Measurement, Journal publishing concept as subtle as ``research quality'' can
really be inferred from counts of citation
Abstract tokens, are openly hostile to what they regard
as the bean-counting tendencies of those who
This paper introduces a new bibliometric tool, the journal
use impact factors for evaluating research
diffusion factor. An argument is presented that the
(Lehrl, 1999).
bibliometric indicators commonly used to measure the
The perspective of this paper is to blissfully
quality of research (journal impact factor, immediacy
ignore both the technical and principled
index and cited half-life) offer little insight into the
objections to ISI1 journal impact factors and
transdisciplinary reception (thus the wider influence) of
to try to strike out instead in a new direction.
journals. The journal diffusion factor describes a neglected
Like them or loathe them, impact factors are
dynamic of citation reception and is intended as a
here to stay. The question posed here is
complementary partial indicator for research evaluation
whether they actually represent what they are
purposes, to be read alongside existing well-established
indicators.
generally taken to represent: a journal's
standing, prestige or (more prosaically) its
research utility, or whether they should be
Electronic access
interpreted more carefully and within
The research register for this journal is available at narrower terms of reference.
http://www.emeraldinsight.com/researchregisters
and reviews. ``Recent'' is an elastic concept, Nevertheless, throughout the scientific and
but in practice usually means two or five research policy literature, impact factors are
years. In the example below, a two-year implicitly and explicitly associated with values
diachronous journal impact factor for Aslib of research quality, utility and influence. Is it
Proceedings for 1998 is calculated. reasonable to impute a single partial indicator
with so much intellectual baggage?
Number of recent citations (the Chambers Dictionary defines ``quality'' as
numerator) relating to a ``high grade of excellence'',
The number of citations received during 1998 ``utility'' as ``usefulness, the power to satisfy
by Aslib Proceedings articles and reviews wants'', and ``influence'' as the ``power of
published in the previous two years, 1996 and producing an effect'' or ``exertions of friends
1997: at court, wire pulling, and the like''. Thinking
about the motivations that structure authors'
1996 + 1997 = 21 citations.
citation and publication behaviour, elements
of all three definitions seem to have some
Number of recent publications (the
relevance! But the lack of clarity surrounding
denominator)
the independent variables which are
The number of articles and reviews published
associated with impact factors in empirical
in Aslib Proceedings during 1996 and 1997:
studies should be a source of some concern,
1996 + 1997 = 75 articles and reviews. especially given the controversial nature of
much data-driven research evaluation.
Impact factor calculation
JIF98= 21/75 = 0.3
Citations and the ripple effect
All things being equal, impact factors should
offer a rational basis for comparing the The dynamics of how ideas are transferred from
relative ``performance'' of two or more titles. one author to another, and from one discipline
However, all things are rarely equal, and it to another, is a central concern within scholarly
will serve us well to remember that impact communication and it is surely relevant to some
factors are heavily influenced by subject field, conception of research quality or influence. If
document type and journal size (Mabe and we regard new ideas as being like pebbles
Amin, 2000), by numbers of citations (Egghe, thrown into a pond (where the surface of the
1988), and by research level, shifting fashions pond represents the general research literature)
and publication policy (Lewison, 2001) as we can draw upon two potentially useful
well as by absolute quality, whatever that metaphors: the size of the splash as the pebble
means. Tomer (1986) concludes that impact hits the surface of the water, and the
factors are useful insofar as they aid the characteristics of the resulting ripples.
identification of what is probably the Mabe and Amin (2000) relate the three
relatively small number of journals that standard bibliometric measures (impact
receive disproportionately high numbers of factor, immediacy index and cited half-life)
citations. He also points out that they are published in ISI1's annual Journal Citation
otherwise uninformative derivations, Reports series to a generalised citation curve.
producing rank orderings which are not Their work is carefully presented, and I
dissimilar to uncorrected rates of citation apologise to both authors by playing fast and
(Tomer, 1986). loose with their ideas and relating them to the
Intellectual influence and quality are pebble-pond metaphor. Mabe and Amin
difficult ideas to conceptualise, let alone argue that impact factors measure the relative
measure, and it is noticeable how careful area under the citation curve within the
some bibliometricians are with their use of two-year time window (i.e. the mass of the
language. Thus Mabe and Amin (2000) note pebble); that the immediacy index represents
simply that impact factors measure the way the initial gradient of the citation curve (i.e.
that a journal ``receives citations to its articles the acceleration of the pebble as it hits the
over time'', while Bordons and Zulueta surface of the water); while cited half-life
(1999) consider that impact factors are better indicates the rate of decline of the citation
regarded as a measure of a journal's curve (i.e. the time taken for calm to begin to
``visibility'' rather than its scientific quality. return to the surface of the pond).
78
Ian Rowlands Volume 54 . Number 2 . 2002 . 77±84
If these analogies hold, then it follows that Two points leap out immediately from this
journal impact factors are best regarded as distribution. The first is that the journal itself,
only partial indicators of research impact, at Scientometrics, accounts for the largest single
least in the limited technical sense of contribution to its own citations (a common
describing particular aspects of citation phenomenon), followed by a highly
dynamics. To fully understand the ``splash'', productive core of closely related journals.
they need to be read carefully alongside The second is that what follows is a long
immediacy and cited half-life. However, and diffuse tail including many citations from
this is the main point of this paper, they tell us titles outside of the immediate discipline.
nothing directly about the subsequent ripples, Yanovsky (1981) suggests that a better
the ``breadth'' of the reception of a particular understanding of the factors that shape
journal in the marketplace for ideas. journal citation patterns such as these could
This paper introduces a simple ``ripple'' be used to inform serials acquisitions policies,
measure which is intended to complement the as well as providing insights into disciplinary
three established ``splash'' indicators structures.
discussed above: a journal diffusion factor.
Figure 1 Retrieving a set of journal citing papers from SSCI1 in Dialog Classic
79
Ian Rowlands Volume 54 . Number 2 . 2002 . 77±84
Figure 3 Online method for calculating journal diffusion factors from SSCI1 in Dialog Classic
The number of citations in the denominator is individual citations rather than citing papers
found by simply inspecting the online since there may be more than one unique
postings using a simple correction citation to Scientometrics in a given article.
(Christensen et al., 1997). It should be noted The journal diffusion factor is expressed
that this method recovers numbers of here as the number of unique citing journal
80
Ian Rowlands Volume 54 . Number 2 . 2002 . 77±84
titles per 100 citations, and may range journal diffusion factors for 1998. The
theoretically between one and 100. As with diffusion factors range between 38.2 (Social
journal impact factors, the size of the time Science Information) to 5.1 (Journal of
window may be varied (a two-year factor is Government Information) with an average value
calculated in this instance), and both of 20.0. The third column shows the rankings
synchronous and diachronous methods might that obtain when the journals are ranked by
be employed (Ingwersen et al., 2000). the comparable journal impact factors as
The data in Table I comprise 42 library and published by ISI1 in the Journal Citation
information science titles ranked by two-year Report for 1998. The two lists are very
81
Ian Rowlands Volume 54 . Number 2 . 2002 . 77±84
distinct: only 14 of the 42 journals even Interpretation of the data in Table I is difficult
remain in the same half of the distribution and possibly premature. It may be, for
when ranked in this way (note that Aslib example, that high diffusion journals play an
Proceedings leaps up 33 places!). important role in science as ideas aggregators,
As a very general principle, journals in the spinning together ideas from closely-related
top third of the list comprise titles which are specialities and knitting them into the fabric
more oriented towards the general academic of neighbouring disciplines. Conversely, low
reader (e.g. Libri), the practitioner (e.g. diffusion journals may play a quite distinct
E-Content, formerly Database), or which have role by providing a sharp focus for specialists
a high social sciences content (e.g. Social in, for example, bibliometrics (Scientometrics)
Science Information), or all three or federal information policy (Journal of
characteristics (e.g. Aslib Proceedings). Titles Government Information) to explore the limits
in the lower third of the list are generally more of their field.
highly specialised and scholarly in nature (e.g.
Scientometrics, Journal of Government
Information). Since diffusion and impact
Formal characteristics of the journal
factors are designed to measure different
diffusion factor
aspects of the reception process, it is perhaps
hardly surprising that the ranked listings are As noted earlier, it is intended that the journal
so different. It is worth bearing in mind that diffusion factor should be read alongside
``high diffusion'' does not at all imply a large existing ISI1 indicators, notably the journal
relative number of citations given that the impact factor, immediacy index and cited
diffusion factor standardises the number of
half-life. This section explores the formal
citing titles per 100 citations. Nonetheless,
characteristics of the diffusion factor in
the introduction of this new indicator does
relation to these existing and widely used
suggest some interesting questions about
tools.
what citations can tell us about research
Journal titles were selected using random
quality or influence. Consider Figure 4.
number tables from four contrasting ISI1
Which scenario depicts the most ``influential''
subject categories:
journal, B or C? This question is posed simply
(1) astronomy and astrophysics;
as an illustration of the additional dimension,
(2) immunology;
which the notion of diffusion brings to
(3) information science and library science;
thinking about what journal ``impact'' really
and
means. Its application to understanding
(4) orthopaedics.
transdisciplinarity and boundary spanning
issues are strikingly evident, and the indicator In each case, the samples represent at least 50
might be of great interest to editors and per cent of all the titles from the relevant
journal publishers. subject category. The fields were selected to
82
Ian Rowlands Volume 54 . Number 2 . 2002 . 77±84
83
Ian Rowlands Volume 54 . Number 2 . 2002 . 77±84
doors, colour, price, or the ready availability program evaluations'', in Cronin, B. and Atkins, H.B.
of spare parts but by all these characteristics (Eds), The Web of Knowledge, ASIS Monograph
and more taken together. The utility and Series, Medford, NJ, pp. 373-87.
Lehrl, S. (1999), ``Evaluating scientific performance by
limitations of the journal diffusion factor
impact factors: the right for equal opportunities'',
should be seen in that light. Strahlentherapie und Onkologie, Vol. 175 No. 4,
pp. 141-53.
Lewison, G. (2001), ``Impact factors revisited'',
unpublished manuscript.
References Mabe, M. and Amin, M. (2000), ``Impact factors: use and
abuse'', Perspectives in Publishing, Vol. 1, available
Bordons, M. and Zulueta, M.A. (1999), ``EvaluacioÂn de la
at: www.elsevier.com/homepage/about/ita/editors/
actividad cientõÂfica a traveÂs de indicadores
bibliomeÂtricos'', Revista EspanÄola de Cardiologia perspectives1.pdf (accessed 6 December 2001).
Online, Vol. 52 No. 10, available at: Moed, H.F. and van Leeuwen, T.N. (1995), ``Improving the
www.revespcardiol.org/cgi-bin/wdbcgi.exe/cardio/ accuracy of Institute for Scientific Information's
mrevista_cardio.fulltext?pident=190 (accessed journal impact factors'', Journal of the American
6 December 2001). Society for Information Science, Vol. 46 No. 6,
Christensen, F.H., Ingwersen, P. and Wormell, I. (1997), pp. 461-7.
``Online determination of the journal impact factor Moed, H.F., Van Leeuwen, T.N. and Reedijk, J. (1996),
and its international properties'', Scientometrics, ``A critical analysis of the journal impact factors of
Vol. 40 No. 3, pp. 529-40. Angewandte Chemie and the Journal of the
Davis, J.D. (1998), ``Problems in using the Social Sciences American Chemical Society: inaccuracies in
Citation Index to rank economics journals'', published impact factors based on overall
American Economist, Vol. 42 No. 2, pp. 59-65. citations only'', Scientometrics, Vol. 37 No. 1,
Egghe, L. (1988), ``Mathematical relations between impact pp. 105-16.
factors and average number of citations'', Tomer, C. (1986), ``A statistical assessment of two
Information Processing & Management, Vol. 24 measures of citation: the impact factor and the
No.5, pp. 567-76. immediacy index'', Information Processing &
Garfield, E. (1994), ``The impact factor'', Current Management, Vol. 22 No. 3, pp. 251-8.
Contents1, Vol. 25, 20 June, pp. 3-7. Yanovsky, V.I. (1981), ``Citation analysis significance of
Ingwersen, P., Larson, B. and Wormell, I. (2000), scientific journals'', Scientometrics, Vol. 3(n),
``Applying diachronous citation analysis to research pp. 223-33.
84