ISJ editorial: Addressing the implications of recent developments in journal impact factors
分析了2020-2021年13本信息系统期刊影响因子的异常飙升，发现少数高被引文章对JIF有显著扭曲效应，并探讨了自引、跨学科引用等影响因素，对编辑、作者和读者有参考价值。
In 2020, we noticed significant spikes in the JIFs of several leading information systems (IS) journals, both inside and outside the Association for Information Systems (AIS) basket of eight (AIS-8) premier journals. A moderate degree of JIF fluctuation is normal, and a few journals increase steadily year after year, but the changes in 2020 were remarkable. For example, the JIF for the Journal of Strategic Information Systems (JSIS) rose from 3.949 in 2019 to 7.838 in 2020; likewise, the JIF for Information & Organisation (I&O), rose from 3.300 to 6.300. Every other major IS journal saw significant increases. In analysing the patterns, our scope embraces 13 journals, beginning with the AIS-8: European Journal of Information Systems (EJIS), Information Systems Journal (ISJ), Information Systems Research (ISR), Journal of Information Technology (JIT), Journal of the Association for Information Systems (JAIS), Journal of Management Information Systems (JMIS), Journal of Strategic Information Systems (JSIS), and Management Information Systems Quarterly (MISQ). We also included five well-regarded and highly cited IS journals outside the AIS-8: Decision Support Systems (DSS), Information & Management (I&M), Information & Organisation (I&O), International Journal of Information Management (IJIM), and IT & People (IT&P). We present the JIFs for 2012–2021 in Table 1 and the year-by-year percentages of change for 2013–2021 in Table 2. Six of the AIS-8 journals and two of the others saw their highest-ever JIF increase (in red with grey background) in 2020. JSIS, for instance, increased by 111% over 2019. In 2021, we found that only one journal (EJIS) had a further dramatic increase in JIF (of 107% over 2020); most journals experienced a more modest rise in JIF (ranging from 4% to 33%), and two declined. The AIS-8 journals saw an average JIF increase of 71% in 2020 and a further increase of 20% in 2021. Figure 1 visually illustrates these trends by providing a line chart of the JIFs for the AIS-8 journals over this period. On the one hand, we see a positive trend of all major IS journals being significantly cited more in recent years than in the past, partially reflecting the growth and increased influence of the IS discipline. On the other hand, citation increases in the last couple of years are dramatic. A variety of stakeholders, including editors, publishers, authors, and readers, are sensitive to such data, so it is fair to ask, “What causes these kinds of fluctuations? Are they predictable? Are the higher JIF scores sustainable?” These questions led us to analyse the data more deeply. Clearly, the more articles that are highly cited and thus the higher the total number of citations received (the numerator), the higher the JIF. At the same time, if many articles are published (the denominator), this will attenuate the JIF. In line with this simple mathematical relationship, we observed that a key contributing factor to a high JIF is the presence of one or more especially highly cited articles. Tables 3 and 4 show the number of citations received by the top 10 most cited articles for each of the same 13 journals in 2020 and 2021, respectively. Articles with 50 or more citations are in red. Some journals have published one or more articles that are very highly cited. For example, all the top 10 articles that contribute to IJIM's 2020 and 2021 JIFs are highly cited; even the 10th most cited article of 2020 has 65 citations, which is exceeded by only three of the AIS-8 journals' most cited articles in that year. The 2021 situation is similarly polarised. The last column in Tables 3 and 4, labelled coefficient of variation (CV), is calculated as “standard deviation/mean” and shows the dispersion of the distribution of citations (for the top 10 most cited articles). A higher CV value suggests a greater dispersion; that is, some of the top 10 papers are much more highly cited than the others. Given that these data are difficult to visualise, Figures 2 and 3 depict the citation rates of the top-10 articles in the AIS-8 journals in 2020 and 2021, respectively, using radar graphs. Although some journals have higher CV rates, the clear trend is that a few key articles make the most contribution to a journal's impact factor, and often this is only a couple of articles, and it quickly spirals down from there. The 2021 data are particularly fascinating, as one highly cited article in JSIS disrupted the entire trend for all journals. Our understanding was this was a highly influential paper for a highly influential special issue, and so in terms of impact and knowledge contribution, this appears to be greatly positive for the field. The separate issue is that this kind of “spike” may skew journal-level impact factor comparisons and lead to possibly incorrect interpretations. When we examine the contribution that these highly cited articles make to the JIF of each journal, some interesting patterns emerge. First, highly cited articles can skew the JIF considerably, especially for journals that publish relatively fewer articles; at the extreme, 41.4% of JIT's 2020 JIF and 53.2% of JSIS's 2021 JIF are attributable to one article each, and in both journals, the CV is higher than 1.5. In contrast, low CV scores are also in evidence. For instance, the CVs for ISR are consistently low in both 2020 and 2021 (0.19 and 0.18, respectively). Which kinds of article are highly cited? Table 5 provides a few hints, with examples from both 2020 and 2021. First, it seems that an article approach focused on theory, reviews, research agendas, frameworks, or methods yields more citations, especially compared to one-off empirical studies. Second, the emerging topics of blockchain, COVID-19, and digital transformation (or combinations of these) also appear to be highly cited, and reviews and research agendas of these are especially highly cited. In fact, in the case of JSIS, the same article (comprising an in-depth review and research agenda) on digital transformation appears twice in Table 5 because it was the journal's most highly cited article in both 2020 (92 citations) and 2021 (264 citations). However, the skewing effect of highly cited articles is less prominent for journals that publish many articles; although IJIM's most cited article received 195 citations in 2021, this accounts for only 2.7% of the 2021 JIF due to the large number of citations that IJIM received (7185). IJIM's 2020 and 2021 CVs are a modest 0.32 and 0.33, confirming that the JIF depends less on a single article. Another aspect of the highly cited articles that contributes to the JIF is the short-term nature of their impact; each article is notionally within the citation window for two calendar years.4 These articles may well be cited for many years into the future, but their contribution to the JIF is limited to two years. Thus, as noted, JIT's most cited article in 2020 had 82 citations, contributing to 41.4% of the JIF for that year. In 2021, however, that article had fallen out of the two-year citation window, so it no longer contributed to the JIF, even though citations to the article had greatly increased since then. Consequently, JIT's JIF dropped 12%, from 5.825 in 2020 to 5.15 in 2021. Meanwhile, JSIS's most cited article in 2021 had 264 citations, contributing to 53.2% of the 2021 JIF (it contributed to 18.5% of the 2020 JIF). This article will drop out of the two-year citation window for the 2022 JIF calculation. Thus, unless it is replaced by something with a similar number of citations, we can expect both that JSIS's JIF will drop next year, and that the CV will fall. Impact factors may also reflect self-citation within a journal (i.e. an article published in Journal A citing other papers published in Journal A). We account for this phenomenon in Table 6, which sorts the percentage of self-citation in rank order (lowest percentage to highest) by years 2020 and 2021. The evidence demonstrates that journal self-citation ranges from 3.1% to 21%, with an average of over 9.5% and standard deviation of around 5%. Authors are (more or less) free to cite what they like, although we occasionally come across situations in which reviewers/editors require authors to cite certain articles as a condition of acceptance (Tarafdar & Davison, 2020). However, it may also be regarded as a matter of respect and proper discourse positioning that authors cite some articles in the journal to which they submit as a way of linking their contribution to the intellectual discussion or discourse taking place in the journal. However, if journal self-citation is taken to an extreme, it could significantly inflate a JIF score, hence our inclusion of it in this analysis. With the available data, it is also insightful to consider the sources of citations. Most of the 13 journals in the sample cite a wide range of sources. Table 7 presents general summary data about citations and sources, including (in the last column) the total number of sources from which citations are drawn. Although we do not include details here for lack of space, this provides some indication that IS research is cited in disciplines far beyond IS, including not only the more familiar management, marketing, operations management, and psychology but also tourism, chemistry, health sciences, engineering, and sustainability. We calculated the rank-order of these journals with self-citation and without self-citation: there is no difference between these in the top-eight journals. In fact, the only journals this affected slightly were ISR (slightly over-ranked from self-citation) and I&O and JAIS, which were slightly under-ranked. We thus surmise that the self-citation trends of the 13 IS journals have little actual meaningful influence on the impact factors, relative to each other. As we examined the data, there appears to be a positive relationship between the total number of citable articles in a journal and the number of different sources of citations and a journal's impact factor. These trends are visualised in Figures 4 and 5, respectively. Tables 8 and 9 present data on the extent to which the 13 journals cite one another, which varies considerably. Although all the journals except JSIS cite themselves more than they are cited by any one other journal, 4.9% of JIT's JIF comes from IT&P, and 4.5% of ISR's JIF comes from JMIS. Among the AIS-8 journals, from 9.1% (JSIS) to 26.7% (MISQ) of the citations received come from within the AIS basket. The figures rise to 12.9%–34.9% when all 13 journals are considered. Figure 6 summarises and visualises the percentage of source citations for each journal, from AIS-8 and all 13 journals, respectively. MISQ is the most dependent on citations from these sets of journals, closely followed by ISR, JAIS, and JMIS. Interestingly, IT&P, IJIM, DSS, and I&M receive relatively few citations from AIS-8 journals and are much more dependent on the other five journals. Clearly, non-IS journals also contribute to the JIFs (see Table 10). However, the numbers tend to be quite low, and it is disappointing that journal articles in the key source disciplines, such as computer science, psychology, and economics, barely cite the IS literature. The most prominent exception is Sustainability, which was the top non-IS source of citations for 11 of the 13 journals in 2020 and seven of 13 in 2021. What is special about Sustainability? It is not listed in the “Information Science and Library Science” index in which most IS journals appear but in the “Environmental Studies” and “Green and Sustainable Science and Technology” indices, where it appears in the second quartile. It is an open access journal published by MDPI, and its 24 issues yearly offer huge numbers of articles: 7183 in 2019 and 10 501 in 2020. It received 68 768 citations to those 17 684 articles in 2021, leading to JIFs of 3.251 in 2020 and 3.889 in 2021. Its most cited articles had 87 citations in 2020 and 170 citations in 2021. Its eight most cited articles in 2021 were all related to COVID-19. Sustainability makes a larger contribution to the JIFs of IS journals than any other IS or non-IS publication except (1) DSS, which received 5.3% of its citations from IEEE Access in 2020, and (2) self-citation; in 2020, Sustainability contributed to between 1.6% (JAIS) and 4.4% (JSIS) and, in 2021, to between 1.3% (JIT) and (JSIS) of the This the of this a short-term by the recent in articles or that the IS is a major to the larger could be that was the of 2021. their papers for in 2020 and some of were published in journals in 2021. was a recent this may have to and more that is a and a Thus, it to be the influence of the Sustainability journal is the general research What can we from this the recent journal citation What are the for the journals and their prominent trends the short-term skewing effect of highly cited articles, the impact of and the extent to which IS research is cited in articles in many other journals and some of which are in disciplines far from A more of the kinds of article that are highly cited may be the here suggests that reviews, research agendas, research frameworks, and methods articles are particularly well has a of and and many articles are well cited, but it may well be that many others are not cited. it is to examine the of (see Table the articles that are not cited at all and thus contribute only to the effect of the and not the effect of the in the JIF calculation. We that these data may be of to and Although a couple of journals have only one several journals figures with as many as of citable articles being numbers of articles will a journal's to see a higher JIF consider to that the articles they publish are cited. one article article a of are many For instance, a of research may to the of a of a may be and more thus more citations. A key could be and of research and wide an an article For instance, most papers have far less citations than their empirical because a number of the as compared to the because most do not and could be key Meanwhile, some papers are cited because similar research even if a trend that seems in articles on topics methods case and so We also expect that a journal is with a also has a large influence on these For example, is the and provides many and for its journals that are not at journals MISQ and JAIS, providing to journals citation in all major and so are at the other and & The other key factor is that these major have much more in and also have to marketing, articles, and so As an if a an article on and is to a for the the will (1) articles, (2) citing articles, and article article information that can be in any to paper cited, for all the and related papers to a citation and receive to article on and so of this is by MISQ or as they both are the AIS which has a more and not even citation or to cited articles. Thus, we surmise that if MISQ or was published by and with a major their JIFs be much higher than at and they also have much rates of It seems to us that the factors that rise to JIFs could a especially for journals not with major Although we the value of research frameworks, agendas, reviews, and all of which tend to be well cited, the for the citation may be that a has all the on a for others do not to all the this one article has the for and or on it in their empirical articles tend to be few and far so their impact on the JIF of a journal is if only by the two-year citation However, there is also the for us to on the of is or interesting to more into and of those or that more but impact may far in the we often to the of and and as these are not or IS journals are so for theory, and that we that many articles are not and to be that are highly cited for the to the extreme, this could lead to a lack of and a to research that one In a where authors do not to an entire article to or not to cite only a to from a the in the JIF may be those journals that publish research articles that are high or and that IS most to an of for those articles. are by because of their as well as although not all journals have this and similar are more the for are highly with authors to their the of research place on an or that of an all have an in the citations that The not be they for that is, both the of research published and the extent to which it is cited. The on may be has for the of research that may if their is because it is to any it may that an could in that a article after several of and after a year or more has out to be cited or but a general with appears to be the A more of and the citation could well patterns the of an or that that In we that the for citations is to be as a for but several could be the lack of or in a research or and the of an with the journal's scope or is to what is interesting or this and are free to they We expect (or that and at top IS journals consider with they also to their of the of a research article and Davison, Authors can also themselves by their of research and it is and of to which This is no more than a variation on & Davison, but it may contribute to the of research that to even though some it and even when it to citations by The of JIFs is to A journal-level the extent to which a research article is of high or it is by the the of articles is and this not be to citation For instance, one can also at the impact of the research beyond in or using JIFs are of journal impact and a for the general of the journal, but they are one of several such Thus, it is that we do not JIFs in such as for and and that they be with other sources of At the same time, JIFs may some indication of the extent to which a journal is to an A higher JIF that more have found the research published by that journal to be of JIF data are to when articles are it is to at trends in the JIF The short-term nature of the two-year JIF is the most often is many articles value over much longer which be in a two-year Journal to JIFs also to be about the of papers are some articles highly cited and others not cited at Some of the can be found in the self-citation scores (see Table some journals appear to authors to cite papers published in the journal to which they are (see numbers in the red some journals receive large numbers of citations from other IS journals, others do not so many citations. For instance, articles published in MISQ were cited by the other journals in our but articles published in and I&O were cited only and respectively, in the other journals, no citations at all from several of the to consider which they to an impact and thus to be cited and examine to that impact to may be a that the of articles in a more sensitive beyond the data we that journal also to at the is and if articles (or on of the articles that are published will have taken longer to the review by the they are they may no longer be to as much as those that are and published more The and of the review may be and we that all journal on the of a review However, the of is a slightly different we of time, in our as and at IS journals, we that some do not even or the (or some of these require Meanwhile, some authors appear not to about the the and so often more time, especially with COVID-19, of and the of these of is by or the for the all are The is longer and, citation as the research more As editors, we and in any is the key issue we with is that all of us do this as when comes to there is a lack of in the journal and in the IS discipline. We with the same of most of the and a large of contributing little to the review or It is to to or even to review an in order to the two to do it to and our systems beyond and The of to the and second to authors, may be by their contribute to the review are with and, with a of their to the of may not be We that if we more consistently those in the review we may also see from and even it is to consider and and as is at all of the top and In this issue of the we present eight In the that IS research on has focused on (i.e. the to and after their to a digital this the authors on can also be a that and (i.e. their and and their to a digital The show that the relationship between and to an which that a low a but a moderate degree of to These and to as both a and as a In the second at the approach with the of on the that around an between data and theory, show that the of in of the of of in a they that a for and by or to empirical data, provides a account of the from In the that in and an in of the effect of has not received with respect to since some have on an The authors a understanding of the condition which and the of in their in two of and using the In the on a case at that in an an the authors both the and the of that have three a and and an of the of as a of in and of into seven the of for further and present for In the the IS and digital with the to both if and which by different of digital in from a with within show significant positive of by by and by systems on However, the relative impact of by these digital with different of for and The authors not only empirical evidence for digital as an of but in line with an they also for and as for the of digital In the that on to information show that can lead to and the of and theory, the authors that and and to and influence and that the of on are on and The all three and and the of on and and increases In the & examine on topics on is on the of and to themselves from but for the more than of the to those with less than others with the to that with greater to out by or In the the of in a that the of from management, and this to be in to their At the same time, between three different that these their case the authors to their We to for contribution to the and in this We also and for insightful on this