Chapter 6 What is Impact Factor, and why is it so important?

The impact factor of a journal relates to the number of times each publication from the journal gets cited in the two years preceding the date of the Impact Factor (IF) (Equation (6.1)). Thus, if you are thinking of publishing in a journal that has an IF of 1 you might expect that in the two years following the publication of your article you may get one citation. But as this is an average for all publications appearing in this journal, it is not necessarily true for your paper. As discussed elsewhere you might be very good at publicising your work and have it extensively cited. One or two extensively cited papers might even change the impact factor of the journal if it doesn’t have so many publications per year. If on the other hand you are thinking of publishing your article in a journal that has an IF of 5, you might expect that your article will be cited five times more than if you published in the first journal (IF = 1).

It’s a relatively simple calculation as seen in equation (6.1):

\[\begin{equation*} \frac{\mbox{The sum of all citations in journal X for year Y }}{(\mbox{No. pubs in journal X for year Y-1}) + (\mbox{No. pubs in journal X for year Y-2})} \tag{6.1} \end{equation*}\]

Because all citations for year Y are needed before the IF can be calculated for each journal, IF for the preceding two years is typically not released until June of Y+1.

Impact Factors are published by a number of different literature databases. For example, for the impact factor calculated by the Web of Science, if your journal is not even listed in the Web of Science then it will not have any Impact Factor. The Web of Science is continually policing the quality of the journals, and this means from time to time journals are excluded. This tends to happen at the lower end of the Impact Factor scale. But recently it happened to some very well known journals and there was a big stink. You can read more about this here.

Note that there are conflicts of interest with publishing impact factors. For example, a publisher, Elsevier, owns Scopus and can decide whether or not a publication can get an Impact Factor. Similarly, the new scholarly database, Dimensions, is owned by Holtzbrinck who also own SpringerNature. This is really only the tip of the iceberg when it comes to each publishing company’s intention to capture the entire academic cycle (see Brembs et al., 2021). Hence, given two equivalent journals, do you think is it more likely that journals published by Elsevier will be included or excluded from Scopus? Organisations that are used by many of our employers in their means of evaluating our effectiveness are owned by private companies. This is certainly cause for concern. There is a group of people who are trying to replace Impact Factor with a group of other metrics, so perhaps by the time you read this Impact Factor will no longer be relevant. If you’re interested read more here.

Do remember that there are a number of other citation indices including Google Scholar. Google Scholar doesn’t currently calculate the Impact Factor for all journals, only the top 100.

6.1 From a simple score to a way of life

When IF was originally devised by Eugene Garfield in 1955, it wasn’t supposed to govern the lives of academics, it was simply intended to be a way of deciding which journals to include in the Science Citation Index (Garfield, 1999). It then became useful for librarians to help them decide which journals to keep and which to ditch under ever constrained budgets (caused by publishers’ ever increasing prices). But along the way, this very simple index is now considered by many people to be a measure of quality, prestige and even academic success (Garfield, 1999). Many people have highlighted how wrong these beliefs are, but the growing trouble is that not only have many academics been misled, but so have administrators responsible for hiring and promotions.

“Like nuclear energy, the impact factor has become a mixed blessing.”

—Eugene Garfield (1999)

In a paper by McKiernan et al. (2019), they found that IF features in the guidelines of many university panels responsible for the fate of academics jobs and therefore lives. Worryingly, many of these institutions don’t actually talk about what IF measures. Instead they equate it with values and qualities that it certainly does not represent. Thus, you may find that your career is influenced by a simple metric that almost all who use it don’t actually understand. The undue influence on lives of scientists that IF has led directly to the San Francisco Declaration on Research Assessment (known as DORA) to which many institutions and publishers have signed up. You should read this very simple declaration and find out whether your institution is a signatory. If they are, then remember to hold them to the DORA principles during any assessment that you undergo.

The Impact Factor now dominates many aspects of life for Early Career Researchers, where the pay-off for a high impact publication might make the difference between having a job or leaving science altogether. The pressure is so high that it leads to misconduct and fraud.

6.2 Five year Impact Factor

Many journals report their ‘Five Year Impact Factor’ in addition the the standard two year timeframe (as seen in equation @ref (eq:Impact-Factor)). This is because many disciplines, such as biological sciences, don’t have maximum impact of articles within the first two years of publication as do subjects like medicine and physics. Papers with immediate impact can be equated with sensational breaking news stories that instantly grab headlines, making money for those news outlets. These ‘hot papers,’ ‘jump-decay’ or ‘fast breaking papers’ (i.e. left skewed distribution) will inflate the IF of the journal, which, if sustained, will earn the publisher more money. Most papers in biological sciences have a ‘delayed peak’ (i.e. low frequency and long tails), consistently gathering small numbers of citations over long periods of time. Papers that suddenly become popular after a long time with no or very few citations are termed ‘sleeping beauties’ (i.e. right skewed; Bornmann & Marx (2012)). The last in the group of four citation types includes those that have a constant, usually low, number of citations over time. Together, each of these four patterns can be described by three variables that describe the citation behaviour of all papers: fitness, immediacy and longevity (see Fortunato et al., 2018). Using fitness, immediacy and longevity of citations to papers within a field can help to normalise citation metrics for different areas of science.

Although the five year Impact Factor might be much more appropriate, most people ignore this metric. While this metric might be more appropriate, because of the bad ways in which people have used impact factor, it is probably better to push back against this metric as you should against others.

6.3 What can you do if you publish in a journal with high IF?

Very high ranking journals for Impact Factor are publications like Cell, Nature and Science. This is because these publications are read by a very great number of people, and so are widely cited. Articles that get published in them receive a lot of attention from the press and media. This results in the prestige that a hiring institution might be looking for. If your academics publish in this journal, your institution may well receive lots of positive publicity. In some countries, notably China, there may be a cash incentive towards publishing in a journal with a high IF (Quan, Chen & Shu, 2017).

One frightening trend that we are seeing in biological sciences is that the higher the Impact Factor the more the journal will charge you to publish in it. At the time of writing in November 2020 Nature has just announced that they will charge $9500 to publish in their highest-ranked journal (see Part II). These sums are an order of magnitude greater than journals like these (with rejection rates >90%) should reasonably charge (Grossmann & Brembs, 2021). This is more money than it cost to publish in any other journal at the moment, and will be greater than the cost of many research projects, or even salaries of Early Career Researchers in some parts of the world.

6.4 Why is IF so important?

Academics are measured by their productivity but also about the quality of their output. Because there are so many different academic disciplines the bean counters who administer us need some way of ranking academics against each other. This is why they use the Impact Factor of the journals in which their academics publish in order to determine the quality of their output. Even though there are other metrics of the actual quality of an academic, most administrators continue to cling to IF and their beliefs of what it stands for.

Some countries reward their academics if they publish in high ranking journals. This can result in a salary bonus (Quan, Chen & Shu, 2017). It may also help with promotion, getting tenure or even just getting an interview for a job (Schimanski & Alperin, 2018; McKiernan et al., 2019). If you’re going to publish and you want a career in academia then you need to be aware of Impact Factors and what they mean to different stakeholders.

Many people will complain that their particular sub-discipline has a range of very low ranking journals with low impact factors. Others complain that journals with high impact factors tend to be edited by an old boys club that facilitates the members. In some cases like PNAS this is certainly true.

6.5 Editors try to increase IF

It’s important to remember that editors care about Impact Factor (see Ioannidis & Thombs, 2019). There are several reasons for this. Firstly, the impact factor of the journal can be used (by the publisher or society) as a simple measure of how well the editor is doing. Secondly, the higher the impact factor of the journal, the better quality of submissions of manuscripts to publish. Being the editor of a journal with a low (or no) impact factor can result in a lot of mediocre manuscripts being submitted. Editors can only choose their content from what is submitted. Poor manuscripts take up much more time than good ones: more rounds of review, more disagreements among reviewers and more time spent making editorial decisions. Thus, by increasing the IF of the journal that you edit, you are likely to increase both the number of submissions (allowing you to reject poorer ones) and retain better ones.

All this means that if editors believe that your paper will not garner the same or more citations in two years as the current Impact Factor of their journal, they may desk reject. This is just one of the ways in which editors are known to manipulate impact factors for their journals. Established ways (Metze, 2010; Martin, 2016) of editors increasing impact factor for their journals include:

  1. Ask authors to cite publications from your journal published within the last 2 years.
  2. Ask reviewers to suggest publications from your journal published within the last two years to authors on which their review is conducted.
  3. Encourage the submission of papers from laboratories with high output and citation rates.
  4. Reject papers that are likely to have no citations. This effectively reduces the size of the denominator in the above equation.
  5. Publishing issues in January means they have a maximum period of the year to get cited. This is now being inflated to having issues published online well ahead of the January date all the time gathering citations.
  6. Encourage review articles which themselves garner more citations.
  7. Editorials that cite every paper in the journal. This tactic is frequently used in special issues.

Evidence of this unscrupulous wrangling of Impact Factor within the Biological Sciences has been found when looking at the differentiation of inflation rates between journals. Wilson (2007) found that for-profit journals managed to increase their IF faster than society journals within the same period.

6.5.1 Negotiating your IF

As the number of citations from your published content is divided by the number of papers, one way of improving the IF of a journal is to reduce the number of papers that are counted towards the denominator in the calculation made by the citations database. It has been known for some time that those journals with the highest IF negotiate the removal of all of their editorial and news content from their denominators, making the number of publications much smaller and hence the IF larger (Garfield, 1999; Adam, 2002). On the other hand, if any of these news articles or commentaries get citations, these are included in the addition to the numerator. Thus our favourite high impact journals, Nature and Science, can publish very citable news at the front of their magazine, but negotiate with the commercially minded database owners about exactly which of their content counts towards their IF (Brembs, Button & Munafò, 2013). Later, we will see evidence of the financial leverage that these negotiations can reward these top tier journals (see Part IV).

6.6 Push back against IF

Just like any metric, Impact Factor is liable for abuse. You need to be aware of how IF is used and abused by many people in the academic community. You also need to be aware of what the rewards are for these individuals. Our problem with Impact Factor is not really the way in which it is manipulated by individuals to achieve their own ends. Instead, we should be worried about the way in which it leads the scientific community towards bad science and dishonesty. People who have benefited from using IF to measure their careers are likely to object if their institutions abandon it (e.g. Chawla, 2021), even though they are already signatories of DORA. Retaining IF benefits these senior academics, their closed practices and the publishing industry who use this metric (that they own and police themselves) to extort money from taxpayers earmarked for research into their own pockets (more on this in Part IV).

If you must calculate IF, one very simple way that you can push back against industry calculated IF is to calculate IF scores for your own papers, and show how they relate to the IF of the journal that you publish in. In this way, you are simply comparing your actual citations in the two years after your paper is published with the mean for the journal. There is an even chance that you generally get more citations than the mean for the journal, and you can convincingly show that your citations are consistently higher than the journal IF. For this to be true, you might need to help your work get cited, and that’s the subject of another section.

References

Adam D. 2002. The counting house. Nature 415:726–729. DOI: 10.1038/415726a.
Bornmann L, Marx W. 2012. The Anna Karenina principle: A way of thinking about success in science. Journal of the American Society for Information Science and Technology 63:2037–2051. DOI: 10.1002/asi.22661.
Brembs B, Button K, Munafò MR. 2013. Deep impact: Unintended consequences of journal rank. Frontiers in Human Neuroscience 7:291. DOI: 10.3389/fnhum.2013.00291.
Brembs B, Huneman P, Schönbrodt F, Nilsonne G, Susi T, Siems R, Perakakis P, Trachana V, Ma L, Rodriguez-Cuadrado S. 2021. Replacing academic journals. Zenodo. DOI: 10.5281/zenodo.5526635.
Chawla DS. 2021. Scientists at odds on Utrecht University reforms to hiring and promotion criteria.
Fortunato S, Bergstrom CT, Börner K, Evans JA, Helbing D, Milojević S, Petersen AM, Radicchi F, Sinatra R, Uzzi B, Vespignani A, Waltman L, Wang D, Barabási A-L. 2018. Science of science. Science 359:eaao0185. DOI: 10.1126/science.aao0185.
Garfield E. 1999. Journal impact factor: A brief review. CMAJ 161:979–980.
Grossmann A, Brembs B. 2021. Current market rates for scholarly publishing services. F1000Research 10:20. DOI: 10.12688/f1000research.27468.2.
Ioannidis JPA, Thombs BD. 2019. A user’s guide to inflated and manipulated impact factors. European Journal of Clinical Investigation 49:e13151. DOI: https://doi.org/10.1111/eci.13151.
Martin BR. 2016. Editors’ JIF-boosting stratagems – Which are appropriate and which not? Research Policy 45:1–7. DOI: 10.1016/j.respol.2015.09.001.
McKiernan EC, Schimanski LA, Nieves CM, Matthias L, Niles MT, Alperin JP. 2019. Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations.
Metze K. 2010. Bureaucrats, researchers, editors, and the impact factor: A vicious circle that is detrimental to science. Clinics 65:937–940. DOI: 10.1590/S1807-59322010001000002.
Quan W, Chen B, Shu F. 2017. Publish or impoverish: An investigation of the monetary reward system of science in China (1999-2016). Aslib Journal of Information Management 69:486–502. DOI: 10.1108/AJIM-01-2017-0014.
Schimanski LA, Alperin JP. 2018. The evaluation of scholarship in academic promotion and tenure processes: Past, present, and future. F1000Research. DOI: 10.12688/f1000research.16493.1.
Wilson AE. 2007. Journal Impact Factors Are Inflated. BioScience 57:550–551. DOI: 10.1641/B570702.