Researchers from Universidad Carlos III de Madrid (UC3M), Universidad de León (ULE) and São Paulo State University (UNESP) in Brazil have developed an indicator which is more robust, clear and fair than ‘impact factor’, which has been widely used for decades to evaluate academic and scientific journals.

This new indicator, which they have called ‘Real Influence’, has numerous practical applications, ranging from the evaluation of research projects and accreditations to tenure applications and the identification of unusual citation patterns in scientific journals. According to one of its authors, Antonio Perianes, professor in the Department of Library and Information Science at UC3M, the indicator “aims at a rational and contextualised evaluation of scientific journals, not a magic formula for decision-making based solely on numbers.” He adds, “It is not intended to be a single metric (since it cannot capture all the complexity of science) and should always be complemented with qualitative evaluations of scientific publications.”

Real Influence seeks to provide an improved alternative to impact factor, which calculates the average number of citations per scientific publication. The issue with this system, according to the researchers, is that it provides information about the individual visibility of each publication but is not immune to distortions caused by highly cited articles, among other drawbacks. For example, the impact factor measurement tends to be higher in fields where publications and citations are more frequent (such as biomedical sciences or technology) or does not reflect the long-term impact of a publication (since it typically measures citations within two years after publication).

“Our initial goal in creating Real Influence was to develop a tool that would make it possible to show the visibility of all publications in a journal and allow for comparisons between them,” explains Perianes. “This approach enables comparisons of journal distributions of different sizes without distortions, which is especially important as it avoids disadvantages for journals with lower output compared to those with significantly higher publication volumes.”

To conduct the study, recently published in the journal Quantitative Science Studies, the team compared the performance of the new indicator across nearly four hundred journals in the fields of Library and Information Science, Biochemistry and Molecular Biology. The analysis results show that Real Influence is less vulnerable to manipulation, better represents the complete distribution of publications and, above all, provides a more detailed and fairer perspective based on the visibility of each article.

The methodology of Real Influence is inspired by the use of percentiles in fields such as economics or paediatric growth. This approach avoids biases caused by exceptionally cited articles by not only evaluating the citations received, but also considering the relative position of each article within the universe of citations within its category. This makes it easily adaptable and implementable in any data system compatible with citation studies, such as WoS or Scopus.

Video: https://www.youtube.com/watch?v=p2qWH4VRyHk