A new tool for evaluating health equity in academic journals; the Diversity Factor.

• Authors: Jack Gallifant, Imperial College London; Joe Zhang, Imperial College London; Stephen Whebell, Townsville University; Justin Quion, University of the East Ramon Magsaysay Memorial Medical Center; Braiam Escobar, Escuela de Ingeniería de Antioquia; Judy Gichoya, Emory University; Karen Herrera, Military Hospital; Ruxana Jina, National Institute for Occupational Health; Swathikan Chidambaram, Imperial College London; Abha Mehndiratta, Center for Global Development; Richard Kimera, Mbarara University of Science and Technology; Alvin Marcelo, University of the Philippines Manila; Portia Grace Fernandez-Marcelo, University of the Philippines Manila; Juan Sebastian Osorio, ScienteLab; Cleva Villanueva, Escuela Superior de Medicina; Lama Nazer, King Hussein Cancer Center; Irene Dankwa-Mullan, George Washington University; Leo Anthony Celi, Massachusetts Institute of Technology
• Journal Title: PLOS Glob Public Health
• Volume: Volume 3, Number 8
• Publisher: PLOS | 2023, Pages e0002252-e0002252
• Type of Work: Article | Final Publisher PDF
• Publisher DOI: http://doi.org/10.1371/journal.pgph.0002252
• Subjects: Health Sciences, Education
• Keywords: citation; Impact Factor; researcher’s contribution; Diversity Factor

Abstract

Current methods to evaluate a journal's impact rely on the downstream citation mapping used to generate the Impact Factor. This approach is a fragile metric prone to being skewed by outlier values and does not speak to a researcher's contribution to furthering health outcomes for all populations. Therefore, we propose the implementation of a Diversity Factor to fulfill this need and supplement the current metrics. It is composed of four key elements: dataset properties, author country, author gender and departmental affiliation. Due to the significance of each individual element, they should be assessed independently of each other as opposed to being combined into a simplified score to be optimized. Herein, we discuss the necessity of such metrics, provide a framework to build upon, evaluate the current landscape through the lens of each key element and publish the findings on a freely available website that enables further evaluation. The OpenAlex database was used to extract the metadata of all papers published from 2000 until August 2022, and Natural language processing was used to identify individual elements. Features were then displayed individually on a static dashboard developed using TableauPublic, which is available at www.equitablescience.com. In total, 130,721 papers were identified from 7,462 journals where significant underrepresentation of LMIC and Female authors was demonstrated. These findings are pervasive and show no positive correlation with the Journal's Impact Factor. The systematic collection of the Diversity Factor concept would allow for more detailed analysis, highlight gaps in knowledge, and reflect confidence in the translation of related research. Conversion of this metric to an active pipeline would account for the fact that how we define those most at risk will change over time and quantify responses to particular initiatives. Therefore, continuous measurement of outcomes across groups and those investigating those outcomes will never lose importance. Moving forward, we encourage further revision and improvement by diverse author groups in order to better refine this concept.