A critical evaluation of the algorithm behind the Relative Citation Ratio (RCR).

• Authors: Anna Janssens, Emory University; Michael Goodman, Emory University; Kimberly Powell, Emory University; Marta Gwinn, Emory University
• Journal Title: PLoS Biology
• Volume: Volume 15, Number 10
• Publisher: Public Library of Science | 2017-10-02, Pages e2002536-e2002536
• Type of Work: Article | Final Publisher PDF
• Publisher DOI: http://doi.org/10.1371/journal.pbio.2002536
• Subjects: Health Sciences, Epidemiology; Health Sciences, Public Health
• Keywords: Citation analysis; Bibliometrics; Scientific publishing; Algorithms; Meta-analysis; Article-level metrics; Health care policy; Peer review

Abstract

The influence of scientific publications is increasingly assessed using quantitative approaches, but most available metrics have limitations that hamper their utility [1]. Hutchins and colleagues recently proposed the Relative Citation Ratio (RCR) [2], which compares the citation rate of an article against the citation rate that is expected for its field. The metric is an attractive and intuitive solution to indicate whether an article is cited more or less frequently than its “peer” publications. As a ratio of rates, RCR is an article-level metric that is field and time independent and strongly correlated with peer review evaluations [2]; the metric was central in the proposed (and withdrawn) grant management policy of the National Institutes of Health (NIH) [3] even though the RCR has been criticized for lacking a theoretical model, having insufficient transparency, and having poor correlation with other peer evaluations [4,5]. We analyzed the algorithm behind the RCR and report several concerns about the calculation of the metric that may limit its utility.