A critical evaluation of the algorithm behind the Relative Citation Ratio (RCR).

• Type of Material: Article
• Authors: Anna Janssens, Emory University; Michael Goodman, Emory University; Kimberly Powell, Emory University; Marta Gwinn, Emory University
• Language: English
• Date: 2017-10-02
• Publisher: Public Library of Science
• Publication Version: Final Published Version
• Copyright Statement: © 2017 Janssens et al
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.1371/journal.pbio.2002536
• Title of Journal or Parent Work: PLoS Biology
• ISSN: 1544-9173
• Volume: 15
• Issue: 10
• Start Page: e2002536
• End Page: e2002536
• Grant/Funding Information: The author(s) received no specific funding for this work.
• Keywords: Citation analysis; Peer review; Scientific publishing; Health care policy; Algorithms; Meta-analysis; Bibliometrics; Article-level metrics
• Research Categories: Health Sciences, Public Health; Health Sciences, Epidemiology

Abstract

The influence of scientific publications is increasingly assessed using quantitative approaches, but most available metrics have limitations that hamper their utility [1]. Hutchins and colleagues recently proposed the Relative Citation Ratio (RCR) [2], which compares the citation rate of an article against the citation rate that is expected for its field. The metric is an attractive and intuitive solution to indicate whether an article is cited more or less frequently than its “peer” publications. As a ratio of rates, RCR is an article-level metric that is field and time independent and strongly correlated with peer review evaluations [2]; the metric was central in the proposed (and withdrawn) grant management policy of the National Institutes of Health (NIH) [3] even though the RCR has been criticized for lacking a theoretical model, having insufficient transparency, and having poor correlation with other peer evaluations [4,5]. We analyzed the algorithm behind the RCR and report several concerns about the calculation of the metric that may limit its utility.