Authors' rebuttal to Integrated Risk Information System (IRIS) response to "Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools"

• Type of Material: Article
• Authors: Stephanie Eick, Emory University; Dana E Goin, University of California San Francisco; Juleen Lam, University of California San Francisco; Tracey J Woodruff, University of California San Francisco; Nicholas Chartres, University of California San Francisco
• Language: English
• Date: 2022-03-23
• Publisher: BMC
• Publication Version: Final Published Version
• Copyright Statement: © The Author(s) 2022
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.1186/s13643-022-01894-8
• Title of Journal or Parent Work: SYSTEMATIC REVIEWS
• Volume: 11
• Issue: 1
• Start Page: 53
• End Page: 53
• Grant/Funding Information: Funding for this work was provided by the JPB Foundation (grant 681) and the Passport Foundation. This work was also supported by grant P01ES022841 from the National Institute of Environmental Health Sciences.
• Keywords: TRIALS; Life Sciences & Biomedicine; QUALITY; Science & Technology; Systematic review; Medicine, General & Internal; General & Internal Medicine; Risk assessment; Risk of bias
• Research Categories: Environmental Sciences; Health Sciences, Public Health

Abstract

This letter responds to the US Environmental Protection Agency’s Integrated Risk Information System (IRIS) program letter by Radke et al. (2021) that was published in response to the application of the IRIS risk of bias tool in our recent study “Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from different tools.” Their letter stated that we misrepresented the IRIS approach. Here, we respond to their three points raised and how we did not misrepresent their tool and also identified areas for improvement: (1) why it should be expected that different reviewers could reach different conclusions with the IRIS tool, as ratings are subject to reviewer judgment; (2) why our interpretation that “low” or “uninformative” studies could be excluded from a body of evidence was reasonable; and (3) why we believe the use of a rating system that generates an overall rating based on an individual domain or a combination of identified deficiencies essentially acts as a score and assumes that we know empirically how much each risk of bias domain should contribute to the overall rating for that study. We have elaborated on these points in our letter.