The Estimating effectiveness from efficacy taxonomy (EFFECT): A tool to estimate the real-world impact of health interventions

• Authors: Karla Galaviz Arredondo, Emory University; Mohammed Ali, Emory University; Jeehea Haw, Emory University; Matthew Magee, Emory University; Alysse Kowalski, Emory University; Jingkai Wei, University of North Carolina, Chapel Hill; Audrey Straus, CommUnityCare Health Centers; Mary Weber, Emory University; Theo Vos, University of Washington; Christopher Murray, University of Washington; Kabayam Venkat Narayan, Emory University
• Journal Title: DIABETES RESEARCH AND CLINICAL PRACTICE
• Volume: Volume 159
• Publisher: ELSEVIER IRELAND LTD | 2020-01-01, Pages 107751-107751
• Type of Work: Article | Post-print: After Peer Review
• Publisher DOI: http://doi.org/10.1016/j.diabres.2019.05.030
• Subjects: Health Sciences, Public Health
• Keywords: Science & Technology; Life Sciences & Biomedicine; Endocrinology & Metabolism; Effectiveness metrics; Implementation science; Global health; LIFE-STYLE INTERVENTIONS; DIABETES PREVENTION; PROMOTION PROGRAMS; TRANSLATION; METAANALYSIS; GUIDELINE

Abstract

Aims: To develop and pilot test a taxonomy that empirically estimates health intervention effectiveness from efficacy data. Methods: We developed a taxonomy to score health interventions across 11 items on a scale from 0–100. The taxonomy was pilot-tested in efficacy and effectiveness diabetes prevention studies identified in two separate systematic reviews; here, the face validity, inter-rater reliability and factor structure of the taxonomy were established. Random effects meta-analyses were used to obtain weight loss and diabetes incidence pooled effects across studies. These effects and taxonomy scores were used to down calibrate efficacy estimates to effectiveness estimates as follows: Efficacy effect*[Efficacy score/highest possible score]. Results: We scored 82 effectiveness lifestyle modification studies (mean score 49.2), 32 efficacy lifestyle modification studies (mean score 69.8) and 20 efficacy studies testing medications (mean score 77.4). The taxonomy had face validity and good inter-rater reliability (ICC = 0.9 [0.87, 0.93]). The between-groups down calibrated weight loss estimate was similar to that observed in the effectiveness meta-analysis (1.7 and 1.8 kg, respectively). The down calibrated diabetes relative risk reduction was also similar to that observed in the effectiveness meta-analysis (30.6% over 2.7 years and 29% over 2 years, respectively). Conclusions: The taxonomy is a promising tool to estimate the real-world impact of health interventions.