External validation of the RISC, RISC-Malawi, and PERCH clinical prediction rules to identify risk of death in children hospitalized with pneumonia

• Type of Material: Article
• Authors: Christopher Rees, Emory University; Shubhada Hooli, Texas Children’s Hospital; Carina King, University College London; Eric D McCollum, Johns Hopkins Bloomberg School of Public Health; Tim Colbourn, University College London; Norman Lufesi, Ministry of Health, Lilongwe, Malawi; Charles Mwansambo, Ministry of Health, Lilongwe, Malawi; Marzia Lazzerini, WHO Collaborating Centre for Maternal and Child Health, Institute for Maternal and Child Health IRCCS Burlo Garofolo, Trieste, Italy; Shabir Ahmed Madhi, University of Witwatersrand; Clare Cutland, University of Witwatersrand; Marta Nunes, University of Witwatersrand; Bradford D Gessner, Pfizer Vaccines; Sudha Basnet, University of Bergen; Cissy B Kartasasmita, Universitas Padjadjaran; Joseph L Mathew, Postgraduate Institute of Medical Education and Research; Syed Mohammad Akram Uz Zaman, Liverpool School of Tropical Medicine, Liverpool; Glaucia Paranhos-Baccala, Fondation Merieux, Lyon; Shinjini Bhatnagar, Translational Health Science and Technology Institute; Nitya Wadhwa, Translational Health Science and Technology Institute; Rakesh Lodha, All India Institute of Medical Sciences; Satinder Aneja, Sharda University; Mathuram Santosham, Johns Hopkins University; Valentina S Picot, Fondation Merieux, Lyon; Mariam Sylla, Gabriel Toure Hosp; Shally Awasthi, King George’s Medical University; Ashish Bavdekar, KEM Hospital Pune; Jean-William Pape, GHESKIO; Vanessa Rouzier, GHESKIO; Monidarin Chou, University of Health Sciences; Mala Rakoto-Andrianarivelo, Centre d'Infectiologie Charles Mérieux; Jianwei Wang, Chinese Academy of Medical Sciences & Peking Union; Pagbajabyn Nymadawa, Mongolian Academy of Sciences; Philippe Vanhems, Université Claude Bernard Lyon; Graciela Russomando, Universidad Nacional de Asuncion; Rai Asghar, Rawalpindi Medical College; Salem Banajeh, Sana’a University; Imran Iqbal, Nishtar Medical College; William MacLeod, Boston University; Irene Maulen-Radovan, Instituto Nactional de Pediatria Division de Investigacion Insurgentes; Greta Mino, Children's Hospital Dr Francisco de Ycaza Bustamante; Samir Saha, Dhaka Shishu Hospital; Sunit Singhi, Medanta, The Medicity, Gurgaon, India; Donald M Thea, Boston University; Alexey W Clara, US Centers for Disease Control, Central American Region; Harry Campbell, University of Edinburgh; Harish Nair, University of Edinburgh; Jennifer Falconer, Queen Margaret University; Linda J Williams, University of Edinburgh; Margaret Horne, University of Edinburgh; Tor Strand, Innlandet Hospital Trust; Shamim A Qazi, World Health Organization; Yasir B Nisar, World Health Organization; Mark I Neuman, Harvard Medical School
• Language: English
• Date: 2021-01-01
• Publisher: INT SOC GLOBAL HEALTH
• Publication Version: Author Accepted Manuscript (After Peer Review)
• Copyright Statement: © 2021 by the Journal of Global Health. All rights reserved.
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.7189/jogh.11.04062
• Title of Journal or Parent Work: JOURNAL OF GLOBAL HEALTH
• Volume: 11
• Start Page: 04062
• End Page: 04062
• Grant/Funding Information: The study was funded by the Bill & Melinda Gates Foundation (#INV-007927) through a grant to the World Health Organization. The funders had no role in the study design or in the collection, analysis, or interpretation of the data. The funders did not write the report and had no role in the decision to submit the paper for publication.
• Keywords: ZINC; EFFICACY; MENINGITIS; RESPIRATORY-INFECTIONS; Life Sciences & Biomedicine; Science & Technology; RANDOMIZED CONTROLLED-TRIAL; Public, Environmental & Occupational Health; BACTERIAL; PNEUMOCOCCAL CONJUGATE VACCINE; PULSE OXIMETRY; DOUBLE-BLIND; MORTALITY
• Research Categories: Health Sciences, Medicine and Surgery; Health Sciences, Public Health

Abstract

Background: Existing scores to identify children at risk of hospitalized pneumonia-related mortality lack broad external validation. Our objective was to externally validate three such risk scores. Methods: We applied the Respiratory Index of Severity in Children (RISC) for HIV-negative children, the RISC-Malawi, and the Pneumonia Etiology Research for Child Health (PERCH) scores to hospitalized children in the Pneumonia REsearch Partnerships to Assess WHO REcommendations (PREPARE) data set. The PREPARE data set includes pooled data from 41 studies on pediatric pneumonia from across the world. We calculated test characteristics and the area under the curve (AUC) for each of these clinical prediction rules. Results: The RISC score for HIV-negative children was applied to 3574 children 0-24 months and demonstrated poor discriminatory ability (AUC = 0.66, 95% confidence interval (CI) = 0.58-0.73) in the identification of children at risk of hospitalized pneumonia-related mortality. The RISC-Malawi score had fair discriminatory value (AUC = 0.75, 95% CI = 0.74-0.77) among 17 864 children 2-59 months. The PERCH score was applied to 732 children 1-59 months and also demonstrated poor discriminatory value (AUC = 0.55, 95% CI = 0.37-0.73). Conclusions: In a large external application of the RISC, RISC-Malawi, and PERCH scores, a substantial number of children were misclassified for their risk of hospitalized pneumonia-related mortality. Although pneumonia risk scores have performed well among the cohorts in which they were derived, their performance diminished when externally applied. A generalizable risk assessment tool with higher sensitivity and specificity to identify children at risk of hospitalized pneumonia-related mortality may be needed. Such a generalizable risk assessment tool would need context-specific validation prior to implementation in that setting.