About this item:

32 Views | 15 Downloads

Author Notes:

Priyanshu Sinha, H-1931 Saya Zion Gaur City 1, Noida 201301, India. Email: Priyanshu_sinha@outlook.com

The authors would like to acknowledge Dr. Elizabeth Krupinski for her assistance with study design and analysis recommendation for the reader analysis.


Research Funding:

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.


  • Science & Technology
  • Life Sciences & Biomedicine
  • Radiology, Nuclear Medicine & Medical Imaging
  • COVID-19
  • Coronavirus
  • Chest X-ray
  • CXR

Multireader evaluation of radiologist performance for COVID-19 detection on emergency department chest radiographs

Show all authors Show less authors


Journal Title:



Volume 82


, Pages 77-82

Type of Work:

Article | Final Publisher PDF


Background: Chest radiographs (CXR) are frequently used as a screening tool for patients with suspected COVID-19 infection pending reverse transcriptase polymerase chain reaction (RT-PCR) results, despite recommendations against this. We evaluated radiologist performance for COVID-19 diagnosis on CXR at the time of patient presentation in the Emergency Department (ED). Materials and methods: We extracted RT-PCR results, clinical history, and CXRs of all patients from a single institution between March and June 2020. 984 RT-PCR positive and 1043 RT-PCR negative radiographs were reviewed by 10 emergency radiologists from 4 academic centers. 100 cases were read by all radiologists and 1927 cases by 2 radiologists. Each radiologist chose the single best label per case: Normal, COVID-19, Other – Infectious, Other – Noninfectious, Non-diagnostic, and Endotracheal Tube. Cases labeled with endotracheal tube (246) or non-diagnostic (54) were excluded. Remaining cases were analyzed for label distribution, clinical history, and inter-reader agreement. Results: 1727 radiographs (732 RT-PCR positive, 995 RT-PCR negative) were included from 1594 patients (51.2% male, 48.8% female, age 59 ± 19 years). For 89 cases read by all readers, there was poor agreement for RT-PCR positive (Fleiss Score 0.36) and negative (Fleiss Score 0.46) exams. Agreement between two readers on 1638 cases was 54.2% (373/688) for RT-PCR positive cases and 71.4% (679/950) for negative cases. Agreement was highest for RT-PCR negative cases labeled as Normal (50.4%, n = 479). Reader performance did not improve with clinical history or time between CXR and RT-PCR result. Conclusion: At the time of presentation to the emergency department, emergency radiologist performance is non-specific for diagnosing COVID-19.

Copyright information:

© 2021 Published by Elsevier Inc.

Export to EndNote