Retrospective Review of the Drop in Observer Detection Performance Over Time in Lesion-enriched Experimental Studies

• Type of Material: Article
• Authors: Sian Taylor-Phillips, University of Warwick; Markus C. Elze, University of Warwick; Elizabeth Krupinski, Emory University; Kathryn Dennick, Kings College London; Alastair G. Gale, Loughborough University; Aileen Clarke, University of Warwick; Claudia Mello-Thoms, University of Sydney
• Language: English
• Date: 2015-02-01
• Publisher: Springer
• Publication Version: Final Published Version
• Copyright Statement: © 2014, The Author(s).
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.1007/s10278-014-9717-9
• Title of Journal or Parent Work: Journal of Digital Imaging
• Volume: 28
• Issue: 1
• Start Page: 32
• End Page: 40
• Grant/Funding Information: Markus C Elze is supported by the University of Warwick’s Chancellor’s Scholarship and a PhD Fellowship from the German National Academic Foundation.; This report is an independent research arising in part from a postdoctoral fellowship for Sian Taylor-Phillips supported by the National Institute for Health Research.; Aileen Clarke is supported by the National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care West Midlands at University Hospitals Birmingham NHS Foundation Trust.
• Keywords: Observer performance evaluation; Science & Technology; VIGILANCE; RADIOLOGY; Life Sciences & Biomedicine; PREVALENCE; Fatigue; Prevalence; Radiologist; Radiology, Nuclear Medicine & Medical Imaging; Vigilance; VISUAL-SEARCH
• Research Categories: Health Sciences, Radiology

Abstract

The vigilance decrement describes a decrease in sensitivity or increase in specificity with time on task. It has been observed in a variety of repetitive visual tasks, but little is known about these patterns in radiologists. We investigated whether there is systematic variation in performance over the course of a radiology reading session. We re-analyzed data from six previous lesion-enriched radiology studies. Studies featured 8–22 participants assessing 27–100 cases (including mammograms, chest CT, chest x-ray, and bone x-ray) in a reading session. Changes in performance and speed as the reading session progressed were analyzed using mixed effects models. Time taken per case decreased 9–23 % as the reading session progressed (p < 0.005 for every study). There was a sensitivity decrease or specificity increase over the course of reading 100 chest x-rays (p = 0.005), 60 bone fracture x-rays (p = 0.03), and 100 chest CT scans (p < 0.0001). This effect was not found in the shorter mammography sessions with 27 or 50 cases. We found evidence supporting the hypothesis that behavior and performance may change over the course of reading an enriched test set. Further research is required to ascertain whether this effect is present in radiological practice.