Could simplified stimuli change how the brain performs visual search tasks? A deep neural network study

• Type of Material: Article
• Authors: David A Nicholson, Emory University; Astrid Prinz, Emory University
• Language: English
• Date: 2022-06-08
• Publisher: Association for Research in Vision and Ophthalmology
• Publication Version: Final Published Version
• Copyright Statement: 2022 The Authors
• License: Creative Commons Attribution 4.0 International
• Final Published Version (URL): http://dx.doi.org/10.1167/jov.22.7.3
• Title of Journal or Parent Work: JOURNAL OF VISION
• Volume: 22
• Issue: 7
• Grant/Funding Information: Research funded by David Nicholson was partially supported by the 2017 William K. and Katherine W. Estes Fund to F. Pestilli, R. Goldstone and L. Smith, Indiana University Bloomington.the Lifelong Learning Machines program, DARPA/Microsystems Technology Office, DARPA cooperative agreement HR0011-18-2-0019.
• Keywords: object recognition; deep learning; selective visual attention; visual search; neural networks
• Research Categories: Biology, General

Abstract

Visual search is a complex behavior influenced by many factors. To control for these factors, many studies use highly simplified stimuli. However, the statistics of these stimuli are very different from the statistics of the natural images that the human visual system is optimized by evolution and experience to perceive. Could this difference change search behavior? If so, simplified stimuli may contribute to effects typically attributed to cognitive processes, such as selective attention. Here we use deep neural networks to test how optimizing models for the statistics of one distribution of images constrains performance on a task using images from a different distribution. We train four deep neural network architectures on one of three source datasets—natural images, faces, and x-ray images—and then adapt them to a visual search task using simplified stimuli. This adaptation produces models that exhibit performance limitations similar to humans, whereas models trained on the search task alone exhibit no such limitations. However, we also find that deep neural networks trained to classify natural images exhibit similar limitations when adapted to a search task that uses a different set of natural images. Therefore, the distribution of data alone cannot explain this effect. We discuss how future work might integrate an optimization-based approach into existing models of visual search behavior.