Improving Classification of Breast Cancer by Utilizing the Image Pyramids of Whole-Slide Imaging and Multi-Scale Convolutional Neural Networks

• Type of Material: Article
• Authors: Li Tong, Georgia Institute of Technology; Ying Sha, Georgia Institute of Technology; Dongmei Wang, Emory University
• Language: English
• Date: 2019-01-01
• Publisher: IEEE
• Publication Version: Author Accepted Manuscript (After Peer Review)
• Copyright Statement: © Copyright 2019 IEEE - All rights reserved.
• Final Published Version (URL): http://dx.doi.org/10.1109/COMPSAC.2019.00105
• Title of Journal or Parent Work: 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC)
• Volume: 1
• Start Page: 696
• End Page: 703
• Grant/Funding Information: The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.; This work was also supported in part by the scholarship from China Scholarship Council (CSC) under the Grant CSC NO. 201406010343.; The work was supported in part by grants from the National Center for Advancing Translational Sciences of the National Institute of Health (NIH) under Award UL1TR000454, the National Science Foundation EAGER Award NSF1651360, Children’s Healthcare of Atlanta and Georgia Tech Partnership Grant, Giglio Breast Cancer Research Fund, and Carol Ann and David D. Flanagan Faculty Fellow Research Fund.
• Keywords: Science & Technology; Whole-Slide Imaging; Pathology; Computer Science, Software Engineering; Image Pyramid; Technology; Computer Science; Multi-Scale Convolutional Neural Network; Computer Science, Interdisciplinary Applications; Breast Cancer
• Research Categories: Health Sciences, Oncology; Biology, Neuroscience; Health Sciences, Pathology

Abstract

Whole-slide imaging (WSI) is the digitization of conventional glass slides. Automatic computer-aided diagnosis (CAD) based on WSI enables digital pathology and the integration of pathology with other data like genomic biomarkers. Numerous computational algorithms have been developed for WSI, with most of them taking the image patches cropped from the highest resolution as the input. However, these models exploit only the local information within each patch and lost the connections between the neighboring patches, which may contain important context information. In this paper, we propose a novel multi-scale convolutional network (ConvNet) to utilize the built-in image pyramids of WSI. For the concentric image patches cropped at the same location of different resolution levels, we hypothesize the extra input images from lower magnifications will provide context information to enhance the prediction of patch images. We build corresponding ConvNets for feature representation and then combine the extracted features by 1) late fusion: concatenation or averaging the feature vectors before performing classification, 2) early fusion: merge the ConvNet feature maps. We have applied the multi-scale networks to a benchmark breast cancer WSI dataset. Extensive experiments have demonstrated that our multi-scale networks utilizing the WSI image pyramids can achieve higher accuracy for the classification of breast cancer. The late fusion method by taking the average of feature vectors reaches the highest accuracy (81.50%), which is promising for the application of multi-scale analysis of WSI.