Joint Region and Nucleus Segmentation for Characterization of Tumor Infiltrating Lymphocytes in Breast Cancer

• Type of Material: Article
• Authors: Mohamed Amgad, Emory University; Anindya Sarkar, Roche Tissue Diagnostics; Chukka Srinivas, Roche Tissue Diagnostics; Rachel Redman, Roche Diagnostics Information Solutions; Simrath Ratra, Roche Tissue Diagnostics; Charles J. Bechert, Roche Diagnostics Information Solutions; Benjamin C. Calhoun, Cleveland Clinic; Karen Mrazeck, Cleveland Clinic; Uday Kurkure, Roche Tissue Diagnostics; Lee Cooper, Emory University; Michael Barnes, Roche Diagnostics Information Solutions
• Language: English
• Date: 2019-02-01
• Publisher: Society of Photo-optical Instrumentation Engineers
• Publication Version: Author Accepted Manuscript (After Peer Review)
• Copyright Statement: © 2019 Society of Photo-Optical Instrumentation Engineers
• Final Published Version (URL): http://dx.doi.org/10.1117/12.2512892
• Title of Journal or Parent Work: Proceedings of SPIE - The International Society for Optical Engineering
• Volume: 10956
• Keywords: convolutional networks; computational pathology; deep learning; tumor infiltrating lymphocytes
• Research Categories: Psychology, Cognitive; Biology, Neuroscience; Health Sciences, Pathology; Health Sciences, Oncology

Abstract

Histologic assessment of stromal tumor infiltrating lymphocytes (sTIL) as a surrogate of the host immune response has been shown to be prognostic and potentially chemo-predictive in triple-negative and HER2-positive breast cancers. The current practice of manual assessment is prone to intra- and inter-observer variability. Furthermore, the inter-play of sTILs, tumor cells, other microenvironment mediators, their spatial relationships, quantity, and other image-based features have yet to be determined exhaustively and systemically. Towards analysis of these aspects, we developed a deep learning based method for joint region-level and nucleus-level segmentation and classification of breast cancer H&E tissue whole slide images. Our proposed method simultaneously identifies tumor, fibroblast, and lymphocyte nuclei, along with key histologic region compartments including tumor and stroma. We also show how the resultant segmentation masks can be combined with seeding approaches to yield accurate nucleus classifications. Furthermore, we outline a simple workflow for calibrating computational scores to human scores for consistency. The pipeline identifies key compartments with high accuracy (Dice= overall: 0.78, tumor: 0.83, and fibroblasts: 0.77). ROC AUC for nucleus classification is high at 0.89 (micro-average), 0.89 (lymphocytes), 0.90 (tumor), and 0.78 (fibroblasts). Spearman correlation between computational sTIL and pathologist consensus is high (R=0.73, p<0.001) and is higher than inter-pathologist correlation (R=0.66, p<0.001). Both manual and computational sTIL scores successfully stratify patients by clinical progression outcomes.