Towards Practical Problems in Deep Learning for Radiology Image Analysis

Transcription

1 Towards Practical Problems in Deep Learning for Radiology Image Analysis Quanzheng Li, Xiang Li, James H.Thrall Center for Clinical Data Science Department of Radiology Massachusetts General Hospital, Harvard Medical School

2 Purpose of AI in medical imaging: creating value in the delivery of medical care and delivery of radiology services: increasing diagnostic certainty decreasing time on task for radiologists faster availability of results reducing costs of care. Interrogating image data for extracting maximum value, with/without pre-defined model structure. Accuracy, Efficiency, Robustness.

3 Automatically detect (i.e. screening) the presence of free air lesion regions in the lung CT images. Manual inspection of the incoming medical images can be time-consuming and lack of the efficiency in handling life-threatening cases (such as pneumothorax). Certain image abnormities can be subtle human inspector, leads to potential mistakes in handling the patients. No systematic way of using learning-based methods for fully automatic screening.

4 Region with free air in the lung. Can be presented anywhere. Low HUT area in the image. Usually associated with other conditions.

5 Data Storage, Transfer, and Sharing Data Preprocessing and Quality Control Preparation for Analysis Analysis Postprocessing PACS Meta-information Patch extraction Model training and Heatmap HDFS processing Data augmentation validation generation Cloud storage Format conversion Network definition Model testing Diagnosis Image enhancement Model application Visualization /de-noising for practice? Positive

6 Data Storage, Transfer, and Sharing Data Preprocessing and Quality Control Preparation for Analysis Analysis Postprocessing PACS Meta-information Patch extraction Model training and Heatmap HDFS processing Data augmentation validation generation Cloud storage Fomat conversion Network definition Model testing Diagnosis Image enhancement Model application Visualization /de-noising for practice? Heterogeneity? Missing / erroneous data items? Online vs. Offline 1

7 High image/feature heterogeneity + lack of training samples: more likely to over-fitting. Data items can be missing or wrong (e.g. in DICOM headers during the scan). Most sophisticated preprocessing (e.g. image restoration, image segmentation) techniques have to be done off-line with group-wise information provided and/or ground-truth. 1

8 Data Storage, Transfer, and Sharing Data Preprocessing and Quality Control Preparation for Analysis Analysis Postprocessing PACS Meta-information Patch extraction Model training and Heatmap HDFS processing Data augmentation validation generation Cloud storage Fomat conversion Network definition Model testing Diagnosis Image enhancement Model application Visualization /de-noising for practice? Low speed and intensive I/O for patch extraction? Lack of training data samples? Arbitrary parameter / model structure 1

9 Deep learning models typically run on small image patches for increased sample size and better feature representation. Normal Control Patches Pneumothorax Patches 1

10 Data Storage, Transfer, and Sharing Data Preprocessing and Quality Control Preparation for Analysis Analysis Postprocessing PACS Meta-information Patch extraction Model training and Heatmap HDFS processing Data augmentation validation generation Cloud storage Fomat conversion Network definition Model testing Diagnosis Image enhancement Model application Visualization /de-noising for practice? Computational time: large data size + complex models? Needs for real-time results. 1

11 More complex models and deeper networks: Increased computational load for the system. Example: >1000 layered Deep Residual Learning network 1 has been evaluated on the ImageNet 2012 dataset consists of 1000 classes, trained on 1.28 million training images. 1

12 Large data size of most medical image types, high performance computing becomes a crucial component for a practical and running solution. Example: A typical CT image has more than 30 million voxels ( ). The pneumothorax project dataset constitutes imaging data from >600 subjects.

13 Dataset consists of 648 subjects with/without pneumothorax, 66 of them are annotated. Network trained on 31 subjects, totally 21, patches. Training a 16-layer, VGG-like 2D CNN for lesion detection on two classes: pneumothorax vs. normal. The whole pipeline takes DICOM images as input, generates a lesion heatmap, provides diagnosis score for the probability of pneumothorax.

14 Data Storage, Transfer, and Sharing Data Preprocessing and Quality Control Preparation for Analysis Analysis Postprocessing PACS Meta-information Patch extraction Model training and Heatmap HDFS processing Data augmentation validation generation Cloud storage Fomat conversion Network definition Model testing Diagnosis Image enhancement Model application Visualization /de-noising for practice? Security? Heterogeneity? Low speed and? Computational? Integration into the? Privacy? Missing / erroneous intensive I/O for time: large data workflow? Transfer speed data items patch extraction size + complex? Real-time feedback? Online vs. Offline? Lack of training data models samples? Arbitrary parameter / model structure Co-development with high radiologist involvement Self-paced learning Asynchronous I/O HPC platform supported by DGX1 CUDA implementation

15 Intensive involvement of radiologist During the training phase: addressing the data heterogeneity and under-coverage of training samples: Four types of mis-classification cases identified, 3 false-positive, 1 false-negative: Extra small pneumothorax lesions (mainly caused by the image view). Empyema. Imaging artifacts (e.g. dark strips). Irregular trachea/branches shapes.

16 Self-paced learning scheme to further increase the sample size. spcnn: Close Looped, Multiple Rounds of Training retraining Bootstrapping Module Original samples retraining retraining Classification Module Virtual samples at round i-1 Original samples retraining Bootstrapping CNN 1,i apply New, unlabeled data obtain Distribution of prediction probability Bootstrapping CNN k,i apply virtual sample selection New CNN, at round i apply Dataset for analyze obtain Classification results and diagnosis

17 Tested 35 subjects, patch-wise accuracy: 93.9%. Subject-wise accuracy is calculated by counting the number of detected patches followed by thresholding. Subject-wise ROC Curve True positive rate False positive rate

18 Although the detection is done on each slice (i.e. 2D network), the detected lesion boundary is stable across slices.

19 Detection (i.e. generating heatmap) of a single subject takes less than 3 minutes. Enabled by the computing power of DGX1: 50 times faster than single K40, 10 times faster than single P100. Most time consuming step is on the patch extraction, further I/O synchronous will help. The detection speed is on the same scale of a typical CT scan (minutes), thus enables real-time screening of the patients.

20 Self-paced learning method helps in increasing the training sample size by 23,100 patches (>100%) from 200 subjects (independent with the current annotation set). Performed in a single round. All the patches are consistently classified by the 10 bootstrapping networks, the accuracy is controlled by the Family-wise Error Rate (FWER) of p=0.05. Training of the 10 bootstrapping networks is done within one hour.

21 The latest NVIVDIA DGX-1 provides us an unprecedented computational power to support our approach of data augmentation and fast user interaction. Such massive data-initiated, computationalintensive solution will be the dominant trend for the medical researches and clinical practices. A new way for researchers and radiologists to apply, interpret and convert their domain knowledges ( fourth paradigm ).

22 Acknowledgement: Radiology at MGH: Subba Digumarthy, Mannudeep Kalra Gordon Center for Medical Imaging at MGH: Ning Guo, Kyungsang Kim, Dufan Wu, Aoxiao Zhong Beijing International Center for Mathematical Research at PKU: Prof. Bin Dong MGH Center for Clinical Data Science: