SUPPLEMENTARY INFORMATION

Transcription

1 VOLUME: 1 ARTICLE NUMBER: 0024 In the format provided by the authors and unedited. An artificial intelligence platform for the multihospital collaborative management of congenital cataracts Erping Long, Haotian Lin, Zhenzhen Liu, Xiaohang Wu, Liming Wang, Jiewei Jiang, Yingying An, Zhuoling Lin, Xiaoyan Li, Jingjing Chen, Jing Li, Qianzhong Cao, Dongni Wang, Xiyang Liu, Weirong Chen, Yizhi Liu Table of Contents 1. Supplementary Algorithm Information Supplementary Figures Supplementary Tables Supplementary Test Paper..10 NATURE BIOMEDICAL ENGINEERING DOI: /s

2 Supplementary Algorithm Information The overall framework of the deep convolutional neural network A deep convolutional neural network (CNN) in our work contained five convolutional or down-sample layers in addition to three fully connected layers. Adjacent layers were connected by edges with trainable parameters. The first seven layers were used to extract 4096 features from the input data, and the Softmax classifier was applied to the last layer. Convolution Each convolution kernel for local feature extracting could identify the relationships between pixels so that the high-quality features can be extracted. Furthermore, shared weights can greatly reduce the number of trainable parameters. The convolution output y L j of L layer can be re-written as equation below, where S j represents all the input maps involved in the convolution operation, L 1 x i stands for the input feature map in the L-1 layer and c L ij indicates the corresponding learnable kernel. y= x c L L 1 L j i ij i Sj Rectified linear unit (ReLU) and overlapping pooling Following the convolution operation, the pre-activation y L j accompanied with the additive bias are entered into the nonlinear activation function ReLU to produce the output feature. The overlapping max pooling layer is adopted instead of the non-overlapping pooling layer, aiming to achieve spatial invariance and enhance anti-noise capacity by reducing the data dimension of the feature maps. Data Augmentation To avoid overfitting, two data augmentation methods, transformed images and horizontal reflections, were adopted to enlarge the training dataset. By randomly extracting patches from the image and horizontally flipping them, the training dataset is NATURE BIOMEDICAL ENGINEERING DOI: /s

3 enlarged. Dropout To further address the overfitting, the dropout technology was integrated into the algorithm. Half of the neurons from the hidden layer are randomly involved in training, the output of all neurons are multiplied by 1/2. By dropout technique, co-adaptations of different neurons can be considerably reduced and thus enhance the performance of the whole network. Stochastic gradient descent (SGD) he SGD was employed to optimize the parameters of the CNN. We set the size of the stochastic batch to 1/8 of the entire training sample to accelerate parameter training convergence. The learning rate was initialized at 0.01 and successively decreased to 1/10 of the original per 1000 iterations, and the maximum number of iterations was NATURE BIOMEDICAL ENGINEERING DOI: /s

4 Supplementary Figures a b c Diagnosis In silico test multihospital clinical trial Website-based study AUC = AUC = AUC = AUC = Area AUC = AUC = Density AUC = AUC = Location AUC = AUC = AUC = AUC = AUC = AUC = AUC = Treatment AUC = AUC = AUC = AUC = Figure S1. ROC Curves for each validation test. a, ROC Curves for in silico test. b, ROC Curves for multihospital clinical trial. c, ROC Curves for website-based study. Footnote: AUC = Area under. NATURE BIOMEDICAL ENGINEERING DOI: /s

5 a b c In silico test Multihospital clinical trial Diagnosis Website-based study Cataract Normal Cataract Normal Cataract Normal Normal Cataract TP=81 FP=1 FN=1 TN=94 Normal Cataract TP=14 FP=1 FN=0 TN=42 Normal Cataract TP=39 FP=4 FN=0 TN=10 Area Extensive Limited Extensive Limited Extensive Limited Limited Extensive TP=46 FP=3 FN=2 TN=32 Limited Extensive TP=7 FP=0 FN=0 TN=7 Limited Extensive TP=19 FP=0 FN=2 TN=18 Density Dense Non-dense Dense Non-dense Dense Non-dense Non-dense Dense TP=44 FP=2 FN=2 TN=33 Non-dense Dense TP=6 FP=0 FN=1 TN=7 Non-dense Dense TP=18 FP=4 FN=2 TN=15 Location Central Peripheral Central Peripheral Central Peripheral Peripheral Central TP=48 FP=0 FN=4 TN=30 Peripheral Central TP=7 FP=0 FN=0 TN=7 Peripheral Central TP=15 FP=2 FN=0 TN=22 Treatment Surgery Follow-up Surgery Follow-up Surgery Follow-up Follow-up Surgery TP=46 FP=1 FN=1 TN=34 Follow-up Surgery TP=7 FP=0 FN=1 TN=6 Follow-up Surgery TP=11 FP=4 FN=0 TN=24 Figure S2. Confusion matrix for each validation test. a, confusion matrix for in silico test. b, confusion matrix for multihospital clinical trial. c, confusion matrix for website-based study. Footnote: TP = True positive; TN =True negative; FP = False positive; FN = False negative. NATURE BIOMEDICAL ENGINEERING DOI: /s

6 a b c d e Figure S3. User interface of the CC-Cruiser website-based platform. a, Prior to using the platform for the first time, a user must register on the system. Demographic information is needed. b, The user has the option to upload a new case to CC-Cruiser. c, After uploading, the output, including the results of the three networks, will be presented on the website. d, For users who wish to use the agent in trials, we have also provided 50 typical sample cases for download. e, and daytime telephone services are offered online for all of the registered patients. The cases classified as requiring surgery were sent to CCPMOH to fast-track their information. The administrator of CCPMOH has access to review all of the cases uploaded and contact the patients if needed. NATURE BIOMEDICAL ENGINEERING DOI: /s

7 Layer 3 Layer 4 Layer 5 Layer 6 Layer 7 Layer 8 Softmax classifier Layer 1 Layer 2 Fully connected layer Convolutions Convolutions Overlapping pooling Convolutions Convolutions Overlapping pooling Convolutions Overlapping pooling Input (RGB with 3 channels) Layers (Convolutions and overlapping involved) Layers (Convolutions involved) Fully connected layer Softmax classifier Figure S4. The architecture of deep convolutional neural network. The input is conducted through RGB (red, green and blue) with 3 channels (gray squares). The network consists of five convolutional and overlapping pooling layers (pooling involved in 1, 2, 5 layer), which are indicated by green squares and blue squares respectively, followed by three fully-connected layers (green rectangles). Softmax classifier (green spots) was applied to the last layer. NATURE BIOMEDICAL ENGINEERING DOI: /s

8 Supplementary Tables Table S1. Summary statistics for the performance of CC-Cruiser. Indices Cataract vs. Normal Extensive vs. Limited Dense vs. Non-dense Central vs. Peripheral Surgery vs. Follow-up ACC 98.87% 93.98% 95.06% 95.12% 97.56% SEN 98.78% 95.83% 95.65% 92.31% 97.87% In silico test Multihospital clinical trial Website-based study SPE 98.95% 91.43% 94.29% % 97.14% PPV 98.78% 93.88% 95.65% % 97.87% NPV 98.95% 94.12% 94.29% 88.24% 97.14% F-Measure 98.78% 94.85% 95.65% 96.00% 97.87% ACC 98.25% % 92.86% % 92.86% SEN % % 85.71% % 87.50% SPE 97.67% % % % % PPV 93.33% % % % % NPV % % 87.50% % 85.71% F-Measure 96.55% % 92.31% % 93.33% ACC 92.45% 94.87% 84.62% 94.87% 89.74% SEN % 90.48% 90.00% % % SPE 71.43% % 78.95% 91.67% 85.71% PPV 90.70% % 81.82% 88.24% 73.33% NPV % 90.00% 88.24% % % F-Measure 95.12% 95.00% 85.71% 93.75% 84.62% Summary statistics of the evaluation indices of CC-Cruiser in silico test, multihospital clinical trial and website-based study are presented. Footnote: Accuracy (ACC) = (TP+TN) / (TP+TN+FP+FN); (SEN) = TP / (TP+FN); Specificity (SPE) = TN / (TN+FP); Positive predictive value (PPV) = TP/ (TP+FP); Negative predictive value (NPV) = TN/ (TN+FN); F-Measure = 2TP/ (2TP+FP+FN); TP = True positive; TN =True negative; FP = False positive; FN = False negative. NATURE BIOMEDICAL ENGINEERING DOI: /s

9 Table S2. Summary for the detailed parameters of each layer. Name Filter Filter LRN Stride Padding Group (Layer) size dimension size Conv (1) / / ReLU (1) / / / / / / LRN (1) / / / / / 5 Pooling (1) 3 / 2 / / / Conv (2) / 2 2 / ReLU (2) / / / / / / LRN (2) / / / / / 5 Pooling (2) 3 / 2 / / / Conv (3) / / ReLU (3) / / / / / / Conv (4) / ReLU (4) / / / / / / Conv (5) / 1 2 / ReLU (5) / / / / / / Pooling (5) 3 / 2 / / / Fc (6) / / / ReLU (6) / / / / / / Dropout (6) / / / / / / Fc (7) / / / ReLU (7) / / / / / / Dropout (7) / / / / / / Fc (8) / / / The name, filter size, filter dimension, stride, padding, group and LRN size were illustrated in the table. Values in the size and dimension column are the size and number of output map in each layer. Footnote: Conv = convolution layer; Fc = fully connected layer; ReLU = Rectified linear unit; LRN = Local response normalization. NATURE BIOMEDICAL ENGINEERING DOI: /s

10 Supplementary Test Paper Congenital cataract comparative test (with expert panel reference) Notices: 1 Going back is not permitted after you have finished a case. 2 Go directly to the next case if you think the current case is normal. 3 Area is extensive when the opacity covers more than 50% of the pupil; otherwise, it is limited. 4 Density is defined as dense when the opacity fully disrupts vision; otherwise, it is non-dense. 5 Location is defined as central when the opacity fully covers the visual axis area; otherwise, it is peripheral. Case 1. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 2. Expert panel: Normal Case 3. Expert panel: Normal Case 4. Expert panel: Cataract, Extensive/ Dense/ Peripheral, Follow-up Case 5. Expert panel: Normal Case 6. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 7. Expert panel: Cataract, Extensive/ Non-dense/ Central, Follow-up Case 8. Expert panel: Normal NATURE BIOMEDICAL ENGINEERING DOI: /s

11 Case 9. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 10. Expert panel: Cataract, Extensive/ Non-dense/ Peripheral, Follow-up Case 11. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 12. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 13. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 14. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 15. Expert panel: Cataract, Extensive/ Dense/ Peripheral, Follow-up Case 16. Expert panel: Normal Case 17. Expert panel: Normal Case 18. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 19. Expert panel: Normal Case 20. Expert panel: Normal Case 21. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery NATURE BIOMEDICAL ENGINEERING DOI: /s

12 Case 22. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 23. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 24. Expert panel: Normal Case 25. Expert panel: Normal Case 26. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 27. Expert panel: Cataract, Limited/ Non-dense/ Peripheral, Follow-up Case 28. Expert panel: Normal Case 29. Expert panel: Normal Case 30. Expert panel: Cataract, Extensive/ Non-dense/ Central, Follow-up Case 31. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery Case 32. Expert panel: Cataract, Extensive/ Non-dense/ Peripheral, Follow-up Case 33. Expert panel: Cataract, Limited/ Non-dense/ Peripheral, Follow-up Case 34. Expert panel: Normal NATURE BIOMEDICAL ENGINEERING DOI: /s

13 Case 35. Expert panel: Cataract, Extensive/ Non-dense/ Peripheral, Follow-up Case 36. Expert panel: Cataract, Extensive/ Non-dense/ Central, Follow-up Case 37. Expert panel: Cataract, Limited/ Dense/ Peripheral, Follow-up Case 38. Expert panel: Cataract, Extensive/ Non-dense/ Central, Follow-up Case 39. Expert panel: Normal Case 40. Expert panel: Cataract, Limited/ Non-dense/ Peripheral, Follow-up Case 41. Expert panel: Cataract, Limited/ Dense/ Peripheral, Follow-up Case 42. Expert panel: Cataract, Limited/ Non-dense/ Peripheral, Follow-up Case 43. Expert panel: Cataract, Extensive/ Dense/ Central, Treatment Case 44. Expert panel: Cataract, Extensive/ Non-dense/ Central, Follow-up Case 45. Expert panel: Normal Case 46. Expert panel: Cataract, Extensive/ Non-dense/ Central, Follow-up Case 47. Expert panel: Cataract, Extensive/ Dense/ Central, Surgery NATURE BIOMEDICAL ENGINEERING DOI: /s

14 Case 48. Expert panel: Cataract, Extensive/ Non-dense/ Central, Follow-up Case 49. Expert panel: Cataract, Extensive/ Non-dense/ Central, Follow-up Case 50. Expert panel: Cataract, Limited/ Dense/ Peripheral, Follow-up NATURE BIOMEDICAL ENGINEERING DOI: /s