LOGOLOGOLOGO
# MUSCLE - MICCAI 2022 This is a repository for paper "MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts" accepted by MICCAI 2022. ## Introduction The goal of MUSCLE (*MUlti-task Self-supervised Continual LEarning*) is to pre-train the deep neural network (DNN) models and deliver decent performance on medical image analysis tasks. All codes are implemented using PaddlePaddle. ## Framework ![image](https://user-images.githubusercontent.com/50255927/189317770-c8c9e866-beb2-4eb5-8116-21ab00850ef0.png) MUSCLE aggregated multiple X-ray image datasets collected from different human body parts, subject to various X-ray analytics tasks. We proposed Multi-Dataset Momentum Contrastive Representation Learning (MD-MoCo) and Multi-task Continual Learning to pre-train the backbone DNNs in a self-supervised continual learning manner. The pre-trained models could be fine-tuned to target tasks using task-specific heads and achieve superb performance. ## Datasets
Datasets Body Part Task Train Valid Test Total
Only Used for the first step (MD-MoCo) of MUSCLE
NIHCC Chest N/A 112,120 N/A N/A 112,120
China-Set-CXR Chest N/A 661 N/A N/A 661
Montgomery-Set-CXR Chest N/A 138 N/A N/A 138
Indiana-CXR Chest N/A 7,470 N/A N/A 7,470
RSNA Bone Age Hand N/A 10,811 N/A N/A 10,811
Used for all three steps of MUSCLE
Pneumonia Chest Classification 4,686 585 585 5,856
MURA Various Bone Classification 32,013 3,997 3,997 40,005
Chest Xray Masks and labels Chest Segmentation 718 89 89 896
TBX Chest Detection 640 80 80 800
Total N/A N/A 169,257 4,751 4,479 178,757
## Experiments ### Experiment setups - Backbone - ResNet-18 and ResNet-50 - Task - Pneumonia classification (Pneumonia), - Skeletal abnormality classification (MURA) - Lung segmentation (Lung) - Tuberculosis detection (TBX) - Head - Fully-Connected (FC) Layer for classification tasks - DeepLab-V3 for segmentation tasks - FasterRCNN for detection tasks - Baselines Pre-training Algorithms - **Scratch**: the models are all initialized using Kaiming’s random initialization and fine-tuned on the target datasets - **ImageNet**: the models are initialized using the officially-released weights pre-trained by the ImageNet dataset and fine-tuned on the target datasets - **MD-MoCo**: the models are pre-trained using multi-dataset MoCo and fine-tuned accordingly - **MUSCLE−−**: all models are pre-trained and fine-tuned with MUSCLE but with Cross-Task Memorization and Cyclic and Reshuffled Learning Schedule turned off ### Results for various body parts + Note: **chest** of Pneumonia and **bones** of MURA
Datasets Backbones Pre-train Acc. Sen. Spe. AUC(95%CI)
Pneumonia ResNet-18 Scratch 91.11 93.91 83.54 96.58(95.09-97.81)
ImageNet 90.09 93.68 80.38 96.05(94.24-97.33)
MD-MoCo 96.58 97.19 94.94 98.48(97.14-99.30)
MUSCLE-- 96.75 97.66 94.30 99.51(99.16-99.77)
MUSCLE 97.26 97.42 96.84 99.61(99.32-99.83)
ResNet-50 Scratch 91.45 92.51 88.61 96.55(95.08-97.82)
ImageNet 95.38 95.78 94.30 98.72(98.03-99.33)
MD-MoCo 97.09 98.83 92.41 99.53(99.23-99.75)
MUSCLE-- 96.75 98.36 92.41 99.58(99.30-99.84)
MUSCLE 98.12 98.36 97.47 99.72(99.46-99.92)
MURA ResNet-18 Scratch 81.00 68.17 89.91 86.62(85.73-87.55)
ImageNet 81.88 73.49 87.70 88.11(87.18-89.03)
MD-MoCo 82.48 72.27 89,57 88.28(87.28-89.26)
MUSCLE-- 82.45 74.16 88.21 88.41(87.54-89.26)
MUSCLE 82.62 74.28 88.42 88.5o(87.46-89.57)
RcsNet-50 Scratch 80.50 65.42 90.97 86.22(85.22-87.35)
ImngeNet 81.73 68.36 91.01 87.87(86.85-88.85)
MD-MoCo 82.35 73.12 88.76 87.89(87.06-88.88)
MUSCLE-- 81.10 69.03 89.48 87.14(86.10-88.22)
MUSCLE 82.60 74.53 88.21 88.37(87.38-89.32)
![image](https://user-images.githubusercontent.com/50255927/189317679-e3c22309-899b-4f8f-a689-d81e406376b5.png) ### Results for various tasks + Note: **segmentation task** for Lung and **detection task** for TBX
Backbones Pre-train Lung TBX
Dice mloU mAP AP-Active AP-Latent
ResNet-18 Scratch 95.24 94.00 30.71 56.71 4.72
ImageNet 95.26 94.10 29.46 56.27 2.66
MD-MoCo 95.31 94.14 36.00 67.17 4.84
MUSCLE-- 95.14 93.90 34.70 63.43 5.97
MUSCLE 95.37 94.22 36.71 64.84 8.59
　
ResNet-50		 Scratch 93.52 92.03 23.93 44.85 3.01
ImageNet 93.77 92.43 35.61 58.81 12.42
MD-MoCo 94.33 93.04 36.78 64.37 9.18
MUSCLE-- 95.04 93.82 35.14 57.32 12.97
MUSCLE 95.27 94.10 37.83 63.46 12.21
![image](https://user-images.githubusercontent.com/50255927/189317479-14ecb3de-da80-4df3-b9a0-f1fece7b953f.png) ## Citation If our work is helpful to you, please kindly cite our paper as: ``` @inproceedings{liao2022muscle, title={MUSCLE: Multi-task Self-supervised Continual Learning to Pre-train Deep Models for X-ray Images of Multiple Body Parts}, author={Weibin, Liao and Haoyi, Xiong and Qingzhong, Wang and Yan, Mo and Xuhong, Li and Yi, Liu and Zeyu, Chen and Siyu, Huang and Dejing, Dou}, booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention}, year={2022}, organization={Springer} } ```