Advertisement

A comprehensive survey of deep learning research on medical image analysis with focus on transfer learning

Published:November 11, 2022DOI:https://doi.org/10.1016/j.clinimag.2022.11.003

      Highlights

      • We review current work in the field of medical image analysis based on medical organ-based.
      • Information on the most effective Transfer Learning techniques, methods and datasets for each anatomical region.
      • Difficulties, limitations and solutions to the implementation of Deep Learning and Transfer Learning.
      • Discussion on current research gaps and future research trends.

      Abstract

      This survey aims to identify commonly used methods, datasets, future trends, knowledge gaps, constraints, and limitations in the field to provide an overview of current solutions used in medical image analysis in parallel with the rapid developments in transfer learning (TL). Unlike previous studies, this survey grouped the last five years of current studies for the period between January 2017 and February 2021 according to different anatomical regions and detailed the modality, medical task, TL method, source data, target data, and public or private datasets used in medical imaging. Also, it provides readers with detailed information on technical challenges, opportunities, and future research trends. In this way, an overview of recent developments is provided to help researchers to select the most effective and efficient methods and access widely used and publicly available medical datasets, research gaps, and limitations of the available literature.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Clinical Imaging
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Jena O.P.
        • Bhushan B.
        • Kose U.
        Machine learning and deep learning in medical data analytics and healthcare applications.
        CRC Press, 2022
        • Litjens G.
        • et al.
        A survey on deep learning in medical image analysis.
        Med Image Anal. 2017; 42: 60-88
        • Pan S.J.
        • Yang Q.
        A survey on transfer learning.
        IEEE Trans Knowl Data Eng. 2009; 22: 1345-1359
        • Lu J.
        • et al.
        Transfer learning using computational intelligence: a survey.
        Knowl-Based Syst. 2015; 80: 14-23
        • Puttagunta M.
        • Ravi S.
        Medical image analysis based on deep learning approach.
        Multimed Tools Appl. 2021; 80: 24365-24398
        • Weiss K.
        • Khoshgoftaar T.M.
        • Wang D.
        A survey of transfer learning.
        J Big Data. 2016; 3: 1-40
        • Zhuang F.
        • et al.
        A comprehensive survey on transfer learning.
        Proc IEEE. 2020; 109: 43-76
        • Dodge J.
        Fine-tuning pretrained language models: weight initializations, data orders, and early stopping.
        arXiv Prepr. arXiv2002.06305, 2020
        • Zhang Y.
        • et al.
        Improving prognostic performance in resectable pancreatic ductal adenocarcinoma using radiomics and deep learning features fusion in CT images.
        Sci Rep. 2021; 11: 1-11
        • Afshar P.
        • et al.
        Covid-caps: a capsule network-based framework for identification of covid-19 cases from x-ray images.
        Pattern Recogn Lett. 2020; 138: 638-643
        • Xie Y.
        • Richmond D.
        Pre-training on grayscale imagenet improves medical image classification.
        in: Proceedings of the European conference on computer vision (ECCV) workshops. 0. 2018
        • Muramatsu C.
        • et al.
        Improving breast mass classification by shared data with domain transformation using a generative adversarial network.
        Comput Biol Med. 2020; 119103698
        • Choong R.Z.J.
        • Harding S.A.
        • Tang B.
        • Liao S.
        3-To-1 pipeline: restructuring transfer learning pipelines for medical imaging classification via optimized GAN synthetic images.
        in: 2020 42nd annual international conference of the IEEE engineering in medicine & biology society (EMBC). 2020: 1596-1599
        • Ayana G.
        • Dese K.
        • Choe S.
        Transfer learning in breast cancer diagnoses via ultrasound imaging.
        Cancers (Basel). 2021; 13: 738
        • Tariq M.
        • et al.
        Medical image based breast cancer diagnosis: state of the art and future directions.
        Expert Syst Appl. 2021; 167114095
        • Zou L.
        • et al.
        A technical review of convolutional neural network-based mammographic breast cancer diagnosis.
        Comput Math Methods Med. 2019; 2019
        • Whitney H.M.
        • Li H.
        • Ji Y.
        • Liu P.
        • Giger M.L.
        Comparison of breast MRI tumor classification using human-engineered radiomics, transfer learning from deep convolutional neural networks, and fusion methods.
        Proc IEEE. 2019; 108: 163-177
        • Valverde J.M.
        • et al.
        Transfer learning in magnetic resonance brain imaging: a systematic review.
        J Imaging. 2021; 7: 66
        • Suri J.S.
        • et al.
        A narrative review on characterization of acute respiratory distress syndrome in COVID-19-infected lungs using artificial intelligence.
        Comput Biol Med. 2021; 130104210
        • Islam M.M.
        • Karray F.
        • Alhajj R.
        • Zeng J.
        A review on deep learning techniques for the diagnosis of novel coronavirus (COVID-19).
        Ieee Access. 2021; 9: 30551-30572
        • Jairam M.P.
        • Ha R.
        A review of artificial intelligence in mammography.
        Clin Imaging. 2022; 88: 33-44https://doi.org/10.1016/j.clinimag.2022.05.005
        • Ke A.
        • Ellsworth W.
        • Banerjee O.
        • Ng A.Y.
        • Rajpurkar P.
        CheXtransfer: performance and parameter efficiency of ImageNet models for chest X-Ray interpretation.
        in: Proceedings of the conference on health, inference, and learning. 2021: 116-124
        • Romero M.
        • Interian Y.
        • Solberg T.
        • Valdes G.
        Targeted transfer learning to improve performance in small medical physics datasets.
        Med Phys. 2020; 47: 6246-6256
        • Krishnan R.
        • Rajpurkar P.
        • Topol E.J.
        Self-supervised learning in medicine and healthcare.
        Nat Biomed Eng. 2022; 1–7
        • Alzubaidi L.
        • et al.
        Novel transfer learning approach for medical imaging with limited labeled data.
        Cancers (Basel). 2021; 13: 1590
        • Azizi S.
        Robust and efficient medical imaging with self-supervision.
        arXiv Prepr. arXiv2205.09723, 2022
        • Bhattacharya S.
        • et al.
        Deep learning and medical image processing for coronavirus (COVID-19) pandemic: a survey.
        Sustain Cities Soc. 2021; 65102589
        • Alhudhaif A.
        • Polat K.
        • Karaman O.
        Determination of COVID-19 pneumonia based on generalized convolutional neural network model from chest X-ray images.
        Expert Syst Appl. 2021; 180115141
        • Moreira I.C.
        • et al.
        Inbreast: toward a full-field digital mammographic database.
        Acad Radiol. 2012; 19: 236-248
        • Huang G.
        • Liu Z.
        • Van Der Maaten L.
        • Weinberger K.Q.
        Densely connected convolutional networks.
        in: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 4700-4708
        • He K.
        • Zhang X.
        • Ren S.
        • Sun J.
        Deep residual learning for image recognition.
        in: Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778
        • Rajpurkar P.
        Chexnet: radiologist-level pneumonia detection on chest x-rays with deep learning.
        arXiv Prepr. arXiv1711.05225, 2017
        • Krizhevsky A.
        • Sutskever I.
        • Hinton G.E.
        Imagenet classification with deep convolutional neural networks.
        Adv Neural Inf Process Syst. 2012; 25: 1097-1105
        • Khan M.A.
        • et al.
        Classification of positive COVID-19 CT scans using deep learning.
        Comput Mater Contin. 2021; : 2923-2938
        • Yu Z.
        • et al.
        Rapid identification of COVID-19 severity in CT scans through classification of deep features.
        Biomed Eng Online. 2020; 19: 1-13
        • Liang G.
        • Zheng L.
        A transfer learning method with deep residual network for pediatric pneumonia diagnosis.
        Comput Methods Programs Biomed. 2020; 187104964
        • Wang X.
        Chestx-ray8: hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases.
        in: Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 2097-2106
        • Simonyan K.
        • Zisserman A.
        Very deep convolutional networks for large-scale image recognition.
        arXiv Prepr. arXiv1409.1556, 2014
        • Devnath L.
        • Luo S.
        • Summons P.
        • Wang D.
        Automated detection of pneumoconiosis with multilevel deep features learned from chest X-ray radiographs.
        Comput Biol Med. 2021; 129104125
        • Trivizakis E.
        • et al.
        Advancing COVID-19 differentiation with a robust preprocessing and integration of multi-institutional open-repository computer tomography datasets for deep learning analysis.
        Exp Ther Med. 2020; 20: 1
        • Panwar H.
        • et al.
        A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-scan images.
        Chaos Solitons Fractals. 2020; 140110190
        • Horry M.J.
        • et al.
        COVID-19 detection through transfer learning using multimodal imaging data.
        IEEE Access. 2020; 8: 149808-149824
        • Niu S.
        • Liu M.
        • Liu Y.
        • Wang J.
        • Song H.
        Distant domain transfer learning for medical imaging.
        IEEE J Biomed Health Inform. 2021; 25: 3784-3793
        • Polat Ç.
        • et al.
        COVID-19 diagnosis from chest X-ray images using transfer learning: enhanced performance by debiasing dataloader.
        J Xray Sci Technol. 2021; 29: 19-36
        • Fuhrman J.D.
        • et al.
        Cascaded deep transfer learning on thoracic CT in COVID-19 patients treated with steroids.
        J Med Imaging. 2020; 8: 14501
        • Xie Q.
        • et al.
        The usage of deep neural network improves distinguishing COVID-19 from other suspected viral pneumonia by clinicians on chest CT: a real-world study.
        Eur Radiol. 2021; 31: 3864-3873
        • Gozes O.
        • Greenspan H.
        Deep feature learning from a hospital-scale chest x-ray dataset with application to TB detection on a small-scale dataset.
        in: 2019 41st annual international conference of the ieee engineering in medicine and biology society (embc). 2019: 4076-4079
        • Behzadi-Khormouji H.
        • et al.
        Deep learning, reusable and problem-based architectures for detection of consolidation on chest X-ray images.
        Comput Methods Programs Biomed. 2020; 185105162
        • Huang X.
        • Yan F.
        • Xu W.
        • Li M.
        Multi-attention and incorporating background information model for chest x-ray image report generation.
        IEEE Access. 2019; 7: 154808-154817
        • Allaouzi I.
        • Ahmed M.Ben
        A novel approach for multi-label chest X-ray classification of common thorax diseases.
        IEEE Access. 2019; 7: 64279-64288
        • Lopes U.K.
        • Valiati J.F.
        Pre-trained convolutional neural networks as feature extractors for tuberculosis detection.
        Comput Biol Med. 2017; 89: 135-143
        • Wang H.
        • Gu H.
        • Qin P.
        • Wang J.
        CheXLocNet: automatic localization of pneumothorax in chest radiographs using deep convolutional neural networks.
        PLoS One. 2020; 15e0242013
        • Joyseeree R.
        • Otálora S.
        • Müller H.
        • Depeursinge A.
        Fusing learned representations from riesz filters and deep CNN for lung tissue classification.
        Med Image Anal. 2019; 56: 172-183
        • Wang C.
        • Elazab A.
        • Wu J.
        • Hu Q.
        Lung nodule classification using deep feature fusion in chest radiography.
        Comput Med Imaging Graph. 2017; 57: 10-18
        • Xu Y.
        • et al.
        Deep learning predicts lung cancer treatment response from serial medical imaging.
        Clin Cancer Res. 2019; 25: 3266-3275
        • Hosny A.
        • et al.
        Deep learning for lung cancer prognostication: a retrospective multi-cohort radiomics study.
        PLoS Med. 2018; 15e1002711
        • Weng S.
        • Xu X.
        • Li J.
        • Wong S.T.C.
        Combining deep learning and coherent anti-stokes Raman scattering imaging for automated differential diagnosis of lung cancer.
        J Biomed Opt. 2017; 22106017
        • Zhao X.
        • et al.
        Deep CNN models for pulmonary nodule classification: model modification, model integration, and transfer learning.
        J Xray Sci Technol. 2019; 27: 615-629
        • Huang C.
        • et al.
        Discrimination between transient and persistent subsolid pulmonary nodules on baseline CT using deep transfer learning.
        Eur Radiol. 2020; 30: 6913-6923
        • Hu X.
        • et al.
        Computer-aided diagnosis of ground glass pulmonary nodule by fusing deep learning and radiomics features.
        Phys Med Biol. 2021; 66: 65015
        • Sajjad M.
        • et al.
        Multi-grade brain tumor classification using deep CNN with extensive data augmentation.
        J Comput Sci. 2019; 30: 174-182
        • Deepak S.
        • Ameer P.M.
        Retrieval of brain MRI with tumor using contrastive loss based similarity on GoogLeNet encodings.
        Comput Biol Med. 2020; 125103993
        • Zhu J.
        • et al.
        Rubik’s cube+: a self-supervised feature learning framework for 3d medical image analysis.
        Med Image Anal. 2020; 64101746
        • Mehrotra R.
        • Ansari M.A.
        • Agrawal R.
        • Anand R.S.
        A transfer learning approach for AI-based classification of brain tumors.
        Mach Learn Appl. 2020; 2100003
        • Swati Z.N.K.
        • et al.
        Brain tumor classification for MR images using transfer learning and fine-tuning.
        Comput Med Imaging Graph. 2019; 75: 34-46
        • Saba T.
        • Mohamed A.S.
        • El-Affendi M.
        • Amin J.
        • Sharif M.
        Brain tumor detection using fusion of hand crafted and deep learning features.
        Cogn Syst Res. 2020; 59: 221-230
        • Wong K.C.L.
        • Syeda-Mahmood T.
        • Moradi M.
        Building medical image classifiers with very limited data using segmentation networks.
        Med Image Anal. 2018; 49: 105-116
        • Khan H.A.
        • Jue W.
        • Mushtaq M.
        • Mushtaq M.U.
        Brain tumor classification in MRI image using convolutional neural network.
        Math Biosci Eng. 2020; 17: 6203-6216
        • Di Ieva A.
        • et al.
        Application of deep learning for automatic segmentation of brain tumors on magnetic resonance imaging: a heuristic approach in the clinical scenario.
        Neuroradiology. 2021; 63: 1253-1262
        • Smedley N.F.
        • Hsu W.
        Using deep neural networks for radiogenomic analysis.
        in: 2018 IEEE 15th international symposium on biomedical imaging (ISBI 2018). 2018: 1529-1533
        • Tufail A.Bin
        • Ma Y.-K.
        • Zhang Q.-N.
        Binary classification of Alzheimer’s disease using sMRI imaging modality and deep learning.
        J Digit Imaging. 2020; 33: 1073-1090
        • Gao F.
        • et al.
        AD-NET: age-adjust neural network for improved MCI to AD conversion prediction.
        NeuroImage Clin. 2020; 27102290
        • Plata D.R.
        • Ramos-Pollan R.
        • Gonzalez F.A.
        Effective training of convolutional neural networks with small, specialized datasets.
        J Intell Fuzzy Syst. 2017; 32: 1333-1342
        • Wingate J.
        • Kollia I.
        • Bidaut L.
        • Kollias S.
        Unified deep learning approach for prediction of Parkinson’s disease.
        IET Image Process. 2020; 14: 1980-1989
        • Wang Z.
        • et al.
        3D compressed convolutional neural network differentiates neuromyelitis optical Spectrum disorders from multiple sclerosis using automated white matter hyperintensities segmentations.
        Front Physiol. 2020; 1736
        • Aghdam M.A.
        • Sharifi A.
        • Pedram M.M.
        Diagnosis of autism spectrum disorders in young children based on resting-state functional magnetic resonance imaging data using convolutional neural networks.
        J Digit Imaging. 2019; 32: 899-918
        • Roy A.G.
        • et al.
        QuickNAT: a fully convolutional network for quick and accurate segmentation of neuroanatomy.
        Neuroimage. 2019; 186: 713-727
        • Bermúdez-Chacón R.
        • Altingövde O.
        • Becker C.
        • Salzmann M.
        • Fua P.
        Visual correspondences for unsupervised domain adaptation on electron microscopy images.
        IEEE Trans Med Imaging. 2019; 39: 1256-1267
        • Nayak D.R.
        • Dash R.
        • Majhi B.
        Automated diagnosis of multi-class brain abnormalities using MRI images: a deep convolutional neural network based method.
        Pattern Recogn Lett. 2020; 138: 385-391
        • Talo M.
        • Yildirim O.
        • Baloglu U.B.
        • Aydin G.
        • Acharya U.R.
        Convolutional neural networks for multi-class brain disease detection using MRI images.
        Comput Med Imaging Graph. 2019; 78101673
        • Grimm F.
        • et al.
        Semantic segmentation of cerebrospinal fluid and brain volume with a convolutional neural network in pediatric hydrocephalus—transfer learning from existing algorithms.
        Acta Neurochir. 2020; 162: 2463-2474
        • Pardamean B.
        • Cenggoro T.W.
        • Rahutomo R.
        • Budiarto A.
        • Karuppiah E.K.
        Transfer learning from chest X-ray pre-trained convolutional neural network for learning mammogram data.
        Procedia Comput Sci. 2018; 135: 400-407
        • Chougrad H.
        • Zouaki H.
        • Alheyane O.
        Deep convolutional neural networks for breast cancer screening.
        Comput Methods Prog Biomed. 2018; 157: 19-30
        • Shen L.
        • et al.
        Deep learning to improve breast cancer detection on screening mammography.
        Sci Rep. 2019; 9: 1-12
        • Agarwal R.
        • Diaz O.
        • Lladó X.
        • Yap M.H.
        • Mart\’\i R.
        Automatic mass detection in mammograms using deep convolutional neural networks.
        J Med Imaging. 2019; 6: 31409
        • Zhang Y.
        • et al.
        Prediction of breast cancer molecular subtypes on DCE-MRI using convolutional neural network with transfer learning between two centers.
        Eur Radiol. 2021; 31: 2559-2567
        • Samala R.K.
        • et al.
        Breast cancer diagnosis in digital breast tomosynthesis: effects of training sample size on multi-stage transfer learning using deep neural nets.
        IEEE Trans Med Imaging. 2018; 38: 686-696
        • Samala R.K.
        • Chan H.-P.
        • Hadjiiski L.M.
        • Helvie M.A.
        • Richter C.D.
        Generalization error analysis for deep convolutional neural network with transfer learning in breast cancer diagnosis.
        Phys Med Biol. 2020; 65105002
        • Khan H.N.
        • Shahid A.R.
        • Raza B.
        • Dar A.H.
        • Alquhayz H.
        Multi-view feature fusion based four views model for mammogram classification using convolutional neural network.
        IEEE Access. 2019; 7: 165724-165733
        • Carneiro G.
        • Nascimento J.
        • Bradley A.P.
        Automated analysis of unregistered multi-view mammograms with deep learning.
        IEEE Trans Med Imaging. 2017; 36: 2355-2365
        • Gao F.
        • Yoon H.
        • Wu T.
        • Chu X.
        A feature transfer enabled multi-task deep learning model on medical imaging.
        Expert Syst Appl. 2020; 143112957
        • Shen R.
        • et al.
        Unsupervised domain adaptation with adversarial learning for mass detection in mammogram.
        Neurocomputing. 2020; 393: 27-37
        • Samala R.K.
        • Chan H.-P.
        • Hadjiiski L.
        • Helvie M.A.
        Risks of feature leakage and sample size dependencies in deep feature extraction for breast mass classification.
        Med Phys. 2021; 48: 2827-2837
        • Yap M.H.
        • et al.
        Automated breast ultrasound lesions detection using convolutional neural networks.
        IEEE J Biomed Health Inform. 2017; 22: 1218-1226
        • Agarwal R.
        • Diaz O.
        • Yap M.H.
        • Llado X.
        • Marti R.
        Deep learning for mass detection in full field digital mammograms.
        Comput Biol Med. 2020; 121103774
        • Arora R.
        • Rai P.K.
        • Raman B.
        Deep feature–based automatic classification of mammograms.
        Med Biol Eng Comput. 2020; 58: 1199-1211
        • Dong N.
        • Zhao L.
        • Wu C.-H.
        • Chang J.-F.
        Inception v3 based cervical cell classification combined with artificially extracted features.
        Appl Soft Comput. 2020; 93106311
        • Shin S.J.
        • et al.
        Style transfer strategy for developing a generalizable deep learning application in digital pathology.
        Comput Methods Programs Biomed. 2021; 198105815
        • Zhao X.
        • et al.
        Deep learning–based fully automated detection and segmentation of lymph nodes on multiparametric-mri for rectal cancer: a multicentre study.
        EBioMedicine. 2020; 56102780
        • Zhang R.
        • Zheng Y.
        • Poon C.C.Y.
        • Shen D.
        • Lau J.Y.W.
        Polyp detection during colonoscopy using a regression-based convolutional neural network with a tracker.
        Pattern Recogn. 2018; 83: 209-219
        • Nadimi E.S.
        • et al.
        Application of deep learning for autonomous detection and localization of colorectal polyps in wireless colon capsule endoscopy.
        Comput Electr Eng. 2020; 81106531
        • Khan M.A.
        • et al.
        Computer-aided gastrointestinal diseases analysis from wireless capsule endoscopy: a framework of best features selection.
        IEEE Access. 2020; 8: 132850-132859
        • Rajpurkar P.
        • et al.
        AppendiXNet: deep learning for diagnosis of appendicitis from a small dataset of CT exams using video pretraining.
        Sci Rep. 2020; 10: 1-7
        • Meng Q.
        • et al.
        Mutual information-based disentangled neural networks for classifying unseen categories in different domains: application to fetal ultrasound imaging.
        IEEE Trans Med Imaging. 2020; 40: 722-734
        • Zhang T.
        • et al.
        Cervical precancerous lesions classification using pre-trained densely connected convolutional networks with colposcopy images.
        Biomed Signal Process Control. 2020; 55101566
        • Nash W.
        • Drummond T.
        • Birbilis N.
        A review of deep learning in the study of materials degradation.
        npj Mater. Degrad. 2018; 2: 1-12
        • Zhu Q.
        • Du B.
        • Yan P.
        Boundary-weighted domain adaptive neural network for prostate MR image segmentation.
        IEEE Trans Med Imaging. 2019; 39: 753-763
        • Jia X.
        • Xiao Y.
        • Yang D.
        • Yang Z.
        • Lu C.
        Multi-parametric MRIs based assessment of hepatocellular carcinoma differentiation with multi-scale ResNet.
        KSII Trans Internet Inf Syst. 2019; 13: 5179-5196
        • Xia T.
        • Kumar A.
        • Feng D.
        • Kim J.
        Patch-level tumor classification in digital histopathology images with domain adapted deep learning.
        in: 2018 40th annual international conference of the IEEE engineering in medicine and biology society (EMBC). 2018: 644-647
        • Mahmood F.
        • Chen R.
        • Sudarsky S.
        • Yu D.
        • Durr N.J.
        Deep learning with cinematic rendering: fine-tuning deep neural networks using photorealistic medical images.
        Phys Med Biol. 2018; 63185012
        • Mahmood F.
        • Chen R.
        • Durr N.J.
        Unsupervised reverse domain adaptation for synthetic medical images via adversarial training.
        IEEE Trans Med Imaging. 2018; 37: 2572-2581
        • Hectors S.J.
        • et al.
        Fully automated prediction of liver fibrosis using deep learning analysis of gadoxetic acid–enhanced MRI.
        Eur Radiol. 2021; 31: 3805-3814
        • Hu Y.
        • et al.
        Computed tomography-based deep-learning prediction of neoadjuvant chemoradiotherapy treatment response in esophageal squamous cell carcinoma.
        Radiother Oncol. 2021; 154: 6-13
        • Peng J.
        • et al.
        Residual convolutional neural network for predicting response of transarterial chemoembolization in hepatocellular carcinoma from CT imaging.
        Eur Radiol. 2020; 30: 413-424
        • Khan M.A.
        • et al.
        Gastrointestinal diseases segmentation and classification based on duo-deep architectures.
        Pattern Recogn Lett. 2020; 131: 193-204
        • Chen J.-R.
        • et al.
        Clinical value of information entropy compared with deep learning for ultrasound grading of hepatic steatosis.
        Entropy. 2020; 22: 1006
        • Xue L.-Y.
        • et al.
        Transfer learning radiomics based on multimodal ultrasound imaging for staging liver fibrosis.
        Eur Radiol. 2020; 30: 2973-2983
        • Jodeiri A.
        • et al.
        Fully automatic estimation of pelvic sagittal inclination from anterior-posterior radiography image using deep learning framework.
        Comput Methods Prog Biomed. 2020; 184105282
        • Zhang L.
        • et al.
        DeepPap: deep convolutional networks for cervical cell classification.
        IEEE J Biomed Health Inform. 2017; 21: 1633-1643
        • Marsh J.N.
        • et al.
        Deep learning global glomerulosclerosis in transplant kidney frozen sections.
        IEEE Trans Med Imaging. 2018; 37: 2718-2728
        • Cheng C.-T.
        • et al.
        Application of a deep learning algorithm for detection and visualization of hip fractures on plain pelvic radiographs.
        Eur Radiol. 2019; 29: 5469-5477
        • Bengani S.
        • et al.
        Automatic segmentation of optic disc in retinal fundus images using semi-supervised deep learning.
        Multimed Tools Appl. 2021; 80: 3443-3468
        • Hervella Á.S.
        • Rouco J.
        • Novo J.
        • Ortega M.
        Learning the retinal anatomy from scarce annotated data using self-supervised multimodal reconstruction.
        Appl Soft Comput. 2020; 91106210
        • Jiang Z.
        • Zhang H.
        • Wang Y.
        • Ko S.-B.
        Retinal blood vessel segmentation using fully convolutional network with transfer learning.
        Comput Med Imaging Graph. 2018; 68: 1-15
        • Zago G.T.
        • Andreão R.V.
        • Dorizzi B.
        • Salles E.O.T.
        Retinal image quality assessment using deep learning.
        Comput Biol Med. 2018; 103: 64-70
        • Zhang W.
        • et al.
        Automated identification and grading system of diabetic retinopathy using deep neural networks.
        Knowl-Based Syst. 2019; 175: 12-25
        • Kermany D.S.
        • et al.
        Identifying medical diagnoses and treatable diseases by image-based deep learning.
        Cell. 2018; 172: 1122-1131
        • Zhou Y.
        • Wang B.
        • Huang L.
        • Cui S.
        • Shao L.
        A benchmark for studying diabetic retinopathy: segmentation, grading, and transferability.
        IEEE Trans Med Imaging. 2020; 40: 818-828
        • Ghazal M.
        • Ali S.S.
        • Mahmoud A.H.
        • Shalaby A.M.
        • El-Baz A.
        Accurate detection of non-proliferative diabetic retinopathy in optical coherence tomography images using convolutional neural networks.
        IEEE Access. 2020; 8: 34387-34397
        • Civit-Masot J.
        • Dominguez-Morales M.J.
        • Vicente-Diaz S.
        • Civit A.
        Dual machine-learning system to aid glaucoma diagnosis using disc and cup feature extraction.
        IEEE Access. 2020; 8: 127519-127529
        • Wang S.
        • Yu L.
        • Yang X.
        • Fu C.-W.
        • Heng P.-A.
        Patch-based output space adversarial learning for joint optic disc and cup segmentation.
        IEEE Trans Med Imaging. 2019; 38: 2485-2495
        • Wang P.
        • Yuan M.
        • He Y.
        • Sun J.
        3D augmented fundus images for identifying glaucoma via transferred convolutional neural networks.
        Int Ophthalmol. 2021; 41: 2065-2072
        • Ju L.
        • et al.
        Leveraging regular fundus images for training UWF fundus diagnosis models via adversarial learning and pseudo-labeling.
        IEEE Trans Med Imaging. 2021; 40: 2911-2925
        • Dallora A.L.
        • et al.
        Age assessment of youth and young adults using magnetic resonance imaging of the knee: a deep learning approach.
        JMIR Med Inform. 2019; 7e16291
        • Tiulpin A.
        • Saarakkala S.
        Automatic grading of individual knee osteoarthritis features in plain radiographs using deep convolutional neural networks.
        Diagnostics. 2020; 10: 932
        • Le Van C.
        • Bao L.N.
        • Puri V.
        • Thao N.T.
        • Le D.-N.
        Detecting lumbar implant and diagnosing scoliosis from Vietnamese X-ray imaging using the pre-trained api models and transfer learning.
        C Comput Mater Contin. 2021; 66: 17-33
        • Kompella G.
        Segmentation of femoral cartilage from knee ultrasound images using mask R-CNN.
        in: 2019 41st annual international conference of the IEEE engineering in medicine and biology society (EMBC). 2019: 966-969
        • Iqbal I.
        • Shahzad G.
        • Rafiq N.
        • Mustafa G.
        • Ma J.
        Deep learning-based automated detection of human knee joint’s synovial fluid from magnetic resonance images with transfer learning.
        IET Image Process. 2020; 14: 1990-1998
        • Papandrianos N.
        • Papageorgiou E.
        • Anagnostis A.
        • Papageorgiou K.
        Bone metastasis classification using whole body images from prostate cancer patients based on convolutional neural networks application.
        PLoS One. 2020; 15e0237213
        • Vrbančič G.
        • Podgorelec V.
        Transfer learning with adaptive fine-tuning.
        IEEE Access. 2020; 8: 196197-196211
        • Yi P.H.
        • et al.
        Automated semantic labeling of pediatric musculoskeletal radiographs using deep learning.
        Pediatr Radiol. 2019; 49: 1066-1070
        • Khatami A.
        • et al.
        A sequential search-space shrinking using CNN transfer learning and a radon projection pool for medical image retrieval.
        Expert Syst Appl. 2018; 100: 224-233
        • Kim D.H.
        • MacKinnon T.
        Artificial intelligence in fracture detection: transfer learning from deep convolutional neural networks.
        Clin Radiol. 2018; 73: 439-445
        • Liao X.
        • et al.
        MMTLNet: multi-modality transfer learning network with adversarial training for 3D whole heart segmentation.
        Comput Med Imaging Graph. 2020; 85101785
        • Gong P.
        • Yu W.
        • Sun Q.
        • Zhao R.
        • Hu J.
        Unsupervised domain adaptation network with category-centric prototype aligner for biomedical image segmentation.
        IEEE Access. 2021; 9: 36500-36511
        • Chen H.-H.
        • et al.
        Automated extraction of left atrial volumes from two-dimensional computer tomography images using a deep learning technique.
        Int J Cardiol. 2020; 316: 272-278
        • Koehler S.
        • et al.
        Unsupervised domain adaptation from axial to short-Axis multi-slice cardiac MR images by incorporating pretrained task networks.
        IEEE Trans Med Imaging. 2021; 40: 2939-2953
        • Papathanasiou N.D.
        • Spyridonidis T.
        • Apostolopoulos D.J.
        Automatic characterization of myocardial perfusion imaging polar maps employing deep learning and data augmentation.
        Hell J Nucl Med. 2020; 23: 125-132
        • Gessert N.
        • et al.
        Automatic plaque detection in IVOCT pullbacks using convolutional neural networks.
        IEEE Trans Med Imaging. 2018; 38: 426-434
        • Gong Y.
        • et al.
        Fetal congenital heart disease echocardiogram screening based on DGACNN: adversarial one-class classification combined with video transfer learning.
        IEEE Trans Med Imaging. 2019; 39: 1206-1222
        • MM-WHS
        Multi-modality whole heart segmentation.
        • Mahbod A.
        • et al.
        Transfer learning using a multi-scale and multi-network ensemble for skin lesion classification.
        Comput Methods Programs Biomed. 2020; 193105475
        • Qin Z.
        • Liu Z.
        • Zhu P.
        • Xue Y.
        A GAN-based image synthesis method for skin lesion classification.
        Comput Methods Programs Biomed. 2020; 195105568
        • Gu Y.
        • Ge Z.
        • Bonnington C.P.
        • Zhou J.
        Progressive transfer learning and adversarial domain adaptation for cross-domain skin disease classification.
        IEEE J Biomed Health Inform. 2019; 24: 1379-1393
        • Liu L.
        • Mou L.
        • Zhu X.X.
        • Mandal M.
        Automatic skin lesion classification based on mid-level feature learning.
        Comput Med Imaging Graph. 2020; 84101765
        • Mahbod A.
        • et al.
        Fusing fine-tuned deep features for skin lesion classification.
        Comput Med Imaging Graph. 2019; 71: 19-29
        • Alzubaidi L.
        • et al.
        Towards a better understanding of transfer learning for medical imaging: a case study.
        Appl Sci. 2020; 10: 4523
        • Armato III, S.G.
        • et al.
        The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans.
        Med Phys. 2011; 38: 915-931
        • Setio A.A.A.
        • et al.
        Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge.
        Med Image Anal. 2017; 42: 1-13
        • Zhao J.
        • Zhang Y.
        • He X.
        • Xie P.
        COVID-CT-dataset: a CT scan dataset about COVID-19.
        arXiv Prepr. arXiv2003.13865, 2020
        • Soares E.
        • Angelov P.
        • Biaso S.
        • Froes M.H.
        • Abe D.K.
        SARS-CoV-2 CT-scan dataset: a large dataset of real patients CT scans for SARS-CoV-2 identification.
        MedRxiv, 2020
        • Irvin J.
        Chexpert: a large chest radiograph dataset with uncertainty labels and expert comparison.
        in: Proceedings of the AAAI conference on artificial intelligence. vol. 33. 2019: 590-597
        • Jaeger S.
        • et al.
        Two public chest X-ray datasets for computer-aided screening of pulmonary diseases.
        Quant Imaging Med Surg. 2014; 4: 475
        • RSNA Pneumonia Detection Challenge
        Radiological society of North America.
      1. Open access biomedical image search engine.
        • Kermany D.
        • Zhang K.
        • Goldbaum M.
        Labeled Optical Coherence Tomography (oct) and Chest x-ray Images for Classification. Mendeley data, 2018: 2
        • Born J.
        POCOVID-Net: automatic detection of COVID-19 from a new lung ultrasound imaging dataset (POCUS).
        arXiv Prepr. arXiv2004.12084, 2020
        • Mueller S.G.
        • et al.
        Ways toward an early diagnosis in Alzheimer’s disease: the Alzheimer’s Disease Neuroimaging Initiative (ADNI).
        Alzheimers Dement. 2005; 1: 55-66
        • Kistler M.
        • et al.
        The virtual skeleton database: an open access repository for biomedical research and collaboration.
        J Med Internet Res. 2013; 15e2930
        • Di Martino A.
        • et al.
        The autism brain imaging data exchange: towards a large-scale evaluation of the intrinsic brain architecture in autism.
        Mol Psychiatry. 2014; 19: 659-667
        • Kwan R.-S.
        • Evans A.C.
        • Pike G.B.
        MRI simulation-based evaluation of image-processing and classification methods.
        IEEE Trans Med Imaging. 1999; 18: 1085-1097
        • Marek K.
        • et al.
        The parkinson progression marker initiative (PPMI).
        Prog Neurobiol. 2011; 95: 629-635
        • Clark K.
        • et al.
        The cancer imaging archive (TCIA): maintaining and operating a public information repository.
        J Digit Imaging. 2013; 26: 1045-1057
        • Heath M.
        Current status of the digital database for screening mammography.
        in: Digital mammography. Springer, 1998: 457-460
        • Suckling J.P.
        The mammographic image analysis society digital mammogram database.
        Digit Mammo. 1994; : 375-386
        • Fedorov A.
        • Vangel M.G.
        • Tempany C.M.
        • Fennessy F.M.
        Multiparametric magnetic resonance imaging of the prostate: repeatability of volume and apparent diffusion coefficient quantification.
        Invest Radiol. 2017; 52: 538
        • Fedorov A.
        • et al.
        An annotated test-retest collection of prostate multiparametric MRI.
        Sci Data. 2018; 5: 1-13
        • Sivaswamy J.
        • et al.
        A comprehensive retinal image dataset for the assessment of glaucoma from the optic nerve head analysis.
        JSM Biomed Imaging Data Pap. 2015; 2: 1004
        • Porwal P.
        • et al.
        Indian diabetic retinopathy image dataset (IDRiD): a database for diabetic retinopathy screening research.
        Data. 2018; 3: 25
        • Decencière E.
        • et al.
        Feedback on a publicly distributed image database: the Messidor database.
        Image Anal Stereol. 2014; 33: 231-234
        • Orlando J.I.
        • et al.
        Refuge challenge: a unified framework for evaluating automated methods for glaucoma assessment from fundus photographs.
        Med Image Anal. 2020; 59101570
        • Arunachalam H.B.
        • et al.
        Viable and necrotic tumor assessment from whole slide images of osteosarcoma using machine-learning and deep-learning models.
        PLoS One. 2019; 14e0210706
        • Sudirman S.
        Lumbar spine MRI dataset. Mendeley Data, 2019: 2
        • Lehmann T.M.
        • et al.
        Content-based image retrieval in medical applications.
        Methods Inf Med. 2004; 43: 354-361
        • Tschandl P.
        • Rosendahl C.
        • Kittler H.
        The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions.
        Sci Data. 2018; 5180161
        • Alzubaidi L.
        • et al.
        Robust application of new deep learning tools: an experimental study in medical imaging.
        Multimed Tools Appl. 2022; 81: 13289-13317
        • Antonelli M.
        • et al.
        The medical segmentation decathlon.
        Nat Commun. 2022; 13: 1-13
        • Barisoni L.
        • Lafata K.J.
        • Hewitt S.M.
        • Madabhushi A.
        • Balis U.G.J.
        Digital pathology and computational image analysis in nephropathology.
        Nat Rev Nephrol. 2020; 16: 669-685
        • Lin T.-Y.
        Microsoft coco: common objects in context.
        in: European conference on computer vision. 2014: 740-755
        • Nam J.G.
        • et al.
        Development and validation of deep learning–based automatic detection algorithm for malignant pulmonary nodules on chest radiographs.
        Radiology. 2019; 290: 218-228
        • Avaner E.
        İleri teknoloji Tıbbi cihazlar ve Sınırlı Kaynakların Adil Paylaşımı; manyeti rezonans (MR) ve Bilgisayarlı tomografi (BT) Cihazları Örnekleri.
        Türk Biyoetik Derg. 2019; 6: 100-108
        • Fedorov A.
        • et al.
        3D slicer as an image computing platform for the quantitative imaging network.
        Magn Reson Imaging. 2012; 30: 1323-1341
        • itk-SNAP
        http://www.itksnap.org/pmwiki/pmwiki.php
        Date accessed: October 14, 2022
        • v7
        • Lefkovits S.
        • Lefkovits L.
        • Szilágyi L.
        HGG and LGG brain tumor segmentation in multi-modal MRI using pretrained convolutional neural networks of Amazon sagemaker.
        Appl Sci. 2022; 12: 3620
        • Wang Y.
        • Yao Q.
        • Kwok J.T.
        • Ni L.M.
        Generalizing from a few examples: a survey on few-shot learning.
        ACM Comput Surv. 2020; 53: 1-34
        • Banik D.
        • Bhattacharjee D.
        Mitigating data imbalance issues in medical image analysis.
        in: Data preprocessing, active learning, and cost perceptive approaches for resolving data imbalance. IGI Global, 2021: 66-89
        • Al-Stouhi S.
        • Reddy C.K.
        Transfer learning for class imbalance problems with inadequate data.
        Knowl Inf Syst. 2016; 48: 201-228
        • Rankin D.
        • et al.
        Reliability of supervised machine learning using synthetic data in health care: model to preserve privacy for data sharing.
        JMIR Med Inform. 2020; 8e18910
        • Mazurowski M.A.
        • Buda M.
        • Saha A.
        • Bashir M.R.
        Deep learning in radiology: an overview of the concepts and a survey of the state of the art with focus on MRI.
        J Magn Reson Imaging. 2019; 49: 939-954
        • NHS Digital
        https://digital.nhs.uk/
        Date accessed: October 14, 2022
      2. Digital Health Knowledge Portal(dHealth, synapse).
        https://dhealth.synapse.org/
        Date accessed: October 14, 2022
        • Jaspers M.W.M.
        • Smeulers M.
        • Vermeulen H.
        • Peute L.W.
        Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings.
        J Am Med Inform Assoc. 2011; 18: 327-334
        • Lyell D.
        • Coiera E.
        Automation bias and verification complexity: a systematic review.
        J Am Med Inform Assoc. 2017; 24: 423-431
        • Sharma A.K.
        • Nandal A.
        • Dhaka A.
        • Dixit R.
        Medical image classification techniques and analysis using deep learning networks: a review.
        Heal Inform A Comput Perspect Healthc. 2021; : 233-258
        • Yoo J.H.
        • Jeong H.
        • Lee J.
        • Chung T.-M.
        Open problems in medical federated learning.
        Int J Web Inf Syst. 2022; 18: 77-99
        • Zhao W.
        Research on the deep learning of the small sample data based on transfer learning.
        in: AIP conference proceedings. vol. 1864. 2017: 20018
        • Islam K.T.
        • Wijewickrema S.
        • O’Leary S.
        A deep learning based framework for the registration of three dimensional multi-modal medical images of the head.
        Sci Rep. 2021; 11: 1-13
        • Altaf F.
        • Islam S.
        • Janjua N.K.
        A novel augmented deep transfer learning for classification of COVID-19 and other thoracic diseases from X-rays.
        Neural Comput Applic. 2021; 33: 14037-14048
        • Saha S.
        • Ahmad T.
        Federated transfer learning: concept and applications.
        Intel Artif. 2021; 15: 35-44
        • Mundhenk T.N.
        • Chen B.Y.
        • Friedland G.
        Efficient saliency maps for explainable ai.
        arXiv Prepr. arXiv1911.11293, 2019
        • Schöttl A.
        A light-weight method to foster the (Grad) CAM interpretability and explainability of classification networks.
        in: 2020 10th international conference on advanced computer information technologies (ACIT). 2020: 348-351
        • Ulas C.
        • Tetteh G.
        • Kaczmarz S.
        • Preibisch C.
        • Menze B.H.
        DeepASL: kinetic model incorporated loss for denoising arterial spin labeled MRI via deep residual learning.
        in: International conference on medical image computing and computer-assisted intervention. 2018: 30-38
        • Sahli Costabal F.
        • Yang Y.
        • Perdikaris P.
        • Hurtado D.E.
        • Kuhl E.
        Physics-informed neural networks for cardiac activation mapping.
        Front Phys. 2020; 8: 42
        • Sahlol A.T.
        • Abd Elaziz M.
        • Tariq Jamal A.
        • Damaševičius R.
        • Farouk Hassan O.
        A novel method for detection of tuberculosis in chest radiographs using artificial ecosystem-based optimisation of deep neural network features.
        Symmetry (Basel). 2020; 12: 1146
        • Abbas A.
        • Abdelsamea M.M.
        • Gaber M.M.
        4S-DT: self-supervised super sample decomposition for transfer learning with application to COVID-19 detection.
        IEEE Trans Neural Networks Learn Syst. 2021; 32: 2798-2808
        • Van Tulder G.
        • de Bruijne M.
        Learning cross-modality representations from multi-modal images.
        IEEE Trans Med Imaging. 2018; 38: 638-648
        • Ng A.Y.
        MLOps: from model-centric to data-centric AI.
        • Yang X.
        Transfer learning or self-supervised learning? A tale of two pretraining paradigms.
        arXiv Prepr. arXiv2007.04234, 2020
        • Zhou S.K.
        A review of deep learning in medical imaging: imaging traits, technology trends, case studies with progress highlights, and future promises.
        Proc IEEE. 2021; 26: 820-838
        • Zhou Z.
        Models genesis: generic autodidactic models for 3d medical image analysis.
        in: International conference on medical image computing and computer-assisted intervention. 2019: 384-393
        • Hinton G.E.
        • Sabour S.
        • Frosst N.
        Matrix capsules with EM routing.
        in: International conference on learning representations. 2018
        • Wang Y.
        • et al.
        Capsule networks showed excellent performance in the classification of hERG blockers/nonblockers.
        Front Pharmacol. 2020; 10: 1631
        • Anteby R.
        • et al.
        Deep learning for noninvasive liver fibrosis classification: a systematic review.
        Liver Int. 2021; 41: 2269-2278
        • Vaswani A.
        • et al.
        Attention is all you need.
        Adv Neural Inf Process Syst. 2017; 30
        • Ghasemi A.
        Vision transformers for transfer learning: an example and comparison to CNN-based architectures.
        • Dai Z.
        • Liu H.
        • Le Q.
        • Tan M.
        Coatnet: marrying convolution and attention for all data sizes.
        Adv Neural Inf Process Syst. 2021; 34