Research Article| Volume 83, P177-183, March 2022

Download started.


Repeatability, robustness, and reproducibility of texture features on 3 Tesla liver MRI


      • 49–67% of texture feature measurements had good-to-excellent repeatability.
      • Only 19–22% of texture features had good-to-excellent robustness to parameter change.
      • Only 25–52% of texture features had good-to-excellent interreader reproducibility.
      • First order features were least repeatable, but were most frequently reproducible.
      • Different parameter changes had varying effects on texture feature outputs.



      Texture features are proposed for classification and prognostication, with lacking information about variability. We assessed 3 T liver MRI feature variability.


      Five volunteers underwent standard 3 T MRI, and repeated with identical and altered parameters. Two readers placed regions of interest using 3DSlicer. Repeatability (between standard and repeat scan), robustness (between standard and parameter changed scan), and reproducibility (two reader variation) were computed using coefficient of variation (CV).


      67%, 49%, and 61% of features had good-to-excellent (CV ≤ 10%) repeatability on ADC, T1, and T2, respectively, least frequently for first order (19–35%). 22%, 19%, and 21% of features had good-to-excellent robustness on ADC, T1, and T2, respectively. 52%, 35%, and 25% of feature measurements had good-to-excellent inter-reader reproducibility on ADC, T1, and T2, respectively, with highest good-to-excellent reproducibility for first order features on ADC/T1.


      We demonstrated large variations in texture features on 3 T liver MRI. Further study should evaluate methods to reduce variability.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Clinical Imaging
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Haralick R.M.
        • Shanmugam K.
        • Dinstein I.
        Textural features for image classification.
        IEEE Trans Syst Man Cybern. 1973; SMC-3: 610-621
        • Galloway M.M.
        Texture analysis using gray level run lengths.
        Comput Graphics Image Process. 1975; 4: 172-179
        • Zhang X.
        • Xu X.
        • Tian Q.
        • et al.
        Radiomics assessment of bladder cancer grade using texture features from diffusion-weighted imaging.
        J Magn Reson Imaging. Nov 2017; 46: 1281-1288
        • Li Z.
        • Mao Y.
        • Huang W.
        • et al.
        Texture-based classification of different single liver lesion based on SPAIR T2W MRI images.
        BMC Med Imaging. Jul 13 2017; 17: 42
        • Gnep K.
        • Fargeas A.
        • Gutierrez-Carvajal R.E.
        • et al.
        Haralick textural features on T2 -weighted MRI are associated with biochemical recurrence following radiotherapy for peripheral zone prostate cancer.
        J Magn Reson Imaging. Jan 2017; 45: 103-117
        • Cattell R.
        • Chen S.
        • Huang C.
        Robustness of radiomic features in magnetic resonance imaging: review and a phantom study.
        Vis Comput Ind Biomed Art. Nov 20 2019; 2: 19
        • Zhao B.
        • Tan Y.
        • Tsai W.Y.
        • et al.
        Reproducibility of radiomics for deciphering tumor phenotype with imaging.
        Sci Rep. Mar 24 2016; 6: 23428
        • Berenguer R.
        • Pastor-Juan M.D.R.
        • Canales-Vazquez J.
        • et al.
        Radiomics of CT features may be nonreproducible and redundant: influence of CT acquisition parameters.
        Radiology. Aug 2018; 288: 407-415
        • Meyer M.
        • Ronald J.
        • Vernuccio F.
        • et al.
        Reproducibility of CT radiomic features within the same patient: influence of radiation dose and CT reconstruction settings.
        Radiology. Oct 1 2019; : 190928
        • Mayerhoefer M.E.
        • Szomolanyi P.
        • Jirak D.
        • Materka A.
        • Trattnig S.
        Effects of MRI acquisition parameter variations and protocol heterogeneity on the results of texture analysis and pattern discrimination: an application-oriented study.
        Med Phys. Apr 2009; 36: 1236-1243
        • Waugh S.A.
        • Lerski R.A.
        • Bidaut L.
        • Thompson A.M.
        The influence of field strength and different clinical breast MRI protocols on the outcome of texture analysis using foam phantoms.
        Med Phys. Sep 2011; 38: 5058-5066
        • Molina D.
        • Perez-Beteta J.
        • Martinez-Gonzalez A.
        • et al.
        Lack of robustness of textural measures obtained from 3D brain tumor MRIs impose a need for standardization.
        PLoS One. 2017; 12e0178843
        • Chirra P.
        • Leo P.
        • Yim M.
        • et al.
        Multisite evaluation of radiomic feature reproducibility and discriminability for identifying peripheral zone prostate tumors on MRI.
        J Med Imaging (Bellingham). Apr 2019; 6024502
        • Whitney H.M.
        • Drukker K.
        • Edwards A.
        • Papaioannou J.
        • Giger M.L.
        Robustness of radiomic breast features of benign lesions and luminal A cancers across MR magnet strengths.
        in: Proceedings of SPIE 10575, Medical Imaging 2018: Computer-Aided Diagnosis. 105750A. 2018
        • Fedorov A.
        • Beichel R.
        • Kalpathy-Cramer J.
        • et al.
        3D slicer as an image computing platform for the quantitative imaging network.
        Magn Reson Imaging. Nov 2012; 30: 1323-1341
        • van Griethuysen J.J.M.
        • Fedorov A.
        • Parmar C.
        • et al.
        Computational radiomics system to decode the radiographic phenotype.
        Cancer Res. 2017; 77: e104-e107
        • Musunuru H.B.
        • Yamamoto T.
        • Klotz L.
        • et al.
        Active surveillance for intermediate risk prostate cancer: survival outcomes in the sunnybrook experience.
        J Urol. Dec 2016; 196: 1651-1658
        • House M.J.
        • Bangma S.J.
        • Thomas M.
        • et al.
        Texture-based classification of liver fibrosis using MRI.
        J Magn Reson Imaging. Feb 2015; 41: 322-328
        • Juntu J.
        • Sijbers J.
        • De Backer S.
        • Rajan J.
        • Van Dyck D.
        Machine learning study of several classifiers trained with texture analysis features to differentiate benign from malignant soft-tissue tumors in T1-MRI images.
        J Magn Reson Imaging. Mar 2010; 31: 680-689
        • Romeo V.
        • Ricciardi C.
        • Cuocolo R.
        • et al.
        Machine learning analysis of MRI-derived texture features to predict placenta accreta spectrum in patients with placenta previa.
        Magn Reson Imaging. May 15 2019;
        • Savio S.J.
        • Harrison L.C.
        • Luukkaala T.
        • et al.
        Effect of slice thickness on brain magnetic resonance image texture analysis.
        Biomed Eng Online. Oct 18 2010; 9: 60
        • Farrance I.
        • Frenkel R.
        Uncertainty of measurement: a review of the rules for calculating uncertainty components through functional relationships.
        Clin Biochem Rev. 2012; 33: 49-75
        • Hu H.T.
        • Shan Q.Y.
        • Chen S.L.
        • et al.
        CT-based radiomics for preoperative prediction of early recurrent hepatocellular carcinoma: technical reproducibility of acquisition and scanners.
        Radiol Med. Aug 2020; 125: 697-705
        • Orlhac F.
        • Frouin F.
        • Nioche C.
        • Ayache N.
        • Buvat I.
        Validation of a method to compensate multicenter effects affecting CT radiomics.
        Radiology. Apr 2019; 291: 53-59
        • Peerlings J.
        • Woodruff H.C.
        • Winfield J.M.
        • et al.
        Stability of radiomics features in apparent diffusion coefficient maps from a multi-centre test-retest trial.
        Sci Rep. 2019; 9 (03 18): 4800
        • Mali S.A.
        • Ibrahim A.
        • Woodruff H.C.
        • et al.
        Making radiomics more reproducible across scanner and imaging protocol variations: a review of harmonization methods.
        J Pers Med. Aug 27 2021; 11