Advertisement

Substantial interreader agreement for biopsy with reduction in biopsy rate: A multireader diagnostic performance study of ACR TI-RADS

Published:February 09, 2022DOI:https://doi.org/10.1016/j.clinimag.2022.02.002

      Highlights

      • Among radiologists with less interpreting experience, there is fair interobserver agreement for final TI-RADS level scoring, and substantial agreement for biopsy recommendation based on established ACR TI-RADS criteria.
      • Retrospective scoring with established ACR TI-RADS criteria would have substantially reduced the number of biopsies performed.

      Abstract

      Purpose

      To assess diagnostic performance of ACR TI-RADS in thyroid cancer detection and evaluate interobserver agreement among radiologists with lower interpreting experience.

      Methods

      Four radiologists retrospectively assessed 295 biopsied thyroid nodules from ultrasound studies performed between 2009 and 2019, blinded to histopathology. Diagnostic performance for cancer detection was determined individually, and interobserver agreement among four readers was evaluated with Fleiss kappa coefficient (ⱪ).

      Results

      245 (83.1%) benign and 50 (16.9%) malignant nodules were evaluated. Diagnostic performance based on final TR level was consistent and without significant difference among four readers, with excellent sensitivity (≥98.0%) and negative predictive value (NPV) [≥94.4%] for TR levels 3 to 5. Diagnostic performance based on recommendation to biopsy has moderate sensitivity (≥62%) and high NPV (≥84.7%). Retrospective scoring with established ACR TI-RADS criteria would have substantially reduced the number of biopsies performed, with 63.2% of malignancy not biopsied meeting criteria for sonographic surveillance. Interobserver agreement on TI-RADS scoring for final TR level was fair (ⱪ = 0.39, 95% CI 0.32, 0.47), with substantial agreement for recommendation to biopsy (ⱪ = 0.64, 95% CI 0.58, 0.70).

      Conclusions

      Substantial reduction in biopsy rate (up to 48%) would have been achieved using the ACR TI-RADS criteria, with 63% of malignancy not biopsied meeting criteria for sonographic surveillance. Interobserver agreement was fair for TI-RADS level scoring and substantial for recommendation to biopsy.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Clinical Imaging
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Tessler F.N.
        • Middleton W.D.
        • Grant E.G.
        • et al.
        ACR thyroid imaging, reporting and data system (TI-RADS): white paper of the ACR TI-RADS committee.
        J Am Coll Radiol. 2017; 14: 587-595
        • Grant E.G.
        • Tessler F.N.
        • Hoang J.K.
        • et al.
        Thyroid ultrasound reporting lexicon: white paper of the ACR thyroid imaging, reporting and data system (TI-RADS) committee.
        J Am Coll Radiol. 2015; 12: 1272-1279
        • Tessler F.N.
        • Middleton W.D.
        • Grant E.G.
        Thyroid imaging reporting and data system (TI-RADS): a user’s guide.
        Radiology. 2018; 287: 29-36
        • Hoang J.K.
        • Middleton W.D.
        • Farjat A.E.
        • et al.
        Reduction in thyroid nodule biopsies and improved accuracy with American College of Radiology Thyroid Imaging Reporting and Data System.
        Radiology. 2018; 287: 185-193
        • Grani G.
        • Lamartina L.
        • Ascoli V.
        • et al.
        Reducing the number of unnecessary thyroid biopsies while improving diagnostic accuracy: toward the “right” TI-RADS.
        J Clin Endocrinol Metab. 2019; 104: 95-102
        • Li W.
        • Wang Y.
        • Wen J.
        • et al.
        Diagnostic performance of American College of Radiology TI-RADS: a systematic review and meta-analysis.
        AJR Am J Roentgenol. 2021; 216: 1-10
        • Middleton W.D.
        • Teefey S.A.
        • Reading C.
        • et al.
        Multi-institutional analysis of thyroid nodule risk stratification using the American College of Radiology Thyroid Imaging, reporting and data system.
        AJR Am J Roentgenol. 2017; 208: 1331-1341
        • Zheng Y.
        • Xu S.
        • Kang H.
        • et al.
        A single-center retrospective validation study of the American college of radiology thyroid imaging reporting and data system.
        Ultrasound Q. 2018; 34: 77-83
        • Middleton W.
        • Teefey S.
        • Reading C.
        • et al.
        Comparison of performance characteristics of American College of Radiology TI-RADS, Korean Society of Thyroid Radiology TIRADS, and American Thyroid Association guidelines.
        AJR Am J Roentgenol. 2018; 210: 1148-1154
        • Chung R.
        • Rosenkrantz A.B.
        • Bennett G.L.
        • et al.
        Intereader concordance of the TI-RADS: impact of radiologist experience.
        AJR Am J Roentgenol. 2020; 214: 1152-1157
        • Hoang J.K.
        • Middleton W.D.
        • Farjat A.E.
        • Teefey S.A.
        • Abinanti N.
        • Boschini F.J.
        • Bronner A.J.
        • Dahiya N.
        • Hertzberg B.S.
        • Newman J.R.
        • Scanga D.
        • Vogler R.C.
        • Tessler F.N.
        Interobserver variability of sonographic features used in the American College of Radiology Thyroid Imaging Reporting and Data System.
        AJR Am J Roentgenol. 2018 Jul; 211: 162-167
        • Cibas E.S.
        • Ali S.Z.
        The 2017 Bethesda system for reporting thyroid cytopathology.
        J Am Soc Cytopathol. 2017; 6: 217-222
        • Huh S.
        • Lee H.S.
        • Yoon J.
        • et al.
        Diagnostic performances and unnecessary US-FNA rates of various TIRADS after application of equal size thresholds.
        Sci Rep. 2020; 10: 10632
        • Cavallo A.
        • Johnson D.N.
        • White M.G.
        • et al.
        Thyroid nodule size at ultrasound as a predictor of malignancy and final pathologic size.
        Thyroid. 2017; 27: 641-650
        • Ito Y.
        • Miyauchi A.
        • Oda H.
        Low-risk papillary microcarcinoma of the thyroid: a review of active surveillance trials.
        Eur J Surg Oncol. 2018; 44: 307-315
        • Krajewska J.
        • Kukulska A.
        • Oczko-Wojciechowska M.
        • et al.
        Early diagnosis of low-risk papillary thyroid cancer results rather in overtreatment than a better survival.
        Front Endocrinol. 2020; 11571421
        • Jegerlehner S.
        • Bulliard J.L.
        • Aujesky D.
        • et al.
        Overdiagnosis and overtreatment of thyroid cancer: a population-based temporal trend study.
        PLoS One. 2017; 12e0179387
        • O’Grady T.J.
        • Gates M.A.
        • Boscoe F.P.
        Thyroid cancer incidence attributable to overdiagnosis in the United States 1981–2011.
        Int J Cancer. 2015; 137: 26642673
        • Welch H.G.
        • Doherty G.M.
        Saving thyroids - overtreatment of small papillary cancers.
        N Engl J Med. 2018; 379: 310-312
        • Ahn H.S.
        • Kim H.J.
        • Welch H.G.
        Korea’s thyroid-cancer “epidemic” – screening and overdiagnosis.
        N Engl J Med. 2014; 371: 1765-1767
        • Frates M.C.
        • Benson C.B.
        • Doubilet P.M.
        • et al.
        Prevalence and distribution of carcinoma in patients with solitary and multiple thyroid nodules on sonography.
        J Clin Endocrinol Metab. 2006; 91: 3411-3417
        • Schenke S.
        • Seifert P.
        • Zimny M.
        • et al.
        Risk stratification of thyroid nodules using the thyroid imaging reporting and data system (TIRADS): the omission of thyroid scintigraphy increases the rate of falsely suspected lesions.
        J Nucl Med. 2019; 60: 342-347