Skip to content
2000
Volume 21, Issue 1
  • ISSN: 1573-4056
  • E-ISSN: 1875-6603

Abstract

Introduction

Detecting Pulmonary Embolism (PE) is critical for effective patient care, and Artificial Intelligence (AI) has shown promise in supporting radiologists in this task. Integrating AI into radiology workflows requires not only evaluation of its diagnostic accuracy but also assessment of its acceptance among clinical staff.

Objective

This study aims to evaluate the performance of an AI algorithm in detecting pulmonary embolisms (PEs) on contrast-enhanced computed tomography pulmonary angiograms (CTPAs) and to assess the level of acceptance of the algorithm among radiology department staff.

Methods

This retrospective study analyzed anonymized computed tomography pulmonary angiography (CTPA) data from a university clinic. Surveys were conducted at three and nine months after the implementation of a commercially available AI algorithm designed to flag CTPA scans with suspected PE. A thoracic radiologist and a cardiac radiologist served as the reference standard for evaluating the performance of the algorithm. The AI analyzed 59 CTPA cases during the initial evaluation and 46 cases in the follow-up assessment.

Results

In the first evaluation, the AI algorithm demonstrated a sensitivity of 84.6% and a specificity of 94.3%. By the second evaluation, its performance had improved, achieving a sensitivity of 90.9% and a specificity of 96.7%. Radiologists’ acceptance of the AI tool increased over time. Nevertheless, despite this growing acceptance, many radiologists expressed a preference for hiring an additional physician over adopting the AI solution if the costs were comparable.

Discussion

Our study demonstrated high sensitivity and specificity of the AI algorithm, with improved performance over time and a reduced rate of unanalyzed scans. These improvements likely reflect both algorithmic refinement and better data integration. Departmental feedback indicated growing user confidence and trust in the tool. However, many radiologists continued to prefer the addition of a resident over reliance on the algorithm. Overall, the AI showed promise as a supportive “second-look” tool in emergency radiology settings.

Conclusion

The AI algorithm demonstrated diagnostic performance comparable to that reported in similar studies for detecting PE on CTPA, with both sensitivity and specificity showing improvement over time. Radiologists’ acceptance of the algorithm increased throughout the study period, underscoring its potential as a complementary tool to physician expertise in clinical practice.

This is an open access article published under CC BY 4.0 https://creativecommons.org/licenses/by/4.0/legalcode
Loading

Article metrics loading...

/content/journals/cmir/10.2174/0115734056367860250630072749
2025-07-23
2025-09-20
Loading full text...

Full text loading...

/deliver/fulltext/cmir/21/1/CMIR-21-E15734056367860.html?itemId=/content/journals/cmir/10.2174/0115734056367860250630072749&mimeType=html&fmt=ahah

References

  1. KonstantinidesS.V. MeyerG. BecattiniC. BuenoH. GeersingG.J. HarjolaV.P. HuismanM.V. HumbertM. JenningsC.S. JiménezD. KucherN. LangI.M. LankeitM. LorussoR. MazzolaiL. MeneveauN. ÁinleF.N. PrandoniP. PruszczykP. RighiniM. TorbickiA. Van BelleE. ZamoranoJ.L. The Task Force for the diagnosis and management of acute pulmonary embolism of the European Society of Cardiology (ESC) 2019 ESC guidelines for the diagnosis and management of acute pulmonary embolism developed in collaboration with the european respiratory society (ERS).Eur. Respir. J.2019543190164710.1183/13993003.01647‑201931473594
    [Google Scholar]
  2. WendelboeA.M. RaskobG.E. Global burden of thrombosis.Circ. Res.201611891340134710.1161/CIRCRESAHA.115.30684127126645
    [Google Scholar]
  3. MartinK.A. MolsberryR. CutticaM.J. DesaiK.R. SchimmelD.R. KhanS.S. Time trends in pulmonary embolism mortality rates in the United States, 1999 to 2018.J. Am. Heart Assoc.2020917e01678410.1161/JAHA.120.01678432809909
    [Google Scholar]
  4. AgnelliG. AndersonF. ArcelusJ. BergqvistD. BrechtJ. GreerI. HeitJ. HutchinsonJ. KakkarA. MottierD. OgerE. SamamaM-M. SpannaglM. CohenA. VTE Impact Assessment Group in Europe (VITAE) Venous thromboembolism (VTE) in Europe.Thromb. Haemost.2007981075676410.1160/TH07‑03‑021217938798
    [Google Scholar]
  5. BĕlohlávekJ. DytrychV. LinhartA. Pulmonary embolism, part I: Epidemiology, risk factors and risk stratification, pathophysiology, clinical presentation, diagnosis and nonthrombotic pulmonary embolism.Exp. Clin. Cardiol.201318212913823940438
    [Google Scholar]
  6. DoğanH. de RoosA. GeleijinsJ. HuismanM. KroftL. The role of computed tomography in the diagnosis of acute and chronic pulmonary embolism.Diagn. Interv. Radiol.201521430731610.5152/dir.2015.1440326133321
    [Google Scholar]
  7. GottschalkA. SteinP.D. GoodmanL.R. SostmanH.D. Overview of prospective investigation of pulmonary embolism diagnosis II.Semin. Nucl. Med.200232317318210.1053/snuc.2002.12417712105798
    [Google Scholar]
  8. SteinP.D. FowlerS.E. GoodmanL.R. GottschalkA. HalesC.A. HullR.D. LeeperK.V.Jr PopovichJ.Jr QuinnD.A. SosT.A. SostmanH.D. TapsonV.F. WakefieldT.W. WegJ.G. WoodardP.K. PIOPED II Investigators Multidetector computed tomography for acute pulmonary embolism.N. Engl. J. Med.2006354222317232710.1056/NEJMoa05236716738268
    [Google Scholar]
  9. Winer-MuramH.T. RydbergJ. JohnsonM.S. TarverR.D. WilliamsM.D. ShahH. NamyslowskiJ. ConcesD. JenningsS.G. YingJ. TrerotolaS.O. KopeckyK.K. Suspected acute pulmonary embolism: Evaluation with multi-detector row CT versus digital subtraction pulmonary arteriography.Radiology2004233380681510.1148/radiol.233303174415564410
    [Google Scholar]
  10. Remy-JardinM. PistolesiM. GoodmanL.R. GefterW.B. GottschalkA. MayoJ.R. SostmanH.D. Management of suspected acute pulmonary embolism in the era of CT angiography: A statement from the Fleischner Society.Radiology2007245231532910.1148/radiol.245207039717848685
    [Google Scholar]
  11. WangR.C. MigliorettiD.L. MarlowE.C. KwanM.L. TheisM.K. BowlesE.J.A. GreenleeR.T. RahmA.K. StoutN.K. WeinmannS. Smith-BindmanR. Trends in imaging for suspected pulmonary embolism across US health care systems, 2004 to 2016.JAMA Netw. Open2020311e202693010.1001/jamanetworkopen.2020.2693033216141
    [Google Scholar]
  12. AzadR. Medical image segmentation review: The success of U-Net.arXiv202210.48550/ARXIV.2211.14830
    [Google Scholar]
  13. XingZ. YeT. YangY. LiuG. ZhuL. SegMamba: Long-range sequential modeling mamba For 3D medical image segmentation.arXiv202410.1007/978‑3‑031‑72111‑3_54
    [Google Scholar]
  14. WangZ. ZhengJ-Q. ZhangY. CuiG. LiL. Mamba-UNet: UNet-like pure visual mamba for medical image segmentation.arXiv202410.48550/ARXIV.2402.05079
    [Google Scholar]
  15. DingY. LiL. WangW. YangY. Clustering propagation for universal medical image segmentation.arXiv202410.1109/CVPR52733.2024.00323
    [Google Scholar]
  16. JinQ. CuiH. SunC. MengZ. WeiL. SuR. Domain adaptation based self-correction model for COVID-19 infection segmentation in CT images.Expert Syst. Appl.202117611484810.1016/j.eswa.2021.11484833746369
    [Google Scholar]
  17. AhmadZ. Al-MaadeedS.A. KhanM.A. Enhanced diagnostic of pulmonary embolism detection using densenet and XGBoost.2024 International Conference on Future Technologies for Smart Society (ICFTSS)Kuala Lumpur, Malaysia, 07-08 August 2024, pp. 106-111.10.1109/ICFTSS61109.2024.10691334
    [Google Scholar]
  18. HeK. ZhangX. RenS. SunJ. Deep residual learning for image recognition.2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)Las Vegas, NV, USA, 27-30 June 2016, pp. 770-778.10.1109/CVPR.2016.90
    [Google Scholar]
  19. BaderA. Society of thoracic radiology abstracts from the 2020 annual meeting and postgraduate course march 8-11, 2020 Hyatt regency indian wells resort & spa in indian wells, California.J. Thorac. Imaging2020356W130W18010.1097/RTI.0000000000000546
    [Google Scholar]
  20. CheikhA.B. GorincourG. NivetH. MayJ. SeuxM. CalameP. ThomsonV. DelabrousseE. CrombéA. How artificial intelligence improves radiological interpretation in suspected pulmonary embolism.Eur. Radiol.20223295831584210.1007/s00330‑022‑08645‑235316363
    [Google Scholar]
  21. EbrahimianS. DigumarthyS.R. HomayouniehF. BizzoB.C. DreyerK.J. KalraM.K. Predictive values of AI-based triage model in suboptimal CT pulmonary angiography.Clin. Imaging202286253010.1016/j.clinimag.2022.03.01135316621
    [Google Scholar]
  22. HuhtanenH. NymanM. MohsenT. VirkkiA. KarlssonA. HirvonenJ. Automated detection of pulmonary embolism from CT-angiograms using deep learning.BMC Med. Imaging20222214310.1186/s12880‑022‑00763‑z35282821
    [Google Scholar]
  23. WeikertT. WinkelD.J. BremerichJ. StieltjesB. ParmarV. SauterA.W. SommerG. Automated detection of pulmonary embolism in CT pulmonary angiograms using an AI-powered algorithm.Eur. Radiol.202030126545655310.1007/s00330‑020‑06998‑032621243
    [Google Scholar]
  24. FangM.C. FanD. SungS.H. WittD.M. SchmelzerJ.R. WilliamsM.S. YaleS.H. BaumgartnerC. GoA.S. Treatment and outcomes of acute pulmonary embolism and deep venous thrombosis: The CVRN VTE study.Am. J. Med.20191321214501457.e110.1016/j.amjmed.2019.05.04031247183
    [Google Scholar]
  25. KennedyS. BhargavanM. SunshineJ.H. FormanH.P. The effect of teleradiology on time to interpretation for CT pulmonary angiographic studies.J. Am. Coll. Radiol.200963180189.e110.1016/j.jacr.2008.09.01319248994
    [Google Scholar]
  26. BarrittD.W. JordanS.C. Anticoagulant drugs in the treatment of pulmonary embolism. A controlled trial.Lancet196027571381309131210.1016/S0140‑6736(60)92299‑613797091
    [Google Scholar]
  27. PollackC.V. SchreiberD. GoldhaberS.Z. SlatteryD. FanikosJ. O’NeilB.J. ThompsonJ.R. HiestandB. BrieseB.A. PendletonR.C. MillerC.D. KlineJ.A. Clinical characteristics, management, and outcomes of patients diagnosed with acute pulmonary embolism in the emergency department: Initial report of EMPEROR (multicenter emergency medicine pulmonary embolism in the real world registry).J. Am. Coll. Cardiol.201157670070610.1016/j.jacc.2010.05.07121292129
    [Google Scholar]
  28. MeyerG. PlanquetteB. SanchezO. Long-term outcome of pulmonary embolism.Curr. Opin. Hematol.200815549950310.1097/MOH.0b013e3283063a5118695374
    [Google Scholar]
  29. O’NeillT.J. XiY. StehelE. BrowningT. NgY.S. BakerC. PeshockR.M. Active reprioritization of the reading worklist using artificial intelligence has a beneficial effect on the turnaround time for interpretation of head CT with intracranial hemorrhage.Radiol. Artif. Intell.202132e20002410.1148/ryai.202020002433937858
    [Google Scholar]
  30. MaxwellS. HaN.T. BulsaraM.K. DoustJ. McrobbieD. O’LearyP. SlavotinekJ. MoorinR. Increasing use of CT requested by emergency department physicians in tertiary hospitals in Western Australia 2003–2015: An analysis of linked administrative data.BMJ Open2021113e04331510.1136/bmjopen‑2020‑04331533664075
    [Google Scholar]
  31. ChilamkurthyS. GhoshR. TanamalaS. BivijiM. CampeauN.G. VenugopalV.K. MahajanV. RaoP. WarierP. Deep learning algorithms for detection of critical findings in head CT scans: A retrospective study.Lancet2018392101622388239610.1016/S0140‑6736(18)31645‑330318264
    [Google Scholar]
  32. QinZ.Z. SanderM.S. RaiB. TitahongC.N. SudrungrotS. LaahS.N. AdhikariL.M. CarterE.J. PuriL. CodlinA.J. CreswellJ. Using artificial intelligence to read chest radiographs for tuberculosis detection: A multi-site evaluation of the diagnostic accuracy of three deep learning systems.Sci. Rep.2019911500010.1038/s41598‑019‑51503‑331628424
    [Google Scholar]
  33. SchaffterT. BuistD.S.M. LeeC.I. NikulinY. RibliD. GuanY. LotterW. JieZ. DuH. WangS. FengJ. FengM. KimH.E. AlbiolF. AlbiolA. MorrellS. WojnaZ. AhsenM.E. AsifU. Jimeno YepesA. YohanandanS. Rabinovici-CohenS. YiD. HoffB. YuT. Chaibub NetoE. RubinD.L. LindholmP. MargoliesL.R. McBrideR.B. RothsteinJ.H. SiehW. Ben-AriR. HarrerS. TristerA. FriendS. NormanT. SahinerB. StrandF. GuinneyJ. StolovitzkyG. MackeyL. CahoonJ. ShenL. SohnJ.H. TrivediH. ShenY. ButurovicL. PereiraJ.C. CardosoJ.S. CastroE. KallebergK.T. PelkaO. NedjarI. GerasK.J. NensaF. GoanE. KoitkaS. CaballeroL. CoxD.D. KrishnaswamyP. PandeyG. FriedrichC.M. PerrinD. FookesC. ShiB. Cardoso NegrieG. KawczynskiM. ChoK. KhooC.S. LoJ.Y. SorensenA.G. JungH. and the DM DREAM Consortium Evaluation of combined artificial intelligence and radiologist assessment to interpret screening mammograms.JAMA Netw. Open202033e20026510.1001/jamanetworkopen.2020.026532119094
    [Google Scholar]
  34. JoshiR. WuK. KaickerJ. ChoudurH. Reliability of on-call radiology residents’ interpretation of 64-slice CT pulmonary angiography for the detection of pulmonary embolism.Acta Radiol.201455668269010.1177/028418511350613524092761
    [Google Scholar]
  35. ShahamD. HeffezR. BogotN.R. LibsonE. BrezisM. CT pulmonary angiography for the detection of pulmonary embolism: Interobserver agreement between on-call radiology residents and specialists (CTPA interobserver agreement).Clin. Imaging200630426627010.1016/j.clinimag.2006.01.00116814143
    [Google Scholar]
  36. TamjeediB. CorreaJ. SemionovA. MesurolleB. Interobserver agreement between on-call radiology resident and general radiologist interpretations of CT pulmonary angiograms and CT venograms.PLoS One2015105e012611610.1371/journal.pone.012611625938666
    [Google Scholar]
  37. VerweijJ.W. HofsteeH.M.A. GoldingR.P. van WaesbergheJ.H.T.M. SmuldersY.M. Interobserver agreement between on-call radiology residents and radiology specialists in the diagnosis of pulmonary embolism using computed tomography pulmonary angiography.J. Comput. Assist. Tomogr.200933695295510.1097/RCT.0b013e3181a2f7fa19940666
    [Google Scholar]
  38. YavasU.S. CalisirC. OzkanI.R. The interobserver agreement between residents and experienced radiologists for detecting pulmonary embolism and DVT with using CT pulmonary angiography and indirect CT venography.Korean J. Radiol.20089649850210.3348/kjr.2008.9.6.49819039265
    [Google Scholar]
/content/journals/cmir/10.2174/0115734056367860250630072749
Loading
/content/journals/cmir/10.2174/0115734056367860250630072749
Loading

Data & Media loading...

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error
Please enter a valid_number test