A study of more than 32,000 CTPA scans found that artificial intelligence and physicians agreed on pulmonary embolism results nearly 98% of the time, though human review caught cases the software missed.
RT’s Three Key Takeaways:
- High Concordance Rates: An artificial intelligence algorithm for detecting pulmonary embolism matched radiologist interpretations in 97.8% of more than 32,000 scans performed in a real-world healthcare system.
- Critical Physician Oversight: While the artificial intelligence tool assisted in triage, radiologists identified 15% of confirmed cases that the software missed, highlighting the importance of human-in-the-loop interpretation.
- Urgent Case Reliability: The agreement between the technology and physicians was highest for acute and central emboli, which are the conditions associated with the greatest mortality risk.
An artificial intelligence (Ai) algorithm for pulmonary embolism (PE) detection demonstrated 97.8% agreement with radiologist interpretations in a large study of more than 32,000 Computed Tomography Pulmonary Angiography (CTPA) scans, according to a study published in Radiology: Artificial Intelligence.
Researchers from Northwell Health analyzed 32,501 CTPAs performed across an integrated healthcare system over an 18-month period. The study found that concordance was higher for negative exams than positive exams, at 98.18% and 93.75%, respectively.
PE is a life-threatening cardiovascular condition responsible for 5% to 10% of in-hospital deaths and more than 300,000 deaths annually in the US. While several FDA cleared Ai tools are available, the researchers noted that few large-scale studies have evaluated their performance in real-world clinical practice using a human-in-the-loop model.
In the study, the Ai algorithm, developed by AIDOC, analyzed CTPAs and flagged suspected positive cases to assist radiologists with triage. When the Ai and radiologists disagreed, expert thoracic radiologists adjudicated the cases. The analysis revealed that radiologists were correct in 88.7% of those disagreements, Ai was correct in 11.3%, and the combined approach improved overall accuracy.
“Ai-informed radiologists achieved a sensitivity of 99.2% for pulmonary embolism detection. Radiologist-Ai agreement was highest for acute and central emboli – the cases associated with the greatest clinical urgency and mortality risk,” said Shlomit Goldberg-Stein, professor of radiology and director of artificial intelligence at the Zucker School of Medicine at Hofstra/Northwell, in a news release.
The findings also showed that 483 positive cases, or 15%, were detected only with radiologist involvement demonstrating the importance of subsequent radiologist review when Ai was negative.
