Training AI to Improve Distinction of Triple-Negative Invasive Breast Cancer from Cysts and Fibroadenomas on Ultrasound
Diagnostics, 2026
Abstract
Introduction: Circumscribed oval, hypoechoic masses are common on screening breast ultrasound (US), and the vast majority are benign. Triple-receptor negative invasive breast cancer (TNBC) can appear similar, resulting in both human and artificial intelligence (AI) interpretive errors. This study sought to improve AI performance in distinguishing common benign masses from TNBC through a retrospective model refinement and validation study.
Methods: In an Institutional Review Board-approved HIPAA-compliant protocol, from five academic medical centers, orthogonal ultrasound images of 1771 breast masses 2 cm or smaller were acquired, consisting of cysts, complicated cysts, other benign, and malignancies. Cases were randomized, controlling for lesion class, site, and patient age, with 1446 (including 402, 27.8%, malignancies) used for training and 325 (including 95, 29.2% malignancies) for validation using Koios DS® (decision support, KDS) software version 2.0. A breast imaging radiologist from each center reviewed images and recorded BI-RADS features and assessment. Demographics, symptoms, and pathology or at least one-year follow-up was recorded. The KDS score was evaluated standalone and in combination with BI-RADS using logistic regression and ROC analysis with focus on specificity at sensitivity of 98%.
Results: In training, KDS standalone performed comparably to BI-RADS, and significantly improved BI-RADS malignancy risk prediction (p < 0.001). The 98%–sensitivity threshold for combined KDS + BI-RADS was estimated and kept fixed during validation. In validation, KDS standalone performed similar to BI-RADS with AUC = 0.97 (CI: 0.95–0.98) versus 0.95 (p = 0.22), with sensitivity of 98% (93/95, CI: 95–100%) for both and specificity of 70.9% (163/230, CI: 65.0–76.7%) for KDS versus 63.9% for BI-RADS (147/230, p = 0.10). Combining KDS + BIRADS significantly improved overall performance (AUC 0.98, p < 0.001) and specificity (74.4%, 171/230, p < 0.001) while maintaining sensitivity at 98% (93/95).
Conclusion: While KDS alone should not replace BI-RADS, when used in combination with BI-RADS, it can significantly improve specificity for highly accurate (98% sensitivity) triaging management of masses representative of those seen on screening US.
