Multicenter evaluation of commercial AI for [18F]FDG PET/CT lesion detection
Annals of Nuclear Medicine, 2026
Abstract
Objective
Commercial Artificial Intelligence (AI) tools for [18F]FDG PET/CT lack real-world evidence. This study evaluated the lesion detection performance of two commercial AI-based algorithms, both alone and as adjuncts to expert reading.
Methods
We retrospectively analyzed 151 [18F]FDG PET/CT scans of patients managed for melanoma or lymphoma in 3 French centers. Lesion were detected by 4 methods: manual expert reading (M1); a PET Assisted Reporting System trained on 629 patients with lymphoma and lung cancers, with SUV 2.5 threshold (M2) or PERCIST-like threshold (M3), and a one-step U-net algorithm trained on 4,906 patients with multiple neoplasia (M4). Expert consensus adjudicated all volumes of interest (VOIs) as the lesion-level reference standard, with neoplastic VOIs designated as true positives (TP). Primary endpoint was per-lesion sensitivity. Secondary endpoints were false-positive (FP) VOIs, and performance of human–AI combinations.
Results
Among 1,544 reference lesions, per-lesion sensitivity was 76.6% (M1), 60.2% (M2), 44.3% (M3), 95.7% (M4), 83.4% (M1 + M2), 79.7% (M1 + M3), and 99.9% (M1 + M4). All methods combining AI and expert reading showed significantly higher sensitivity than that obtained by the expert alone. Higher sensitivity coincided with greater false-positive burdens: FP VOIs were 837 (M2), 151 (M3), and 1,435 (M4). Subgroup analyses showed human sensitivity dropped with > 5 lesions, while AI sensitivity was preserved.
Conclusion
Our study of several commercially available software solutions reveals that deep learning algorithms are currently not accurate enough to be used without being combined with expert interpretation. The number of false positive VOIs was higher for solution offering the higher sensitivity, which may hinder their widespread use.
