Evaluation of Auto-Contouring Software for Brain Metastasis Detection in Stereotactic Radiosurgery: A Comparison to Physician Review

Objectives: : Accurate detecting and contouring are essential for brain metastases stereotactic radiosurgery (SRS). Auto-detection and contouring software aims to enhance contouring accuracy while significantly reducing the time needed for manual segmentation, a process known to be both labor-intensive and subject to variability among physicians. Despite its advantages, the software’s accuracy in delineating metastases may not always match clinical observations, leading to potential over-treatment or under-treatment. This study evaluates the efficacy of an auto-contouring software by comparing it with physician-generated contours to determine its impact on decision-making in SRS for brain metastases.

Methods: : A retrospective cohort study was conducted after obtaining institutional review board approval. The medical records of patients that underwent radiosurgery for brain metastases at our institution between 2016 and 2022 were reviewed. Physician contoured metastases and auto-contoured metastases were compared. Most of the contoured targets matched between the physicians and auto-contouring software. Discrepant contours were identified and reviewed by three independent physicians. They reviewed each of the unmatched objects on the 3D T1 contrasted MRI study and rated whether it was a true brain metastasis with a confidence score on a 0 ~ 100 scale. We then assessed the subsequent course of the discrepant contoured objects to determine whether they were true metastases (progressed and then identified and treated in subsequent SRS courses).

Results: : A total of 42 patients with multiple brain mets were included in the study. 223 mets were contoured clinically and 236 mets were contoured by the auto-contour software. 203 were matched and 53 were unmatched (33 drawn by auto-contouring algorithm only and 20 clinically contoured metastases not detected by the software). Among the 33 lesions identified solely by the auto-lesion detection algorithm, 19 (57.6%) were confirmed as true positives, while 14 (42.4%) were false positives. The auto-contouring algorithm's performance was evaluated against the three physicians. When reviewing the 33 lesions detected by the software, Physicians 1, 2, and 3 identified 48.5%, 42.4%, and 48.5% of the lesions, respectively, as true positives. In the five cases where physicians provided ambiguous ratings (with at least one rating in the 66%-100% range and another in the 0%-33% range), the brain metastases were small, with a mean volume of 0.024 cc.

Conclusion(s):: The auto-lesion detection algorithm shows potential to improve the speed and consistency of brain metastasis detection, allowing for faster treatment. Preliminary results suggest that the algorithm may have utility in detecting metastases. However, the risk of false positives and missed lesions highlights the need for careful interpretation and physician oversight. Further research is needed to refine the software’s accuracy and to assess its long-term impact on clinical outcomes.