Automated Breast Cancer Diagnosis with YOLO and Generative AI

Abstract

Breast cancer is a leading cause of mortality among women in Brazil, and human interpretation of mammograms still has limitations. This study evaluated a BI-RADS (Breast Imaging Reporting and Data System) classification pipeline that uses the YOLO (You Only Look Once) model for lesion segmentation and the ChatGPT language model for interpretation and assisted diagnosis. 120 images classified with the following distribution were used to evaluate the results: BI-RADS 3 (68 images), 4 (44 images), and 5 (images). Three approaches were given: (i) direct AI diagnosis; (ii) a pre-trained model without preprocessing getting used in the ChatGPT; and (iii) the same model with detailed morphological descriptions and enhancement filters (CLAHE and Sharpen), with ChatGPT producing the classification. The overall accuracies were 30%, 42%, and 70%, respectively. The methodology improved the accuracy of the results, although it still presents limitations in the model's ability to differentiate specific cases, or even creating a confusion in the interpretability, indicating space for future improvements.