Transfer deep learning mammography diagnostic model from public datasets to clinical practice: A comparison of model performance and mammography datasets

Literatures have showed that deep learning models can detect a breast cancer with high diagnostic accuracy in the publicly available mammography datasets. The objective of this study is to examine whether the high performance (accuracy) of a deep learning model, trained by the public mammography dataset, can be transferred into the clinic practice by applying it to a new mammography dataset obtained in an academic breast center. An end-to-end CNN architecture was trained on DDSM dataset and transferred to INbreast dataset and the in-house collected dataset. The model achieved validation AUC of 0.82 on DDSM dataset and 0.93 on INbreast dataset. However, it only achieved 0.70 when applied to the in-house dataset. Reviewing the images revealed that the in-house dataset is more challenging to classify. The mean subtlety score for DDSM dataset is 3.64 and median is 4. For in-house dataset, the mean and median scores are 2.65 and 2, respectively. In addition, the in-house dataset has more co-existing benign abnormalities as more patients with benign biopsy or prior surgery return for mammography. These observations are in line with other institutes' finding that the relative percentage of early stage cancer cases from mammography diagnosis has more than tripled since 2002. This indicates that currently available public open datasets may be inadequate to represent the mammography seen in today's clinical practice. It is necessary to build an updated mammography database that contains sufficient pathological heterogeneity of breast cancer and coexisting benign abnormalities that reflect the cases seen in current practice.