Evaluating the clinical acceptability of deep learning contours of prostate and organs-at-risk in an automated prostate treatment planning process

Jingwei Duan, Mark Bernard, Laura Downes, Brooke Willows, Xue Feng, Waleed F. Mourad, William St Clair, Quan Chen

Research output: Contribution to journalArticlepeer-review

16 Scopus citations


Background: Radiation treatment is considered an effective and the most common treatment option for prostate cancer. The treatment planning process requires accurate and precise segmentation of the prostate and organs at risk (OARs), which is laborious and time-consuming when contoured manually. Artificial intelligence (AI)-based auto-segmentation has the potential to significantly accelerate the radiation therapy treatment planning process; however, the accuracy of auto-segmentation needs to be validated before its full clinical adoption. Purpose: A commercial AI-based contouring model was trained to provide segmentation of the prostate and surrounding OARs. The segmented structures were input to a commercial auto-planning module for automated prostate treatment planning. This study comprehensively evaluates the performance of this contouring model in the automated prostate treatment planning process. Methods and materials: A 3D U-Net-based model (INTContour, Carina AI) was trained and validated on 84 computed tomography (CT) scans and tested on an additional 23 CT scans from patients treated in our local institution. Prostate and OARs contours generated by the AI model (AI contour) were geometrically evaluated against reference contours. The prostate contours were further evaluated against AI, reference, and two additional observer contours for comparison using inter-observer variation (IOV) and 3D boundaries discrepancy analyses. A blinded evaluation was introduced to assess subjectively the clinical acceptability of the AI contours. Finally, treatment plans were created from an automated prostate planning workflow using the AI contours and were evaluated for their clinical acceptability following the Radiation Therapy Oncology Group-0815 protocol. Results: The AI contours demonstrated good geometric accuracy on OARs and prostate contours, with average Dice similarity coefficients (DSC) for bladder, rectum, femoral heads, seminal vesicles, and penile bulb of 0.93, 0.85, 0.96, 0.72, and 0.53, respectively. The DSC, 95% directed Hausdorff distance (HD95), and mean surface distance for the prostate were 0.83 ± 0.05, 6.07 ± 1.87 mm, and 2.07 ± 0.73 mm, respectively. No significant differences were found when comparing with IOV. In the double-blinded evaluation, 95.7% of the AI contours were scored as either “perfect” (34.8%) or “acceptable” (60.9%), while only one case (4.3%) was scored as “unacceptable with minor changes required.” In total, 69.6% of the AI contours were considered equal to or better than the reference contours by an independent radiation oncologist. Automated treatment plans created from the AI contours produced similar and clinically acceptable dosimetric distributions as those from plans created from reference contours. Conclusions: The investigated AI-based commercial model for prostate segmentation demonstrated good performance in clinical practice. Using this model, the implementation of an automated prostate treatment planning process is clinically feasible.

Original languageEnglish
Pages (from-to)2570-2581
Number of pages12
JournalMedical Physics
Issue number4
StatePublished - Apr 2022

Bibliographical note

Funding Information:
Xue Feng and Quan Chen are co‐founders of Carina Medical LLC. University of Kentucky receives an NIH SBIR subcontract from Carina Medical LLC and a research grant from Varian Medical System. Xue Feng, Quan Chen, and Mark Bernard are partially supported by these grants. Jingwei Duan, Laura Downes, Brooke Willows, Waleed Mourad, and William St Clair have no conflict of interest to disclose.

Funding Information:
Xue Feng and Quan Chen are partially supported by NIH R44CA254844 and NIH NCI 75N91020C00048. Xue Feng, Quan Chen, and Mark Bernard are partially supported by Research Grant from Varian Inc. William St Clair is partially supported by NIH R01CA205400. We like to thank Dr. James R. Castle for additional proofreading of this manuscript.

Publisher Copyright:
© 2022 American Association of Physicists in Medicine.


  • 3D U-Net
  • auto-segmentation
  • deep learning
  • dosimetry metrics
  • evaluation methodology
  • inter-observer variability

ASJC Scopus subject areas

  • Biophysics
  • Radiology Nuclear Medicine and imaging


Dive into the research topics of 'Evaluating the clinical acceptability of deep learning contours of prostate and organs-at-risk in an automated prostate treatment planning process'. Together they form a unique fingerprint.

Cite this