An End-to-End Natural Language Processing System for Automatically Extracting Radiation Therapy Events From Clinical Texts

Danielle S. Bitterman, Eli Goldner, Sean Finan, David Harris, Eric B. Durbin, Harry Hochheiser, Jeremy L. Warner, Raymond H. Mak, Timothy Miller, Guergana K. Savova

Research output: Contribution to journalArticlepeer-review

Abstract

Purpose: Real-world evidence for radiation therapy (RT) is limited because it is often documented only in the clinical narrative. We developed a natural language processing system for automated extraction of detailed RT events from text to support clinical phenotyping. Methods and Materials: A multi-institutional data set of 96 clinician notes, 129 North American Association of Central Cancer Registries cancer abstracts, and 270 RT prescriptions from HemOnc.org was used and divided into train, development, and test sets. Documents were annotated for RT events and associated properties: dose, fraction frequency, fraction number, date, treatment site, and boost. Named entity recognition models for properties were developed by fine-tuning BioClinicalBERT and RoBERTa transformer models. A multiclass RoBERTa-based relation extraction model was developed to link each dose mention with each property in the same event. Models were combined with symbolic rules to create a hybrid end-to-end pipeline for comprehensive RT event extraction. Results: Named entity recognition models were evaluated on the held-out test set with F1 results of 0.96, 0.88, 0.94, 0.88, 0.67, and 0.94 for dose, fraction frequency, fraction number, date, treatment site, and boost, respectively. The relation model achieved an average F1 of 0.86 when the input was gold-labeled entities. The end-to-end system F1 result was 0.81. The end-to-end system performed best on North American Association of Central Cancer Registries abstracts (average F1 0.90), which are mostly copy–paste content from clinician notes. Conclusions: We developed methods and a hybrid end-to-end system for RT event extraction, which is the first natural language processing system for this task. This system provides proof-of-concept for real-world RT data collection for research and is promising for the potential of natural language processing methods to support clinical care.

Original languageEnglish
JournalInternational Journal of Radiation Oncology Biology Physics
DOIs
StateAccepted/In press - 2023

Bibliographical note

Funding Information:
The work was supported by National Institutes of Health grants UH3CA243120 (E.G., S.F., D.H., E.B.D., H.H., J.L.W., G.K.S.), U24CA248010 (E.G., S.F., H.H., J.L.W., G.K.S.), R01LM010090 (T.M., G.K.S.), R01LM013486 (T.M., G.K.S.), 5R01GM11435 (T.M., G.K.S.), U24CA265879 (J.L.W.), HHSN261201800013I / HHSN26100001 (E.B.D.), and P30 CA177558 (E.B.D.).

Funding Information:
Disclosures: J.L.W. received funding from Brown Physicians Inc outside the submitted work; is a consultant for Westat, Melax Tech, Roche, and Flatiron all outside the submitted work; and is the deputy editor of HemOnc.org and cofounder of HemOnc.org LLC. D.S.B. is an associate editor of the Radiation Oncology section of HemOnc.org.

Publisher Copyright:
© 2023 Elsevier Inc.

ASJC Scopus subject areas

  • Radiation
  • Oncology
  • Radiology Nuclear Medicine and imaging
  • Cancer Research

Fingerprint

Dive into the research topics of 'An End-to-End Natural Language Processing System for Automatically Extracting Radiation Therapy Events From Clinical Texts'. Together they form a unique fingerprint.

Cite this