An End-to-End Natural Language Processing System for Automatically Extracting Radiation Therapy Events From Clinical Texts

Danielle S. Bitterman, Eli Goldner, Sean Finan, David Harris, Eric B. Durbin, Harry Hochheiser, Jeremy L. Warner, Raymond H. Mak, Timothy Miller, Guergana K. Savova

Research output: Contribution to journalArticlepeer-review

12 Scopus citations

Abstract

Purpose: Real-world evidence for radiation therapy (RT) is limited because it is often documented only in the clinical narrative. We developed a natural language processing system for automated extraction of detailed RT events from text to support clinical phenotyping. Methods and Materials: A multi-institutional data set of 96 clinician notes, 129 North American Association of Central Cancer Registries cancer abstracts, and 270 RT prescriptions from HemOnc.org was used and divided into train, development, and test sets. Documents were annotated for RT events and associated properties: dose, fraction frequency, fraction number, date, treatment site, and boost. Named entity recognition models for properties were developed by fine-tuning BioClinicalBERT and RoBERTa transformer models. A multiclass RoBERTa-based relation extraction model was developed to link each dose mention with each property in the same event. Models were combined with symbolic rules to create a hybrid end-to-end pipeline for comprehensive RT event extraction. Results: Named entity recognition models were evaluated on the held-out test set with F1 results of 0.96, 0.88, 0.94, 0.88, 0.67, and 0.94 for dose, fraction frequency, fraction number, date, treatment site, and boost, respectively. The relation model achieved an average F1 of 0.86 when the input was gold-labeled entities. The end-to-end system F1 result was 0.81. The end-to-end system performed best on North American Association of Central Cancer Registries abstracts (average F1 0.90), which are mostly copy–paste content from clinician notes. Conclusions: We developed methods and a hybrid end-to-end system for RT event extraction, which is the first natural language processing system for this task. This system provides proof-of-concept for real-world RT data collection for research and is promising for the potential of natural language processing methods to support clinical care.

Original languageEnglish
Pages (from-to)262-273
Number of pages12
JournalInternational Journal of Radiation Oncology Biology Physics
Volume117
Issue number1
DOIs
StatePublished - Sep 1 2023

Bibliographical note

Publisher Copyright:
© 2023 Elsevier Inc.

ASJC Scopus subject areas

  • Radiation
  • Oncology
  • Radiology Nuclear Medicine and imaging
  • Cancer Research

Fingerprint

Dive into the research topics of 'An End-to-End Natural Language Processing System for Automatically Extracting Radiation Therapy Events From Clinical Texts'. Together they form a unique fingerprint.

Cite this