TY - JOUR
T1 - An End-to-End Natural Language Processing System for Automatically Extracting Radiation Therapy Events From Clinical Texts
AU - Bitterman, Danielle S.
AU - Goldner, Eli
AU - Finan, Sean
AU - Harris, David
AU - Durbin, Eric B.
AU - Hochheiser, Harry
AU - Warner, Jeremy L.
AU - Mak, Raymond H.
AU - Miller, Timothy
AU - Savova, Guergana K.
N1 - Publisher Copyright:
© 2023 Elsevier Inc.
PY - 2023/9/1
Y1 - 2023/9/1
N2 - Purpose: Real-world evidence for radiation therapy (RT) is limited because it is often documented only in the clinical narrative. We developed a natural language processing system for automated extraction of detailed RT events from text to support clinical phenotyping. Methods and Materials: A multi-institutional data set of 96 clinician notes, 129 North American Association of Central Cancer Registries cancer abstracts, and 270 RT prescriptions from HemOnc.org was used and divided into train, development, and test sets. Documents were annotated for RT events and associated properties: dose, fraction frequency, fraction number, date, treatment site, and boost. Named entity recognition models for properties were developed by fine-tuning BioClinicalBERT and RoBERTa transformer models. A multiclass RoBERTa-based relation extraction model was developed to link each dose mention with each property in the same event. Models were combined with symbolic rules to create a hybrid end-to-end pipeline for comprehensive RT event extraction. Results: Named entity recognition models were evaluated on the held-out test set with F1 results of 0.96, 0.88, 0.94, 0.88, 0.67, and 0.94 for dose, fraction frequency, fraction number, date, treatment site, and boost, respectively. The relation model achieved an average F1 of 0.86 when the input was gold-labeled entities. The end-to-end system F1 result was 0.81. The end-to-end system performed best on North American Association of Central Cancer Registries abstracts (average F1 0.90), which are mostly copy–paste content from clinician notes. Conclusions: We developed methods and a hybrid end-to-end system for RT event extraction, which is the first natural language processing system for this task. This system provides proof-of-concept for real-world RT data collection for research and is promising for the potential of natural language processing methods to support clinical care.
AB - Purpose: Real-world evidence for radiation therapy (RT) is limited because it is often documented only in the clinical narrative. We developed a natural language processing system for automated extraction of detailed RT events from text to support clinical phenotyping. Methods and Materials: A multi-institutional data set of 96 clinician notes, 129 North American Association of Central Cancer Registries cancer abstracts, and 270 RT prescriptions from HemOnc.org was used and divided into train, development, and test sets. Documents were annotated for RT events and associated properties: dose, fraction frequency, fraction number, date, treatment site, and boost. Named entity recognition models for properties were developed by fine-tuning BioClinicalBERT and RoBERTa transformer models. A multiclass RoBERTa-based relation extraction model was developed to link each dose mention with each property in the same event. Models were combined with symbolic rules to create a hybrid end-to-end pipeline for comprehensive RT event extraction. Results: Named entity recognition models were evaluated on the held-out test set with F1 results of 0.96, 0.88, 0.94, 0.88, 0.67, and 0.94 for dose, fraction frequency, fraction number, date, treatment site, and boost, respectively. The relation model achieved an average F1 of 0.86 when the input was gold-labeled entities. The end-to-end system F1 result was 0.81. The end-to-end system performed best on North American Association of Central Cancer Registries abstracts (average F1 0.90), which are mostly copy–paste content from clinician notes. Conclusions: We developed methods and a hybrid end-to-end system for RT event extraction, which is the first natural language processing system for this task. This system provides proof-of-concept for real-world RT data collection for research and is promising for the potential of natural language processing methods to support clinical care.
UR - http://www.scopus.com/inward/record.url?scp=85153183126&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85153183126&partnerID=8YFLogxK
U2 - 10.1016/j.ijrobp.2023.03.055
DO - 10.1016/j.ijrobp.2023.03.055
M3 - Article
C2 - 36990288
AN - SCOPUS:85153183126
SN - 0360-3016
VL - 117
SP - 262
EP - 273
JO - International Journal of Radiation Oncology Biology Physics
JF - International Journal of Radiation Oncology Biology Physics
IS - 1
ER -
