Identification of Naloxone in Emergency Medical Services Data Substantially Improves by Processing Unstructured Patient Care Narratives

Objectives: Structured data fields, including medication fields involving naloxone, are routinely used to identify opioid overdoses in emergency medical services (EMS) data; between January 2021 and March 2024, there were approximately 1.2 million instances of naloxone administration in the United States. To improve the accuracy of naloxone reporting, we developed methodology for identifying naloxone administration using both structured fields and unstructured patient care narratives for events documented by EMS. Methods: We randomly sampled 30,000 records from Kentucky’s state-wide EMS database during 2019. We applied regular expressions (RegEx) capable of recognizing naloxone-related text patterns in each EMS patient’s case narrative. Additionally, we applied natural language processing (NLP) techniques to extract important contextual factors such as route and dosage from these narratives. We manually reviewed cases where the structured data and unstructured data disagreed and developed an aggregate indicator for naloxone administration using either structured or unstructured data for each patient case. Results: There were 437 (1.45%) records with structured documentation of naloxone. Our RegEx method identified 547 naloxone administrations in the narratives; after manual review, we determined RegEx yielded acceptable false positives (N = 31, 5.6%), false negatives (N = 23, 4.2%) and performance (precision = 0.94, recall = 0.93). In total, 552 patients had naloxone administered after combining indicators from both structured fields and verified results from unstructured narratives. The NLP approach also identified 246 (47.4%) records that specified route of administration and 358 (69.0%) records with dosage delivered. Conclusions: An additional 115 (26.3%) patients receiving naloxone were identified by using unstructured case narratives compared to structured data. New surveillance methods that incorporate unstructured EMS narratives are critically needed to avoid substantial underestimation of naloxone utilization and enumeration of opioid overdoses.