Do You See What I See? Researcher–Participant Agreement on Single-Item Measures of Emotion Regulation Behaviors in Borderline Personality Disorder

Researchers use ecological momentary assessment (EMA) to study a range of behaviors related to psychopathology. However, it is unclear whether brief measures of coping behaviors accurately capture the intended responses. In this secondary analysis of a single-case experimental design, eight individuals with borderline personality disorder (M_age = 21.57, 63% female, 63% Asian American) completed daily diary entries for 12 weeks, along with hourly EMA entries on 2 days. Participants provided qualitative descriptions of their behaviors and classified them into one of five functional categories. Independent researchers also classified each qualitative description into the same categories. Overall, agreement between participants and researchers was low, Krippendorff’s α =.47, 95% confidence interval [0.43, 0.52]. The type of emotion experienced, researcher confidence, and word count of responses affected agreement. Generating items that capture the breadth of possible behaviors, are brief enough for frequent administration, and are consistently understood by participants is an important continued challenge in EMA research.