Indexing Dynamic Collective Constructs Using Computer-Aided Text Analysis: Construct Validity Evidence and Illustrations Featuring Team Processes

John E. Mathieu, Mikhail A. Wolfson, Semin Park, Margaret M. Luciano, Wendy L. Bedwell-Torres, P. Scott Ramsay, Elizabeth A. Klock, Scott I. Tannenbaum

Research output: Contribution to journalArticlepeer-review

16 Scopus citations

Abstract

Organizational processes have been widely recognized as both multilevel and dynamic, yet traditional methods of measurements limit our ability to model and understand such phenomena. Featuring a popular model of team processes advanced by Marks et al. (2001), we illustrate a method to use individuals’ communications as construct valid unobtrusive measures of collective constructs occurring over time. Thus, the purpose of this investigation is to develop computer-aided text analysis (CATA) techniques that can score members’ communications into valid team process measures. We apply a deductive content validitybased method to construct CATA dictionaries for Marks et al.’s dimensions. We then demonstrate their convergent validity with subject matter experts’ (SMEs) hand-coded team communications and different SMEs’ behaviorally anchored rating scales based on video recordings of team interactions, using multitrait– multimethod analyses in two samples. Using a third sample of paramedics performing a high-fidelity mass casualty incident exercise, we further demonstrate the convergent validity of the CATA and SME scorings of communications. We then model the relationships among processes across episodes using all three samples. Next, we test criterion-related validity using a longitudinal dual-discontinuous change growth modeling design featuring the paramedic CATA-scored team processes as related to a dynamic performance criterion. Finally, we integrate behavioral data from wearable sensor badges to illustrate how CATA can be scored at the individual level and then leveraged to model dynamic networks of team interactions. Implications, limitations, directions for the future research, and guidelines for the application of these techniques to other collective constructs are discussed.

Original languageEnglish
Pages (from-to)533-559
Number of pages27
JournalJournal of Applied Psychology
Volume107
Issue number4
DOIs
StatePublished - Sep 6 2021

Bibliographical note

Publisher Copyright:
© 2021. American Psychological Association

Keywords

  • Cata
  • Construct validity
  • Multitrait–multimethod
  • Team process

ASJC Scopus subject areas

  • Applied Psychology

Fingerprint

Dive into the research topics of 'Indexing Dynamic Collective Constructs Using Computer-Aided Text Analysis: Construct Validity Evidence and Illustrations Featuring Team Processes'. Together they form a unique fingerprint.

Cite this