Indexing Dynamic Collective Constructs Using Computer-Aided Text Analysis: Construct Validity Evidence and Illustrations Featuring Team Processes

John E. Mathieu, Mikhail A. Wolfson, Semin Park, Margaret M. Luciano, Wendy L. Bedwell-Torres, P. Scott Ramsay, Elizabeth A. Klock, Scott I. Tannenbaum

Research output: Contribution to journalArticlepeer-review

10 Scopus citations


Organizational processes have been widely recognized as both multilevel and dynamic, yet traditional methods of measurements limit our ability to model and understand such phenomena. Featuring a popular model of team processes advanced by Marks et al. (2001), we illustrate a method to use individuals’ communications as construct valid unobtrusive measures of collective constructs occurring over time. Thus, the purpose of this investigation is to develop computer-aided text analysis (CATA) techniques that can score members’ communications into valid team process measures. We apply a deductive content validitybased method to construct CATA dictionaries for Marks et al.’s dimensions. We then demonstrate their convergent validity with subject matter experts’ (SMEs) hand-coded team communications and different SMEs’ behaviorally anchored rating scales based on video recordings of team interactions, using multitrait– multimethod analyses in two samples. Using a third sample of paramedics performing a high-fidelity mass casualty incident exercise, we further demonstrate the convergent validity of the CATA and SME scorings of communications. We then model the relationships among processes across episodes using all three samples. Next, we test criterion-related validity using a longitudinal dual-discontinuous change growth modeling design featuring the paramedic CATA-scored team processes as related to a dynamic performance criterion. Finally, we integrate behavioral data from wearable sensor badges to illustrate how CATA can be scored at the individual level and then leveraged to model dynamic networks of team interactions. Implications, limitations, directions for the future research, and guidelines for the application of these techniques to other collective constructs are discussed.

Original languageEnglish
Pages (from-to)533-559
Number of pages27
JournalJournal of Applied Psychology
Issue number4
StatePublished - Sep 6 2021

Bibliographical note

Funding Information:
NNX09AK48G: Cohesion Optimization in NASA Spaceflight Teams Experiencing Long-duration Assignment: Training and Indices for Optimizing Performance and Gaining Astronaut Synergy Under Stress, Principle Investigator: Eduardo Salas, University of Central Florida; NASA grant # NNX11AR22G: Composing and Developing Resilient, Adaptive, and Self-Sustaining Teams for Long Duration Space Exploration, Principle Investigator Scott Tannenbaum, The Group for Organizational Effectiveness; and the Army Research Institute grant # W911NF-15-1-0014: The Development and Construct Validation of Unobtrusive Dynamic Measures of Team Processes and Emergent States, Principle Investigators Scott Tannenbaum and John Mathieu, The Group for Organizational Effectiveness, Inc. The opinions expressed herein, however, are strictly those of the authors and do not necessary represent the opinions of the sponsoring organizations.

Publisher Copyright:
© 2021. American Psychological Association


  • Cata
  • Construct validity
  • Multitrait–multimethod
  • Team process

ASJC Scopus subject areas

  • Applied Psychology


Dive into the research topics of 'Indexing Dynamic Collective Constructs Using Computer-Aided Text Analysis: Construct Validity Evidence and Illustrations Featuring Team Processes'. Together they form a unique fingerprint.

Cite this