Coding Small Group Communication with AI: RNNs and Transformers with Context

Andrew Pilny; Joseph Bonito; Aaron Schecter

doi:10.1177/10464964251314197

Coding Small Group Communication with AI: RNNs and Transformers with Context

Andrew Pilny, Joseph Bonito, Aaron Schecter

Research output: Contribution to journal › Article › peer-review

1 Scopus citations

Abstract

This study compares the performance of recurrent neural networks (RNNs) and transformer-based models (DistilBERT) in classifying utterances as dialogue acts. The results show that transformers consistently outperform RNNs, highlighting their usefulness in coding small group interaction. Furthermore, the study explores the impact of incorporating context, in the form of preceding and following utterances. The findings reveal that adding context leads to modest improvements in model performance. Moreover, in some cases, adding context can lead to a slight decrease in performance. The study discusses the implications of these findings for small group researchers employing AI models for text classification tasks.

Original language	English
Pages (from-to)	864-893
Number of pages	30
Journal	Small Group Research
Volume	56
Issue number	5
DOIs	https://doi.org/10.1177/10464964251314197
State	Published - 2025

Bibliographical note

Publisher Copyright:
© The Author(s) 2025.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research used the Delta advanced computing and data resource which is supported by the National Science Foundation (award OAC 2005572) and the State of Illinois. Delta is a joint effort of the University of Illinois Urbana-Champaign and its National Center for Supercomputing Applications.

Funders	Funder number
University of Illinois, Urbana-Champaign
National Science Foundation Arctic Social Science Program	OAC 2005572

Keywords

communication
content analysis
interaction analysis
meetings

ASJC Scopus subject areas

Social Psychology
Applied Psychology

Access to Document

10.1177/10464964251314197

Cite this

@article{7601dba160be4cd694f925c75d0ca500,

title = "Coding Small Group Communication with AI: RNNs and Transformers with Context",

abstract = "This study compares the performance of recurrent neural networks (RNNs) and transformer-based models (DistilBERT) in classifying utterances as dialogue acts. The results show that transformers consistently outperform RNNs, highlighting their usefulness in coding small group interaction. Furthermore, the study explores the impact of incorporating context, in the form of preceding and following utterances. The findings reveal that adding context leads to modest improvements in model performance. Moreover, in some cases, adding context can lead to a slight decrease in performance. The study discusses the implications of these findings for small group researchers employing AI models for text classification tasks.",

keywords = "communication, content analysis, interaction analysis, meetings",

author = "Andrew Pilny and Joseph Bonito and Aaron Schecter",

note = "Publisher Copyright: {\textcopyright} The Author(s) 2025.",

year = "2025",

doi = "10.1177/10464964251314197",

language = "English",

volume = "56",

pages = "864--893",

journal = "Small Group Research",

issn = "1046-4964",

publisher = "SAGE Publications Inc.",

number = "5",

}

TY - JOUR

T1 - Coding Small Group Communication with AI

T2 - RNNs and Transformers with Context

AU - Pilny, Andrew

AU - Bonito, Joseph

AU - Schecter, Aaron

N1 - Publisher Copyright: © The Author(s) 2025.

PY - 2025

Y1 - 2025

N2 - This study compares the performance of recurrent neural networks (RNNs) and transformer-based models (DistilBERT) in classifying utterances as dialogue acts. The results show that transformers consistently outperform RNNs, highlighting their usefulness in coding small group interaction. Furthermore, the study explores the impact of incorporating context, in the form of preceding and following utterances. The findings reveal that adding context leads to modest improvements in model performance. Moreover, in some cases, adding context can lead to a slight decrease in performance. The study discusses the implications of these findings for small group researchers employing AI models for text classification tasks.

AB - This study compares the performance of recurrent neural networks (RNNs) and transformer-based models (DistilBERT) in classifying utterances as dialogue acts. The results show that transformers consistently outperform RNNs, highlighting their usefulness in coding small group interaction. Furthermore, the study explores the impact of incorporating context, in the form of preceding and following utterances. The findings reveal that adding context leads to modest improvements in model performance. Moreover, in some cases, adding context can lead to a slight decrease in performance. The study discusses the implications of these findings for small group researchers employing AI models for text classification tasks.

KW - communication

KW - content analysis

KW - interaction analysis

KW - meetings

UR - http://www.scopus.com/inward/record.url?scp=85215696368&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85215696368&partnerID=8YFLogxK

U2 - 10.1177/10464964251314197

DO - 10.1177/10464964251314197

M3 - Article

AN - SCOPUS:85215696368

SN - 1046-4964

VL - 56

SP - 864

EP - 893

JO - Small Group Research

JF - Small Group Research

IS - 5

ER -

Coding Small Group Communication with AI: RNNs and Transformers with Context

Abstract

Bibliographical note

Funding

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this