Speech and language processing for assessing child–adult interaction based on diarization and location

John H.L. Hansen, Maryam Najafian, Rasa Lileikyte, Dwight Irvin, Beth Rous

Research output: Contribution to journalArticlepeer-review

4 Scopus citations

Abstract

Understanding and assessing child verbal communication patterns is critical in facilitating effective language development. Typically speaker diarization is performed to explore children’s verbal engagement. Understanding which activity areas stimulate verbal communication can help promote more efficient language development. In this study, we present a two-stage children vocal engagement prediction system that consists of (1) a near to real-time, noise robust system that measures the duration of child-to-adult and child-to-child conversations, and tracks the number of conversational turn-takings, (2) a novel child location tracking strategy, that determines in which activity areas a child spends most/least of their time. A proposed child–adult turn-taking solution relies exclusively on vocal cues observed during the interaction between a child and other children, and/or classroom teachers. By employing a threshold optimized speech activity detection using a linear combination of voicing measures, it is possible to achieve effective speech/non-speech segment detection prior to conversion assessment. This TO-COMBO-SAD reduces classification error rates for adult-child audio by 21.34% and 27.3% compared to a baseline i-Vector and standard Bayesian Information Criterion diarization systems, respectively. In addition, this study presents a unique location tracking system adult-child that helps determine the quantity of child–adult communication in specific activity areas, and which activities stimulate voice communication engagement in a child–adult education space. We observe that our proposed location tracking solution offers unique opportunities to assess speech and language interaction for children, and quantify the location context which would contribute to improve verbal communication.

Original languageEnglish
Pages (from-to)697-709
Number of pages13
JournalInternational Journal of Speech Technology
Volume22
Issue number3
DOIs
StatePublished - Sep 1 2019

Bibliographical note

Publisher Copyright:
© 2019, Springer Science+Business Media, LLC, part of Springer Nature.

Keywords

  • Child speech
  • I-Vector
  • Language environment monitoring
  • Speaker diarization
  • Speech activity detection

ASJC Scopus subject areas

  • Software
  • Language and Linguistics
  • Human-Computer Interaction
  • Linguistics and Language
  • Computer Vision and Pattern Recognition

Fingerprint

Dive into the research topics of 'Speech and language processing for assessing child–adult interaction based on diarization and location'. Together they form a unique fingerprint.

Cite this