Speech and Language Processing for Assessing Child–Adult Interaction Based on Diarization and Location

dc.contributor.ORCID0000-0003-1382-9929 (Hansen, JHL)
dc.contributor.authorHansen, John H. L.
dc.contributor.authorNajafian, Maryam
dc.contributor.authorLileikyte, Rasa
dc.contributor.authorIrvin, D.
dc.contributor.authorRous, B.
dc.contributor.utdAuthorHansen, John H. L.
dc.contributor.utdAuthorNajafian, Maryam
dc.contributor.utdAuthorLileikyte, Rasa
dc.descriptionDue to copyright restrictions and/or publisher's policy full text access from Treasures at UT Dallas is limited to current UTD affiliates (use the provided Link to Article).
dc.description.abstractUnderstanding and assessing child verbal communication patterns is critical in facilitating effective language development. Typically speaker diarization is performed to explore children’s verbal engagement. Understanding which activity areas stimulate verbal communication can help promote more efficient language development. In this study, we present a two-stage children vocal engagement prediction system that consists of (1) a near to real-time, noise robust system that measures the duration of child-to-adult and child-to-child conversations, and tracks the number of conversational turn-takings, (2) a novel child location tracking strategy, that determines in which activity areas a child spends most/least of their time. A proposed child–adult turn-taking solution relies exclusively on vocal cues observed during the interaction between a child and other children, and/or classroom teachers. By employing a threshold optimized speech activity detection using a linear combination of voicing measures, it is possible to achieve effective speech/non-speech segment detection prior to conversion assessment. This TO-COMBO-SAD reduces classification error rates for adult-child audio by 21.34% and 27.3% compared to a baseline i-Vector and standard Bayesian Information Criterion diarization systems, respectively. In addition, this study presents a unique location tracking system adult-child that helps determine the quantity of child–adult communication in specific activity areas, and which activities stimulate voice communication engagement in a child–adult education space. We observe that our proposed location tracking solution offers unique opportunities to assess speech and language interaction for children, and quantify the location context which would contribute to improve verbal communication. © 2019, Springer Science+Business Media, LLC, part of Springer Nature.
dc.description.departmentErik Jonsson School of Engineering and Computer Science
dc.description.departmentCenter for Robust Speech Systems
dc.identifier.bibliographicCitationHansen, J. H. L., M. Najafian, R. Lileikyte, D. Irvin, et al. 2019. "Speech and language processing for assessing child–adult interaction based on diarization and location." International Journal of Speech Technology 22: 697-709, doi: 10.1007/s10772-019-09590-0
dc.publisherSpringer New York LLC
dc.rights©2019 Springer Science+Business Media, LLC.
dc.source.journalInternational Journal of Speech Technology
dc.subjectOral communication
dc.titleSpeech and Language Processing for Assessing Child–Adult Interaction Based on Diarization and Location


Original bundle

Now showing 1 - 1 of 1
Thumbnail Image
183.1 KB
Adobe Portable Document Format
Link to Article