Continuous prediction of backchannel timing for human-robot interaction

Michael Paierl; Martin Hagmueller; Barbara Schuppler

doi:10.21437/Interspeech.2025-770

Continuous prediction of backchannel timing for human-robot interaction

Michael Paierl^*
, Martin Hagmueller
, Barbara Schuppler

^*Corresponding author for this work

Institute of Signal Processing and Speech Communication (4420)

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Abstract

This paper explores the continuous prediction of backchannel timing in conversational speech, with the aim to make turntaking in human-robot interaction more natural. To assure real-time prediction, we present regressionbased models based exclusively on acoustic features that can be extracted continuously from the user’s speech. Comparing different machine learning models, we found lightGBM models to perform best with respect to accuracy (mean absolute error: approx. 130 ms) and efficiency, while meeting the real-time requirement. Our analysis
of feature importances revealed that speaking duration, intensity and fundamental frequency are among the most important predictors of backchannel timing, when extracted in the window from 275-875 ms before a backchannel in the interlocutor’s
turn. Given the strong predictive performance of our models, this work provides a foundation for implementing more natural and responsive conversational agents.

Original language	English
Title of host publication	Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2025
Pages	3020 - 3024
Number of pages	5
DOIs	https://doi.org/10.21437/Interspeech.2025-770
Publication status	Published - 2025
Event	26th Interspeech Conference 2025 - Rotterdam, Netherlands Duration: 17 Aug 2025 → 21 Aug 2025

Publication series

Name	Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
ISSN (Print)	2308-457X

Conference

Conference	26th Interspeech Conference 2025
Country/Territory	Netherlands
City	Rotterdam
Period	17/08/25 → 21/08/25

Keywords

backchannels
human-robot interaction
turn-taking
prosodic features

ASJC Scopus subject areas

Software
Signal Processing
Language and Linguistics
Modelling and Simulation
Human-Computer Interaction

Fields of Expertise

Information, Communication & Computing

Access to Document

10.21437/Interspeech.2025-770Licence: Unspecified

https://www.isca-archive.org/interspeech_2025/paierl25_interspeech.pdf

Cite this

Paierl, M., Hagmueller, M., & Schuppler, B. (2025). Continuous prediction of backchannel timing for human-robot interaction. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2025 (pp. 3020 - 3024). (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH). https://doi.org/10.21437/Interspeech.2025-770

Continuous prediction of backchannel timing for human-robot interaction. / Paierl, Michael ; Hagmueller, Martin ; Schuppler, Barbara.
Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2025. 2025. p. 3020 - 3024 (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH).

Research output: Chapter in Book/Report/Conference proceeding › Conference paper › peer-review

Paierl, M , Hagmueller, M & Schuppler, B 2025, Continuous prediction of backchannel timing for human-robot interaction. in Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2025. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, pp. 3020 - 3024, 26th Interspeech Conference 2025, Rotterdam, Netherlands, 17/08/25. https://doi.org/10.21437/Interspeech.2025-770

Paierl M , Hagmueller M , Schuppler B. Continuous prediction of backchannel timing for human-robot interaction. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2025. 2025. p. 3020 - 3024. (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH). doi: 10.21437/Interspeech.2025-770

Paierl, Michael ; Hagmueller, Martin ; Schuppler, Barbara. / Continuous prediction of backchannel timing for human-robot interaction. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2025. 2025. pp. 3020 - 3024 (Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH).

@inproceedings{460d3cea8ab1441b983fd32a2a0e74f6,

title = "Continuous prediction of backchannel timing for human-robot interaction",

abstract = "This paper explores the continuous prediction of backchannel timing in conversational speech, with the aim to make turntaking in human-robot interaction more natural. To assure real-time prediction, we present regressionbased models based exclusively on acoustic features that can be extracted continuously from the user{\textquoteright}s speech. Comparing different machine learning models, we found lightGBM models to perform best with respect to accuracy (mean absolute error: approx. 130 ms) and efficiency, while meeting the real-time requirement. Our analysis of feature importances revealed that speaking duration, intensity and fundamental frequency are among the most important predictors of backchannel timing, when extracted in the window from 275-875 ms before a backchannel in the interlocutor{\textquoteright}s turn. Given the strong predictive performance of our models, this work provides a foundation for implementing more natural and responsive conversational agents.",

keywords = "backchannels, human-robot interaction, turn-taking, prosodic features",

author = "Michael Paierl and Martin Hagmueller and Barbara Schuppler",

note = "Publisher Copyright: {\textcopyright} 2025 International Speech Communication Association. All rights reserved.; 26th Interspeech Conference 2025 ; Conference date: 17-08-2025 Through 21-08-2025",

year = "2025",

doi = "10.21437/Interspeech.2025-770",

language = "English",

series = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",

pages = "3020 -- 3024",

booktitle = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2025",

}

TY - GEN

T1 - Continuous prediction of backchannel timing for human-robot interaction

AU - Paierl, Michael

AU - Hagmueller, Martin

AU - Schuppler, Barbara

PY - 2025

Y1 - 2025

N2 - This paper explores the continuous prediction of backchannel timing in conversational speech, with the aim to make turntaking in human-robot interaction more natural. To assure real-time prediction, we present regressionbased models based exclusively on acoustic features that can be extracted continuously from the user’s speech. Comparing different machine learning models, we found lightGBM models to perform best with respect to accuracy (mean absolute error: approx. 130 ms) and efficiency, while meeting the real-time requirement. Our analysis of feature importances revealed that speaking duration, intensity and fundamental frequency are among the most important predictors of backchannel timing, when extracted in the window from 275-875 ms before a backchannel in the interlocutor’s turn. Given the strong predictive performance of our models, this work provides a foundation for implementing more natural and responsive conversational agents.

AB - This paper explores the continuous prediction of backchannel timing in conversational speech, with the aim to make turntaking in human-robot interaction more natural. To assure real-time prediction, we present regressionbased models based exclusively on acoustic features that can be extracted continuously from the user’s speech. Comparing different machine learning models, we found lightGBM models to perform best with respect to accuracy (mean absolute error: approx. 130 ms) and efficiency, while meeting the real-time requirement. Our analysis of feature importances revealed that speaking duration, intensity and fundamental frequency are among the most important predictors of backchannel timing, when extracted in the window from 275-875 ms before a backchannel in the interlocutor’s turn. Given the strong predictive performance of our models, this work provides a foundation for implementing more natural and responsive conversational agents.

KW - backchannels

KW - human-robot interaction

KW - turn-taking

KW - prosodic features

UR - https://www.scopus.com/pages/publications/105020041328

U2 - 10.21437/Interspeech.2025-770

DO - 10.21437/Interspeech.2025-770

M3 - Conference paper

AN - SCOPUS:105020041328

T3 - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

SP - 3020

EP - 3024

BT - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH 2025

T2 - 26th Interspeech Conference 2025

Y2 - 17 August 2025 through 21 August 2025

ER -

Continuous prediction of backchannel timing for human-robot interaction

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Fields of Expertise

Access to Document

Other files and links

Fingerprint

FWF - Spontansprache - Cross-layer language models for conversational speech