Communicative Function Classification Using Voice Activity Projection

Spontaneous conversation involves complex turn-taking behavior that needs to be recognized in real time for natural human-robot interaction. A promising approach to this challenge is the Voice Activity Projection (VAP) model, which continuously predicts upcoming voice activity. However, it is trained primarily on relatively clean, English conversational speech, lacks evaluation on lively conversation and does not distinguish the communicative function of the current turn. We address these gaps using the Graz Corpus of Read and Spontaneous Speech, an Austrian German dataset with fine-grained communicative function labels, where zero-shot evaluation shows that VAP predictions do not generalize well. Hypothesizing that VAP embeddings nonetheless encode rich turn-structural information, we use them as input to a causal Long Short-Term Memory model to predict five communicative function classes (i.e., change, hold, hrt (backchannel), laughter, breathing) plus an absence class. We achieve a mean macro F1-score of 57.3% (SD 3.5%) across leave-one-out cross-validation for 13 speaker pairs in this six-class setup. Since hold and change represent the central turn-taking distinction and proved hardest to classify, we further conduct an acoustic analysis, finding that duration, intensity, and voice quality features strongly affect classification outcome. These findings highlight the challenges of fine-grained communicative function prediction in natural and spontaneous speech and provide a foundation for building real-time turn-taking components in human-robot interaction systems.