PhD Studentship – Speaker Tracking in Complex Conversations

Background:
It is known that many modalities (e.g. articulation, mouth movements, eye gaze, head nods, back channels and gestures) play a role in communication success in speech-based interaction. Active speaker-tracking can currently exploit audio and facial movement in recorded conversations to track the active speaker. If you want to change emphasis to instead predict the next speaker in a more complex multispeaker environment, that’s significantly more challenging and would benefit from exploiting more non-verbal cues, just as humans do. This PhD student will focus on approaches to tracking a conversation that incorporate knowledge around the H&H theory of speech and multimodality in human communication. By gaining deeper insights into how multimodal cues are exploited in different conversation situations, this can be leveraged into new neural architectures to intelligently follow what’s happening in a conversation. This PhD research is part of a larger Research Irela...