Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue System. (arXiv:2205.15060v2 [cs.CL] UPDATED)

In this paper, we present Duplex Conversation, a multi-turn, multimodal
spoken dialogue system that enables telephone-based agents to interact with
customers like a human. We use the concept of full-duplex in telecommunication
to demonstrate what a human-like interactive experience should be and how to
achieve smooth turn-taking through three subtasks: user state detection,
backchannel selection, and barge-in detection. Besides, we propose
semi-supervised learning with multimodal data augmentation to leverage
unlabeled data to increase model generalization. Experimental results on three
sub-tasks …

arxiv conversation human human-like