SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models

arXiv:2312.09818v2 Announce Type: replace
Abstract: Despite the recent advances of the artificial intelligence, building social intelligence remains a challenge. Among social signals, laughter is one of the distinctive expressions that occurs during social interactions between humans. In this work, we tackle a new challenge for machines to understand the rationale behind laughter in video, Video Laugh Reasoning. We introduce this new task to explain why people laugh in a particular video and a dataset for this task. Our proposed dataset, …

arxiv cs.ai cs.cl dataset language language models multimodal smile type understanding video