Contextual Emotion Recognition using Large Vision Language Models

arXiv:2405.08992v1 Announce Type: new
Abstract: "How does the person in the bounding box feel?" Achieving human-level recognition of the apparent emotion of a person in real world situations remains an unsolved task in computer vision. Facial expressions are not enough: body pose, contextual knowledge, and commonsense reasoning all contribute to how humans perform this emotional theory of mind task. In this paper, we examine two major approaches enabled by recent large vision language models: 1) image captioning followed by a …

abstract arxiv box commonsense computer computer vision cs.cv emotion human humans knowledge language language models person reasoning recognition type unsolved vision world