Facial Affective Behavior Analysis with Instruction Tuning

arXiv:2404.05052v1 Announce Type: new
Abstract: Facial affective behavior analysis (FABA) is crucial for understanding human mental states from images. However, traditional approaches primarily deploy models to discriminate among discrete emotion categories, and lack the fine granularity and reasoning capability for complex facial behaviors. The advent of Multi-modal Large Language Models (MLLMs) has been proven successful in general visual understanding tasks. However, directly harnessing MLLMs for FABA is challenging due to the scarcity of datasets and benchmarks, neglecting facial prior knowledge, …

abstract analysis arxiv behavior behavior analysis capability cs.cv deploy emotion however human images language language models large language large language models mllms modal multi-modal reasoning type understanding