Supporting Experts with a Multimodal Machine-Learning-Based Tool for Human Behavior Analysis of Conversational Videos

arXiv:2402.11145v1 Announce Type: cross
Abstract: Multimodal scene search of conversations is essential for unlocking valuable insights into social dynamics and enhancing our communication. While experts in conversational analysis have their own knowledge and skills to find key scenes, a lack of comprehensive, user-friendly tools that streamline the processing of diverse multimodal queries impedes efficiency and objectivity. To solve it, we developed Providence, a visual-programming-based tool based on design considerations derived from a formative study with experts. It enables experts to …

abstract analysis arxiv behavior behavior analysis communication conversational conversations cs.cv cs.hc cs.lg dynamics experts human insights key knowledge machine multimodal search skills social tool tools type videos