Human Action Co-occurrence in Lifestyle Vlogs using Graph Link Prediction

arXiv:2309.06219v3 Announce Type: replace-cross
Abstract: We introduce the task of automatic human action co-occurrence identification, i.e., determine whether two human actions can co-occur in the same interval of time. We create and make publicly available the ACE (Action Co-occurrencE) dataset, consisting of a large graph of ~12k co-occurring pairs of visual actions and their corresponding video clips. We describe graph link prediction models that leverage visual and textual information to automatically infer if two actions are co-occurring. We show that …

action arxiv cs.cl cs.cv cs.cy cs.ir graph human lifestyle link prediction prediction replace type