An Effective-Efficient Approach for Dense Multi-Label Action Detection

arXiv:2406.06187v1 Announce Type: new
Abstract: Unlike the sparse label action detection task, where a single action occurs in each timestamp of a video, in a dense multi-label scenario, actions can overlap. To address this challenging task, it is necessary to simultaneously learn (i) temporal dependencies and (ii) co-occurrence action relationships. Recent approaches model temporal information by extracting multi-scale features through hierarchical transformer-based networks. However, the self-attention mechanism in transformers inherently loses temporal positional information. We argue that combining this with …

abstract action arxiv cs.cv dependencies detection learn relationships temporal type video