Adapting Short-Term Transformers for Action Detection in Untrimmed Videos

arXiv:2312.01897v2 Announce Type: replace
Abstract: Vision Transformer (ViT) has shown high potential in video recognition, owing to its flexible design, adaptable self-attention mechanisms, and the efficacy of masked pre-training. Yet, it remains unclear how to adapt these pre-trained short-term ViTs for temporal action detection (TAD) in untrimmed videos. The existing works treat them as off-the-shelf feature extractors for each short-trimmed snippet without capturing the fine-grained relation among different snippets in a broader temporal context. To mitigate this issue, this paper …

abstract adapt arxiv attention attention mechanisms cs.cv design detection pre-training recognition self-attention temporal training transformer transformers type video videos vision vit