LoSA: Long-Short-range Adapter for Scaling End-to-End Temporal Action Localization

arXiv:2404.01282v1 Announce Type: new
Abstract: Temporal Action Localization (TAL) involves localizing and classifying action snippets in an untrimmed video. The emergence of large video foundation models has led RGB-only video backbones to outperform previous methods needing both RGB and optical flow modalities. Leveraging these large models is often limited to training only the TAL head due to the prohibitively large GPU memory required to adapt the video backbone for TAL. To overcome this limitation, we introduce LoSA, the first memory-and-parameter-efficient …

abstract adapter arxiv cs.cv emergence flow foundation large models localization optical optical flow scaling temporal training type video