End-to-End Temporal Action Detection with 1B Parameters Across 1000 Frames

arXiv:2311.17241v2 Announce Type: replace
Abstract: Recently, temporal action detection (TAD) has seen significant performance improvement with end-to-end training. However, due to the memory bottleneck, only models with limited scales and limited data volumes can afford end-to-end training, which inevitably restricts TAD performance. In this paper, we reduce the memory consumption for end-to-end training, and manage to scale up the TAD backbone to 1 billion parameters and the input video to 1,536 frames, leading to significant detection performance. The key to …

arxiv cs.cv detection parameters temporal type