Entity-aware and Motion-aware Transformers for Language-driven Action Localization in Videos. (arXiv:2205.05854v1 [cs.CV])

Language-driven action localization in videos is a challenging task that
involves not only visual-linguistic matching but also action boundary
prediction. Recent progress has been achieved through aligning language query
to video segments, but estimating precise boundaries is still under-explored.
In this paper, we propose entity-aware and motion-aware Transformers that
progressively localizes actions in videos by first coarsely locating clips with
entity queries and then finely predicting exact boundaries in a shrunken
temporal region with motion queries. The entity-aware Transformer incorporates …

arxiv cv language localization transformers videos