Structured Video Tokens @ Ego4D PNR Temporal Localization Challenge 2022. (arXiv:2206.07689v1 [cs.CV])

This technical report describes the SViT approach for the Ego4D Point of No
Return (PNR) Temporal Localization Challenge. We propose a learning framework
StructureViT (SViT for short), which demonstrates how utilizing the structure
of a small number of images only available during training can improve a video
model. SViT relies on two key insights. First, as both images and videos
contain structured information, we enrich a transformer model with a set of
\emph{object tokens} that can be used across images …

arxiv challenge cv localization temporal tokens video