AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?

arXiv:2307.16368v3 Announce Type: replace
Abstract: Can we better anticipate an actor's future actions (e.g. mix eggs) by knowing what commonly happens after his/her current action (e.g. crack eggs)? What if we also know the longer-term goal of the actor (e.g. making egg fried rice)? The long-term action anticipation (LTA) task aims to predict an actor's future behavior from video observations in the form of verb and noun sequences, and it is crucial for human-machine interaction. We propose to formulate the …

arxiv cs.cv language language models large language large language models long-term type videos