Learning Long-form Video Prior via Generative Pre-Training

arXiv:2404.15909v1 Announce Type: new
Abstract: Concepts involved in long-form videos such as people, objects, and their interactions, can be viewed as following an implicit prior. They are notably complex and continue to pose challenges to be comprehensively learned. In recent years, generative pre-training (GPT) has exhibited versatile capacities in modeling any kind of text content even visual locations. Can this manner work for learning long-form video prior? Instead of operating on pixel space, it is efficient to employ visual locations …

arxiv cs.cv form generative pre-training prior training type via video