A Survey of Video Datasets for Grounded Event Understanding

arXiv:2406.09646v1 Announce Type: new
Abstract: While existing video benchmarks largely consider specialized downstream tasks like retrieval or question-answering (QA), contemporary multimodal AI systems must be capable of well-rounded common-sense reasoning akin to human visual understanding. A critical component of human temporal-visual perception is our ability to identify and cognitively model "things happening", or events. Historically, video benchmark tasks have implicitly tested for this ability (e.g., video captioning, in which models describe visual events with natural language), but they do not …

abstract ai systems arxiv benchmarks cs.ai cs.cv datasets event human identify multimodal multimodal ai perception question reasoning retrieval sense survey systems tasks temporal things type understanding video visual while