STREAM: Spatio-TempoRal Evaluation and Analysis Metric for Video Generative Models

arXiv:2403.09669v1 Announce Type: new
Abstract: Image generative models have made significant progress in generating realistic and diverse images, supported by comprehensive guidance from various evaluation metrics. However, current video generative models struggle to generate even short video clips, with limited tools that provide insights for improvements. Current video evaluation metrics are simple adaptations of image metrics by switching the embeddings with video embedding networks, which may underestimate the unique characteristics of video. Our analysis reveals that the widely used Frechet …

abstract analysis and analysis arxiv cs.ai cs.cv current diverse evaluation evaluation metrics generate generative generative models guidance however image images improvements insights metrics progress struggle temporal tools type video