[R] Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers

Training AI to understand and describe video content requires datasets which are expensive for humans to annotate manually. Now researchers from Snap, UC Merced, and the University of Trento have put together a new dataset called Panda-70M that aims to help.

This new dataset has 70 million high-res YouTube clips paired with descriptive captions. The key is they used an automated pipeline with multiple cross-modal "teacher" AI models to generate captions based on different inputs like video, subtitles, images, etc. …

captioning dataset datasets humans machinelearningnews multiple researchers snap teachers together training training ai university video videos youtube