Describing Differences in Image Sets with Natural Language

arXiv:2312.02974v2 Announce Type: replace-cross
Abstract: How do two sets of images differ? Discerning set-level differences is crucial for understanding model behaviors and analyzing datasets, yet manually sifting through thousands of images is impractical. To aid in this discovery process, we explore the task of automatically describing the differences between two $\textbf{sets}$ of images, which we term Set Difference Captioning. This task takes in image sets $D_A$ and $D_B$, and outputs a description that is more often true on $D_A$ than …

abstract arxiv cs.cl cs.cv cs.cy cs.lg datasets differences discovery explore image images language natural natural language process set through type understanding