Does SAM dream of EIG? Characterizing Interactive Segmenter Performance using Expected Information Gain

arXiv:2404.16155v1 Announce Type: cross
Abstract: We introduce an assessment procedure for interactive segmentation models. Based on concepts from Bayesian Experimental Design, the procedure measures a model's understanding of point prompts and their correspondence with the desired segmentation mask. We show that Oracle Dice index measurements are insensitive or even misleading in measuring this property. We demonstrate the use of the proposed procedure on three interactive segmentation models and subsets of two large image segmentation datasets.

abstract arxiv assessment bayesian concepts cs.cv cs.it cs.lg design dice experimental index information interactive math.it oracle performance prompts sam segmentation show type understanding