Towards a More Rigorous Science of Blindspot Discovery in Image Models. (arXiv:2207.04104v2 [cs.LG] UPDATED)

A growing body of work studies Blindspot Discovery Methods ("BDM"s): methods
that use an image embedding to find semantically meaningful (i.e., united by a
human-understandable concept) subsets of the data where an image classifier
performs significantly worse. Motivated by observed gaps in prior work, we
introduce a new framework for evaluating BDMs, SpotCheck, that uses synthetic
image datasets to train models with known blindspots and a new BDM, PlaneSpot,
that uses a 2D image representation. We use SpotCheck to run …

2d image arxiv classifier concept data datasets discovery embedding framework human identify image image datasets prior representation science studies synthetic united work