CIFAR-10-Warehouse: Broad and More Realistic Testbeds in Model Generalization Analysis

arXiv:2310.04414v3 Announce Type: replace
Abstract: Analyzing model performance in various unseen environments is a critical research problem in the machine learning community. To study this problem, it is important to construct a testbed with out-of-distribution test sets that have broad coverage of environmental discrepancies. However, existing testbeds typically either have a small number of domains or are synthesized by image corruptions, hindering algorithm design that demonstrates real-world effectiveness. In this paper, we introduce CIFAR-10-Warehouse, consisting of 180 datasets collected by …

abstract analysis arxiv cifar-10 community construct coverage cs.cv distribution environmental environments however machine machine learning model generalization performance research study test type warehouse