A Benchmark Study on Calibration

arXiv:2308.11838v4 Announce Type: replace
Abstract: Deep neural networks are increasingly utilized in various machine learning tasks. However, as these models grow in complexity, they often face calibration issues, despite enhanced prediction accuracy. Many studies have endeavored to improve calibration performance through the use of specific loss functions, data preprocessing and training frameworks. Yet, investigations into calibration properties have been somewhat overlooked. Our study leverages the Neural Architecture Search (NAS) search space, offering an exhaustive model architecture space for thorough calibration …

abstract accuracy arxiv benchmark complexity cs.ai cs.lg data data preprocessing face frameworks functions investigations loss machine machine learning networks neural networks performance prediction stat.ml studies study tasks through training type