Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networks

arXiv:2306.17630v2 Announce Type: replace
Abstract: Enhancing the generalisation abilities of neural networks (NNs) through integrating noise such as MixUp or Dropout during training has emerged as a powerful and adaptable technique. Despite the proven efficacy of noise in NN training, there is no consensus regarding which noise sources, types and placements yield maximal benefits in generalisation and confidence calibration. This study thoroughly explores diverse noise modalities to evaluate their impacts on NN's generalisation and calibration under in-distribution or out-of-distribution settings, …

abstract arxiv consensus cs.lg dropout networks neural networks nns noise study through training type