Test-time Batch Normalization. (arXiv:2205.10210v1 [cs.LG])

Deep neural networks often suffer the data distribution shift between
training and testing, and the batch statistics are observed to reflect the
shift. In this paper, targeting of alleviating distribution shift in test time,
we revisit the batch normalization (BN) in the training process and reveals two
key insights benefiting test-time optimization: $(i)$ preserving the same
gradient backpropagation form as training, and $(ii)$ using dataset-level
statistics for robust optimization and inference. Based on the two insights, we
propose a novel …

arxiv normalization test time