Fine-tuning with Very Large Dropout

arXiv:2403.00946v1 Announce Type: new
Abstract: It is impossible today to pretend that the practice of machine learning is compatible with the idea that training and testing data follow the same distribution. Several authors have recently used ensemble techniques to show how scenarios involving multiple data distributions are best served by representations that are both richer than those obtained by regularizing for the best in-distribution performance, and richer than those obtained under the influence of the implicit sparsity bias of common …

abstract arxiv authors cs.cv cs.lg data distribution dropout ensemble fine-tuning machine machine learning multiple practice show testing training type