Deep Reference Priors: What is the best way to pretrain a model?. (arXiv:2202.00187v2 [stat.ML] UPDATED)

Web: http://arxiv.org/abs/2202.00187

What is the best way to exploit extra data -- be it unlabeled data from the
same task, or labeled data from a related task -- to learn a given task? This
paper formalizes the question using the theory of reference priors. Reference
priors are objective, uninformative Bayesian priors that maximize the mutual
information between the task and the weights of the model. Such priors enable
the task to maximally affect the Bayesian posterior, e.g., reference priors
depend upon the …

arxiv deep ml model reference