Unleashing the Power of Meta-tuning for Few-shot Generalization Through Sparse Interpolated Experts

arXiv:2403.08477v1 Announce Type: cross
Abstract: Conventional wisdom suggests parameter-efficient fine-tuning of foundation models as the state-of-the-art method for transfer learning in vision, replacing the rich literature of alternatives such as meta-learning. In trying to harness the best of both worlds, meta-tuning introduces a subsequent optimization stage of foundation models but has so far only shown limited success and crucially tends to underperform on out-of-domain (OOD) tasks. In this paper, we introduce Sparse MetA-Tuning (SMAT), a method inspired by sparse mixture-of-experts …

abstract art arxiv best of cs.cv cs.lg experts few-shot fine-tuning foundation harness literature meta meta-learning optimization power stage state through transfer transfer learning type vision