MoPEFT: A Mixture-of-PEFTs for the Segment Anything Model

arXiv:2405.00293v1 Announce Type: new
Abstract: The emergence of foundation models, such as the Segment Anything Model (SAM), has sparked interest in Parameter-Efficient Fine-Tuning (PEFT) methods that tailor these large models to application domains outside their training data. However, different PEFT techniques modify the representation of a model differently, making it a non-trivial task to select the most appropriate method for the domain of interest. We propose a new framework, Mixture-of-PEFTs methods (MoPEFT), that is inspired by traditional Mixture-of-Experts (MoE) methodologies …

abstract application arxiv cs.cv data domains emergence fine-tuning foundation however large models making peft representation sam segment segment anything segment anything model training training data type