IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT

arXiv:2404.02059v2 Announce Type: replace-cross
Abstract: Multimodal foundation models are transformative in sequential recommender systems, leveraging powerful representation learning capabilities. While Parameter-efficient Fine-tuning (PEFT) is commonly used to adapt foundation models for recommendation tasks, most research prioritizes parameter efficiency, often overlooking critical factors like GPU memory efficiency and training speed. Addressing this gap, our paper introduces IISAN (Intra- and Inter-modal Side Adapted Network for Multimodal Representation), a simple plug-and-play architecture using a Decoupled PEFT structure and exploiting both intra- and inter-modal …

abstract adapt arxiv capabilities cs.cv cs.ir efficiency fine-tuning foundation gpu memory multimodal peft recommendation recommender systems representation representation learning research speed systems tasks training type