Can LLMs' Tuning Methods Work in Medical Multimodal Domain?

arXiv:2403.06407v1 Announce Type: new
Abstract: While large language models (LLMs) excel in world knowledge understanding, adapting them to specific subfields requires precise adjustments. Due to the model's vast scale, traditional global fine-tuning methods for large models can be computationally expensive and impact generalization. To address this challenge, a range of innovative Parameters-Efficient Fine-Tuning (PEFT) methods have emerged and achieved remarkable success in both LLMs and Large Vision-Language Models (LVLMs). In the medical domain, fine-tuning a medical Vision-Language Pretrained (VLP) model …

abstract arxiv challenge cs.cv domain excel fine-tuning global impact knowledge language language models large language large language models large models llms medical multimodal scale them type understanding vast work world