Directed Domain Fine-Tuning: Tailoring Separate Modalities for Specific Training Tasks

arXiv:2406.16346v1 Announce Type: new
Abstract: Large language models (LLMs) and large visual language models (LVLMs) have been at the forefront of the artificial intelligence field, particularly for tasks like text generation, video captioning, and question-answering. Typically, it is more applicable to train these models on broader knowledge bases or datasets to increase generalizability, learn relationships between topics, and recognize patterns. Instead, we propose to provide instructional datasets specific to the task of each modality within a distinct domain and then …

abstract artificial artificial intelligence arxiv captioning cs.ai cs.cv datasets domain fine-tuning intelligence knowledge language language models large language large language models llms question tasks text text generation train training tuning type video visual visual language models