ProtLLM: An Interleaved Protein-Language LLM with Protein-as-Word Pre-Training

arXiv:2403.07920v1 Announce Type: cross
Abstract: We propose ProtLLM, a versatile cross-modal large language model (LLM) for both protein-centric and protein-language tasks. ProtLLM features a unique dynamic protein mounting mechanism, enabling it to handle complex inputs where the natural language text is interspersed with an arbitrary number of proteins. Besides, we propose the protein-as-word language modeling approach to train ProtLLM. By developing a specialized protein vocabulary, we equip the model with the capability to predict not just natural language but also …

abstract arxiv cs.ai cs.cl cs.lg dynamic enabling features inputs language language model large language large language model llm modal natural natural language pre-training protein proteins q-bio.bm tasks text training type word