ChatMusician: Understanding and Generating Music Intrinsically with LLM

arXiv:2402.16153v1 Announce Type: cross
Abstract: While Large Language Models (LLMs) demonstrate impressive capabilities in text generation, we find that their ability has yet to be generalized to music, humanity's creative language. We introduce ChatMusician, an open-source LLM that integrates intrinsic musical abilities. It is based on continual pre-training and finetuning LLaMA2 on a text-compatible music representation, ABC notation, and the music is treated as a second language. ChatMusician can understand and generate music with a pure text tokenizer without any …

abstract arxiv capabilities continual creative cs.ai cs.cl cs.lg cs.mm cs.sd eess.as finetuning generalized humanity intrinsic language language models large language large language models llama2 llm llms music pre-training text text generation training type understanding