NumLLM: Numeric-Sensitive Large Language Model for Chinese Finance

arXiv:2405.00566v1 Announce Type: cross
Abstract: Recently, many works have proposed various financial large language models (FinLLMs) by pre-training from scratch or fine-tuning open-sourced LLMs on financial corpora. However, existing FinLLMs exhibit unsatisfactory performance in understanding financial text when numeric variables are involved in questions. In this paper, we propose a novel LLM, called numeric-sensitive large language model (NumLLM), for Chinese finance. We first construct a financial corpus from financial textbooks which is essential for improving numeric capability of LLMs during …

abstract arxiv chinese cs.ce cs.cl finance financial fine-tuning however language language model language models large language large language model large language models llm llms novel paper performance pre-training q-fin.gn questions scratch text training type understanding variables