Construction of a Japanese Financial Benchmark for Large Language Models

arXiv:2403.15062v1 Announce Type: cross
Abstract: With the recent development of large language models (LLMs), models that focus on certain domains and languages have been discussed for their necessity. There is also a growing need for benchmarks to evaluate the performance of current LLMs in each domain. Therefore, in this study, we constructed a benchmark comprising multiple tasks specific to the Japanese and financial domains and performed benchmark measurements on some models. Consequently, we confirmed that GPT-4 is currently outstanding, and …

abstract arxiv benchmark benchmarks construction cs.cl current development domain domains financial focus japanese language language models languages large language large language models llms performance q-fin.cp study type