InfiCoder-Eval: Systematically Evaluating the Question-Answering Capabilities of Code Large Language Models

arXiv:2404.07940v1 Announce Type: cross
Abstract: Large Language Models for understanding and generating code (code LLMs) have witnessed tremendous progress in recent years. With the rapid development of code LLMs, many popular evaluation benchmarks, such as HumanEval, DS-1000, and MBPP, have emerged to measure the performance of code LLMs with a particular focus on code generation tasks. However, they are insufficient to cover the full range of expected capabilities of code LLMs, which span beyond code generation to answering diverse coding-related …

arxiv capabilities code cs.lg cs.se language language models large language large language models question type