EffiBench: Benchmarking the Efficiency of Automatically Generated Code

Code generation models have increasingly become integral to aiding software development, offering assistance in tasks such as code completion, debugging, and code translation. Although current research has thoroughly examined the correctness of code produced by code generation models, a vital aspect, i.e., the efficiency of the generated code, has often been neglected. This paper presents EffiBench, a benchmark with 1,000 efficiency-critical coding problems for assessing the efficiency of code generated by code generation models. EffiBench contains a diverse set of …

become benchmarking code code completion code generation cs.cl cs.se current debugging development efficiency generated integral research software software development tasks translation vital