CodeHalu: Code Hallucinations in LLMs Driven by Execution-based Verification

arXiv:2405.00253v1 Announce Type: new
Abstract: Large Language Models (LLMs) have made significant advancements in the field of code generation, offering unprecedented support for automated programming and assisting developers. However, LLMs sometimes generate code that appears plausible but fails to meet the expected requirements or executes incorrectly. This phenomenon of hallucinations in the coding field has not been explored. To advance the community's understanding and research on code hallucinations in LLMs, we propose a definition method for these hallucinations based on …

arxiv code cs.cl cs.se hallucinations llms type verification