LLM-Powered Test Case Generation for Detecting Tricky Bugs

arXiv:2404.10304v1 Announce Type: cross
Abstract: Conventional automated test generation tools struggle to generate test oracles and tricky bug-revealing test inputs. Large Language Models (LLMs) can be prompted to produce test inputs and oracles for a program directly, but the precision of the tests can be very low for complex scenarios (only 6.3% based on our experiments). To fill this gap, this paper proposes AID, which combines LLMs with differential testing to generate fault-revealing test inputs and oracles targeting plausibly correct …

abstract arxiv automated bugs case cs.lg cs.se generate inputs language language models large language large language models llm llms low precision struggle test test case tests tools type