Automated Program Repair: Emerging trends pose and expose problems for benchmarks

arXiv:2405.05455v1 Announce Type: cross
Abstract: Machine learning (ML) now pervades the field of Automated Program Repair (APR). Algorithms deploy neural machine translation and large language models (LLMs) to generate software patches, among other tasks. But, there are important differences between these applications of ML and earlier work. Evaluations and comparisons must take care to ensure that results are valid and likely to generalize. A challenge is that the most popular APR evaluation benchmarks were not designed with ML techniques in …

abstract algorithms applications arxiv automated benchmarks cs.lg cs.se deploy differences generate language language models large language large language models llms machine machine learning machine translation neural machine translation repair software tasks translation trends type work