CodeT: Code Generation with Generated Tests. (arXiv:2207.10397v2 [cs.CL] UPDATED)

The task of generating code solutions for a given programming problem can
benefit from the use of pre-trained language models such as Codex, which can
produce multiple diverse samples. However, a major challenge for this task is
to select the most appropriate solution from the multiple samples generated by
the pre-trained language models. A natural way to evaluate the quality and
correctness of a code solution is to run it against a set of test cases, but
the manual creation …

arxiv code code generation generated tests