CERT: Continual Pre-Training on Sketches for Library-Oriented Code Generation. (arXiv:2206.06888v1 [cs.SE])

Code generation is a longstanding challenge, aiming to generate a code
snippet based on a natural language description. Usually, expensive text-code
paired data is essential for training a code generation model. Recently, thanks
to the success of pre-training techniques, large language models are trained on
large-scale unlabelled code corpora and perform well in code generation. In
this paper, we investigate how to leverage an unlabelled code corpus to train a
model for library-oriented code generation. Since it is a common …

arxiv code code generation continual generation library pre-training training