Evaluating In-Context Learning of Libraries for Code Generation

arXiv:2311.09635v2 Announce Type: replace
Abstract: Contemporary Large Language Models (LLMs) exhibit a high degree of code generation and comprehension capability. A particularly promising area is their ability to interpret code modules from unfamiliar libraries for solving user-instructed tasks. Recent work has shown that large proprietary LLMs can learn novel library usage in-context from demonstrations. These results raise several open questions: whether demonstrations of library usage is required, whether smaller (and more open) models also possess such capabilities, etc. In this …

abstract arxiv capability code code generation context cs.cl in-context learning language language models large language large language models learn libraries library llms modules novel proprietary tasks type usage work