Evaluation of the Programming Skills of Large Language Models

arXiv:2405.14388v1 Announce Type: cross
Abstract: The advent of Large Language Models (LLM) has revolutionized the efficiency and speed with which tasks are completed, marking a significant leap in productivity through technological innovation. As these chatbots tackle increasingly complex tasks, the challenge of assessing the quality of their outputs has become paramount. This paper critically examines the output quality of two leading LLMs, OpenAI's ChatGPT and Google's Gemini AI, by comparing the quality of programming code generated in both their free …

abstract arxiv become challenge chatbots cs.cl cs.cr cs.se efficiency evaluation innovation language language models large language large language models llm productivity programming quality skills speed tasks through type