Training Human-AI Teams

Abstract: AI systems, including large language models (LLMs), are augmenting the capabilities of humans in settings such as healthcare and programming. I first showcase preliminary evidence of the productivity gains of LLMs in programming tasks. To understand opportunities for model improvements, I developed a taxonomy to understand how programmers interact with a popular LLM extension, GitHub Copilot. This taxonomy reveals how much programming behavior has changed; particularly, time spent verifying LLM suggestions dominates other activities. I then show how we …

abstract ai systems capabilities copilot evidence extension github healthcare human humans improvements language language models large language large language models llm llms opportunities popular productivity programmers programming systems tasks taxonomy teams training