Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models

arXiv:2402.11723v1 Announce Type: cross
Abstract: Advances in language modeling have paved the way for novel human-AI co-writing experiences. This paper explores how varying levels of scaffolding from large language models (LLMs) shape the co-writing process. Employing a within-subjects field experiment with a Latin square design, we asked participants (N=131) to respond to argumentative writing prompts under three randomly sequenced conditions: no AI assistance (control), next-sentence suggestions (low scaffolding), and next-paragraph suggestions (high scaffolding). Our findings reveal a U-shaped impact of …

abstract advances ai collaboration arxiv collaboration cs.cl cs.hc design experiment human language language models large language large language models llms modeling novel paper process square type writing