Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena

arXiv:2403.06965v1 Announce Type: new
Abstract: Argument Structure Constructions (ASCs) are one of the most well-studied construction groups, providing a unique opportunity to demonstrate the usefulness of Construction Grammar (CxG). For example, the caused-motion construction (CMC, ``She sneezed the foam off her cappuccino'') demonstrates that constructions must carry meaning, otherwise the fact that ``sneeze'' in this context causes movement cannot be explained. We form the hypothesis that this remains challenging even for state-of-the-art Large Language Models (LLMs), for which we devise …

abstract arxiv cappuccino construction cs.cl evaluation example grammar her human hybrid llm meaning type