Navigating Prompt Complexity for Zero-Shot Classification: A Study of Large Language Models in Computational Social Science

arXiv:2305.14310v3 Announce Type: replace
Abstract: Instruction-tuned Large Language Models (LLMs) have exhibited impressive language understanding and the capacity to generate responses that follow specific prompts. However, due to the computational demands associated with training these models, their applications often adopt a zero-shot setting. In this paper, we evaluate the zero-shot performance of two publicly accessible LLMs, ChatGPT and OpenAssistant, in the context of six Computational Social Science classification tasks, while also investigating the effects of various prompting strategies. Our experiments …

abstract applications arxiv capacity classification complexity computational cs.cl generate however instruction-tuned language language models language understanding large language large language models llms prompt prompts responses science social social science study training type understanding zero-shot