Framework-Based Qualitative Analysis of Free Responses of Large Language Models: Algorithmic Fidelity

Today, using Large-scale generative Language Models (LLMs) it is possible to simulate free responses to interview questions like those traditionally analyzed using qualitative research methods. Qualitative methodology encompasses a broad family of techniques involving manual analysis of open-ended interviews or conversations conducted freely in natural language. Here we consider whether artificial "silicon participants" generated by LLMs may be productively studied using qualitative methods aiming to produce insights that could generalize to real human populations. The key concept in our analysis …

analysis conversations cs.ai cs.cl family fidelity framework free generative interview interview questions interviews language language models large language large language models llms methodology natural natural language questions research responses scale