Can Large Language Models (or Humans) Distill Text?

arXiv:2403.16584v1 Announce Type: new
Abstract: We investigate the potential of large language models (LLMs) to distill text: to remove the textual traces of an undesired forbidden variable. We employ a range of LLMs with varying architectures and training approaches to distill text by identifying and removing information about the target variable while preserving other relevant signals. Our findings shed light on the strengths and limitations of LLMs in addressing the distillation and provide insights into the strategies for leveraging these …

abstract architectures arxiv cs.cl humans information language language models large language large language models llms text textual traces training type