Knowledge Sanitization of Large Language Models

arXiv:2309.11852v2 Announce Type: replace
Abstract: We explore a knowledge sanitization approach to mitigate the privacy concerns associated with large language models (LLMs). LLMs trained on a large corpus of Web data can memorize and potentially reveal sensitive or confidential information, raising critical security concerns. Our technique efficiently fine-tunes these models using the Low-Rank Adaptation (LoRA) method, prompting them to generate harmless responses such as ``I don't know'' when queried about specific information. Experimental results in a closed-book question-answering task show …

abstract arxiv concerns cs.cl data explore information knowledge language language models large language large language models llms low low-rank adaptation privacy security security concerns type web