Rethinking Machine Unlearning for Large Language Models

arXiv:2402.08787v1 Announce Type: new
Abstract: We explore machine unlearning (MU) in the domain of large language models (LLMs), referred to as LLM unlearning. This initiative aims to eliminate undesirable data influence (e.g., sensitive or illegal information) and the associated model capabilities, while maintaining the integrity of essential knowledge generation and not affecting causally unrelated information. We envision LLM unlearning becoming a pivotal element in the life-cycle management of LLMs, potentially standing as an essential foundation for developing generative AI that …

abstract arxiv capabilities cs.cl cs.lg data domain explore influence information integrity knowledge language language models large language large language models llm llms machine type unlearning