Offset Unlearning for Large Language Models

arXiv:2404.11045v1 Announce Type: new
Abstract: Despite the strong capabilities of Large Language Models (LLMs) to acquire knowledge from their training corpora, the memorization of sensitive information in the corpora such as copyrighted, harmful, and private content has led to ethical and legal concerns. In response to these challenges, unlearning has emerged as a potential remedy for LLMs affected by problematic training data. However, previous unlearning techniques are either not applicable to black-box LLMs due to required access to model internal …

abstract arxiv capabilities challenges concerns cs.cl ethical information knowledge language language models large language large language models legal llms training type unlearning