Second-Order Information Matters: Revisiting Machine Unlearning for Large Language Models

arXiv:2403.10557v1 Announce Type: new
Abstract: With the rapid development of Large Language Models (LLMs), we have witnessed intense competition among the major LLM products like ChatGPT, LLaMa, and Gemini. However, various issues (e.g. privacy leakage and copyright violation) of the training corpus still remain underexplored. For example, the Times sued OpenAI and Microsoft for infringing on its copyrights by using millions of its articles for training. From the perspective of LLM practitioners, handling such unintended privacy violations can be challenging. …

abstract arxiv chatgpt competition copyright cs.ai cs.cl cs.lg development example gemini however information language language models large language large language models llama llm llms machine major privacy products the times training type unlearning