Understanding Privacy Risks of Embeddings Induced by Large Language Models

arXiv:2404.16587v1 Announce Type: new
Abstract: Large language models (LLMs) show early signs of artificial general intelligence but struggle with hallucinations. One promising solution to mitigate these hallucinations is to store external knowledge as embeddings, aiding LLMs in retrieval-augmented generation. However, such a solution risks compromising privacy, as recent studies experimentally showed that the original text can be partially reconstructed from text embeddings by pre-trained language models. The significant advantage of LLMs over traditional pre-trained models may exacerbate these concerns. To …

abstract artificial artificial general intelligence arxiv cs.ai cs.cl embeddings general hallucinations however intelligence knowledge language language models large language large language models llms privacy retrieval retrieval-augmented risks show solution store struggle studies type understanding