all AI news
Do LLMs dream of elephants (when told not to)? Latent concept association and associative memory in transformers
June 27, 2024, 4:42 a.m. | Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam
cs.CL updates on arXiv.org arxiv.org
Abstract: Large Language Models (LLMs) have the capacity to store and recall facts. Through experimentation with open-source models, we observe that this ability to retrieve facts can be easily manipulated by changing contexts, even without altering their factual meanings. These findings highlight that LLMs might behave like an associative memory model where certain tokens in the contexts serve as clues to retrieving facts. We mathematically explore this property by studying how transformers, the building blocks of …
abstract arxiv association capacity concept cs.cl cs.lg elephants experimentation facts language language models large language large language models llms memory observe open-source models recall stat.ml store through transformers type
More from arxiv.org / cs.CL updates on arXiv.org
ReFT: Reasoning with Reinforced Fine-Tuning
1 day, 20 hours ago |
arxiv.org
Exploring Defeasibility in Causal Reasoning
1 day, 20 hours ago |
arxiv.org
Jobs in AI, ML, Big Data
Quantitative Researcher – Algorithmic Research
@ Man Group | GB London Riverbank House
Software Engineering Expert
@ Sanofi | Budapest
Senior Bioinformatics Scientist
@ Illumina | US - Bay Area - Foster City
Senior Engineer - Generative AI Product Engineering (Remote-Eligible)
@ Capital One | McLean, VA
Graduate Assistant - Bioinformatics
@ University of Arkansas System | University of Arkansas at Little Rock
Senior AI-HPC Cluster Engineer
@ NVIDIA | US, CA, Santa Clara