all AI news
Detecting Conceptual Abstraction in LLMs
April 25, 2024, 5:44 p.m. | Michaela Regneri, Alhassan Abdelhalim, S\"oren Laue
cs.CL updates on arXiv.org arxiv.org
Abstract: We present a novel approach to detecting noun abstraction within a large language model (LLM). Starting from a psychologically motivated set of noun pairs in taxonomic relationships, we instantiate surface patterns indicating hypernymy and analyze the attention matrices produced by BERT. We compare the results to two sets of counterfactuals and show that we can detect hypernymy in the abstraction mechanism, which cannot solely be related to the distributional similarity of noun pairs. Our findings …
abstract abstraction analyze arxiv attention bert cs.cl cs.lg language language model large language large language model llm llms novel patterns relationships results set surface type
More from arxiv.org / cs.CL updates on arXiv.org
Jobs in AI, ML, Big Data
Artificial Intelligence – Bioinformatic Expert
@ University of Texas Medical Branch | Galveston, TX
Lead Developer (AI)
@ Cere Network | San Francisco, US
Research Engineer
@ Allora Labs | Remote
Ecosystem Manager
@ Allora Labs | Remote
Founding AI Engineer, Agents
@ Occam AI | New York
AI Engineer Intern, Agents
@ Occam AI | US