Exploring the Reversal Curse and Other Deductive Logical Reasoning in BERT and GPT-Based Large Language Models

arXiv:2312.03633v2 Announce Type: replace
Abstract: The term "Reversal Curse" refers to the scenario where auto-regressive decoder large language models (LLMs), such as ChatGPT, trained on "A is B" fail to learn "B is A," assuming that B and A are distinct and can be uniquely identified from each other, demonstrating a basic failure of logical deduction. This raises a red flag in the use of GPT models for certain general tasks such as constructing knowledge graphs, considering their adherence to …

abstract arxiv auto bert chatgpt cs.ai cs.cl cs.lg decoder fail gpt language language models large language large language models learn llms reasoning replace type