Mitigating Reversal Curse via Semantic-aware Permutation Training

arXiv:2403.00758v1 Announce Type: cross
Abstract: While large language models (LLMs) have achieved impressive performance across diverse tasks, recent studies showcase that causal LLMs suffer from the "reversal curse". It is a typical example that the model knows "A's father is B", but is unable to reason "B's child is A". This limitation poses a challenge to the advancement of artificial general intelligence (AGI), as it suggests a gap in the models' ability to comprehend and apply bidirectional reasoning. In this …

abstract arxiv child cs.ai cs.cl cs.lg diverse example language language models large language large language models llms performance reason semantic studies tasks training type via