A Relational Inductive Bias for Dimensional Abstraction in Neural Networks

arXiv:2402.18426v1 Announce Type: cross
Abstract: The human cognitive system exhibits remarkable flexibility and generalization capabilities, partly due to its ability to form low-dimensional, compositional representations of the environment. In contrast, standard neural network architectures often struggle with abstract reasoning tasks, overfitting, and requiring extensive data for training. This paper investigates the impact of the relational bottleneck -- a mechanism that focuses processing on relations among inputs -- on the learning of factorized representations conducive to compositional coding and the attendant …

abstract abstraction architectures arxiv bias capabilities cognitive contrast cs.ai cs.lg data environment flexibility form human inductive low network networks neural network neural networks overfitting paper reasoning relational standard struggle tasks the environment training type