Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication

arXiv:2111.06464v2 Announce Type: replace
Abstract: Communication is compositional if complex signals can be represented as a combination of simpler subparts. In this paper, we theoretically show that inductive biases on both the training framework and the data are needed to develop a compositional communication. Moreover, we prove that compositionality spontaneously arises in the signaling games, where agents communicate over a noisy channel. We experimentally confirm that a range of noise levels, which depends on the model and the data, indeed …

abstract arxiv biases combination communication cs.ai cs.cl cs.lg data emergence framework inductive noise paper role show training type