Generative Kaleidoscopic Networks

arXiv:2402.11793v1 Announce Type: new
Abstract: We discovered that the Deep ReLU networks (or Multilayer Perceptron architecture) demonstrate an 'over-generalization' phenomenon. That is, the output values for the inputs that were not seen during training are mapped close to the output range that were observed during the learning process. In other words, the MLP learns a many-to-one mapping and this effect is more prominent as we increase the number of layers or depth of the MLP. We utilize this property of …

abstract architecture arxiv cs.ai cs.lg generative inputs mapped mapping mlp networks perceptron process relu training type values words