Emergence of Latent Binary Encoding in Deep Neural Network Classifiers

arXiv:2310.08224v3 Announce Type: replace
Abstract: We investigate the emergence of binary encoding within the latent space of deep-neural-network classifiers. Such binary encoding is induced by the integration of a linear penultimate layer, which employs during training a loss function specifically designed to compress the latent representations. As a result of a trade-off between compression and information retention, the network learns to assume only one of two possible values for each dimension in the latent space. The binary encoding is provoked …

abstract arxiv binary classifiers cs.lg deep neural network emergence encoding function integration layer linear loss network neural network space training type