Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learning

arXiv:2404.09403v1 Announce Type: new
Abstract: Integrating and processing information from various sources or modalities are critical for obtaining a comprehensive and accurate perception of the real world in autonomous systems and cyber-physical systems. Drawing inspiration from neuroscience, we develop the Information-Theoretic Hierarchical Perception (ITHP) model, which utilizes the concept of information bottleneck. Different from most traditional fusion models that incorporate all modalities identically in neural networks, our model designates a prime modality and regards the remaining modalities as detectors in …

abstract arxiv autonomous autonomous systems concept cs.lg cyber hierarchical information inspiration multimodal multimodal learning neuro neuroscience perception processing systems the information type world