Complementary Random Masking for RGB-Thermal Semantic Segmentation

arXiv:2303.17386v2 Announce Type: replace
Abstract: RGB-thermal semantic segmentation is one potential solution to achieve reliable semantic scene understanding in adverse weather and lighting conditions. However, the previous studies mostly focus on designing a multi-modal fusion module without consideration of the nature of multi-modality inputs. Therefore, the networks easily become over-reliant on a single modality, making it difficult to learn complementary and meaningful representations for each modality. This paper proposes 1) a complementary random masking strategy of RGB-T images and 2) …

arxiv cs.ai cs.cv cs.ro masking random segmentation semantic type