BM-NAS: Bilevel Multimodal Neural Architecture Search. (arXiv:2104.09379v2 [cs.CV] UPDATED)

Deep neural networks (DNNs) have shown superior performances on various
multimodal learning problems. However, it often requires huge efforts to adapt
DNNs to individual multimodal tasks by manually engineering unimodal features
and designing multimodal feature fusion strategies. This paper proposes Bilevel
Multimodal Neural Architecture Search (BM-NAS) framework, which makes the
architecture of multimodal fusion models fully searchable via a bilevel
searching scheme. At the upper level, BM-NAS selects the inter/intra-modal
feature pairs from the pretrained unimodal backbones. At the lower …

architecture arxiv cv multimodal nas neural architecture search search