Video Relationship Detection Using Mixture of Experts

arXiv:2403.03994v1 Announce Type: cross
Abstract: Machine comprehension of visual information from images and videos by neural networks faces two primary challenges. Firstly, there exists a computational and inference gap in connecting vision and language, making it difficult to accurately determine which object a given agent acts on and represent it through language. Secondly, classifiers trained by a single, monolithic neural network often lack stability and generalization. To overcome these challenges, we introduce MoE-VRD, a novel approach to visual relationship detection …

abstract agent arxiv challenges computational cs.cv cs.lg detection experts gap images inference information language machine making mixture of experts networks neural networks object relationship through type video videos vision visual