Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs. (arXiv:2206.04674v2 [cs.CV] UPDATED)

To build an artificial neural network like the biological intelligence
system, recent works have unified numerous tasks into a generalist model, which
can process various tasks with shared parameters and do not have any
task-specific modules. While generalist models achieve promising results on
various benchmarks, they have performance degradation on some tasks compared
with task-specialized models. In this work, we find that interference among
different tasks and modalities is the main factor to this phenomenon. To
mitigate such interference, we …

arxiv cv learning moe perceiver