Challenging Common Paradigms in Multi-Task Learning

arXiv:2311.04698v3 Announce Type: replace
Abstract: While multi-task learning (MTL) has gained significant attention in recent years, its underlying mechanisms remain poorly understood. Recent methods did not yield consistent performance improvements over single task learning (STL) baselines, underscoring the importance of gaining more profound insights about challenges specific to MTL. In our study, we challenge paradigms in MTL in the context of STL: First, the impact of the choice of optimizer has only been mildly investigated in MTL. We show the …

abstract arxiv attention challenge challenges consistent cs.ai cs.cv cs.lg importance improvements insights multi-task learning performance stl study type