Medusa: Universal Feature Learning via Attentional Multitasking. (arXiv:2204.05698v1 [cs.LG])

Recent approaches to multi-task learning (MTL) have focused on modelling
connections between tasks at the decoder level. This leads to a tight coupling
between tasks, which need retraining if a new task is inserted or removed. We
argue that MTL is a stepping stone towards universal feature learning (UFL),
which is the ability to learn generic features that can be applied to new tasks
without retraining.

We propose Medusa to realize this goal, designing task heads with dual
attention mechanisms. …

arxiv learning multitasking