Hyperdecoders: Instance-specific decoders for multi-task NLP. (arXiv:2203.08304v2 [cs.CL] UPDATED)

We investigate input-conditioned hypernetworks for multi-tasking in NLP,
generating parameter-efficient adaptations for a decoder using a hypernetwork
conditioned on the output of an encoder. This approach produces a unique
decoder for every input instance, allowing the network a larger degree of
flexibility than prior work that specializes the decoder for each task. We
apply our method to sequence classification tasks, extractive QA, and
summarisation and find that it surpasses previous parameter efficient
fine-tuning methods and often outperforms fully finetuning the …

arxiv nlp