To Adapt or to Fine-tune: A Case Study on Abstractive Summarization. (arXiv:2208.14559v1 [cs.CL])

Recent advances in the field of abstractive summarization leverage
pre-trained language models rather than train a model from scratch. However,
such models are sluggish to train and accompanied by a massive overhead.
Researchers have proposed a few lightweight alternatives such as smaller
adapters to mitigate the drawbacks. Nonetheless, it remains uncertain whether
using adapters benefits the task of summarization, in terms of improved
efficiency without an unpleasant sacrifice in performance. In this work, we
carry out multifaceted investigations on fine-tuning …

arxiv case case study study summarization