all AI news
A Comprehensive Analysis of Adapter Efficiency. (arXiv:2305.07491v1 [cs.CL])
cs.CL updates on arXiv.org arxiv.org
Adapters have been positioned as a parameter-efficient fine-tuning (PEFT)
approach, whereby a minimal number of parameters are added to the model and
fine-tuned. However, adapters have not been sufficiently analyzed to understand
if PEFT translates to benefits in training/deployment efficiency and
maintainability/extensibility. Through extensive experiments on many adapters,
tasks, and languages in supervised and cross-lingual zero-shot settings, we
clearly show that for Natural Language Understanding (NLU) tasks, the parameter
efficiency in adapters does not translate to efficiency gains compared to …
analysis arxiv benefits deployment efficiency fine-tuning languages through training