Refining Joint Text and Source Code Embeddings for Retrieval Task with Parameter-Efficient Fine-Tuning

arXiv:2405.04126v1 Announce Type: new
Abstract: The latest developments in Natural Language Processing (NLP) have demonstrated remarkable progress in a code-text retrieval problem. As the Transformer-based models used in this task continue to increase in size, the computational costs and time required for end-to-end fine-tuning become substantial. This poses a significant challenge for adapting and utilizing these models when computational resources are limited. Motivated by these concerns, we propose a fine-tuning framework that leverages Parameter-Efficient Fine-Tuning (PEFT) techniques. Moreover, we adopt …

abstract arxiv become code computational costs cs.lg cs.se embeddings fine-tuning language language processing latest natural natural language natural language processing nlp processing progress retrieval text transformer transformer-based models type