Fine tunning a PLM using LoRA.

Hey there, I am new to NLP and currently working on a semantic role labelling model where i am using Word and sentence embeddings from a PLM like deberta/T5 and using It for the downstream task. The Palm Is not involved in the training and its embeddings are made and stored during pre-processing.

My understanding Is that if i can finetune the plm weights on my own dataset before the training, It should in theory provide Better embeddings which Will …

embeddings hey labelling languagetechnology lora nlp palm pre-processing processing role semantic training word