An Overview of the LoRA Family

LoRA, DoRA, AdaLoRA, Delta-LoRA, and more variants of low-rank adaptation.

LoRA comes in different shapes and varieties. Photo by Lucas George Wendt on Unsplash.

Low-Rank Adaptation (LoRA) can be considered a major breakthrough towards the ability to train large language models for specific tasks efficiently. It is widely used today in many applications and has inspired research on how to improve upon its main ideas to achieve better performance or train models even …

adapter dora editors pick llm lora