all AI news
674: Parameter-Efficient Fine-Tuning of LLMs using LoRA (Low-Rank Adaptation) — with Jon Krohn
April 28, 2023, 11 a.m. | Super Data Science Podcast with Jon Krohn
Super Data Science Podcast with Jon Krohn www.youtube.com
Models like Alpaca, Vicuña, GPT4All-J and Dolly 2.0 have relatively small model architectures, but they're prohibitively expensive to train even on a small amount of your own data. The standard model-training protocol can also lead to catastrophic forgetting. In this week's episode, @JonKrohnLearns explores a solution to these problems, introducing listeners to Parameter-Efficient Fine-Tuning (PEFT) and the leading approach: Low-Rank Adaptation (LoRA).
Additional materials: https://www.superdatascience.com/674
Interested in sponsoring a SuperDataScience Podcast episode? Visit https://www.jonkrohn.com/podcast for sponsorship information.
alpaca architectures data dolly dolly 2.0 fine-tuning gpt4all jon llms lora low model-training protocol small solution standard training
More from www.youtube.com / Super Data Science Podcast with Jon Krohn
Can AI Replace Doctors Completely?
1 day, 2 hours ago |
www.youtube.com
Abridge AI Relieves Doctors of Admin Work
1 day, 9 hours ago |
www.youtube.com
769: Generative AI for Medicine — with Prof. Zack Lipton
2 days, 9 hours ago |
www.youtube.com
How to Make a Live While Contributing Open-Source
3 days, 2 hours ago |
www.youtube.com
The 3 Best Alternatives to RLHF
3 days, 9 hours ago |
www.youtube.com
Multi-Query vs Multi-Head Attention
4 days, 9 hours ago |
www.youtube.com
Jobs in AI, ML, Big Data
Senior ML Researcher - 3D Geometry Processing | 3D Shape Generation | 3D Mesh Data
@ Promaton | Europe
Software Engineer, Data Platforms
@ Whatnot | San Francisco, CA, Los Angeles, CA, New York City, Phoenix, AZ, Seattle, WA, Denver, CO
Staff Data Engineer, Data Platform
@ Lilt | Indianapolis
Business Data Analyst - New Division
@ Breakthru Beverage Group | Toronto, ON, Canada
Data Operations Associate
@ iCapital | New York City, United States
Senior Data Scientist, R&D
@ Plusgrade | Toronto, Ontario