all AI news
Training attention QKV matrices [D]
March 10, 2024, 6:10 p.m. | /u/datashri
Machine Learning www.reddit.com
What i don't understand is how the values of Q, K, and V matrices are trained. I've read the cross validation stack exchange answers and some others. I get that QKV come from the previous layer, but how does the previous layer determine/train their values?
Some form of backprop, sure. But what's the goal …
attention head machinelearning paper product significance stack stack exchange training validation values
More from www.reddit.com / Machine Learning
Jobs in AI, ML, Big Data
Founding AI Engineer, Agents
@ Occam AI | New York
AI Engineer Intern, Agents
@ Occam AI | US
AI Research Scientist
@ Vara | Berlin, Germany and Remote
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne