June 8, 2024, 5:12 p.m. | /u/NeatFox5866

Deep Learning www.reddit.com

Hi guys!

This is just a clarification post. As far as I understand, key (K), query (Q), and value (V) vectors come from the **same** **embeddings**. Let me explain: we project the same embeddings **into different weight matrices** (WK, WQ, and WV) and we operate with those. Am I getting this right?

Thank you!

attention attention mechanisms deeplearning embeddings key project query value vectors

Senior Data Engineer

@ Displate | Warsaw

Junior Data Analyst - ESG Data

@ Institutional Shareholder Services | Mumbai

Intern Data Driven Development in Sensor Fusion for Autonomous Driving (f/m/x)

@ BMW Group | Munich, DE

Senior MLOps Engineer, Machine Learning Platform

@ GetYourGuide | Berlin

Data Engineer, Analytics

@ Meta | Menlo Park, CA

Data Engineer

@ Meta | Menlo Park, CA