May 20, 2024, 12:18 p.m. | /u/BigConsequence3915

Deep Learning www.reddit.com

Hi all,

I am learning cross attention recently and I know how it works, with query and key/values from different embeddings. But I want to know how to understand this intuitively and in a high-level context, because when I design a network architecture, it's difficult for me to know which one should be key and which one should be queries or values, who's attending to who and why. Some tips from this perspective would be very useful! Thanks in advance

architecture attention context deeplearning design embeddings key network network architecture query values

Senior Data Engineer

@ Displate | Warsaw

Director of Data Science (f/m/x)

@ AUTO1 Group | Berlin, Germany

Business Intelligence Analyst I [BI Analyst I]

@ Capitec Bank | Stellenbosch, Western Cape, ZA

Data Governance Associate Director

@ Publicis Groupe | London, United Kingdom

Technical Lead - Power BI

@ Birlasoft | INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI, IN

Data Analyst

@ FirstRand Corporate Centre | 1 First Place, Cnr Simmonds & Pritchard Streets, Johannesburg, 2001