May 20, 2024, 12:18 p.m. | /u/BigConsequence3915

Deep Learning www.reddit.com

Hi all,

I am learning cross attention recently and I know how it works, with query and key/values from different embeddings. But I want to know how to understand this intuitively and in a high-level context, because when I design a network architecture, it's difficult for me to know which one should be key and which one should be queries or values, who's attending to who and why. Some tips from this perspective would be very useful! Thanks in advance

architecture attention context deeplearning design embeddings key network network architecture query values

Senior Data Engineer

@ Displate | Warsaw

Associate Director, Technology & Data Lead - Remote

@ Novartis | East Hanover

Product Manager, Generative AI

@ Adobe | San Jose

Associate Director – Data Architect Corporate Functions

@ Novartis | Prague

Principal Data Scientist

@ Salesforce | California - San Francisco

Senior Analyst Data Science

@ Novartis | Hyderabad (Office)