Jan. 15, 2024, 5:49 p.m. | /u/karun_kodes

Machine Learning www.reddit.com

So I was just reading about absolute positional encoding and then about relative positional embedding.

All I could understand is how's it done with relative to each word. But I really couldn't think of the advantages over absolute one as the "Attention is all you need" also states that ""We chose this function because we hypothesized it would allow the model to easily learn to attend by *relative positions.." So what's the advantage relative positional encoding carries ?.*


And can …

advantages attention attention is all you need embedding encoding machinelearning positional encoding reading think word

AI Engineer Intern, Agents

@ Occam AI | US

AI Research Scientist

@ Vara | Berlin, Germany and Remote

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Data Engineer - Takealot Group (Takealot.com | Superbalist.com | Mr D Food)

@ takealot.com | Cape Town