April 16, 2024, 4:53 p.m. | /u/Ok-Cheesecake-8881

Machine Learning www.reddit.com

I've seen many posts saying that absolute positional embeddings work only on fixed sequence lengths... But theoretically if we look at the formulae, ( Positional encoding matrix using sinusoids ) we can process any number of time steps and dimensions..... So my question here is

1. Is this true ( Sequence lengths aspect ) ? If true why and if false why?
2. What are the advantages of using Relative positional embeddings over absolute??
3. Why are relative positional embeddings …

dimensions embeddings encoding look machinelearning matrix positional encoding process question relational true work

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Lead GNSS Data Scientist

@ Lurra Systems | Melbourne

Senior Machine Learning Engineer (MLOps)

@ Promaton | Remote, Europe

Research Scientist (Computer Science)

@ Nanyang Technological University | NTU Main Campus, Singapore

Intern - Sales Data Management

@ Deliveroo | Dubai, UAE (Main Office)