April 16, 2024, 4:53 p.m. | /u/Ok-Cheesecake-8881

Machine Learning www.reddit.com

I've seen many posts saying that absolute positional embeddings work only on fixed sequence lengths... But theoretically if we look at the formulae, ( Positional encoding matrix using sinusoids ) we can process any number of time steps and dimensions..... So my question here is

1. Is this true ( Sequence lengths aspect ) ? If true why and if false why?
2. What are the advantages of using Relative positional embeddings over absolute??
3. Why are relative positional embeddings …

dimensions embeddings encoding look machinelearning matrix positional encoding process question relational true work

Software Engineer for AI Training Data (School Specific)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Python)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Tier 2)

@ G2i Inc | Remote

Data Engineer

@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania

Artificial Intelligence – Bioinformatic Expert

@ University of Texas Medical Branch | Galveston, TX

Lead Developer (AI)

@ Cere Network | San Francisco, US