Sept. 4, 2023, 7:43 p.m. | /u/MRMohebian

Machine Learning www.reddit.com

Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but also from applying feedforward and projection layers to every token. However, not all tokens are equally important, especially for longer documents.

CoLT5, a long-input Transformer model that builds on this intuition by employing conditional computation, devoting more resources to important tokens in both feedforward and attention layers. CoLT5 can effectively and tractably make use …

attention benefit complexity documents faster language language processing machinelearning natural natural language natural language processing processing projection tasks token tokens transformer transformer model transformers

Senior AI/ML Developer

@ Lemon.io | Remote

Earthquake Forecasting Post-doc in ML at the USGS

@ U. S. Geological Survey | Remote, US

Senior Data Scientist - Remote - Colombia

@ FullStack Labs | Soacha, Cundinamarca, Colombia

Senior Data Engineer

@ Reorg | Remote - US

Quantitative / Data Analyst

@ Talan | London, United Kingdom

Senior Data Scientist

@ SoFi | CA - San Francisco; US - Remote