NeurIPS 2022: Overcoming Communication Bottlenecks for Decentralized Training (2/2)

Dec. 5, 2022, 4:01 p.m. | Together

Blog Content - TOGETHER www.together.xyz

We introduce a compression technique that can train models on networks with
20X less bandwidth with minimal impact on training time. Another step
toward democratizing the training & tuning of foundation models!

communication compression decentralized foundation impact networks neurips neurips 2022 research training

Visit resource

More from www.together.xyz / Blog Content - TOGETHER

RedPajama-Data-v2: an Open Dataset with 30 Trillion Tokens for Training Large Language Models 6 months, 1 week ago | www.together.xyz

annotations data data quality dataset +12

Flash-Decoding for long-context inference 6 months, 4 weeks ago | www.together.xyz

attention context decoding faster +3

Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads 8 months ago | www.together.xyz

api context decoding framework +6

Llama-2-7B-32K-Instruct — and fine-tuning for Llama-2 models with Together API 8 months, 3 weeks ago | www.together.xyz

api context fine-tuning llama +2

Faster inference enables up to 5x price reduction on Together API 9 months ago | www.together.xyz

ai stack api cost efficiency +12

Preparing for the era of 32K context: Early learnings and explorations 9 months, 1 week ago | www.together.xyz

context document understanding llama research +2

Monarch Mixer: A new model architecture for increased efficiency 9 months, 2 weeks ago | www.together.xyz

architecture efficiency exploration look +2

Introducing Together AI Chief Scientist Tri Dao, as he releases FlashAttention-2 to speed up model … 9 months, 3 weeks ago | www.together.xyz

ai models dao fine-tuning inference +5

Together AI and Snorkel AI empower enterprises to build proprietary LLMs 9 months, 3 weeks ago | www.together.xyz

build data enterprises environments +5

Data Engineer

@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania

View on ai-jobs.net

Artificial Intelligence – Bioinformatic Expert

@ University of Texas Medical Branch | Galveston, TX

View on ai-jobs.net

Lead Developer (AI)

@ Cere Network | San Francisco, US

View on ai-jobs.net

Research Engineer

@ Allora Labs | Remote

View on ai-jobs.net

Ecosystem Manager

@ Allora Labs | Remote

View on ai-jobs.net

Founding AI Engineer, Agents

@ Occam AI | New York

View on ai-jobs.net

View more jobs

all AI news

NeurIPS 2022: Overcoming Communication Bottlenecks for Decentralized Training (2/2)

More from www.together.xyz / Blog Content - TOGETHER

Jobs in AI, ML, Big Data

Data Engineer

Artificial Intelligence – Bioinformatic Expert

Lead Developer (AI)

Research Engineer

Ecosystem Manager

Founding AI Engineer, Agents