all AI news
Snowflake Arctic Cookbook
Simon Willison's Weblog simonwillison.net
Today's big model release was Snowflake Arctic, an enormous 480B model with a 128×3.66B MoE (Mixture of Experts) architecture. It's Apache 2 licensed and Snowflake state that "in addition, we are also open sourcing all of our data recipes and research insights."
The research insights will be shared on this Arctic Cookbook blog - which currently has two articles covering the MoE architecture and describing how they optimized their training run in great detail.
They also list …
ai apache architecture arctic big blog data experts generativeai insights llms mixture of experts moe recipes release research snowflake state will