all AI news
Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs
Simon Willison's Weblog simonwillison.net
Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs
There's a lot to absorb about this one. Mosaic trained this model from scratch on 1 trillion tokens, at a cost of $200,000 taking 9.5 days. It's Apache-2.0 licensed and the model weights are available today.
They're accompanying the base model with an instruction-tuned model called MPT-7B-Instruct (licensed for commercial use) and a non-commercially licensed MPT-7B-Chat trained using OpenAI data. They also announced MPT-7B-StoryWriter-65k+ - "a model designed to read …
ai apache cost generativeai homebrewllms llms opensource standard tokens