Let’s BLOOM with BigScience’s New AI Model

Deploy large language models with bnb-Int8 for Hugging Face

What is this about?

In this tutorial we will deploy BigScience’s BLOOM model, one of the most impressive large language models (LLMs), in an Amazon SageMaker endpoint. To do so, we will leverage the bitsandbytes (bnb) Int8 integration for models from the Hugging Face (HF) Hub. With these Int8 weights we can run large models that previously wouldn’t fit into our GPUs.

The code …

ai ai model bigscience bloom hugging face machine learning nlp sagemaker