[P] Haven: Deploy Open Source LLMs on Your Own Cloud

Open source LLMs are a great **privacy-preserving** alternative to using the ChatGPT API, but deploying them in production is still really hard - especially for people without ML experience and knowledge about ML infrastructure. With Haven, we make it possible to deploy LLMs with just a few lines of code!

**GitHub:** [https://github.com/havenhq/haven](https://github.com/havenhq/haven)

**Website:** [https://haven.run/](https://haven.run/)

**Colab Demo:** [https://colab.research.google.com/drive/1eGGSisS9Du5-\_KcaejY5y9vk9v7EIfba?authuser=2#scrollTo=YYECIKqAGId8](https://colab.research.google.com/drive/1eGGSisS9Du5-_KcaejY5y9vk9v7EIfba?authuser=2#scrollTo=YYECIKqAGId8)

​

Haven’s main component is the manager, which comes as a container image that can be deployed in your GCP environment. The manager …

api chatgpt cloud code deploy experience infrastructure knowledge llms machinelearning open source open source llms people privacy production them