Create Your Own Local Chatbot with Next.js, Llama.cpp, and ModelFusion

In this blog post, we'll build a Next.js chatbot that runs on your computer. We'll use Llama.cpp to serve the OpenHermes 2.5 Mistral LLM (large language model) locally, the Vercel AI SDK to handle stream forwarding and rendering, and ModelFusion to integrate Llama.cpp with the Vercel AI SDK. The chatbot will be able to generate responses to user messages in real-time.

The architecture looks like this:

You can find a full Next.js, Vercel AI SDK, Llama.cpp & ModelFusion starter with …

ai beginners blog build chatbot computer cpp javascript language language model large language large language model llama llm mistral mistral llm next next.js opensource rendering sdk serve vercel vercel ai sdk