Scalable Evaluation and Serving of Open Source LLMs // Waleed Kadous // LLMs in Prod Conference

This portion is sponsored by Anyscale.
Website: https://www.anyscale.com/
Anyscale is a unified compute platform that makes it easy to develop, deploy, and manage scalable AI and Python applications using Ray.

//Abstract
While we've seen great progress on Open Source LLMs, we haven't seen the same level of progress on systems to serve those LLMs in production contexts. In this presentation, I work through some of the challenges of taking open source models and serving them in production.

//Bio
Dr. Waleed …

abstract anyscale applications compute conference deploy easy evaluation llms open source open source llms platform prod progress python ray scalable sponsored