No Rose Without a Thorn - Obstacles to Successful LLM Deployments // Tanmay Chopra // LLMs in Prod

// Abstract
LLMs have garnered immense attention in a short span of time - with their capabilities usually being conveyed to the world in low-precision demanding scenarios like demos and MVPs, but as we all know, deploying to prod is a whole other ballgame. In this talk, we'll discuss some pitfalls expected in deploying LLMs to production use cases both at the terminal layer (direct-to-user) as well as intermediate layers. We'll approach this topic from both infrastructural and output-focused lenses …

abstract attention llm llms low precision prod world