A Testing Library for AI Prompt Engineering // Maxime Beauchemin // MLOps Podcast #167 clip

MLOps Coffee Sessions #167 with Maxime Beauchemin, Treating Prompt Engineering More Like Code.

The problem of generating SQL queries is open-ended but deterministic, making it suitable for assessing AI performance. To evaluate the AI's success, he emphasizes the need to build a bank of test cases with various prompts and corresponding queries to assess accuracy and iterate on the system. Maxime also talks about "Promptimize", a testing library he created to evaluate different prompts and models in the AI system. …

ai performance bank build clip code coffee engineering library making mlops performance podcast prompt sql sql queries success test testing