[D] Referenceless NLP Evaluation

Hey all, I'm building this open source project that helps ML engineers evaluate LLM applications (its like unit testing for LLMs), and it works great in development since users can just write a test\_file.py like how you would normally do it in pytest, but as I'm going onto the next phase I'm thinking how to bring evaluation to production, especially on metrics such as factual consistency where I need a ground truth. I'm hoping to get some ideas around this. …

applications building development engineers evaluation hey llm llm applications llms machinelearning next nlp normally open source project pytest test testing thinking