promptfoo: How to benchmark Llama2 Uncensored vs. GPT-3.5 on your own inputs

promptfoo: How to benchmark Llama2 Uncensored vs. GPT-3.5 on your own inputs

promptfoo is a CLI and library for "evaluating LLM output quality". This tutorial in their documentation about using it to compare Llama 2 to gpt-3.5-turbo is a good illustration of how it works: it uses YAML files to configure the prompts, and more YAML to define assertions such as "not-icontains: AI language model".

ai benchmark cli documentation generativeai good gpt gpt-3 gpt-3.5 gpt-3.5-turbo illustration library llama llama 2 llama2 llm llms quality testing turbo tutorial yaml