[P] OpenAI vs Open Source LLM Comparison for Document Q&A

Ran a fun comparison between OpenAI vs open source (Apache 2.0) LLMs for Wikipedia document Q&A -- open source is looking good (and getting better).

TLDR:

For simple Wikipedia article Q&A, I compared OpenAI GPT 3.5, FastChat-T5, FLAN-T5-XXL, and FLAN-T5-XL. GPT 3.5 provided the best answers, but FastChat-T5 was very close in performance (with a basic guardrail). The T5 models I tested are all licensed under Apache 2.0, so they are commercially viable.

For the embedding model, I compared OpenAI …

apache article comparison fun good gpt llm llms machinelearning openai openai gpt open source open source llm wikipedia