[D] Gemma vs Mistral (and other open models)

A lot of comparisons out there in terms of model size and quantitative performance against benchmarks. Wanted to get people's thoughts on qualitative, ad-hoc performance.

\> Playground to Compare: [https://huggingface.co/spaces/lastmileai/gemma-playground](https://huggingface.co/spaces/lastmileai/gemma-playground)

In this example, both Gemma 2B and 7B doesn't seem to do well against Mistral for CoT tasks. Curious how it's doing for instruct, question-answering, and creativity tasks.

benchmarks creativity example gemma machinelearning mistral open models people performance quantitative question tasks terms thoughts