GEMINI PRO Outclassed by Mixtral 8x7B? Live Test!

Gemini-PRO is now (starting today) available on Vertex-AI. So we test it in a causal reasoning test against the open source MoE Mixtral 8x7B. Can a Mixture of Experts with eight 7B LLM system have a similar performance to the latest LLM from Google? We expect the new GEMINI Ultra not before next year.

Simple (and non-scientific) test of Mixtral 8x7B against GEMINI-PRO on my standard causal reasoning prompt, with which I tested already 7 other LLMs for you to …

expect experts gemini gemini pro gemini ultra google llm mixtral mixtral 8x7b mixture of experts moe open source performance reasoning test vertex vertex-ai