MIXTRAL 8x7B MoE Instruct: LIVE Performance Test

Mistral AI's new Sparse Mixture-of-Experts system (SMoE) is now available: MIXTRAL 8x7B. Also in a DPO instruction tuned version MIXTRAL 8x 7B Instruct, which we test on real world causal reasoning, in a live recoding.

Plus Python code to run inference on the fp32 Mixtral 8x7B, on the fp16 Mixtral 8x7B, on the 4-bit quantized version of Mixtral 8x7B and also w/ Flash-attention_2.

Plus costs for inference API (costs per token) and embedding API.

00:00 Live test of Mixtral 8x7B …

code experts fp16 inference mistral mistral ai mixtral 8x7b moe performance python reasoning test world