How do mixture-of-experts layers affect transformer models?

This new LLM technique has started improving the results of models without additional training.

experts generative-ai improving llm mixture of experts results se-stackoverflow se-tech training transformer transformer models