[D] How does a MoE router learn when it has made a wrong choice?

Looking at the code for current mixture of experts models, they seem to use argmax, with k=1 (picking only the top expert) to select the router choice. Since argmax is non differentiable, the gradient cannot flow to the other experts. Thus it seems to me that only the weights of the selected expert will be updated if it performs poorly. However, it could be the case that a different expert was in fact a better choice for the given input, …

code current differentiable expert experts flow gradient learn machinelearning mixture of experts moe