[D] Interview w/ Google Brain researchers on Sparse Expert Models (Switch Transformers, GLAM, and more...)

[https://youtu.be/ccBMRryxGog](https://youtu.be/ccBMRryxGog)

This video is an interview with Barret Zoph and William Fedus of Google Brain about Sparse Expert Models.

Sparse Expert models have been hugely successful at distributing parts of models, mostly Transformers, across large array of machines and use a routing function to effectively route signals between them. This means that even though these models have a huge number of parameters, the computational load for a given signal does not increase because the model is only sparsely activated. Sparse …

brain expert glam google google brain interview machinelearning researchers sparse expert models transformers