MatFormer: The Universal Elastic Transformer Capable to Generate Submodels With Zero Extra Training Costs

In a new paper MatFormer: Nested Transformer for Elastic Inference, a research team proposes MatFormer, a Transformer architecture that is inherently designed for elasticity, enables the training of a single universal model capable of generating numerous smaller submodels without the need for additional training.

ai architecture artificial intelligence costs deep-neural-networks elastic elasticity extra generate inference machine learning machine learning & data science ml paper research research team team technology training training costs transformer transformer architecture