Google Trains a 540B Parameter Language Model With Pathways, Achieving ‘Breakthrough Performance’

A Google Research team further explores the scaling approach for improving language modelling, leveraging the new Pathways distributed ML system to train a 540 billion parameter autoregressive transformer, Pathways Language Model (PaLM), that achieves state-of-the-art few-shot performance.

ai artificial intelligence few-shot learning google language language model machine learning machine learning & data science ml neural network performance pretrained language model research technology trains