Large Language Models, GPT-1 — Generative Pre-Trained Transformer

Large Language Models, GPT-1 — Generative Pre-Trained Transformer

Diving deeply into the working structure of the first ever version of gigantic GPT-models

Introduction

2017 was a historical year in machine learning. Researchers from the Google Brain team introduced Transformer which rapidly outperformed most of the existing approaches in deep learning. The famous attention mechanism became the key component in the future models derived from Transformer. The amazing fact about Transformer’s architecture is its vaste flexibility: it can be efficiently used …

attention brain deep-dives deep learning generative generative pre-trained transformer google google brain gpt gpt-1 language language models large language large language models machine machine learning researchers team transformer transformers