GPT-3, RNNs and All That: A Deep Dive into Language Modelling

How Neural networks are used in language modelling: Multi-layer Perceptrons, RNNs and Transformers.

As I’ve been working on Chai I’ve been exposed to large language models (LLMs), something I didn’t really know anything about previously. In this article I’ll summarise everything I have since learned on the subject. We’ll go from the very simple (what researchers were doing 40-ish years ago) to the state of the art, staying at a big picture …

artificial intelligence chatbots gpt gpt-3 language machine learning modelling