Web: https://www.reddit.com/r/LanguageTechnology/comments/sh5vlm/gpt2_from_scratch_pytorch_video_tutorial/

Jan. 31, 2022, 4:53 p.m. | /u/mildlyoverfitted

Natural Language Processing reddit.com

Hey everybody,

I created a video tutorial on GPT-2 and how to implement it "from scratch".

It only focuses on the inference (training is not discussed). It covers concepts like masked self attention, decoder blocks and generating new tokens. It is heavily based on minGPT (see link below).

Hope some of you could find it useful!


Relevant links:

submitted by /u/mildlyoverfitted
[link] [comments]

gpt languagetechnology pytorch tutorial video

Engineering Manager, Machine Learning (Credit Engineering)

@ Affirm | Remote Poland

Sr Data Engineer

@ Rappi | [CO] Bogotá

Senior Analytics Engineer

@ GetGround | Porto

Senior Staff Software Engineer, Data Engineering

@ Galileo, Inc. | New York City or Remote

Data Engineer

@ Atlassian | Bengaluru, India

Data Engineer | Hybrid (Pune)

@ Velotio | Pune, Maharashtra, India