Deep Dive into LLMs like ChatGPT - YouTube

I set aside some time today and yesterday and just finished watching Andrej Karpathy’s latest video on LLMs 101. https://youtu.be/7xTGNNLPyMI

In his video he went over LLM foundation model training, supervised finetuning, and the latest exploratory RL and RLHF training. And there is a section where he explains why DeepSeek R1 paper is a big deal (at a high level, yet super helpful). I feel like I have a much better understanding of how RL is helping LLM post-training and how RLHF is (and is not) really RL and its shortcomings.

I watched the whole video (3.5hours @ 2x) and would recommend it anyone who’s interested in LLM or working with LLM since it will help you understand its quirks.

Note, I watched many of Andrej’s previous videos. For example, the one on tokenization (which explains why asking GPT to output YAML instead of JSON saves more tokens, among other things.) He is a great educator.