Karpathy's Neural Networks: Zero to Hero: study plan and how long it takes
The Andrej Karpathy playlist is 10 videos and 19.4 hours long. Here's how long it takes at different paces, a week-by-week schedule built from the real videos, and a planner that turns it into your own daily plan.
10 videos · 19.4 hours · Andrej Karpathy
In Neural Networks: Zero to Hero, Andrej Karpathy builds neural networks from scratch in Python, starting with a tiny autograd engine and ending with a reproduction of GPT-2. It is one of the most recommended ways to really understand how modern language models work.
The videos are long, between one and four hours each, and they are meant to be coded along with. Plan for more time than the runtime: pausing, typing and debugging is where the learning happens.
Who it's for: Programmers comfortable with Python who want to understand deep learning and language models from the ground up. High-school calculus is enough for the maths.
How long does it take?
The full playlist is 19.4 hours across 10 videos. Here's roughly how many days that is at different daily watch times and speeds, counting video time only:
| Daily time | At 1x | At 1.5x | At 2x |
|---|---|---|---|
| 30 min a day | 39 days | 26 days | 20 days |
| 1 h a day | 20 days | 13 days | 10 days |
| 2 h a day | 10 days | 7 days | 5 days |
Make your own schedule
The playlist link is already filled in. Press Plan, set how much time you have each day and your speed, and you'll get a day-by-day schedule with a finish date and a calendar file. It reads the live playlist, so new videos are included.
Week-by-week schedule
A sample plan at 1 hour a day and normal speed, using whole videos so none is split across days. That's 2 weeks in total. For a different pace, start date or speed, use the planner above.
Week 1Videos 1–7 · 12h 23m
Day 1 · 2h 26m
- 1The spelled-out intro to neural networks and backpropagation: building micrograd2h 26m
Day 2 · 1h 58m
- 2The spelled-out intro to language modeling: building makemore1h 58m
Day 3 · 1h 16m
- 3Building makemore Part 2: MLP1h 16m
Day 4 · 1h 56m
- 4Building makemore Part 3: Activations & Gradients, BatchNorm1h 56m
Day 5 · 1h 55m
- 5Building makemore Part 4: Becoming a Backprop Ninja1h 55m
Day 6 · 56m
- 6Building makemore Part 5: Building a WaveNet56m
Day 7 · 1h 56m
- 7Let's build GPT: from scratch, in code, spelled out.1h 56m
Week 2Videos 8–10 · 6h 58m
Day 8 · 43m
- 8State of GPT | BRK216HFS43m
Day 9 · 2h 14m
- 9Let's build the GPT Tokenizer2h 14m
Day 10 · 4h 1m
- 10Let's reproduce GPT-2 (124M)4h 1m
Study plan, phase by phase
Video numbers are positions in the playlist. To schedule just one phase, enter its numbers in the planner's From and To fields.
- 01
Backpropagation: building micrograd
Video 1 · 2h 26m
A small autograd engine and neural network library, built step by step. Everything later builds on this, so don't rush it; split it over two or three sessions.
- 02
Language modelling: the makemore series
Videos 2–6 · 8h 1m
A character-level language model built five ways: bigrams, an MLP, activations and BatchNorm, manual backprop ("Becoming a Backprop Ninja") and a WaveNet-style network.
- 03
Transformers: building GPT
Videos 7–8 · 2h 39m
Building a GPT from scratch, following the Attention Is All You Need architecture, and the State of GPT talk, which is a conference talk rather than a coding lecture and can be watched at a higher speed.
- 04
Tokenizers and reproducing GPT-2
Videos 9–10 · 6h 15m
Building the GPT tokenizer (byte-pair encoding), then a four-hour session reproducing GPT-2 (124M) end to end. Plan the GPT-2 video across several days.
Tips for finishing
- Set the planner's daily time to about half your study time. If you have 2 hours a day, plan 1 hour of video and keep the rest for coding along.
- Keep playback at 1x for the coding videos. Speed helps with talks, but in live coding you will be pausing constantly anyway.
- Where a video's description lists exercises, do them before moving on. They are short and catch gaps early.