Stanford CS234: Reinforcement Learning: study plan and how long it takes
The Stanford Online playlist is 16 videos and 20.2 hours long. Here's how long it takes at different paces, a week-by-week schedule built from the real videos, and a planner that turns it into your own daily plan.
16 videos · 20.2 hours · Stanford Online
Stanford CS234 (Spring 2024), taught by Emma Brunskill, covers reinforcement learning from tabular MDPs and policy evaluation to Q-learning, policy search and policy gradients, offline RL, a guest lecture on DPO, exploration, and multi-agent RL.
It balances theory with the methods behind modern RLHF and LLM fine-tuning.
Who it's for: Students with ML basics and probability who want a rigorous introduction to reinforcement learning.
How long does it take?
The full playlist is 20.2 hours across 16 videos. Here's roughly how many days that is at different daily watch times and speeds, counting video time only:
| Daily time | At 1x | At 1.5x | At 2x |
|---|---|---|---|
| 30 min a day | 41 days | 27 days | 21 days |
| 1 h a day | 21 days | 14 days | 11 days |
| 2 h a day | 11 days | 7 days | 6 days |
Make your own schedule
The playlist link is already filled in. Press Plan, set how much time you have each day and your speed, and you'll get a day-by-day schedule with a finish date and a calendar file. It reads the live playlist, so new videos are included.
Week-by-week schedule
A sample plan at 1 hour a day and normal speed, using whole videos so none is split across days. That's 3 weeks in total. For a different pace, start date or speed, use the planner above.
Week 1Videos 1–7 · 8h 58m
Day 1 · 1h 19m
- 1Stanford CS234 Reinforcement Learning I Introduction to Reinforcement Learning I 2024 I Lecture 11h 19m
Day 2 · 1h 13m
- 2Stanford CS234 Reinforcement Learning I Tabular MDP Planning I 2024 I Lecture 21h 13m
Day 3 · 1h 20m
- 3Stanford CS234 Reinforcement Learning I Policy Evaluation I 2024 I Lecture 31h 20m
Day 4 · 1h 19m
- 4Stanford CS234 Reinforcement Learning I Q learning and Function Approximation I 2024 I Lecture 41h 19m
Day 5 · 1h 8m
- 5Stanford CS234 Reinforcement Learning I Policy Search 1 I 2024 I Lecture 51h 8m
Day 6 · 1h 19m
- 6Stanford CS234 Reinforcement Learning I Policy Search 2 I 2024 I Lecture 61h 19m
Day 7 · 1h 19m
- 7Stanford CS234 Reinforcement Learning I Policy Search 3 I 2024 I Lecture 71h 19m
Week 2Videos 8–14 · 8h 49m
Day 8 · 1h 14m
- 8Stanford CS234 Reinforcement Learning I Offline RL 1 I 2024 I Lecture 81h 14m
Day 9 · 1h 19m
- 9Stanford CS234 I Guest Lecture on DPO: Rafael Rafailov, Archit Sharma, Eric Mitchell I Lecture 91h 19m
Day 10 · 1h 20m
- 10Stanford CS234 Reinforcement Learning I Offline RL 3 I 2024 I Lecture 101h 20m
Day 11 · 1h 15m
- 11Stanford CS234 Reinforcement Learning I Exploration 1 I 2024 I Lecture 111h 15m
Day 12 · 1h 18m
- 12Stanford CS234 Reinforcement Learning I Exploration 2 I 2024 I Lecture 121h 18m
Day 13 · 1h 10m
- 13Stanford CS234 Reinforcement Learning I Exploration 3 I 2024 I Lecture 131h 10m
Day 14 · 1h 13m
- 14Stanford CS234 Reinforcement Learning I Multi-Agent Game Playing I 2024 I Lecture 141h 13m
Week 3Videos 15–16 · 2h 23m
Day 15 · 1h 14m
- 15Stanford CS234 Reinforcement Learning I Emma Brunskill & Dan Webber I 2024 I Lecture 151h 14m
Day 16 · 1h 10m
- 16Stanford CS234 Reinforcement Learning I Value Alignment I 2024 I Lecture 161h 10m
Tips for finishing
- Plan two lectures a week alongside the course assignments.
- The policy gradient lectures (5 to 7) are the foundation for RLHF. Give them extra time.