AI Fundamentals
A complete, free, self-paced introduction to how modern AI works — structured around Andrej Karpathy's open Neural Networks: Zero to Hero series. Watch in order, code along, and you'll go from zero to building a working GPT.
How to take this course
Set aside 2–3 hours per session. Open the linked video, follow along in your own editor, and don't move on until you can explain the lesson out loud. The lessons build on each other — order matters.
- 01
What is an LLM, really?
Start with a high-level mental model of what large language models are, what they can do, and the rough shape of the system you'll spend the rest of the course building up to.
[1hr Talk] Intro to Large Language ModelsAndrej Karpathy 1h 0m - 02
Neural networks & backpropagation, from scratch
Build a tiny autograd engine and a neural net by hand. After this lesson, terms like 'gradient', 'loss' and 'backprop' will stop being magic.
The spelled-out intro to neural networks and backpropagation: building microgradAndrej Karpathy 2h 25m - 03
Your first language model: bigrams
Train the simplest possible character-level language model. You'll see the entire LM lifecycle — tokens, training, sampling — in its smallest form.
The spelled-out intro to language modeling: building makemoreAndrej Karpathy 1h 57m - 04
From bigrams to a real neural net (MLP)
Replace the lookup table with a multi-layer perceptron. Learn embeddings, hidden layers, and why deeper models can capture more.
Building makemore Part 2: MLPAndrej Karpathy 1h 16m - 05
Activations, gradients & BatchNorm
Understand what actually makes deep networks trainable — initialization, activation distributions, and normalization layers.
Building makemore Part 3: Activations & Gradients, BatchNormAndrej Karpathy 1h 55m - 06
Becoming a backprop ninja
Manually backpropagate through every operation in your network. This is the lesson that promotes neural nets from 'I trust the framework' to 'I understand it'.
Building makemore Part 4: Becoming a Backprop NinjaAndrej Karpathy 1h 55m - 07
Let's build GPT — from scratch
The headline lesson: code a transformer-based GPT character by character. By the end you'll know exactly what self-attention does and why GPTs work.
Let's build GPT: from scratch, in code, spelled out.Andrej Karpathy 1h 56m - 08
Tokenizers — the hidden layer
Tokens are where most LLM weirdness comes from. Build a Byte-Pair Encoding tokenizer from scratch and finally understand why models can't spell.
Let's build the GPT TokenizerAndrej Karpathy 2h 13m
Ready for something more applied?
Continue with our practical course on actually using LLMs day to day.
Practical AI for Beginners