AI & Machine Learning · Intermediate

Neural Networks & Deep Learning

From a single neuron to building a language model yourself — taught by the people who explain it best.

Lessons
29 lessons
Total runtime
66 hours of video
Modules
8 modules
Optional lessons
17 optional
Start course

Free. No account.

About this course

There is an enormous amount of writing about machine learning and very little of it explains anything. Most of it is either a metaphor about brains, which is wrong, or an equation with no picture attached, which is useless. This course takes the narrow path between those: build the intuition visually first, then watch someone write the actual code, line by line, with nothing hidden.

The first half is Grant Sanderson and Josh Starmer covering the same ground twice, on purpose. 3Blue1Brown gives you the geometry — a network as a function, training as descending a landscape — and StatQuest works through the arithmetic on small numbers you can follow by hand. Neither alone is sufficient. Together they are the clearest explanation of backpropagation available for free, and probably at any price.

Then the course changes character entirely. Andrej Karpathy's micrograd video is two and a half hours of building an automatic differentiation engine and a neural network from an empty file, in Python, with every line explained. It is demanding and it is the most valuable single video here. Everything abstract in the first half becomes concrete during it, and there is no substitute for having typed it yourself.

The last part is about language models, which is what most people actually came for. The transformer chapters explain attention properly rather than by analogy, and Karpathy's build-a-GPT video closes the loop: the same ideas from module two, scaled up, producing text. Roughly eight hours required, and around sixty-six in total — the optional lessons add convolutional networks, four separate PyTorch courses, a module on fine-tuning existing models, and two Stanford lectures. It is a genuinely difficult course. Do not watch it passively — the code videos in particular are meant to be paused and typed along with.

What you'll be able to do

  • Explain what a neural network computes, without reaching for a metaphor about brains
  • Describe gradient descent and backpropagation well enough to implement them
  • Build a small automatic differentiation engine and train a network with it, in plain Python
  • Explain what attention does in a transformer, and why it replaced what came before
  • Read a modern machine learning paper's architecture section and follow the shapes
  • Judge what these systems can and cannot do from how they are built rather than from marketing

Curriculum

29 lessons · 66 hours

What a Neural Network Is

4 lessons · 2h 7m

  1. 01CompletedMachine Learning Explained in 100 SecondsFireshipVideoWhere machine learning sits relative to everything else called AI. Two minutes of vocabulary so the rest of the course is not also a glossary.3m
  2. 02CompletedBut what is a neural network?3Blue1BrownVideoA network as a function with thirteen thousand knobs on it, shown recognising handwritten digits. The single best introduction to the subject, and it never once mentions brains.19m
  3. 03CompletedThe Essential Main Ideas of Neural NetworksStatQuest with Josh StarmerVideoThe same object, built up from arithmetic on numbers small enough to check by hand. Watch it straight after the last lesson — the two explanations reinforce rather than repeat each other.19m
  4. 04CompletedDeep Learning Crash Course for BeginnersOptionalfreeCodeCamp.orgVideoAn hour and a half covering the whole field in outline — the vocabulary, the main architectures and what each is for. Useful as a map before the careful explanations that follow.1h 26m

How It Learns

5 lessons · 1h 27m

  1. 05CompletedGradient descent, how neural networks learn3Blue1BrownVideoTraining as walking downhill on a surface with thirteen thousand dimensions. This is where the word learning stops being a metaphor and becomes a specific, mechanical procedure.21m
  2. 06CompletedGradient Descent, Step-by-StepStatQuest with Josh StarmerVideoThe same algorithm worked through numerically, including the learning rate and why it matters. Slower than the previous lesson and worth every minute.24m
  3. 07CompletedBackpropagation, intuitively3Blue1BrownVideoThe idea that makes training possible at all: each training example voting on how every weight should nudge. Understand this lesson and the rest of deep learning is engineering.14m
  4. 08CompletedNeural Networks Pt. 2: Backpropagation Main IdeasOptionalStatQuest with Josh StarmerVideoOptional, and the one to take if the previous lesson left you nodding along without quite being able to reproduce it.18m
  5. 09CompletedBackpropagation calculus3Blue1BrownVideoThe chain rule, applied. Short and dense, and it is what the code in the next module is implementing — watch it immediately before starting micrograd.10m

Building One Yourself

1 lesson · 2h 25m

  1. 10CompletedThe spelled-out intro to neural networks and backpropagation: building microgradAndrej KarpathyVideoThe most valuable video on this site. An automatic differentiation engine and a working neural network, built from an empty file with nothing hidden. Type it out as you go — watching it is worth perhaps a tenth of doing it.2h 25m

Convolutions and Vision

3 lessons · 2h 17m

  1. 11CompletedBut What Is a Convolution?Optional3Blue1BrownVideoThe mathematical operation itself, explained before any mention of networks. Same treatment as the earlier 3Blue1Brown lessons, and the right way in.23m
  2. 12CompletedHow Convolutional Neural Networks WorkOptionalBrandon RohrerVideoConvolution applied to images, worked through by hand on a tiny example. Nothing else explains filters and pooling this concretely.26m
  3. 13CompletedHot Dog or Not Hot Dog — CNN CourseOptionalfreeCodeCamp.orgVideoBuilding and training an image classifier end to end. The practical counterpart to the two lessons above.1h 28m

PyTorch

4 lessons · 40h 58m

  1. 14CompletedPyTorch in 1 HourOptionalZachary HuangVideoKarpathy's videos build everything from scratch, which is the right way to understand it and the wrong way to work. This is the shortest bridge to the library everyone actually uses.1h 3m
  2. 15CompletedDeep Learning With PyTorch — Full CourseOptionalPatrick LoeberVideoFour and a half hours, structured as short focused chapters. The best-paced of the PyTorch options here.4h 36m
  3. 16CompletedPyTorch for Deep Learning — Full CourseOptionalfreeCodeCamp.orgVideoTen hours covering the library properly, including the data-loading and training-loop machinery that shorter courses gloss over.9h 42m
  4. 17CompletedPyTorch for Deep Learning & Machine Learning — Full CourseOptionalfreeCodeCamp.orgVideoTwenty-five hours, and the most exhaustive free PyTorch course in existence. A reference to work through over months, not a video to watch.25h 37m

Language Models

7 lessons · 8h 39m

  1. 18CompletedIntro to Large Language ModelsAndrej KarpathyVideoAn hour with no code: what these systems are, how they are trained in stages, and what the honest open questions are. The best available answer to what is actually going on inside one.1h
  2. 19CompletedTransformers, the tech behind LLMs3Blue1BrownVideoThe architecture, visually, with the tensor shapes actually shown. Embeddings and the unembedding step are the parts most explanations skip and this one does not.27m
  3. 20CompletedAttention in transformers, step-by-step3Blue1BrownVideoQueries, keys and values explained as an operation rather than an analogy. This is the central idea of the last decade of machine learning and it deserves the twenty-six minutes.26m
  4. 21CompletedHow might LLMs store factsOptional3Blue1BrownVideoOptional. The multilayer perceptron blocks, which are two thirds of the parameters and get a fraction of the attention. Speculative in places, and honest about which places.22m
  5. 22CompletedLet's build GPT: from scratch, in code, spelled outAndrej KarpathyVideoThe capstone. A working transformer language model, written from nothing, trained on Shakespeare. Two hours that turn every diagram in this module into code you have run.1h 56m
  6. 23CompletedDeep Dive into LLMs like ChatGPTOptionalAndrej KarpathyVideoOptional, and three and a half hours. The full pipeline from pretraining data to fine-tuning and reinforcement learning. The best thing to watch after finishing the course, not during it.3h 30m
  7. 24CompletedRecurrent Neural Networks, Transformers and Attention — MIT 6.S191OptionalAlexander AminiVideoThe academic version of the transformer chapters above, tracing the path from recurrent networks to attention and explaining why the older approach was abandoned. Good for seeing the same idea derived rather than illustrated.58m

Working With Existing Models

3 lessons · 3h 27m

  1. 25CompletedThe Easiest Way to Fine-Tune an LLMOptionalTech With TimVideoMost people will adapt an existing model rather than train one. Twenty minutes to do that once, end to end, on your own machine.22m
  2. 26CompletedFine-Tuning Large Language ModelsOptionalShaw TalebiVideoThe concepts underneath — what LoRA is doing, when fine-tuning beats prompting, and when it is the wrong tool entirely.28m
  3. 27CompletedFine Tuning LLM Models — Generative AI CourseOptionalfreeCodeCamp.orgVideoThe full treatment: datasets, quantisation, evaluation and deployment. Take it once you have fine-tuned something small and want to do it deliberately.2h 37m

The Wider Field

2 lessons · 5h 9m

  1. 28CompletedMachine Learning for Everybody – Full CourseOptionalfreeCodeCamp.orgVideoOptional. Everything that is not a neural network — regression, classification, clustering, and the practical business of splitting data and measuring a model. Worth knowing, because a great many problems do not need deep learning at all.4h
  2. 29CompletedMIT Introduction to Deep Learning — 6.S191OptionalAlexander AminiVideoThe opening lecture of MIT's deep learning course, rerecorded each year and kept current. An hour that places everything in this course inside the wider field, and the best single lecture here.1h 9m

Credits

Every lesson in this course was made by one of these channels. Tubeversity sequenced them; it did not make them. If a course is useful to you, the people below are the ones who earned it.