Papers

Learning Representations by Back-propagating Errors

• The famous paper introducing backpropagation for machine learning

Lectures

Backpropagation Lecture

• Andrej Karpathy’s lecture on backpropagtion

Lecture 6: Backpropagation

• A University of Michigan lecture on backprop by Justin Johnson

Lecture 4: Backpropagation and Neural Networks

• A Stanford Lecture on backprop by Serena Yeung

Videos

3Blue1Brown on Backpropagation Part 1 and Part 2

• A couple videos explaining backpropagation and the underlying math

• While you’re at it, check out the rest of his videos on neural networks

Gradient Descent for Neural Networks and Backpropagation Intuition

• Andrew Ng’s walkthrough of backpropagation from his Coursera course

Posts

Yes, You Should Understand Backprop

• A medium post by Andrej Karpathy about why learning backprop is still important

Backpropagation Intution

• A lesson on backpropagation from Karpathy’s Stanford course

Backpropagation | Brilliant

• A brilliant (pun intended), detailed lesson on backprop by Brilliant.org

Backpropagation | ML Glossary

• A review of the chain rule, backpropagation, and some code

Code Examples

Pytorch Example

• A simple example of training a neural network in pytorch using backpropagation

Neural Network backpropagation from scratch in Python

• A cool example of training a neural network from scratch with just numpy