Residual Connections (Skip Connections)
Know a great resource on Residual Connections? Send it to us at contactbackprop@gmail.com!
Tags: Neural Networks (MLPs)
Papers
Deep Residual Learning for Image Recognition
Papers with Code explanation and link to paper introducing residual connections
Videos
A pair of videos explaining residual connections by Andrew Ng
Lecture section on residual connections at UC Berkeley
Deep Residual Learning for Image Recognition (Paper Explained)
Paper explanation by Yannic Kilcher
Posts
Skip Connections and Residual Blocks
An explanation of why and how residual connections are used
Intuitive Explanation of Skip Connections in Deep Learning
An explanation of how skip connections address the vanishing/exploding gradients problem
Residual Blocks — Building blocks of ResNet
Short towardsdatascience post on residual connections
Code Examples
Pytorch implementation of the famous ResNet model which uses residual connections
Basic implementation of a residual block with Keras
Textbook Sections
Residual Blocks | Chapter 7.6.2
Section on residual blocks from the Deep Dive into Deep Learning textbook
Tagged Pages: Transformers
Know a great resource on Residual Connections? Send it to us at contactbackprop@gmail.com!