Residual Connections (Skip Connections)

Know a great resource on Residual Connections? Send it to us at contactbackprop@gmail.com!

Tags: Neural Networks (MLPs)

Papers

Deep Residual Learning for Image Recognition

  • Papers with Code explanation and link to paper introducing residual connections

Videos

ResNets and Why ResNets Work

  • A pair of videos explaining residual connections by Andrew Ng


ResNet (Residual Networks)

  • Lecture section on residual connections at UC Berkeley


Deep Residual Learning for Image Recognition (Paper Explained)

  • Paper explanation by Yannic Kilcher

Posts

Skip Connections and Residual Blocks

  • An explanation of why and how residual connections are used


Intuitive Explanation of Skip Connections in Deep Learning

  • An explanation of how skip connections address the vanishing/exploding gradients problem


Residual Blocks — Building blocks of ResNet

  • Short towardsdatascience post on residual connections

Code Examples

ResNet

  • Pytorch implementation of the famous ResNet model which uses residual connections


Residual Block

  • Basic implementation of a residual block with Keras

Textbook Sections

Residual Blocks | Chapter 7.6.2

  • Section on residual blocks from the Deep Dive into Deep Learning textbook

Tagged Pages: Transformers

Know a great resource on Residual Connections? Send it to us at contactbackprop@gmail.com!