Residual Connections (Skip Connections)

Know a great resource on Residual Connections? Send it to us at contactbackprop@gmail.com!

Tags: Neural Networks (MLPs)

Papers

Deep Residual Learning for Image Recognition

Papers with Code explanation and link to paper introducing residual connections

Videos

A pair of videos explaining residual connections by Andrew Ng

Lecture section on residual connections at UC Berkeley

Deep Residual Learning for Image Recognition (Paper Explained)

Paper explanation by Yannic Kilcher

Posts

Skip Connections and Residual Blocks

An explanation of why and how residual connections are used

Intuitive Explanation of Skip Connections in Deep Learning

An explanation of how skip connections address the vanishing/exploding gradients problem

Residual Blocks â€” Building blocks of ResNet

Short towardsdatascience post on residual connections

Code Examples

Pytorch implementation of the famous ResNet model which uses residual connections

Basic implementation of a residual block with Keras

Textbook Sections

Residual Blocks | Chapter 7.6.2

Section on residual blocks from the Deep Dive into Deep Learning textbook

Tagged Pages: Transformers

Know a great resource on Residual Connections? Send it to us at contactbackprop@gmail.com!