Batch Normalization

Know a great resource on Batch Normalization? Send it to us at contactbackprop@gmail.com!

Tags: Neural Networks (MLPs)

Papers

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Original Paper introducing Batch Normalization

Videos

Batch Normalization, 2, 3, 4

A sequence of videos by Andrew Ng explaining batch normalization in depth

Batch Normalization Walkthrough

A walkthrough of the Batch Norm paper by Yannic Kilcher

A visual, beginner friendly introduction to Batch Norm with Tensorflow code by Deep Lizard

Video of using Batch Norm in a pytorch neural network by Deep Lizard

Posts

Batch Normalization Explained | Papers with Code

A summary of Batch Norm, along with a link to code and the paper

Code Examples

Batch Norm example on MNIST dataset using numpy another option: here

Numpy implementation of neural net with batch norm

Example of using Batch Norm in a pytorch neural network by Deep Lizard

Textbook Sections

Dive into Deep Learning Chapter 7.5 - Batch Normalization

Explanation of Batch Norm with mxnet, pytorch, and tensorflow implementations

Deep Learning Chapter 8.7.1 Batch Normalization (Page 313)

Section on Batch Norm from Ian Goodfellow’s Textbook

Tagged Pages: Dropout, Layer Normalization

Know a great resource on Batch Normalization? Send it to us at contactbackprop@gmail.com!